H3 Acceleration Arena

 About

01  What this is

A human-judged ranking of MiniMax-H3 speed-ups.

Generating video with MiniMax-H3 at full quality takes 27 model evaluations per clip. A crowd of published methods claim to get most of that quality in four to eight — turbo LoRAs, distillation checkpoints, sparse-attention kernels. Automated metrics cannot settle which of them actually hold up: they measure global statistics and are blind to the localised smearing and noise a person spots immediately. So this asks people, one pair at a time.

—
methods compared
200
prompts
1344×768
canvas
124
frames, with audio

02  What is being compared

The set

Roughly twenty-odd acceleration methods, run on identical inputs. They fall into a few kinds: turbo LoRAs that distil the sampler into four to eight steps, merges of those LoRAs, sparse-attention variants that change how attention is computed rather than how many steps are taken, a 28-step reference that takes no shortcut at all, and a hosted service for comparison against what a vendor actually ships.

The reference is the yardstick: every rating is expressed relative to it, so a number reads directly as how much quality you give up against the full-step run, and what speed you get back.

This page describes the set, not the legs. Which method is which is hidden while you vote, and nothing here says whether any particular one is good — that is the question the votes answer, and telling you first would spoil the measurement.

The corpus

200 prompts, built as a deliberate 2×2: text-to-video against image-to-video, crossed with audio-emphasis against visual-emphasis, fifty in each cell. H3 generates picture and sound jointly, so half the corpus exists to stress the audio specifically — speech, footsteps, impacts, music — because a method can hold the picture together and lose the sound completely.

03  Why the comparison is fair

Every pair you see is the same prompt rendered two ways. Held constant across both clips:

  • The prompt, and the full expansion fed to the model — one keystroke away behind i on the vote page. Prompt expansion is disabled, so what the model receives is what is written.
  • The random seed, derived from the prompt, so both clips denoise the same starting noise. This is the control the whole thing rests on: it is what makes a visible difference attributable to the method rather than to luck.
  • The canvas — 1344×768, 124 frames at 24 fps, 16:9 — and the dtype, the conditioning frame on image-to-video prompts, and the guidance setting.
  • Loudness. Every clip is normalised to the same integrated loudness at ingest with a constant gain, because the measured spread between two methods on the same prompt reached 12.4 dB — enough to win a blind test on volume alone. The native measurement is not hidden: what each method actually shipped is reported alongside the rating.

One exception, stated plainly. The hosted leg is reached through a vendor API, and a hosted endpoint samples its own noise server-side — the integer we send does not reproduce our latent. Everything else about it is matched (same prompt, canvas, frame count, expansion disabled), but its clips are independent draws rather than our noise denoised differently.

So its comparisons are not noise-controlled, and its interval is published as a floor rather than an estimate: the resampling that produces every other interval is structurally blind to between-draw variance, and that component does not shrink as more votes arrive. Labelling it beats excluding it — "what does the vendor actually ship" is one of the more interesting questions here.

04  How voting works

  • Blind until you vote. The two clips are A and B, sides randomised per pair, and the page carries nothing per-leg — no cost, no family, not even a carefully worded warning, because anything that fires on some pairs and not others is a name for those methods.
  • Both clips play in parallel, started together and held in sync. With about five seconds of material, watching the same moment in both at once is the only way they are comparable at all.
  • Exactly one audio track is audible, and it alternates at each loop boundary, so you hear each side in whole passes rather than fragments. The live card is marked; press 1 / 2 to switch by hand.
  • The gate depends on the prompt. On an audio-emphasis prompt the vote controls stay locked until each clip has genuinely been heard through — coverage is credited only while a clip is unmuted, playing and actually advancing, so scrubbing cannot satisfy it. On a visual-emphasis prompt both clips start muted and immediately, and the vote opens part-way through the first pass; audio stays one keystroke away and is never required.
  • Tie and "both bad" are first-class outcomes, behind the same gate. On a set of near-identical methods, "these two cannot be told apart" is information, and it is modelled as such rather than being split into half a win each.

After the vote: a one-line toast naming which method won, then the next pair, already decoded and waiting. No account is needed.

05  How the ratings are computed

Two estimators, side by side, because they fail differently.

  • Bradley–Terry is the headline: order-independent, uses every vote at once, fitted by maximum likelihood with a weak prior that stops a method which has only ever won from running off to infinity. Ties are modelled with a Davidson tie term rather than discarded. Intervals — and rank intervals, which are the honest answer to "is this leaderboard real" — come from resampling the votes and refitting.
  • Elo runs alongside because it is what people can check by hand. It is the weaker estimator here: it is sequential, so the answer depends on the order the votes arrived, and the share of a rating that is an artifact of that order is reported as its own number.

Both are reported in Elo units, and the scale floats: there is no fixed zero, so a single number means nothing on its own and only the gaps between methods carry information — which is how every Elo ladder works. The un-accelerated 28-step reference competes as an ordinary entrant and is marked on its row, so you can see at a glance which accelerations landed above and below it.

The leaderboard is not published until there are enough votes to rank these fairly. A table looks authoritative whatever banner sits over it, and a reader takes the ordering away with them — so until the votes support one, there is nothing to take away. It opens by itself. Alongside every rating, when it does open: the interval, the rank range, the number of votes behind it, the "both bad" rate, the cost in seconds per clip, and the speed-up against the reference.

06  What is excluded, and why that matters

Some methods are in the dataset but out of the voting pool, and the reasons are not the same reason. Published in full on the leaderboard page, grouped by kind, they are mostly:

  • Probes — two or three clips from an investigation, kept because they are the controls that make a measurement a measurement, and never meant for a voter.
  • Parked — working artifacts taken out deliberately, usually because a question has already been answered by direct measurement and every extra method costs human attention.
  • Superseded — we ran it at the wrong setting, and the corrected re-run is in the pool instead. The original clips stay as the control for that comparison.
  • Open investigations — status genuinely undetermined. Neither "broken" nor "fine", and the group most likely to come back.

Exactly one exclusion is a defect confirmed in the artifact itself. That framing is the point: a column of exclusions reads as a broken corpus, and it is not one. Most are states of an investigation or deliberate curation, and an exclusion is provisional by default — every entry records whose fault it is, because that is what decides whether it comes back. Nothing is deleted: the clips stay in the dataset and every vote stays in the log.

One method is also judged on half the corpus by our own decision rather than by any absence — it is rated on the visual-emphasis prompts because its audio was distilled against a sampler convention that no longer exists, which we could reproduce but not make faithful. It says so on its row, and its interval is wider for it.

07  Provenance and credit

Every artifact judged here is someone else's published work, downloaded from the Hub and run at the settings its author documented. Where an author's settings were ambiguous we went to their own inference code and their repo's discussions rather than guessing, and where we got it wrong the corrected re-run replaced ours. The source repository for each method is linked from its row on the leaderboard page.

Generation is 🤗 diffusers throughout, on the released MiniMax-H3 weights, with every method's attachment verified against an expected module census before its clips were generated — so a rating is about the artifact rather than about our loader. Sampling was checked against ComfyUI's grid for equivalence, because that is what most of these authors tuned against.

The prompts, the generated clips and the vote log all live in a dataset on the Hub. Votes are append-only and nothing is ever removed from the record — a withdrawal or a partial exclusion changes what is counted, never what was kept.