Sebastian HaasAI Security Engineer
Research/Case study
Case study · GPU kernel engineering · gfx906

Six hypotheses, four refutations, +31% throughput

An AMD Instinct MI50 from 2018 used only 31% of its memory bandwidth when generating text. The obvious explanation was wrong. So was the second. How a team of AI agents found the cause, and what the project’s most expensive failure ultimately delivered.

Qwen2.5 1.5B / 7B · Llama-3.1 8B llama.cpp fork, 38 commits ROCm 10.1 · Wave64 · 60 CUs As of August 12, 2026
+38% Decode throughput, Qwen2.5-1.5B
232.1 → 320.0 tokens/s
+33% Decode throughput, Qwen2.5-7B
88.6 → 118.1 tokens/s
545→266 Kernel launches per token
after fusion and epilogues
4 / 6 Tested hypotheses falsified
including our own leading hypothesis
Starting point

The problem: a card waiting for itself

The MI50 is a data-center card with 16 GB of HBM2 and roughly 1 TB/s of theoretical memory bandwidth. During text generation, the decode step that produces one token after another, language-model inference is almost pure memory work: every model weight must pass through the chip once for every token. Across all model sizes, the achieved rate was consistently around 320 GB/s, or 31% of the possible bandwidth.

A decode step consisted of 545 individual GPU kernel launches: normalization, quantization, projections, positional encoding, attention, feed-forward, repeated 28 times, plus the output layer. This led to the project’s central hypothesis, and it sounded compelling.

Core investigation

Hypothesis ledger

Every hypothesis was implemented, measured, and either confirmed or rejected. The verdicts below are measurements, not opinions; each includes the number that decided it.

T1
Falsified

Decode is dispatch-limited: 545 kernel launches per token never let the card get going.

Timestamp analysis of every kernel transition: gaps between kernels account for 1–2% of token time. An enqueue costs the CPU 1.86 µs, while the average kernel runs longer, so the queue stays filled. The losses are inside the kernels, not between them.

T2
Falsified, then successful, depending on the model

A persistent megakernel, one launch per token for all 28 layers, unlocks threefold potential.

Fully implemented: six internal phases, attention with context splitting, synchronization through a custom grid barrier. First measurement 234.6 t/s versus 269.4 for the comparison build: −13%. After porting the memory technique discovered later: +5.3%. Once the same gain reached the normal path: +2.8%. On the 7B model, however, +13.5%; there the kernel reaches 75% of the bandwidth reference instead of 34%. The threefold potential does not materialize, but the hypothesis is correct depending on model size.

T3
Falsified

The bandwidth gap comes from the q4_0 format: 18-byte blocks cannot be loaded with proper alignment.

The prime suspect was innocent. Machine code showed five memory instructions per loop iteration, three for the activation, which is already cached and generates zero main-memory traffic, but occupies 60% of memory instructions. The real cause.

T4
Falsified, and reversed

More concurrently running thread groups hide memory latency better.

First, the prototype showed that 240 instead of 168 resident workgroups cost 4.3%. Then the stronger finding: a kernel reaching the hardware maximum of 40 waves per compute unit collapses to 192 GB/s, while just one fewer wave (through 1664 bytes of unused LDS at launch) delivers 441 GB/s, a factor of 2.3, with byte-identical kernel code. Full occupancy is a cliff on this architecture, not a goal.

T5
Confirmed

Fewer, longer kernels help by fusing neighboring compute steps.

Q, K, and V projections in one launch (−112), normalization writes the quantized format directly (−56), positional encoding and cache writes as epilogues (−28 each). Result: +16% on the 1.5B, with bit-exactly unchanged results.

T6
Falsified

Standard remedies work: HIP graphs against launch costs, draft speculation against latency.

HIP graphs are slower on this architecture and leak memory (~370 B per node and invocation, demonstrated with a standalone reproducer). Draft speculation using a 1B model for an 8B model: +0.9%. Both rejected.

The turning point

What the failure left behind

The failed megakernel delivered something nobody had before: a reference for what this card can actually achieve. Its output head, a trivially parallel matrix-vector kernel on the same chip, reached 768 GB/s. Not a data-sheet figure, but a measurement.

For the first time, this quantified that the production kernels used only 55% of that bandwidth. A ladder of isolated microbenchmarks then cleanly separated the hypotheses:

The second row is the answer: +47% simply by omitting activation loads. The third shows that the compute chain costs nothing; the fourth independently confirms the reference. The resulting redesign, a full data block per lane through wide loads, multiple output rows per workgroup, delivered +20.5% on the 7B model, reproducibly confirmed by an A/B/A run to within 0.04 t/s.

Results

Benchmarks

All figures come from alternating measurement arms with 150-second cooldowns (explained below), five repetitions each, flash attention enabled.

Decode throughput (tokens/s, tg64) across project phases
Model Initial After fusion After memory fix Current Total
Qwen2.5-1.5B Q4_0 232.1269.4297.5 320.0+38%
Qwen2.5-7B Q4_0 88.693.0114.5 118.1+33%
Llama-2-7B Q4_0 96.8¹ 117.4+21%
Llama-3.1-8B Q4_K_M 70.670.873.7 73.9+5%

¹ The Llama-2 gain has an uncomfortable history: much of it fixes a self-inflicted regression, a kernel-selection window validated on only one model that slowed the most common matrix width in the field (K=4096: Llama-2, Mistral, Llama-3) by 16%, while gaining 0.4% on the validated case. Details below under “Near-miss conclusions.”

Structural and efficiency metrics
MetricBeforeAfterNote
Kernel launches / token (1.5B)545294Fusion + epilogues
Effective bandwidth (7B)384 GB/s471 GB/sOut of 768 achievable
Grid barrier, custom implementation10.8–13 µs1.07 µsVersus AMD Cooperative Groups
Kernel execution floor1.3–2.1 µs3.2–4 µsIsolated versus in the decode stream
Occupancy cliff (single matvec)192 GB/s441 GB/sOne fewer wave per CU
Wait instructions in the matvec hot loop13–157–9After address-space correction
LDS instructions per weight byte0.5560.056Data-layout redesign

Why the megakernel won, and why that does not help

Breaking down its advantage explains the recommendation against building it. Positive items are gains over the normal path; the negative item is the cost of persistence itself:

The persistence-specific balance is negative: barriers cost 333 µs and save only 129 µs in launch gaps, a net −204 µs. The kernel wins despite its persistence, not because of it. Three of its four gains do not depend on persistence at all and were subsequently ported into the normal path. Its lead then shrank from +5.3% to +2.8%.

The size effect: the same method, four times the gain

Porting to the 7B model was the last open question, and it changed the assessment. The estimate had predicted +2 to +5%, assuming the advantage came from matrix-vector multiplication. It does not; the normal path is actually faster there.

Persistent kernel versus normal path, by model size
ModelNormal pathPersistent ΔLong contextBandwidth achieved
Qwen2.5-1.5B Q4_0320.0317.5 −0.8%Advantage disappears34% of reference
Qwen2.5-7B Q4_0118.1130.4 +10.4%+11.4%75% of reference

This table shows the final state after all portable parts of the prototype’s advantage were integrated into the normal path, most recently positional encoding as an epilogue to the preceding step (+4.0% on 1.5B, +2.3% on 7B). On the small model, the normal path has now overtaken the megakernel. On the large one, it remains 10.4% behind, and that gap no longer melts away, because every grid-wide dependency in the normal path costs a kernel boundary of 3.2–4 µs, where the persistent kernel pays 1.15 µs. That is precisely persistence, and it cannot be fused away.

The explanation is in the last column. The large model has enough work between synchronization points to reach streaming behavior at all. Its advantage comes from 1302 µs per token spent in normal-path kernels that perform no computation: 708 µs of normalization and quantization across 143 launches at the execution floor, 296 µs of positional encoding, 136 µs of attention combination, and 162 µs of launch gaps. Exactly the costs a persistent kernel does not have by construction.

Approach

How the agents searched

The investigation used specialized AI agents, each handling one stage: forensics, fusion, prototype, root-cause analysis, and porting. The decisive factor was not their compute power, but the instruments used to separate claims from measurements. Twelve techniques that proved useful:

Kernel inventory through a profiler trace

Every kernel invocation in a decode step captured with name, duration, and timestamp. The problem: where does a token end? Solved by using the output layer as a marker; it runs exactly once per token and cleanly segments the stream.

Found: 545 launches/token, distributed across 19 kernels per layer

Gap analysis instead of summing durations

Not adding kernel runtimes, but measuring the differences between one kernel’s end and the next start. This metric alone falsified the project’s leading hypothesis.

Found: gaps only 1–2%; the cause is inside kernels

Isolated microprograms

Launch, synchronization, and transfer costs measured in standalone miniature programs rather than estimated from the overall picture, then cross-checked against trace analysis.

Found: enqueue 1.86 µs, barrier 1.07 µs, transfer 49.6 µs

Reading machine code

The decisive finding came not from a profiler, but from disassembly of the generated kernel: count memory instructions, check their width, inspect wait placement.

Found: 3 of 5 memory instructions serve cached data

Differential benchmark ladder

Rather than optimizing a guess, variants were built that each omit one ingredient. The difference between two rungs identifies the cause without having to guess it in advance.

Found: +47% when activation loads are omitted

Switchable changes

Every optimization received a runtime switch. This allows both states to be compared in the same program: no rebuild, no version differences, no excuses.

Enabled: clean A/B comparisons across 38 commits

A/B/A instead of A/B

After every comparison, the initial state was measured again. If it failed to return to its own value, the measurement was worthless.

Confirmed: 95.04 → 114.54 → 95.08 t/s, returning within 0.04

A second reference to classify errors

When the prototype exceeded the accuracy threshold, the threshold was not raised. Instead, a second reference implementation was created with identical code but single rather than double precision.

Proven: the deviation is a rounding artifact, not a computation error

Live bit comparison during operation

Verification modes compare every generated buffer byte by byte against the reference implementation during real inference, across thousands of buffers rather than a handful of test cases.

Demonstrated: over 3000 buffers, zero deviations

Deliberately provoking failure

Before building a persistent kernel, the condition that makes it hang was forced: more thread groups than can fit on the chip simultaneously.

Consequence: a runtime check became mandatory, not optional

Cross-check through reversal

To prove a change does not alter results, it was reverted and checked for exactly the same numbers and text output.

Demonstrated: bit-identical to the initial state

Measure alternatives, do not assert them

To answer “would splitting outperform the monolith?”, the kernel was restructured so individual branches could be compiled and measured separately instead of estimating the answer.

Result: splitting only approximately 10% better, at best parity
Near-miss conclusions

Four traps that would have fabricated success

Each of these traps would have introduced a false result into the assessment. All four were caught by verification mechanisms, three only after the mechanism itself had been refined.

Metric

1504 GB/s: one and a half times the physically possible

The analysis hard-coded the bytes of a full layer while the prototype executed only 7% of it. The same line extrapolated 999 tokens/s. Every bandwidth figure has since been interpreted against a concurrent reference.

Accuracy

A threshold that could conveniently have been raised

The prototype exceeded the error threshold by 78%. Instead of relaxing it, the yardstick was changed: a single-precision reference showed the same deviation, and at half the positions an even larger one.

Compiler

An address space the compiler could not infer

Because the pointer type was ambiguous, the compiler emitted a generic rather than specialized load instruction. It counts in two wait queues simultaneously, forcing every following access to serialize individually.

Verification

A verification mode checks only what it checks, four times

Four bit-level traps, and each time the existing verification reported zero errors because it compared the wrong stage: downstream quantization instead of raw values, the f32 intermediate instead of the f16 final values, and the fourth time exactly the reverse. The fix for the third trap (conversion directly at the expression, one rounding) would have been precisely wrong for the sibling variant, where the double-rounded value matched the reference. There is no universally correct verification stage, only the stage where the change acts.

Validation

A gate validated on one model is not validated

A kernel-selection window was measured on one model (K=3584) and applied to the whole range through K=4096. Result: 16% loss at the field’s most common matrix width, 0.4% gain on the validated case. The error persisted unnoticed for weeks and surfaced only because another measurement happened to hit the right model. The rule since then: thresholds are measured on at least three architectures with different widths.

Hardware

The hardware maximum is a cliff

The project’s slowest kernel was also its “most efficient”: 23 registers, no LDS, maximum occupancy of 40 waves per compute unit, and at that point 2.3 times slower than necessary with identical cache counters. The fingerprint of this cliff: kernel time is independent of data volume. The only effective diagnostic control: unused dynamic LDS at launch, reducing occupancy by one wave without changing a byte of code.

And a measurement error that would have invalidated every series

The card is capped at 150 watts and throttles to half speed under uninterrupted load: the 7B model falls from 92.7 to 44.9 tokens/s. Running a measurement series without pauses means that from the third row onward, you measure temperature rather than code. Every number in this case study comes from alternating arms with 150-second cooldowns.

Validation

What correctness was measured against

Speed without correctness evidence is worthless in this field: a kernel that computes incorrectly can be arbitrarily fast. The same anchors applied after every single step:

Correctness anchors after every change
CheckCriterionResult
Perplexity, Qwen2.5-1.5BBit-exactly unchanged9.4145
Perplexity, Llama-3.1-8BBit-exactly unchanged7.0867
Matrix multiplicationAll green1066 / 1066
Flash attentionAll green3344 / 3344
Expert routingAll green690 / 690
Prototype endurance runNo deviation10.000 iterations
Token by token against referenceIdentical selection64 / 65
Text comparison, 300 tokensByte-identicalPassed

The one deviation in the token-by-token comparison occurred exactly at the position with the smallest margin between the best and second-best choices. There the last bit decides, and the prototype deliberately uses a different format for the output layer.

Assessment

What a carefully measured failure is worth

The criterion was fixed in advance: harvest the portable parts of the prototype’s advantage into the normal path, and if a gap of five percent or more remains, integration is justified. The harvest is complete, and the answer is opposite for the two model sizes.

For small models, the megakernel is finished: the normal path has overtaken it at 320.0 versus 317.5 tokens/s. Reducing 545 kernel launches to 266 achieved exactly what the monolith was meant to achieve, without its costs. For the 7B class, the normal path remains 10.4% behind, justifying integration for the first time, with open caveats: fixed model geometry, only one quantization format, and the prototype figure is an extrapolation that integration would first have to demonstrate end to end.

The real return lies elsewhere. Without the megakernel being built, there would be no 768 GB/s reference; without that reference, no diagnosis of activation loads; without dissecting its victory, no quantization fix. The +31% comes almost entirely from insights produced by a failure.

Roughly half of the original bandwidth gap is closed, but not where the hypothesis placed it. It was never in the number of kernels, but in their memory-access efficiency: wasted load instructions for already cached data, an address space the compiler could not infer, and dozens of kernel launches for microseconds of work.

What transfers beyond this card

Ordered by transferability, not effect size:

Methodology (applies everywhere): The differential ladder that identifies a cause instead of guessing it. A/B/A rather than A/B: returning to the initial value authenticates the measurement. The insight that verification checks only the stage it targets. And the distinction between ranking and recommendation: a top result measures distance from the status quo; a build decision weighs it against downstream costs.

Architecture knowledge (applies to GCN, partly beyond): Full occupancy as a cliff, with its fingerprint and diagnostic control. The 3–4 µs kernel execution floor in a live stream: every kernel with less than ten microseconds of compute is a fusion candidate. The finding that load instructions for cached data cost throughput even when they generate no memory traffic.

Project result (applies to this card): The throughput gains themselves, the repaired formats, the ranked lever list with a measured value behind every entry, including the eight rejected entries. A documented dead end is a result: it saves the next person the journey.

A carefully measured negative result is not a lost project. It is a measurement instrument.

Subject: a llama.cpp fork for gfx906 (AMD Instinct MI50, 16 GB, Radeon VII firmware), ROCm 10.1, 48 commits. All measurements on identical hardware and in the same session, with cooldowns between arms; key figures independently cross-checked. Open caveat: since the prototype’s last redesign, one cache value from position 15 in layer 7 differs from the previous version by the smallest representable unit. Deterministically reproducible, with the code-generation cause not yet found. It appeared only in the raw comparison; the error metric continued to report the same value.

Back to all research →