Spreadsheet view. Each row is a change/action that either improved speed or unlocked the next measurable result. Percent gains are relative to the closest comparable prior run noted in the source artifact.
Words used below: screen = a quick speed check without the full quality battery · A/B = the same run with one change, compared · TP N / PP N = the model split across N cards (by tensor / by pipeline) · MTP = multi-token prediction, a small draft the big model verifies · ubatch = tokens processed per step · XCCL = the cards' communication library. Patch links ending in .b64 are base64-encoded patch files. This log follows the Qwen and MiniMax optimization programs through August 2026; Ornith, Gemma, Muse, and Laguna have their own result pages.
| Track | Date | Change / action | Result after change | Gain | Why it stayed in the path | Source |
|---|---|---|---|---|---|---|
| Qwen3.6 27B Q4_0 GGUF, llama.cpp/SYCL | ||||||
| Qwen Q4 | May 3 | Set the quality-preserving baseline: SYCL graph-enabled single B70, Q4_0 weights, f16 KV. | 24.249 tok/s | Baseline | Established the comparison point: no quantization change, no speculation, no power tuning. | result |
| Qwen Q4 | May 3-4 | Make tensor split viable: split-buffer fixes plus async tensor copy. | 37.690 tok/s, 2x | +40.26% | Moved multi-GPU from slower-than-single-card into useful scaling. | result / patch |
| Qwen Q4 | May 4 | Fix Qwen split anchoring: recurrent projections share a stable split plan. | 38.365 tok/s, 3x | Enabled 3x | Made equal three-card tensor split legal and reproducible. | result / patch |
| Qwen Q4 | May 4 | Replace copy/sum with single-kernel allreduce. | 41.737 tok/s, 3x | +8.79% | Reduced repeated small cross-card reductions in the tensor-parallel path. | result / patch |
| Qwen Q4 | May 4 | Cache Q8 activations across sibling Q4 matvecs. | 42.432 tok/s screen | +3.65% | Avoided repeated activation quantization/copy work inside tensor split. | note / patch |
| Qwen Q4 | May 4 | Use event barrier sync after allreduce. | 43.605 tok/s | +3.21% screen | Kept the same math while lowering synchronization overhead. | data / patch |
| Qwen Q4 | May 5 | Fuse allreduce + residual ADD. | 44.004 tok/s | +0.92% | Did the residual add where the cross-card sum was already happening. | data / patch |
| Qwen Q4 | May 5 | Fuse reshape-through-ADD graph sites. | 44.813 tok/s | +1.30% | Collapsed 48 repeated view-plus-residual sites into the backend helper. | data / patch |
| Qwen Q4 | May 5 | Add runtime health gate: reboot/peer-read before trusting new data. | 45.624 tok/s | +1.81% | Separated real code wins from degraded xe/Level Zero state. | data |
| Qwen Q4 | May 6 | Fuse adjacent Q4 matvec pairs. | 46.118 tok/s | +1.08% | Cut repeated launch/activation work for same-input matvec pairs. | data / patch |
| Qwen Q4 | May 6 | Replace broad sync with reduce.wait mode. | 46.194 tok/s | +0.17% | Small but clean synchronization win on the fused MMVQ2 stack. | data / repro |
| Qwen Q4 | May 6 | Fuse gate/up matvec + SwiGLU. | 46.805 tok/s | +2.32% | Removed more repeated FFN work while preserving deterministic output. | data / patch |
| Qwen Q4 | May 6 | Fuse RMS_NORM + scale MUL. | 49.366 tok/s | +5.47% | Largest late Q4 gain; removed a repeated normalization/scale boundary. | data / patch |
| Qwen Q4 | May 6 | Fuse final allreduce + GET_ROWS. | 49.404 tok/s | +1.18% A/B | Removed the final small plain allreduce path without changing output. | data / patch |
| Qwen Q4 | May 7 | Fix over-broad Q8 guard. | 49.553 tok/s | +0.30% | Restored the validated allreduce+ADD fast path after a regression. | data / patch |
| Qwen Q4 | May 7 | Use flat fused beta/alpha GGUF, root-residual off. | 50.130 tok/s | +1.16% | Current quality-cleared Q4 high-water mark. | data / patch |
| Qwen3.6 27B FP8, vLLM/XPU | ||||||
| Qwen FP8 | May 4 | Fix singleton compressed-tensors attention scales for XPU FA2. | 41.503 tok/s | Enabled TP4 | Made the static FP8 checkpoint run through XPU FlashAttention2. | note / patch |
| Qwen FP8 | May 4 | Add verified CPU n-gram speculative decode, depth 2. | 42.489 tok/s | +2.38% | Speculative tokens were still verified by the FP8 target model. | data / patch |
| Qwen FP8 | May 5 | Use n-gram depth 4 with default CCL topology. | 47.675 tok/s | +14.87% | Best stable speculative setting before the library-ordering cleanup. | data / patch |
| Qwen FP8 | May 6 | Fix vLLM library ordering around XCCL. | 49.582 tok/s | +4.00% | Kept the fast TP4+n-gram path stable with the venv libraries first. | data / patch |
| Qwen dense B70 program, Q8 baseline to AutoRound/MTP research (different lane identities) | ||||||
| Qwen dense | Aug 8 | Establish the one-B70 Qwen3.6 Q8 decode baseline. | 15.550 tok/s | Program baseline | Created the measured starting point for the later Qwen kernel and runtime program. | result |
| Qwen dense | Aug 18 | Transfer the lab's AutoRound, XPU runtime, graph, sampler, and MTP stack to Qwen3.8. | 91.926 tok/s | New lane | Reused the lab's Qwen3.6 engineering on the architecture-compatible Qwen3.8 checkpoint. | audit |
| Qwen dense | Aug 18 | Increase MTP depth and tune the two-B70 lane past 100 tok/s. | 100.497 / 101.922 | Invalidated | The speed was measured here, but an output-changing greedy margin contaminated candidate and baseline; withdrawal is recommended. | audit |
| Qwen dense | Aug 20 | Disable the margin and rerun the full 25-prompt lane across three arms. | 101.170 tok/s | Research anchor | Honest margin-free result; not promoted because pairwise token repeatability is only 21–22/25. | status |
| Qwen dense | Aug 26 | Add block-W8A16 dispatch to the official FP8 TP2 service. | 35.011 / 1,112.570 tok/s | Qualified profile | Raised matched target-only single-user decode by 60.07% and c128 aggregate by 29.30%; the 1,112.570 figure is a separate 256-token service profile. | result |
| Qwen dense | Aug 27 | Audit dynamic-MTP promotion after the harness accepted a short output cap. | No promoted single-user headline | Corrected / pending | The 58.391 varied-prompt result used a 128-token cap and the 146.814 result used a selected high-acceptance fixture. Both were demoted to diagnostic evidence; the LocalMaxxing row is marked for withdrawal. Target-only aggregate/context results remain separately scoped. | audit |
| MiniMax M2.7, GGUF/RPC and AutoRound/vLLM | ||||||
| MiniMax | May 5 | Prototype split expert placement instead of monolithic fallback. | Graph reserved | Diagnostic | Turned "does not run" into a specific code-path blocker: expert placement and early GPU kernels. | data / patch |
| MiniMax | May 7 | Switch to process-per-GPU RPC+SYCL and fix device mapping. | 14.292 tok/s | First usable run | Produced the first reproducible MiniMax four-B70 completion. | data / patch |
| MiniMax | May 8 | Enable K/Q/V offload on the RPC+SYCL layer path. | 16.384 tok/s | +14.64% | Moved more repeated attention-side work onto the GPUs. | data / note |
| MiniMax | May 8 | Add fast IQ4_XS expert-down MUL_MAT_ID path. | 17.336 tok/s | +5.67% | Targeted the expert hot path instead of generic placement flags. | data / note |
| MiniMax | May 8 | Use runtime MMV row packing. | 17.547 tok/s | +1.22% | Small clean kernel scheduling gain on the fast-MMID stack. | data / note |
| MiniMax | May 8 | Allow fused RMSNorm and keep ubatch 64. | 17.698 tok/s | +0.86% | Current GGUF/RPC MiniMax baseline; deterministic smoke matched prior output. | data / note |
| MiniMax | May 8 | Repair vLLM/INC FusedMoE quantization for AutoRound. | 13.450 tok/s | Enabled vLLM | Made the AutoRound W4A16 safetensors model fit and generate on vLLM/XPU TP4. | data / patch |
| MiniMax | May 8 | Use pidfd IPC and P2P=1 for vLLM/XPU TP4. | 19.850 tok/s | +47.58% | Removed a communication setup bottleneck in the AutoRound path. | data |
| MiniMax | May 8 | Add hybrid B70 MoE config for AutoRound. | 20.110 tok/s | +1.31% | Small model-level win from using a decode-tuned MoE config. | config / payload |
| MiniMax | May 9 | Route tiny decode MoE batches through llm-scaler unsigned u4 kernel. | 29.748 tok/s | +47.49% | Largest MiniMax speed jump; kept prefill on vLLM and accelerated decode-sized MoE work. | data / kernel / vllm |
| MiniMax | May 9 | Remove per-layer FP32-to-FP16 route-weight cast. | 34.158 tok/s | +3.40% | Cut avoidable bridge overhead around the custom u4 decode path. | data / patch |
| MiniMax | May 9 | Leave oneCCL IPC at default for the u4 decode path. | 34.578 tok/s | +1.23% | Best p512/n256 MiniMax AutoRound result; pidfd was not best for this final stack. | data / payload |
| MiniMax | May 9 | Validate longer decode window with same best stack. | 37.136 tok/s | Validation | Current MiniMax high-water mark; longer output shows setup/prompt overhead was diluting short runs. | data / payload |
| Not promoted | May 9 | Triton attention A/B, graph capture, PP2/TP2, n-gram speculation, KV fp8. | Regressed, blocked, or neutral | None | Useful as screens, but not part of the improvement path. | data |
Short share version: Qwen taught the B70 playbook: split carefully, keep data local, fuse repeated work, and verify clean runtime. Qwen Q4 went from 24.2 to 50.1 tok/s; the broader dense-Qwen lab program went from a 15.55 tok/s one-card Q8 baseline to an honest 101.17 tok/s two-card AutoRound/MTP research anchor. The later official-FP8/W8A16 dynamic-MTP singleton screens are diagnostic pending a compliant varied-prompt final gate. Those are different model, quantization, and card identities, not a like-for-like percentage speedup. MiniMax then moved from early 101 GB GGUF bring-up to 17.7 tok/s on GGUF/RPC and 37.1 tok/s on vLLM AutoRound with the llm-scaler u4 decode path.