Qwen3.5 9B W4A16 with its own MTP head on one or two Intel Arc Pro B70
RedHatAI's W4A16 Qwen3.5-9B served by vLLM XPU on one B70 through the lab's R276 image, with the publisher's MTP head as a lossless speculative draft. vLLM's compressed-tensors path selects the lab's wNa16 INT4 kernel, whose fixed-K two-tier strategy is row-count invariant: without speculation this route is byte-exact against a single request at every concurrency through 64 users, where the FP8 route on the same model flips a near-tie token from 16 users up. It is also 28% faster without speculation and 15% faster with it. The kernel is not the only reduction on that path: the RMSNorm this route runs is itself row-count dependent (measured 2026-09-08), so the route is exact in the regimes measured rather than exact by construction.
What we measured Lab-measured
113tok/s MTP depth 3 with the draft-only INT4 lm_head, one card, strict completions suite (center of two fresh servers, 113.627 / 112.904)
Center of two fresh-server class-balanced medians over the strict fixed 12-prompt six-class suite through the completions API, 512-token cap, separate empty compile caches, 12/12 complete token arrays exact versus a same-configuration MTP0 oracle (64.332 / 64.338), canaries on every server, cache zero; one B70, R276 image, qwen3_5_mtp depth 3, draft-only INT4 lm_head, full decode-only XPU graph capture. proof file
MTP = multi-token prediction, a small draft the main model verifies · oracle = the fixed prompt set whose exact outputs are checked
Still missing before this becomes an install guide
clean-host replay
Measured performance profiles Lab-measured
Qwen3.5-9B W4A16 one card, MTP depth 3 with the draft INT4 head: decode over active context (2K-32K) Lab-measured
Campaign w4 (2026-09-07): one B70, one-slot vLLM HTTP completions at exactly 2K, 4K, 8K, 16K, 24K and 32K active context of unrepeated real content (technical prose, Python, structured documents; three requests per depth, median shown), 128 output tokens, max-model-len 33024, cache zero, canaries before and after; every answer matched the same-configuration MTP0 oracle (18/18). No value is interpolated. evidence
Qwen3.5-9B W4A16 one card, no speculation: decode over active context (2K-32K) Lab-measured
Campaign w4 MTP0 arm (the oracle for the depth-3 profile), same workload and shape. evidence
Many people at once Lab-measured
Qwen3.5-9B W4A16 one card, no speculation, XPU graph capture, identity-qualified aggregate decode vs concurrent users (c1-c64; warm pass) Lab-measured
Campaign w1 (2026-09-07): one B70, R276 image, strict launcher env, FULL_DECODE_ONLY capture sizes 1-64, no speculation, max-model-len 256, max-num-seqs 64, max-num-batched-tokens 512, 128 returned raw token IDs per response on the small-context suite, two passes on one server, warm pass shown. Every rung is output-identity-qualified in both passes, including c64 (64/64 twice). Nothing is withheld on this profile. evidence
Qwen3.5-9B W4A16 one card, MTP depth 3 with the draft INT4 head, identity-qualified aggregate decode vs concurrent users (c1-c16; warm pass) Lab-measured
As above with qwen3_5_mtp depth 3 and the draft-only INT4 lm_head. c1-c16 output-identity-qualified in both passes. Measured but withheld: c32 827.3 (32/32 warm, 31/32 cold) and c64 789.2 (61/64). evidence
Qwen3.5-9B W4A16 two cards (TP2), no speculation, identity-qualified aggregate decode vs concurrent users (c1-c32; warm pass) Lab-measured
Campaign w3 (2026-09-07): two B70s, otherwise as the one-card no-speculation profile. c1-c32 output-identity-qualified in both passes. Measured but withheld: c64 2092.9 (63/64 in both passes); the one-card run of the same kernel held 64/64 there, so the residual flip comes from the cross-card reduction. evidence
Qwen3.5-9B W4A16 two cards (TP2), MTP depth 3 with the draft INT4 head, identity-qualified aggregate decode vs concurrent users (c1-c16; warm pass) Lab-measured
As above with qwen3_5_mtp depth 3 and the draft-only INT4 lm_head. c1-c16 output-identity-qualified in both passes. Measured but withheld: c32 1350.9 (32/32 warm, 31/32 cold) and c64 1464.6 (60/64). evidence
Qwen3.5-9B W4A16 one card, draft depth scheduled by batch size (3 to 8 users, 1 to 16, none above), full decode graphs per depth, draft-state catch-up, aggregate decode vs concurrent users (c1-c64; warm pass) Lab-measured
Campaign cudynm1 (2026-09-11, repeated as cudynm1r on the same boot): one B70, R276 image plus the three overlays in repro/qwen35-9b-w4a16-b70/docker, FULL_DECODE_ONLY capture sizes 1-64 with one full decode graph per scheduled draft depth, schedule [[1,8,3],[9,16,1],[17,64,0]], max-model-len 256, max-num-seqs 64, max-num-batched-tokens 512, 128 returned raw token IDs per response on the small-context suite, two passes on one server, warm pass shown. Identity by the two-run rule: exact against the sequential oracle at every rung through 32 users in both passes of both runs; at 64 users near-exact (63-64/64), the same band as the no-speculation server on this route, so that point is a measured rate, not an identity-qualified one. One user on the strict suite: 110.69/110.60 and 110.65/110.60 tok/s, G1/G2/G3 12/12 in both runs; the 2K-32K real-content ladder is 18/18 exact. evidence
Qwen3.5-9B W4A16 one card, no speculation, CLASSPAD=1 (R293): identity-qualified aggregate decode vs concurrent users Lab-measured
Chain s1 (2026-09-11): one B70, R293 image with VLLM_XPU_FP16_LINEAR_CLASSPAD=1 (verified in the container and by the op's census line), strict launcher env, FULL_DECODE_ONLY capture sizes 1-64, no speculation, max-model-len 256, max-num-seqs 64, max-num-batched-tokens 512, 128 returned raw token IDs per response on the small-context suite, four passes, pass 2 shown. Points are rungs output-identity-qualified in the shown pass (c1-c24 and c64). Measured but withheld: c32 1208.7 (127/128). With max-num-seqs 128 and max-num-batched-tokens 1024 (s6) c64/c96/c128 are exact at 1641.6 / 1852.8 / 1955.3. evidence
Qwen3.5-9B W4A16 two cards, no speculation, CLASSPAD=1 (R293): identity-qualified aggregate decode vs concurrent users Lab-measured
Chain s2 (2026-09-11): two B70s (TP2), otherwise as the one-card R293 profile. Points are rungs output-identity-qualified in the shown pass (c1-c32). Measured but withheld: c64 2614.1 (255/256). With max-num-seqs 128 and max-num-batched-tokens 1024 (s7): c64 2614.2 (256/256), c96 2978.7 (383/384), c128 3227.4 (512/512). With the 5 ms admission stagger on this lane's tie-site suite (s5) c64 is 1280/1280 over twenty passes at 2557.1, harness-certified. evidence
Qwen3.5-9B W4A16 one card, scheduled drafts (depth 3 to 8 users, 1 to 16, none above) on R293 with CLASSPAD=1: aggregate decode vs concurrent users (exact rungs) Lab-measured
dyn293 (2026-09-11): one B70, the scheduled-draft overlays rebuilt on the R293 image, VLLM_XPU_FP16_LINEAR_CLASSPAD=1, strict launcher env, full decode graphs per scheduled K, max-model-len 256, max-num-seqs 64, 128 returned raw token IDs per response on the small-context suite, four passes, pass 2 shown. Points are rungs output-identity-qualified in the shown pass (c1-c16). Measured but withheld: c32 1192.2 (120/128), c64 1630.6 (255/256; the no-speculation R293 server in the same arm reads 1641.1). Same-host R276 baseline (dyn276): c16 942.7, c32 1153.1 (115/128), c64 1243.4 (249/256). evidence
How much faster could this get? Projected — not measured
The ML Bottleneck physics engine projects what stock software, a tuned run, and the physical ceiling look like for this exact model, compression, card count, and software. The grade is optimization headroom against the tuned-run target, not model quality.