Qwen3.5 4B W4A16 with its own MTP head on one or two Intel Arc Pro B70
RedHatAI's W4A16 Qwen3.5-4B served by vLLM XPU on one B70 through the lab's R276 image, with the publisher's MTP head as a lossless speculative draft. The FP8-dynamic build of the same model cannot pass the base identity gate on this stack (three fresh-server pairs scored 11/12, 9/12, 11/12 with three tie-prone prompts); on the row-invariant W4A16 INT4 kernel the same gate passes 12/12, which is why this is the packaged route. The kernel is not the only reduction on that path: the RMSNorm this route runs is itself row-count dependent (measured 2026-09-08), so the route is exact in the regimes measured rather than exact by construction.
What we measured Lab-measured
177tok/s MTP depth 3 with the draft-only INT4 lm_head, one card, strict completions suite (center of two fresh servers, 177.406 / 177.168)
Center of two fresh-server class-balanced medians over the strict fixed 12-prompt six-class suite through the completions API, 512-token cap, separate empty compile caches, 12/12 complete token arrays exact versus a same-configuration MTP0 oracle (102.625 / 102.376), canaries on every server, cache zero; one B70, R276 image, qwen3_5_mtp depth 3, draft-only INT4 lm_head, full decode-only XPU graph capture. proof file
MTP = multi-token prediction, a small draft the main model verifies · oracle = the fixed prompt set whose exact outputs are checked
Still missing before this becomes an install guide
clean-host replay
Measured performance profiles Lab-measured
Qwen3.5-4B W4A16 one card, MTP depth 3 with the draft INT4 head: decode over active context (2K-32K) Lab-measured
Campaign v2 (2026-09-07): one B70, one-slot vLLM HTTP completions at exactly 2K, 4K, 8K, 16K, 24K and 32K active context of unrepeated real content (three requests per depth, median shown), 128 output tokens, max-model-len 33024, cache zero, canaries before and after; every answer matched the same-configuration MTP0 oracle (18/18). No value is interpolated. evidence
Qwen3.5-4B W4A16 one card, no speculation: decode over active context (2K-32K) Lab-measured
Campaign v2 (2026-09-07): one B70, one-slot vLLM HTTP completions at exactly 2K, 4K, 8K, 16K, 24K and 32K active context of unrepeated real content (three requests per depth, median shown), 128 output tokens, max-model-len 33024, cache zero, canaries before and after; the oracle arm for the depth-3 profile. No value is interpolated. evidence
Many people at once Lab-measured
Qwen3.5-4B W4A16 one card, no speculation, XPU graph capture, identity-qualified aggregate decode vs concurrent users (c1-c32; warm pass) Lab-measured
Campaign v1 (2026-09-07): one B70, R276 image, strict launcher env, FULL_DECODE_ONLY capture sizes 1-64, no speculation, max-model-len 256, max-num-seqs 64, max-num-batched-tokens 512, 128 returned raw token IDs per response on the small-context suite, two passes on one server, warm pass shown. c1-c32 output-identity-qualified in both passes. Measured but withheld: c64 1725.1 (63/64 warm, 64/64 first pass). evidence
Qwen3.5-4B W4A16 one card, MTP depth 3 with the draft INT4 head, identity-qualified aggregate decode vs concurrent users (c1-c16; warm pass) Lab-measured
As above with qwen3_5_mtp depth 3 and the draft-only INT4 lm_head. c1-c16 output-identity-qualified in both passes. Measured but withheld: c32 1147.4 (30/32) and c64 1201.0 (55/64). evidence
Qwen3.5-4B W4A16 one card, no speculation, CLASSPAD=1 (R293): identity-qualified aggregate decode vs concurrent users Lab-measured
Chain 15 r1 (2026-09-11): one B70, R293 image with VLLM_XPU_FP16_LINEAR_CLASSPAD=1 (verified in the container and by the op's own census line), strict launcher env, FULL_DECODE_ONLY capture sizes 1-64, no speculation, max-model-len 256, max-num-seqs 64, max-num-batched-tokens 512, 128 returned raw token IDs per response on the small-context suite, four passes on one server, pass 2 shown. Points are rungs output-identity-qualified in the shown pass. Measured but withheld: c24 1354.9 (95/96), c64 2163.6 (255/256). With max-num-seqs 128 and max-num-batched-tokens 1024 (r6) c64/c96/c128 are exact at 2165.2 / 2400.1 / 2520.0; with the 5 ms admission stagger on the tie-site suite (r5) c64 is 1280/1280 over twenty passes at 2103.6, harness-certified. evidence
Qwen3.5-4B W4A16 two cards, no speculation, CLASSPAD=1 (R293): identity-qualified aggregate decode vs concurrent users Lab-measured
Chain 15 r2 (2026-09-11): two B70s (TP2), otherwise as the one-card R293 profile. Points are rungs output-identity-qualified in the shown pass (c1-c16). Measured but withheld: c20 1695.8 (78/80), c24 1947.3 (94/96), c32 2350.1 (126/128), c64 3316.7 (255/256). With max-num-seqs 128 and max-num-batched-tokens 1024 (r7): c64 3323.9 (255/256), c96 3810.2 (383/384), c128 4015.3 (512/512). With the 5 ms admission stagger on the tie-site suite (r10) c64 is 1280/1280 over twenty passes at 3163.9, harness-certified. evidence
Qwen3.5-4B W4A16 one card, MTP depth 3 with the draft INT4 head, CLASSPAD=1 (R293): identity-qualified aggregate decode vs concurrent users Lab-measured
Chain 15 r1 (2026-09-11), depth-3 lane of the same server pair. Points are rungs exact in the shown pass (c1-c16). Measured but withheld: c20 1355.3 (79/80), c24 1489.9 (92/96), c32 1607.9 (120/128), c64 1830.7 (222/256). Depth 2 (r8) leads depth 3 above c16: 1766.6 at c32 and 2030.6 at c64. evidence
How much faster could this get? Projected — not measured
The ML Bottleneck physics engine projects what stock software, a tuned run, and the physical ceiling look like for this exact model, compression, card count, and software. The grade is optimization headroom against the tuned-run target, not model quality.