Alibaba's Qwen3.8 27B on one Arc Pro B70 in 4-bit form with llama.cpp, no draft model. The lab's promoted one-card package, rebuilt from pinned source.
What we measured Lab-measured
27.8tok/s decode
Class-balanced median of per-input-class medians using conventional 99-interval rates on the fixed cold 12-prompt suite; target-only and cache-zero. The all-prompt median is 27.824790 tok/s. proof file
target-only = no draft model assisting · cache-zero = a fresh start, nothing pre-computed · 99-interval median = the middle rate across 99 measured stretches of a long answer
Still missing before this becomes an install guide
tested clean-host Intel driver and oneAPI installation
clean-host source build and endpoint replay
batch-shape-invariant greedy output for multi-user serving
What to know
The beginner launch remains a conservative one-slot 8K profile. Separate audits measured exact one-slot HTTP active context through 32K and a stable output-audited 1→64 HTTP aggregate curve. The 64-slot profile nearly fills the card and multi-user greedy token identity is batch-shape-dependent, so it is a capacity result rather than a deterministic serving recommendation.
Qwen3.8 27B is dense; do not compare this one-card aggregate rate as if it were a sparse Qwen-derived MoE or an NVIDIA NVFP4 deployment.
Measured performance profiles Lab-measured
Q4_K_M raw decode with F16 KV Lab-measured
Direct llama-bench raw-engine tg128 using the exact Q4_K_M TP1 lane, one B70, flash attention on, F16 K/V, and 5 repetitions at every displayed depth. This is a context-shape profile, not the 27.82 tok/s realistic-suite headline. Every marker is measured; no point is interpolated or extrapolated. evidence
Q4_K_M raw pp2048 with F16 KV Lab-measured
Direct llama-bench raw-engine pp2048 using the exact Q4_K_M TP1 lane, one B70, flash attention on, F16 K/V, and 5 repetitions at every displayed depth. Every marker is measured; no point is interpolated or extrapolated. evidence
Q4_K_M raw decode with Q8_0 KV Lab-measured
Alternative Q8_0 K/V operating profile on the same exact Q4_K_M TP1 lane, one B70, flash attention on, raw-engine tg128, and 5 repetitions per depth. It is shown separately because KV precision materially changes decode. Every marker is measured; no point is interpolated or extrapolated. evidence
Q4_K_M raw pp2048 with Q8_0 KV Lab-measured
Alternative Q8_0 K/V operating profile on the same exact Q4_K_M TP1 lane, one B70, flash attention on, raw-engine pp2048, and 5 repetitions per depth. Every marker is measured; no point is interpolated or extrapolated. evidence
Qualified HTTP decode over exact active context Lab-measured
One-slot llama-server HTTP completions on the exact accepted Q4_K_M TP1 stack, one B70, F16 KV, 33,024 configured context, cache zero, no truncation or context shift, and 128 returned token IDs. The registered repeated-token fixture is evidence grade C: it fixes context shape but is not natural prose. Every marker is measured; no point is interpolated or extrapolated. evidence
Qualified HTTP TTFT over exact active context Lab-measured
TTFT from the same one-slot exact-token HTTP receipts as the qualified decode curve. Cache was zero and prompt counts were exact at every marker. The repeated-token fixture is evidence grade C and is not a natural-prose latency claim. Every marker is measured; no point is interpolated or extrapolated. evidence
Many people at once Lab-measured
Raw aggregate decode over parallel sequences Lab-measured
Direct llama-batched-bench raw-engine continuous batching with independent pp128 prompts, interleaved tg256 decode, c32768, flash attention on, F16 KV, one B70, and the exact accepted package stack. This is a mechanism ceiling: it does not emit auditable completions and excludes HTTP, JSON, queueing, and server-scheduler overhead. No point is scaled, interpolated, or extrapolated. evidence
Output-audited HTTP aggregate decode Lab-measured
llama.cpp native HTTP /completion on one B70 with 64 slots, 32K total F16 KV context, prompt cache and slot similarity disabled, unique short prompts, and 128 returned raw token IDs per request. Each marker is the median of two preregistered fresh-server attempts whose relative range was at most 2.03%. Output isolation passed with no cross-base oracle collision, but multi-user greedy token identity is batch-shape-dependent. No point is interpolated or extrapolated. evidence
How much faster could this get? Projected — not measured
The ML Bottleneck physics engine projects what stock software, a tuned run, and the physical ceiling look like for this exact model, compression, card count, and software. The grade is optimization headroom against the tuned-run target, not model quality.