# Qwen3.6 27B Q8_0 one-B70 baseline and DNN exactness

Date: 2026-08-08

## Outcome

The pinned target-only Q8_0 artifact fully offloads and serves the requested
32K context on one 32 GiB B70 with F16 KV. The correctness-qualified short
baseline is `15.550257 tok/s` median over the fixed 12-prompt suite. The
complete 4K/17K/31.8K retrieval ladder passed with zero cached tokens.

The archived community-validation build has one important selector boundary:
`GGML_SYCL_ENABLE_DNN=1` is fast but greedy-output nondeterministic for this
model. `GGML_SYCL_ENABLE_DNN=0`, with the broader optimization selector still
enabled, restores replay exactness without a measurable short-decode loss.

## Validated lane identity

```text
model_sha256=f93f517f38e696d35a1a7df2c0e3155a64f4c4dcd662107a146ae263f7fb14ce
runtime_sha256=1a093f09122ceb2851157042c2bbc6281ddb9d4e2de50137502890f9b52fa7d7
gpu_count=1
ctx=4096 short / 32768 long
cache_k=f16
cache_v=f16
GGML_SYCL_ENABLE_VMM=1
GGML_SYCL_ENABLE_GRAPH=0
GGML_SYCL_ENABLE_DNN=0
GGML_SYCL_ENABLE_OPT=1
speculation=none
vision_projector=none
```

This is a validated baseline identity, not yet a localmaxxing submission. A
promotion packet still needs the workspace-standard full-512 timing and the
matching TTFT/request-wall fields.

## Short exact result

Evidence:

`/mnt/fast-ai/bench-results/qwen36-27b-q8-gguf-b70/runs/qwen36-27b-q8_0-f16kv-short-dnn0-exact-20260808T232639Z`

- 12/12 distinct prompts completed both the timed stream and exact replay;
- replay token count matched native `predicted_n` on every row;
- streamed token IDs uniquely aligned to the complete replay on every row;
- stream/replay content and stop type matched on every row;
- native cache-reuse count was zero on both passes for every row;
- median token-1--100 rate `15.550256588703753 tok/s`;
- p10 `15.548172179501558`, mean `15.550044283229964`;
- full offload `65/65`, loaded VRAM `26,573 MiB`;
- clean TERM teardown, port closed, VRAM returned to 43 MiB, empty fault scans.

The `technical-guide` stream contained 127 token-ID events while native
`predicted_n` and the replay contained 128 tokens. Source review showed that
llama.cpp suppresses the SSE partial while a generated byte sequence is
incomplete UTF-8. The unique subsequence alignment located the absent event at
generated-token index 89; generated tokens 1 and 100 both had observed timing
events, so the 99-interval metric remains exact.

## 32K F16-KV result

Correctness-qualified evidence:

`/mnt/fast-ai/bench-results/qwen36-27b-q8-gguf-b70/runs/qwen36-27b-q8_0-f16kv-long32k-dnn0-20260808T233129Z`

- exact retrieval passed at 4,369, 17,274, and 31,846 prompt tokens;
- output counts were 95, 96, and 94 tokens;
- every row reported zero cached tokens;
- median approximate prefill `156.043068 tok/s`;
- decode after TTFT `15.239550`, `14.025002`, and `12.782894 tok/s`;
- full offload `65/65`;
- loaded VRAM `28,372 MiB`;
- model/KV/recurrent/compute device buffers were 25,972.29 / 2,048.00 /
  149.62 / 38.50 MiB;
- clean teardown and empty fault scans.

F16 KV therefore meets the requested 32K ceiling. Q8 KV is not required and
would be a separate quality identity rather than a baseline improvement.

## DNN selector diagnosis

The initial DNN-on fixed suite retained a stable rate near 15.55 tok/s but
failed exact replay on four of twelve prompts. An immediate one-prompt A/A
repeat also diverged, first at token 106, so the failure did not require
cross-prompt state contamination.

Controls:

- DNN on, OPT on: fast, but same-runtime greedy replay failed;
- DNN on, OPT off: immediate A/A exactness passed, but only `5.032736 tok/s`;
- DNN off, OPT on: immediate A/A exactness passed at `15.550969 tok/s`;
- DNN off, OPT on, full suite: 12/12 exact at `15.550257 tok/s` median.

The earlier DNN-on 32K diagnostic also passed semantic retrieval and measured
`160.484166 tok/s` median approximate prefill. The corrected DNN-off row was
`156.043068 tok/s`, about 2.8% lower in this single A/B, while median decode was
effectively unchanged (`14.023740` versus `14.025002`). Use DNN-off as the
correctness authority; retain DNN-on only as a negative performance control.

Failed/diagnostic run directories are preserved under the same external run
root. In particular:

- first sealed offload-log audit failure:
  `qwen36-27b-q8_0-f16kv-smoke-verifiedusb-20260808T225118Z`;
- DNN-on 12-row replay failure:
  `qwen36-27b-q8_0-f16kv-short-twopass-20260808T230629Z`;
- DNN-on immediate A/A replay failure:
  `qwen36-27b-q8_0-f16kv-smoke-aa-replay-20260808T232003Z`;
- OPT-off exact but slow control:
  `qwen36-27b-q8_0-f16kv-smoke-aa-opt0b-20260808T232327Z`;
- DNN-off immediate A/A pass:
  `qwen36-27b-q8_0-f16kv-smoke-aa-dnn0-20260808T232518Z`.

## Next action

Use the DNN-off exact-token JSON as the initial Q8 regression oracle. If a
publishable speed result is wanted, add a separately labeled 512-token run
with TTFT and request-wall fields before submitting anything. Optimization
lanes should branch from isolated source worktrees and must beat this oracle;
MTP and vision remain optional follow-ups, not baseline requirements.