B70 fit High
Quality evidence Strong scoped deployment evidence
Interest 123.24K
Measured results 9
Packets and recipes The deployment variants of this family, at every maturity.
What has been classified Dense scorecards summarize every declared combination; measured slices retain exact evidence links.
serving profile × cards Target-only · graph off · F16 KV · standard 8K serving profile. Rows keep stock and patched identities separate; a blank topology is never inferred from another row. Fixed: mtp=0 · graph=off · kv=f16 · configured_max_context_tokens=8192.
✓ Measured 1, 9B Q8_0 · stock 1·9B Q8_0 · stock 49.59 tok/s guide · evidence ✓ Measured 1, 35B-A3B Q4_K_M · current optimized 1·35B-A3B Q4_K_M · current optimized 132.47 tok/s guide · evidence ✓ Measured 1, 35B-A3B Q4_K_M · stock 1·35B-A3B Q4_K_M · stock 104.84 tok/s evidence ✓ Measured 2, 35B-A3B Q4_K_M · stock 2·35B-A3B Q4_K_M · stock 102.2 tok/s evidence Codes in the rows: D = decode tok/s · P = prefill tok/s · T = ms to first token · AR = share of drafted tokens accepted · A = combined tok/s.
5 untested combinations 2·9B Q8_0 · stock4·9B Q8_0 · stock2·35B-A3B Q4_K_M · current optimized4·35B-A3B Q4_K_M · current optimized4·35B-A3B Q4_K_M · stock
Full matrix and exact selectors 9B Q8_0 context × cards Stock 9B Q8_0 · target-only · graph off · F16 KV · raw pp2048/tg128 · five repetitions per marker. Fixed: revision=ornith-1.5-9b · variant=Q8_0 dense · runtime=llama.cpp SYCL 9fee29e · profile_id=ornith-9b-q8-stock-context · split_mode=single device · mtp=0 · graph=off · kv=f16.
✓ Measured 1, 0 1·0 D50.29 · P3,184 evidence ✓ Measured 1, 2048 1·2048 D49.34 · P1,623 evidence ✓ Measured 1, 4096 1·4096 D48.55 · P1,601 evidence ✓ Measured 1, 8192 1·8192 D47.04 · P1,569 evidence ✓ Measured 1, 16384 1·16384 D44.35 · P1,489 evidence ✓ Measured 1, 24576 1·24576 D41.96 · P1,449 evidence ✓ Measured 1, 32768 1·32768 D39.84 · P1,314 evidence Codes in the rows: D = decode tok/s · P = prefill tok/s · T = ms to first token · AR = share of drafted tokens accepted · A = combined tok/s.
35B-A3B current context × cards Current twelve-feature Q4_K_M overlay plus accepted copy-offload setting · target-only · graph off · F16 KV · raw pp2048/tg128 · five repetitions per marker. Fixed: revision=ornith-1.5-35b-a3b · variant=Q4_K_M MoE twelve-feature stack plus copy-offload setting · runtime=llama.cpp SYCL 9fee29e plus lab patch · profile_id=ornith-35b-q4-twelve-feature-context · split_mode=single device · mtp=0 · graph=off · kv=f16.
✓ Measured 1, 0 1·0 D141.92 · P1,422 evidence ✓ Measured 1, 2048 1·2048 D136.85 · P1,344 evidence ✓ Measured 1, 4096 1·4096 D133.33 · P1,338 evidence ✓ Measured 1, 8192 1·8192 D126.83 · P1,301 evidence ✓ Measured 1, 16384 1·16384 D116.27 · P1,239 evidence ✓ Measured 1, 24576 1·24576 D106.97 · P1,213 evidence ✓ Measured 1, 32768 1·32768 D99.61 · P1,116 evidence Codes in the rows: D = decode tok/s · P = prefill tok/s · T = ms to first token · AR = share of drafted tokens accepted · A = combined tok/s.
35B concurrent sequences × cards Accepted Ornith 35B Q4_K_M stack · one-card raw-engine continuous batching · independent 1,024-token prompts · 256 generated tokens per sequence · graph off · F16 KV. Fixed: revision=ornith-1.5-35b-a3b · variant=Q4_K_M MoE accepted twelve-feature stack · runtime=llama.cpp SYCL 9fee29e plus accepted lab patch · profile_id=ornith-35b-q4-accepted-stack-batched · measurement_class=raw-engine-continuous-batching · split_mode=single device · mtp=0 · graph=off · kv=f16 · configured_max_context_tokens=65536.
✓ Measured 1, c1 1·c1 D98.02 · A98.02 evidence ✓ Measured 1, c2 1·c2 D51.34 · A102.67 evidence ✓ Measured 1, c4 1·c4 D29.72 · A118.88 evidence ✓ Measured 1, c8 1·c8 D18.29 · A146.28 evidence ✓ Measured 1, c16 1·c16 D10.16 · A162.5 evidence ✓ Measured 1, c32 1·c32 D6.77 · A216.51 evidence Codes in the rows: D = decode tok/s · P = prefill tok/s · T = ms to first token · AR = share of drafted tokens accepted · A = combined tok/s.
Measured results Every number links to its proof.
family data
Speed as the conversation grows Separate dense Q8_0 and MoE Q4_K_M architectures on one B70; shared axes do not imply transferable speed
Decode Prefill
0 39.7 79.5 119.2 158.9 0 2K 4K 8K 16K 24K 32K active context tokens · tok/s 9B dense · Q8_0 · Decode (decode_tok_s) · active context tokens=0.0 · value=50.291618 tok/s 9B dense · Q8_0 · Decode (decode_tok_s) · active context tokens=2048.0 · value=49.338288 tok/s 9B dense · Q8_0 · Decode (decode_tok_s) · active context tokens=4096.0 · value=48.554943 tok/s 9B dense · Q8_0 · Decode (decode_tok_s) · active context tokens=8192.0 · value=47.044737 tok/s 9B dense · Q8_0 · Decode (decode_tok_s) · active context tokens=16384.0 · value=44.349773 tok/s 9B dense · Q8_0 · Decode (decode_tok_s) · active context tokens=24576.0 · value=41.957791 tok/s 9B dense · Q8_0 · Decode (decode_tok_s) · active context tokens=32768.0 · value=39.83848 tok/s 35B-A3B MoE · Q4_K_M · current 12-feature · Decode (decode_tok_s) · active context tokens=0.0 · value=141.917514 tok/s 35B-A3B MoE · Q4_K_M · current 12-feature · Decode (decode_tok_s) · active context tokens=2048.0 · value=136.848608 tok/s 35B-A3B MoE · Q4_K_M · current 12-feature · Decode (decode_tok_s) · active context tokens=4096.0 · value=133.329784 tok/s 35B-A3B MoE · Q4_K_M · current 12-feature · Decode (decode_tok_s) · active context tokens=8192.0 · value=126.829116 tok/s 35B-A3B MoE · Q4_K_M · current 12-feature · Decode (decode_tok_s) · active context tokens=16384.0 · value=116.267169 tok/s 35B-A3B MoE · Q4_K_M · current 12-feature · Decode (decode_tok_s) · active context tokens=24576.0 · value=106.967195 tok/s 35B-A3B MoE · Q4_K_M · current 12-feature · Decode (decode_tok_s) · active context tokens=32768.0 · value=99.614237 tok/s 0 891.7 1,783 2,675 3,567 0 2K 4K 8K 16K 24K 32K active context tokens · tok/s 9B dense · Q8_0 · Prefill (prefill_tok_s) · active context tokens=0.0 · value=3184.490007 tok/s 9B dense · Q8_0 · Prefill (prefill_tok_s) · active context tokens=2048.0 · value=1623.026366 tok/s 9B dense · Q8_0 · Prefill (prefill_tok_s) · active context tokens=4096.0 · value=1601.167744 tok/s 9B dense · Q8_0 · Prefill (prefill_tok_s) · active context tokens=8192.0 · value=1568.596728 tok/s 9B dense · Q8_0 · Prefill (prefill_tok_s) · active context tokens=16384.0 · value=1489.240859 tok/s 9B dense · Q8_0 · Prefill (prefill_tok_s) · active context tokens=24576.0 · value=1448.815985 tok/s 9B dense · Q8_0 · Prefill (prefill_tok_s) · active context tokens=32768.0 · value=1313.738568 tok/s 35B-A3B MoE · Q4_K_M · current 12-feature · Prefill (prefill_tok_s) · active context tokens=0.0 · value=1422.407346 tok/s 35B-A3B MoE · Q4_K_M · current 12-feature · Prefill (prefill_tok_s) · active context tokens=2048.0 · value=1343.876428 tok/s 35B-A3B MoE · Q4_K_M · current 12-feature · Prefill (prefill_tok_s) · active context tokens=4096.0 · value=1338.212192 tok/s 35B-A3B MoE · Q4_K_M · current 12-feature · Prefill (prefill_tok_s) · active context tokens=8192.0 · value=1301.167019 tok/s 35B-A3B MoE · Q4_K_M · current 12-feature · Prefill (prefill_tok_s) · active context tokens=16384.0 · value=1238.569995 tok/s 35B-A3B MoE · Q4_K_M · current 12-feature · Prefill (prefill_tok_s) · active context tokens=24576.0 · value=1212.927246 tok/s 35B-A3B MoE · Q4_K_M · current 12-feature · Prefill (prefill_tok_s) · active context tokens=32768.0 · value=1115.599876 tok/s
9B dense · Q8_0 0: 50.29 · 2K: 49.34 · 4K: 48.55 · 8K: 47.04 · 16K: 44.35 · 24K: 41.96 · 32K: 39.8435B-A3B MoE · Q4_K_M · current 12-feature 0: 141.92 · 2K: 136.85 · 4K: 133.33 · 8K: 126.83 · 16K: 116.27 · 24K: 106.97 · 32K: 99.61
9B dense · Q8_0 0: 3,184 · 2K: 1,623 · 4K: 1,601 · 8K: 1,569 · 16K: 1,489 · 24K: 1,449 · 32K: 1,31435B-A3B MoE · Q4_K_M · current 12-feature 0: 1,422 · 2K: 1,344 · 4K: 1,338 · 8K: 1,301 · 16K: 1,239 · 24K: 1,213 · 32K: 1,116
All measured metric summaries Decode 9B dense · Q8_0 0: 50.29 · 2K: 49.34 · 4K: 48.55 · 8K: 47.04 · 16K: 44.35 · 24K: 41.96 · 32K: 39.8435B-A3B MoE · Q4_K_M · current 12-feature 0: 141.92 · 2K: 136.85 · 4K: 133.33 · 8K: 126.83 · 16K: 116.27 · 24K: 106.97 · 32K: 99.61Prefill 9B dense · Q8_0 0: 3,184 · 2K: 1,623 · 4K: 1,601 · 8K: 1,569 · 16K: 1,489 · 24K: 1,449 · 32K: 1,31435B-A3B MoE · Q4_K_M · current 12-feature 0: 1,422 · 2K: 1,344 · 4K: 1,338 · 8K: 1,301 · 16K: 1,239 · 24K: 1,213 · 32K: 1,116
evidence 1 · evidence 2
Tuned 35B speed as the conversation grows Historical eleven-feature profile and current twelve-feature plus copy-offload profile; every marker is measured independently
Decode Prefill
0 39.7 79.5 119.2 158.9 0 2K 4K 8K 16K 24K 32K active context tokens · tok/s historical 11-feature · Decode (decode_tok_s) · active context tokens=0.0 · value=138.97752 tok/s historical 11-feature · Decode (decode_tok_s) · active context tokens=2048.0 · value=133.765934 tok/s historical 11-feature · Decode (decode_tok_s) · active context tokens=4096.0 · value=130.620489 tok/s historical 11-feature · Decode (decode_tok_s) · active context tokens=8192.0 · value=124.209778 tok/s historical 11-feature · Decode (decode_tok_s) · active context tokens=16384.0 · value=113.31086 tok/s historical 11-feature · Decode (decode_tok_s) · active context tokens=24576.0 · value=104.315441 tok/s historical 11-feature · Decode (decode_tok_s) · active context tokens=32768.0 · value=96.995532 tok/s current 12-feature + copy-offload · Decode (decode_tok_s) · active context tokens=0.0 · value=141.917514 tok/s current 12-feature + copy-offload · Decode (decode_tok_s) · active context tokens=2048.0 · value=136.848608 tok/s current 12-feature + copy-offload · Decode (decode_tok_s) · active context tokens=4096.0 · value=133.329784 tok/s current 12-feature + copy-offload · Decode (decode_tok_s) · active context tokens=8192.0 · value=126.829116 tok/s current 12-feature + copy-offload · Decode (decode_tok_s) · active context tokens=16384.0 · value=116.267169 tok/s current 12-feature + copy-offload · Decode (decode_tok_s) · active context tokens=24576.0 · value=106.967195 tok/s current 12-feature + copy-offload · Decode (decode_tok_s) · active context tokens=32768.0 · value=99.614237 tok/s 0 398.3 796.5 1,195 1,593 0 2K 4K 8K 16K 24K 32K active context tokens · tok/s historical 11-feature · Prefill (prefill_tok_s) · active context tokens=0.0 · value=1397.348168 tok/s historical 11-feature · Prefill (prefill_tok_s) · active context tokens=2048.0 · value=1326.651018 tok/s historical 11-feature · Prefill (prefill_tok_s) · active context tokens=4096.0 · value=1312.167851 tok/s historical 11-feature · Prefill (prefill_tok_s) · active context tokens=8192.0 · value=1284.796086 tok/s historical 11-feature · Prefill (prefill_tok_s) · active context tokens=16384.0 · value=1220.200931 tok/s historical 11-feature · Prefill (prefill_tok_s) · active context tokens=24576.0 · value=1196.724116 tok/s historical 11-feature · Prefill (prefill_tok_s) · active context tokens=32768.0 · value=1101.625369 tok/s current 12-feature + copy-offload · Prefill (prefill_tok_s) · active context tokens=0.0 · value=1422.407346 tok/s current 12-feature + copy-offload · Prefill (prefill_tok_s) · active context tokens=2048.0 · value=1343.876428 tok/s current 12-feature + copy-offload · Prefill (prefill_tok_s) · active context tokens=4096.0 · value=1338.212192 tok/s current 12-feature + copy-offload · Prefill (prefill_tok_s) · active context tokens=8192.0 · value=1301.167019 tok/s current 12-feature + copy-offload · Prefill (prefill_tok_s) · active context tokens=16384.0 · value=1238.569995 tok/s current 12-feature + copy-offload · Prefill (prefill_tok_s) · active context tokens=24576.0 · value=1212.927246 tok/s current 12-feature + copy-offload · Prefill (prefill_tok_s) · active context tokens=32768.0 · value=1115.599876 tok/s
historical 11-feature 0: 138.98 · 2K: 133.77 · 4K: 130.62 · 8K: 124.21 · 16K: 113.31 · 24K: 104.32 · 32K: 97current 12-feature + copy-offload 0: 141.92 · 2K: 136.85 · 4K: 133.33 · 8K: 126.83 · 16K: 116.27 · 24K: 106.97 · 32K: 99.61
historical 11-feature 0: 1,397 · 2K: 1,327 · 4K: 1,312 · 8K: 1,285 · 16K: 1,220 · 24K: 1,197 · 32K: 1,102current 12-feature + copy-offload 0: 1,422 · 2K: 1,344 · 4K: 1,338 · 8K: 1,301 · 16K: 1,239 · 24K: 1,213 · 32K: 1,116
All measured metric summaries Decode historical 11-feature 0: 138.98 · 2K: 133.77 · 4K: 130.62 · 8K: 124.21 · 16K: 113.31 · 24K: 104.32 · 32K: 97current 12-feature + copy-offload 0: 141.92 · 2K: 136.85 · 4K: 133.33 · 8K: 126.83 · 16K: 116.27 · 24K: 106.97 · 32K: 99.61Prefill historical 11-feature 0: 1,397 · 2K: 1,327 · 4K: 1,312 · 8K: 1,285 · 16K: 1,220 · 24K: 1,197 · 32K: 1,102current 12-feature + copy-offload 0: 1,422 · 2K: 1,344 · 4K: 1,338 · 8K: 1,301 · 16K: 1,239 · 24K: 1,213 · 32K: 1,116
evidence 1 · evidence 2
35B: stock software by card count Q4_K_M stock serving comparison; the optimized one-card packet is a separate patched identity
stock layer placement · B70 cards 1 104.81–104.84 tok/s Decode lab measurement; quality scope in evidence · evidence stock layer placement · B70 cards 2 102.01–102.2 tok/s Decode lab measurement; quality scope in evidence · evidence
evidence 1
35B: many users at once Raw llama-batched-bench engine throughput; combined and per-sequence rates are both measured, and sequence count is not relabeled as HTTP users
Combined Decode
0 60.6 121.2 181.9 242.5 1 2 4 8 16 32 concurrent engine sequences · tok/s accepted 12-feature stack · Combined (aggregate_tok_s) · concurrent engine sequences=1.0 · value=98.024651 tok/s accepted 12-feature stack · Combined (aggregate_tok_s) · concurrent engine sequences=2.0 · value=102.671738 tok/s accepted 12-feature stack · Combined (aggregate_tok_s) · concurrent engine sequences=4.0 · value=118.882942 tok/s accepted 12-feature stack · Combined (aggregate_tok_s) · concurrent engine sequences=8.0 · value=146.283051 tok/s accepted 12-feature stack · Combined (aggregate_tok_s) · concurrent engine sequences=16.0 · value=162.503937 tok/s accepted 12-feature stack · Combined (aggregate_tok_s) · concurrent engine sequences=32.0 · value=216.513077 tok/s 0 27.4 54.9 82.3 109.8 1 2 4 8 16 32 concurrent engine sequences · tok/s accepted 12-feature stack · Decode (decode_tok_s) · concurrent engine sequences=1.0 · value=98.024651 tok/s accepted 12-feature stack · Decode (decode_tok_s) · concurrent engine sequences=2.0 · value=51.335869 tok/s accepted 12-feature stack · Decode (decode_tok_s) · concurrent engine sequences=4.0 · value=29.720736 tok/s accepted 12-feature stack · Decode (decode_tok_s) · concurrent engine sequences=8.0 · value=18.285381 tok/s accepted 12-feature stack · Decode (decode_tok_s) · concurrent engine sequences=16.0 · value=10.156496 tok/s accepted 12-feature stack · Decode (decode_tok_s) · concurrent engine sequences=32.0 · value=6.766034 tok/s
accepted 12-feature stack 1: 98.02 · 2: 102.67 · 4: 118.88 · 8: 146.28 · 16: 162.5 · 32: 216.51
accepted 12-feature stack 1: 98.02 · 2: 51.34 · 4: 29.72 · 8: 18.29 · 16: 10.16 · 32: 6.77
All measured metric summaries Combined accepted 12-feature stack 1: 98.02 · 2: 102.67 · 4: 118.88 · 8: 146.28 · 16: 162.5 · 32: 216.51Decode accepted 12-feature stack 1: 98.02 · 2: 51.34 · 4: 29.72 · 8: 18.29 · 16: 10.16 · 32: 6.77
evidence 1 Fine print Qwen3.5-derived sibling architectures. Publisher/version grouping only: the dense qwen35 9B and qwen35moe 35B-A3B variants are architecture-distinct, so no model-specific kernel, speed, output, or quality transfer is implied. Measurements, artifact hashes, outputs, quality decisions, and speed stay pinned to their exact recorded identity.
1 combination the lab has stopped pursuing: Both artifacts declare a 262,144-token native window, but these two packet profiles have measured active-context evidence only through 32,768 tokens. evidence Exact selectors are in the family data .
Many people at once Lab-measured 32 simultaneous users share 217 combined tok/s on 1 card (Q4_K_M MoE accepted twelve-feature stack); the full curve is under Measured results and in the multi-user report .
How much faster could this get? No projection No projection is shown: the family has no curated headline measurement yet. The measured numbers above stand on their own.