neural.download

Learn · concept guides

The levers that set your tokens per second

Plain-language guides to the variables that actually move local-LLM speed and capacity. Each one explains what the knob is, shows what we measured when we turned it, tells you what to pick, and warns you where it bites. Numbers are labelled by where they came from.

How to read these

Every figure carries a source badge: Lab-measured we ran it on our own B70s; Community reported by a contributor, checked for consistency but not re-run here; Spec / vendor a published number we have not benchmarked. We never present a spec figure as a measured result.

Speed comes from a stack of choices — the model, how it's quantized, how the KV cache is stored, how long your context is, which decoding trick you use, which runtime, and which card. Pull the wrong lever and you can halve your rate without noticing. Start anywhere; they cross-link.

Pick your setup

Guide 01

Models

Dense vs mixture-of-experts, parameters vs VRAM, and which model on our board fits your card and job.
Read guide →
Guide 02

Quantization

Q4/Q5/Q8/INT4/FP8 explained — and the surprise that a smaller file can decode slower.
Read guide →
Guide 08

Hardware

B65 vs B70 vs Intel Max 1100 vs AMD R9700 Pro vs GB10 Spark — VRAM, bandwidth, and what we've actually tested.
Read guide →
Guide 08b

B70 ECC & VRAM

How to choose between roughly 28 GB with memory error correction and 32 GB without it—and change the setting safely.
Read guide →
Guide 10

Host tuning

Why the same recipe ran 1.8x slower on a second host, when XPU graph capture gets it back, a five-minute probe, and which host fixes changed speed, stability, or nothing.
Read guide →
Guide 07

Runtimes

vLLM vs llama.cpp SYCL vs IPEX/XPU — which one gives you MTP, which one just works on one card.
Read guide →

Tune for speed

Guide 03

KV precision

KV8 vs KV16 vs FP8 — why an 8-bit KV cache saves memory but its decode cost grows with context.
Read guide →
Guide 04

Context length

How decode, prefill, and time-to-first-token change as the prompt fills from 0 to 32K.
Read guide →
Guide 05

Prefill vs clock

Why prefill scales with clock and power but decode scales with memory bandwidth — and what that means for tuning.
Read guide →
Guide 06

MTP 1·2·3·4

Multi-token prediction: how the draft-depth ladder trades cost for acceptance, and why the sweet spot moves and how to find yours.
Read guide →
Guide 06b

Spec methods

DFlash (a small separate draft model) vs MTP vs no speculation — what each buys you and what it can cost in determinism.
Read guide →
Guide 09

Many users

One B70 measured serving 64 people at once — 1,039 combined tok/s — and why per-user speed falls while total climbs.
Read guide →
What to be aware of

These guides describe our lab's Intel Arc Pro B70 lane and the community reports around it. The directions (bigger KV precision costs more at long context, MTP2 is a common knee — but sweep it; some quant/runtime combinations regress, prefill tracks clock) generalize well; the exact numbers are specific to our tuned build and hardware. Re-measure on your own machine before treating any figure as a guarantee.