Skip to content

docs(gemma4): specify the two-group KV cache contract #809

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Draft
FeathBow wants to merge 1 commit into
base: main
Choose a base branch
from
Draft

docs(gemma4): specify the two-group KV cache contract #809

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions docs/index.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -64,6 +64,7 @@ Organized by domain (model line / subsystem / playbook / lesson) instead of by l
| --- | --- |
| `models/gemma4/tokenizer.md` | Gemma 4 tokenizer/chat-template contracts, gated against a Hugging Face reference (token ids plus all five chat renders, content flattened to strings): BOS comes only from the standalone `chat_template.jinja` (which opens a thought channel and accepts a native system role), EOS is declared three times with three values, the published defaults are sampled rather than greedy, image/audio tokens encode straight from user text so text-only serving must reject them at admission, and one divergence stays open — the server's default content format adds a trailing space to system turns. |
| `models/gemma4/hf-golden.md` | The Hugging Face reference for 12B: layer-boundary activations at both ends of both layer types plus top-64 logprobs, over a single-token, a nine-token and a 1024-token (exactly the sliding window) case. Pins three facts the forward path has to match — the embedding scale is bf16 62.0 rather than `sqrt(3840)`, text attention is causal, and `layer_scalar` applies to the layer output after both residual adds. Regeneration is byte-identical and checked with sha256. |
| `models/gemma4/kv-cache.md` | The two-family KV contract (256 sliding / 512 full attention, two lifetimes): two coordinate systems with the never-conflate invariant frozen — slots derive in-kernel from the resident origin; storage declared along orthogonal axes (lifecycle, paged/slab live storage, reference/copy checkpoint materialization) with observable semantics frozen, not a closed pairing; staged serving — unreclaimed pool first, paged window with batching; a frozen checkpoint query — the largest position at which every required family is exactly recoverable, and a global prefix without its local window is worthless; capacity and TP accounting; four frozen behaviors with a default-off prefix-cache amendment. |

## models / glm52

Expand Down
Loading