Repository navigation
docs(llm): record the local runtime screen and the sufficiency swap on the floor (#1586, #1587) - #1589
Merged
Conversation
This was referenced Oct 20, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #1586
Closes #1587
Part of #1556. The run followed the registration merged in #1588 (
30ff7774). It took 3.71 h of measured model time against a 4.0 h budget. Only numbers are added: no weights, outputs or AMI text.Verdict
llama.cpp(serving Ollama's own GGUF blobs): faster but not equivalent.qwen3:8b, no regression.gemma4:26b-a4bmatched Ollama call for call (delta 0.000).qwen3.6:27bin evals: choose the concept-extraction model with an annotation-free, fast screen #1557.bge-m3fit in 23.56 GiB.qwen3:8bis equivalent on both harnesses.gemma4:26b-a4bshows a clear loss on the sufficiency arm: -0.019 [-0.044, -0.002].gemma4.OLLAMA_MAX_LOADED_MODELS=1). The warm ratio is 0.96 against a 1.5 limit, and the cold question takes 23.88 s against 30 s. The 24 GB emulation throughOLLAMA_GPU_OVERHEADdid not evict: all three models stayed resident in 17 of 18 arm-(b) questions. It is reported as void, not as a pass. No superseding ADR is needed.Full tables, deviations and limits are in
evals/runtime_screen/RESULTS.md. The limits include the 32 GB floor's page cache, so load and swap times here are lower bounds.Checks