You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Measure ADR-0076's sufficiency model swap on the 32 GB floor #1587
The PR #1585 bar passed only on the 48 GB development machine, where qwen3:8b and gemma4:26b-a4b stayed resident together. On the 32 GB product floor (24 GB budget) they do not fit together: the Ollama server log puts the pair at about 24.1 GiB. So every query there would swap the two models twice (check, then answer). That cost is unmeasured, and the bar written for #1585 did not separate the host machine from the floor.
Measurement
Repeat the #1585 protocol, which is 3 interleaved sequences per arm of one cold question then 5 warm, with one loaded model enforced. Use a second Ollama server bound to another port with OLLAMA_MAX_LOADED_MODELS=1 and the same model store, and do not touch the user's server. Apply the same bar: warm median at most 1.5x of arm (a), and the cold first question at most 30 s.
Done when
The result is recorded.
If the bar fails, a superseding ADR changes the sufficiency default to null (follow model:), or to the tied qwen3:14b if it fits with qwen3:8b. Or the default is made memory-aware.
Recorded in #1589 (evals/runtime_screen/RESULTS.md). ADR-0076's bar passes with one chat model resident, so no superseding ADR.
Configuration A: a second Ollama server with OLLAMA_MAX_LOADED_MODELS=1, where only qwen3:8b stayed resident after each question.
Cold first question: 23.88 s against 22.96 s for arm (a); the bar is 30 s.
Warm median: 18.58 s against 19.34 s, a ratio of 0.96; the bar is 1.5.
Configuration B, a 24 GB budget through OLLAMA_GPU_OVERHEAD, is void. Ollama never evicted: all three models stayed resident after 17 of 18 arm-(b) questions, because ollama ps puts gemma4 at 985 MB.
As an estimate, not a measurement: with bge-m3 resident on the floor, arm (b) would pay two swaps of about 3.3 s plus 1.1 s, giving roughly 16.4 / 14.24 = 1.15. That is still under 1.5, provided the floor's page cache keeps both weight files.
Activity
docs(llm): pre-register the local runtime screen and the sufficiency …
docs(llm): record the local runtime screen and the sufficiency swap o…
jasonssdev commentedon Oct 20, 2026
Recorded in #1589 (
evals/runtime_screen/RESULTS.md). ADR-0076's bar passes with one chat model resident, so no superseding ADR.Configuration A: a second Ollama server with
OLLAMA_MAX_LOADED_MODELS=1, where onlyqwen3:8bstayed resident after each question.Configuration B, a 24 GB budget through
OLLAMA_GPU_OVERHEAD, is void. Ollama never evicted: all three models stayed resident after 17 of 18 arm-(b) questions, becauseollama psputs gemma4 at 985 MB.As an estimate, not a measurement: with
bge-m3resident on the floor, arm (b) would pay two swaps of about 3.3 s plus 1.1 s, giving roughly 16.4 / 14.24 = 1.15. That is still under 1.5, provided the floor's page cache keeps both weight files.