feat(eval): add --runs N flag with median scoring - #111

Closed

yogeshchoudhary147 wants to merge 10 commits into

mainfrom

feat/runs-n-median-scoring

Commits on Jul 22, 2026

feat(eval): add --runs N flag with median scoring
yogeshchoudhary147
committed
fix(eval): address coderabbit review comments
yogeshchoudhary147
committed
revert: restore merged check for hasErrors
yogeshchoudhary147
committed
fix(eval): normalize tools in mergeIntoOutput key to match resultKey
yogeshchoudhary147
committed
docs(eval): add docstrings to medianAgentResult and medianBaselineResult
yogeshchoudhary147
committed
fix(eval): reject float values for --runs; document --runs in AGENTS.md
yogeshchoudhary147
committed
fix(eval): warn on partial run failures excluded from aggregation
yogeshchoudhary147
committed
fix(eval): strip --runs from subprocess args to prevent infinite recursion
yogeshchoudhary147
committed
fix(eval): cap --runs at MAX_RUNS=10 to prevent unbounded subprocess expansion
yogeshchoudhary147
committed

Commits on Jul 23, 2026

fix(eval): address coderabbit review comments on aggregation correctness
yogeshchoudhary147
committed