Run the first real audit: projection vs single-scale

[esaran1]

Depends on

• Preregister the paper estimand and seed-dependence assumptions #3 preregistered estimand
• completed runs.parquet from the 45-job grid

Why

The headline comparison is whether scaling-law projection beats ranking by the largest cheap run. Synthetic demos are not evidence.

This repo also reports ensemble, gate, and racing when those rankers are available. Those are extra comparisons. They must not silently receive budgets the projection/single-scale pair does not get. The paper headline remains projection vs single_scale_ranker.

What to do

Fast laptop check:

asla audit \
  --runs runs.parquet \
  --target 16 \
  --budgets 1 2 4 \
  --intermediate-budget 8 \
  --estimand <PREREGISTERED> \
  --fast \
  --out results/audit_fast.json

Then paper-grade (no --fast, preferably HPC):

asla audit \
  --runs runs.parquet \
  --target 16 \
  --budgets 1 2 4 \
  --intermediate-budget 8 \
  --estimand <PREREGISTERED> \
  --out results/audit.json

Inspect:

• rankers.projection_ranker vs rankers.single_scale_ranker
• whether intervals overlap
• under_seeded_cells
• noise.noise_band_estimated
• ranker_availability.gate_ranker (gate only if intermediate budget was supplied)
• audit_metadata.input_sha256

Acceptance

• Fast and full JSON exist and share the same input hash
• Write-up states whether projection clearly beats single-scale, is similar, or is worse, using intervals
• Under-seeded cells are reported as a limitation, not hidden
• Gate/ensemble/race results are omitted or labeled if their budget contract was not met

Do not fabricate

• Report --fast intervals as paper numbers
• Claim a population mis-selection rate from one table
• Refit using the target or intermediate budget in the projection ranker