Skip to content

Add Chinchilla uncertainty only if an identifiable N,D grid exists #11

Description

@esaran1

Why

--fit-form chinchilla can point-project when params_n and tokens_d are present. Gate uncertainty and projection bootstrap still raise NotImplementedError for that form. A one-dimensional compute fit can also look like saturation when the true dependence is separate in N and D.

What to do

Only start this if the real grid varies N and D independently. Then:

  1. Add a stratified bootstrap (or another stated method) for Chinchilla projections.
  2. Fail loudly on collinear log N, log D designs; do not fall back to compute power laws.
  3. Project to a stated target (N, D), not by reading those values from the target BPB rows if the target design is supposed to be chosen in advance.
  4. Tests: recover known two-term parameters; reject coupled D=N^k designs; reject missing columns.

Acceptance

  • Uncertainty exists or the CLI still refuses gate/CIs for Chinchilla rather than pretending
  • No silent fallback to compute_power_law
  • Identifiability checks are tested

Files

  • asla/models.py
  • asla/analysis/fits.py
  • asla/analysis/gate.py
  • tests/test_chinchilla.py

Do not fabricate

  • Invent N,D for compute-only runs
  • Enable Chinchilla on the placeholder 45-run relative-compute grid unless those columns are actually logged

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or requeststatsStatistical methods, uncertainty, and tests

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions