Skip to content

Decide paired-seed resampling if seeds are shared experimental units #10

Description

@esaran1

Why

resample_runs_by_cell currently draws seeds independently inside each (intervention, compute) cell. That is correct if cells are independent replicates. It is wrong if seed 0 is one paired experimental unit reused across recipes or scales.

What to do

  1. From the recipe card (Define the actual recipes and what compute=1,2,4,8,16 means #4), state how seeds are assigned.
  2. If seeds are paired, implement a bootstrap that resamples the seed once and applies it across the paired cells.
  3. Add a test with a constructed paired table where the independent-cell bootstrap and the paired bootstrap disagree.
  4. Document which bootstrap asla audit uses in NOTES_ESTIMAND.md and the audit JSON.

Acceptance

  • Pairing policy is written down
  • Implementation matches that policy
  • A regression test locks the chosen resampling unit
  • Audit metadata names the resampling unit

Files

  • asla/analysis/audit.py
  • NOTES_ESTIMAND.md

Do not fabricate

  • Change the bootstrap after seeing the real-data intervals

Metadata

Metadata

Assignees

No one assigned

    Labels

    scienceScientific design, estimands, and claim languagestatsStatistical methods, uncertainty, and tests

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions