Skip to content

Add Stage 2 target catalog selection#1084

Draft
anth-volk wants to merge 1 commit into
agent/stage-2/pr-2c-payload-reader-writerfrom
agent/stage-2/pr-2d-target-catalog-selection
Draft

Add Stage 2 target catalog selection#1084
anth-volk wants to merge 1 commit into
agent/stage-2/pr-2c-payload-reader-writerfrom
agent/stage-2/pr-2d-target-catalog-selection

Conversation

@anth-volk
Copy link
Copy Markdown
Collaborator

Fixes #1083

Summary

  • Add a reusable Stage 2 target catalog/selection boundary with TargetCatalogReader, TargetCatalog, TargetSelectionPolicy, and TargetSelectionResult.
  • Route target config filtering through the selection service and keep UnifiedMatrixBuilder as the matrix consumer via selected target metadata.
  • Emit calibration_targets.jsonl and calibration_target_facets.json, include selection checksum metadata, and reference both artifacts from the Stage 2 contract and manifest output bundle.
  • Document the target catalog/selection artifacts in the pipeline map and cover target rows, facets, disabled reporting, additive expressions, stable order/checksum, and fake fit joins.

Validation

  • uv run --no-sync python -m py_compile policyengine_us_data/calibration_package/targets.py policyengine_us_data/calibration_package/specs.py policyengine_us_data/calibration_package/__init__.py policyengine_us_data/calibration/unified_calibration.py policyengine_us_data/calibration/unified_matrix_builder.py policyengine_us_data/stage_contracts/calibration_package.py
  • uv run --no-sync ruff check policyengine_us_data/calibration/unified_calibration.py policyengine_us_data/calibration/unified_matrix_builder.py policyengine_us_data/calibration_package/__init__.py policyengine_us_data/calibration_package/specs.py policyengine_us_data/calibration_package/targets.py policyengine_us_data/stage_contracts/calibration_package.py tests/unit/calibration_package/test_specs.py tests/unit/calibration_package/test_targets.py tests/unit/test_calibration_package_stage_contract.py tests/unit/test_pipeline_docs_extractor.py
  • uv run --no-sync ruff format --check policyengine_us_data/calibration/unified_calibration.py policyengine_us_data/calibration/unified_matrix_builder.py policyengine_us_data/calibration_package/__init__.py policyengine_us_data/calibration_package/specs.py policyengine_us_data/calibration_package/targets.py policyengine_us_data/stage_contracts/calibration_package.py tests/unit/calibration_package/test_specs.py tests/unit/calibration_package/test_targets.py tests/unit/test_calibration_package_stage_contract.py tests/unit/test_pipeline_docs_extractor.py
  • uv run --no-sync python -m pytest tests/unit/calibration_package/test_targets.py tests/unit/calibration_package/test_specs.py tests/unit/test_calibration_package_stage_contract.py tests/unit/test_pipeline_docs_extractor.py (47 passed)
  • uv run --no-sync python -m pytest tests/unit/calibration/test_target_config.py tests/unit/calibration/test_unified_matrix_builder.py::TestQueryTargets (39 passed)
  • uv run --no-sync python -m pytest tests/unit/test_remote_calibration_runner.py tests/unit/test_pipeline_doc_guards.py tests/unit/calibration_package/test_payload.py tests/unit/build_outputs/test_geography_loader.py (24 passed, 1 skipped from tests/unit/test_pipeline.py when run separately due local Modal import skip)
  • uv run --no-sync python -m pytest tests/unit/test_pipeline_source_contracts.py tests/unit/test_pipeline_docs_extractor.py tests/unit/test_pipeline_doc_guards.py (18 passed)
  • uv run --no-sync --with pyyaml python scripts/extract_pipeline_docs.py --json /private/tmp/us-data-pr-2d-pipeline-docs/pipeline_map.json --api-json /private/tmp/us-data-pr-2d-pipeline-docs/pipeline_api.json --markdown /private/tmp/us-data-pr-2d-pipeline-docs/pipeline-map.md
  • uv run --no-sync --with pyyaml python scripts/run_quality_guards.py
  • git diff --check
  • make lint

Notes

  • The selector also emitted the full tests/unit/calibration/test_unified_calibration.py and several broader matrix/ETL/query tests. I ran the directly affected target-config and query-target tests plus the new target artifact/contract/pipeline docs coverage; broader full selector pytest was not run to avoid the known long collection/runtime behavior in this local environment.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant