Skip to content

fix(conversion): isolate malformed groups and trust converter output (fixes #112) - #131

Draft
jakobtfaber wants to merge 1 commit into
mainfrom
agent/convert-groups-isolation
Draft

fix(conversion): isolate malformed groups and trust converter output (fixes #112)#131
jakobtfaber wants to merge 1 commit into
mainfrom
agent/convert-groups-isolation

Conversation

@jakobtfaber

Copy link
Copy Markdown
Contributor

Summary

Fixes both conversion follow-ups reported in #112:

  • moves timestamp extraction and ISOT parsing inside the per-group exception boundary, so one malformed group cannot abort later conversions
  • uses the authoritative converted and skipped group IDs returned by convert_subband_groups_to_ms instead of reconstructing output names with a selector timestamp glob
  • only reuses skipped MS paths when skip_existing=True and the returned path exists
  • consolidates timestamp-string coverage into the existing conversion behavior tests

Root cause

convert_groups parsed the two-minute conversion window before its per-group try, so malformed timestamps escaped the isolation contract. After conversion it searched for {selector_timestamp}*.ms, but the core converter derives its output group ID from a separate database query; small timestamp differences could therefore create an MS that the caller silently failed to return.

Validation

  • PYTHONPATH=/data/dsa110-continuum /opt/miniforge/envs/casa6/bin/python -m pytest tests/test_calibrator_ms_generator.py -q — 26 passed
  • /opt/miniforge/envs/casa6/bin/python -m ruff check dsa110_continuum/conversion/calibrator_ms_generator.py tests/test_calibrator_ms_generator.py — passed
  • python -m py_compile dsa110_continuum/conversion/calibrator_ms_generator.py — passed
  • make test-cloud import-migration check passed, but the local run was interrupted after check_contimg_mentions.py remained blocked in an H17 filesystem request during repository-wide traversal; GitHub CI should run the same gate on the clean checkout

Closes #112.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

convert_groups: per-group failure isolation and rep_time/MS-name coupling (follow-ups from #106 review)

1 participant