Repository navigation
Preserve and recover archive frame IDs for calibrations - #490
Conversation
| # If the frame id was not included originally but we were able to pull it from the archive, | ||
| # we store it for future use | ||
| if 'frameid' not in master_calibration_file_info and master_calibration_image.frame_id is not None: | ||
| if frame_id_was_missing and master_calibration_image.frame_id is not None: |
There was a problem hiding this comment.
Can we add if frame_id_was_missing case that then populates master_calibration_image.frame_id in the database?
There was a problem hiding this comment.
The db update is just below in dbs.update_calibration_frameid(...). The old condition prevented it from running because it just looked for whether the key existed and not whether it was populated vs None, which is what this change fixes.
There was a problem hiding this comment.
Copilot review overview
🔵 Needs a closer look
Add coverage for re-importing an existing filename and updating its frame ID instead of NULL.
Review effort: Lite
Findings: None
What changed in this PR
This PR preserves archive frame IDs during calibration registration and recovery.
Changes:
- Passes archive and ingester IDs into calibration records.
- Persists IDs recovered through archive lookup.
- Adds database tests for registration and recovery.
| File | Description |
|---|---|
banzai/tests/test_dbs.py |
Tests registration and recovery behavior. |
banzai/main.py |
Preserves frame IDs during calibration registration. |
banzai/calibrations.py |
Saves IDs recovered through archive lookup. |
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
cmccully
left a comment
There was a problem hiding this comment.
I think this looks fine. I do want to make sure that we store the newly found frame id in the database and I'm not finding the code to do that. We should probably also add a change log note and bump the version (though we can wait to tag the final version until we've merged all the prs).
Problem
Archived calibrations can have
calimages.frameid = NULLbecause two registration commands omit the archive ID when constructing theirDataProduct:banzai_populate_bpms --db-address "$DB_ADDRESS".banzai_add_super_calibration /path/to/readnoise.fits --db-address "$DB_ADDRESS" --upload-to-archive.A reduction can find these calibrations through the Archive basename lookup, but the existing save-back condition never persists the recovered ID: database records already include a
frameidkey, and opening the file updates that dictionary in place.Changes
idintoDataProductinadd_bpms_from_archive.frameidintoDataProductinadd_super_calibration, removing the ineffectivecal_image.frameidassignment.Recovery writes to
cal_db_address, the same database used to select the calibration. This defaults todb_addressin the main AWS configuration; an explicit calibration database remains the target at sites.Registration without upload still stores NULL, including when replacing an existing filename. Existing NULL rows can be repaired through corrected re-imports or basename fallback during calibration use.