Repository navigation
Conversation
Analysis runs off the control PC and needs three things AlabOS does not expose: a write of metadata.analysis.<kind> onto an existing sample, a query for which samples still need a given analysis, and an atomic claim so two workers never refine the same sample twice. Implements them as a view next to SampleView with the HTTP routes as a thin adapter, so there is one write implementation behind both, per the data-architecture doc's D7 and D9. Where a lab stores its measurements is read from the config file rather than hardcoded here.
Fold-in note (unified
|
Closing without merge — landed via #107Thank you — this PR’s intent (analysis pending / claim / release / write-back without opening Mongo as a peer) is on What changed vs this branch
Please point desta / analysis clients at the contract in alab_one — Emma (via Cursor on Sauron) |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Based on
origin/main6aed0ef. Opened at your request. Prompted by D7 and D9 inalab_one'sdocs/data-architecture.mdand the companiondocs/alabos-mongo-write-paths.md.What it adds
An analysis service runs off the control PC. To work without opening Mongo as a peer (P1) it needs three things
AlabOS does not expose, plus one read-only helper:
POST /api/analysis/results/<sample_id>metadata.analysis.<kind>, and mirror it toAlab(completed)if the sample is archived therePOST /api/analysis/claim/<sample_id>POST /api/analysis/claim/<sample_id>/releaseGET /api/analysis/pendingparams_hashGET /api/analysis/data-sourcesOne
AnalysisViewnext toSampleView, with the blueprint as a thin adapter over it: one write implementationbehind both the HTTP and in-process paths, which is D9's shape. Additive, and tested.
Design
AlabOS never looks inside a result.
write_resultstores an opaque object and reads one field,params_hash, which it only compares for equality. It is a cache validator, like an ETag, so a change to ourrefinement output needs no change here.
Where a lab stores its measurements is config, not code.
[data_api.input_sources.<name>]declares a sourceas a field path plus an optional BSON type, and AlabOS builds the query; the caller sends a name, never a
filter, because accepting an arbitrary Mongo filter over HTTP is the same surface as handing out the connection
string. In config because the field is a fact about a deployment:
alab_oneandalab-gpssdiffer today, and analab_onechange in flight will move its pattern again. Each of those becomes a config edit instead of a PR.The claim is a lease, not a lock. One conditional
update_one, so of two racers exactly one matches. Theexpiry is stored on the document, so a crashed worker's claim frees itself and nothing has to notice the crash.
It is a fixed-width ISO-8601 UTC string rather than a BSON
Datebecause$ltethen orders correctly by stringcomparison; a
Datewould never compare against the strings the existing off-box worker already writes.An optional shared token guards the routes.
[data_api] token, orALABOS_DATA_API_TOKEN, compared inconstant time and never echoed back. With nothing set the check does not exist, so merging this changes no
deployment unless you turn it on. We added it because these are write routes and our analysis host is a shared
machine, so a forwarded port is not ours alone. If you would rather use mTLS, a proxy, per-client tokens, or
treat the lab network as the boundary, say so and we will follow it on our side.
The result write goes through
SampleView.update_sample_metadatawith a dotted key, so{"analysis.xrd": ...}becomes a
$setofmetadata.analysis.xrd. Worth flagging because we checked: every existing caller passes aflat key and no test covers the dotted form, so the nesting is a consequence of how that method builds its update
rather than a stated contract. We rely on it because the alternative is writing the
$setourselves andbypassing the one method everything else goes through.
The mirror is asymmetric, deliberately. This API copies a result into
Alab(completed)when the sample isalready archived there, because a result written after the experiment finished would otherwise be invisible to
the Data page and lost when the live document is pruned.
lab_view.update_sample_metadataduring a task does notmirror, and we did not change it: that is a core method on the lab's hot path, and unifying the two belongs in
its own change rather than in an analysis PR.
One operational note:
GET /pendingis an unindexed scan ofsamples, fine at current scale but it wants anindex on
metadata.analysis.<kind>.params_hashif the collection grows.Not in this PR
The SEM client's three paths from section 5 of the architecture doc (link to an existing sample, create a
manual sample, measure without persisting). This is the analysis side only: no sample search, no manual-sample
creation, no do-not-persist gate. If the two are meant to share one contract, the natural shape is for "register
a measurement" and "write an analysis result" to be two endpoint groups under one
/api/data. We have not builtthat and are not the right people to design the instrument-UI half.
Discovery for anything but XRD.
pendingneeds one configured source per technique, and for SEM the spectraare files in a shared folder, so that query depends on how the transfer records them. The write path is already
technique-agnostic, so a
sem_edsoreisresult can be written the moment there is one.