Repository navigation
Conversation
…r results
Changing the plot at the end of a workflow cost a full re-run: Studio never
told Pegasus where earlier results were, and generated workflows commonly set
register_replica=False, so even `pegasus-plan --reuse` finds nothing.
services/rerun.py writes the catalog itself, from files that exist: a replica
file listing earlier outputs of every step not being re-run, passed to the
planner with --inherited-rc-files. Which files are current comes from the run
directories (the abstract workflow copy and the stampede DB): the newest run
in which a file's producer succeeded owns it, and the file is reused only if
written after that job started. The preview mirrors Pegasus's pruning as
measured on 6.0.0.dev1; the Run flow reports what Pegasus actually planned.
- Run flow: rerun_from on the request; an unknown step fails the flow
rather than quietly running everything.
- API: GET /projects/{id}/steps and /projects/{id}/rerun-preview.
- Run panel: a "Start from" choice with the preview (what runs, what is
reused and from which run, which intermediates were never kept).
- Assistant: a prepare_rerun MCP tool that writes the replica file and
returns the pegasus-plan command, which runs through the gated shell; the
Studio environment guide tells the assistant to use it.
Refs #680
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #680
Changing only the plot at the end of a workflow used to re-run everything. With this change, a run, from the Run panel or from the assistant, can start from a chosen step and reuse what earlier runs left in the output folder for the steps before it. Keeping intermediates, so a re-run can start mid-workflow, is the follow-up #683.
How
Pegasus already skips a job whose outputs are all in the replica catalog. It had nothing to find: each run's catalog sits in its own submit directory, and generated workflows commonly set
register_replica=False, so even--reusefinds nothing.services/rerun.pywrites the catalog itself, from files that exist on disk:--inherited-rc-files, which has lower priority than the workflow's own catalog, so it can only add locations, never override one. Nothing in the user's workflow changes.Where it shows up:
rerun_fromon the request. An unknown step fails the flow instead of quietly running everything, since running everything is what Re-run a workflow from a chosen step, reusing earlier results #680 was about.GET /projects/{id}/stepsandGET /projects/{id}/rerun-preview?from_step=(writes nothing).prepare_rerunMCP tool. It writes the replica file and returns the preview plus thepegasus-plan … --inherited-rc-files …command, which the assistant runs through the gated shell as it does any plan. The tool itself starts nothing.STUDIO_ENV.mdtells the assistant to use it when only a step changed.Verified
climate-data-analysis-workflow(prep_data → analyze_station ×3 → aggregate), without submitting:aggregateis planned, with--cacheand--inherited-rc-filesalike.aggregate, that station'sanalyze, andprep_dataare planned; the other twoanalyzejobs are reused.aggregate's outputs listed: theanalyzejobs are not dropped, because their staged outputs are missing.--reusewould reuse nothing, because the workflow registers no replicas.rerun.pyagainst that real run (preview only): fromaggregate, 1 step runs and 4 are reused (6 files from run0001). Fromanalyze_station, everything runs, and the preview names the threestation_*.csvintermediates as the reason. Both match Pegasus above.test_rerun.py, 2 intest_run_flow.py),tsc --noEmitclean, vitest passes.Not verified
output/andaggregaterunning on the pool. Planning shows the stage-in jobs, but none of these runs was submitted.:develop.Limits