HPC Site Preflight

HPC Site Preflight constructs an evidence-backed profile of an HPC system and uses that profile to determine whether a portable workflow package can be deployed safely.

The research paper framing is Evidence-Backed HPC Site Policies with Agentic Discovery.

login-measurements.json is the single site input. It contains measured or simulated identity, storage, platform, and scheduler facts. A site profile is the larger evidence-backed, actionable output constructed from measurements, documentation, and eventually pilot jobs.

The working machine-readable site-profile contract, field semantics, evidence boundary, and a full illustrative example are described in docs/SITE_PROFILE.md. Common scheduler-independent login-node observations and their safety boundary are defined in docs/MEASUREMENT_FIELDS.md. The versioned measurement result and its profile mappings are described in docs/LOGIN_MEASUREMENTS.md.

The project combines four evidence sources:

1. existing site profiles;
2. login-node measurements;
3. official documentation analyzed with bounded AI assistance;
4. predefined pilot-job results.

One bounded discovery agent uses reviewed web-search and page-download tools, then AI selects among the fetched sources and proposes structured facts. Measurements, pilot scripts, evidence validation, reconciliation, and the final preflight decision remain deterministic.

Main workflows

Build a site profile

A backpack is not required.

login measurements
→ profile lookup
→ documentation evidence
→ approved pilot results
→ reconciliation
→ site profile

Preflight a portable workflow

backpack + site profile
→ compatibility checks
→ execution plan or early failure

Evaluate only the AI/documentation subsystem

login measurements
→ documentation discovery
→ corpus
→ context selection
→ extraction
→ evidence validation

The implemented context modes are full-corpus, bm25, and llm-expanded-bm25. See docs/DOCUMENTATION_WORKFLOW.md for the code and artifact flow. For a repository-wide reading order, see docs/CODE_GUIDE.md. The evaluated Anvil live-AI example is traced in docs/RUN_RESULT.md.

Site, model, and web modes

The three concerns are independent:

• --site-mode simulate is the default, requires --measurements, and never queries local hardware. live reuses --measurements when supplied or captures them from the login node.
• --model-mode live is the default and calls the model's inferred provider. simulate replays model responses.
• --web-mode live is the default and searches and fetches allowed official domains. simulate replays search results and pages.

The normal laptop workflow therefore simulates only the HPC site while using live web discovery and live model calls. Offline tests explicitly simulate all three external inputs. Evidence retains whether it was simulated or measured.

Documentation discovery also accepts optional user guidance. --site-name supplies a search name and is required only for live collection when measurements are absent. --discovery-note adds free-text context for the discovery model, and each repeatable --discovery-keyword adds a bounded search query. There is no exclusion-keyword option: source scope and allowed domains remain deterministic controls.

--model accepts a provider-neutral model identifier. The current registry maps gpt- and OpenAI o-series names to OpenAI, claude- names to Anthropic, and gemini- names to Gemini. Only the OpenAI adapter is implemented today; recognized future providers fail explicitly.

Installation

conda env create -f environment.yml
conda activate hpc-site-preflight

Or with an existing Python environment:

python -m pip install -e ".[dev]"

CLI

hpc-site-preflight --help
hpc-site-preflight profile build --help
hpc-site-preflight profile validate --help
hpc-site-preflight profile show --help
hpc-site-preflight evidence capture-login --help
hpc-site-preflight evidence run-pilots --help
hpc-site-preflight evaluate documentation --help
hpc-site-preflight preflight --help

evidence capture-login writes the structured site input without running documentation or profile construction. profile build can combine measurement, documentation, and explicitly approved pilot evidence. evaluate documentation runs the documentation subsystem alone.

Generated profile JSON follows the SiteProfile schema order instead of alphabetical key order. The evidence-report reference and field-evidence links are the final top-level fields.

For a normal laptop run, set OPENAI_API_KEY, then run:

hpc-site-preflight profile build \
  --measurements examples/simulate/anvil/login-measurements.json \
  --model gpt-5-mini \
  --output-dir artifacts/anvil-live

Progress is written to standard error before slow searches, downloads, and model calls. When using conda run, add --no-capture-output so Conda streams that output instead of holding it until the process exits:

conda run --no-capture-output -n hpc-site-preflight hpc-site-preflight profile build --help

Add --model-mode simulate --web-mode simulate for a fully offline replay.

On a real login node, capture only the site input:

hpc-site-preflight evidence capture-login --short-site-name Anvil

Or let profile construction capture and retain it under the output directory:

hpc-site-preflight profile build \
  --site-mode live \
  --short-site-name Anvil \
  --model gpt-5-mini \
  --run-pilots \
  --output-dir artifacts/anvil-live

For a simulated site, --run-pilots loads a flat partial result rather than submitting:

hpc-site-preflight profile build \
  --measurements examples/simulate/anvil/login-measurements.json \
  --model-mode simulate \
  --web-mode simulate \
  --run-pilots \
  --pilot-results pilot-result.json

To compare retrieval modes against one real, frozen corpus, supply a directory containing manifest.json, documents.jsonl, and chunks.jsonl. This skips discovery and web access:

hpc-site-preflight profile build \
  --measurements examples/simulate/anvil/login-measurements.json \
  --model gpt-5-mini \
  --corpus-input examples/simulate/anvil/corpus \
  --context-mode bm25

The separate pilot command requires the site inputs normally derived from login measurements:

hpc-site-preflight evidence run-pilots \
  --scheduler slurm \
  --site-id anvil \
  --login-host login01.anvil.rcac.purdue.edu \
  --storage home=/home/user \
  --output pilot-result.json \
  --approve

Preflight a Floability backpack

Preflight reads compute.yml, parses but never executes the supplied Floability command, and compares the normalized worker request with a validated site profile:

hpc-site-preflight preflight \
  --backpack examples/backpacks/preflight-case-study \
  --site-profile artifacts/anvil/site-profile.json \
  --floability-command \
    "floability execute --backpack examples/backpacks/preflight-case-study" \
  --scheduler-value account=my-allocation \
  --output artifacts/preflight-result.json

The structured result is ready, blocked, or unknown. A ready result contains a selected resource and modified Floability command. --explain-with-model --model <model> adds prose but cannot change the deterministic decision. This first implementation checks scheduler type, per-worker CPU, memory and GPU shape, required submission options, and TaskVine network settings. A requested local-disk amount remains unknown until the site profile contains disk evidence.

Performance reporting

Every command creates one tracker and passes it through the pipeline. The tracker aggregates:

• total runtime;
• runtime per pipeline step;
• model requests;
• input and output tokens;
• retries;
• tool calls;
• produced artifacts;
• failed-step information.

Reports are written to:

runs/<run-id>/performance.json
runs/<run-id>/trace.jsonl

Usage is reported per pipeline step and for the whole run. Missing provider token data is recorded as unavailable rather than estimated.

Repository relationship

The existing hpc-site-policy-agent repository is the reference implementation for documentation discovery and extraction. Its useful components are adapted into the bounded documentation pipeline; its top-level CLI and control loop are not copied into this project.

Trust boundary

The project must not allow an LLM to:

• execute arbitrary shell commands;
• generate unrestricted network tests;
• submit unapproved pilot jobs;
• directly certify a site policy;
• approve or launch a real workflow;
• silently resolve conflicting evidence.