[ApGa]

deploy/abci/toolathlon.env now pins the ABCI SIF of ghcr.io/apga/toolathlon-gym:c66ff0d6a827c803e4a0db09c24367d50b9bd06b, in place of the cd86c47 SIF:

• the SIF is openreward-toolathlon-c66ff0d-pbs-7034e1a61a4a.sif, sha256 2fb0ee8ba5377a79206af36c4645c6e32242300aad5cf08677addc8e25ba468d;
• the image is ApGa/toolathlon-gym apga/session-db-shim after PR Allow positive rollout refills and fix multi-turn sample merging #4, amd64 sha256:b0e8ed72c025018e930b9dc0a9585f5df7cbfd502b95eb01a52c15554851ca30.

The image keeps the grader fixes of cd86c47 and adds:

• The snowflake tools. Its own tool catalog lists all 14, where c1e58af's had no snowflake entry.
• Fail-closed graders. 41 more graders fail closed on deliverables they can't read. In binary mode, random bytes no longer pass the two PDF tasks.

The SIF is the image plus the PBS portability patch. Its build recipe is in rasp's containers/, ported to c66ff0d in rasp's companion PR.

Changes

• deploy/abci/toolathlon.env takes the identity block the SIF's build wrote, byte for byte (toolathlon-c66ff0d.env). The labels now name this image's own digest and the SIF pulled from it (BASE_SIF_SHA256 7034db8a…). The old block still named 18e62c0d's. The comment says:
  • what the image fixes;
  • where the grader protection holds;
  • that the tamper hole below is open.
• tests/fast/fork/test_deploy_dry_run.py, deploy/abci/README.md and docs/fork/deploy-pbs-15node.md name the new SIF and its sha256.

Verification

Kit: /groups/gai51740/aci18912it/platoon-next/toolathlon-audit/ghcr-c66ff0d/ (PROGRESS.md).

Check	Job	Result
Registry and pull, anonymous	2672319	The index (fe1219c6…), amd64 manifest and config hash to their digests. Revision label c66ff0d; 26 layers. Pulled by tag, and the tag was unchanged afterwards
Files against git archive of c66ff0d	2672319	/app: 4,686 of 4,686 identical. /opt/local_servers differs only in build outputs. The catalog (f893c306) has 25 servers and 441 tools, 14 of them Snowflake's
run_check on the image, both modes	2672333	12 of 12 checks pass. Nothing is paid for a do-nothing episode, a planted result file or a rewritten grader copy (503 tasks each). Junk files pay nothing in binary mode. Correct submissions score 1.0
ABCI SIF build	2672392	singularity build --fakeroot. The structural verifier, labels and full verifier as the user pass: rootless Chromium, and all 13 Python MCPs. /app/tasks is unchanged
The SIF through deploy/abci/toolathlon_service.sh (the same file as here), as the user, both modes	2672482	5,054 sessions ran, and all 25 servers started with live name parity. 0 MCPBridge errors or warnings; 12 of 12 checks pass. Rewards are identical to the OCI run's on all 5,054, and to audit 2654028 (the cd86c47 SIF) on 3,018
rasp's builder rebuilt the SIF from its ported containers/	2673025, 2673126	Same patch identity. 309,444 entries each, and only 2 build-metadata files in /.singularity.d differ
Review, independent of the kit's scripts	2673234	The registry, fetched again, serves the same index, amd64 manifest and config. Each of the 26 cached layers matches its digest and its diff_id. Applied in order, with whiteouts, they give the pulled SIF's tree file for file (content and modes), apart from Singularity's own files and ownership, which the unprivileged pull sets to root. The ABCI SIF differs from the pulled SIF only in the patch's paths, plus 13 MCP venvs' empty .venv/.lock files, which the patcher's permission pass takes from mode 0666 to 0644 (as rasp's earlier patcher did). git archive of c66ff0d matches as above, and server.py and the 25 MCP configs equal the cd86c47 SIF's

Tests

Check	Job	Result
The env file sources cleanly, and its identity block equals the build's	2673155	ok
tests/fast/fork/test_deploy_dry_run.py	2673155	32 passed, including the Toolathlon dry run, which hashes the new SIF and checks it
deploy/abci/toolathlon_service.sh with DRY_RUN=1 and this env file, from a git archive of the branch	2673369	Exit 0: it hashes the pinned SIF (2fb0ee8b…), checks its base, OCI and portability labels and prints the container line. rasp's env file passes the same way
tests/fast/doc (miles' test-venv)	2673161	25 passed, here and on apga/dev f57034b2
tests/fast/fork (miles' test-venv)	2673161	151 passed, 1 skipped, 1 failed, the same here and on apga/dev f57034b2. The failure is test_smoke_q36_step9.py::test_platoon_env_is_the_c10_c11_client_contract

Rollout

• Nothing running changes. toolathlon_service.sh reads this file and stages the SIF when a service starts. A production job picks up the new image only when its operator restarts it.

• Re-baseline first. Against the cd86c47 SIF:

  • 41 graders and 3 tool descriptions changed;
  • the runtime is a fresh build (45 version changes in /opt/venv, among them openreward 0.1.154 → 0.1.167, and Snowflake's MCP 1.26 → 1.30).

  The no-op, planted-file and tamper audits give identical rewards on both SIFs.

• Location. The SIF sits in the audit area, …/toolathlon-audit/ghcr-c66ff0d/abci-sif/. Copying it to the canonical images directory changes only the path; the sha256 stays the same.

Still open

Tamper hole, in every SIF so far (job 2672675).

• toolathlon_service.sh binds the entrypoint to /app/openreward-toolathlon-resilient-entrypoint.sh, a path no SIF has.
• Singularity creates that mount point in the user's overlay, so /app becomes owned by the user the agent's tools run as.
• That user can then delete and replace /app/server.py and /app/task_process.py. /app/tasks and /app/runtime_patches stay protected.

It isn't fixed here. Two fixes would work: bind the entrypoint outside /app, or bake a placeholder at that path into the image.

Grading holes, unchanged:

• In binary mode, two tasks (162, 205) pass with empty but valid files.
• In partial mode, junk files earn partial credit on 170 of 503 tasks.

rasp's companion PR pins the same SIF and ports the SIF build: https://github.com/ApGa/rasp/compare/main...apga/toolathlon-image-c66ff0d?expand=1