Skip to content

fix(bootstrap): unify executable and zipapp startup - #4183

Open
jrandolf wants to merge 1 commit into
bazel-contrib:mainfrom
jrandolf:codex/bootstrap-runtime-cleanup
Open

jrandolf wants to merge 1 commit into
bazel-contrib:mainfrom
jrandolf:codex/bootstrap-runtime-cleanup

Conversation

@jrandolf

@jrandolf jrandolf commented Sep 24, 2026 •

Copy link
Copy Markdown

Temporary virtual environments and executable ZIPs currently leave a launcher
process in front of Python so it can remove files afterward. A termination signal
sent directly to that PID can leave the application running and skip its cleanup.
The different executable and ZIP startup paths also disagree about interpreter
arguments, cache identity, and ownership during failed or concurrent preparation.

Describe the application once, then launch the same prepared image through an
ordinary executable, a legacy executable ZIP, or either py_zipapp_binary entry
point. On POSIX the application replaces the launcher in its original PID, keeping
native signal delivery, process groups, terminal input, and exit status. A detached
watcher removes only temporary files after that PID exits. Windows starts and
reaps the application child before removal, allowing the application's own
Ctrl-C cleanup to finish without forwarding another interrupt.

The implementation separates the private application description, environment
preparation, storage ownership, process launch, and thin entry adapters:

  • A private provider and versioned application specification carry the runtime,
    virtual environment, entry point, and explicit links. Both ZIP rule families
    use the same packager. Ordinary builds keep their runfiles without copying the
    dependency tree.
  • Preparation owns one temporary workspace and registers resources before
    honoring cancellation. The bundled ZIP shell reaps its preparation children
    and validates a fixed six-field result before native execution.
  • Persistent images publish only after preparation succeeds, using an exclusive
    relative directory symlink to an owned backing directory. Publication cannot
    replace an existing entry, including one created concurrently. Content, modes,
    layout, bootstrap code, and resolved runtime facts contribute to identity.
    Completed bundled ZIP cache hits can execute directly from the shell.
  • Public providers and flags remain unchanged. Stage two retains import, main,
    precompilation, and coverage behavior. Custom templates preserve their bytes
    and historical substitutions; the original raw templates remain usable by
    older expanders. Public-only executable providers retain a compatibility path.

Validation uses Bazel 9.1.1 and --config=fast-tests for test targets:

  • Root suite: 727 test targets passed, seven skipped. The generated launcher
    matrix has 315 cases (313 passed, two skipped), passing under macOS Bash 3.2
    and Bash 5.3.20. The final 315-case matrix also passes on Bazel 7.7.1 in
    WORKSPACE mode and Bazel 9.1.1, each with the same two expected skips.
  • Native Buildbarn execution with local fallback disabled: macOS arm64 13/13
    targets, Linux arm64 13/13, Linux x86_64 13/13, and Windows x86_64 7/7
    passed. Final affected custom/public-provider checks also pass on all four
    platform pools.
    Execution logs distinguish actual remote test execution from cache hits.
  • Generated ZIPs also pass with a Python 3.9 host interpreter. A loader-protocol
    regression covers ZIP loaders that predate exec_module; all ten affected
    local ZIP/template targets pass. The final loader and packaged ZIP follow-up
    also passes three targets on each of the four native Buildbarn pools.
  • Regression coverage includes direct/group signals and interrupted finally,
    inherited ignored signals and masks, terminal job control, nesting, arguments,
    wrapper runtimes, setup/exec failures, cache publication races, Windows console
    handling and exit status, custom templates, and watcher readiness/descriptor
    lifetime. Native macOS kqueue and Linux subreaper checks pass. Linux pidfd
    and procfs checks pass in a separate Debian guest where those APIs are available;
    these backend proofs reject fallback and verify owned-child/descriptor lifetime.
  • Gazelle: 85/85; examples with real coverage: 32/32; Sphinx private verification
    module: 11/11. The Sphinx check privately registers its required Python 3.11;
    no dependency configuration changes are included.
  • Changed-file pre-commit hooks, generated-file checks, Bash syntax, diff checks,
    documentation and distribution builds pass. Original ordinary and ZIP launchers
    reproduce the direct-PID termination failure; the new launchers preserve the
    PID, execute application cleanup, and remove their temporary files.

Temporary execution costs one resident watcher, measured at roughly 4.7–6.8 MiB
RSS locally. Runtime-created virtual environments also enter Python for
preparation. In the latest six-sample, 500-file comparison on a loaded shared
host, baseline/current startup medians were 75/73 ms for prepared executables,
144/289 ms for runtime-created venvs, 1724/1559 ms for cold ZIPs, and 77/74 ms for
warm ZIPs. Earlier quieter runs varied substantially; these are measurements,
not performance guarantees. Both builds spawn three build actions; the new
description adds three internal metadata writes.

Removal is asynchronous and follows the original interpreter, including exec;
it does not extend to descendants that outlive it. Namespace PID 1 and subreapers
can adopt the watcher, and namespace/cgroup shutdown or SIGKILL before setup
registration can prevent cleanup. A persistent extract root avoids the temporary
watcher where supported. Its published backing directories belong to the caller;
removing only a cache symlink does not reclaim the backing image. These lifetime
limits and cache cleanup semantics are documented.

Older custom rules exposing only PyExecutableInfo use the unchanged raw-template
adapter. Its Windows persistent extraction can fail when a repeated launch meets
an existing directory symlink. This also reproduces with the original ZIP rule,
templates, and packaging tools; normal native launch and archive structure pass
in both versions. Temporary extraction is the documented workaround for that
compatibility path. The new default image path passes repeated persistent-launch
checks on Windows.

@jrandolf
jrandolf marked this pull request as ready for review September 24, 2026 06:38
@jrandolf jrandolf changed the title fix(bootstrap): preserve signals with temporary runtimes fix(bootstrap): unify executable and zipapp startup Sep 24, 2026
Describe executables and ZIPs with one private application specification,
and share environment preparation, storage ownership and packaging across
ordinary binaries, legacy executable ZIPs and py_zipapp entry points.

Use native POSIX exec to preserve the application PID, signals and terminal
behavior, with a detached watcher for temporary-file cleanup. On Windows,
wait for the application child and its console cleanup before removal.
Publish complete persistent images without replacing concurrent entries.

Preserve public providers, custom templates and stage-two behavior. Keep
historical raw-template adapters available and support Python 3.9 ZIP
loaders. Document cleanup lifetime and legacy compatibility limitations.

Validation: 727 root targets pass; the launcher matrix passes 313 cases
with two skips. Native checks pass on macOS, Windows and Linux arm64/x64,
including final loader/ZIP checks on all four pools. Changed-file hooks,
documentation, distribution and independent implementation audit pass.
@jrandolf
jrandolf force-pushed the codex/bootstrap-runtime-cleanup branch from 70c2c88 to 7fac02f Compare September 24, 2026 14:54

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant