Skip to content

Investigate unexplained multi-minute release command latency #90

Description

@deverman

Problem

Release commands sometimes appeared to take several minutes even when the work
inside the command was fast or eventually returned normal output. This made the
v0.10.0-beta release feel stalled and made it difficult to distinguish real
build time from network, approval, package-manager, or command-orchestration
latency.

This is a development/release-tooling investigation, not evidence of slow
FocusRelay queries or an oversized Swift test suite.

Evidence from the v0.10.0-beta release

Fast project work:

  • Incremental swift test: 133 tests completed in about 0.04 seconds after a
    0.36-second build.
  • Local release builds: approximately 21–24 seconds.
  • GitHub CI: 2 minutes 5 seconds.
  • GitHub tag release workflow: 4 minutes 14 seconds.

Unexpected command-layer waits observed by the agent environment:

  • gh pr ready 84: approximately 329 seconds before succeeding.
  • brew update plus tap style/audit: approximately 391 seconds.
  • brew reinstall: approximately 1,000 seconds before returning output that
    said the formula itself built in 2 seconds; the only reported problem was an
    expected link conflict with the temporary UAT symlink.
  • Final read-only release audit batch: approximately 499 seconds despite all
    checks returning healthy results.
  • Some gh calls failed immediately with Could not resolve host: github.com
    or error connecting to api.github.com, then succeeded on later attempts.
  • A few local operations, including patch application and process inspection,
    also showed unusually long wrapper wall times, so network latency alone is
    not yet a sufficient explanation.

Some waits may include time awaiting an approval handoff or a yielded command
session. Current logs do not separate that time from actual subprocess runtime.

Questions to answer

  1. Is the delay inside the subprocess, DNS/network access, Homebrew auto-update,
    an approval handoff, or the agent command/session wrapper?
  2. Do the same commands have normal timings when run directly in Terminal?
  3. Does an explicit brew update followed by HOMEBREW_NO_AUTO_UPDATE=1 for the
    remaining release commands remove redundant waits without weakening release
    verification?
  4. Are GitHub DNS failures correlated with the multi-minute waits?
  5. Are commands completing promptly but their output/result delivery being
    delayed?

Investigation plan

  • Capture separate timestamps for:
    • agent/tool invocation;
    • approval requested and granted;
    • subprocess start and exit;
    • first output received;
    • final result delivered.
  • Compare representative commands in both environments:
    • direct Terminal execution with /usr/bin/time -p;
    • agent command execution with the same working directory and environment.
  • Test lightweight read-only commands first: gh api rate_limit, DNS lookup,
    brew config, brew list --versions focusrelay, and git status.
  • For a future release rehearsal, enable narrow diagnostics such as GH_DEBUG
    or Git trace timing for one command at a time; do not leave verbose logging on
    by default.
  • Record whether Homebrew is updating taps, downloading bottles/source,
    cleaning up, checking PATH shadowing, or waiting before any output appears.
  • Confirm whether PTY allocation, command yielding, or concurrent tool calls
    changes the observed latency.

Acceptance criteria

  • Identify the responsible layer or layers with timestamped evidence.
  • Produce a short release runbook that separates expected build/CI time from
    command-orchestration waits.
  • Remove redundant Homebrew updates or retries only where correctness remains
    unchanged.
  • Add bounded timeouts/retry guidance for GitHub DNS/API failures.
  • A rehearsal of the publication-critical sequence completes without an
    unexplained multi-minute silent wait.

Guardrails

  • Do not reduce Swift, MCP, Homebrew checksum, plugin-version, or bridge-health
    validation merely to make the release appear faster.
  • Do not treat this as a FocusRelay runtime performance regression unless
    product-path measurements demonstrate one independently.

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or requestperformanceLatency, throughput, memory, or reliability performance work

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions