Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -974,6 +974,9 @@ the terminal, with both a full-screen TUI and scriptable CLI commands.
unchanged saved object back to SetAccessControl can write nothing. Restore
parent then payload then file after inheritance propagation, and avoid changing
the source owner when testing read-only backup permissions.
Fixture owner normalization must inspect the no-follow object and skip an
already-current-user owner. Redundant owner-only `SetSecurityInfo` can strip
inherited OWNER RIGHTS access; retain file/directory/link regression coverage.

- Every Unix descriptor duplication must atomically retain close-on-exec through
`F_DUPFD_CLOEXEC`; never use plain `dup` followed by `F_SETFD`. Darwin ARM64
Expand Down
49 changes: 49 additions & 0 deletions agent_docs/one-off-acceptance-summary.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,49 @@
# One-off acceptance summary

PR #86 adds manually invoked testing tools, fixture corrections and startup
documentation. It changes no product source or dependency locks. Personal
reports, machine dumps, command logs and terminal captures are excluded from
the PR and its commit history. This record contains aggregate results only.

## Fixes retained

- The runner uses CI-compatible sequential locked AOT publishes, rejects result
directories beneath any ancestor `.git` marker, and reports partial or
interrupted coverage explicitly.
- The collision-guard fixture isolates Git context so a short output budget
exercises the intended listing rather than an ambient checkout path.
- Windows fixture owner normalization avoids redundant owner writes that can
remove inherited OWNER RIGHTS access. File, directory and link regressions
failed with the original helper and passed with the correction.
- Startup documentation explains the inventory-driven transition to Installed.
Integration tests exercise initial Discover query input at two terminal sizes.
No product focus change was needed.

## Verification

| Coverage | Result | Limit |
| --- | --- | --- |
| Combined WSL unit suite | 2,698 passed, 30 explicitly skipped, 0 failed | Windows-only test bodies that return early do not establish Windows coverage. |
| Combined WSL integration suite | 11 passed | ANSI-driver tests do not establish native Windows terminal behavior. |
| Python runner smoke tests | 10 passed | Runner contracts, not product acceptance. |
| Combined Release builds and scoped formatting | Passed | No dependency lock changes. |
| Focused macOS collision/startup tests | 6 passed | Focused confirmation rather than a repeated full suite. |
| Settled-startup macOS PTY checks | 2 passed; complete queries and successful Ctrl+Q exits | Early input during delayed inventory was not tested. |
| Windows fixture regressions | 3 passed after failing with the original helper | Focused native verification. |
| Windows metadata test class | 21 passed | Includes refusal paths; does not prove complete SACL publication. |
| Supplemental native AOT CLI proofs | 16 passed on each of Windows and macOS | Original proof matrices were not repeated after test-only corrections. |

## Remaining release acceptance

- Verify positive Windows security-metadata-dependent retention and unpin
publication in a context that already has `SeSecurityPrivilege`. Restricted
runs failed those positive assertions; passing refusal paths are not a waiver.
- Complete a native Windows terminal walkthrough for search, install, reviewed
update, Undo, cleanup and keyboard quit. Process startup alone is insufficient.
- A full Windows suite pass after the fixture correction remains unverified.
An earlier suite timed out; a later diagnostic run completed with failures,
and the focused follow-up did not repeat the full suite.

These gaps remain release follow-ups, separate from merging the testing kit.
Required PR checks must pass before merge. Future device results stay outside
Git; retain only sanitized aggregate conclusions and durable regression tests.
26 changes: 19 additions & 7 deletions agent_docs/tui-pty-testing.md
Original file line number Diff line number Diff line change
Expand Up @@ -109,12 +109,24 @@ immediately.

## Key PTY lessons from this repo

### 1. Startup focus is in the query field
### 1. Wait for startup inventory before judging focus

At startup, plain letters go into the query field. If you want single-letter
global shortcuts like `d`, `u`, `c`, or `I`, first send `Esc`. `Ctrl+Q` is the
unconditional quit path and works without leaving the field; plain `q` quits
only when a top-level read-only view has focus.
Startup initially paints Discover. When the first inventory contains skills,
SkillView can automatically open Installed, unless the user has already
interacted. The first Discover frame and its initial `ready` status do not prove
that startup inventory has finished. Wait for the environment/inventory result
and inspect the active view before sending keys.

With an empty inventory, Discover starts with query focus. From the Installed list,
press `/` to open Discover and focus its query. Typing `f` while the Installed
list has focus opens its filter; that shortcut is consumed rather than inserted.
Do not interpret the remaining filter text as a lost Discover query without
checking which view had focus when the key arrived.

In the Discover query, printable keys enter text. To use single-letter global
shortcuts like `d`, `u`, `c`, or `I`, first leave the field. `Ctrl+Q` is the
unconditional quit path and works without leaving it; plain `q` quits only when
a top-level read-only view has focus.

In a baseline headless PTY, Terminal.Gui can hold a bare Esc byte and combine
it with the next key as Alt+key, even after a delay. Use Tab to leave the query
Expand All @@ -136,8 +148,8 @@ Screen text can mislead because:
- result tables truncate repo names
- old frames remain in the PTY buffer

For startup smoke detection, wait for the `Search:` field rather than relying
on a transient status message.
For Discover readiness, wait for the `Search:` field after explicitly entering
Discover. Startup may legitimately settle on Installed instead.

### 3. Modal screens need unique wait conditions

Expand Down
135 changes: 135 additions & 0 deletions docs/one-off-testing-agent.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,135 @@
# Agent task: one-off release-candidate acceptance

This is a bounded testing task for Codex or Claude Code on one machine. Follow
the platform prompt in [one-off-testing.md](one-off-testing.md). Test the supplied
full commit SHA. Do not pull more features from PR #73, change product code,
weaken tests, change lock files, merge PRs, or establish recurring automation.

## Prepare and run

1. Read `AGENTS.md`. Create a fresh dedicated clone/worktree at the requested
commit, outside any dirty development checkout. Fetch the `v1.0.1` tag if
missing. Record the SHA and `git status --porcelain`; stop on a mismatch or
dirty source. Do not stash, reset or clean someone else's work.
2. Check Git, Python 3.9+, gh >=2.97.0 and the exact SDK in `global.json`.
Record OS, filesystem, terminal and architecture. If a prerequisite is
missing, install only those prerequisites from official sources into user-owned
locations when your session permits. Use a private SDK installation rather
than changing the machine's default SDK. If setup requires administrator access
or a lengthy native-toolchain installation, report the exact commands needed
and continue independent checks. Keep project dependencies unchanged.
3. Run the Python smoke tests, then `tools/test_once.py --expect-commit SHA
--live` with the appropriate Python command. Add `--dotnet PATH` if needed.
Add `--aot` only if the native build toolchain is already available for a
committed release RID. Use existing gh auth; never print/copy tokens into
commands, logs, reports or committed files. No auth means remote checks are
blocked, while offline checks can still run.
4. Read every command result and test summary, including skipped tests. Keep
failures visible. A runner exit of 0 establishes command success, not that
all platform-specific tests or UX checks ran. Record which host binaries
were actually exercised and whether they were managed or AOT.

Let the automated checks finish with their existing timeouts. Then spend at
most 30 minutes on the terminal checks below and 15 minutes reproducing new
failures. Stop investigating a failure after one focused reproduction unless
the next check is necessary to distinguish a fixture error from a product bug.
Do not wait for CI or another reviewer to finish this local report.
Avoid broad code reviews or reading the whole repository. Read only the guides
and failure-specific code needed for this pass. Use an inexpensive model for
command execution and log collection if available, reserving stronger models
for diagnosis. Do not create a swarm of review agents or repeatedly poll CI.

## Drive the actual terminal

Read [the PTY guide](../agent_docs/tui-pty-testing.md) before launching the built
host reported in `report.json`. Use a real PTY and terminal emulator/probe
responses on WSL/macOS, or a native Windows terminal/ConPTY on Windows.
Redirected CLI output, direct key-handler calls and a WSL session do not prove
native Windows TUI behavior. If the available agent tools cannot drive the real
terminal, mark that coverage blocked and provide a short manual checklist.
Do not build a new terminal automation framework for this pass.

Use a separate private fixture workspace, HOME/USERPROFILE, config, state and
temp directories. Strip inherited `GIT_*` and agent overrides as described in
AGENTS.md; keep the parent process environment unchanged. Capture existing auth
privately before isolating HOME when a remote operation requires it. Never
apply, remove, unpin or undo against actual user installations. Do not launch
AI clients to populate effective-loading evidence.

Start at **80×24**, then repeat any clipped or awkward view at a larger size.
Use help/current documentation for keys rather than guessing mutation flags.
After each mutation, check payload hashes, link targets, owner/registry evidence
and unchanged independent copies from the shell. Capture meaningful screen
states and terminal identity. Distinguish fixture creation from actions actually
performed through the TUI.

Prioritize these user journeys, in this order:

| Journey | Evidence to collect |
| --- | --- |
| Start, browse, search, preview | Wait for startup inventory to finish. An empty fixture stays in Discover with query focus; an existing inventory can open Installed. From the Installed list, press `/` to open Discover and focus its query. Verify the whole typed query, Tab, shortcuts, help and quit at 80×24. Search failure has a clear explanation when offline. |
| Install and inspect copies | Install into the fixture using actual gh; inspect two independent copies with differing content and, where supported, a shared link. Explain which physical copy and consumers are selected. Refresh preserves correct scope/workspace context. |
| Review a version/update | Use Updates, review a specific existing tag, cancel first, then approve retention and apply one target. Check exact source/version and local-change messaging, queue/focus behavior, and that independent peers stay unchanged. A pinned fixture must remain protected until separately reviewed unpin. |
| Undo | Inspect the saved operation, cancel, then approve the exact reviewed Undo and verify the original. If time permits, edit after inspection and confirm stale approval refuses. Check refresh and retry guidance. |
| Cleanup | Show an empty leftover directory and a broken link beside a healthy skill. Review/cancel first, then apply the safe fixture cleanup. Verify healthy content and independent copies remain; deliberately review-only items should be explained rather than counted as attempted destructive failures. |

Use [usage](usage.md) and [saved updates](saved-updates.md) for current behavior.
Avoid a large matrix of every supported agent or every adversarial native case;
the automated suites/proofs cover those contracts. Inspect static package-check
and effective-loading output if convenient, but do not displace the journeys
above or install real AI clients just to test them.

On WSL/macOS, the existing
`agent_docs/evidence/prove-reviewed-update-tui.py` can automate a live reviewed
update journey; inspect its `--help` and retain its captures. It uses POSIX PTYs
and is not a Windows proof. Run it with a new output directory and the built
host; it creates private fixtures. Report the exact shortcut/disposition tested.
Do not rerun every variant automatically. On Windows, if symlink privileges or
security-metadata visibility are unavailable, record the refusal and missing
coverage; do not elevate or bypass guards to manufacture a pass.

## Return a useful report

Write `AGENT-REPORT.md` beside the runner's `REPORT.md`, outside the checkout.
Use this shape:

```markdown
# SkillView acceptance — WINDOWS / WSL / MACOS
Commit: FULL_SHA
Machine: OS, architecture (hardware/process/SDK), filesystem, terminal, SDK, gh
Builds: managed/AOT, standalone/extension paths
Overall: PASS / FAIL / PARTIAL / BLOCKED

## Automated checks
Commands and logs; actual test totals and skipped counts/reasons.
Offline and remote proofs: passed/failed/blocked, separately.
AOT and platform-specific/native coverage actually exercised or missing.

## Terminal journeys
Each journey: passed/failed/not-run/blocked; actual terminal size and input path.
Mutation evidence and unchanged-peer checks; relevant captures/transcripts.

## Findings, most important first
Product / environment / fixture / authentication or network.
Exact reproduction, expected/actual behavior, log/capture paths.
Any risk of wrong target, lost local edits, misleading approval or missing Undo.
Say 'pre-existing' only with baseline reproduction evidence.

## Remaining manual checks
Only the specific checks that could not be automated and why.
```

If there is a new failure, reproduce it once in a fresh fixture. If comparing to
the release is necessary, use a separate checkout and clearly label that result;
never change the requested candidate checkout mid-run. Preserve original failing
logs even if the retry passes. Do not hide skips, retry until green, waive unknown
native failures, or treat authentication/rate-limit failures as successful tests.

Keep reports, logs, machine dumps and captures outside Git; do not commit personal
or device evidence. Return a sanitized summary with aggregate results and missing
coverage, omitting usernames, hostnames, absolute local paths and device
identifiers. Keep investigation paths in the local report. Share raw evidence
separately only when explicitly requested, after checking it for personal data
and credentials; exclude tokens, private homes, state archives and package caches.
Leave fixtures available for investigation. Report any owned process that could
not be stopped.
Loading
Loading