Skip to content

Record plugin installs in the project lock file - #6314

Open
samuv wants to merge 4 commits into
mainfrom
plugins-lock/03-install-hooks
Open

Record plugin installs in the project lock file#6314
samuv wants to merge 4 commits into
mainfrom
plugins-lock/03-install-hooks

Conversation

@samuv

@samuv samuv commented Aug 13, 2026

Copy link
Copy Markdown
Contributor

Summary

  • Why: RFC THV-0080's plugins: key is only useful if project-scope installs actually pin into toolhive.lock.yaml, and a failed lock write must not leave an install silently unpinned.
  • What:
    • Gate the whole rollout behind TOOLHIVE_PLUGINS_LOCK_ENABLED so plugin entries stay out of the live skills trust document until verification is wired.
    • Project-scope Install upserts a plugins: entry (source, resolvedReference, digest, contentDigest) and marks the store record managed. Lock-write failure rolls back the DB record (restore pre-existing, otherwise delete).
    • Uninstall of a managed plugin removes the lock entry first; a lock-write failure aborts while the install is still intact.
    • contentDigest is the frozen skills dirhash over the canonical plugin tree (in-memory ExtractPlugin file set), not marketplace.json / settings.json.
    • Plugin requires is not materialized; requiredBy is unused in this v1.

Part of #6300. Stack 3/5 — schema → lock-service → install-hooks → sync → upgrade.

Type of change

  • New feature

Test plan

  • Unit tests (./pkg/plugins and ./pkg/plugins/pluginsvc with the Taskfile race/ldflags flags; lock tests cover record, gate-off, user-scope, skills-key isolation, lock-write rollback, pre-existing restore, uninstall, and uninstall abort-before-destroy)
  • Linting (task lint-fix)

Does this introduce a user-facing change?

No by default — the feature is inert unless TOOLHIVE_PLUGINS_LOCK_ENABLED=true. With the gate on, project-scope thv ai-plugin install / uninstall write and remove plugins: entries in toolhive.lock.yaml.

Implementation plan

Approved implementation plan (PR3 slice)

Mirror skills PR3 (#5894) at the installAndRegister choke point:

  • Capture LockSource as the caller's original name before internal resolution.
  • Compute contentDigest from the layer file set (same files ExtractPlugin writes) before recording.
  • Fail the entire install on lock-write error; rollback restores PreExisting or deletes the new record.
  • Uninstall removes the plugins: entry first (no requires cascade).
  • Do not add SyncRestore yet — that bypass of isExtractionNoOp lands with sync (PR4).

Special notes for reviewers

  • No requires materialization and no uninstall cascade — agreed v1: lock only explicit plugin installs.
  • isExtractionNoOp has a comment that PR4's SyncRestore will need to bypass it for on-disk drift repair at the same digest.
  • LockSource / LockResolvedReference are on InstallOptions now so sync/upgrade can preserve the original source without a follow-up options change.

@samuv
samuv requested a review from JAORMX as a code owner August 13, 2026 16:19
@github-actions github-actions Bot added the size/L Large PR: 600-999 lines changed label Aug 13, 2026
@codecov

codecov Bot commented Aug 13, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 70.52632% with 112 lines in your changes missing coverage. Please review.
✅ Project coverage is 72.99%. Comparing base (cfba580) to head (c70b9bc).

Files with missing lines Patch % Lines
pkg/plugins/pluginsvc/install_extraction.go 63.51% 34 Missing and 20 partials ⚠️
pkg/plugins/pluginsvc/lock.go 68.00% 10 Missing and 6 partials ⚠️
pkg/plugins/pluginsvc/install.go 84.61% 5 Missing and 5 partials ⚠️
pkg/plugins/pluginsvc/content_digest.go 52.63% 5 Missing and 4 partials ⚠️
pkg/plugins/adapters/claudecode.go 46.66% 4 Missing and 4 partials ⚠️
pkg/plugins/pluginsvc/uninstall.go 89.36% 3 Missing and 2 partials ⚠️
pkg/plugins/adapters/codex.go 55.55% 2 Missing and 2 partials ⚠️
pkg/plugins/pluginsvc/install_git.go 0.00% 4 Missing ⚠️
pkg/plugins/pluginsvc/install_oci.go 50.00% 1 Missing and 1 partial ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main    #6314      +/-   ##
==========================================
+ Coverage   72.94%   72.99%   +0.04%     
==========================================
  Files         742      745       +3     
  Lines       78236    78557     +321     
==========================================
+ Hits        57070    57341     +271     
+ Misses      17188    17182       -6     
- Partials     3978     4034      +56     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@samuv samuv self-assigned this Aug 13, 2026

@JAORMX JAORMX left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Panel review found three transaction-boundary issues that can leave filesystem, DB, group, and lock state inconsistent. Please address the inline findings before merge.

Comment thread pkg/plugins/pluginsvc/install.go Outdated
Comment thread pkg/plugins/pluginsvc/install.go Outdated
Comment thread pkg/plugins/pluginsvc/uninstall.go Outdated
@github-actions github-actions Bot added size/XL Extra large PR: 1000+ lines changed and removed size/L Large PR: 600-999 lines changed labels Aug 14, 2026
@samuv
samuv requested a review from amirejaz as a code owner August 14, 2026 08:46
@github-actions github-actions Bot added size/XL Extra large PR: 1000+ lines changed and removed size/XL Extra large PR: 1000+ lines changed labels Aug 14, 2026
Comment thread pkg/plugins/pluginsvc/install_extraction.go Dismissed
Comment thread pkg/plugins/pluginsvc/install_extraction.go Dismissed
Comment thread pkg/plugins/pluginsvc/install_extraction.go Dismissed
Comment thread pkg/plugins/pluginsvc/install_extraction.go Dismissed
@samuv
samuv force-pushed the plugins-lock/03-install-hooks branch from 1e86051 to 7a77833 Compare August 14, 2026 08:57
@github-actions github-actions Bot added size/XL Extra large PR: 1000+ lines changed and removed size/XL Extra large PR: 1000+ lines changed labels Aug 14, 2026

@JAORMX JAORMX left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The original three findings were addressed, but the follow-up panel found two remaining rollback hazards: unresolved client paths can be mistaken for absent installs, and compensation failures are still discarded after destructive replacement.

Comment on lines +257 to +260
dir, err := s.pluginInstallPath(ct, name, scope, projectRoot)
if err != nil {
continue
}

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

High — path-resolution failures are treated as “this client was not previously installed.” WithClientManager is explicitly optional, but pluginInstallPath requires it. When it is absent (or rejects one client), this continue produces no backup. A later rollback then classifies that existing client as newly added and dematerializes it, turning a lock/group failure into data loss. Please return this error and abort before mutation; only os.ErrNotExist from a successfully resolved path should mean “no prior tree.”

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed. snapshotClientTrees now returns if pluginInstallPath fails (no client manager, rejected client). Only os.ErrNotExist after a successful path resolve is treated as “no prior tree.” Upgrade tests that omit a client manager now fail before mutation. c70b9bc13.

Comment on lines +163 to +167
return &plugins.InstallResult{
Plugin: pl,
RestoreFiles: func(ctx context.Context) {
_ = s.restoreClientTrees(ctx, opts.Name, scope, opts.ProjectRoot, backups, allClients)
},

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

High — the new compensation path still reports success restoring metadata when file restoration failed. restoreDir removes the whole live tree before rewriting it, so a write/registration error can leave the plugin missing or partial. This closure discards that error, and rollbackInstall also ignores DB/group/lock compensation errors; the caller receives only the original lock failure while the restored DB/lock claims corrupt files are installed. Make RestoreFiles return an error and join every compensation failure into the returned install/uninstall error (the DB-delete path in uninstall.go currently discards restoreClientTrees too).

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed. RestoreFiles now returns error. rollbackInstall joins compensation failures with the original error, uninstall store.Delete failure joins restoreClientTrees errors, and materializeForClients joins dematerializeAll on a later-client failure. c70b9bc13.

samuv added 4 commits August 14, 2026 14:52
Project-scope installs must pin plugins: in toolhive.lock.yaml,
and a lock-write failure must roll back the install so nothing
is left silently unpinned. Gated until sync, upgrade, and
Sigstore land.
Hold the per-plugin lock across materialize, DB, group, and
lock-file writes so uninstall cannot race. Restore on-disk
trees and lock entries when a later step fails, matching the
AC that rollback undoes DB and dematerialization together.
Group membership, executable modes, and client marketplace
registration must come back with the files so a failed lock
write cannot leave a half-installed plugin.
A missing client path must abort before mutation, and every
compensation failure has to travel with the original error.
@samuv
samuv force-pushed the plugins-lock/03-install-hooks branch from 7a77833 to c70b9bc Compare August 14, 2026 12:52
@github-actions github-actions Bot added size/XL Extra large PR: 1000+ lines changed and removed size/XL Extra large PR: 1000+ lines changed labels Aug 14, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

size/XL Extra large PR: 1000+ lines changed

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants