feat(providers): resolve external broker credentials just in time

[drew]

User Story

I need to integrate long-running OpenShell agent sandboxes with an external credential broker that owns authorization, token issuance, renewal, and upstream revocation. When an agent makes an allowed API or repository request, OpenShell should obtain the appropriate credential just in time and inject it without exposing it to the agent. The broker should remain the source of truth; I should not need to push each replacement token into OpenShell provider storage or restart the agent after rotation.

Problem Statement

OpenShell has provider placeholders, credential storage drivers, and on-demand dynamic token grants for supported SPIFFE/OAuth flows. The missing workflow is a generic external-broker binding: configure an opaque credential reference, authenticate OpenShell to the broker, and resolve that reference under the requesting sandbox's current authorization at egress time without storing the upstream token durably.

The existing External refresh strategy does not itself provide broker lookup. Ordinary provider updates require an external delivery loop, and ordinary static references remain revision-scoped. Existing SPIFFE/OAuth token grants should be reused where compatible, but should not require every external broker to implement those particular grant flows.

Impact / Why This Matters

Operators otherwise need to synchronize short-lived tokens into provider records, coordinate rotation and withdrawal across two systems, or implement a separate forwarding integration. That duplicates credential lifecycle state and can leave running processes using old references. OpenShell needs to enforce sandbox policy while the existing broker remains authoritative for credential eligibility and issuance.

Proposed Design

1. An operator registers a trusted broker connection and a provider credential binding containing an opaque broker reference. The binding identifies the owning workspace, provider, permitted sandboxes, and allowed destinations. Broker connection credentials stay outside workloads.
2. An authorized controller attaches that provider to a sandbox. The workload uses a stable placeholder where needed, or existing supported automatic header injection. It receives no upstream token or broker authentication material.
3. For a matching outbound request, OpenShell checks process and destination policy, then resolves the configured reference using authenticated service communication and trusted sandbox/binding context. Workload-controlled headers or reference strings cannot select another owner, broker endpoint, or broader scope.
4. The broker evaluates current authorization and returns a credential with explicit validity and authorization metadata. OpenShell validates the response and substitutes the credential only into the authorized request. OpenShell does not independently renew it with the upstream issuer.
5. Persist references and policy metadata, not returned upstream tokens. Permit only transient in-memory custody of resolved credentials. After restart, resolution requires a fresh broker authorization.

Provide a mode that consults the broker before every dispatch. If caching is supported, make it an explicit bounded policy: credential expiry and authorization-cache expiry are distinct, and the earliest deadline wins. Document the maximum withdrawal delay. A cached token is usable only while both authorizations remain valid; once validation is required, broker failure must fail closed. An explicit denial or applied local detach invalidates matching cached authority. Late lookup responses must not restore a detached or superseded binding. Previously dispatched upstream operations cannot be undone.

Reuse existing provider, dynamic token-grant, credential-driver, and injection extension points where they satisfy this workflow. External can continue to express external refresh ownership; the final schema and adapter placement are design decisions. This request does not require a parallel broker framework or a provider-specific issuer implementation.

Suggested UX

Illustrative configuration only; these fields are not existing OpenShell syntax:

external_credential_binding:
  provider: repository-access
  workspace: engineering
  sandbox: coding-session
  credential_key: API_TOKEN
  broker: corporate-credentials  # separately configured trusted connection
  reference: grants/repository-read
  destination:
    host: api.example.com
    port: 443
    path: /repos/team/project/**
  cache:
    mode: none

The operator configures broker authentication separately, attaches the provider, and runs the client normally. A permitted request triggers resolution; subsequent credential replacement requires no provider-value update or client restart. Configuration and status surfaces distinguish authorization denial, broker unavailability, expiry, and local withdrawal without exposing secrets.

Acceptance Criteria

• A provider can reference an external broker credential without supplying or durably storing its upstream token. A denied process/destination request does not trigger credential acquisition.
• A real sandbox request resolves through an authenticated broker integration and reaches a controlled HTTPS upstream with the expected credential. Neither the workload nor logs/status expose the token or broker authentication material.
• The same running client continues across broker token replacement using its unchanged placeholder or configured injection behavior, without a provider-value update or restart.
• Broker requests carry trusted sandbox/binding context. Cross-workspace, cross-sandbox, detached-provider, destination-mismatch, and reference-substitution attempts are rejected.
• With caching disabled, each new dispatch requires successful current broker authorization. After broker denial, no subsequent request is dispatched using previously resolved credentials.
• Any supported cache is memory-only, isolated by authorization binding, bounded by token and authorization expiry, and has a documented withdrawal interval. Applied denial/detach invalidates matching entries; delayed responses cannot restore withdrawn authority.
• Missing, expired, malformed, mismatched, or unauthorized responses fail closed. Broker timeout/outage cannot cause use beyond the permitted cache window, upstream issuance fallback, or replay of an uncertain upstream mutation.
• Restart discards resolved tokens and requires fresh authorization before reuse. Concurrent resolution, rotation, withdrawal, and failure recovery have bounded behavior without mixing bindings.
• End-to-end coverage proves rotation in a persistent client, isolation, no-cache authorization, any supported cache deadlines, outage behavior, withdrawal races, and restart recovery. Existing static, gateway-refresh, and SPIFFE/OAuth token-grant behavior remains covered.
• Published provider documentation and relevant contributor/public skills explain configuration, supported authentication/injection paths, custody, failure behavior, and revocation limits.

Alternatives Considered

• Push updates using External: useful for externally rotated credentials; Keep credential references current after external rotation #3336 addresses stable references for that model. This request additionally removes the mandatory push loop and resolves current broker authority on demand.
• Existing SPIFFE/OAuth dynamic token grants: already provide on-demand acquisition, in-memory caching, and injection. Prefer that path for compatible brokers and extend it where possible. The requested addition covers opaque broker references and an explicit authorization/withdrawal contract beyond those specific grant flows.
• Credential storage drivers: already resolve handles and optional expiry, but their current provider/workspace storage contract does not by itself establish request-time sandbox authorization and broker withdrawal behavior. Reuse or extend this boundary if appropriate.
• Supervisor middleware or gateway interceptors: may implement parts of the adapter, policy checks, or profile delivery. Prefer an existing extension if it meets the full identity, credential custody, injection, and lifecycle requirements; extension choice is not prescribed here.
• OpenShell-owned issuer refresh: appropriate when OpenShell owns the refresh grant. It does not meet the requirement that an external broker retain issuance and authorization ownership.

Agent Investigation

Reviewed checkout 48d9ab3d0d9a343365dea1b0cd87565050ac658e; these are source findings, not runtime validation:

• Dynamic token grants already resolve on demand using SPIFFE JWT-SVID-backed OAuth grants and support disabling the token cache.
• Credential-driver protocol resolves gateway-owned handles with provider/workspace ownership and optional expiry.
• Provider refresh skips External in the gateway refresh worker; it does not implement generic broker acquisition.
• Provider environment construction currently restricts refresh-managed stable handles to gateway-mintable strategies.

Related work: #3336 (externally pushed rotation), #3654 (sandbox credential RPC identity), #1883 (closed request/session credential proposal), and #1755 (closed broader credential-delivery tracking). This issue focuses on generic reference-based broker resolution at request time.

Checklist

• I've reviewed existing issues and the published docs.
• This is a design proposal with observable acceptance criteria.