Skip to content

feat(web): storage wizard and browser uploads to your own storage - #37

Merged
PastaPastaPasta merged 5 commits into
masterfrom
feat/web-launch-ux
Sep 26, 2026
Merged

PastaPastaPasta merged 5 commits into
masterfrom
feat/web-launch-ux

Conversation

@PastaPastaPasta

Copy link
Copy Markdown
Owner

What

This is the web half of P0 #4 in docs/design/ux-dx-spec.md §11: bring your own storage from the browser. It adds a storage wizard at /settings/storage and the upload path that browser writes pushing packs will use: merges (#13), fork follow-ups and release assets. Nothing here is hosted by Forge. Credentials stay encrypted in this browser, bytes go to the user's bucket or IPFS node, and Platform stores only the signed packManifest.

Storage wizard (§3.1)

  • Provider tiles:
    • Cloudflare R2 (recommended: free egress)
    • Backblaze B2
    • AWS S3
    • MinIO / other S3
    • IPFS (kubo)
    • IPFS pinning service
    • Dash Platform, last (permanent, about 0.28 DASH/MiB)
  • Per-provider form. Each field has a "where to find this" hint with a console link. Secret fields carry a lock and the line "Stored encrypted in this browser only. Never sent to Forge (there is no Forge server) and never written on-chain."
  • Live test, run from the page, so it checks what the browser will actually do, CORS included.
    • S3 rows: signed PUT → signed GET → anonymous GET via the public URL → ranged read (Content-Range visible) → CORS preflight for PUT (browser pushes) → delete probe.
    • IPFS rows: kubo API → add + CID check + pin → gateway re-read → public gateway → pinning service → unpin.
    • A failing CORS row shows a copy-paste fix per provider (R2 JSON, AWS put-bucket-cors, b2 bucket update, mc, ipfs config), prefilled with the bucket and this app's origin, plus Re-test. Reads are allowed from any origin; writes only from this app's origin.
    • The test tells a CORS refusal apart from a dead host and from a redirect (wrong region or endpoint).
  • Replication: one place, every chosen place, or Platform as a fallback (costed, and it asks first).
  • Per-repo override: repo Settings → Your browser pushes.
  • Cost comparison card, always visible.

Credentials: in the vault, never localStorage

The #24 vault now holds a second AES-256-GCM blob next to the limited-key record:

  • It is sealed under an HKDF of the vault's data key, with additional authenticated data bound to the network and identity.
  • While the vault is unlocked, only a non-extractable CryptoKey is in memory, and it is dropped on lock.
  • The record and the blob are written in one IndexedDB transaction.
  • A key renewal re-seals the blob under the new key. A renewal made while locked tells the user their settings could not be carried over.
  • Settings that can no longer be opened can be discarded from the wizard.
  • Editing a profile never puts stored secrets back into the page, and every field has autocomplete="off".

Upload path (lib/storage, lib/repo/push.ts)

  • SigV4 on WebCrypto (sigv4.ts) is pinned to the same vectors as forge-core backends/sigv4.rs: the AWS test suite and the S3 reference examples. The signed host and path are exactly what fetch sends (redirect: 'error', credentials: 'omit').
  • kubo: Forge's pinned import parameters, with the CID re-derived locally (cid.ts, which matches ipfs add --only-hash, multi-chunk DAG roots included) and a pin check. The Pinning Service API pin is reused when one already exists.
  • Verified before it counts, as in the CLI:
    • S3: a signed re-read of the whole object up to 16 MiB, or head and tail windows above that, with one re-upload if it fails. Then an anonymous head-and-tail read through the public URL.
    • IPFS: CID and pin, then the gateway re-read, then the public gateway.
  • Only public https addresses are recorded on chain or fetched by readers (lib/net.ts). Loopback, private and plain-http URLs are refused, and the storage test fails a public URL like 127.0.0.1 with the reason.
    • The CSP allows http://127.0.0.1:* and http://localhost:* so the storage settings can reach the user's own local kubo or MinIO.
    • That is not a read path. Readers skip loopback and private URLs even if a hostile manifest records them.
  • Every Platform write asks first, with its price. That covers a Platform target in the policy, the fallback, and the case where nothing is configured.
  • The manifest comes last. writePackManifest matches forge-core write_pack_manifest field for field and is idempotent on the signer's (repoId, $ownerId, packHash) slot. It is written only after at least replicas copies are verified.
    • platform:// URIs come first, then https, then the rest; s3:// is dropped first when the list is over budget (parity with Replication::uris / manifest_uris).
    • Chunks split exactly like pack::split.
  • Timeouts follow forge-core's clients: 15 s to connect and 120 s idle, not a total cap, so large uploads work.
  • kubo trust: the wizard says plainly that kubo's RPC API is the node's admin interface. The fix block:
    • creates a full-access owner token first, because any API.Authorizations entry locks the RPC API for the CLI and WebUI;
    • creates a Forge token limited to add, pin/ls, pin/rm, id and version;
    • merges this origin into the existing allowed list. Tested against a fresh kubo 0.42 repo and against one with an existing list.

Reviews

  • An independent security review (mandatory for credentials and SigV4) found 6 major and 12 minor issues and no blockers. They are fixed in f1a5da4. The reviewer verified that commit, and its follow-ups are fixed in 3b8d715.
  • A simplification pass is in 56b9d58.
  • A contrast fix is included. danger text failed WCAG AA on both themes, so danger-400 and danger-700 were added and applied to the danger button. lib/design/contrast.test.ts pins the ratios.

Testing

  • Unit: pnpm typecheck, pnpm lint and pnpm test pass (923 passed). Both builds pass (testnet, and devnet moutai). New unit suites:
    • sigv4.test.ts: the AWS suite and the S3 examples.
    • cid.test.ts: the kubo-derived CIDs.
    • storage.test.ts (49 tests): profile rules, published-URL rules, reader URL filtering, policies and renames, vault sealing, renewal and discard, chunk split, URI order and budget, the CORS blocks, and the replication engine against fake S3 and kubo (public re-read, re-upload, the policy failing, Platform asking first).
  • Live, moutai + local MinIO (lib/storage/upload.live.test.ts):
    • MinIO was exposed over a public https tunnel. SigV4 upload → verification by re-read, including the anonymous public read → packManifest on moutai (0.00067 DASH) → the manifest read back from Platform → the bytes fetched and hashed through the web reader.
    • A rerun writes nothing: cost 0, same manifest id.
  • Playwright, moutai build (e2e/storage-wizard.spec.ts), 4/4 pass:
    • s1, MinIO: write, read, range and CORS rows pass, and the public row fails with the loopback reason. The profile is sealed in the vault, with no secret in localStorage or IndexedDB plaintext, and it comes back after a reload and unlock.
    • s2, a host without CORS: the CORS fix block appears, prefilled.
    • s3: the per-repo override.
    • s4: axe finds 0 serious violations, and the page works at 390 px.

Screenshots: empty, MinIO tested, profile saved, CORS fix, repo policy, mobile.

Not in this PR

  • Nothing calls storeAndRecordPack from the UI yet. Its callers come in the browser merge, fork and release PRs; the live test exercises the whole path.
  • Base costs for the packManifest and chunk documents are estimates until measured from the browser (the live manifest cost 0.00067 DASH against a 0.0006 estimate).
  • Pinning services receive kubo's multiaddrs, the same as the CLI.

🤖 Generated with Claude Code

PastaPastaPasta and others added 5 commits September 26, 2026 17:26
Adds /settings/storage (ux-dx-spec 3.1): provider tiles for R2, B2, AWS S3, MinIO, IPFS kubo, an IPFS pinning service and Dash Platform; per-provider forms with where-to-find hints; a live browser test (signed PUT, signed GET, anonymous public GET, CORS for Range reads and for PUT, delete probe) with copy-paste CORS fixes per provider; replication choice; the cost card.

Credentials are sealed in the vault next to the limited key (AES-GCM under an HKDF of the vault data key), never in localStorage. The upload path (lib/storage) signs S3 requests with SigV4 on WebCrypto, pinned to the forge-core test vectors, adds to kubo with the CID re-derived locally, pins remotely, verifies every copy by re-read, and only then writes the packManifest (lib/repo/push.ts), matching forge-core field for field.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Shared byte and error helpers, one StoreOptions type, a status-hint switch, and fewer duplicated failure paths; no behavior change.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Only public https addresses are recorded on chain or fetched by readers (lib/net.ts): profiles refuse a loopback or private public URL for uploads, fitManifestUris refuses non-https URIs, externalFetchUrls skips them, and the CSP opens loopback http only in devnet and dev builds. A copy counts only after an anonymous head-and-tail read through its public URL. Every Platform write asks first with its price, and the policy error says whether chunks were written.

Requests use a 15 s connect and 120 s idle deadline instead of a 60 s total cap. The kubo fix creates a token limited to add, pin and id and merges this origin into the allowed list; the form explains the admin-API trust. Test results are tied to the exact values tested. Also: the reserved platform name, renames kept in policies, the record and settings written in one transaction with a discard option, secrets never put back into the page, header-unsafe secrets refused without echoing them, redirect vs CORS vs down told apart, live regions.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Amber and green status text in the storage wizard now pairs the -700 shade on light surfaces with the base on dark, matching the Verification card.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
A local kubo or MinIO works in every build: the CSP allows loopback http again (readers fetch only public https URLs themselves), and only the local API and endpoint may be loopback, never a published address. The kubo fix script works on a fresh node (the origin merge no longer relies on a failing config read) and creates a full-access owner token first, since any API.Authorizations entry locks the RPC API for the CLI and WebUI.

Also: the chunks-written flag only after a chunk is written, request deadlines cleared when the body is not read, the test fingerprint hashed, AWS's bare wrong-region 301 told apart from CORS, and the renewal notice cleared once shown.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
@coderabbitai

coderabbitai Bot commented Sep 26, 2026

Copy link
Copy Markdown

Warning

Review limit reached

Next included review available in 13 minutes.

Check out review usage here.

View limit details

Limit details: You’ve used the included review currently available.

You've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository.

Learn how review limits work.

Review configuration:

⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Advanced

Run ID: 70191c42-fcd7-44da-bf13-2e12b54c6ad7

📥 Commits

Reviewing files that changed from the base of the PR and between aa1cf88 and 3b8d715.

📒 Files selected for processing (40)
  • forge-web/app/layout.tsx
  • forge-web/app/settings/page.tsx
  • forge-web/app/settings/storage/page.tsx
  • forge-web/components/repo/settings-content.tsx
  • forge-web/components/storage/copy-block.tsx
  • forge-web/components/storage/cost-card.tsx
  • forge-web/components/storage/profile-form.tsx
  • forge-web/components/storage/repo-storage-policy.tsx
  • forge-web/components/storage/storage-test.tsx
  • forge-web/components/storage/storage-wizard.tsx
  • forge-web/components/ui/button.tsx
  • forge-web/contexts/auth-context.tsx
  • forge-web/e2e/helpers.ts
  • forge-web/e2e/storage-wizard.spec.ts
  • forge-web/hooks/use-storage-config.ts
  • forge-web/lib/auth/controller.ts
  • forge-web/lib/auth/vault.ts
  • forge-web/lib/design/contrast.test.ts
  • forge-web/lib/idb.ts
  • forge-web/lib/net.ts
  • forge-web/lib/repo/push.ts
  • forge-web/lib/sdk/cost.ts
  • forge-web/lib/storage/cid.test.ts
  • forge-web/lib/storage/cid.ts
  • forge-web/lib/storage/cors.ts
  • forge-web/lib/storage/index.ts
  • forge-web/lib/storage/ipfs.ts
  • forge-web/lib/storage/probe.ts
  • forge-web/lib/storage/profiles.ts
  • forge-web/lib/storage/s3.ts
  • forge-web/lib/storage/sigv4.test.ts
  • forge-web/lib/storage/sigv4.ts
  • forge-web/lib/storage/storage.test.ts
  • forge-web/lib/storage/store.ts
  • forge-web/lib/storage/upload.live.test.ts
  • forge-web/lib/storage/upload.ts
  • forge-web/lib/storage/util.ts
  • forge-web/lib/view/browse-fallback.test.ts
  • forge-web/lib/view/browse-source.ts
  • forge-web/tailwind.config.js

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@PastaPastaPasta
PastaPastaPasta merged commit 464f12b into master Sep 26, 2026
4 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant