feat(relayer): gRPC write path for the JSON-RPC sunset (2026-07-31)#355
Open
ducnmm wants to merge 11 commits into
Open
feat(relayer): gRPC write path for the JSON-RPC sunset (2026-07-31)#355ducnmm wants to merge 11 commits into
ducnmm wants to merge 11 commits into
Conversation
…N-RPC sunset 2026-07-31)
For dev/testnet validation only. Config-gated and OFF by default (SUI_GRPC_URL
empty -> unchanged JSON-RPC behaviour), so this is a no-op until the env is set.
- config.ts: add SUI_GRPC_URL.
- clients.ts: shared write-path suiClient (Walrus/SEAL/Enoki build+certify) uses
SuiGrpcClient when SUI_GRPC_URL is set, else JSON-RPC. Keep a dedicated
suiJsonRpcClient for the query/restore path (getOwnedObjects /
getDynamicFieldObject / suix_queryTransactionBlocks are JSON-RPC-only).
getSuiBalanceMist now reads both {totalBalance} (JSON-RPC) and
{balance:{balance}} (gRPC) shapes — found via live smoke test.
- walrus-query.ts: route index-method reads through suiJsonRpcClient.
NOT validated e2e: full Walrus register/certify/get_blob + Enoki tx.build + SEAL
decrypt over gRPC need a live relayer — that is what this dev deploy tests.
Query path + Rust side JSON-RPC are stage 2.
Under the gRPC client the Walrus certify upload failed at certify_sponsor with 'Transaction was not signed by the correct sender ... given owner/signer 0x0': the SDK's certify tx is built without a sender (register sets its own via setSenderIfNotSet, certify does not), and the gRPC client validates owned-object inputs against the tx sender during build/resolution. Set the sender in the Enoki sponsor path before tx.build; Enoki still sponsors with its own sender and onlyTransactionKind excludes the sender from the bytes, so this is resolution-only and a no-op on JSON-RPC. Surfaced by the dev testnet gRPC relayer test.
Two bugs found while verifying PR #355's SUI_GRPC_URL write path: - enoki.ts direct-sign fallback read `direct.digest` assuming SuiJsonRpcClient's flat SuiTransactionBlockResponse shape. Under SuiGrpcClient, signAndExecuteTransaction resolves to the core-API union `{Transaction: {digest}} | {FailedTransaction: {digest}}` with no top-level `.digest`, so this silently returned undefined once SUI_GRPC_URL was set. Added extractTransactionDigest() to handle both shapes explicitly. - retry/rpc.ts's isRetryableRpcError only matched JSON-RPC-style error text ("429", "503", "timeout"). gRPC errors from SuiGrpcClient carry a status code (RpcError.code) instead, so transient gRPC failures (UNAVAILABLE, RESOURCE_EXHAUSTED, DEADLINE_EXCEEDED, ABORTED) were not retried on the write path. Added a code-based check alongside the existing string matching. Verified both fixes locally: sidecar boots clean on JSON-RPC and gRPC (real mainnet fullnode), getSuiBalanceMist resolves identically on both, and a simulated gRPC transient error now retries and recovers.
Not part of PR #355's own diff (services/server/scripts/sidecar/*) — these are pre-existing bugs in apps/app surfaced only because testnet's public JSON-RPC endpoint is returning HTTP 404 right now (confirmed live), ahead of the documented 2026-07-31 sunset. @mysten/dapp-kit's SuiClientProvider defaults every network to JSON-RPC, so every useSuiClient() consumer broke identically once exercised against testnet. - App.tsx: SuiClientProvider now uses a SuiGrpcClient for testnet via the createClient override (mainnet untouched, still JSON-RPC-healthy). - SetupWizard.tsx / Dashboard.tsx: account/delegate-key lookups used JSON-RPC-shaped getObject/getDynamicFieldObject calls (RpcError: INVALID_ARGUMENT under gRPC). Rewritten for gRPC's flatter .json shape and base64-encoded public_key (both verified live against real testnet objects, not guessed from docs). - useSponsoredTransaction.ts: the direct-sign fallback reused the same Transaction object already .build()'d with onlyTransactionKind:true (no sender attached), causing wrong-sender execution (ENotOwner on-chain, reproduced across two different wallets). Now rebuilds via Transaction.fromKind(). Also added an execute() override since dapp-kit's default calls the JSON-RPC-only client.executeTransactionBlock(), which doesn't exist on SuiGrpcClient (core API only has executeTransaction(), different response shape). - vite.config.ts: dev-only proxy for /api, /health, /version, /config, /sponsor to the real dev backend — its CORS allowlist has no localhost entry, so local testing needs a same-origin proxy (no path rewrite, so signed-request signatures stay valid).
Real, live incident: testnet's public JSON-RPC endpoint (fullnode.testnet.sui.io) already returns HTTP 404 today, ahead of the documented 2026-07-31 sunset. verify_delegate_key_onchain (storage/sui.rs) called this endpoint directly via raw sui_getObject JSON-RPC for every signed request's account resolution. Any failure there — including a dead endpoint — gets mapped to a uniform 401 by auth.rs's constant_time_reject() (deliberate: prevents timing oracles between "wrong key" and "account not found"), so the real cause was indistinguishable from bad credentials. In practice this means relayer.dev.memwal.ai currently cannot authenticate any signed request on testnet, regardless of whether the caller's account/delegate key is correct. This is the same JSON-RPC sunset PR #355 addresses for the sidecar's Walrus write path — just in the Rust backend's auth path, which #355 didn't touch. - Config: add SUI_GRPC_URL (opt-in, mirrors the sidecar/web-app pattern — empty keeps the existing JSON-RPC behavior unchanged). - storage/sui.rs: verify_delegate_key_onchain now branches to a gRPC implementation (LedgerService.GetObject via the sui-rpc crate) when SUI_GRPC_URL is set. gRPC's object.json representation is flatter than JSON-RPC's parsed .fields shape and encodes delegate key public_key as base64 (not a byte array) — verified against real, live testnet objects with a real registered delegate key (not guessed from docs, which for this crate were largely unhelpful — see the two live #[ignore] tests). - find_account_by_delegate_key (registry-scan fallback, used only when the caller sends no x-account-id hint) stays JSON-RPC-only: gRPC has no single-key dynamic-field lookup, only paginated ListDynamicFields, and modern SDKs always send the account-id hint, which resolve_account's Strategy 2 checks first — that's the path this fix actually needed to cover for the live incident. Verified: cargo check clean, cargo clippy clean (no new warnings), full test suite passes (294/299 — the 5 failures are pre-existing, unrelated jobs.rs tests needing a local Postgres connection, not present in this environment). The two new #[ignore] tests hit real testnet gRPC and pass.
Code review of the prior frontend fix (e89788c) found it introduced a real regression: SetupWizard.tsx and Dashboard.tsx were rewritten to call gRPC's getObject/getDynamicField shapes unconditionally, but App.tsx's createClientForNetwork only swaps testnet to SuiGrpcClient — mainnet still gets SuiJsonRpcClient. Those two files would have broken on mainnet (wrong getObject params, and getDynamicField doesn't exist on SuiJsonRpcClient at all). Separately, ConnectMcp.tsx was missed entirely and still used the original JSON-RPC-only shape, breaking on testnet (the default network) — the exact bug class this whole fix line was meant to close, just in a fourth file nobody looked at. Replaced all three call sites with shared helpers in the new utils/suiClientCompat.ts that duck-type the client (`typeof client.getDynamicField === 'function'`) and branch accordingly, instead of assuming one transport. fetchObjectJson's JSON-RPC branch recursively unwraps Move's {fields: {...}} struct envelopes so both transports return the same flat shape to callers — matches gRPC's .json representation at every nesting level, not just the top one. publicKeyToHex handles both delegate-key encodings (JSON-RPC's number[], gRPC's base64 string). Verified: tsc --noEmit clean, dev server HMR-reloaded all three files with no runtime errors.
Both were explicitly marked "TEMPORARY LOCAL PATCH (not for upstream)" — resolving that before this could reasonably be considered for merge. - config.ts / App.tsx: VITE_SUI_GRPC_URL (opt-in, empty keeps JSON-RPC unchanged) replaces the hardcoded testnet-only fullnode URL. createClientForNetwork now applies to whichever network is actually configured (config.suiNetwork), not a hardcoded 'testnet' string. - vite.config.ts: DEV_BACKEND_PROXY_TARGET (opt-in, unset by default — no proxy configured, matching upstream's actual behavior) replaces the hardcoded relayer.dev.memwal.ai target. Uses loadEnv() since vite.config.ts needs .env.local values in its own Node-side execution, not just the client bundle. Verified: tsc --noEmit clean, dev server boots clean with the new env-driven config (.env.local updated to set VITE_SUI_GRPC_URL explicitly).
Four parallel cleanup reviews (reuse, simplification, efficiency, altitude) over the fix commits; net -21 lines. Fixes applied: - [efficiency, HIGH] services/server: sui_rpc::Client was constructed on every auth verification — sui_rpc's Client::new parses the OS root-cert store and opens a fresh TLS channel per call, on the per-request auth hot path, the opposite of how the JSON-RPC path reuses the pooled reqwest::Client. Now built once at startup into AppState (fails fast on a bad SUI_GRPC_URL) and cloned per request — clones share the tonic channel. Both live testnet gRPC tests still pass against the refactored path. - [efficiency] suiClientCompat.fetchAccountIdForOwner: the registry's inner Table ID is an immutable on-chain constant but was re-fetched on every account lookup — now memoized per registryId, halving round trips on repeat lookups. - [reuse] suiClientCompat: dropped all hand-rolled hex/base64 encoders for toHex/fromHex/fromBase64/normalizeSuiAddress from @mysten/sui/utils (already a dependency). - [reuse+altitude] useSponsoredTransaction: executeTransactionCompat moved into suiClientCompat next to the other transport-compat helpers, so the app has one shared isGrpcClient discriminator instead of two divergent method-presence heuristics in two files, and one base64 decode instead of a second inline atob copy. - [simplification] deleted apps/app/src/utils/suiFields.ts — dead after the compat refactor removed its last importer, and its typed shapes had already drifted from the real on-chain encodings (public_key: number[] only, while gRPC returns base64 strings). - [review nits] fixed the stale "testnet-only" header comment in suiClientCompat; publicKeyToHex now warns on an unrecognized encoding instead of silently returning ''. Skipped (reviewed, deliberately not changed): macro-consolidating the four 3-line grpc_value_as_* accessors (idiomatic as-is); a transport trait for verify_delegate_key_onchain (no such convention in storage/*, and the registry-scan fallback genuinely needs per-call-site transport choice); page-local bytesToHex/hexToBytes copies still in live use for key derivation (pre-existing, out of diff scope). Verified: cargo check --tests clean, clippy no new warnings, live gRPC tests pass, tsc --noEmit clean, dev server HMR-reloads without errors.
harrymove-ctrl
approved these changes
Jul 9, 2026
… has 1.88)
Railway build failure: sui-sdk-types (transitive via sui-rpc) pulled in
roaring 0.11.4, which requires rustc >=1.90.0. The Dockerfile pins
rust:1.88-bookworm, so the build failed at `cargo build --release` before
reaching any of our own code:
error: rustc 1.88.0 is not supported by the following package:
roaring@0.11.4 requires rustc 1.90.0
0.11.3 requires rustc 1.82.0 (compatible) and is otherwise identical for our
purposes (roaring is a transitive dep neither our code nor sui-rpc/
sui-sdk-types code paths we hit exercise directly).
Not verifiable via local `docker build` on this machine: the pinned
linux/amd64 base image is emulated via QEMU on Apple Silicon, and QEMU
itself segfaults on rustc invocation before any dependency compiles
(pre-existing environment issue, unrelated to this fix — same crash occurs
on the pre-fix Cargo.lock too). `cargo update -p roaring --precise 0.11.3`
+ `cargo check` locally (rustc 1.92) confirms the resolution is valid; the
actual rustc-1.88 constraint can only be verified by Railway's real amd64
build.
Dockerfile only declared ARG/ENV for a fixed list of VITE_ vars, so Railway's VITE_SUI_GRPC_URL service variable never reached the Vite build — the bundle always baked in an empty value and silently fell back to the dead JSON-RPC client.
The restore path still called suix_queryTransactionBlocks and getDynamicFieldObject over JSON-RPC, which testnet no longer serves — POST /api/restore 500'd with 'returned non-JSON (404)'. The stage-2 GraphQL note is obsolete: the current SDK's gRPC client covers this with listOwnedObjects (server-side type filter + json content) and getDynamicField. With SUI_GRPC_URL set, one paginated listOwnedObjects call replaces both the transaction scan and multiGetObjects, and blob metadata is BCS-decoded from getDynamicField (Metadata = VecMap<String,String>). Unset, the original JSON-RPC path still runs — same reversible opt-in as the write path. Verified against live testnet gRPC: real account returns all blobs with correct memwal_* metadata; namespace filtering, full-scan and empty-owner paths OK; 78/78 unit tests pass.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Migrate the relayer's write path off JSON-RPC to gRPC ahead of the Sui JSON-RPC sunset on 2026-07-31. Opt-in and off by default (
SUI_GRPC_URLempty → unchanged JSON-RPC behaviour), so merging is a no-op until the env is set.What changes
config.ts— addSUI_GRPC_URL. When set, the shared core client uses gRPC; empty keeps JSON-RPC.clients.ts— the write-pathsuiClient(Walrus register/certify, SEAL, Enoki build) is aSuiGrpcClientwhenSUI_GRPC_URLis set, else JSON-RPC. A dedicatedsuiJsonRpcClientstays for the blob query/restore path, which uses JSON-RPC index methods (getOwnedObjects,getDynamicFieldObject,suix_queryTransactionBlocks) that gRPC doesn't expose — migrating that to GraphQL is stage 2.getSuiBalanceMistreads both{totalBalance}(JSON-RPC) and{balance:{balance}}(gRPC) shapes.walrus-query.ts— route the index-method reads throughsuiJsonRpcClient.enoki.ts— set the tx sender beforetx.buildin the sponsor path: the gRPC client validates owned-object inputs against the tx sender during resolution, and the SDK'scertifytx is built without a sender (register sets its own, certify doesn't), so gRPC rejected it (Transaction was not signed by the correct sender ... given owner/signer 0x0). Enoki still sponsors with its own sender;onlyTransactionKindexcludes the sender from the bytes, so this is resolution-only (no-op on JSON-RPC).Verified end-to-end on dev (testnet,
SUI_GRPC_URL=https://fullnode.testnet.sui.io)register_sponsor → certify_sponsor → get_blob → metadata_transfer → blobId, allphase_ok, 0phase_failed, balance logs non-null.seal decrypt-batch ok: 1/1, 0 errors, HTTP 200.Two gRPC-vs-JSON-RPC bugs were found and fixed via this dev test (getBalance shape; certify sender).
Notes / follow-ups
spike/indexer-checkpoint), not here.SUI_RPC_URLdefault; fix(relayer): recover Walrus register destroy_zero from stale WAL price (#351) #352 adds the env override.