Repository navigation
op-reth: keep proofs history shard keys valid on unwind - #23334
akatsukiwork0320 wants to merge 2 commits into
Conversation
Restore the history sentinel after suffix unwinds, including legacy finite tails. Keep retention pruning unchanged and cover historical reads and shard structure. Based on omerfirmak's root-cause analysis and reproduction in ethereum-optimism#22501. Prepared with AI assistance.
|
/ci authorize bc364c2 |
geoknee
left a comment
There was a problem hiding this comment.
Reviewed against develop. The bug is confirmed and the fix looks correct.
Confirming the issue. I ran this PR's new regression tests against develop's write.rs: 13 of 17 fail. The headline failure is the one from #22501: after unwinding to 1801 and refilling, reads at 1800–1999 return the tip's nonce (2100), and the boundary shard stays keyed at 2000 while its blocks end at 1800. Both live paths reach the buggy code: unwind_history (CLI unwind / revert) and replace_updates (reorgs).
The fix. Walking back from the key's real last shard and putting the surviving tail under u64::MAX keeps both the find_source invariant and the append invariant. The precondition that the range ends at latest holds for both callers. With the PR's write.rs, all 142 db::store tests pass locally.
Nit (optional): unwind_changesets_and_history only documents that its range must end at latest. A debug_assert! would make that rule enforced rather than doc-only.
Existing nodes are not healed, as the description says. I checked this with a store corrupted by the old unwind logic and then run on this branch. Bad reads stayed (800/800 dense, 5600/5600 sparse), even after more appends and another unwind/refill. A one-off pass that re-keys each shard to its own last block, and the key's final shard to u64::MAX, brought them to 0, and they stayed at 0 through later unwinds. I'll open that pass as a follow-up op-reth proofs subcommand.
One correction to the issue's timeline: the partial-prune branch came in with the v2 store itself (#19928, first released in op-reth v2.2.1), not 3bccc60. Nodes that opted into --proofs-history.storage-version=v2 since then, and every proofs-history node on v2.7.0-rc.1 and rc.2, should be treated as potentially affected.
unwind_changesets_and_history documented that its range must end at the proof window's latest block; check it in debug builds, as suggested in review. The seeded-history fixtures that call it directly now set the proof window first. Prepared with AI assistance.
|
Thanks for reproducing the issue and checking the fix so carefully, and for opening #23341 for the repair pass. I added the suggested Locally, |
|
/ci authorize 50bde65 |
Unwinding proofs history could leave the surviving boundary shard under its old key. After new appends, historical reads could fall through to current state, making
eth_getProofserve chain-tip trie nodes for historical blocks, as reported in #22501.Unwind now walks backwards from each key's actual last shard and restores the surviving tail under
u64::MAX. This preserves the sentinel expected by appends, including when an earlier version left a tail without one. Re-keying to the surviving maximum would fix the lookup but leave the append invariant broken. Retention pruning is unchanged.This prevents the reported stale-key defect during subsequent unwinds; it is not a migration of previously corrupted history. Untouched malformed intermediate shards can still produce incorrect historical proofs and need separate recovery.
Local validation:
just lintpassed.just test-unit, restricted to the changed crate and its reverse dependency closure (17 of 76 workspace packages), passed with 822 tests and 3 skipped. Online tests were excluded; no live-chain/RPC proof replay was performed. Regression tests cover table round trips, boundary-shard structure, legacy finite tails and neighbouring keys. An equivalent AI review against therust-code-reviewerguide found no blocking issues; no findings were dismissed.Root-cause analysis and original reproduction by @omerfirmak.
Closes #22501
Prepared with AI assistance, including implementation, tests and local validation.