Repository navigation
Conversation
7b51ebc to
becef55
Compare
- revm 36 -> 43.0.3, revm-inspectors 0.44, revm-bytecode 43, alloy-evm 0.39, alloy / alloy-contract 2.5, alloy-sol-types 1.7.3, plus alloy-chains/hardforks/rlp - vendor op-revm (ported to revm 43 upstream in ethereum-optimism/optimism#23046) into crates/op-revm until a compatible op-revm is published on crates.io - port all chain modules (arbitrum, tempo, bsc, polygon, cosmos, citrea, monad, rsk, moonbeam, iotex) and rpc to the new handler / gas / precompile / instruction APIs, preserving pre-Amsterdam behavior - standalone: default mainnet-derived spec AMSTERDAM -> OSAKA (revm 43 AMSTERDAM enables Glamsterdam gas); keep --spec-id on the legacy revm 36 numbering; OpSpecId::OSAKA -> KARST - bump rust-version to 1.95 and Docker RUST_VERSION to 1.97.1 - fix flaky cpu recorder / spawn_blocking_with_cancel tests
becef55 to
02ac617
Compare
joshklop
left a comment
There was a problem hiding this comment.
Full reth/revm/alloy update sweep found six runtime/build risks and two compatibility notes; details are inline. Coverage included all 283 reth commits, the complete revm/alloy ranges, both lockfiles, fork ancestry, and all changed mirror annotations. just mirrors stale --json and git diff --check passed; affected-package tests passed. The revm/asyncdb feature-unification failure was reproduced locally despite the standard CI matrix being green.
f029d8c to
c099617
Compare
sebastianst
left a comment
There was a problem hiding this comment.
Review against docs/ai/reth-update-review.md, including the partitioned full-range sweep (#23063): every commit in reth 3a83ccc546..dfcb10454c (v2.5.2 fork base → v2.6.0), revm v115..v119, alloy-evm 0.38→0.39, revm-inspectors, alloy-eip7928, alloy-hardforks and reth-core 0.6→0.7. The results are a per-commit verdict ledger, the mirror funnel, and a check of the adaptation diff. Mirrors: all 63 advanced tags match their upstream diffs. Old fork cherry-picks are all contained in v2.6.0.
Four blocking findings. For each, a red test fails on this head and passes with the fix. Fixes and tests are ready to share.
1. [blocking, consensus] Isthmus withdrawals_root is wrong over a masked state trie.
parent_state() is now an OverlayStateProvider (reth f0166c32e2), and state masking is on by default (--engine.num-state-masking-blocks 30, bd61f3919e). Upstream OverlayStateProvider::storage_root (reth crates/storage/storage-overlay/src/provider.rs:573) builds the overlay but passes only its hashed storage to DbStorageRoot::overlay_root. That drops the overlay trie nodes and walks the DB storage trie directly. storage_proof and storage_multiproof have the same problem. Under partial persistence, disjointed_merge_batch skips DB trie nodes that the masking suffix overwrites. Their cached child hashes therefore predate the persisted hashed storage, and siblings outside the prefix set are hashed with stale values.
Ethereum never uses storage_root for consensus. OP does, twice: the validator's Isthmus check (node/src/engine.rs:149) and the sequencer's header (evm/src/build.rs:81). Each node's DB layout depends on its own persistence timing, so op-reth nodes can disagree with each other, not only with kona/op-program.
A regression test in consensus/src/validation/isthmus.rs writes L2ToL1MessagePasser storage across the persisted batch, the masking suffix and an in-memory block, over v1 and v2 storage. It compares withdrawals_root via OverlayStateProviderFactory::database_provider_ro() against a from-scratch root. The masked cases fail (0x9334cfa9…acc4 vs 0xa976ad4c…672a); the unmasked controls pass. The same scenario as an upstream storage-overlay test is red at v2.6.0 and green once storage_root/storage_proof/storage_multiproof read nodes through InMemoryTrieCursorFactory over the overlay nodes, with the full reth-storage-overlay suite passing.
The fix belongs upstream, then backported to the fork pin. Until then, the only mitigation I see is forcing num_state_masking_blocks = 0 in op-reth. The same flaw affects the conditional-tx storage roots (rpc/src/eth/ext.rs:93). eth_getProof uses proof and is not affected.
2. [blocking] op-reth proofs init seeds a corrupt proofs store after an unclean shutdown.
With masking on, a datadir left by a crash, OOM or SIGKILL, or a snapshot of a running node, has Finish.block_number at the tip. The hashed and trie tables, however, hold a disjointed merge at Finish.partial_state_trie. reth repairs that only at node launch (launch/common.rs:553). proofs init opens with RoInconsistent, anchors at chain_info().best_number and copies the tables unchecked (cli/src/commands/op_proofs/init.rs:161,179, trie/src/initialize.rs). This PR adapted the ExEx to partial_state_trie but not init.
A repro persists blocks via the real save_blocks with frontiers 6/3. Init returns Ok, seeding slot values from before the tip and a state root that differs from header #6. With V2 plus backfill (the default), backfill then fails with StateRootMismatch, and re-running init reports "already initialized", so the proofs dir must be deleted. With --proofs-history.skip-backfill or V1, the node silently serves wrong historical proofs; --proofs-history.verification-interval defaults to 0.
Suggested fix: InitializationJob::run reads the Finish checkpoint in its copy transaction and refuses when partial_state_trie < block_number, with an actionable error ("start op-reth node once and stop it cleanly"). Reading both in one transaction also closes the existing race between chain_info() and the copy.
3. [blocking] BAL disablement is incomplete (inline on node.rs).
4. [blocking, downstream API] Filtered flashblock log streams return nothing.
reth #26946 (1ca1201167) added if !filter.matches_bloom(header.logs_bloom()) { return Ok(vec![]) } to matching_block_logs_with_tx_hashes. The synthetic header in OpEthApi::flashblock_receipts_stream (rpc/src/eth/mod.rs:180) leaves logs_bloom zeroed. So since this bump, any filter with an address or topic yields no logs. Only the empty filter still works.
A unit test fails for the address and topic filters (0 logs, expected 1). It passes once the header carries alloy_primitives::logs_bloom(fb.metadata.receipts.values().flat_map(|r| r.logs())). Nothing in-tree calls the method (eth_subscribe logs uses upstream's canonical log_stream), so only downstream users of this public API are affected.
Follow-ups referenced in threads were closed. #23096 (+12 pending timestamp) and #23103 (pending logs in latest..pending) were closed unmerged after the replies pointing at them. Only issues #22685 and #23104 remain, and #23104 still names the closed PR. #23106 (proofs ExEx acks) is an open draft; see the inline note on exex/src/lib.rs, which shows the problem also applies in live mode.
Migration notes: missing items
- Startup failure.
--engine.persistence-threshold≤ 35 now fails startup unless--engine.num-state-masking-blocks 0is also set (EngineArgs::validate). This includes the old default of 7 when it is pinned via flag orRETH_ENGINE_PERSISTENCE_THRESHOLD. - Env vars. The engine flags now also read
RETH_ENGINE_PERSISTENCE_THRESHOLDandRETH_ENGINE_NUM_STATE_MASKING_BLOCKS. - Priority fee.
--txpool.minimum-priority-feenow also rejects local and private (conditional) txs (74ce7f1b2b). - Tx fetcher. New per-peer cap of 4096 announced hashes, with no CLI flag.
--max-tx-pending-fetchnow also counts inflight hashes (b2d7bd7f3f). eth_getLogs. Errors instead of[]on missing receipts, for example pre-Bedrock ranges oninit-state --without-ovmdatadirs (d5cd94331e). Also errors whenfromBlockis beyond head (1e9a9438ad).- RocksDB.
max_open_filesis now -1 (keep all open) when RLIMIT_NOFILE ≥ 131072 (853a42db1c). - Tx pool.
--txpool.max-batch-size 0is now rejected.
payload_handoff.rs removal. The removal is fine, but the stated reason isn't quite right. Upstream #26917 (68877d5fd7) removed the persistence lease/handoff mechanism entirely, including #26708's own test, rather than the cherry-pick becoming redundant.
The revm/asyncdb dismissal holds: the feature is enabled in neither lock nor in op-rbuilder. Other non-blocking notes are inline.
🤖 Co-created with Claude Opus 5.5
d5cc28b to
e8453a0
Compare
Moves op-reth onto the reth v2.6.0 dependency stack so it remains aligned with current reth, REVM, and Alloy APIs. Every cherry-pick on the previous fork pin is in v2.6.0 or superseded upstream. The new pin (ethereum-optimism/reth#15) carries the upstream fixes paradigmxyz/reth#27129, #27247, #27252, #27286, #27429, #27634, #27782 and #27832. Problems the reviews found that already exist on develop are tracked separately on the Protocol Team board.
Migration notes:
--db.rocksdb-block-cache-sizenow takes effect. Before, it was ignored in favor of the 128 MiB builder default, so operators with a larger configured value may see higher memory use after upgrading.--engine.persistence-thresholdnow defaults to 50 instead of 7.--rpc-cache.max-cached-tx-hashesnow defaults to 100000 instead of 30000.--rpc-cache.max-headersis now a hidden no-op.--engine.num-state-masking-blocks, default 30 upstream). op-reth keeps it off: the flag defaults to 0, is hidden, and any nonzero value is rejected at startup. Turning it back on is tracked in https://github.com/ethereum-optimism/protocol-team/issues/457.ethwire protocol between upgraded peers changes frometh/69toeth/71. Compatibility with older peers is retained through highest-common-version negotiation.--txpool.minimum-priority-feenow also rejects local and private (conditional) transactions; before, it applied only to transactions received over p2p. On a sequencer that covers nearly all RPC submissions; those below the minimum are now rejected with an error (fix(txpool): enforce minimum priority fee for local transactions paradigmxyz/reth#26861).eth_getLogsreturns errors where it used to return[]:op-reth init-state --without-ovm(fix(rpc): reject eth_getLogs queries over pruned receipts paradigmxyz/reth#27002);fromBlockbeyond the head returns -32602, which polling indexers that requestlastSeen + 1on fast chains will hit (fix(rpc): return invalid params error for eth_getLogs fromBlock beyond head paradigmxyz/reth#26891).--txpool.max-batch-size 0is now rejected at startup (fix(txpool): reject zero transaction batch size paradigmxyz/reth#26806).--engine.persistence-thresholdcan now also be set withRETH_ENGINE_PERSISTENCE_THRESHOLD. A nonzeroRETH_ENGINE_NUM_STATE_MASKING_BLOCKSis rejected like the flag.--max-tx-pending-fetchnow also counts in-flight hashes.max_open_files = -1) whenRLIMIT_NOFILEis at least 131072 (fix(rocksdb): better max_open_files config paradigmxyz/reth#26812).Proofs history (
--proofs-history):Finishcheckpoint for every bytecode and block-hash read. That fails during pipeline sync and gets expensive during catch-up. The proofs ExEx now reads bytecode and block hashes from a snapshot of reth's in-memory chain, then from reth's database.op-reth proofs initrefuses to copy a state trie that lags reth's tip.BREAKING CHANGE: op-reth no longer accepts reth's block access list (BAL) options, because OP blocks do not carry EIP-7928 block access lists. This affects operators whose
op-rethcommand lines or unit files pass any of these flags, which previous releases accepted:--rpc-cache.max-bals--db.balstore-cache-size, onnodeand every other subcommand (db,init,prune,stage,proofs, ...)--engine.disable-bal-parallel-execution--engine.disable-bal-parallel-state-root--engine.disable-bal-batch-ioThey are hidden from help, and using any of them now makes
op-rethexit with "Block access list options are not supported by op-reth". The BAL options new in reth v2.6.0 (--rpc-cache.cache-computed-bals,--rpc-cache.prewarm-bals) are rejected the same way. To migrate, remove these flags. None of them had any effect on OP chains.op-reth also does not serve the block access list RPC methods
eth_getBlockAccessList,eth_getBlockAccessListByBlockHash,eth_getBlockAccessListByBlockNumber,eth_getBlockAccessListRawanddebug_getRawBlockAccessList. They now return method not found (-32601).Verified with the Rust workspace checks, feature combinations, mirror audit, and the affected CL sync, EL sync, and restart acceptance scenarios using both op-node and kona-node. After 4dcdefc, the op-reth trie and ExEx tests and
just lintwere rerun. The acceptance scenarios were not.Fixes https://github.com/ethereum-optimism/protocol-team/issues/354