Skip to content

feat: stop history pruning after certain checkpoint - #25656

Merged
alexghr merged 1 commit into
v5-nextfrom
stack/feat-stop-history-pruning-after-certain-checkpoint
Oct 16, 2026
Merged

feat: stop history pruning after certain checkpoint#25656
alexghr merged 1 commit into
v5-nextfrom
stack/feat-stop-history-pruning-after-certain-checkpoint

Conversation

@alexghr

@alexghr alexghr commented Oct 15, 2026

Copy link
Copy Markdown
Contributor

Allow pinning history to a certain checkpoint number.

Related to A-2390

@alexghr alexghr added the port-to-aztec-node PRs need to be ported to the new repo label Oct 15, 2026
@alexghr

alexghr commented Oct 15, 2026

Copy link
Copy Markdown
Contributor Author

This will have to be ported manually to aztec-node.

@alexghr
alexghr force-pushed the stack/feat-stop-history-pruning-after-certain-checkpoint branch from aa50227 to 6962aea Compare October 15, 2026 16:59
@alexghr
alexghr requested a review from IlyasRidhuan as a code owner October 15, 2026 16:59
@alexghr
alexghr force-pushed the stack/feat-stop-history-pruning-after-certain-checkpoint branch from 6962aea to fa395e3 Compare October 15, 2026 17:47
spalladino
spalladino requested changes Oct 15, 2026
View reviewed changes

@spalladino spalladino left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Requesting changes: the new history validation also runs on nodes without a pin, and it changes startup and snapshot-sync behaviour for every v5 node. Details are in the inline comment on snapshot-sync.ts.

Written by Claude

Comment thread yarn-project/node-lib/src/actions/snapshot-sync.test.ts Resolved
Comment thread yarn-project/node-lib/src/actions/snapshot-sync.ts Outdated
Comment on lines +69 to +75
if (worldStateDirectory && (await hasWorldStateDatabase(worldStateDirectory))) {
await validateLocalWorldStateHistory(config, log, genesis);
if (historyStartCheckpoint !== undefined) {
log.info('Using existing world-state database with compatible retained history', { historyStartCheckpoint });
return false;
}
}

@spalladino spalladino Oct 15, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

IIUC what happens here is that if the historyStartCheckpoint is set, then we'll refuse to use a snapshot? I'm worried this could lead to excessively long syncing times. Maybe we could sync anyway, but regenerate world state from archive?

EDIT: I think I understand now. You cannot "backfill" world state entries you already pruned. So the best you can do is get a new archiver snapshot, an old world state one from another snapshot bundle, and then have world state sync the missing entries from the archiver. But that's extra complexity, so we stick with picking a snapshot with the old world state. Is that the case?

@alexghr alexghr Oct 16, 2026

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

no, the scenario is:

  1. we have local state
  2. our config is updated to say "we need to retain full history from this block onwards"
  3. node restarts (since it's an env var change)
  4. on boot we check if our local state is compatible. If it isn't (ie our local db has state from block N but env var says to keep state from block N - 100) we refuse to start
  5. otherwise we print the above log and continue.

Comment thread barretenberg/sol/lib/forge-std

@spalladino spalladino Oct 15, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why the submodule changes?

@alexghr alexghr Oct 16, 2026

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'll double check they are the same commits as v5-next but I hadn't bootstrapped properly before working on this code and committed with deleted submodules. See 6962aea

Comment thread yarn-project/node-lib/src/actions/snapshot-sync.ts
Comment on lines +271 to +276
if (checkpoint) {
maxSnapshotBlock = BlockNumber(checkpoint.startBlock + checkpoint.blockCount - 1);
}
throw new Error(
`Snapshot is at checkpoint ${latestBlock.checkpointNumber}, after requested checkpoint ${historyStartCheckpoint}`,
);

@spalladino spalladino Oct 15, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I find this bit confusing. Why is it a problem for the snapshot to be at a later point in time than the requested history?

EDIT: I think I understand now: we want to sync from a snapshot that has an old archive and world state, so we can sync state and regen world state from there, without pruning, right?

@alexghr alexghr Oct 16, 2026

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

correct

Comment thread yarn-project/node-lib/src/actions/snapshot-sync.ts Resolved
@alexghr
alexghr force-pushed the stack/feat-stop-history-pruning-after-certain-checkpoint branch from fa395e3 to 3c139ba Compare October 16, 2026 09:36
@alexghr
alexghr force-pushed the stack/feat-stop-history-pruning-after-certain-checkpoint branch from 3c139ba to c08b2d1 Compare October 16, 2026 09:41
@alexghr
alexghr merged commit 0d09d31 into v5-next Oct 16, 2026
12 checks passed
@alexghr
alexghr deleted the stack/feat-stop-history-pruning-after-certain-checkpoint branch October 16, 2026 12:29
@alexghr alexghr mentioned this pull request Oct 16, 2026
Merged
@fcarreiro fcarreiro mentioned this pull request Oct 26, 2026
Merged
fcarreiro added a commit that referenced this pull request Oct 26, 2026
@fcarreiro
chore(release): 5.3.0 (#25688)
ddb0adb 
Promotes `v5-next` onto `v5` for the **v5.3.0** release.

The head is a single merge commit with parents `v5` (`1d7578e433`) and
`v5-next` (`4aab531902`). Land it with a merge commit, not a squash, so
`v5-next` stays in the ancestry of `v5`.

### Node.js 24.22.1

Included via #25676. `.nvmrc` reads `v24.22.1`, and
`release-image/Dockerfile.base` installs Node 24.22.1 from the
nodejs.org tarball, sha256-checked per architecture, with `apt upgrade`
in the image and `docker build --pull` for the base. Both files are
byte-identical to `v5-next` on this branch.

The CI build image (`build-images/src/Dockerfile`) and
`scripts/setup-container.sh` pin `nodejs=24.15.0-1nodesource1`. The
published `aztecprotocol/aztec` image takes its Node runtime from
`Dockerfile.base`, not from either of those.

### Carried from `v5`

- #25629 (`PROVER_NODE_DISABLE_FULL_EPOCH_PROVING`) is on `v5` only. It
is not on `v5-next`, so no nightly has run it.
- The v5.2.1 cherry-picks (#25616 via #25617, and #25304) are on both
sides and merge without conflict.

### Manifest

`.release-please-manifest.json` is the only conflict: `5.2.1` on `v5`,
`5.3.0` on `v5-next`. It resolves to `5.3.0`.

### Scope

97 commits (56 non-merge), 2026-08-17 to 2026-10-26.

| Area | Changes |
|---|---|
| toolchain | Node.js 24.22.1 (#25676) |
| `aztec-up` | Pin verdaccio 7.0.1 and publish to the local test
registry with a login token (#25690) |
| `aztec-node` | Stop history pruning after a configured checkpoint
(#25656); archive witnesses at the current tip (#25540) |
| `sequencer` | Publisher instance cleanup (#25490); fee provider
refreshes in the background (#25278) |
| `pxe` | Per-change-set staging store and coordinator hardening
(#25335, #25336, #25337, #25338, #25339, #25367); speculative sync of
predicted contract call trees (#25214) |
| `txe` | Immutables hash in TXE deployments (#25541, `feat(txe)!`) |
| `bb` | Reject mixed-infinity recursive accumulators (#25617, also in
v5.2.1) |
| merge trains | `fairies-v5` (#25256, #25329, #25425, #25471, #25483,
#25535, #25539, #25583); `spartan-v5` (#25284, #25325, #25382, #25421) |
| compat | 5.2.0 legacy compat artifact (#25250) |

**One breaking change**: #25541 (`feat(txe)!`).

### Protocol

`v5..v5-next` touches nothing under
`noir-projects/noir-protocol-circuits`, `l1-contracts/src`,
`barretenberg`, `noir` or `yarn-project/constants`.
`pinned-build.tar.gz`, `pinned-protocol-contracts.tar.gz` and
`pinned-standard-contracts.tar.gz` are the same blobs as on `v5`. The
`noir-projects` changes are in `aztec-nr`, `handshake_registry_contract`
(standard, pinned), `schnorr_initializerless_account_contract` and
`tx_effect_oracle_test_contract`.

### Validation

`v5.3.0-nightly.20261022` through `v5.3.0-nightly.20261026` all point at
`31aa3e90f2`, and their CI3 runs are green. The only `v5-next` change
after `31aa3e90f2` is #25690, which touches `aztec-up` CI scripts only.
No v5.3.0 nightly is deployed on testnet:
`v5.testnet.rpc.aztec-labs.com` and
`canonical.testnet.rpc.aztec-labs.com` both report `nodeVersion=5.0.0`.

Not run on this head: the from-source protocol-constants gate, and a
testnet soak.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@spalladino spalladino spalladino approved these changes
@IlyasRidhuan IlyasRidhuan Awaiting requested review from IlyasRidhuan

Assignees

No one assigned

Labels

port-to-aztec-node PRs need to be ported to the new repo

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@alexghr @spalladino