Skip to content

fix(spectest): promote gossip attestations before head selection in fork-choice runner - #999

Open
zclawz wants to merge 4 commits into
mainfrom
fix/hive-fork-choice-gossip-attestation-weight
Open

fix(spectest): promote gossip attestations before head selection in fork-choice runner#999
zclawz wants to merge 4 commits into
mainfrom
fix/hive-fork-choice-gossip-attestation-weight

Conversation

@zclawz

@zclawz zclawz commented Jun 15, 2026

Copy link
Copy Markdown
Contributor

Problem

The fork-choice spec test runner calls updateHead() after onAttestation(attestation, false), but gossip attestations are stored in latestNew while updateHead()computeFCHeadUnlocked(true, 0) only reads from latestKnown. Attestation weights were invisible to head computation, causing all weight-dependent fork-choice tests to pick heads by depth/lexicographic order instead of weight.

Root Cause

In fork_choice_runner.zig:

  1. processAttestationStep calls onAttestation(attestation, false) → stores in latestNew
  2. Then calls updateHead() → reads only latestKnown → attestation weight ignored
  3. Same pattern in processGossipAggregatedAttestationStep

Block-included attestations (onAttestation(true) at line 835) go directly to latestKnown and were unaffected.

Fix

Replace updateHead() with acceptNewAttestations() in both attestation processing steps. acceptNewAttestations() promotes latestNewlatestKnown then calls updateHeadUnlocked(), matching the production consensus time-loop path.

Regression Timeline

Bisected via Hive CI artifacts:

  • May 20 (590f4b2): 85/85 fork-choice tests passing ✅
  • May 21 (d89b642): 80/86 — 6 failures appeared ❌

The code bug existed since the fork-choice test driver was introduced (#859), but was latent until the Hive simulator added weight-dependent test fixtures around May 20-21.

Fixes 6 Hive lean-spec-tests-fork-choice failures

  • test_head_selection_by_weight_not_depth
  • test_head_with_deep_fork_split
  • test_back_and_forth_reorg_oscillation
  • test_three_block_deep_reorg
  • test_three_way_fork_competition
  • test_two_block_reorg_progressive_building

Not addressed (pre-existing)

The rpc-compat (5 fails) and reqresp (7 fails) failures are separate, pre-existing issues that have been improving over time (May 15 had 17 reqresp failures vs 7 now).

zclawz and others added 3 commits June 12, 2026 17:57
…ork-choice runner

The fork-choice spec test runner called `updateHead()` after
`onAttestation(attestation, false)`, but gossip attestations are stored
in `latestNew` while `updateHead()` only reads from `latestKnown`.
This caused all weight-dependent fork-choice tests to fail because
attestation weights were invisible to head computation.

Replace `updateHead()` with `acceptNewAttestations()` in both
`processAttestationStep` and `processGossipAggregatedAttestationStep`
to promote new→known before head selection, matching the production
consensus time-loop path.

Fixes 6 Hive lean-spec-tests-fork-choice failures:
- test_head_selection_by_weight_not_depth
- test_head_with_deep_fork_split
- test_back_and_forth_reorg_oscillation
- test_three_block_deep_reorg
- test_three_way_fork_competition
- test_two_block_reorg_progressive_building
…o tracker.latestNew to promote

storeAggregatedPayload only writes to latest_new_aggregated_payloads, not the
per-validator tracker.latestNew. Using acceptNewAttestations() here drained the
new pool immediately on every gossip aggregate step, causing the
test_finalization_prunes_stale_attestation_signatures check to see an empty
latestNewAggregatedTargetSlots instead of the expected all_targets.

processAttestationStep correctly uses acceptNewAttestations() because
onAttestation() does update tracker.latestNew and the weights need to be
visible to head selection. processGossipAggregatedAttestationStep does not
update the tracker, so updateHead() (which reads tracker.latestKnown) is
the right call — the new→known promotion for aggregated payloads happens
via the periodic tick at slot_interval==4.
@zclawz

zclawz commented Jun 15, 2026

Copy link
Copy Markdown
Contributor Author

🔧 CI fix pushed — f7f541f

Failure: test_finalization_prunes_stale_attestation_signatures step #21latestNewAggregatedTargetSlots mismatch

Root cause: processGossipAggregatedAttestationStep was changed to call acceptNewAttestations() instead of updateHead(). But storeAggregatedPayload only writes to latest_new_aggregated_payloads — it does not update the per-validator tracker.latestNew. So acceptNewAttestations() was draining the "new" aggregated payload pool immediately on every gossip aggregate step, leaving it empty by the time the pre-finalization check at step #21 ran (which expected all 5 targets in latestNewAggregatedTargetSlots).

Fix: Revert processGossipAggregatedAttestationStep to use updateHead(). The acceptNewAttestations() call in processAttestationStep is still correct — individual gossip attestations DO update tracker.latestNew and those weights need promotion before head selection. Aggregated gossip payloads are promoted by the periodic tick at slot_interval==4, not by each gossip step.

…load migration

processAttestationStep was calling acceptNewAttestations() to make gossip
votes visible to head selection, but that also migrated latest_new_aggregated_payloads
→ latest_known, which drained the 'new' pool prematurely.

The test_finalization_prunes_stale_attestation_signatures fixture expects the
second batch of gossip aggregated payloads to still be in latest_new at the
pre-finalization check step — calling acceptNewAttestations() at every individual
gossip attestation step emptied the pool too early.

Add ForkChoice.promoteGossipVotes() which only promotes per-validator
latestNew → latestKnown in the attestation tracker (no aggregated-payload
migration), then recomputes head. Use it from processAttestationStep.

Aggregated-payload migration belongs exclusively at the periodic acceptance
tick (slot_interval == 4), matching the production acceptNewAttestations path.
@zclawz

zclawz commented Jun 15, 2026

Copy link
Copy Markdown
Contributor Author

🔧 CI fix pushed (commit 1af41c6)

Root cause: processAttestationStep was calling acceptNewAttestations() to promote gossip votes into latestKnown for head selection. But acceptNewAttestations() also migrates latest_new_aggregated_payloads → latest_known, draining the "new" pool prematurely.

The failing test (test_finalization_prunes_stale_attestation_signatures) submits a second batch of gossip aggregated payloads and then checks at step #21 that they're still in latestNew — but by then all 5 calls to acceptNewAttestations() (one per individual gossip attestation) had already promoted them all to known.

Fix: Added ForkChoice.promoteGossipVotes() — promotes per-validator latestNew → latestKnown in the attestation tracker (matching production intent) without touching latest_new_aggregated_payloads. Used this in processAttestationStep instead of acceptNewAttestations().

Aggregated-payload migration now correctly stays exclusive to the periodic acceptance tick (slot_interval == 4).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant