feat(metrics): add gossip arrival timing metrics

[MegaRedHand]

🗒️ Description

Every consensus message has an interval it was due in. A block belongs at the start of its own slot, an attestation one interval later, an aggregate at the aggregation interval. Nothing measured how far off a message actually landed, so a slow network looked the same as aggregators that were slow to prove.

Six metrics cover it. Three histograms carry the absolute distance from the boundary, and three counters label each arrival by position: before, inside, or after the interval it was due in. Absolute distance keeps an early arrival and a late one of the same size in a single bucket, and the position label is what tells them apart.

Metric	Type	Anchor
lean_gossip_block_arrival_delay_seconds	Histogram	interval 0 of the block's own slot
lean_gossip_attestation_arrival_delay_seconds	Histogram	interval 1 of the attestation's data slot
lean_gossip_aggregation_arrival_delay_seconds	Histogram	most recent aggregation boundary at or before the arrival
lean_gossip_block_arrival_total	Counter	position=before,inside,after
lean_gossip_attestation_arrival_total	Counter	position=before,inside,after
lean_gossip_aggregation_arrival_total	Counter	position=inside,after

Aggregates anchor to the latest boundary rather than their own data slot. An aggregate catching up on an earlier group carries a data slot several slots back, and anchoring there would report a multi-slot delay that is not a health problem. The boundary anchor also bounds the value to one slot and rules out before, so that counter exports two series rather than three. It wraps rather than clamps, so a message that beat its boundary reads as nearly a slot late instead of as early; the helper documents that limitation.

Only messages that crossed the network are sampled, all three kinds. A proposer's own block and a validator's own votes come back through the same handlers with no peer id (see publish_and_process_block in node.py), and stamping those would sample local build latency as if it were network timing. Aggregates this node produced are left out for the same reason: their timing is proving cost, which lean_pq_sig_aggregated_signatures_building_time_seconds already measures directly.

A message's slot is attacker-controlled until the store validates it, and Slot is a Uint64. One block claiming slot 2**64-1 would push a single observation of roughly 7e19 seconds into the histogram and pin _sum for the life of the process, so arrivals further than MAX_MEASURABLE_SLOT_DISTANCE from the clock are dropped rather than measured. The aggregate metric takes no slot and needs no such guard.

SlotClock._milliseconds_since_genesis becomes public: the arrival helpers need millisecond resolution, which the slot and interval accessors have already rounded away.

Names, types, and buckets match the leanMetrics registry entry, which adds these six rows and a "Gossip arrival timing" dashboard row. EthLambda implements all six today.

Validation

Checked against a live 2-node devnet (test XMSS scheme, 8 validators split 4/4, real QUIC gossip) rather than unit tests, matching how the rest of node/metrics is covered. For every family on both nodes, the counter totals matched the histogram counts exactly, and each delta recomputed independently from the log timestamps landed inside the window that second-resolution logs allow.

The anchor difference is visible in the data: aggregation delay stayed bounded under one slot (mean ~1.7–2.1s) while block and attestation delays ran to several seconds, since this client lags and those two anchor to their own slot. before never fired on any family.

One thing the run turned up, out of scope here and left alone: cli/run.py subscribes to the block and attestation-subnet topics but never to the aggregation topic, so a node publishes aggregates that no peer receives (Empty mesh ... subscribed=0 every heartbeat). Until that is fixed, lean_gossip_aggregation_* stays at zero on a real node. I subscribed locally to exercise the metric; that patch is not part of this PR. Happy to open a separate one if it is not already known.

🔗 Related Issues or PRs

N/A

✅ Checklist

• Ran local quality checks to avoid unnecessary CI fails:

  just check

• Considered adding appropriate tests for the changes.
• Considered updating the online docs in the ./docs/ directory.