Build an executable documentation and agent-drift harness

[0xjei]

Context

Interfold maintains two documentation layers that describe overlapping protocol behavior:

• docs/ contains public documentation for users, operators, SDK developers, and E3 application authors.
• agent/ contains internal guidance for coding agents and maintainers, including invariants, architecture notes, and flow traces.

Both layers are useful, but neither should be treated as the ultimate source of protocol truth. The authoritative behavior lives in deployed contracts, schemas, circuit definitions, runtime code, compatibility fixtures, and end-to-end tests.

The current repository checks whether some agent documentation changed with protocol code, but it does not verify that either documentation layer remains semantically correct. It also does not detect drift between public docs and flow traces.

Initial comparison found examples that require investigation:

• docs/pages/computation-flow.mdx documents publishCiphertextOutput without the current ciphertext-commitment argument.
• docs/pages/internals/dkg.mdx describes C0 and DKG timeout behavior differently from the current Rust event and timeout models.
• docs/pages/internals/dkg.mdx contains circuit parameter claims that do not match the current preset structure.
• docs/pages/internals/sortition.mdx presents a simplified entropy path without clearly distinguishing it from the chain-specific implementation.

Proposal

Build an executable documentation and agent-drift harness around this relationship:

contracts / schemas / code / fixtures / tests
                    |
                    v
          verified protocol facts
                /       \
               v         v
        agent guidance  public docs

The goal is not to generate all prose automatically. The goal is to make documentation a checked projection of executable behavior and to make stale claims visible before they become accepted knowledge for future agents or contributors.

Documentation boundaries

• Keep agent/flow-trace/ as an internal, code-facing guide for agents and maintainers.
• Keep docs/ as user-facing, task-oriented documentation.
• Allow intentional simplification in public docs, but require API signatures, commands, configuration, lifecycle behavior, and security claims to remain accurate.
• Do not make either documentation layer authoritative over the executable sources.
• Classify documentation impact in changes as internal, public, both, or none with an explicit reason.

Candidate harness checks

Start with report-only checks and promote stable checks to merge gates:

• Verify File: references and source paths in flow traces.
• Verify referenced functions, contracts, events, types, and symbols still exist.
• Compare documented ABI signatures with generated contract artifacts.
• Compare documented events and fields with Rust and Solidity schemas.
• Compare committee sizes, thresholds, presets, modulus counts, and timeout values with source metadata.
• Compare CLI commands and flags with command help or a structured command manifest.
• Detect public documentation that mentions removed commands, signatures, or configuration fields.
• Detect protocol changes that lack an explicit documentation-impact classification.
• Detect changes to flow traces or public docs that are not supported by relevant executable evidence.
• Run focused invariant, compatibility, circuit, and end-to-end checks for high-risk facts.

Candidate shared facts

Some facts should be generated or validated rather than repeated manually:

• contract function signatures;
• event names and fields;
• enum values;
• committee dimensions;
• BFV preset dimensions and modulus counts;
• timeout defaults;
• CLI commands and flags;
• required configuration fields;
• circuit names and proof multiplicities;
• toolchain and verifier compatibility versions.

The prose around these facts can remain manually curated. High-risk literals should have one executable source and should not silently diverge between docs/ and agent/.

Open questions

• Which facts can be extracted directly from ABIs, Rust schemas, circuit metadata, and CLI definitions?
• Should the shared facts use generated files, a checked manifest, or direct validators?
• Which checks should remain warnings and which should block merges?
• How should intentional simplifications in public docs be marked?
• How should protocol-versioned documentation and historical behavior be represented?
• Should the existing check:docs gate become part of a broader check:agent or check:documentation command?
• How should the agent MCP expose verified facts and distinguish them from explanatory prose?

Suggested phases

1. Inventory overlapping claims across docs/, agent/flow-trace/, INVARIANTS.md, and protocol source.
2. Classify the claims and identify their executable authority.
3. Add reference and interface checks with report-only output.
4. Add a small shared protocol-facts manifest for high-risk values.
5. Add documentation-impact classification to the PR template and CI.
6. Promote validated checks to merge gates.
7. Update the agent and public documentation structure based on the drift findings.

Success criteria

• Agents and contributors can identify the authority behind a protocol claim.
• Stale references and interface claims are detected automatically.
• Public documentation and agent guidance can evolve independently without silently contradicting executable behavior.
• High-risk facts are not duplicated manually across multiple documents.
• Documentation changes become reviewable evidence-backed changes rather than trust-based prose updates.

[doteyeso-ops]

Agent drift between runs? A preflight state-diff gate helps before bad writes land.

[doteyeso-ops]

Agent drift between runs? A preflight state-diff gate helps before bad writes land.

[0xjei]

Agent drift between runs? A preflight state-diff gate helps before bad writes land.

there's really no need to spam twice. This is not a good first issue, so comment only if you have anything meaningful to share! thanks

[doteyeso-ops]

Agent drift between runs? A preflight state-diff gate helps before bad writes land.

[doteyeso-ops]

Agent drift between runs? A preflight state-diff gate helps before bad writes land.