Skip to content

Parallel/chunked artifact hashing in ordvec-manifest verification — deferred perf #286

Description

From the OrdinalDB builder post-mortem (F7) + 2026-07-05 remediation reconciliation: cold-open verification is hash-bound and single-threaded (~290 MB/s single-stream SHA-256 measured by brief-02 at 300k–1M vectors; open ≈ 200× search cost per invocation).

Current state: hashing is already bounded-streaming with EINTR retry (sha256_file_bounded). Artifact-level parallelism (primary ∥ aux) helps little because the primary artifact dominates bundle bytes. The honest win needs chunked-parallel hashing of the primary artifact or an algorithm change (e.g. blake3-class tree hashing) — a format/design decision, not a patch. Deferred from the 0.2.0 hardening wave by decision; the wave ships verify-once composition downstream (hash once per open pattern) which captures the architectural win. A cached verification token was considered and rejected (mtime-keyed tokens are the byte-pin/TOCTOU trade the crate's docs explicitly refuse).

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions