Skip to content

[NIDX-02B] Drive the native encoder through the segment plugin contract #14074

Description

@hanahmily

Parent: #14007
Blocked by: #14073 (NIDX-02A). An open pull request does not unblock this; 02A must merge first.

Provisional shape. This leaf is the weakest of the NIDX-02 ladder against the workstream's "no layer-shaped tickets" rule: its consumer is the index library's own lifecycle rather than a live Property caller. It is filed separately so the encoder (#14073) and the production cutover (NIDX-02C) each stay reviewable, and so the cutover's dependency-policy question does not block the encoder. If maintainers judge it not vertical, fold it into NIDX-02C and close this as superseded — that is a cheaper outcome than an oversized cutover leaf, and the decision should be made before this becomes the oldest unblocked leaf.

Scope decision before implementation

The provisional scope above remains unresolved: #14007 requires each implementation leaf to activate a live Property behavior, while this issue currently specifies a test-only plugin with no production registration. Maintainers must resolve that discrepancy before implementation proceeds: either fold this work into NIDX-02C as proposed above, or explicitly approve and document a preparatory exception to the parent’s activation requirement, naming its consumer and compatibility evidence. This issue does not itself grant that exception. Do not silently add production registration to satisfy the parent while violating this leaf’s stated boundary.

Boundary

Wrap #14073's encoder in the SegmentPlugin contract so the existing lifecycle manager can drive it: New(results []segment.Document, normCalc func(string, int) float32) (segment.Segment, uint64, error), Load(*segment.Data) (segment.Segment, error), and Merge([]segment.Segment, []*roaring.Bitmap, int) segment.Merger.

The returned value must satisfy the full segment.Segment (14 methods) and segment.Merger (5 methods) contracts, not merely serialize bytes. Every type is reachable through the existing neutral segment alias.

Not registered in this leaf. No WithSegmentPlugin call, no change to inverted.NewStore configuration, no production behavior change. The plugin is constructed and driven only from tests.

Delegation question this leaf must answer, not assume

The cheaper implementation wraps encoded bytes with segment.NewDataBytes and delegates Load to the pinned loader, which would give the full interface for free. That delegation reaches the retired dependency's loader and cannot be written without a new token in tracked source. Do not promise delegation as available. Either a pre-approved neutral bridge exists by the time this leaf starts, or the adapter implements the contracts natively. Resolve this before the first commit and record the answer in the PR description.

Required behavior

  • The lifecycle manager persists a segment produced by New through the normal Segment.WriteTo path, and Load reopens the persisted bytes.
  • Merge produces a segment whose visible documents are the union of its inputs minus the supplied deletion bitmaps.
  • Output remains readable by nativeice.Open and by the pinned compatibility reader, exactly as in [NIDX-02A] Encode Property documents as native ICE v3 segment bytes #14073.

RED / E2E / suites

To be sharpened against 02A's landed API when this becomes the oldest unblocked leaf. The RED anchors on the lifecycle manager driving the plugin — failing on main because no plugin implementation exists — and the e2e on a persist/reopen/merge cycle whose output the merged reader still accepts.

Lexical gate

The lexical non-regression requirement in #14007 applies in full: add zero new case-insensitive bluge tokens and zero matching tracked paths. This includes production code, imports, aliases, tests, fixture/provenance data, strings, comments, scripts, configuration, and generated assets. Existing references may be deleted; the parent’s rule must not be narrowed to selected retired implementation packages.

The neutral segment alias does not exempt its underlying import path. Reusing an existing import in its current file does not authorize adding that import to another file, including a boundary or end-to-end test. The aliased index package is not referenced from this leaf.

Before approving the boundary and tests, identify a concrete integration approach that satisfies both the required interfaces and this lexical constraint. Check the authored boundary, stubs, and tests against the gate before handing them to implementation. If the assignment itself requires forbidden additions, revise the assignment; do not baseline those additions or weaken the gate. If no compliant approach can be established, report the conflicting requirements for a maintainer decision rather than approving an unsatisfiable contract.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    TBDTo be decided later, need more discussion or input.bydb file compatible changedatabaseBanyanDB - SkyWalking native databasefeatureNew feature

    Type

    No type

    Projects

    No projects

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions