[Core] Add snapshot-pinned full-text search with scalar filters

[JingsongLi]

Purpose

Move the local full-text search capabilities already available in PyPaimon into Rust core. The existing Rust builder cannot express partition/data predicates or provide an owned, snapshot-pinned Scan -> Plan -> Read contract for the Python wrapper.

Brief change log

• Add structured with_query, filters, and owned FullTextScan, FullTextScanPlan, and FullTextRead types. Preserve the existing plain-text SQL builder API.
• Resolve a snapshot once, honor REST/branch/time-travel selection, prune partitions, and preserve the bound read schema across schema-only renames.
• Follow Java full-text/scalar FAST, FULL and DETAIL coverage semantics. Refine inexact scalar candidates only when configured; exclude composite scalar indexes from search pre-filters as Java does.
• Keep complete indexed/raw corpora for BM25, collect Boolean/boost leaf candidates before final Top-K, and honor global-index.thread-num.
• Share exact row-ID filtering with vector searches and expose thin Python bindings plus type stubs. Index search, raw reading, scalar evaluation and ranking remain in core.

Tests

• Full-text builder: 24 passed; vector regression: 61 passed; hybrid search: 12 passed.
• Clippy 1.98.0, all targets for core and Python binding with full-text enabled: passed.
• Rustfmt and minimal-feature core compile: passed.
• Built the local Python wheel and ran 106 PyPaimon tests with native plan/read/write/update/commit gates enabled. All passed; actual counters: 113 plans, 105 reads, 10 writes, 1 native commit, and every update category exercised.
• REST parity covers structured/boosted queries, Top-K, partial coverage, BTree/bitmap filters, candidate refinement, partitions, typed analyzer options, snapshot selection, rename, deletion vectors, text updates, missing indexes and execution errors. Python scores are compared with the existing reader.

API and Format

Additive core and Python APIs. No storage format change or new dependency. Python integration targets REST Data Evolution tables with global row IDs; PK materialized search remains on its existing path.

Documentation

Document the core builder and snapshot/coverage/filtering semantics in the SQL guide. Paired Python PR: apache/paimon#10494 (draft until this change reaches Rust main). Python CI continues to install Rust main.

[leaves12138]

Reviewed head e1069c5 together with apache/paimon#10494, using the paired Java implementation as the semantic reference.

Snapshot pinning, partition/data filtering, scalar FAST/FULL/DETAIL coverage, candidate refinement, composite-index exclusion, full-corpus BM25 scoring, Boolean/boost candidates before final Top-K, and no retry after native execution starts look consistent in the reviewed paths.

Local validation: 4,034 core tests passed, 6 ignored; the freshly built Python extension passed 106 paired PyPaimon tests with all five native gates enabled and every update category exercised. Rustfmt and changed-file Python lint passed. Java comparison was source-level; no Java engine interoperability or production performance benchmark was run.

One new Windows test failure must be addressed before approval; details are inline. This does not ask for an unrelated filesystem implementation rewrite.

[leaves12138]

Re-reviewed head 3badea1 with paired Python head ab44d7142b4d6b739553dadd79514281e37d75d2.

The previous Windows fixture issue is addressed: the rename/snapshot test now uses a unique memory:/ warehouse, eliminating the drive-rooted filesystem listing while retaining the actual schema-rename assertions. The increment changes only that test; production Rust and binding code are unchanged from the previous reviewed head.

Local validation: 4,034 core tests passed, 6 ignored; rustfmt passed. The paired Python native suite passed 106 cases with all five native gates and every update category exercised, using the previously built extension after verifying identical production Rust/binding sources. The updated classic regression selection passed 83 cases.

No additional Rust blocking issue found. Java comparison remains source-level, not a Java engine interoperability or performance run. Hosted CI, including Windows unit tests, is still in progress at review time; this approval is not a claim that all CI checks have completed.

The separate bitmap/refinement mismatch reported on the Python PR is in the classic Python policy; the Rust exactness policy matches the current Java implementation.