Skip to content

proto: backport FTS/TiCI protocols to release-8.5-fts - #1538

Open
AilinKid wants to merge 5 commits into
pingcap:feature/release-8.5-ftsfrom
AilinKid:codex/fts-on-release-8.5-fts
Open

AilinKid wants to merge 5 commits into
pingcap:feature/release-8.5-ftsfrom
AilinKid:codex/fts-on-release-8.5-fts

Conversation

@AilinKid

@AilinKid AilinKid commented Sep 17, 2026

Copy link
Copy Markdown

Summary

Backport the existing FTS/TiCI protocol stack to feature/release-8.5-fts for pingcap/tidb#71280.

Cherry-pick the five previously adapted commits from cp-fts-858-kvproto (47bcf5dc38ad): #1319, #1332, #1356, #1457 and #1432. They add shard metadata, versioned ranges and services, estimate-count RPCs, and MPP shard dispatch. Each commit retains its cherry-pick provenance.

The target starts at ab2787838a97; retain its #1524 resource-usage change. The resulting tree differs from the old FTS branch only by that target change. No backup metadata schema changes.

Validation

  • go test -mod=readonly ./pkg/... passed (generated Go packages compile; most have no tests).
  • Patch integrity and exact source/target delta checks passed.
  • Rust/C++ builds and live TiCI integration were not run.

Summary by CodeRabbit

  • New Features
    • Added version-aware point-range lookups, allowing requests to specify the read timestamp for each range.
    • Added TiCI full-text search shard information to coprocessor and MPP requests, including shard retry details.
    • Added an API for estimating TiCI full-text search result counts.
    • Added a versioned coprocessor service for processing timestamp-aware requests.
    • Versioned lookup requests require point ranges and must not include standard range specifications.

wshwsh12 and others added 5 commits September 17, 2026 10:59
* tici

* address comments

* address comments

* test

(cherry picked from commit d86730c)
* save work

Signed-off-by: xufei <xufeixw@mail.ustc.edu.cn>

* typo

* add rpc

* fix

* proto: model versioned ranges as VersionedKeyRange

* proto: drop reserved range_versions

* proto: use uint64 for read_ts

---------

Signed-off-by: xufei <xufeixw@mail.ustc.edu.cn>
Co-authored-by: xufei <xufeixw@mail.ustc.edu.cn>
(cherry picked from commit fa40b23)
Co-authored-by: cfzjywxk <cfzjywxk@gmail.com>
(cherry picked from commit f4a0f7a)
(cherry picked from commit 78eb8c3)
(cherry picked from commit 47bcf5d)
@ti-chi-bot

ti-chi-bot Bot commented Sep 17, 2026

Copy link
Copy Markdown

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by:
Once this PR has been reviewed and has the lgtm label, please assign cfzjywxk, zhangjinpeng87 for approval. For more information see the Code Review Process.
Please ensure that each of them provides their approval before proceeding.

The full list of commands accepted by this bot can be found here.

Details Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@ti-chi-bot
ti-chi-bot Bot requested a review from tangwz September 17, 2026 03:01
@ti-chi-bot ti-chi-bot Bot added the size/XXL Denotes a PR that changes 1000+ lines, ignoring generated files. label Sep 17, 2026
@coderabbitai

coderabbitai Bot commented Sep 17, 2026

Copy link
Copy Markdown

Review Change StackReview Change Stack

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Advanced

Run ID: 74d720e6-2709-49a1-a3da-a2aa99026fc8

📥 Commits

Reviewing files that changed from the base of the PR and between ab27878 and d7957dc.

⛔ Files ignored due to path filters (4)
  • pkg/coprocessor/coprocessor.pb.go is excluded by !**/*.pb.go
  • pkg/mpp/mpp.pb.go is excluded by !**/*.pb.go
  • pkg/tikvpb/tikvpb.pb.go is excluded by !**/*.pb.go
  • scripts/proto.lock is excluded by !**/*.lock
📒 Files selected for processing (3)
  • proto/coprocessor.proto
  • proto/mpp.proto
  • proto/tikvpb.proto

Included review availability: Your plan provides up to 2 included reviews per hour; 1 remains after this review.


📝 Walkthrough

Walkthrough

The protobuf contracts add versioned point-range lookups, TiCI shard metadata, estimate-count messages, MPP routing data, and two RPC endpoints.

Changes

TiCI protocol extensions

Layer / File(s) Summary
Versioned lookup contracts
proto/coprocessor.proto
Adds VersionedKeyRange and versioned-range fields to Request and StoreBatchTask. The fields require point ranges.
Shard metadata and estimate contracts
proto/coprocessor.proto, proto/mpp.proto
Adds shard metadata, TiCI estimate-count messages, batch retry shard data, and MPP table shard information.
TiCI RPC endpoints
proto/tikvpb.proto
Adds GetEstimateTiCICount to Tikv and adds the VersionedKv.VersionedCoprocessor service. The latter documents the versioned-range request invariant.

Priority: ➖ Normal

Estimated code review effort: 3 (Moderate) | ~20 minutes

Change: Feature

Merge Risk: ⚪ Minimal · up to d7957

The protocol additions have no identified merge-blocking risk.

🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title clearly summarizes the main change: backporting the FTS/TiCI protocol additions to the release-8.5-fts branch.
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check. Docstring coverage is scoped to functions touched by this diff. Analyzed 0 functions across 0…
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Warning

Some tools did not complete. Review the errors below.

🔧 Buf (1.72.0)
proto/coprocessor.proto

fatal: unable to access 'https://github.com/pingcap/kvproto.git/': Failed to connect to github.com:443 over proxy 127.0.0.1 after 0 ms: Could not connect to server
fatal: could not fetch d0f83a0dabdfe96285e9aaf7b0006ff64590e4f6 from promisor remote

proto/mpp.proto

fatal: unable to access 'https://github.com/pingcap/kvproto.git/': Failed to connect to github.com:443 over proxy 127.0.0.1 after 0 ms: Could not connect to server
fatal: could not fetch d0f83a0dabdfe96285e9aaf7b0006ff64590e4f6 from promisor remote

proto/tikvpb.proto

fatal: unable to access 'https://github.com/pingcap/kvproto.git/': Failed to connect to github.com:443 over proxy 127.0.0.1 after 0 ms: Could not connect to server
fatal: could not fetch d0f83a0dabdfe96285e9aaf7b0006ff64590e4f6 from promisor remote


Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

A rabbit carries ranges bright,
With read timestamps tucked in tight.
Shards join the routing trail,
Count estimates cross the rail,
New RPC paths complete the flight.

Comment @coderabbitai help to get the list of available commands.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

size/XXL Denotes a PR that changes 1000+ lines, ignoring generated files.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants