Repository navigation
refactor: extract shared virtual memory primitives. - #2468
Merged
liutongxuan merged 9 commits intoOct 13, 2026
Merged
Conversation
liutongxuan
requested review from
DongheJin,
DragonFive,
JimHsiung,
Kang-Meng,
XuZhang99,
xiao-yu-chen,
yingxudeng and
zhang-minchao
as code owners
October 10, 2026 11:39
Codex Review SummaryThis comment shows the latest Codex review activity on this pull request.
ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings. |
|
Important Review skippedToo many files! This PR contains 487 files, which is 187 over the limit of 300. To get a review, reduce the PR to 300 files or fewer by splitting it into smaller PRs or changing its base branch. Usage-priced reviews support at most 300 files. ⚙️ Run configuration
📒 Files selected for processing (487)
You can disable this status message by setting the
Comment |
liutongxuan
changed the title
liutongxuan
force-pushed
the
features/kv_cache_refactor
branch
2 times, most recently
from
October 11, 2026 13:56
b36fca1 to
72baefc
Compare
XuZhang99
reviewed
Oct 11, 2026
Collaborator
XuZhang99
reviewed
Oct 11, 2026
|
Codex Review: Something went wrong. Try again later by commenting “@codex review”. ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
If Codex has suggestions, it will comment; otherwise it will react with 👍. Codex can also answer questions or update the PR. Try commenting "@codex address that feedback". |
liutongxuan
force-pushed
the
features/kv_cache_refactor
branch
from
October 12, 2026 02:55
72baefc to
32218a2
Compare
XuZhang99
reviewed
Oct 12, 2026
liutongxuan
force-pushed
the
features/kv_cache_refactor
branch
2 times, most recently
from
October 12, 2026 12:38
66df8c9 to
d8dc205
Compare
- Synchronize English and Chinese directory maps and architecture diagrams. - Explain injected KV backends, weight ownership, mapping leases and shared RPC compatibility. - Record the remaining Sequence and configuration coupling for subsequent slices.
- Keep worker-local resources in runtime and distributed dispatch in explicit coordinators and RPC classes. - Move KV page queues and preallocation behind an injected mapping backend in kv_cache storage. - Reserve KV and weight wakeup capacity atomically and drain pending mappings before suspension. - Migrate consumers, test targets, and architecture documentation while preserving the RPC wire schema.
- rename the memory RPC registration address to worker_memory_master_node_addr - use the virtual_memory KV layout signature and synchronize docs and tests - keep capability flag splitting for a separate pull request
- Open and close weight data sessions without using the Mooncake data endpoint as a BRPC control endpoint. - Use ScalarType throughout the mapped-memory tensor and KV cache APIs. - Remove the outdated directory listing from AGENTS.md.
liutongxuan
force-pushed
the
features/kv_cache_refactor
branch
from
October 12, 2026 14:38
d8dc205 to
80e01e7
Compare
XuZhang99
reviewed
Oct 13, 2026
liutongxuan
force-pushed
the
features/kv_cache_refactor
branch
from
October 13, 2026 08:04
bb4adc4 to
9ff71fb
Compare
XuZhang99
reviewed
Oct 13, 2026
- Distinguish virtual mappings and physical page budgets across cache, weight, runtime and protocol APIs while preserving protobuf field tags. - Reuse external tensor construction with an explicit device and preserve empty-view storage and legacy caller behavior. - Isolate device tests and move tensor validation death tests to a host-only target whose fixture permits GoogleTest inheritance. - Align sampling tests with separate model and sampling storage accounting. - Clarify KV block token units and consistently name virtual KV offset RPCs, mapped tensor views and shared physical page mappings.
liutongxuan
force-pushed
the
features/kv_cache_refactor
branch
from
October 13, 2026 09:30
9ff71fb to
edbe940
Compare
XuZhang99
approved these changes
Oct 13, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
Related Issues
Change Type
Pull Request Checklist
Thank you for contributing to xLLM. Before requesting review, please make sure the following items are complete.
PR Title and Commit Messages
<type>: <subject>.Pre-commit Checks
pre-commitby runningpip install pre-commitor an equivalent command.pre-commit install.pre-commit run --all-filesand fixed any reported issues.Self Review
.agents/skills/code-review/references/custom-code-style.md, especially code written or assisted by AI.mainbranch.Build and Test Coverage
python setup.py build testhas passed on a CUDA machine.python setup.py build testhas passed on an NPU machine.python setup.py build testhas passed on an MLU machine.Reviewer Notes