fix(index): key rows on (app_id, project_id, id)

[gloryfromca]

Summary

Derived-index rows (LanceDB / Milvus) were merged on id alone, and id is only unique within one (app_id, project_id): a user's episodes, atomic facts, agent cases, foresights, skills and profile written under a second project replaced the first project's rows (markdown was never affected). Every business table now declares PRIMARY_KEY = ("app_id", "project_id", "id") on a backend-neutral mixin; LanceDB merges on it and Milvus keys its collections on the composed __pk column; every single-row read and write goes through primary_key(app_id, project_id, id). A multi-project LanceDB root indexed before this release gets its lost rows back by itself: the first full pass after upgrading (server start, or cascade sync) re-reconciles every indexed markdown file once and records it in .index/.index_reconcile_version; cascade sync --force-all is the manual entry to the same pass.

The first four commits are the behaviour-neutral refactor (declarations onto IndexRecordMeta, record models to infra/persistence/records/, startup check); the rest is the fix. Full write-up, acceptance cases and evidence: .work_context/profile_skill_scope_collision/ (local, not committed).

Area

• Architecture method
• Benchmark
• Use case
• Documentation
• Developer experience
• CI, build, or release

Verification

make lint                      # ruff + import-linter (4 contracts kept) + repo gates + openapi drift
uv run pytest tests/unit       # 2655 passed, 4 skipped
uv run pytest tests/integration# 186 passed, 7 skipped (LanceDB backend)
uv run pytest tests/e2e/test_get_endpoint_e2e.py -k lancedb   # 23 passed
Real-host acceptance (acceptance-cases.md, L/E/U/M tiers, 35/35 cases): two projects, one user, one agent;
  v1.4.1 roots upgraded in place; starting the new server on an old root recovers the 10 overwritten rows
  by itself (log: cascade_index_reconcile_enqueued count=8); interrupted --force-all converges; rebuild;
  failed-row requeue; backfill leaves the other project's vector byte-identical; Milvus tier on a local
  Docker Milvus 2.6: old collection refused at startup, rebuild, then two rows per table.

Checklist

• I kept the change scoped to the relevant area.
• I am opening this from a separate branch, not pushing directly to main.
• I updated docs, examples, or setup notes when behavior changed.
• I added or updated tests when the change affects behavior.
• I did not commit secrets, .env files, dependency folders, or generated output.
• Active relative links in Markdown files resolve.

Notes for Reviewers

• Upgrade: LanceDB roots need nothing — the first full pass reconciles the ledger once (reconcile_index_once, marker-guarded, no lock; a healthy root pays one read pass and no embedding: 1568 entries in under a second locally); Milvus roots must run everos cascade --root <root> rebuild --yes once (the server refuses to start on an old collection and says so). CHANGELOG has both.
• handlers/user_profile.py, search/recall/profile.py and records/user_profile.py are touched here; @juwei.yue's feat/llm-multiround-retrieval edits the same files (profile multi-subject) and will need a rebase on top of this.
• The repository contract deliberately has no add / get_by_id any more: the only write is a merge on the declared key. The chassis LanceRepoBase.add stays for non-business tables and is unreachable from the business layers (import-linter).
• Known follow-ups left out on purpose: Decouple index record models from LanceModel / Vector #476 BTree scalar index on id for business index tables #477 case→skill bridge matches index row ids against md-side source_case_ids #478 cascade status: report duplicate primary keys in the derived index #479 IndexRepository: retire scope-blind read methods (find_by_owner, …) #480; cascade rebuild has no --root alias (sibling commands do).

🤖 Generated with Claude Code