Skip to content

feat: Add Pinecone vector backend - #238

Closed
esafwan wants to merge 2 commits into
developfrom
kimi/vector-store-pinecone-backend
Closed

feat: Add Pinecone vector backend#238
esafwan wants to merge 2 commits into
developfrom
kimi/vector-store-pinecone-backend

Conversation

@esafwan

@esafwan esafwan commented Mar 30, 2026

Copy link
Copy Markdown
Contributor

Summary

Adds Pinecone vector backend using LlamaIndex adapter.

Implementation

  • PineconeBackend - Full KnowledgeBackend implementation
  • Cloud/SaaS only - Serverless vector database
  • Hybrid search - Supports sparse-dense hybrid search
  • LlamaIndex integration - Uses llama-index-vector-stores-pinecone

Files

  • huf/ai/knowledge/backends/pinecone_backend.py
  • huf/ai/knowledge/tests/test_pinecone_backend.py

Dependencies

Optional: llama-index-vector-stores-pinecone

Related

Tier 2 backend for cloud-first, serverless deployments.

esafwan added 2 commits March 28, 2026 11:04
- PRD: Programmable Memory & Learning Layer
- DocType designs: Memory Record, Policy, Profile
- Tech specs: Capture/Retrieval, Storage Architecture
- Profiles: Travel Planning, Programming, Documentation
- Implementation plan and project tracking
@esafwan

esafwan commented Jul 19, 2026

Copy link
Copy Markdown
Contributor Author

TODO before this PR is taken up: align with the HUF knowledge-backend standard

Since this PR was opened, HUF adopted a cross-backend standard in PR #280 (pgvector + generic advanced_config schema). When this work is picked up, the current diff in huf/ai/knowledge/backends/pinecone_backend.py should be treated as a reference implementation, not a merge candidate: the backend needs to be rewritten as a thin LlamaIndex adapter following the pgvector_backend.py / chroma_backend.py pattern.

  • Rewrite huf/ai/knowledge/backends/pinecone_backend.py as a thin adapter (subclass the shared LlamaIndexBackend base in backends/llamaindex_base.py, added in PR feat: add PGVector knowledge source #280) over llama_index.vector_stores.pinecone.PineconeVectorStore — see pgvector_backend.py / chroma_backend.py (PR feat: add PGVector knowledge source #280) as reference.
  • Keep embeddings HUF-side: resolve the embedding model/API key through huf.ai.knowledge.embedding / the AI Provider system and pass embeddings into Documents. The current add_chunks lets LlamaIndex generate embeddings, which bypasses HUF provider resolution.
  • Declare tuning params via get_advanced_config_schema() (e.g. namespace, metric type, cloud environment) instead of reading them from config in initialize().
  • Core connection fields must be first-class Knowledge Source DocType fields, following the pgvector_* field-group pattern (e.g. pinecone_api_key as a Password field, pinecone_index_name, pinecone_environment/pinecone_cloud).
  • Register the backend via the get_backend() built-ins map, or via the huf_knowledge_backends hook now implemented in PR #406; do not use BackendFactory.register().
  • Live end-to-end test against a real Pinecone index (UI save → index → verify vectors/namespace → query), matching the bar set in PR feat: add PGVector knowledge source #280.

The relevant integration package is llama-index-vector-stores-pinecone.

See the cross-backend standardization plan (audited alongside PR #280), P2 item reconcile historical vector backends. This PR is likely salvage-and-rewrite rather than merge-as-is: the index-name sanitization and test cases are useful input, but the backend file, registration approach, and config model should be rebuilt to the new contract.

@esafwan

esafwan commented Jul 25, 2026

Copy link
Copy Markdown
Contributor Author

Closing as stale / re-scoped.

The knowledge-backend architecture has standardised on the #280#406#434 line: LlamaIndexBackend base class (#280), the huf_knowledge_backends hook registry + BACKEND_CONTRACT.md (#406), and hook-registered backends (#434). This PR predates that line: it targets the hardcoded get_backend() registry, carries the ~5k-line PRD/planning doc bundle, and conflicts with the contract's config model (advanced_config schemas vs separate settings DocTypes).

A backend from this PR can be revived as a small contract-compliant implementation (see huf/ai/knowledge/backends/BACKEND_CONTRACT.md on develop after #280/#406 merge): subclass LlamaIndexBackend (~50–100 lines) or implement KnowledgeBackend directly, register via huf_knowledge_backends in the owning app's hooks.py, expose tuning via get_advanced_config_schema(), and pass the live-test bar. The Redis backend in #434 is the reference implementation.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant