Skip to content

Latest commit

 

History

History
187 lines (134 loc) · 4.84 KB

File metadata and controls

187 lines (134 loc) · 4.84 KB

VecStore Python Bindings

High-performance vector database with RAG toolkit for Python, powered by Rust.

Status: Python bindings track the 0.1.0 alpha release (December 2025). APIs may change between versions.

Installation

pip install vecstore-rs

With built-in embedding support (sentence-transformers):

pip install vecstore-rs[embeddings]

Note: The package is published as vecstore-rs on PyPI, but imports as vecstore in Python.

Quick Start

from vecstore import VecStore, Query

# Create or open a vector store
store = VecStore.open("./my_db")

# Insert vectors with metadata
store.upsert(
    id="doc1",
    vector=[0.1, 0.2, 0.3, ...],
    metadata={"text": "Hello world", "category": "greeting"}
)

# Query for similar vectors
results = store.query(
    vector=[0.1, 0.2, 0.3, ...],
    k=5
)

for result in results:
    print(f"ID: {result.id}, Score: {result.score}")
    print(f"Metadata: {result.metadata}")

LangChain Integration

VecStore provides native LangChain-compatible classes for seamless integration with LLM applications:

from vecstore import LangChainVectorStore, Document

# Create a LangChain-compatible vector store
store = LangChainVectorStore("./langchain_db")

# Add documents with embeddings (from your embedding model)
store.add_embeddings(
    texts=["Hello world", "Goodbye world"],
    embeddings=[[0.1, 0.2, 0.3], [0.4, 0.5, 0.6]],
    metadatas=[{"source": "doc1"}, {"source": "doc2"}]
)

# Similarity search
results = store.similarity_search_by_vector(
    embedding=[0.1, 0.2, 0.3],
    k=5
)

for doc in results:
    print(f"Content: {doc.page_content}")
    print(f"Metadata: {doc.metadata}")
    print(f"Score: {doc.score}")

Built-in Embeddings

VecStore includes optional built-in embedding support via sentence-transformers. No need to manage embeddings manually:

pip install vecstore-rs[embeddings]
from vecstore import VecStoreWithEmbeddings

# Create store with automatic embedding generation
store = VecStoreWithEmbeddings("./my_db", model_name="all-MiniLM-L6-v2")

# Add texts - embeddings generated automatically
store.add_texts(
    texts=["Hello world", "Machine learning is great", "AI revolution"],
    metadatas=[{"source": "a"}, {"source": "b"}, {"source": "c"}]
)

# Search by text - query embedded automatically
results = store.search("artificial intelligence", k=5)
for doc in results:
    print(f"{doc.document.page_content} (score: {doc.score:.3f})")

Supported Models

Any model from sentence-transformers:

Model Dimensions Speed Quality Use Case
all-MiniLM-L6-v2 (default) 384 Fast Good General purpose
all-mpnet-base-v2 768 Medium High Best quality
multi-qa-MiniLM-L6-cos-v1 384 Fast Good Q&A optimized
paraphrase-multilingual-MiniLM-L12-v2 384 Fast Good Multilingual

With Custom Embedding Model

from vecstore import LangChainVectorStore
# Use with any embedding model (OpenAI, HuggingFace, etc.)
from sentence_transformers import SentenceTransformer

model = SentenceTransformer('all-MiniLM-L6-v2')
store = LangChainVectorStore("./my_rag_db")

# Add documents
texts = ["Document 1 content", "Document 2 content"]
embeddings = model.encode(texts).tolist()
store.add_embeddings(texts=texts, embeddings=embeddings)

# Query
query_embedding = model.encode("search query").tolist()
results = store.similarity_search_by_vector(query_embedding, k=3)

Features

  • Fast: Rust core avoids Python hot loops for distance calculations
  • Built-in Embeddings: Optional sentence-transformers integration
  • Complete RAG Toolkit: Text splitting, reranking, evaluation
  • LangChain Compatible: Native Document and VectorStore classes
  • Operational Features: Persistence, namespaces, server mode
  • Pythonic API: Type hints, familiar patterns
  • Zero Config: Works out of the box

Documentation

See the main repository documentation:

Examples

See the examples/ directory for complete examples:

  • basic_rag.py - Simple RAG workflow
  • fastapi_integration.py - FastAPI REST API
  • evaluation.py - RAG quality measurement
  • production.py - Production deployment

Development

Build Requirements

  • Rust 1.92+ (Edition 2024)
  • Python 3.8+

Building from source:

# Install Rust (if needed)
curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh
rustup update stable  # Ensure Rust 1.92+

# Install maturin
pip install maturin

# Build in development mode
maturin develop --features python

# Run tests
pytest tests/

License

MIT License - see LICENSE file for details