Chroma Vector Database Integration

Overview

Chroma is an open-source embedding database designed for building AI applications that need fast, filterable similarity search. Unlike general-purpose databases that bolt on vector support as an afterthought, Chroma is purpose-built around the embedding workflow: store documents with their vector representations and arbitrary metadata, then query by semantic similarity with optional metadata filters—all through a simple API that feels native to Python and Go applications.

In a typical AI application stack, the search platform uses Chroma as its primary vector store via langchaingo abstractions. This integration layer provides a uniform interface for storing, indexing, and querying embeddings regardless of whether the underlying store is Chroma, Pinecone, or an in-memory index. The langchaingo vector store interface wraps Chroma’s collection API, handling serialization of metadata filters, pagination of large upsert batches, and automatic retry on transient failures.

Understanding how Chroma works under the hood—its HNSW indexing, metadata filtering pipeline, and persistence model—is essential for tuning query performance and making informed decisions about when Chroma is the right choice versus alternatives like pgvector or Pinecone.

Core Concepts

How It Works

The integration follows a layered architecture: application code calls the langchaingo VectorStore interface, which delegates to a Chroma-specific adapter that communicates with the Chroma server over its REST or gRPC API. Documents flow through an embedding function (registered separately) before being upserted into a collection.

Initializing the Chroma Store

package search

import (
    "context"
    "log"

    "github.com/tmc/langchaingo/embeddings"
    "github.com/tmc/langchaingo/vectorstores/chroma"
)

// NewChromaStore creates a langchaingo-compatible vector store backed by Chroma.
func NewChromaStore(ctx context.Context, chromaURL string, collectionName string, embedder embeddings.Embedder) (*chroma.Store, error) {
    store, err := chroma.New(
        chroma.WithChromaURL(chromaURL),
        chroma.WithNameSpace(collectionName),
        chroma.WithEmbedder(embedder),
        chroma.WithDistanceFunction("cosine"),
    )
    if err != nil {
        return nil, fmt.Errorf("creating chroma store: %w", err)
    }
    log.Printf("Connected to Chroma collection %q at %s", collectionName, chromaURL)
    return &store, nil
}

Adding Documents with Metadata

package search

import (
    "context"
    "fmt"

    "github.com/tmc/langchaingo/schema"
    "github.com/tmc/langchaingo/vectorstores"
)

// IndexDocuments embeds and stores documents in the Chroma collection.
func IndexDocuments(ctx context.Context, store vectorstores.VectorStore, docs []schema.Document) error {
    // langchaingo handles batching internally based on the embedder's limits
    ids, err := store.AddDocuments(ctx, docs)
    if err != nil {
        return fmt.Errorf("adding documents to chroma: %w", err)
    }
    fmt.Printf("Indexed %d documents, IDs: %v\n", len(ids), ids[:min(3, len(ids))])
    return nil
}

// PrepareDocument creates a langchaingo Document with metadata for Chroma storage.
func PrepareDocument(content string, source string, docType string, tenantID string) schema.Document {
    return schema.Document{
        PageContent: content,
        Metadata: map[string]any{
            "source":    source,
            "doc_type":  docType,
            "tenant_id": tenantID,
            "indexed_at": time.Now().UTC().Format(time.RFC3339),
        },
    }
}

Querying with Metadata Filters

package search

import (
    "context"
    "fmt"

    "github.com/tmc/langchaingo/vectorstores"
)

// SemanticSearch queries the Chroma collection with optional metadata filtering.
func SemanticSearch(ctx context.Context, store vectorstores.VectorStore, query string, tenantID string, topK int) ([]schema.Document, error) {
    // Build filter: restrict results to the requesting tenant
    filter := map[string]any{
        "tenant_id": map[string]any{
            "$eq": tenantID,
        },
    }

    results, err := store.SimilaritySearch(ctx, query, topK,
        vectorstores.WithFilters(filter),
        vectorstores.WithScoreThreshold(0.7),
    )
    if err != nil {
        return nil, fmt.Errorf("chroma similarity search: %w", err)
    }

    fmt.Printf("Found %d results for query %q (tenant: %s)\n", len(results), query, tenantID)
    return results, nil
}

Practical Applications

The search platform in a production AI application stack uses the Chroma integration for its core semantic search pipeline. When a user submits a natural-language query, the service embeds the query using the same model that produced the stored embeddings, then calls SimilaritySearch on the tenant’s Chroma collection. Results are ranked by cosine similarity and filtered by metadata (document type, recency, access level) before being returned to the frontend.

During development and testing, the team runs Chroma in-memory mode, which avoids the need for a running Chroma server. Integration tests create ephemeral collections, index a known corpus, run queries, and verify that expected documents appear in the results with appropriate similarity scores. This pattern catches embedding model regressions and filter logic bugs before they reach production.

For multi-tenant deployments, each tenant gets a dedicated Chroma collection. This provides strong data isolation without the operational overhead of running separate Chroma instances. The collection-per-tenant model also allows per-tenant tuning of HNSW parameters based on corpus size and query patterns.

Implementation Patterns

Batch Upsert with Progress Tracking

package search

import (
    "context"
    "fmt"

    "github.com/tmc/langchaingo/schema"
    "github.com/tmc/langchaingo/vectorstores"
)

const upsertBatchSize = 100

// BatchIndex indexes a large corpus in chunks, reporting progress.
func BatchIndex(ctx context.Context, store vectorstores.VectorStore, docs []schema.Document) error {
    total := len(docs)
    for i := 0; i < total; i += upsertBatchSize {
        end := min(i+upsertBatchSize, total)
        batch := docs[i:end]

        if _, err := store.AddDocuments(ctx, batch); err != nil {
            return fmt.Errorf("batch %d-%d failed: %w", i, end, err)
        }
        fmt.Printf("Indexed %d/%d documents (%.0f%%)\n", end, total, float64(end)/float64(total)*100)
    }
    return nil
}

Collection Health Check

package search

import (
    "context"
    "net/http"
    "fmt"
    "io"
)

// HealthCheck verifies the Chroma server is reachable and the collection exists.
func HealthCheck(ctx context.Context, chromaURL string) error {
    resp, err := http.Get(chromaURL + "/api/v1/heartbeat")
    if err != nil {
        return fmt.Errorf("chroma unreachable: %w", err)
    }
    defer resp.Body.Close()

    if resp.StatusCode != http.StatusOK {
        body, _ := io.ReadAll(resp.Body)
        return fmt.Errorf("chroma health check failed (status %d): %s", resp.StatusCode, body)
    }
    return nil
}

Common Pitfalls

• Mismatched embedding dimensions: If you change the embedding model without recreating the Chroma collection, new vectors will have different dimensions than existing ones, causing silent search quality degradation or outright errors. Always recreate or migrate collections when switching embedding models.
• Over-relying on metadata filters for access control: Chroma metadata filters are post-retrieval—the HNSW search runs first, then filters are applied. With highly selective filters and large topK values, you may get fewer results than expected. For strict access control, use separate collections per access boundary rather than metadata filters alone.
• Ignoring HNSW tuning: The default HNSW parameters work well for small collections (under 100K vectors) but degrade on larger ones. Monitor recall by comparing HNSW results against brute-force search on a sample, and adjust M and ef_search accordingly.
• Not handling Chroma server restarts: Chroma’s persistent mode writes index data to disk, but in-memory collections are lost on restart. Ensure your deployment uses persistent storage and that the langchaingo client handles reconnection gracefully.
• Embedding the query with a different model: The query embedding must come from the same model and configuration (dimensions, normalization) as the stored document embeddings. Using a different model or version produces vectors in a different space, making similarity scores meaningless.
• Large metadata values: Storing large text blobs in metadata (e.g., full document text) bloats the index and slows metadata filter evaluation. Store large content externally and keep metadata lean—IDs, timestamps, categories, and short labels.