A vector database is a specialized storage system designed to index, store, and query high-dimensional vector embeddings efficiently. Unlike traditional databases that match rows by exact key lookups or SQL predicates, vector databases find data points based on semantic similarity — returning the vectors (and their associated metadata) that are closest to a query vector in high-dimensional space.
Vector databases are the backbone of modern AI applications. Every time a Retrieval-Augmented Generation (RAG) system fetches relevant context to ground an LLM response, it queries a vector database. Every time a recommendation engine finds “similar items,” or a visual search system matches images, the underlying operation is a vector similarity search.
Purpose-built vector databases like Pinecone, Weaviate, Milvus, Qdrant, and Chroma have emerged alongside vector-capable extensions to traditional databases (PostgreSQL with pgvector, Elasticsearch with dense vector fields). Understanding their architecture is essential for building performant AI systems.
Before data enters a vector database, it passes through an embedding model — a neural network that converts text, images, or audio into dense numerical vectors. For example, OpenAI’s text-embedding-3-small produces 1536-dimensional vectors, while Cohere’s embed-v3 outputs 1024 dimensions. Each dimension captures a learned semantic feature, and the position of a vector in this high-dimensional space encodes its meaning.
A vector embedding is a fixed-length array of floating-point numbers (e.g., [0.023, -0.187, 0.445, ...]) that represents the semantic meaning of a piece of content. Semantically similar content produces vectors that are close together in the embedding space, enabling similarity-based retrieval.
Vector databases measure “closeness” using mathematical distance functions. The choice of metric affects both result quality and computational cost:
# Distance metric examples (Python with NumPy)
import numpy as np
a = np.array([0.1, 0.3, 0.5])
b = np.array([0.2, 0.4, 0.4])
# Cosine similarity
cosine_sim = np.dot(a, b) / (np.linalg.norm(a) * np.linalg.norm(b))
# Result: 0.9899...
# Euclidean distance
euclidean = np.linalg.norm(a - b)
# Result: 0.1732...
# Dot product
dot = np.dot(a, b)
# Result: 0.34A vector database’s architecture consists of several interconnected layers:
Stores the raw vector data alongside associated metadata (text, document IDs, timestamps, tags). Vectors are stored as contiguous arrays of 32-bit or 16-bit floats. For a collection of 10 million 1536-dimensional vectors at 32-bit precision, raw storage alone requires ~57 GB. Many databases support scalar quantization (converting float32 to int8) or product quantization (PQ) to compress vectors by 4–32×, trading slight accuracy for dramatic memory savings.
The indexing layer builds data structures that enable sub-linear search — finding nearest neighbors without scanning every vector. Common index types include:
O(N × D) per query, impractical above ~100K vectors.nprobe clusters are searched. Reduces search scope by 10–100× with ~95% recall.Exact nearest neighbor search guarantees finding the true closest vectors but scales linearly with dataset size. Approximate Nearest Neighbor (ANN) algorithms like HNSW and IVF trade a small amount of accuracy (measured as recall@k — the fraction of true top-k neighbors returned) for orders-of-magnitude faster queries. In practice, ANN algorithms achieve 95–99% recall with 100–1000× speedup.
Handles query parsing, vector similarity computation, and result assembly. Advanced features include:
category = "legal" AND date > 2024-01-01). This is called filtered search or pre-filtering.Production vector databases distribute data across multiple nodes for scalability and fault tolerance:
The most common AI use case for vector databases. A user’s question is embedded, the vector database returns the top-k most semantically similar document chunks, and those chunks are injected into the LLM’s context window as grounding information. Pinecone reports that typical RAG queries retrieve 5–20 chunks with p99 latencies under 50ms for collections of 10M+ vectors.
Unlike keyword search, semantic search finds documents by meaning. A query like “How do I fix a leaking faucet?” matches documents about “plumbing repair” and “dripping tap solutions” even without shared keywords. Vector databases power this by returning vectors with high cosine similarity to the query embedding.
User behavior and item characteristics are encoded as vectors. Finding “similar items” or “users who liked similar things” becomes a nearest-neighbor query. Spotify, for instance, uses embedding-based similarity to power its Discover Weekly playlist recommendations.
Vectors representing normal system behavior cluster together. New observations that are far from any cluster (high distance to nearest neighbors) are flagged as anomalies. This is used in fraud detection, network security, and manufacturing quality control.
Key considerations when selecting a vector database for production AI systems: