Vector search finds the nearest neighbors to a query vector in a high-dimensional space. The definition of “nearest” depends entirely on the distance metric used. Choosing the wrong metric can make a perfectly good index return irrelevant results—not because the index is broken, but because the geometric notion of “closeness” doesn’t match the semantic similarity the embedding model was trained to capture.
The three dominant distance metrics in production vector search are L2 Squared Euclidean distance, Inner Product (dot product), and Cosine Similarity. Each defines a different geometry over the vector space. L2 measures absolute spatial distance, inner product measures directional alignment scaled by magnitude, and cosine similarity measures pure directional alignment regardless of magnitude. The choice depends on the embedding model: models trained with contrastive loss typically produce normalized vectors suited for cosine similarity, while models trained with regression objectives may produce vectors where magnitude carries meaning, favoring inner product.
In production vector databases, vector search typically supports all three metrics as configurable options on each vector index. The distance computation is the innermost loop of both brute-force and HNSW index queries, so these implementations are heavily optimized with SIMD intrinsics for both x86 (AVX2/AVX-512) and ARM (NEON) architectures.
The sum of squared differences between corresponding vector components: d(a,b) = Σ(a_i - b_i)². Lower values mean more similar. The squared variant is preferred over true Euclidean distance (which adds a square root) because the square root is monotonic and doesn’t change the ranking of nearest neighbors.
The sum of element-wise products: ip(a,b) = Σ(a_i × b_i). Higher values mean more similar. Unlike L2, inner product is sensitive to vector magnitude—a longer vector can achieve a higher dot product simply by being larger, not by being more aligned. This is useful when magnitude encodes relevance (e.g., document importance).
The inner product of unit-normalized vectors: cos(a,b) = ip(a,b) / (||a|| × ||b||). Values range from -1 (opposite) to 1 (identical direction). Cosine similarity ignores magnitude entirely, comparing only the angle between vectors. When vectors are pre-normalized to unit length, cosine similarity reduces to inner product, and L2 distance becomes a monotonic function of cosine similarity.
The distance metric is not just a query-time parameter—it must be fixed when the index is built. HNSW graphs, IVF clusters, and quantization codebooks all encode the metric into their structure. Using an index built with L2 to answer cosine queries (or vice versa) produces wrong results without any error.
When all vectors are normalized to unit length (||v|| = 1), the three metrics become equivalent for ranking purposes. Specifically: L2²(a,b) = 2 - 2×cos(a,b) and cos(a,b) = ip(a,b). This means that for normalized vectors, you can use whichever metric has the fastest implementation, since they will all produce identical rankings. Most modern embedding models (sentence-transformers, OpenAI embeddings, Cohere) output normalized vectors, making this equivalence practically useful.
Some embedding architectures intentionally encode information in vector magnitude. For example, a retrieval model might produce longer vectors for more “confident” embeddings. In these cases, inner product should be used instead of cosine similarity, because normalizing the vectors would discard the magnitude signal. ColBERT-style late-interaction models and some learned sparse representations also produce non-normalized vectors where magnitude is meaningful.
The scalar implementations are straightforward loops. These serve as reference implementations and fallbacks for platforms without SIMD support:
// Zig implementations of the three distance metrics
const std = @import("std");
/// L2 Squared Euclidean distance between two vectors.
pub fn l2Squared(a: []const f32, b: []const f32) f32 {
std.debug.assert(a.len == b.len);
var sum: f32 = 0;
for (a, b) |ai, bi| {
const diff = ai - bi;
sum += diff * diff;
}
return sum;
}
/// Inner product (dot product) of two vectors.
/// Returns negative for use as a distance (higher IP = lower distance).
pub fn innerProduct(a: []const f32, b: []const f32) f32 {
std.debug.assert(a.len == b.len);
var sum: f32 = 0;
for (a, b) |ai, bi| {
sum += ai * bi;
}
return -sum; // negate so that "nearest" = highest IP
}
/// Cosine distance = 1 - cosine_similarity.
pub fn cosineDistance(a: []const f32, b: []const f32) f32 {
std.debug.assert(a.len == b.len);
var dot: f32 = 0;
var norm_a: f32 = 0;
var norm_b: f32 = 0;
for (a, b) |ai, bi| {
dot += ai * bi;
norm_a += ai * ai;
norm_b += bi * bi;
}
const denom = @sqrt(norm_a) * @sqrt(norm_b);
if (denom == 0) return 1.0; // zero vectors are maximally dissimilar
return 1.0 - dot / denom;
}The distance function is called billions of times during index construction and millions of times per query. SIMD (Single Instruction, Multiple Data) vectorization processes 8 float32 values per instruction on AVX2, or 16 on AVX-512. The key optimization is loop unrolling combined with SIMD accumulation:
// Go implementation with manual loop unrolling for L2 distance
package vector
import "math"
// L2SquaredUnrolled computes L2 squared distance with 8x unrolling.
// The compiler may auto-vectorize this with SIMD instructions.
func L2SquaredUnrolled(a, b []float32) float32 {
n := len(a)
var sum0, sum1, sum2, sum3 float32
var sum4, sum5, sum6, sum7 float32
i := 0
for ; i+8 <= n; i += 8 {
d0 := a[i] - b[i]
d1 := a[i+1] - b[i+1]
d2 := a[i+2] - b[i+2]
d3 := a[i+3] - b[i+3]
d4 := a[i+4] - b[i+4]
d5 := a[i+5] - b[i+5]
d6 := a[i+6] - b[i+6]
d7 := a[i+7] - b[i+7]
sum0 += d0 * d0
sum1 += d1 * d1
sum2 += d2 * d2
sum3 += d3 * d3
sum4 += d4 * d4
sum5 += d5 * d5
sum6 += d6 * d6
sum7 += d7 * d7
}
total := sum0 + sum1 + sum2 + sum3 + sum4 + sum5 + sum6 + sum7
for ; i < n; i++ {
d := a[i] - b[i]
total += d * d
}
return total
}
// CosineDistance computes 1 - cosine_similarity.
func CosineDistance(a, b []float32) float32 {
var dot, normA, normB float32
for i := range a {
dot += a[i] * b[i]
normA += a[i] * a[i]
normB += b[i] * b[i]
}
denom := float32(math.Sqrt(float64(normA))) * float32(math.Sqrt(float64(normB)))
if denom == 0 {
return 1.0
}
return 1.0 - dot/denom
}Product quantization (PQ) and scalar quantization (SQ) compress vectors from float32 to uint8 or smaller, reducing memory by 4–32x. Distance computation on quantized vectors uses lookup tables: for PQ, each sub-vector’s distance contribution is precomputed in a table indexed by centroid ID. The total distance is the sum of table lookups—much faster than floating-point arithmetic. The accuracy loss from quantization depends on the metric: L2 is relatively robust to quantization noise, while inner product on non-normalized vectors is more sensitive because quantization errors in magnitude compound.
// Go: Asymmetric distance computation with product quantization
package vector
// PQDistanceTable precomputes distances from a query vector to all centroids
// for each sub-vector, enabling fast approximate distance computation.
type PQDistanceTable struct {
NumSubvectors int
NumCentroids int // typically 256 (uint8 codes)
Table [][]float32 // [subvector_idx][centroid_idx] -> partial distance
}
// BuildTable precomputes the distance table for a query vector.
func BuildPQDistanceTable(query []float32, centroids [][][]float32) *PQDistanceTable {
numSub := len(centroids)
subDim := len(query) / numSub
table := make([][]float32, numSub)
for s := 0; s < numSub; s++ {
table[s] = make([]float32, len(centroids[s]))
qSub := query[s*subDim : (s+1)*subDim]
for c, centroid := range centroids[s] {
table[s][c] = L2SquaredUnrolled(qSub, centroid)
}
}
return &PQDistanceTable{
NumSubvectors: numSub,
NumCentroids: len(centroids[0]),
Table: table,
}
}
// Distance computes approximate distance using precomputed lookup table.
func (t *PQDistanceTable) Distance(codes []uint8) float32 {
var dist float32
for s, code := range codes {
dist += t.Table[s][code]
}
return dist
}When creating a vector index in a production system, the user specifies the distance metric alongside the dimension and index type. The metric is stored in the index metadata and checked at query time to ensure consistency. Attempting to query an L2 index with cosine similarity should return an error rather than silently producing wrong results.
The most common use case is semantic search using embeddings from models like sentence-transformers. These models output normalized 768-dimensional vectors. Because the vectors are normalized, a well-optimized vector index uses inner product (equivalent to cosine for unit vectors) as it avoids the per-query normalization check and enables the fastest SIMD path.
Image embeddings from models like CLIP produce 512-dimensional vectors that may not be normalized. For image search where visual similarity is the goal (not just directional alignment), L2 distance captures both directional and magnitude differences. This is important when the embedding model encodes “image quality” or “relevance confidence” in vector magnitude.
In a hybrid search system, combining BM25 text scores with vector similarity scores. The vector distance must be converted to a score in the same range as BM25 for meaningful combination. For cosine similarity, the value is already in [0,1]. For L2, a common conversion is score = 1 / (1 + distance). The choice of metric affects how naturally scores combine with BM25.
To support multiple metrics without branching in the hot loop, the distance function is resolved once at index creation and stored as a function pointer. All index operations (insert, search, merge) use this pointer without checking which metric is active:
// Zig: Distance metric as a comptime-resolved function pointer
const DistanceFn = *const fn (a: []const f32, b: []const f32) f32;
const Metric = enum {
l2_squared,
inner_product,
cosine,
pub fn distanceFn(self: Metric) DistanceFn {
return switch (self) {
.l2_squared => l2Squared,
.inner_product => innerProduct,
.cosine => cosineDistance,
};
}
};
const VectorIndex = struct {
distFn: DistanceFn,
dimension: u32,
vectors: [][]f32,
pub fn init(metric: Metric, dimension: u32) VectorIndex {
return .{
.distFn = metric.distanceFn(),
.dimension = dimension,
.vectors = &.{},
};
}
pub fn search(self: *const VectorIndex, query: []const f32, k: usize) []SearchResult {
// distFn is called here without any metric branching
_ = self.distFn(query, self.vectors[0]);
// ... HNSW or brute-force search logic
}
};If the index uses cosine similarity, normalizing vectors at ingest time converts all future distance computations to simple inner products. This avoids computing norms at query time and enables using the faster inner product SIMD path. The normalization is a one-time O(d) cost per vector at insert versus O(d) per comparison at query time.
For brute-force search over a segment, computing distances one vector at a time underutilizes SIMD and cache bandwidth. Batching computes distances from the query to 4–8 candidate vectors simultaneously, interleaving their memory accesses. This saturates the SIMD pipeline and reduces loop overhead.
During kNN search, once the k-th nearest distance is known, any candidate whose partial distance (computed over the first few dimensions) already exceeds this bound can be skipped. For L2, partial sums are monotonically increasing, so early termination is straightforward. For inner product, this is harder because partial sums can increase or decrease, requiring tighter bounds from quantization or other pre-filtering.
The most common and most harmful mistake is using L2 distance with embeddings trained for cosine similarity (or vice versa). The embedding model’s training objective determines which metric preserves its notion of similarity. Always check the model card or documentation for the recommended distance metric.
Most nearest-neighbor libraries define “nearest” as “smallest distance.” For inner product, higher is more similar, so the distance must be negated: distance = -ip(a,b). Forgetting this negation inverts the results—the most dissimilar vectors are returned as “nearest.”
When an index has multiple segments built at different times, all segments must use the same metric. A bug that allows one segment to be built with L2 and another with cosine will produce correct per-segment results but meaningless merged results, since the distance values from different segments are incomparable.
Summing hundreds of float32 products accumulates rounding error. For 1024-dimensional vectors, the relative error can reach 0.1%, which occasionally changes nearest-neighbor rankings. Using a float64 accumulator for the final sum (or Kahan compensated summation) eliminates this issue with minimal performance cost.
Scalar quantization (float32 → uint8) introduces uniform noise. For L2 distance on normalized vectors, this noise is well-behaved. For inner product on vectors with varying magnitudes, quantization can distort distances significantly because magnitude information is lost. Always benchmark recall with your actual data before deploying quantized indexes.