Single Instruction, Multiple Data (SIMD) is a class of CPU instructions that apply the same operation to multiple data elements simultaneously. Modern processors provide SIMD registers that are 128 bits (SSE), 256 bits (AVX2), or 512 bits (AVX-512) wide, allowing a single instruction to process 4, 8, or 16 floating-point values at once. For vector databases and search engines that compute millions of distance calculations per query, SIMD is not an optimization—it is a requirement for competitive performance.
Zig provides first-class SIMD support through its @Vector type, which maps directly to hardware SIMD registers. Unlike C intrinsics (_mm256_add_ps) that are architecture-specific and unreadable, Zig’s vector operations use natural operators (+, *, <) that the compiler lowers to the best available SIMD instructions for the target architecture. This makes SIMD code portable across x86 (SSE/AVX) and ARM (NEON/SVE) without architecture-specific branches.
In a typical vector search engine, SIMD-accelerated vector operations power the core distance computations: Euclidean distance, cosine similarity, dot product, and Hamming distance for binary vectors. These operations also underpin quantized vector support, where 32-bit floats are compressed to 8-bit integers and distance computations use integer SIMD lanes with popcount for binary quantization.
A SIMD register holds multiple “lanes” of data. A 256-bit AVX2 register can hold 8 lanes of f32, 4 lanes of f64, 32 lanes of u8, or 16 lanes of u16. Operations execute on all lanes in parallel: adding two 8-lane f32 vectors performs 8 floating-point additions in a single instruction cycle. The throughput gain is proportional to the number of lanes, minus overhead for loading data into registers and extracting results.
Vertical operations apply independently to each lane: a + b adds lane 0 of a to lane 0 of b, lane 1 to lane 1, etc. Horizontal operations combine lanes within a single register: summing all 8 lanes of an f32 vector to produce a scalar result. Vertical operations map perfectly to SIMD; horizontal operations (like computing the final sum in a dot product) require additional shuffle and add instructions, which are slower.
Vector quantization compresses high-dimensional float vectors into lower-precision representations. Scalar quantization maps each f32 to a u8 (256 levels), allowing 4x more vectors per cache line. Binary quantization maps each dimension to a single bit. Distance between binary vectors is computed using popcount (population count—counting set bits in an XOR result), which modern CPUs execute in a single instruction.
@Vector(N, T) creates a fixed-size SIMD vector of N elements of type T. Arithmetic operators (+, *, -) work element-wise. The compiler maps these to the best available hardware instructions (SSE, AVX2, NEON) based on the target architecture. No intrinsics or platform-specific code needed.
SIMD loads and stores are fastest when data is aligned to the register width (16-byte for SSE, 32-byte for AVX2). Misaligned access may work but incurs a performance penalty or fault on some architectures. In Zig, use @alignCast and aligned allocators to ensure vector data meets alignment requirements.
The Hamming distance between two binary vectors is popcount(a XOR b)—the number of bit positions where they differ. Modern CPUs provide hardware POPCNT instructions. In Zig, @popCount on a @Vector of integers compiles to SIMD popcount operations, processing hundreds of dimensions per instruction cycle.
Euclidean distance between two vectors is sqrt(sum((a[i] - b[i])^2)). The SIMD version processes multiple dimensions per iteration, accumulating squared differences in a SIMD accumulator and reducing to a scalar at the end:
const VEC_SIZE = 8; // 8 x f32 = 256 bits (AVX2)
pub fn euclideanDistance(a: []const f32, b: []const f32) f32 {
std.debug.assert(a.len == b.len);
var sum_vec: @Vector(VEC_SIZE, f32) = @splat(0.0);
const n = a.len;
const aligned = n - (n % VEC_SIZE);
// SIMD loop: process VEC_SIZE elements per iteration
var i: usize = 0;
while (i < aligned) : (i += VEC_SIZE) {
const va: @Vector(VEC_SIZE, f32) = a[i..][0..VEC_SIZE].*;
const vb: @Vector(VEC_SIZE, f32) = b[i..][0..VEC_SIZE].*;
const diff = va - vb;
sum_vec += diff * diff; // fused multiply-add on supported hardware
}
// Horizontal reduction: sum all lanes
var sum: f32 = @reduce(.Add, sum_vec);
// Scalar tail: handle remaining elements
while (i < n) : (i += 1) {
const diff = a[i] - b[i];
sum += diff * diff;
}
return @sqrt(sum);
}Cosine similarity is dot(a, b) / (norm(a) * norm(b)). When vectors are pre-normalized (common in embedding spaces), cosine similarity reduces to just the dot product:
pub fn dotProduct(a: []const f32, b: []const f32) f32 {
std.debug.assert(a.len == b.len);
var dot_vec: @Vector(VEC_SIZE, f32) = @splat(0.0);
const n = a.len;
const aligned = n - (n % VEC_SIZE);
var i: usize = 0;
while (i < aligned) : (i += VEC_SIZE) {
const va: @Vector(VEC_SIZE, f32) = a[i..][0..VEC_SIZE].*;
const vb: @Vector(VEC_SIZE, f32) = b[i..][0..VEC_SIZE].*;
dot_vec += va * vb;
}
var dot: f32 = @reduce(.Add, dot_vec);
while (i < n) : (i += 1) {
dot += a[i] * b[i];
}
return dot;
}For binary-quantized vectors, each dimension is a single bit packed into u64 words. Hamming distance uses XOR and popcount:
pub fn hammingDistance(a: []const u64, b: []const u64) u32 {
std.debug.assert(a.len == b.len);
const VEC_LEN = 4; // 4 x u64 = 256 bits
var count_vec: @Vector(VEC_LEN, u64) = @splat(0);
const n = a.len;
const aligned = n - (n % VEC_LEN);
var i: usize = 0;
while (i < aligned) : (i += VEC_LEN) {
const va: @Vector(VEC_LEN, u64) = a[i..][0..VEC_LEN].*;
const vb: @Vector(VEC_LEN, u64) = b[i..][0..VEC_LEN].*;
const xor_result = va ^ vb;
count_vec += @popCount(xor_result);
}
var count: u32 = @intCast(@reduce(.Add, count_vec));
while (i < n) : (i += 1) {
count += @popCount(a[i] ^ b[i]);
}
return count;
}@reduce(.Add, vec) sums all lanes of a SIMD vector into a scalar. Other reduction operations include .Min, .Max, .And, .Or, and .Xor. Reductions are the bridge between the SIMD world (parallel lanes) and the scalar world (single values). Minimize reductions in inner loops— accumulate in SIMD registers and reduce once at the end.
The HNSW (Hierarchical Navigable Small World) graph index performs distance computations on every edge traversal during search. A single query may evaluate thousands of distance calculations. SIMD acceleration reduces the per-distance cost from microseconds to tens of nanoseconds, making real-time approximate nearest neighbor search feasible for million-scale vector collections.
Product quantization divides vectors into subspaces and quantizes each subspace independently. At query time, a distance lookup table is precomputed for each subspace, and the total distance is the sum of table lookups. SIMD accelerates both the table precomputation (distance from query subvector to each centroid) and the table-lookup-and-sum step, using gather instructions to load non-contiguous table entries into SIMD registers.
When scoring a candidate list against a query vector, a typical search engine processes candidates in batches. Each batch loads multiple candidate vectors into SIMD registers, computes distances against the query, and writes results to a score buffer. This amortizes the cost of loading the query vector and maximizes instruction-level parallelism by interleaving loads and arithmetic.
For example, scalar quantization can compress f32 vectors to u8 using min/max normalization per dimension. The quantization and dequantization loops use SIMD: a vector of f32 values is scaled, clamped to [0, 255], and truncated to u8 in a single SIMD pipeline. Distance computation on quantized vectors uses u8 SIMD lanes (32 per AVX2 register), achieving 4x throughput over f32 operations.
The fundamental SIMD coding pattern is “strip mining”: process data in chunks of the SIMD width, then handle the remaining elements with a scalar tail. This pattern appears in every SIMD distance function:
@splat(0.0)@Vector(N, T) from slices@reduceDifferent CPUs support different SIMD widths. A robust implementation dispatches to the widest available instruction set at runtime. In Zig, this can be done with comptime feature detection or runtime CPU feature queries:
pub fn selectDistanceFn(comptime T: type) DistanceFn {
const features = std.Target.x86.featureSet;
if (comptime std.Target.x86.featureSetHas(features, .avx512f)) {
return euclidean_avx512;
} else if (comptime std.Target.x86.featureSetHas(features, .avx2)) {
return euclidean_avx2;
} else {
return euclidean_sse2; // baseline x86-64 always has SSE2
}
}
// Build with: zig build -Dcpu=native
// This enables all features supported by the build machineWhen scanning through a large array of vectors, memory latency dominates over compute. Software prefetch instructions (@prefetch in Zig) hint the CPU to load the next vector into cache before it is needed:
pub fn batchDistance(query: []const f32, vectors: []const []const f32,
scores: []f32) void {
for (vectors, 0..) |vec, i| {
// Prefetch next vector while computing current distance
if (i + 1 < vectors.len) {
@prefetch(vectors[i + 1].ptr, .{
.rw = .read,
.locality = 1, // L2 cache hint
});
}
scores[i] = euclideanDistance(query, vec);
}
}Building with -Dcpu=native enables all SIMD instruction sets supported by the build machine. For deployment, either build with a known baseline (-Dcpu=x86_64_v3 for AVX2) or use runtime dispatch to select the optimal code path. Shipping AVX-512 code to a machine without AVX-512 causes an illegal instruction fault.
if statements that break the pipeline.n % VEC_SIZE remaining elements. Off-by-one errors here silently produce incorrect distance results.u8 * u8 products (range 0–65025) across hundreds of dimensions overflows u16. Use u32 accumulators for quantized distance sums, or accumulate in smaller chunks and widen periodically.@Vector usage guarantees SIMD codegen regardless of optimization level.