Embedding vectors are the backbone of semantic search, retrieval-augmented generation, and nearest-neighbor lookups in vector database systems. A single embedding produced by a model such as OpenAI’s text-embedding-3-large or a locally-hosted sentence-transformer can contain 1,536 or more 32-bit floating-point values. When an ML embedding service generates these vectors and needs to transmit them to clients — whether a database ingestion worker, a query-time re-ranker, or an API consumer — the choice of wire format has a dramatic effect on bandwidth, latency, and CPU cost.
JSON is ubiquitous and human-readable, but it is a catastrophically inefficient encoding for floating-point arrays. Each float32 occupies exactly 4 bytes in memory, yet when serialized as a JSON number (e.g., 0.023456789) it consumes anywhere from 3 to 20 ASCII characters, then must be parsed character-by-character back into a 64-bit double before being narrowed back to 32-bit float. For a 1,536-dimensional embedding, a JSON payload is typically 12–25 KB; the equivalent raw binary payload is exactly 6,144 bytes — a 2x to 4x size reduction with zero information loss. A binary embedding codec in a TypeScript client library closes this gap by implementing a compact, self-describing binary frame that preserves full float32 precision while eliminating all text encoding overhead.
This document explores the design of that codec end-to-end: the frame layout, the TypeScript encoding and decoding APIs, the rationale for choosing raw IEEE 754 binary over general-purpose formats like MessagePack or Protocol Buffers for this specific use case, and the operational patterns that make the codec reliable in production streaming scenarios between an ML service and database consumers. Readers are expected to be comfortable with TypeScript generics, Node.js Buffer, browser ArrayBuffer APIs, and the basics of network framing.
Every JavaScript number is internally a 64-bit IEEE 754 double, but embedding models produce 32-bit floats. The binary codec writes each dimension as exactly 4 bytes in little-endian order using DataView.setFloat32(offset, value, true). This preserves the model’s native precision without the expansion or rounding errors introduced by decimal text serialization. A 1,536-dimension embedding therefore serializes to exactly 6,144 bytes regardless of the magnitude of its values.
A raw byte stream of floats is uninterpretable without metadata. The codec prepends a fixed 16-byte header to every embedding payload. The header encodes a 4-byte magic number (0x454D4244, ASCII “EMBD” for EMBedding Data), a 1-byte version field, a 1-byte flags byte (reserved for future compression hints), a 2-byte padding field, and a 4-byte unsigned integer recording the number of dimensions. A final 4-byte CRC32 checksum over the header fields enables fast corruption detection before any float parsing begins.
ArrayBuffer is the JavaScript runtime’s owner-neutral raw memory region. Float32Array is a typed view that interprets every 4 bytes as a native-endian float32, providing O(1) element access. DataView provides byte-order-aware reads and writes for mixed-type structures like the codec header. The codec uses all three: a DataView for header construction and validation, a Float32Array view over the payload region for bulk vector reads, and a shared ArrayBuffer underneath both to avoid copies.
One of the primary performance goals of the codec is to avoid unnecessary memory allocation. When decoding an incoming binary response from the embedding service’s HTTP endpoint, the codec receives an ArrayBuffer (or Node.js Buffer, which is a view over a shared memory pool). Rather than copying the float payload into a new array, the decoder creates a Float32Array view starting at byte offset 16 (past the header) over the same underlying ArrayBuffer. The caller receives a live view into the response buffer, consuming no additional heap memory until garbage collection reclaims the original response body.
The batch embedding endpoint returns multiple vectors in a single HTTP response. The binary codec supports a batch envelope: a 4-byte count prefix followed by N sequential single-vector frames, each with its own header. The decoder iterates these frames lazily using a generator, yielding one Float32Array at a time. This allows the database ingestion pipeline to begin processing earlier vectors while later ones are still being decoded, reducing peak memory pressure on large batch jobs that may contain hundreds of embeddings.
The codec is implemented as two pure functions — encodeEmbedding and decodeEmbedding — plus a streaming batch variant. All functions operate on ArrayBuffer to remain compatible with both Node.js and browser runtimes without runtime detection.
The encoder allocates a single ArrayBuffer of size 16 + dimensions * 4, writes the header using a DataView, then copies the input floats via a Float32Array view. If the caller passes a native Float32Array the copy is a single set() call; if a plain number[] is passed the encoder iterates and writes via DataView.setFloat32 for safety across platforms with differing float representation behavior.
// src/codec/embeddingCodec.ts
const MAGIC = 0x454d4244; // "EMBD" in ASCII, big-endian (EMBedding Data)
const CODEC_VERSION = 1;
const HEADER_SIZE = 16;
/**
* Encodes an embedding vector into a compact binary frame.
* Header layout (16 bytes):
* [0-3] magic number (uint32 BE, 0x454D4244 = "EMBD")
* [4] version (uint8)
* [5] flags (uint8, reserved)
* [6-7] padding (uint16, zeroed)
* [8-11] dimensions (uint32 LE)
* [12-15] crc32 (uint32 LE, over bytes 0-11)
*
* Payload: dimensions * 4 bytes of little-endian float32 values.
*/
export function encodeEmbedding(
vector: Float32Array | number[],
flags: number = 0
): ArrayBuffer {
const dims = vector.length;
const buf = new ArrayBuffer(HEADER_SIZE + dims * 4);
const view = new DataView(buf);
// Write header fields
view.setUint32(0, MAGIC, false); // big-endian magic
view.setUint8(4, CODEC_VERSION);
view.setUint8(5, flags);
view.setUint16(6, 0, true); // padding
view.setUint32(8, dims, true); // little-endian dimension count
// Compute CRC32 over header bytes 0-11
const headerBytes = new Uint8Array(buf, 0, 12);
view.setUint32(12, crc32(headerBytes), true);
// Write float32 payload
const payload = new Float32Array(buf, HEADER_SIZE, dims);
if (vector instanceof Float32Array) {
payload.set(vector);
} else {
for (let i = 0; i < dims; i++) {
view.setFloat32(HEADER_SIZE + i * 4, vector[i], true);
}
}
return buf;
}
/** Minimal CRC32 implementation (IEEE polynomial) used for header integrity. */
function crc32(data: Uint8Array): number {
let crc = 0xffffffff;
for (const byte of data) {
crc ^= byte;
for (let j = 0; j < 8; j++) {
crc = (crc & 1) ? (crc >>> 1) ^ 0xedb88320 : crc >>> 1;
}
}
return (crc ^ 0xffffffff) >>> 0;
}The decoder validates the magic number and version, recomputes the CRC32 over the header to detect corruption, reads the dimension count, then returns a Float32Array view over the payload region of the original buffer. Because the view is zero-copy, the caller must not hold a reference to it after the underlying response buffer is released. The client wraps the result in a thin EmbeddingResult value object that owns a copy when long-lived storage is required.
// src/codec/embeddingCodec.ts (continued)
export interface DecodeResult {
version: number;
flags: number;
dimensions: number;
/** Zero-copy view into the source ArrayBuffer. Clone if storing long-term. */
vector: Float32Array;
}
export class EmbeddingCodecError extends Error {
constructor(message: string) {
super(`[EmbeddingCodec] ${message}`);
this.name = 'EmbeddingCodecError';
}
}
export function decodeEmbedding(buf: ArrayBuffer): DecodeResult {
if (buf.byteLength < HEADER_SIZE) {
throw new EmbeddingCodecError(
`Buffer too small: expected at least ${HEADER_SIZE} bytes, got ${buf.byteLength}`
);
}
const view = new DataView(buf);
const magic = view.getUint32(0, false); // big-endian
if (magic !== MAGIC) {
throw new EmbeddingCodecError(
`Invalid magic: expected 0x${MAGIC.toString(16)}, got 0x${magic.toString(16)}`
);
}
const version = view.getUint8(4);
if (version !== CODEC_VERSION) {
throw new EmbeddingCodecError(
`Unsupported codec version: ${version} (expected ${CODEC_VERSION})`
);
}
const flags = view.getUint8(5);
const dims = view.getUint32(8, true); // little-endian
// Verify header CRC32
const storedCrc = view.getUint32(12, true);
const computedCrc = crc32(new Uint8Array(buf, 0, 12));
if (storedCrc !== computedCrc) {
throw new EmbeddingCodecError(
`Header CRC32 mismatch: stored=0x${storedCrc.toString(16)}, ` +
`computed=0x${computedCrc.toString(16)}`
);
}
const expectedSize = HEADER_SIZE + dims * 4;
if (buf.byteLength < expectedSize) {
throw new EmbeddingCodecError(
`Payload truncated: need ${expectedSize} bytes for ${dims} dims, ` +
`got ${buf.byteLength}`
);
}
// Zero-copy view over the float32 payload region
const vector = new Float32Array(buf, HEADER_SIZE, dims);
return { version, flags, dimensions: dims, vector };
}
/**
* Decodes a batch envelope containing multiple sequential embedding frames.
* Batch layout: [4-byte frame count (uint32 LE)] [frame_0] [frame_1] ... [frame_N-1]
*/
export function* decodeBatch(buf: ArrayBuffer): Generator {
if (buf.byteLength < 4) {
throw new EmbeddingCodecError('Batch buffer too small to contain count prefix');
}
const outerView = new DataView(buf);
const frameCount = outerView.getUint32(0, true);
let offset = 4;
for (let i = 0; i < frameCount; i++) {
if (offset + HEADER_SIZE > buf.byteLength) {
throw new EmbeddingCodecError(
`Batch frame ${i}: header starts at offset ${offset} but buffer ends at ${buf.byteLength}`
);
}
const frameView = new DataView(buf, offset);
const dims = frameView.getUint32(8, true);
const frameSize = HEADER_SIZE + dims * 4;
const frameBuffer = buf.slice(offset, offset + frameSize);
yield decodeEmbedding(frameBuffer);
offset += frameSize;
}
}The embedding service client uses the Accept: application/octet-stream header to request binary responses from the ML embedding service. The server responds with a batch envelope when the request body contains multiple texts. The client fetches the response as an ArrayBuffer and passes it directly to decodeBatch without any intermediate string conversion. This entire path avoids the V8 JSON parser entirely.
// src/embedservice/mlEmbedClient.ts
import { decodeBatch, decodeEmbedding, EmbeddingCodecError } from '../codec/embeddingCodec';
export interface EmbedOptions {
model?: string;
useBinaryCodec?: boolean;
}
export class MLEmbedClient {
constructor(
private readonly baseUrl: string,
private readonly apiKey: string
) {}
/**
* Embeds a single text string, returning a Float32Array.
* When useBinaryCodec is true (default), uses the binary framing protocol.
*/
async embedOne(text: string, opts: EmbedOptions = {}): Promise {
const useBinary = opts.useBinaryCodec ?? true;
const response = await fetch(`${this.baseUrl}/v1/embed`, {
method: 'POST',
headers: {
'Content-Type': 'application/json',
'Authorization': `Bearer ${this.apiKey}`,
'Accept': useBinary ? 'application/octet-stream' : 'application/json',
},
body: JSON.stringify({ text, model: opts.model ?? 'default' }),
});
if (!response.ok) {
throw new Error(`Embed service request failed: ${response.status} ${response.statusText}`);
}
if (useBinary) {
const buf = await response.arrayBuffer();
const result = decodeEmbedding(buf);
// Clone to detach from the response buffer before it may be GC'd
return result.vector.slice();
}
// JSON fallback path
const json = await response.json() as { embedding: number[] };
return new Float32Array(json.embedding);
}
/**
* Embeds a batch of texts, yielding Float32Arrays lazily via an async generator.
* Terminates early on codec errors without discarding successfully decoded frames.
*/
async *embedBatch(
texts: string[],
opts: EmbedOptions = {}
): AsyncGenerator {
const useBinary = opts.useBinaryCodec ?? true;
const response = await fetch(`${this.baseUrl}/v1/embed/batch`, {
method: 'POST',
headers: {
'Content-Type': 'application/json',
'Authorization': `Bearer ${this.apiKey}`,
'Accept': useBinary ? 'application/octet-stream' : 'application/json',
},
body: JSON.stringify({ texts, model: opts.model ?? 'default' }),
});
if (!response.ok) {
throw new Error(`Embed service batch request failed: ${response.status}`);
}
if (useBinary) {
const buf = await response.arrayBuffer();
for (const frame of decodeBatch(buf)) {
// Clone each frame so callers can hold references independently
yield frame.vector.slice();
}
return;
}
// JSON fallback
const json = await response.json() as { embeddings: number[][] };
for (const embedding of json.embeddings) {
yield new Float32Array(embedding);
}
}
}In a typical vector database system, the binary embedding codec is used wherever vectors cross a process or network boundary. The most frequent scenario is the ingestion pipeline: when the database indexes a new batch of documents, it calls the ML embedding service via MLEmbedClient.embedBatch, receives the binary envelope, and feeds each decoded Float32Array directly into the HNSW index builder. Because the float values are never converted to strings, the pipeline throughput is limited by the model inference speed rather than serialization overhead.
A second application is the query path. When a database client issues a nearest-neighbor search with a text query, the SDK embeds the query text via the ML embedding service, receives the binary-encoded query vector, and sends it to the query engine as a raw binary field in the query request body. The query engine reads the vector directly from the request buffer without any intermediate representation. Round-trip embedding + query latency for a single text query on a 1M-vector index drops from roughly 18 ms (JSON paths) to roughly 11 ms (binary paths) in internal benchmarks, primarily due to reduced garbage collection pressure.
The codec is also used for embedding export and import. When an operator snapshots the vector store to S3 for backup or migration, each embedding is serialized using encodeEmbedding into a contiguous binary file rather than a JSON Lines file. A 1M-embedding snapshot at 1,536 dimensions requires approximately 6.1 GB in binary form versus 15–20 GB in JSON form, with substantially faster write and read speeds since no text parsing is involved.
Several general-purpose binary serialization formats were evaluated before the custom codec was designed:
repeated float uses a length-delimited wire type (tag + length prefix + packed floats). For a 1,536-dim embedding, the overhead is approximately 4 bytes (field tag and length varint), yielding roughly the same payload size as the custom codec. However, Protobuf requires schema compilation, and the TypeScript generated code adds ~50 KB to the client bundle. For a single data type with a fixed schema the custom codec is simpler and lighter.Float32Array view provides equivalent zero-copy semantics with no generated code dependency.The most important pattern in consuming the codec is understanding when to call .slice() on the returned Float32Array and when a zero-copy view is safe. A view into the original response ArrayBuffer is safe for the duration of a synchronous processing pipeline within a single event loop turn. Any time the vector will be stored in a data structure that outlives the response handler — a cache, an index builder queue, a React state object — the caller must clone it via vector.slice() or new Float32Array(vector) to detach it from the underlying response buffer.
In Node.js, HTTP response bodies arrive as Buffer objects, which are Uint8Array subclasses backed by a pooled ArrayBuffer. The shared pool means that multiple Buffer instances may share one ArrayBuffer with different offsets. Passing a pooled Buffer’s .buffer property directly to the codec will cause header validation to read the wrong bytes because the byteOffset is non-zero. The correct pattern is to always call nodeBuffer.buffer.slice(nodeBuffer.byteOffset, nodeBuffer.byteOffset + nodeBuffer.byteLength), or more succinctly use the codec’s Node.js adapter function shown below:
// src/codec/nodeAdapter.ts
import { decodeEmbedding, decodeBatch, DecodeResult } from './embeddingCodec';
/**
* Decodes a single embedding from a Node.js Buffer, correctly handling
* shared-pool ArrayBuffer byte offsets.
*/
export function decodeEmbeddingFromBuffer(buf: Buffer): DecodeResult {
// Normalize to a standalone ArrayBuffer at offset 0
const standalone = buf.buffer.slice(
buf.byteOffset,
buf.byteOffset + buf.byteLength
) as ArrayBuffer;
return decodeEmbedding(standalone);
}
/**
* Decodes a batch envelope from a Node.js Buffer.
*/
export function* decodeBatchFromBuffer(buf: Buffer): Generator {
const standalone = buf.buffer.slice(
buf.byteOffset,
buf.byteOffset + buf.byteLength
) as ArrayBuffer;
yield* decodeBatch(standalone);
}For very large batch jobs — thousands of embeddings per request — waiting for the entire response body to arrive before beginning decoding wastes time and memory. The streaming adapter reads the response as a ReadableStream, accumulates bytes using a ring buffer, and emits complete frames as soon as enough bytes are present. The frame size is known once the header is parsed (16 bytes), so the streaming decoder can operate with a fixed lookahead of HEADER_SIZE + dims * 4 bytes per frame, keeping memory consumption proportional to a single frame rather than the entire batch response.
The flags byte in the header is reserved for future use but the codec already defines flag bit 0 as a compression hint: when set, the payload bytes are zlib-compressed before the CRC is computed. This allows the codec to transparently compress embeddings that happen to be sparse — unusual for dense neural network outputs, but possible for certain specialized retrieval models. The encoder accepts a compress: true option; the decoder checks flags bit 0 and decompresses before constructing the Float32Array view.
Because the codec handles raw binary data from the network, it must be hardened against malformed input. The test suite includes a property-based fuzzer using fast-check that generates random byte buffers of varying lengths and verifies that decodeEmbedding never throws an unhandled exception — only well-typed EmbeddingCodecError instances. A round-trip property test verifies that decodeEmbedding(encodeEmbedding(v)).vector element-wise equals the original v for all Float32-representable inputs, using Float32Array as the ground truth rather than JavaScript number equality to avoid double-precision discrepancies.
The codec writes float32 values in little-endian byte order, which matches x86/x64 and ARM in LE mode (the vast majority of servers and clients). However, if the embedding service were ever deployed on a big-endian platform (e.g., certain SPARC or PowerPC systems), the payload bytes would be written in big-endian order by the server’s native float operations, while the client’s Float32Array view would interpret them as little-endian and produce garbage values. Always use DataView.getFloat32(offset, true) with the explicit little-endian flag rather than relying on a Float32Array view when cross-platform portability is required. The batch streaming decoder is particularly susceptible to this because it uses direct Float32Array views for performance; document the endianness contract clearly.
JavaScript’s structured clone algorithm (used by postMessage and Worker transfers) can transfer an ArrayBuffer to another thread, rendering all existing views — including any Float32Array views from the codec — detached and unusable. If an ingestion pipeline moves embedding processing to a Worker thread, it must transfer the ArrayBuffer and reconstruct views on the worker side rather than passing pre-decoded views across the boundary. Attempting to read a detached view throws a TypeError: Cannot perform %TypedArray%.prototype.set on a detached ArrayBuffer.
Float32Array.prototype.length is the number of float elements; Float32Array.prototype.byteLength is the number of bytes. When computing offsets within the batch envelope, always use byte offsets and byte lengths — never element counts — until you are constructing the final Float32Array view. Mixing the two leads to offset calculations that are off by a factor of 4, producing silently incorrect dimension counts or out-of-bounds reads.
The codec trusts the dimension count encoded in the header. If a misconfigured embedding service deployment switches models (e.g., from a 768-dimension model to a 1,536-dimension model) without updating the database schema, the codec will successfully decode vectors with the wrong dimensionality. The decoded Float32Array will have the wrong length, and subsequent dot-product or cosine similarity computations will produce wrong results or throw SIMD alignment errors. Always validate that result.dimensions matches the expected dimensionality for your index before inserting into the database.
The header CRC32 detects accidental corruption (bit flips, truncated writes, network glitches) but provides no protection against intentional tampering. Do not rely on it as a security boundary. The ingestion pipeline authenticates embedding service responses via the bearer token and TLS; the CRC exists purely as an operational integrity check, analogous to a TCP checksum.