HTTP/1.1 suffers from head-of-line blocking: a browser that needs ten resources from the same origin must either open multiple TCP connections or wait for each response to finish before sending the next request. HTTP/2 solves this by multiplexing many logical streams over a single TCP connection, allowing requests and responses to interleave at the frame level. Combined with HPACK header compression—which shrinks repetitive headers by up to 90%—HTTP/2 dramatically reduces latency and bandwidth consumption for modern web applications.
As an example of a low-level implementation, one can implement HTTP/2 from scratch in Zig, building the binary framing layer, stream state machines, HPACK encoder/decoder, and flow control windows directly on top of a TLS connection. This low-level implementation reveals every design decision in the RFC: why streams use odd/even numbering, how WINDOW_UPDATE frames prevent memory exhaustion, and why HPACK uses both a static table and a dynamic table with FIFO eviction. Understanding these internals is essential for anyone building high-performance network services or debugging mysterious latency spikes in production HTTP/2 deployments.
This document walks through the HTTP/2 framing layer, stream lifecycle, HPACK compression algorithm, and flow control mechanisms with Zig code examples.
HTTP/2 replaces HTTP/1.1’s text-based protocol with a binary framing layer. Every piece of data—headers, body, settings, pings—is wrapped in a frame with a fixed 9-byte header: 3 bytes for payload length (max 16,384 bytes by default), 1 byte for frame type, 1 byte for flags, and 4 bytes for stream identifier. This structure enables multiplexing because each frame is tagged with the stream it belongs to, allowing the receiver to reassemble interleaved frames into complete messages.
Each HTTP/2 stream progresses through a well-defined state machine: idle → open → half-closed (local or remote) → closed. A stream enters the “open” state when HEADERS are sent or received, transitions to “half-closed” when one side sends END_STREAM, and reaches “closed” when both sides have finished. RST_STREAM can abort a stream from any state. Violating the state machine (e.g., sending DATA on a half-closed stream) triggers a stream error or connection error depending on severity.
HPACK compresses HTTP headers using three mechanisms: a static table of 61 common header field/value pairs (like :method: GET), a dynamic table that caches recently seen headers with FIFO eviction, and Huffman coding for literal values. Each header is encoded as either an indexed reference (1 byte for common headers), an indexed name with a literal value, or a fully literal name-value pair. The dynamic table is maintained independently for each direction of a connection, and its maximum size is negotiated via SETTINGS frames.
HTTP/2 implements flow control at both the connection and stream level using WINDOW_UPDATE frames. Each stream and the overall connection start with a 65,535-byte window. When DATA frames are sent, the window shrinks; the receiver sends WINDOW_UPDATE frames to replenish it. This prevents a fast sender from overwhelming a slow receiver and ensures that one stream cannot monopolize the connection’s bandwidth. Flow control applies only to DATA frames—HEADERS, SETTINGS, and other control frames are exempt.
HTTP/2 allows clients to express stream dependencies and weights, forming a priority tree. A stream can depend on another stream, and the server should allocate bandwidth proportionally to stream weights among siblings. In practice, many servers implement simplified priority schemes, and HTTP/3 replaced this model with the simpler Extensible Priority scheme (RFC 9218). Nonetheless, understanding the priority tree is important for diagnosing performance issues in HTTP/2 deployments.
Every HTTP/2 frame begins with a 9-byte header. The parser reads this header, validates the length against the connection’s MAX_FRAME_SIZE setting, and dispatches to the appropriate handler based on the frame type. Here is an example of parsing the frame header in Zig:
const FrameHeader = struct {
length: u24,
frame_type: FrameType,
flags: u8,
stream_id: u31,
pub fn parse(buf: *const [9]u8) FrameHeader {
const length: u24 = @as(u24, buf[0]) << 16 | @as(u24, buf[1]) << 8 | buf[2];
return .{
.length = length,
.frame_type = @enumFromInt(buf[3]),
.flags = buf[4],
.stream_id = @as(u31, @truncate(
std.mem.readInt(u32, buf[5..9], .big) & 0x7FFFFFFF
)),
};
}
pub fn serialize(self: FrameHeader, buf: *[9]u8) void {
buf[0] = @truncate(self.length >> 16);
buf[1] = @truncate(self.length >> 8);
buf[2] = @truncate(self.length);
buf[3] = @intFromEnum(self.frame_type);
buf[4] = self.flags;
std.mem.writeInt(u32, buf[5..9], @as(u32, self.stream_id), .big);
}
};
const FrameType = enum(u8) {
data = 0x0,
headers = 0x1,
priority = 0x2,
rst_stream = 0x3,
settings = 0x4,
push_promise = 0x5,
ping = 0x6,
goaway = 0x7,
window_update = 0x8,
continuation = 0x9,
};The connection maintains a hash map of active streams keyed by stream ID. When a HEADERS frame arrives for a new stream ID, the connection creates a stream object in the “open” state. The stream tracks its state, flow control window, pending data, and header block fragments (which may span HEADERS and CONTINUATION frames).
const StreamState = enum {
idle,
reserved_local,
reserved_remote,
open,
half_closed_local,
half_closed_remote,
closed,
};
const Stream = struct {
id: u31,
state: StreamState,
recv_window: i32,
send_window: i32,
header_buf: std.ArrayList(u8),
data_buf: std.ArrayList(u8),
pub fn applyHeaders(self: *Stream, end_stream: bool) !void {
switch (self.state) {
.idle => {
self.state = if (end_stream) .half_closed_remote else .open;
},
.reserved_remote => {
self.state = if (end_stream) .closed else .half_closed_local;
},
.open => {
if (end_stream) self.state = .half_closed_remote;
},
else => return error.StreamError,
}
}
pub fn applyData(self: *Stream, len: u32, end_stream: bool) !void {
self.recv_window -= @intCast(len);
if (self.recv_window < 0) return error.FlowControlError;
if (end_stream) {
self.state = switch (self.state) {
.open => .half_closed_remote,
.half_closed_local => .closed,
else => return error.StreamError,
};
}
}
};The HPACK encoder checks the static table first, then the dynamic table, encoding headers as indexed references when possible. For headers not in either table, it emits a literal representation and optionally inserts the header into the dynamic table. Integer values use a variable-length encoding with a prefix of N bits, and string values can be Huffman-coded for additional savings.
const HpackEncoder = struct {
dynamic_table: DynamicTable,
huffman_enabled: bool,
pub fn encode(self: *HpackEncoder, headers: []const Header, out: *std.ArrayList(u8)) !void {
for (headers) |h| {
if (self.findStaticIndex(h)) |idx| {
if (h.value_matches) {
// Indexed Header Field (prefix 1, 7-bit index)
try self.encodeInteger(out, idx, 7, 0x80);
} else {
// Literal with Incremental Indexing (prefix 01, 6-bit index)
try self.encodeInteger(out, idx, 6, 0x40);
try self.encodeString(out, h.value);
self.dynamic_table.insert(h);
}
} else {
// New name - literal with indexing
try out.append(0x40);
try self.encodeString(out, h.name);
try self.encodeString(out, h.value);
self.dynamic_table.insert(h);
}
}
}
fn encodeInteger(self: *HpackEncoder, out: *std.ArrayList(u8), value: u32, prefix_bits: u4, pattern: u8) !void {
const max_prefix = (@as(u32, 1) << prefix_bits) - 1;
if (value < max_prefix) {
try out.append(@truncate(pattern | value));
} else {
try out.append(pattern | @as(u8, @truncate(max_prefix)));
var remaining = value - max_prefix;
while (remaining >= 128) {
try out.append(@truncate((remaining & 0x7F) | 0x80));
remaining >>= 7;
}
try out.append(@truncate(remaining));
}
}
};Flow control windows are tracked per-stream and per-connection. When the application receives DATA frames, it decrements both windows. A background task monitors the windows and sends WINDOW_UPDATE frames when they drop below a threshold, preventing the sender from stalling.
const Connection = struct {
streams: std.AutoHashMap(u31, *Stream),
conn_recv_window: i32 = 65535,
conn_send_window: i32 = 65535,
pub fn handleData(self: *Connection, header: FrameHeader, payload: []const u8) !void {
const stream = self.streams.get(header.stream_id) orelse return error.StreamClosed;
const end_stream = (header.flags & 0x01) != 0;
try stream.applyData(header.length, end_stream);
self.conn_recv_window -= @intCast(header.length);
if (self.conn_recv_window < 32768) {
const increment: u32 = 65535 - @as(u32, @intCast(self.conn_recv_window));
try self.sendWindowUpdate(0, increment);
self.conn_recv_window += @intCast(increment);
}
if (stream.recv_window < 16384) {
const increment: u32 = 65535 - @as(u32, @intCast(stream.recv_window));
try self.sendWindowUpdate(stream.id, increment);
stream.recv_window += @intCast(increment);
}
try stream.data_buf.appendSlice(payload);
}
};In a distributed database system, HTTP/2 serves as the transport layer for REST APIs and gRPC endpoints. A Zig-native HTTP/2 implementation provides several benefits in this context:
Every HTTP/2 connection begins with the client sending a 24-byte connection preface (PRI * HTTP/2.0\r\n\r\nSM\r\n\r\n) followed by a SETTINGS frame. The server responds with its own SETTINGS frame, and both sides must acknowledge the other’s settings before proceeding. Key settings include MAX_CONCURRENT_STREAMS (limits resource consumption), INITIAL_WINDOW_SIZE (controls flow control), and MAX_FRAME_SIZE (limits frame payload size).
When a header block is too large for a single HEADERS frame (exceeding MAX_FRAME_SIZE), it is split across HEADERS and one or more CONTINUATION frames. The receiver must buffer all fragments and only decompress them once the END_HEADERS flag is set. Between HEADERS and the final CONTINUATION, no other frames may be sent on the connection for any stream—this prevents interleaving that would corrupt the HPACK decoder state.
To shut down gracefully, a server sends a GOAWAY frame with the last stream ID it will process. The client knows that streams with higher IDs were not processed and can retry them on a new connection. The server continues processing streams up to the advertised ID, allowing in-flight requests to complete. This pattern enables zero-downtime deployments by draining connections before stopping the old server process.