The Write-Ahead Log (WAL) is one of the most fundamental concepts in database engineering. Its principle is simple but powerful: before any change is applied to the main data structures, the raw bytes of that change are first appended to a sequential log file on durable storage. If the system crashes at any point, the log can be replayed on restart to recover every committed operation.
WAL provides two of the four ACID guarantees — Atomicity (operations either fully complete or fully roll back) and Durability (committed data survives crashes). It is used in virtually every production database: PostgreSQL, MySQL/InnoDB, SQLite, RocksDB, Pebble, Cassandra, and the storage layers beneath vector databases.
For AI engineers, WAL is the mechanism that ensures your embeddings, metadata, and index state are never lost during crashes or power failures. When you bulk-ingest a million document embeddings into a vector database, the WAL is what guarantees that a server restart mid-ingestion doesn’t corrupt the entire dataset. Understanding WAL also helps you tune the critical tradeoff between write throughput and durability guarantees.
The WAL protocol enforces a strict ordering of operations. Every write follows this sequence:
fsync() or fdatasync() to ensure the record reaches durable storage (not just the OS page cache)The critical invariant is: the WAL record must be durably stored before the write is acknowledged. This means that even if the process crashes immediately after acknowledging, the operation is recoverable from the log.
No modification to the actual data pages is written to disk until the corresponding log record has been flushed to stable storage. This single rule is what makes crash recovery possible — the log is always at least as up-to-date as the data files, and usually ahead of them.
Each WAL record is a self-contained unit that includes enough information to replay the operation. A typical format looks like:
// WAL record structure (simplified)
struct WALRecord {
uint32 checksum; // CRC32 of the payload for corruption detection
uint16 record_type; // FULL, FIRST, MIDDLE, LAST (for records spanning blocks)
uint16 payload_length; // Size of the data portion
uint64 sequence_number; // Monotonically increasing operation ID
uint8 operation; // PUT = 0x01, DELETE = 0x02, MERGE = 0x03
bytes key; // Variable-length key
bytes value; // Variable-length value (empty for DELETE)
}
// WAL file is divided into fixed-size blocks (typically 32 KB)
// Records that don't fit in one block are split across blocks
// using FIRST/MIDDLE/LAST record typesWAL is fast because it only performs sequential writes — appending to the end of a file. This matters enormously for storage performance:
This speed advantage is why LSM-Trees pair a WAL with an in-memory memtable: the WAL provides durability at sequential-write speed, while the memtable provides sorted access at memory speed. The expensive random I/O of updating sorted on-disk structures is deferred to background compaction.
Calling write() on a file only guarantees the data reaches the OS page cache — a power failure can still lose it. fsync() forces the OS to flush the file’s data and metadata to the physical storage device. This is the most expensive part of a WAL write (typically 0.1-2ms per sync on SSD), and is the primary knob for trading durability against throughput.
When a system restarts after a crash, the recovery process follows these steps:
// Simplified crash recovery pseudocode
function recover():
checkpoint_seq = read_manifest().last_flushed_sequence
for record in wal.read_from(checkpoint_seq):
if not record.checksum_valid():
// Corrupted record = end of committed data
break
memtable.apply(record) // Re-insert into memory
last_recovered_seq = record.sequence_number
log("Recovered %d operations", last_recovered_seq - checkpoint_seq)
// System is now consistent and ready for new writesWhile the WAL concept is universal, its role differs between storage architectures:
Most storage engines offer configurable sync behavior, letting you trade durability for throughput:
In RocksDB, this is controlled by WriteOptions::sync. In Pebble, you pass pebble.Sync or pebble.NoSync to write operations. PostgreSQL uses synchronous_commit and wal_sync_method settings.
When you ingest embeddings into a vector database like Weaviate or Milvus, the WAL ensures that each batch of vectors is durable before the ingestion API returns success. If the database crashes mid-ingestion, only the last uncommitted batch is lost — all previously acknowledged batches are recovered from the WAL on restart. This is critical for ETL pipelines that need to know exactly where to resume.
The Raft consensus protocol, used in CockroachDB, etcd, and many distributed systems, requires each node to persist log entries to a WAL before voting to commit them. The WAL is literally the Raft log — without it, a node that restarts would lose its voting history and could cause data inconsistency across the cluster. The performance of the WAL directly determines the throughput of the entire distributed system.
AI systems that process real-time data streams (user interactions, sensor data, log events) use WAL-backed stores as their first persistence layer. Apache Kafka’s log-structured storage is essentially a distributed WAL — every message is appended sequentially and retained for replay. This makes it possible to reprocess historical data through updated embedding models without data loss.
When embedding millions of documents, a WAL-based checkpoint strategy lets you resume from the last successful batch after any failure. Rather than re-embedding the entire corpus (which could cost hundreds of dollars in API fees), the WAL tells you exactly which operations completed, enabling efficient resumption.
fsync() call is the critical durability boundary — without it, data only reaches the OS page cache, not the physical device