Fixed-size Chunking

Overview

Fixed-size chunking is the simplest and most widely used method for splitting documents into smaller pieces for AI pipelines. It divides text into uniform segments of a predetermined length — measured in characters, tokens, or words — regardless of the document’s semantic structure. Every chunk produced is approximately the same size, making it highly predictable and computationally efficient.

In Retrieval-Augmented Generation (RAG) systems, chunking is a foundational preprocessing step. Before documents can be embedded into vector space and retrieved at query time, they must be broken into pieces small enough to fit within embedding model limits (typically 512 tokens) and focused enough to return relevant context. Fixed-size chunking is the baseline approach that most teams start with because it requires no NLP models, no sentence boundary detection, and no structural parsing.

However, this simplicity comes at a cost. By ignoring the natural boundaries of text — sentences, paragraphs, sections — fixed-size chunking can sever ideas mid-thought, splitting a sentence across two chunks and degrading retrieval quality. Understanding when this tradeoff is acceptable, and how to mitigate its downsides, is essential knowledge for any AI engineer building data pipelines.

How It Works

The Basic Algorithm

Fixed-size chunking operates on a straightforward principle: walk through the text from beginning to end, extracting segments of exactly chunk_size units. The unit can be characters, tokens (as defined by a tokenizer like tiktoken), or words. The algorithm requires only two parameters:

• chunk_size — the maximum number of units per chunk (e.g., 256 tokens)
• chunk_overlap — the number of units shared between consecutive chunks (e.g., 20 tokens)

Implementation Example

Here is a basic implementation of fixed-size character chunking with overlap:

# Fixed-size chunking with overlap
def fixed_size_chunk(text, chunk_size=1000, overlap=200):
    chunks = []
    start = 0
    while start < len(text):
        end = start + chunk_size
        chunks.append(text[start:end])
        start = end - overlap  # slide back by overlap amount
    return chunks

# Example usage
document = open("report.txt").read()  # 15,000 characters
chunks = fixed_size_chunk(document, chunk_size=1000, overlap=200)
# Produces ~19 chunks, each ~1000 chars, with 200-char overlap

Using LangChain’s CharacterTextSplitter

In practice, most teams use a library rather than writing splitting logic from scratch. LangChain provides CharacterTextSplitter for fixed-size chunking:

from langchain.text_splitter import CharacterTextSplitter

splitter = CharacterTextSplitter(
    separator="",          # no separator = pure fixed-size
    chunk_size=1000,       # max characters per chunk
    chunk_overlap=200,     # overlap between chunks
    length_function=len    # measure by character count
)

chunks = splitter.split_text(document)

Token-Based vs Character-Based

Character-based chunking is the simplest variant but has a significant drawback: character counts don’t map cleanly to token counts, and embedding models have token limits, not character limits. A 1000-character chunk might be 180 tokens or 320 tokens depending on the text. Token-based chunking solves this by measuring chunk size in the same units the model consumes:

import tiktoken

def token_chunk(text, chunk_size=256, overlap=20, model="text-embedding-3-small"):
    enc = tiktoken.encoding_for_model(model)
    tokens = enc.encode(text)
    chunks = []
    start = 0
    while start < len(tokens):
        end = start + chunk_size
        chunk_tokens = tokens[start:end]
        chunks.append(enc.decode(chunk_tokens))
        start = end - overlap
    return chunks

Choosing Chunk Size

The optimal chunk size depends on your embedding model, retrieval strategy, and content type. Here are practical guidelines:

• Small chunks (128-256 tokens): Higher precision retrieval. Each chunk is tightly focused. Better for factual Q&A where you need a specific fact, not a broad context.
• Medium chunks (256-512 tokens): Balanced approach. Good for most RAG applications. A sensible starting point is ~250 tokens (~1000 characters).
• Large chunks (512-1024 tokens): More context per retrieval. Better for summarization tasks or when questions require multi-paragraph reasoning. Risks diluting relevance.

Practical Applications

When Fixed-size Chunking Works Well

Fixed-size chunking is the right choice in several common scenarios:

• Homogeneous text: Documents with uniform density and no strong structural hierarchy — transcripts, chat logs, continuous prose — chunk well at fixed sizes because there are no natural boundaries to respect.
• High-volume pipelines: When processing millions of documents, the O(n) simplicity of fixed-size chunking matters. No embedding model calls, no NLP parsing — just string slicing. This can be 10-100x faster than semantic chunking.
• Baseline and prototyping: When building a new RAG pipeline, start with fixed-size chunking to establish baseline retrieval metrics. Then compare against more sophisticated strategies to see if the added complexity is justified.
• Embedding model constraints: When your embedding model has a strict token limit (e.g., 512 tokens for many BERT-based models), fixed-size token chunking guarantees no chunk exceeds the limit.

When It Falls Short

Fixed-size chunking struggles with structured documents — technical manuals with headers, legal contracts with numbered clauses, or code with function boundaries. In these cases, a chunk boundary might split a table in half, separate a header from its body text, or cut a code function mid-logic. The resulting chunks are incomplete thoughts that embed poorly and retrieve inaccurately.

Real-World Example: Customer Support Knowledge Base

Consider a knowledge base with 10,000 FAQ articles averaging 800 words each. Using fixed-size chunking at 256 tokens with 25-token overlap:

• Each article produces ~4-5 chunks
• Total: ~45,000 chunks to embed and index
• Processing time: minutes (vs. hours for semantic chunking)
• Retrieval quality: adequate for straightforward factual queries but may struggle with multi-sentence answers that span chunk boundaries

Key Takeaways

• Fixed-size chunking divides text into uniform segments by character, token, or word count — the simplest chunking strategy available.
• chunk_overlap is essential: it prevents information loss at boundaries by repeating text between adjacent chunks. Use 10-20% of your chunk size.
• Token-based chunking is preferred over character-based when your embedding model has token limits, since character-to-token ratios vary.
• Start with ~250 tokens as a baseline chunk size and tune based on measured retrieval quality.
• Fixed-size chunking is fastest and simplest but ignores semantic structure — it will split sentences, paragraphs, and ideas mid-thought.
• Best suited for homogeneous text, high-volume pipelines, and establishing baseline metrics before trying more complex strategies.
• For structured documents (manuals, contracts, code), consider recursive or semantic chunking instead.