Every Transformer block has two main sub-layers: a self-attention mechanism and a feed-forward network (FFN). While attention gets most of the spotlight, the FFN is where the majority of a Transformer’s parameters actually live — roughly two-thirds of the total parameter count in a standard architecture. Understanding FFN layers is essential for anyone working on model compression, fine-tuning, or architecture design because these layers are the primary target for pruning, quantization, and MoE conversion.
A landmark 2021 paper by Mor Geva et al. reframed FFN layers as key-value memories: the first weight matrix stores “keys” that match textual patterns in the input, and the second weight matrix stores “values” that push the output distribution toward tokens associated with those patterns. This interpretation has become foundational for mechanistic interpretability and for understanding why techniques like knowledge editing, neuron pruning, and sparse upcycling work.
For AI engineers, the FFN layer is the component you will most frequently modify when optimizing a model. Whether you are converting a dense model into a Mixture of Experts, quantizing weights to 4-bit integers, or pruning neurons for faster inference, you are operating directly on the FFN.
In the original Transformer (Vaswani et al. 2017), each FFN consists of two linear projections with a non-linearity in between. Given a hidden state x of dimension d_model:
# Standard Transformer FFN
FFN(x) = W2 · activation(W1 · x + b1) + b2
# W1: (d_model, d_ff) — "up-projection"
# W2: (d_ff, d_model) — "down-projection"
# d_ff is typically 4 × d_modelThe up-projection W1 expands the representation into a higher-dimensional space (e.g., from 4096 to 16384 dimensions), the activation function introduces non-linearity, and the down-projection W2 maps back to the model dimension. This expansion-then-compression pattern gives each FFN layer the capacity to represent complex transformations.
In a standard Transformer with d_model = 4096 and d_ff = 16384, each FFN layer has 2 × d_model × d_ff ≈ 134M parameters. The self-attention layer (Q, K, V, O projections) has 4 × d_model² ≈ 67M parameters. The FFN is thus roughly two-thirds of each block’s parameters, making it the primary target for compression and optimization.
Most modern LLMs (LLaMA, Mistral, Gemma, Qwen, DeepSeek) have replaced the original two-matrix FFN with a gated linear unit variant, typically SwiGLU (Shazeer 2020). SwiGLU uses three weight matrices instead of two:
# SwiGLU FFN (used in LLaMA, Mistral, etc.)
SwiGLU(x) = W2 · (swish(W_gate · x) ⊙ (W_up · x))
# W_gate: (d_model, d_ff) — gate projection
# W_up: (d_model, d_ff) — up projection
# W2: (d_ff, d_model) — down projection
# ⊙ = element-wise multiplicationTo keep the total parameter count comparable, the hidden dimension d_ff is reduced to roughly ⅔ × 4 × d_model. For example, LLaMA-2 7B uses d_model = 4096 and d_ff = 11008 instead of 16384. Despite the smaller hidden dimension, SwiGLU consistently yields better perplexity and downstream performance than ReLU or GeLU variants.
Geva et al. (2021) demonstrated that each row of W1 acts as a key that activates on specific textual patterns, and the corresponding column of W2 acts as a value that encodes a distribution over the output vocabulary. The FFN output is a weighted sum of these value vectors, with weights determined by how strongly the input matches each key.
# Interpreting FFN as key-value memory
keys = W1 # Each row k_i is a pattern detector
values = W2 # Each column v_i is a vocabulary distribution
# For input x:
memory_coefficients = activation(keys @ x) # Which patterns match?
output = values @ memory_coefficients # Weighted sum of value distributionsLower layers tend to detect shallow patterns like n-grams and positional features, while upper layers detect semantic patterns like topics and entity types. This explains why knowledge editing techniques (like ROME and MEMIT) target specific FFN layers to update factual associations — they are literally editing entries in the model’s memory.
Each FFN layer’s output is added to the residual stream via a skip connection: x_out = x + FFN(LayerNorm(x)). This means the FFN contributes an additive update to the running representation, and later layers can read, modify, or override earlier contributions. The model’s final prediction is the cumulative composition of all FFN memory retrievals refined through the residual stream.
Empirical studies (Meng et al. 2022, Dai et al. 2022) have shown that factual associations like “The Eiffel Tower is located in [Paris]” are stored in specific FFN layers, typically in the middle-to-late layers of the network. When a subject token activates a particular FFN neuron (key), the corresponding value vector pushes the output probability distribution toward the correct factual completion. This localization of knowledge enables targeted editing: you can change where the model thinks the Eiffel Tower is located by modifying a handful of FFN weight rows, without affecting other knowledge.
Because FFN layers hold the majority of parameters and exhibit structured, interpretable behavior, they are the primary target for nearly every model optimization technique:
The key-value memory interpretation enables direct knowledge editing. ROME (Rank-One Model Editing) identifies which FFN layer stores a specific fact and applies a rank-one update to the value matrix to change that fact. MEMIT extends this to batch edits across multiple layers. These techniques are practical for correcting model hallucinations or updating outdated factual knowledge without full retraining.
Understanding FFN layers is central to mechanistic interpretability research. Tools like TransformerLens and Baukit allow researchers to analyze individual FFN neurons, identify which patterns they detect, and trace how their activations influence the model’s output. This bottom-up understanding of FFN behavior is essential for building trustworthy AI systems.