Scheduled sampling is a curriculum-learning technique for autoregressive sequence models. It addresses a practical mismatch in teacher-forced training: during training the model usually receives the correct previous token, but during inference it must condition on tokens it generated itself. When an early prediction is wrong, the model can enter states it never saw during training, and later errors compound along the generated sequence.
The original 2015 work introduced the method for recurrent neural networks used in machine translation, image captioning, constituency parsing, and speech recognition. The core idea is simple: instead of always feeding the ground-truth previous token during training, randomly choose between the ground-truth token and the model’s own previous prediction. Early in training, the probability of using ground truth is high; later, a schedule lowers that probability so the model practices recovering from its own mistakes.
For engineers, scheduled sampling is useful because it separates two concerns that are often blurred together: maximum-likelihood next-token accuracy and robustness under autoregressive rollout. It is not a universal replacement for likelihood training, and it has known theoretical caveats, but it remains a clear example of aligning the training loop with deployment-time behavior.
Teacher forcing trains a sequential decoder by feeding the true previous token at each step. If the target sequence is y1, y2, y3, the model learns to predict y2 given y1, then y3 given y2. This makes optimization stable because the conditioning context is always valid, even when the current model would have predicted something else.
Exposure bias is the gap between the contexts a model sees during teacher-forced training and the contexts it sees during free-running inference. In deployment, a wrong token becomes part of the next input. The model may then have to recover from unfamiliar states, even if its single-step validation loss looked strong.
Scheduled sampling turns the source of the previous token into a curriculum. The probability of using ground truth starts near one, then decays toward a lower value. This lets the model first learn the task under clean conditions and later experience noisier, self-generated prefixes.
The method is less about adding noise and more about changing the state distribution. A model that will generate from its own outputs should practice on prefixes that include its own outputs, not only perfect training prefixes.
During training, the decoder still predicts the next token and receives the usual supervised loss. The difference is in the input token used for the next recurrent step. With probability epsilon, use the true token. With probability 1 - epsilon, use a token produced by the model, either by sampling from its predicted distribution or by taking the highest-probability token.
def scheduled_sampling_step(model, x, target, epsilon):
state = model.encode(x)
prev = BOS
losses = []
for t, gold in enumerate(target):
logits, state = model.decode_step(prev, state)
losses.append(cross_entropy(logits, gold))
predicted = sample_or_argmax(logits)
if random_uniform() < epsilon:
prev = gold
else:
prev = stop_gradient(predicted)
return mean(losses)The original paper describes several schedules for reducing epsilon: linear decay, exponential decay, and inverse-sigmoid decay. The schedule is a training-control parameter, similar in spirit to a learning-rate schedule. Too much self-sampling too early makes optimization difficult because the model is not yet competent enough to generate useful prefixes.
def inverse_sigmoid_epsilon(step, k):
# High at the start, then gradually decreases.
return k / (k + exp(step / k))Scheduled sampling does not require reinforcement learning or sequence-level rewards. The loss is still computed against the target next token at each step. This makes it easy to add to an existing sequence-to-sequence training loop, but it also means the sampled token decision is usually treated as non-differentiable in simple implementations.
The original paper used scheduled sampling in an LSTM image-captioning system. The model begins from an image representation and emits words autoregressively. Because early word choices constrain later captions, practicing with model-generated previous words can improve robustness during rollout.
Any encoder-decoder model that generates a target sequence token by token can suffer from compounding errors. Scheduled sampling is most natural when the decoder state depends directly on the previous emitted token, as in recurrent decoders. Transformer variants require more care because the decoder attends to the whole generated prefix rather than only the immediate previous token.
The paper also evaluated speech recognition and constituency parsing. These tasks show that the technique is not limited to natural-language generation; it applies whenever a model predicts a sequence of discrete decisions and future predictions depend on earlier predicted decisions.
Log the current epsilon, the schedule type, and the fraction of sampled tokens per batch. Without this telemetry, it is hard to distinguish a bad model architecture from an overly aggressive schedule.
for step, batch in enumerate(loader):
epsilon = schedule(step)
loss = scheduled_sampling_step(model, batch.x, batch.y, epsilon)
loss.backward()
optimizer.step()
metrics.log({
"loss": loss.item(),
"teacher_forcing_probability": epsilon,
})A common production pattern is to train with pure teacher forcing for an initial warm-up period, then begin decaying epsilon. This avoids wasting early updates on random model outputs that carry little task structure.
Do not rely only on teacher-forced validation loss. Add rollout metrics that decode full sequences using the same strategy used at inference, then score the completed sequence. Scheduled sampling is meant to improve rollout behavior, so rollout metrics are the most relevant validation signal.
epsilon too quickly can destabilize training because the model sees low-quality self-generated prefixes before it has learned the basic task.epsilon.