Persona vectors are directions in a language model’s activation space that correspond to behavioral traits such as sycophancy, hallucination tendency, evasiveness, or other deployment-relevant personas. Instead of judging behavior only after text has been generated, the technique probes internal hidden states and asks whether the model is moving toward a trait before or during generation.
The Anthropic persona-vectors work frames the problem as a safety and interpretability problem: production assistants can drift because of user instructions, jailbreak attempts, long conversations, or fine-tuning data. A model may still pass ordinary output checks on many prompts while its internal state becomes more aligned with an undesirable trait. Persona vectors make that drift measurable by projecting activations onto a learned trait direction.
The method is not just an evaluation score. Once a direction is validated, it can support three operational workflows: deployment monitoring, post-hoc activation steering, and training-data auditing. The same vector can estimate whether a prompt is about to elicit a trait, whether fine-tuning shifted the model toward that trait, and whether a candidate training sample is likely to push the model in that direction.
This topic covers the engineering pattern behind persona vectors: building contrastive trait datasets, extracting activation differences, validating causal effect through steering, using projections as monitoring signals, and integrating the result into safety review without treating it as a perfect explanation of the model’s internals.
A transformer does not store a trait as a named variable. It stores intermediate representations across layers and token positions. A persona vector approximates a direction in that high-dimensional space: moving activations further along the direction tends to increase the target trait, while moving against it tends to suppress the trait.
The common extraction strategy is contrastive. Run the same or comparable prompts under trait-eliciting and trait-discouraging conditions, record hidden states, then compute a mean difference between positive and negative activation sets. The result is usually layer-specific because the relevant representation may be stronger in some transformer layers than others.
Once a vector exists, a runtime activation can be projected onto it. A large positive projection means the current hidden state is aligned with the target direction; a low or negative projection means it is not. This converts internal model state into a scalar signal that can be logged, thresholded, trended, and compared across prompts or model versions.
Projection is useful because it can be computed before the final answer is fully emitted. That makes it complementary to output classifiers: classifiers see what the model said, while vector projections can reveal whether the model is internally leaning toward a behavior that may soon appear in the response.
A direction becomes operationally useful only after intervention tests. If adding the vector reliably increases the trait and subtracting or ablating it reduces the trait, the direction is more than a correlational probe. If steering has no behavioral effect, the vector may still be a weak classifier but should not be used as a control surface.
The pipeline starts with trait artifacts: a natural-language trait description, prompts likely to elicit the trait, prompts that discourage it, and evaluation questions. The official persona-vectors implementation generates positive and negative system-prompt runs, stores activations, and computes mean activation differences. The paper reports experiments on Qwen 2.5-7B-Instruct and Llama 3.1-8B-Instruct, which makes the workflow practical for open-weight model safety research.
# Conceptual extraction workflow, not a drop-in library API.
positive = collect_activations(
model=model,
prompts=trait_prompts,
system_instruction="Respond in a sycophantic style",
layer=20,
token_scope="response",
)
negative = collect_activations(
model=model,
prompts=trait_prompts,
system_instruction="Respond honestly and directly",
layer=20,
token_scope="response",
)
persona_vector = mean(positive) - mean(negative)
persona_vector = persona_vector / norm(persona_vector)The official repository exposes the same idea with command-line scripts: evaluate positive and negative persona instructions, call a vector-generation script, and save response-token average differences such as response_avg_diff.pt. Each saved vector has a layer-by-hidden-dimension shape, so downstream jobs can choose which layer to monitor or steer.
Validation asks whether manipulating the activation changes behavior. At inference time, the system can add alpha * vector to a hidden state for selected layers and tokens, then measure trait expression with a judge or benchmark. Steering toward a trait should increase trait expression; steering away from it should reduce expression. The coefficient controls strength and must be tuned because over-steering can degrade general capability.
def apply_steering(hidden_state, vector, coefficient):
# Add a normalized trait direction to the current layer activation.
return hidden_state + coefficient * vector
def project_trait_score(hidden_state, vector):
centered = hidden_state - hidden_state.mean(axis=-1, keepdims=True)
return dot(centered, vector) / (norm(centered) * norm(vector))The Anthropic report distinguishes post-hoc inference-time steering from preventative training-time steering. Post-hoc steering can reduce an acquired trait after fine-tuning but may reduce broad capability. Preventative steering applies an intervention during fine-tuning so the model does not need to move its weights as far in the undesirable direction to fit the data, preserving behavior with less measured degradation in their experiments.
For high-risk assistants, projection scores can be recorded alongside prompt, model version, tool calls, refusal decisions, and output-classifier results. A safety dashboard can track whether specific prompt families, traffic segments, or conversation lengths correlate with rising sycophancy or hallucination-vector activation. This is most useful when monitored in aggregate rather than used as a single hard block on one request.
A practical deployment pattern is shadow-mode monitoring. Compute projections for sampled traffic, compare them with human reviews and output classifiers, then establish alert thresholds only after observing the false-positive and false-negative profile. If a model update shifts projection distributions significantly, the release can be held for targeted red-team review.
Persona vectors can audit candidate training data before a fine-tuning run. For each prompt-response pair, compare the projection of the dataset response with the model’s natural response. A large projection difference suggests that training on the sample may push the model toward the trait. The paper validates this idea on both synthetic trait-inducing datasets and real-world conversation data.
This creates a data-review loop that is more precise than keyword filtering. A sample can avoid explicit trait language while still activating the direction strongly. Engineers can quarantine high-projection samples, run human review, or balance them with counterexamples before launching a fine-tune.
Keep vector extraction offline and reproducible. Store the model checkpoint, trait definition, prompt artifacts, layer selection, token scope, judge model, random seeds, and saved activation files. Persona vectors are sensitive to model family, layer, and prompt construction, so an unlabeled tensor file is not enough for auditability.
{
"trait": "sycophancy",
"model": "Qwen/Qwen2.5-7B-Instruct",
"activation_scope": "response_tokens",
"layers": [20],
"vector_path": "persona_vectors/qwen2.5-7b/sycophancy_response_avg_diff.pt",
"extraction_set": "trait_data_extract/sycophancy.json",
"evaluation_set": "trait_data_eval/sycophancy.json",
"judge": "gpt-4.1-mini",
"created_at": "2025-09-05"
}For production monitoring, isolate activation capture behind an internal probe service. The serving path sends a sampled request through an instrumented model replica or a shadow evaluator, receives projection scores, and writes them to telemetry. This avoids coupling the user-facing latency path to experimental interpretability code while still giving safety engineers statistically useful signal.
class PersonaProbe:
def __init__(self, model, vectors, layers):
self.model = model
self.vectors = vectors
self.layers = layers
def score(self, messages):
activations = self.model.forward_with_activations(messages, layers=self.layers)
scores = {}
for trait, vector in self.vectors.items():
hidden = activations[vector.layer].response_token_mean()
scores[trait] = cosine_projection(hidden, vector.tensor)
return scoresDo not treat the probe as a replacement for output validation. The strongest architecture combines persona-vector telemetry, output classifiers, adversarial evaluation sets, and human review. Each layer catches different failure modes.