Semantic memory is a long-term memory system that stores generalized facts, knowledge, and learned relationships extracted from the agent’s interactions — without tying them to specific events or timestamps. Where episodic memory answers “What happened on March 15th?”, semantic memory answers “What do I know about this user?” or “What are the rules for handling refund requests?” It is the distilled knowledge that an agent carries forward, stripped of event context and reduced to reusable facts.
In human cognition, semantic memory is what lets you know that Paris is the capital of France without remembering when or where you learned it. For AI agents, semantic memory serves the same purpose — it stores user preferences (“prefers TypeScript over JavaScript”), domain facts (“the booking API requires ISO 8601 date formats”), and learned relationships (“when users ask about pricing, they usually need the enterprise tier breakdown”) that inform behavior across all future interactions.
Semantic memory is what makes the difference between an agent that treats every conversation as a blank slate and one that genuinely “knows” its users and domain. It is the most impactful memory type for personalization, and frameworks like Mem0 and Zep have made it the centerpiece of their memory architectures. For AI engineers, understanding how to extract, store, update, and retrieve semantic memories is essential for building agents that feel intelligent and contextually aware.
Semantic memories are atomic facts and relationships — individual pieces of knowledge that stand on their own. Unlike episodic memories (which are event narratives), semantic memories are declarative statements:
Episodic: “On March 15th, the user reported a CORS bug in the API gateway. We traced it to a missing header and deployed a fix.” Semantic: “The user’s API gateway has had CORS configuration issues. The user is experienced with debugging network errors.” Semantic memory extracts the reusable knowledge from episodes and discards the event-specific details.
Semantic memories don’t appear from nowhere — they must be extracted from conversations and interactions through a deliberate pipeline. This is typically an LLM-powered process that runs after each conversation:
# Semantic memory extraction using an LLM
EXTRACTION_PROMPT = """
Analyze this conversation and extract factual knowledge about
the user, their preferences, and their domain. Output each fact
as a standalone statement that would be useful in future
conversations.
Rules:
- Extract only facts that are likely to remain true over time
- Do NOT extract ephemeral task details
- Each fact should be a single, atomic statement
- Include confidence level (high/medium/low)
Conversation:
{conversation_text}
Output format (JSON):
[
{
"fact": "User prefers async/await over .then() chains",
"category": "preference",
"confidence": "high",
"source_turn": 7
},
...
]
"""
# After extraction, each fact is processed
for fact in extracted_facts:
embedding = embed_model.encode(fact["fact"])
existing = memory_store.search(
embedding=embedding,
threshold=0.92 # High threshold for dedup
)
if existing:
# Update existing fact (merge or replace)
memory_store.update(existing[0].id, fact)
else:
# Add new fact
memory_store.add(fact, embedding=embedding)A critical challenge in semantic memory is handling contradictions and duplicates. Over time, the same fact may be extracted multiple times in slightly different forms, or a user’s preferences may change. The memory system must:
valid_at and invalid_at timestamps to facts, creating a temporal knowledge graph where the system knows not just what’s true, but when it became true.Mem0 uses an LLM-powered tool-call mechanism to manage semantic memories. After extracting new facts, a second LLM call compares them against existing memories and decides for each: ADD (new fact, no existing match), UPDATE (existing fact needs modification), or DELETE (fact is no longer true). This keeps the memory store clean and current without human intervention.
Semantic memories are typically stored in systems optimized for both structured queries and semantic search:
User → prefers → TypeScript, User → manages → Team Alpha). Graph traversal enables multi-hop queries: “What tools does the user’s team prefer?”When a new conversation begins, the system retrieves relevant semantic memories and injects them into the agent’s working memory as part of the system prompt or context:
# Retrieve relevant semantic memories for current context
def build_memory_context(user_id: str, current_query: str) -> str:
# Always include core user profile facts
core_facts = memory_store.query(
filter={"user_id": user_id, "category": "profile"},
top_k=10
)
# Retrieve contextually relevant facts
query_embedding = embed_model.encode(current_query)
relevant_facts = memory_store.query(
embedding=query_embedding,
filter={"user_id": user_id},
top_k=10
)
# Deduplicate and format
all_facts = deduplicate(core_facts + relevant_facts)
memory_block = "## What you know about this user:\n"
for fact in all_facts:
memory_block += f"- {fact.content}\n"
return memory_block
# Inject into system prompt
system_prompt = f"""You are a helpful assistant.
{build_memory_context(user_id, user_query)}
Use the above knowledge to personalize your responses.
"""The key design decision is what to inject and when. Injecting all known facts wastes context window space; injecting too few misses personalization opportunities. A common approach is to always inject a small “user profile” (5-10 core facts) and then add contextually relevant facts based on the current query.
A coding assistant with semantic memory knows that a developer prefers functional programming patterns, uses Vim keybindings, deploys to AWS, and is color-blind (so code highlighting suggestions should use high-contrast themes). This knowledge shapes every response — from code suggestions to documentation formatting — without the user needing to re-state their preferences in every session.
In enterprise settings, semantic memory stores organizational knowledge: “The payments team uses gRPC for service-to-service communication,” “Production deployments require a VP-level approval on Fridays,” “The legacy billing system uses SOAP XML and cannot be modified.” This institutional knowledge prevents agents from making suggestions that violate organizational constraints or norms.
A support agent with semantic memory knows that Customer X is on the enterprise plan, has a technical background, prefers email communication, and has previously expressed frustration with slow response times. The agent adjusts its communication style, technical depth, and escalation urgency based on these stored facts. Studies show that personalized support interactions reduce average handle time by 20-30% and increase customer satisfaction scores by 15-25%.
In specialized domains, semantic memory stores domain-specific rules and knowledge: “Patient has penicillin allergy,” “Client’s jurisdiction requires GDPR compliance for EU data subjects,” “The applicable statute of limitations is 6 years in this case.” These critical facts must be reliably stored, accurately retrieved, and never lost or contradicted — making the ADD/UPDATE/DELETE pipeline and conflict resolution especially important.
from mem0 import Memory
# Initialize with configuration
config = {
"vector_store": {
"provider": "qdrant",
"config": {"collection_name": "user_memories"}
},
"llm": {
"provider": "openai",
"config": {"model": "gpt-4o"}
},
"graph_store": {
"provider": "neo4j",
"config": {"url": "bolt://localhost:7687"}
}
}
memory = Memory.from_config(config)
# Add memories from a conversation
memory.add(
messages=[
{"role": "user", "content": "I'm working on a React Native app "
"for hotel bookings. We use TypeScript and deploy to AWS."},
{"role": "assistant", "content": "Great! I can help with that..."}
],
user_id="dev_jane"
)
# Later: retrieve relevant memories
results = memory.search(
query="How should I set up CI/CD?",
user_id="dev_jane"
)
# Returns: ["User works with React Native",
# "User uses TypeScript",
# "User deploys to AWS"]