Multi-tenant memory systems face a fundamental design challenge: how do you give each team, project, or agent namespace its own isolated memory store without the operational overhead of managing hundreds of separate databases? Namespace-provisioned tables solve this by automatically creating dedicated database tables, AKNN embedding indexes, and graph edge structures for each namespace on first access. Every namespace gets full vector-search and graph-traversal capabilities from the moment it’s provisioned, with no manual setup required.
The approach draws on the database’s native support for dynamically created tables and its built-in approximate k-nearest-neighbor (AKNN) indexing. When a new namespace is registered, the provisioning layer creates a memories table with columns for content, embeddings, metadata, and timestamps; an AKNN index over the embedding column for fast similarity search; and a graph_v0 edge table that lets memories link to entities, topics, and other memories. This means each team’s memories are physically isolated — queries can never accidentally return another team’s data — while sharing the same underlying database cluster for efficiency.
This pattern is central to the memorydb system in the exampledb ecosystem, where it enables SaaS-style memory serving. A single memorydb deployment can host hundreds of isolated namespaces, each with its own embedding indexes and entity graphs, all provisioned lazily and garbage-collected when inactive.
A namespace is a logical boundary within a shared database cluster that guarantees data isolation. Each namespace maps to its own set of tables (memories, edges, entities) with naming conventions like {namespace}_memories, {namespace}_edges. Queries within a namespace can never read or write data belonging to another namespace, enforced at the table-naming level rather than through row-level security — providing hard isolation with zero filtering overhead at query time.
Tables and indexes are created on first access rather than upfront. When a memory store or search request arrives for a namespace that doesn’t yet have tables, the provisioning layer creates them atomically before processing the request. This avoids the need for a separate “create namespace” API and means the system scales to thousands of namespaces without pre-allocating resources for inactive ones.
The database’s approximate k-nearest-neighbor (AKNN) index is a built-in vector index that supports fast similarity search over embedding columns. It uses locality-sensitive hashing or graph-based algorithms (similar to HNSW) to provide sub-millisecond approximate nearest neighbor lookups. Each namespace gets its own AKNN index, sized appropriately for its memory count, and the index is automatically maintained as memories are inserted or deleted.
The graph_v0 schema defines how memories connect to each other and to entities. Edge tables store directional relationships with typed edges (e.g., mentions, related_to, derived_from), edge weights, and timestamps. This graph layer enables traversal queries like “find all memories that mention entity X” or “find the chain of reasoning that led to decision Y” — capabilities that pure vector search cannot provide.
Each provisioned namespace records the schema version used to create its tables. When the memory system upgrades (new columns, new index types, changed edge schemas), a migration layer detects outdated namespaces and applies incremental DDL changes. This ensures that long-lived namespaces automatically receive new capabilities without manual intervention or downtime.
The provisioning system follows an idempotent check-and-create pattern. A distributed lock prevents race conditions when multiple requests arrive simultaneously for a new namespace.
package memory
import (
"context"
"fmt"
"sync"
db "github.com/example/exampledb"
)
type NamespaceProvisioner struct {
client *db.Client
mu sync.Mutex
provisioned map[string]bool
}
func NewProvisioner(client *db.Client) *NamespaceProvisioner {
return &NamespaceProvisioner{
client: client,
provisioned: make(map[string]bool),
}
}
func (p *NamespaceProvisioner) EnsureNamespace(ctx context.Context, ns string) error {
p.mu.Lock()
defer p.mu.Unlock()
if p.provisioned[ns] {
return nil
}
memoriesTable := fmt.Sprintf("%s_memories", ns)
exists, err := p.client.TableExists(ctx, memoriesTable)
if err != nil {
return fmt.Errorf("checking table existence: %w", err)
}
if !exists {
if err := p.createNamespaceTables(ctx, ns); err != nil {
return fmt.Errorf("provisioning namespace %q: %w", ns, err)
}
}
p.provisioned[ns] = true
return nil
}
func (p *NamespaceProvisioner) createNamespaceTables(ctx context.Context, ns string) error {
memoriesTable := fmt.Sprintf("%s_memories", ns)
edgesTable := fmt.Sprintf("%s_edges", ns)
entitiesTable := fmt.Sprintf("%s_entities", ns)
// Create the memories table with embedding column
err := p.client.CreateTable(ctx, memoriesTable, db.Schema{
Columns: []db.Column{
{Name: "id", Type: db.TypeUUID, PrimaryKey: true},
{Name: "content", Type: db.TypeText},
{Name: "embedding", Type: db.TypeVector, Dimensions: 1536},
{Name: "tags", Type: db.TypeStringArray},
{Name: "importance", Type: db.TypeString},
{Name: "source", Type: db.TypeString},
{Name: "created_at", Type: db.TypeTimestamp},
{Name: "updated_at", Type: db.TypeTimestamp},
{Name: "access_count", Type: db.TypeInt64},
{Name: "schema_version", Type: db.TypeInt32},
},
})
if err != nil {
return fmt.Errorf("creating memories table: %w", err)
}
// Create AKNN index on the embedding column
err = p.client.CreateAKNNIndex(ctx, memoriesTable, db.AKNNConfig{
Column: "embedding",
Dimensions: 1536,
Metric: db.MetricCosine,
IndexType: db.IndexHNSW,
HNSWConfig: &db.HNSWConfig{
M: 16,
EfConstruction: 200,
},
})
if err != nil {
return fmt.Errorf("creating AKNN index: %w", err)
}
// Create the entity graph tables
err = p.client.CreateTable(ctx, entitiesTable, db.Schema{
Columns: []db.Column{
{Name: "id", Type: db.TypeUUID, PrimaryKey: true},
{Name: "name", Type: db.TypeString},
{Name: "entity_type", Type: db.TypeString},
{Name: "mention_count", Type: db.TypeInt64},
{Name: "created_at", Type: db.TypeTimestamp},
},
})
if err != nil {
return fmt.Errorf("creating entities table: %w", err)
}
err = p.client.CreateTable(ctx, edgesTable, db.Schema{
Columns: []db.Column{
{Name: "id", Type: db.TypeUUID, PrimaryKey: true},
{Name: "source_id", Type: db.TypeUUID},
{Name: "target_id", Type: db.TypeUUID},
{Name: "edge_type", Type: db.TypeString},
{Name: "weight", Type: db.TypeFloat64},
{Name: "created_at", Type: db.TypeTimestamp},
},
})
if err != nil {
return fmt.Errorf("creating edges table: %w", err)
}
return nil
}The memory store wraps every operation with a namespace check. Callers never need to think about provisioning — they just read and write memories, and the namespace materializes on demand.
package memory
import (
"context"
"fmt"
"time"
"github.com/google/uuid"
db "github.com/example/exampledb"
)
type MemoryStore struct {
client *db.Client
provisioner *NamespaceProvisioner
embedder Embedder
}
type Memory struct {
ID string
Content string
Embedding []float32
Tags []string
Importance string
Source string
CreatedAt time.Time
UpdatedAt time.Time
}
func (s *MemoryStore) Store(ctx context.Context, ns string, content string, tags []string, importance string) (string, error) {
if err := s.provisioner.EnsureNamespace(ctx, ns); err != nil {
return "", err
}
embedding, err := s.embedder.Embed(ctx, content)
if err != nil {
return "", fmt.Errorf("generating embedding: %w", err)
}
id := uuid.New().String()
table := fmt.Sprintf("%s_memories", ns)
now := time.Now()
err = s.client.Insert(ctx, table, db.Row{
"id": id,
"content": content,
"embedding": embedding,
"tags": tags,
"importance": importance,
"source": "agent",
"created_at": now,
"updated_at": now,
"access_count": 0,
"schema_version": 1,
})
if err != nil {
return "", fmt.Errorf("inserting memory: %w", err)
}
return id, nil
}
func (s *MemoryStore) Search(ctx context.Context, ns string, query string, limit int) ([]Memory, error) {
if err := s.provisioner.EnsureNamespace(ctx, ns); err != nil {
return nil, err
}
queryEmb, err := s.embedder.Embed(ctx, query)
if err != nil {
return nil, fmt.Errorf("embedding query: %w", err)
}
table := fmt.Sprintf("%s_memories", ns)
results, err := s.client.AKNNSearch(ctx, table, db.AKNNQuery{
Vector: queryEmb,
Column: "embedding",
Limit: limit,
})
if err != nil {
return nil, fmt.Errorf("AKNN search: %w", err)
}
memories := make([]Memory, 0, len(results))
for _, row := range results {
memories = append(memories, Memory{
ID: row["id"].(string),
Content: row["content"].(string),
Tags: row["tags"].([]string),
Importance: row["importance"].(string),
CreatedAt: row["created_at"].(time.Time),
UpdatedAt: row["updated_at"].(time.Time),
})
}
return memories, nil
}Namespaces that have been inactive beyond a configurable TTL can be garbage-collected. The GC process checks the most recent updated_at timestamp across all memories in a namespace and drops the tables if the namespace has been dormant.
func (p *NamespaceProvisioner) GarbageCollect(ctx context.Context, maxInactiveDays int) ([]string, error) {
namespaces, err := p.listNamespaces(ctx)
if err != nil {
return nil, err
}
cutoff := time.Now().AddDate(0, 0, -maxInactiveDays)
var dropped []string
for _, ns := range namespaces {
table := fmt.Sprintf("%s_memories", ns)
lastActivity, err := p.client.QueryScalar(ctx,
fmt.Sprintf("SELECT MAX(updated_at) FROM %s", table))
if err != nil {
continue
}
if lastActivity.(time.Time).Before(cutoff) {
p.dropNamespaceTables(ctx, ns)
dropped = append(dropped, ns)
}
}
return dropped, nil
}For high-throughput deployments, maintain a pool of prepared statements per active namespace. Since table names are embedded in queries (not parameterizable), each namespace needs its own set of prepared inserts, searches, and deletes. Use an LRU cache to bound the number of active prepared-statement sets.
Support nested namespaces like team-alpha/project-x by encoding the hierarchy in table name prefixes with a separator (e.g., team_alpha__project_x_memories). This enables queries scoped to a project or broadened to an entire team by querying all tables matching the prefix pattern.
When an agent needs to search across multiple namespaces (e.g., finding shared knowledge), run parallel AKNN queries against each namespace’s table and fuse results using reciprocal rank fusion. This preserves namespace isolation in storage while enabling controlled cross-namespace discovery.