Hybrid Search with Entity Graph Expansion

Overview

Traditional search systems choose between keyword matching (full-text search) and semantic similarity (vector search), but each approach has blind spots. Keyword search misses paraphrased content; vector search misses exact terms and named entities. Hybrid search combines both approaches using Reciprocal Rank Fusion (RRF) to merge ranked result lists from different retrieval strategies into a single, higher-quality ranking.

The memory system takes hybrid search further by adding a third retrieval channel: entity graph expansion. When a query mentions entities (people, projects, concepts), the system recognizes them on the fly, looks up their neighbors in a knowledge graph, and includes those neighbors in the search context. If you search for “Alice’s architecture decisions,” the system not only finds memories mentioning Alice but also surfaces memories about projects Alice is connected to in the entity graph—even if those memories don’t mention Alice by name.

This three-channel approach—full-text, semantic, and graph-expanded—produces significantly richer recall for memory-oriented applications where relationships between entities carry as much meaning as the content of individual documents.

Core Concepts

How It Works

When a search query arrives, the system runs entity recognition and all three retrieval channels concurrently. The entity recognizer feeds its results to the graph expansion channel while full-text and vector search proceed independently. Once all channels return their ranked results, RRF merges them into a final ranking.

RRF Score Computation

package search

// RRFConstant is the dampening factor for reciprocal rank fusion.
// A value of 60 is standard in the literature (Cormack et al., 2009).
const RRFConstant = 60

// RankedDoc represents a document with its rank from a single retrieval channel.
type RankedDoc struct {
    ID   string
    Rank int
}

// FuseResults merges ranked lists from multiple retrieval channels using RRF.
// Each input slice is a ranked list (index 0 = rank 1).
func FuseResults(channels ...[]RankedDoc) []ScoredDoc {
    scores := make(map[string]float64)

    for _, channel := range channels {
        for _, doc := range channel {
            scores[doc.ID] += 1.0 / float64(RRFConstant+doc.Rank)
        }
    }

    // Convert to sorted slice
    result := make([]ScoredDoc, 0, len(scores))
    for id, score := range scores {
        result = append(result, ScoredDoc{ID: id, Score: score})
    }

    sort.Slice(result, func(i, j int) bool {
        return result[i].Score > result[j].Score
    })

    return result
}

// ScoredDoc holds a document ID and its fused RRF score.
type ScoredDoc struct {
    ID    string
    Score float64
}

Entity Recognition and Graph Expansion

package search

import (
    "context"
    "strings"
)

// EntityGraph provides neighbor lookup for known entities.
type EntityGraph interface {
    // Lookup returns the entity ID if the name is known, or empty string.
    Lookup(ctx context.Context, name string) (entityID string, found bool)
    // Neighbors returns 1-hop neighbor entity names for a given entity ID.
    Neighbors(ctx context.Context, entityID string) ([]string, error)
}

// ExtractAndExpand recognizes entities in the query and returns expanded terms.
func ExtractAndExpand(ctx context.Context, query string, graph EntityGraph) ([]string, error) {
    // Simple entity extraction: split on spaces, check capitalized tokens
    // against the known entity graph.
    tokens := strings.Fields(query)
    var seedEntities []string

    for _, token := range tokens {
        cleaned := strings.Trim(token, ".,!?\"'")
        if len(cleaned) == 0 {
            continue
        }
        if entityID, found := graph.Lookup(ctx, cleaned); found {
            seedEntities = append(seedEntities, entityID)
        }
    }

    // Also try multi-word entity names (bigrams)
    for i := 0; i < len(tokens)-1; i++ {
        bigram := strings.Trim(tokens[i], ".,!?") + " " + strings.Trim(tokens[i+1], ".,!?")
        if entityID, found := graph.Lookup(ctx, bigram); found {
            seedEntities = append(seedEntities, entityID)
        }
    }

    // Expand: collect 1-hop neighbors for all seed entities
    expanded := make([]string, 0)
    seen := make(map[string]bool)
    for _, eid := range seedEntities {
        neighbors, err := graph.Neighbors(ctx, eid)
        if err != nil {
            continue // non-fatal: skip this entity's expansion
        }
        for _, neighbor := range neighbors {
            if !seen[neighbor] {
                seen[neighbor] = true
                expanded = append(expanded, neighbor)
            }
        }
    }

    return expanded, nil
}

Full Hybrid Search Pipeline

package search

import (
    "context"
    "fmt"
    "sync"
)

// HybridSearcher orchestrates three-channel retrieval with RRF fusion.
type HybridSearcher struct {
    FullText    FullTextIndex
    Vector      VectorIndex
    Graph       EntityGraph
    Embedder    Embedder
}

// Search runs all three channels concurrently and fuses results.
func (h *HybridSearcher) Search(ctx context.Context, query string, topK int) ([]ScoredDoc, error) {
    var (
        ftResults    []RankedDoc
        vecResults   []RankedDoc
        graphResults []RankedDoc
        ftErr, vecErr, graphErr error
        wg sync.WaitGroup
    )

    // Channel 1: Full-text search
    wg.Add(1)
    go func() {
        defer wg.Done()
        ftResults, ftErr = h.FullText.Search(ctx, query, topK*2)
    }()

    // Channel 2: Vector similarity search
    wg.Add(1)
    go func() {
        defer wg.Done()
        queryVec, err := h.Embedder.Embed(ctx, query)
        if err != nil {
            vecErr = err
            return
        }
        vecResults, vecErr = h.Vector.Search(ctx, queryVec, topK*2)
    }()

    // Channel 3: Graph-expanded search
    wg.Add(1)
    go func() {
        defer wg.Done()
        expanded, err := ExtractAndExpand(ctx, query, h.Graph)
        if err != nil || len(expanded) == 0 {
            graphErr = err
            return
        }
        // Search for graph-expanded entity terms via full-text
        expandedQuery := query + " " + strings.Join(expanded, " ")
        graphResults, graphErr = h.FullText.Search(ctx, expandedQuery, topK*2)
    }()

    wg.Wait()

    // Collect available channels (graph expansion is optional)
    channels := make([][]RankedDoc, 0, 3)
    if ftErr == nil {
        channels = append(channels, ftResults)
    }
    if vecErr == nil {
        channels = append(channels, vecResults)
    }
    if graphErr == nil && len(graphResults) > 0 {
        channels = append(channels, graphResults)
    }

    if len(channels) == 0 {
        return nil, fmt.Errorf("all search channels failed: ft=%v, vec=%v, graph=%v", ftErr, vecErr, graphErr)
    }

    fused := FuseResults(channels...)

    // Trim to requested topK
    if len(fused) > topK {
        fused = fused[:topK]
    }

    return fused, nil
}

Practical Applications

The memory system uses this hybrid search architecture for its core memory retrieval pipeline. When a user or AI agent queries for relevant memories, the system needs to surface results that match the exact query terms (keyword search), results that are semantically related but use different wording (vector search), and results that are contextually connected through entity relationships (graph expansion). A query like “what did the team decide about the migration?” retrieves memories that mention “migration” literally, memories about database schema changes (semantically similar), and memories about the team members and projects involved in the migration (graph-expanded).

The graph expansion channel is particularly valuable for memory applications because human conversations are full of implicit references. People say “the project” instead of naming it, or refer to decisions by their outcomes rather than the meeting where they were made. The entity graph bridges these implicit connections, recovering context that neither keyword nor semantic search can find alone.

The RRF fusion approach also makes the system robust to individual channel failures. If the vector index is temporarily unavailable, the system degrades gracefully to full-text plus graph-expanded search, rather than returning empty results.

Common Pitfalls

• Graph explosion on highly connected entities: Popular entities (common project names, team leads) may have hundreds of graph neighbors. Expanding all of them dilutes the search signal. Limit expansion to the top-N neighbors by edge weight (co-occurrence frequency) or apply a relevance threshold to prune low-quality edges.
• Over-weighting the graph channel: If graph expansion returns many results, it can dominate the RRF fusion. Consider weighting channels by applying a multiplier to the RRF scores or limiting the number of results each channel contributes to fusion.
• Entity recognition false positives: Simple capitalization-based entity extraction produces false positives (common words at sentence starts, abbreviations). These false entities trigger irrelevant graph expansions. Validate candidates against the known-entity set in the graph before expanding.
• Stale entity graph: If the entity graph is not updated as new memories are ingested, graph expansion returns outdated connections. Ensure the graph update pipeline runs synchronously with memory ingestion or within a bounded delay.
• Ignoring the RRF constant tuning: The k constant in RRF controls how much rank position matters. The standard value of 60 works well in general, but for memory retrieval where the top result matters much more than the 10th, a lower k (e.g., 20) may improve top-heavy precision.
• Not measuring per-channel contribution: Without metrics on how often each channel contributes unique results to the final ranking, you cannot tell if graph expansion is helping or just adding noise. Log which channels each result came from and monitor the distribution over time.