Modern embedding systems support a growing variety of input modalities—plain text, images, audio, PDFs, HTML pages, and combinations of these in a single request. Each embedding provider (OpenAI, Cohere, Voyage, local ONNX models) has different constraints: maximum batch sizes, supported MIME types, default output dimensions, whether it can handle URLs directly versus requiring downloaded bytes, and whether it supports late-interaction or fusion embeddings. Without a systematic way to advertise and query these capabilities, application code becomes riddled with provider-specific conditionals.
An embedder capability registry solves this by defining a standard interface that every embedder implementation must satisfy. The interface declares what the embedder can do—which content types it accepts, how many items it can process per batch, what dimensionality it produces—and the registry provides helper functions that enforce these constraints at the call boundary. When application code needs to embed a mixed-media document, it queries the registry for a compatible embedder rather than hard-coding provider logic.
A typical Go SDK project implements this pattern to support pluggable embedding backends. A central EmbedderRegistry maps provider names to their capability descriptors, and a set of middleware functions validate inputs (checking MIME types, enforcing batch limits, falling back to text extraction for unsupported media) before dispatching to the underlying provider. This architecture makes it straightforward to add new embedding providers or swap them without touching application-level code.
The capability interface is a Go struct (or interface) that describes what an embedder can do. Fields include SupportedMIMETypes (a set of content types like text/plain, image/png, application/pdf), MaxBatchSize (maximum items per API call), DefaultDimensions (output vector size), SupportsFusion (whether it can embed mixed-modality inputs in a single vector), and SupportsURL (whether it accepts URLs directly or requires raw bytes).
Before sending data to an embedder, the registry’s middleware checks each input item’s MIME type against the embedder’s declared capabilities. If an item’s type is not supported, the middleware either rejects it with a clear error or invokes a fallback pipeline (e.g., extracting text from a PDF using a document parser, then sending the text to a text-only embedder). This enforcement layer prevents cryptic API errors from providers that silently ignore unsupported inputs.
When an embedder only supports text/plain but receives a PDF or HTML document, the fallback pipeline extracts text content before embedding. This involves MIME-type-specific extractors: a PDF parser for application/pdf, an HTML-to-text converter for text/html, and an OCR pipeline for images when text extraction is needed. The registry chains these extractors transparently so callers do not need to know which embedder handles which format.
Embedding APIs impose limits on how many items can be processed in a single request. The registry automatically chunks input batches to respect each provider’s MaxBatchSize, dispatches the chunks concurrently (with configurable parallelism), and reassembles the results in the original order. This batching logic lives in the registry middleware rather than in each provider implementation, eliminating duplication.
Some providers support variable output dimensions (e.g., OpenAI’s text-embedding-3-small can produce 256, 512, or 1536 dimensions). The capability descriptor advertises the default and available dimensions. When the application requires a specific dimensionality (to match an existing vector index), the registry selects the appropriate configuration or applies post-hoc dimensionality reduction if the provider does not support the exact size natively.
The registry is initialized at application startup. Each embedding provider registers itself with a name and a capability descriptor. When application code calls registry.Embed(ctx, providerName, inputs), the registry looks up the provider, runs the input through validation middleware (MIME checks, batch splitting, fallback extraction), calls the provider’s Embed method with validated and chunked inputs, and returns the concatenated results.
package embedder
// Capabilities describes what an embedding provider supports.
type Capabilities struct {
// SupportedMIMETypes lists content types this embedder accepts.
SupportedMIMETypes map[string]bool
// MaxBatchSize is the maximum number of items per API call.
MaxBatchSize int
// DefaultDimensions is the output vector dimensionality.
DefaultDimensions int
// AvailableDimensions lists all supported output sizes.
AvailableDimensions []int
// SupportsFusion indicates whether multi-modal inputs can be
// fused into a single embedding vector.
SupportsFusion bool
// SupportsURL indicates whether the provider can fetch content
// from a URL directly, avoiding a separate download step.
SupportsURL bool
}
// CanEmbed checks whether the embedder supports a given MIME type.
func (c *Capabilities) CanEmbed(mimeType string) bool {
return c.SupportedMIMETypes[mimeType]
}package embedder
import "context"
// Input represents a single item to be embedded.
type Input struct {
Content []byte
MIMEType string
URL string // optional: provider fetches directly if SupportsURL
}
// Result holds the embedding vector for a single input.
type Result struct {
Vector []float32
Tokens int // token count consumed, for billing tracking
}
// Embedder is the interface every provider must implement.
type Embedder interface {
Embed(ctx context.Context, inputs []Input) ([]Result, error)
Capabilities() *Capabilities
}
// Registry maps provider names to embedder implementations.
type Registry struct {
providers map[string]Embedder
fallbacks map[string]TextExtractor // MIME type -> text extractor
}
// NewRegistry creates an empty registry.
func NewRegistry() *Registry {
return &Registry{
providers: make(map[string]Embedder),
fallbacks: make(map[string]TextExtractor),
}
}
// Register adds a provider to the registry.
func (r *Registry) Register(name string, e Embedder) {
r.providers[name] = e
}
// RegisterFallback adds a text extractor for a MIME type.
func (r *Registry) RegisterFallback(mimeType string, ext TextExtractor) {
r.fallbacks[mimeType] = ext
}package embedder
import (
"context"
"fmt"
"sync"
)
// Embed validates inputs, applies fallbacks, splits batches, and calls the provider.
func (r *Registry) Embed(ctx context.Context, providerName string, inputs []Input) ([]Result, error) {
provider, ok := r.providers[providerName]
if !ok {
return nil, fmt.Errorf("unknown embedder: %s", providerName)
}
caps := provider.Capabilities()
// Step 1: Validate and apply fallbacks
validated := make([]Input, 0, len(inputs))
for _, inp := range inputs {
if caps.CanEmbed(inp.MIMEType) {
validated = append(validated, inp)
continue
}
// Try text extraction fallback
extractor, hasFallback := r.fallbacks[inp.MIMEType]
if !hasFallback {
return nil, fmt.Errorf("unsupported MIME type %s for provider %s", inp.MIMEType, providerName)
}
text, err := extractor.Extract(ctx, inp.Content)
if err != nil {
return nil, fmt.Errorf("fallback extraction failed for %s: %w", inp.MIMEType, err)
}
validated = append(validated, Input{Content: text, MIMEType: "text/plain"})
}
// Step 2: Split into batches
batches := splitBatches(validated, caps.MaxBatchSize)
// Step 3: Process batches concurrently
allResults := make([]Result, len(validated))
var mu sync.Mutex
var firstErr error
var wg sync.WaitGroup
offset := 0
for _, batch := range batches {
batchOffset := offset
batchCopy := batch
offset += len(batch)
wg.Add(1)
go func() {
defer wg.Done()
results, err := provider.Embed(ctx, batchCopy)
mu.Lock()
defer mu.Unlock()
if err != nil && firstErr == nil {
firstErr = err
return
}
copy(allResults[batchOffset:], results)
}()
}
wg.Wait()
if firstErr != nil {
return nil, firstErr
}
return allResults, nil
}In a production implementation, the embedder registry powers the semantic search pipeline. Documents ingested into the system can be plain text, Markdown, HTML, or PDF. The ingestion service calls registry.Embed("voyage", inputs) for production workloads and registry.Embed("local-onnx", inputs) for development and testing. The registry handles the differences transparently—Voyage supports text and images natively, while the local ONNX model only handles text, so the registry’s fallback pipeline extracts text from PDFs and HTML before embedding.
When the team added a new Cohere embedding provider with different batch limits and dimension options, they implemented the Embedder interface, registered it with the appropriate Capabilities, and the entire pipeline worked immediately. No changes to the ingestion service, search service, or batch processing jobs were required.
The registry also enables A/B testing of embedding providers. By routing a percentage of traffic to a new provider and comparing retrieval quality metrics, the team can evaluate providers in production without code changes—just configuration updates to the provider routing table.
MaxConcurrency configuration.Embed method and surface them in metrics and cost dashboards.Embedder interface in tests is convenient but hides real-world issues like rate limits, content filtering, and API version changes. Include integration tests that call the actual provider (with a small test corpus) in the CI pipeline.