Genkit LLM Judge Prompts

Overview

Genkit LLM Judge Prompts are code-defined evaluation prompts within the Firebase Genkit framework that use LLM providers as automated judges. Rather than writing hand-crafted heuristics to evaluate LLM output quality, this pattern delegates judgment to another LLM call with a carefully structured prompt. The judge receives the original input, the system’s output, and evaluation criteria, then returns a structured JSON response containing a pass/fail decision, a numeric score, a reasoning explanation, and extracted keywords.

In this pattern, judge prompts are defined as Go structs with typed EvalInput and EvalOutput schemas. The Genkit framework handles serialization, provider routing, and retry logic. Each judge prompt is registered as a reusable evaluator that can be composed into evaluation suites run by the evaluation runner.

The key advantage of this approach over rule-based evaluation is flexibility: judge prompts can assess subjective qualities like “helpfulness,” “factual accuracy,” or “tone appropriateness” that resist simple pattern matching. The tradeoff is cost (each evaluation requires an additional LLM call) and the meta-problem of evaluating the evaluator itself. This pattern addresses both challenges through confidence metadata that flags low-confidence judgments for human review.

Core Concepts

How It Works

The judge prompt lifecycle is: define the prompt template and IO types, register it as a Genkit prompt, wrap it in an evaluator adapter, and plug it into the evaluation runner. Each evaluation call serializes the input, invokes the LLM, parses the structured output, and returns the result.

EvalInput/EvalOutput Type Definitions

package eval

type EvalInput struct {
    Query     string `json:"query"`
    Response  string `json:"response"`
    Reference string `json:"reference,omitempty"`
    Criteria  string `json:"criteria"`
}

type EvalOutput struct {
    Pass       bool     `json:"pass"`
    Score      float64  `json:"score"`
    Reason     string   `json:"reason"`
    Keywords   []string `json:"keywords"`
    Confidence float64  `json:"confidence"`
}

Judge Prompt Definition

package eval

import (
    "context"
    "encoding/json"
    "fmt"

    "github.com/firebase/genkit/go/ai"
    "github.com/firebase/genkit/go/genkit"
)

const factualAccuracyPrompt = `You are an expert evaluator assessing the factual accuracy of an AI response.

## Input
User Query: {{.Query}}
AI Response: {{.Response}}
{{if .Reference}}Reference Answer: {{.Reference}}{{end}}

## Evaluation Criteria
{{.Criteria}}

## Instructions
Evaluate the AI response for factual accuracy. Consider:
1. Are all stated facts correct?
2. Are there any hallucinated claims?
3. Is the response consistent with the reference answer (if provided)?

Respond with a JSON object matching this exact schema:
{
  "pass": true/false,
  "score": 0.0 to 1.0,
  "reason": "explanation of your judgment",
  "keywords": ["key", "quality", "signals"],
  "confidence": 0.0 to 1.0
}`

type FactualAccuracyJudge struct {
    model  ai.Model
    prompt *genkit.Prompt[EvalInput, EvalOutput]
}

func NewFactualAccuracyJudge(g *genkit.Genkit, model ai.Model) (*FactualAccuracyJudge, error) {
    prompt, err := genkit.DefinePrompt(g, "factual-accuracy-judge",
        ai.WithPromptText(factualAccuracyPrompt),
        ai.WithInputType[EvalInput](),
        ai.WithOutputType[EvalOutput](),
        ai.WithModel(model),
        ai.WithConfig(&ai.GenerationCommonConfig{
            Temperature: 0.0, // deterministic judging
        }),
    )
    if err != nil {
        return nil, fmt.Errorf("define prompt: %w", err)
    }

    return &FactualAccuracyJudge{model: model, prompt: prompt}, nil
}

Evaluator Adapter

package eval

import (
    "context"
    "fmt"
)

func (j *FactualAccuracyJudge) Name() string {
    return "factual-accuracy"
}

func (j *FactualAccuracyJudge) Evaluate(
    ctx context.Context, ex Example, out Output,
) (EvalResult, error) {
    input := EvalInput{
        Query:    fmt.Sprintf("%v", ex.Input["query"]),
        Response: out.Response,
        Criteria: "Assess factual accuracy. Flag any hallucinated or unsupported claims.",
    }

    // Check for reference answer
    if ref, ok := ex.Input["reference"]; ok {
        input.Reference = fmt.Sprintf("%v", ref)
    }

    // Invoke the judge prompt
    resp, err := j.prompt.Execute(ctx, input)
    if err != nil {
        return EvalResult{}, fmt.Errorf("judge prompt execution: %w", err)
    }

    evalOut := resp.Output()

    return EvalResult{
        Pass:     evalOut.Pass,
        Score:    evalOut.Score,
        Reason:   evalOut.Reason,
        Keywords: evalOut.Keywords,
    }, nil
}

func (j *FactualAccuracyJudge) MinConfidence() float64 {
    return 0.7
}

Composing Multiple Judges into a Suite

package eval

import (
    "github.com/firebase/genkit/go/ai"
    "github.com/firebase/genkit/go/genkit"
)

func BuildEvalSuite(g *genkit.Genkit, capableModel, cheapModel ai.Model) ([]Evaluator, error) {
    factual, err := NewFactualAccuracyJudge(g, capableModel)
    if err != nil {
        return nil, err
    }

    // Tone and helpfulness use a cheaper model
    tone, err := NewToneJudge(g, cheapModel)
    if err != nil {
        return nil, err
    }

    format, err := NewFormatComplianceJudge(g, cheapModel)
    if err != nil {
        return nil, err
    }

    return []Evaluator{factual, tone, format}, nil
}

Practical Applications

LLM judge prompts are the primary mechanism for evaluating RAG (Retrieval-Augmented Generation) pipeline quality. The factual accuracy judge checks whether retrieved context was faithfully used in the response. A separate “groundedness” judge verifies that every claim in the response can be traced back to a retrieved document, catching hallucinations where the model invents plausible-sounding facts.

The confidence metadata powers a human-in-the-loop review workflow. When a judge reports confidence below 0.7, the example is routed to a review queue where a human annotator makes the final pass/fail decision. This is especially common for ambiguous cases where the evaluation criteria are subjective (e.g., “is the tone appropriate for a technical audience?”). Over time, the human annotations are fed back as few-shot examples in the judge prompt, improving its calibration.

Provider model routing enables cost-efficient evaluation suites. Teams typically run factual accuracy evaluations (which require strong reasoning) on a capable model, while format compliance checks (which are mechanical) run on a fast, cheap model. A typical suite of 5 evaluators across 500 examples costs roughly $2–5 when mixing model tiers, compared to $15–25 if all evaluators used the most capable model.

Common Pitfalls

• Judge prompt sensitivity to wording: Small changes in the evaluation criteria or prompt structure can significantly shift pass rates. Always version-control judge prompts and track pass rate changes when prompts are modified. Run A/B comparisons on a held-out set before deploying prompt changes to production evaluation suites.
• Temperature drift in judging: Using temperature > 0 for judge calls introduces randomness—the same input may pass on one run and fail on the next. Always set temperature to 0 for judge prompts to ensure deterministic, reproducible evaluations. If you want diversity in judgments, run multiple judge calls and aggregate rather than relying on temperature sampling.
• Circular evaluation: Using the same model to generate responses and judge them creates a blind spot: the model is unlikely to catch its own systematic biases. Use a different model family or provider for judging than for generation, or supplement LLM judges with rule-based checks for known failure modes.
• JSON parsing failures on edge cases: Despite structured output instructions, LLMs occasionally produce invalid JSON (unclosed strings, trailing commas, or wrapping JSON in markdown). Genkit’s retry-with-correction handles most cases, but set a retry limit (typically 2) to avoid infinite loops when the model consistently fails to produce valid output.
• Cost explosion from evaluation-heavy suites: Each judge prompt adds one LLM call per example per evaluator. A suite of 10 judges across 1,000 examples is 10,000 LLM calls. Budget evaluation costs explicitly, use cheaper models for mechanical checks, and consider sampling strategies (evaluate 10% of examples) for expensive judges during development iterations.
• Ignoring confidence signals: Treating all judge outputs as equally reliable regardless of confidence leads to noisy evaluation metrics. Low-confidence judgments should be excluded from aggregate statistics or weighted proportionally. If more than 20% of judgments are low-confidence, the judge prompt likely needs refinement.