AI Engine Brand Mention Monitoring Pipeline

Overview

As large language models and AI search engines become primary channels through which users discover products and services, brand visibility in AI-generated responses has become a critical marketing metric. Traditional SEO monitors web search rankings, but AI engines like ChatGPT, Perplexity, and Google Gemini synthesize answers from multiple sources — a brand might appear, be misrepresented, or be entirely absent from these responses. Monitoring brand mentions across AI engines requires a fundamentally different approach: you must send prompts to each engine, capture the generated response, and analyze the text for brand presence, sentiment, and competitive positioning.

The GEO (Generative Engine Optimization) project implements a brand mention monitoring pipeline as a Next.js service. It fans out configurable prompts to multiple AI engines simultaneously, stores each response in a database, runs keyword-based brand detection combined with sentiment heuristics, and persists structured results — prompt_checks and brand_mentions records — for downstream dashboards and alerting. The pipeline runs on a scheduled cadence, enabling longitudinal tracking of how brand visibility changes over time as AI models are retrained and competitors adjust their strategies.

This topic covers the architecture of a multi-engine prompt fanout system, response parsing and brand detection, sentiment analysis heuristics, data modeling for longitudinal brand tracking, and the operational considerations of running automated queries against third-party AI APIs.

Core Concepts

How It Works

The GEO monitoring pipeline operates in four stages: prompt configuration (defining what to ask), engine fanout (sending prompts to AI APIs), response analysis (detecting brands and scoring sentiment), and persistence (storing results for dashboards and alerting).

Data Model

// prisma/schema.prisma — Core tables for the monitoring pipeline

model PromptCheck {
  id          String   @id @default(cuid())
  promptText  String   @db.Text
  engine      String   // "chatgpt", "perplexity", "google_ai"
  response    String   @db.Text
  status      String   @default("pending") // "pending", "completed", "failed"
  latencyMs   Int?
  tokenCount  Int?
  costCents   Int?     // estimated API cost in cents
  checkedAt   DateTime @default(now())
  createdAt   DateTime @default(now())
  updatedAt   DateTime @updatedAt

  brandMentions BrandMention[]

  @@index([engine, checkedAt])
  @@index([status])
}

model BrandMention {
  id            String   @id @default(cuid())
  promptCheckId String
  promptCheck   PromptCheck @relation(fields: [promptCheckId], references: [id])
  brand         String
  mentioned     Boolean
  sentiment     String?  // "positive", "negative", "neutral"
  position      Int?     // character offset in response
  context       String?  @db.Text // surrounding text window
  confidence    Float?   // detection confidence 0.0–1.0
  createdAt     DateTime @default(now())

  @@index([brand, createdAt])
  @@index([promptCheckId])
}

Engine Fanout Service

// src/services/engineFanout.ts
import { PromptCheck } from "@prisma/client";

interface EngineClient {
  name: string;
  query(prompt: string): Promise<{ text: string; tokens: number; latencyMs: number }>;
}

export class EngineFanout {
  private engines: EngineClient[];

  constructor(engines: EngineClient[]) {
    this.engines = engines;
  }

  async fanout(prompt: string): Promise<EngineResult[]> {
    // Send prompt to all engines concurrently.
    const results = await Promise.allSettled(
      this.engines.map(async (engine) => {
        const start = Date.now();
        try {
          const response = await engine.query(prompt);
          return {
            engine: engine.name,
            status: "completed" as const,
            response: response.text,
            tokenCount: response.tokens,
            latencyMs: response.latencyMs,
          };
        } catch (error) {
          return {
            engine: engine.name,
            status: "failed" as const,
            response: "",
            tokenCount: 0,
            latencyMs: Date.now() - start,
            error: error instanceof Error ? error.message : "Unknown error",
          };
        }
      })
    );

    return results.map((r) =>
      r.status === "fulfilled" ? r.value : r.reason
    );
  }
}

interface EngineResult {
  engine: string;
  status: "completed" | "failed";
  response: string;
  tokenCount: number;
  latencyMs: number;
  error?: string;
}

Brand Detection & Sentiment Analysis

// src/services/brandDetector.ts

interface BrandConfig {
  name: string;
  aliases: string[];     // alternative spellings, abbreviations
  competitors: string[]; // track competitive mentions
}

interface DetectedMention {
  brand: string;
  mentioned: boolean;
  sentiment: "positive" | "negative" | "neutral";
  position: number;
  context: string;
  confidence: number;
}

const POSITIVE_SIGNALS = [
  "recommended", "best", "leading", "powerful", "fast",
  "reliable", "excellent", "top", "popular", "preferred",
];
const NEGATIVE_SIGNALS = [
  "slow", "outdated", "expensive", "limited", "worst",
  "avoid", "deprecated", "complicated", "unreliable", "lacking",
];
const NEGATION_WORDS = ["not", "no", "never", "hardly", "barely", "isn't", "doesn't"];

export function detectBrands(
  response: string,
  brands: BrandConfig[]
): DetectedMention[] {
  const mentions: DetectedMention[] = [];
  const lowerResponse = response.toLowerCase();

  for (const brand of brands) {
    const allTerms = [brand.name, ...brand.aliases];
    let found = false;

    for (const term of allTerms) {
      const lowerTerm = term.toLowerCase();
      let searchFrom = 0;

      while (true) {
        const idx = lowerResponse.indexOf(lowerTerm, searchFrom);
        if (idx === -1) break;

        found = true;
        // Extract context window (100 chars each side).
        const ctxStart = Math.max(0, idx - 100);
        const ctxEnd = Math.min(response.length, idx + term.length + 100);
        const context = response.slice(ctxStart, ctxEnd);

        const sentiment = scoreSentiment(context);
        mentions.push({
          brand: brand.name,
          mentioned: true,
          sentiment,
          position: idx,
          context,
          confidence: 1.0,
        });

        searchFrom = idx + term.length;
      }
    }

    // Record absence — brand not mentioned at all.
    if (!found) {
      mentions.push({
        brand: brand.name,
        mentioned: false,
        sentiment: "neutral",
        position: -1,
        context: "",
        confidence: 1.0,
      });
    }
  }

  return mentions;
}

function scoreSentiment(context: string): "positive" | "negative" | "neutral" {
  const lower = context.toLowerCase();
  const words = lower.split(/\s+/);

  let score = 0;
  for (let i = 0; i < words.length; i++) {
    const word = words[i].replace(/[^a-z]/g, "");
    const isNegated = i > 0 && NEGATION_WORDS.includes(words[i - 1].replace(/[^a-z']/g, ""));

    if (POSITIVE_SIGNALS.includes(word)) {
      score += isNegated ? -1 : 1;
    }
    if (NEGATIVE_SIGNALS.includes(word)) {
      score += isNegated ? 1 : -1;
    }
  }

  if (score > 0) return "positive";
  if (score < 0) return "negative";
  return "neutral";
}

Scheduled Monitoring Job

// src/jobs/runMonitoringCycle.ts
import { prisma } from "../lib/prisma";
import { EngineFanout } from "../services/engineFanout";
import { detectBrands } from "../services/brandDetector";
import { brandConfig, promptTemplates } from "../config/monitoring";

export async function runMonitoringCycle() {
  const fanout = new EngineFanout(getConfiguredEngines());

  for (const prompt of promptTemplates) {
    const results = await fanout.fanout(prompt.text);

    for (const result of results) {
      // Persist the prompt check.
      const check = await prisma.promptCheck.create({
        data: {
          promptText: prompt.text,
          engine: result.engine,
          response: result.response,
          status: result.status,
          latencyMs: result.latencyMs,
          tokenCount: result.tokenCount,
        },
      });

      if (result.status === "completed") {
        // Run brand detection on the response.
        const mentions = detectBrands(result.response, brandConfig);

        // Persist each detected mention.
        await prisma.brandMention.createMany({
          data: mentions.map((m) => ({
            promptCheckId: check.id,
            brand: m.brand,
            mentioned: m.mentioned,
            sentiment: m.sentiment,
            position: m.position,
            context: m.context,
            confidence: m.confidence,
          })),
        });
      }
    }
  }
}

Practical Applications

The GEO monitoring pipeline provides three primary capabilities for a product or service. First, it tracks brand visibility across AI engines — answering questions like “when someone asks ChatGPT for the best time-series database, does our product appear in the response?” This baseline visibility metric, tracked over time, reveals whether content marketing and documentation efforts are improving the brand’s presence in AI-generated recommendations.

Second, the pipeline enables competitive intelligence. By tracking mentions of competing products alongside the monitored brand, the marketing team can see exactly how AI engines position the brand relative to competitors. A shift where a competitor starts appearing in responses where the monitored brand was previously mentioned alone signals a competitive threat worth investigating.

Third, the sentiment analysis identifies reputation risks early. If AI engines start associating the brand with negative sentiment — perhaps due to a widely-discussed outage or a critical blog post that made it into training data — the pipeline surfaces this before it’s noticed in sales conversations. Early detection enables proactive response: publishing correction content, updating documentation, or engaging with the AI engine providers.

Typical Monitoring Prompts

• “What are the best vector databases for production use?” — tracks visibility in the vector DB category.
• “Compare ExampleDB vs Elasticsearch for log analytics” — monitors direct competitive positioning.
• “What database should I use for real-time analytics?” — tracks category presence.
• “What are common problems with ExampleDB?” — monitors negative sentiment and known issues.
• “Recommend a database for a SaaS application with multi-tenancy” — tracks feature-specific visibility.

Implementation Patterns

Engine Adapter Pattern

Each AI engine has a different API shape, authentication mechanism, and response format. Use the adapter pattern to normalize these differences behind a common interface. Each adapter handles authentication, request formatting, response parsing, and error mapping for its engine. New engines can be added by implementing the adapter interface without modifying the fanout or analysis logic.

// src/engines/chatgpt.ts
import OpenAI from "openai";

export class ChatGPTEngine implements EngineClient {
  name = "chatgpt";
  private client: OpenAI;

  constructor(apiKey: string) {
    this.client = new OpenAI({ apiKey });
  }

  async query(prompt: string) {
    const start = Date.now();
    const completion = await this.client.chat.completions.create({
      model: "gpt-4o",
      messages: [{ role: "user", content: prompt }],
      max_tokens: 2048,
    });

    const text = completion.choices[0]?.message?.content ?? "";
    return {
      text,
      tokens: completion.usage?.total_tokens ?? 0,
      latencyMs: Date.now() - start,
    };
  }
}

// src/engines/perplexity.ts
export class PerplexityEngine implements EngineClient {
  name = "perplexity";
  private apiKey: string;

  constructor(apiKey: string) {
    this.apiKey = apiKey;
  }

  async query(prompt: string) {
    const start = Date.now();
    const resp = await fetch("https://api.perplexity.ai/chat/completions", {
      method: "POST",
      headers: {
        Authorization: `Bearer ${this.apiKey}`,
        "Content-Type": "application/json",
      },
      body: JSON.stringify({
        model: "llama-3.1-sonar-large-128k-online",
        messages: [{ role: "user", content: prompt }],
      }),
    });

    const data = await resp.json();
    return {
      text: data.choices[0]?.message?.content ?? "",
      tokens: data.usage?.total_tokens ?? 0,
      latencyMs: Date.now() - start,
    };
  }
}

Cost Tracking & Circuit Breaker

Track the estimated cost of every API call. Each engine adapter estimates cost based on token count and the engine’s per-token pricing. Accumulate costs per monitoring cycle and enforce a budget ceiling — if the cumulative cost exceeds the configured limit, the circuit breaker trips and remaining prompts are skipped. This prevents runaway costs from a misconfigured prompt list or unexpected API pricing changes.

Idempotent Check Runs

Each monitoring cycle generates a batch ID (date + cycle number). If a cycle is interrupted and retried, previously completed prompt-engine combinations (identified by the batch ID, prompt hash, and engine name) are skipped. This prevents duplicate entries in the database and avoids wasting API quota on already-collected responses.

Common Pitfalls

Non-Deterministic AI Responses

AI engines return different responses for the same prompt on different days (or even different minutes). A brand that appears in one response might be absent in the next. Don’t treat a single missing mention as a signal — use rolling averages over multiple checks. A brand’s “mention rate” (percentage of checks where it appears) is more meaningful than any single binary result. Set alerting thresholds on the rolling average, not individual checks.

Terms of Service Violations

Automated querying of AI engine APIs may violate their terms of service, especially at high volumes or for competitive monitoring purposes. Review each engine’s ToS and acceptable use policy before deploying the pipeline. Some engines offer dedicated API tiers for monitoring use cases; others explicitly prohibit automated benchmarking. Violations can result in API key revocation or legal action.

Keyword Matching False Positives

Simple substring matching produces false positives when the brand name is a common word or part of another word. “Fly” matching inside “butterfly”, or “DB” matching every database reference. Use word boundary detection, require minimum surrounding context, and maintain a stop-list of known false positive patterns. Consider fuzzy matching thresholds carefully — too loose catches noise, too strict misses legitimate variations.

Ignoring Response Structure

AI responses have implicit structure — numbered lists, comparisons, pros/cons sections. A brand mentioned first in a recommendation list is positioned more favorably than one mentioned last. Brands in a “cons” section have different sentiment than those in a “pros” section. Flat text analysis misses these structural signals. Parse response structure where possible to weight mentions by position and context.

Stale Prompt Lists

The prompts that matter change as the market evolves. Prompts about “best NoSQL databases” might have been relevant last year, but the market conversation has shifted to “best vector databases for RAG.” Review and update the prompt list quarterly. Add new prompts that reflect emerging market categories, and retire prompts that no longer match real user queries.

Further Reading