[SL_CORE]
< CD ..||6 min read

Hallucination Detection: Systematic Mitigation in Prod

# LLM hallucinations are inevitable but manageable. Learn the detection techniques — self-consistency checks, factuality validators, source grounding, and LLM-as-judge — that reduce hallucinations in production.

[hallucination][llm][reliability][production]

Hallucinations — LLMs stating false things with confidence — are the most trust-eroding failure mode in production AI. You can't eliminate them entirely, but you can detect them reliably and reduce them to acceptable rates. Here's the systematic approach we use.

Why LLMs Hallucinate

Understanding the cause helps choose the right mitigation:

  1. Training data gaps — the model was never trained on the specific fact
  2. Pattern completion — the model generates plausible-sounding text without factual grounding
  3. Context forgetting — in long contexts, the model ignores provided facts and generates from priors
  4. Instruction-following pressure — the model is trained to always answer, so it invents rather than saying "I don't know"

Different causes require different fixes.

Detection Layer 1: Self-Consistency Checks

Sample the same query multiple times. If the model gives different answers, at least one is hallucinated.

async function selfConsistencyCheck(
  prompt: string,
  samples = 5,
  model = "claude-haiku-4-5-20251001"
): Promise<{ answer: string; confidence: number }> {
  // Generate multiple responses
  const responses = await Promise.all(
    Array.from({ length: samples }, () =>
      anthropic.messages.create({
        model,
        max_tokens: 200,
        messages: [{ role: "user", content: prompt }],
      }).then((r) => r.content[0].text)
    )
  );

  // Find the most common answer via LLM comparison
  const consensus = await anthropic.messages.create({
    model,
    max_tokens: 200,
    messages: [{
      role: "user",
      content: `These are ${samples} answers to the same question. 
Find the answer most consistently supported.
Return JSON: {"answer": "consensus answer", "agreementRate": 0.0-1.0, "conflicting": true/false}
Answers:\n${responses.map((r, i) => `[${i+1}] ${r}`).join("\n")}`,
    }],
  });

  const { answer, agreementRate } = JSON.parse(consensus.content[0].text);
  return { answer, confidence: agreementRate };
}

If agreementRate < 0.6, flag as potentially hallucinated. Best for factual questions with definitive answers.

Detection Layer 2: Source Grounding Verification

Force the model to cite sources, then verify the citation exists:

async function groundedGeneration(
  question: string,
  context: string[]
): Promise<{ answer: string; grounded: boolean; claims: Array<{ claim: string; supported: boolean }> }> {
  // Generate answer with citations
  const response = await anthropic.messages.create({
    model: "claude-sonnet-4-6",
    max_tokens: 1024,
    system: `Answer based ONLY on the provided context. 
For each factual claim, note which source it comes from as [Source N].
If information isn't in the context, say "Not found in provided context."`,
    messages: [{
      role: "user",
      content: `Context:\n${context.map((c, i) => `[Source ${i+1}] ${c}`).join("\n\n")}\n\nQuestion: ${question}`,
    }],
  });

  const answer = response.content[0].text;

  // Verify each claim is actually in the cited source
  const claimsCheck = await anthropic.messages.create({
    model: "claude-haiku-4-5-20251001",
    max_tokens: 500,
    messages: [{
      role: "user",
      content: `Verify each factual claim in this answer is supported by the cited source.
Answer: ${answer}
Context: ${context.map((c, i) => `[Source ${i+1}] ${c}`).join("\n\n")}
Return JSON: {"claims": [{"claim": "...", "citedSource": N, "supported": true/false}], "overallGrounded": true/false}`,
    }],
  });

  const { claims, overallGrounded } = JSON.parse(claimsCheck.content[0].text);
  return { answer, grounded: overallGrounded, claims };
}

Detection Layer 3: LLM-as-Judge

A second, stronger model evaluates the first model's output for factual accuracy:

const FACTUALITY_JUDGE_SYSTEM = `You are a factuality judge. 
Evaluate whether the given answer is factually accurate based on general knowledge.
Be strict — mark anything uncertain as potentially hallucinated.`;

async function judgeFactuality(
  question: string,
  answer: string
): Promise<{ score: number; issues: string[]; safe: boolean }> {
  const judgment = await anthropic.messages.create({
    model: "claude-opus-4-8",  // use strongest model as judge
    max_tokens: 300,
    system: FACTUALITY_JUDGE_SYSTEM,
    messages: [{
      role: "user",
      content: `Question: ${question}\nAnswer: ${answer}\n\nRate factual accuracy 1-10 and list any issues.
Return JSON: {"score": 1-10, "issues": ["issue1", ...], "safe": true/false}`,
    }],
  });

  return JSON.parse(judgment.content[0].text);
}

Use this for high-stakes outputs (medical, legal, financial). It adds latency and cost but catches subtle hallucinations.

Detection Layer 4: Knowledge Graph Validation

For domain-specific facts (products, APIs, internal knowledge), validate against a structured source of truth:

interface FactCheck {
  claim: string;
  verified: boolean;
  source?: string;
  correction?: string;
}

async function validateAgainstKnowledgeBase(
  claims: string[],
  knowledgeBase: Map<string, string> // fact → source
): Promise<FactCheck[]> {
  return Promise.all(
    claims.map(async (claim) => {
      // Search KB for relevant facts
      const relevant = await vectorSearch(claim, 3);
      
      const check = await anthropic.messages.create({
        model: "claude-haiku-4-5-20251001",
        max_tokens: 150,
        messages: [{
          role: "user",
          content: `Does this claim match the knowledge base facts?
Claim: "${claim}"
KB facts: ${relevant.join("\n")}
Reply JSON: {"verified": true/false, "correction": "if wrong, what's correct", "source": "relevant KB fact"}`,
        }],
      });

      const result = JSON.parse(check.content[0].text);
      return { claim, ...result };
    })
  );
}

Mitigation: Reduce Hallucinations at the Source

Detection catches hallucinations after the fact. These techniques reduce them upfront:

1. Chain-of-thought before factual claims

const GROUNDED_SYSTEM = `Before making any factual claim:
1. Check if this fact is in the provided context
2. If not, explicitly say "I don't have information about this"
3. Never infer or guess facts — only state what you can directly verify`;

2. Uncertainty quantification

Ask the model to express uncertainty:

const messages: Anthropic.MessageParam[] = [{
  role: "user",
  content: `${question}

For each factual claim in your answer, indicate your confidence:
[HIGH] = certain, directly in provided context
[MEDIUM] = likely correct but not directly stated  
[LOW] = uncertain, may need verification`,
}];

3. Refusal training (via system prompt)

const SYSTEM_WITH_REFUSAL = `You are a precise assistant. 
Rules:
- If you don't know something, say "I don't have reliable information about this"
- Never make up statistics, dates, names, or technical specifications
- If asked about something outside your knowledge, say so clearly
- It's better to say "I don't know" than to hallucinate`;

Production Hallucination Pipeline

Combine all layers for high-stakes outputs:

async function safeGeneration(question: string, context: string[]) {
  // 1. Generate with source grounding
  const { answer, grounded, claims } = await groundedGeneration(question, context);

  // 2. Check unsupported claims
  const unsupported = claims.filter((c) => !c.supported);
  if (unsupported.length > 0) {
    // Regenerate with explicit prohibition
    return regenerateWithConstraints(question, context, unsupported.map((c) => c.claim));
  }

  // 3. For high-stakes: LLM-as-judge
  if (isHighStakes(question)) {
    const { safe, issues } = await judgeFactuality(question, answer);
    if (!safe) {
      return { answer: null, error: "Could not verify accuracy", issues };
    }
  }

  return { answer, grounded, confidence: grounded ? "high" : "medium" };
}

FAQ

What's an acceptable hallucination rate for production? Depends on stakes. Customer support: <5% is acceptable. Medical/legal: near 0% required with human review. Code generation: use a linter/compiler as ground truth.

Do bigger models hallucinate less? Generally yes — Claude Opus hallucinates less than Haiku. But even the largest models hallucinate on obscure facts. Don't rely on model size alone.

Does RAG eliminate hallucinations? RAG greatly reduces them for knowledge-based questions. But the model can still hallucinate by ignoring the context, mis-quoting sources, or generating outside the provided docs. Use Agentic RAG with verification for best results.

How do I measure hallucination rate in production? Sample 1–5% of responses for human review or LLM-as-judge evaluation. Track as a dashboard metric. Set an alert at your threshold.

Should I tell users when an answer might be hallucinated? Yes — for low-confidence answers, show an "AI may be uncertain — verify important facts" disclaimer. Users appreciate transparency and it builds long-term trust.