AIHero

    Failure Modes

    Hallucination

    Confidently-wrong model output. Two flavors: factuality (invented facts) and faithfulness (drift from loaded context).

    Matt Pocock
    Matt Pocock

    Confidently-wrong model output. Two flavors with different causes and fixes:

    FlavorWhat goes wrongCauseFix
    FactualityInvented or wrong facts about the world — a function that doesn't exist, a wrong API signature, a fake citationParametric knowledge gaps, often past the knowledge cutoffLoad the right contextual knowledge
    FaithfulnessOutput drifts from the contextual knowledge that's loaded, the user's instructions, or the model's own prior reasoningAttention degradation; worsens in the dumb zoneClear or compact

    Next-token prediction produces fluent output whether or not the underlying fact is real — the model has no internal signal that it doesn't know something, so an invented method arrives in the same assured register as a correct one. Hallucinated code is plausible by construction: it's what the API would look like if it existed, which is exactly what makes it slip past a skim-level review and fail only when run.

    You need to know which flavor you're looking at, because the fix for one makes the other worse. Factuality means missing knowledge: the fix is adding context — the docs, the type definitions, the file. Faithfulness means the knowledge is present but losing the competition for attention: the fix is removing context. Misdiagnose faithfulness as factuality and you paste in more docs, which grows the context and makes the drift worse. When the agent gets something wrong, check whether the correct information was already in context before deciding which problem you have.

    Avoid: "hallucination" as a bare synonym for "wrong" — without naming the flavor, the term has no diagnostic value.

    Usage:

    "It hallucinated a parseAsync method on the schema."

    "Factuality or faithfulness?"

    "The method exists in the docs I pasted — it just stopped reading them after turn forty."

    "Faithfulness then. Compact and reload, don't bother adding more docs."

    Want more than vocabulary?

    Join AI Hero for practical skills, thinking on AI engineering, and resources that keep you ahead of the curve.

    I respect your privacy. Unsubscribe at any time.

    Share