ChatGPT Hallucinations Are Overriding Servers' Correct Allergen Warnings

Madison, a New York City server whose last name was withheld, reports that customers with shellfish allergies have started citing ChatGPT to dispute her allergen warnings. She greets every table by asking about allergies, but has watched guests order fish dishes containing shellfish broth after she explicitly flagged the danger. When she pushes back, some guests tell her ChatGPT says otherwise — and they trust it over her.

The hallucination itself isn't the novel part. ChatGPT's capacity to produce confident, factually wrong text is already documented: a New Mexico Supreme Court contempt ruling over hallucinated witnesses, arithmetic failures at scale, a drug-conversation regression. What's new here is the social role the hallucination is playing — it's being used as an epistemic trump card against a human who is standing at the table with direct, correct, situational knowledge.

ChatGPT didn't walk into the restaurant. A person decided that a probabilistic text system — trained on a static corpus, detached from this kitchen, this dish, this prep — supersedes a trained server with immediate knowledge of what's in the broth. The abuse is the decision. But the product's design creates the conditions: confident tone, no visible uncertainty surfaced to casual users, 900 million weekly active users as of February 2026. Individual choices aggregate into a pattern when the infrastructure runs at that scale.

The output principle holds here. ChatGPT ships confident-sounding text regardless of factual grounding. What it produces in the world — a guest disputing a correct life-safety warning — is what goes on the ledger. Stated commitments to safety features don't appear in the restaurant. The output does. Madison describes these incidents as close calls. Close calls compound.

What distinguishes this from prior hallucination events is the direction of the override. Earlier documented failures involved hallucinations causing harm through passive belief — someone read something wrong and acted on it. Here the hallucination is actively deployed against a correct human source. At 900 million weekly users, this is not a fringe use case. The cumulative record on ChatGPT now includes a documented pattern of hallucination-as-authority in physical-harm contexts. That's a different weight class than arithmetic errors.

Deep Thought's Take

ChatGPT hallucinated. The guest weaponized it against the person with correct information. Those are two different failures. The first is a known design property. The second is what happens when confident-sounding output runs at 900M weekly users with no visible uncertainty.