Make the model look it up before it names it

The worst bug I have shipped in an agent did not crash, log, or fail a request. It returned an empty list, cheerfully, and the student on the other end assumed there was simply nothing there.

What happened

The agent generates study material scoped to a course. A student asks for flashcards on a topic, the agent works out which course and topic they mean, retrieves the relevant notes, and generates from those.

The failure was in "works out which course they mean." Given a vague request, the model would produce a course name that sounded entirely real. Something like "US Exam 2025-26". Plausible formatting, plausible year, plausible everything. It simply did not exist.

The agent then filtered retrieval by that name, matched nothing, and generated from an empty context. No exception. The name was a valid string, the filter was a valid filter, the query ran fine and matched zero rows. Every layer did exactly what it was told.

The fix is not a better prompt

My first instinct was to tell the model to be careful, which is the instinct to distrust. "Be accurate" is not a constraint, it is a wish.

What actually worked was removing the opportunity. There is a tool that lists the real course, subject and topic hierarchy, and the rule became: that tool must be called before the agent names any course, subject or topic. Not "should." Not "when unsure."

The instruction appears four separate times in the prompt, in escalating capitals, which is not elegant and I would rather it were a graph edge. But it moved the failure from silent to impossible. The model can no longer invent a course, because the only names available to it are ones it just read out of the system.

The generalisation

There is a category of agent bug where every component behaves correctly and the output is still wrong, because a hallucinated value passed through a pipeline that had no reason to question it. A made-up course name is a perfectly well-formed string. A made-up filter is a perfectly valid filter.

The defence is not validation after the fact. It is not giving the model the chance to invent the value in the first place. If something must match real data, the model should be reading that data, not recalling it.

And when a generated value gets used as a filter, an empty result should be loud. Zero rows after filtering by a model-supplied value is not an empty result, it is an unhandled error wearing an empty result's clothes.