Non-deterministic does not mean untestable.
AI responses can vary. Product expectations still need a stable yardstick: clear rules, meaningful test cases and a deliberate release decision.
At Webwoven, one thing was clear to me: the rules of the game had to remain understandable. AI could help build it, but whether a route was valid could not depend on a model's mood. I described that separation in “Not the model. The workflow.”. That leaves a natural question: how do I test the parts of a product whose responses are allowed to vary?
The expectation needs to be more precise than the answer
A summary can be correct in several different ways. A test accepting only one exact wording would reject useful answers. But “sounds good” is not enough either.
I would start by defining what the summary needs to achieve. Which statements must survive? Which uncertainty must it preserve? What must it never invent? This gives an open-ended task a framework that can be checked. The wording stays flexible; the expectation becomes concrete.
A small collection of difficult cases
I would begin with a manageable golden set: carefully chosen inputs with expectations reviewed by someone who understands the subject. Alongside ordinary cases, I would include incomplete information and conflicting sources. When information is missing, a clarifying question can be the right answer.
For example, suppose a system turns a handover note into a task with a deadline and an owner. If the note contains no deadline, the date field needs to stay empty. Fluent wording does not make an invented date any better. Cases like this force me to describe quality more precisely before automating its assessment.
Rules and judgement need different checks
Required fields and allowed values can be checked directly. Whether a text clearly explains an important limitation also needs informed judgement. Anthropic's article on agent evals distinguishes between checks performed by code, models and people. Model-based assessments need calibration against human assessments.
For me, this suggests a practical division of work: unambiguous rules belong in unambiguous tests. Language quality needs a defined standard and someone who regularly checks the assessments.
Releasing is a product decision
Before changing a model or prompt, I would run the same cases several times. I want to know whether important requirements are met reliably and where behaviour varies. A good average can conceal a failure that is unacceptable for the particular use case.
The golden set is still a sample. New failures from production need to expand the collection; some cases should be held back for an independent check. Otherwise I may eventually optimise for my own test.
Not every answer needs to look the same. But before releasing a change, I want to be able to explain how I know that an answer serves its purpose.