Glossary
LLM evaluation
Also: Evals
A repeatable procedure for measuring whether answers from a language model are getting better or worse.
Without evaluation, every change to an AI system is a guess. A new prompt feels better, a new model sounds more fluent, and whether the hit rate went up or down nobody knows. It only surfaces when users complain, and by then several changes sit in between.
The core is a fixed test set: real cases with a known correct answer. It does not have to be large. Fifty well chosen examples covering the typical and the difficult cases are worth more than a thousand random ones. What matters is that they are frozen, otherwise you measure against a moving target.
How you measure depends on the task. With classification and extraction there is one correct answer, so you compute accuracy, precision and recall. With freely worded answers a second model as a judge helps, with its verdict calibrated against a sample of human ratings.
With RAG systems the chain is measured in parts: was the right excerpt found, was it ranked high enough, and did the model use it correctly. A single overall score hides where the problem sits and then leads to treating the wrong one.
Two numbers always belong alongside, because otherwise they get traded away: the cost per operation and the response time. A prompt that lifts quality by two points and doubles the cost is not an unambiguous improvement.
The evaluation belongs in the pipeline, not in a notebook. If it only runs when somebody remembers it, it stops running after three weeks and the test set quietly goes stale.
How you notice it
- Prompt changes are checked against a few random examples.
- After a model change nobody knows whether quality went up.
- Complaints from users are the first source of quality problems.
- There is no number for cost and response time per operation.
Not to be confused with
- Benchmarks
- Compare models on public tasks. They say little about how a model performs in your use case, because your data and your prompt are missing.
- Monitoring
- Watches the running operation: error rates, latency, cost. Evaluation checks quality against known correct answers, mostly before rollout.
- A/B test
- Measures effect on real users, takes longer and needs volume. Evaluation gives an answer in minutes and catches the coarse regressions beforehand.
When it fits
- As soon as more than one person works on the prompt or on the chain.
- Before every model change, including a new version of the same model.
- When answers have operational or financial effect.
When it does not
- In the first exploratory phase, as long as the task itself is still changing.
- As a substitute for observation in production. A test set only knows the cases somebody put into it.
How to approach it
- Build the test set from real casesFifty to two hundred examples from actual usage, with the correct answer. Deliberately include the edge cases the system has failed on so far.
- Pick the metric to match the taskAccuracy and precision for fixed answers, a judge model with calibrated scoring for free text. A score without a defined metric is a feeling with a decimal place.
- Measure the chain in partsWith RAG, measure search, ranking and wording separately. Otherwise you know it got worse but not where.
- Carry cost and time alongEvery measurement records price per operation and response time. Otherwise quality gets optimized at the expense of the other two.
- Put it in the pipelineAutomatically on every change to prompt, chain or model version. A regression beyond a threshold blocks the rollout.
- Feed failures back inEvery reported failure becomes a new test case. That way the set grows by exactly the cases that go wrong in reality.
Frequently asked
How large does a test set have to be?
Smaller than most people think. Fifty to two hundred well chosen cases are enough to detect coarse regressions reliably. More important than the quantity is the selection: typical cases, edge cases, and the ones the system has already failed on.
Can one model rate another model?
Yes, and for freely worded answers that is often the only practical route. What matters is the calibration: rate a sample by hand and check whether the judge model comes to similar verdicts. Without that comparison you are measuring the opinion of one model about another.
How often should the evaluation run?
On every change to prompt, chain, knowledge base or model version, automatically in the pipeline. In addition regularly without a change, because provider models can shift without the version number revealing it.
What do you measure with a RAG system?
Three things separately: whether the right excerpt was found, whether it ranked high enough, and whether the answer used it correctly. Only that separation tells you whether the work belongs in the search, in the ranking or in the prompt.