Glossary
Characterization tests
Also: Golden master
Tests that record what a system does today, rather than checking what it should do. The safety net before the first change.
The difference to ordinary tests is the direction. A normal test checks an expectation: this input should produce this result. A characterization test records what actually comes out, with no judgement about whether it is right. It documents current behaviour.
That sounds like a compromise and it is exactly the right thing when nobody knows any more what a system is supposed to do. In grown systems that is the normal case: there is no specification, the requirements are twenty years old, and the code is the only reliable source on what happens.
In practice they are created by sending real inputs through the system and pinning the outputs down as the expected result. For larger outputs, whole documents or JSON structures for example, you compare against a stored reference file, which is where the name golden master comes from.
The value shows itself at the first refactoring. Without this net every restructuring is a bet that you understood everything. With it an unintended change in behaviour becomes visible immediately, and at the place where it was caused, not three weeks later in production.
The tests are explicitly disposable. Once an area has been modernized and given real tests, they may go. They are scaffolding for the rebuild, not a permanent quality instrument.
One trap: they cement bugs as well. If you find an obviously wrong behaviour while writing them, you still record it first and raise it separately for a decision. Correcting it on the spot mixes safety net and change, and that destroys exactly the benefit.
How you notice it
- An area needs to change but has no tests.
- There is no specification, or it is visibly out of date.
- Nobody can say reliably what the system does in edge cases.
- Earlier changes led to surprises in seemingly unrelated areas.
Not to be confused with
- Unit test
- Checks an expectation against a small unit. It assumes you know what is right. That knowledge is precisely what is missing in legacy code.
- Integration test
- Checks the interplay against defined expectations. Characterization tests often sit at the same level, but without an expectation: they write down what is.
- Snapshot test
- Technically the same principle, usually applied to interfaces. The word characterization stresses the purpose: securing an unknown system before the rebuild.
When it fits
- Before the first change to an area without tests.
- When the intended behaviour is undocumented and nobody knows it for certain.
- Before a replacement, so the new system can be checked against the same behaviour.
When it does not
- In areas that already have meaningful tests.
- As a permanent substitute for real tests. They are scaffolding, not a foundation.
- For code that will be switched off without replacement.
How to approach it
- Collect real inputsFrom logs, from the database, from live traffic. Invented examples miss the edge cases, and the edge cases are the reason for the exercise.
- Pin outputs down without judging themWhatever comes out becomes the expected result, even when it looks odd. The assessment comes later and separately.
- Note oddities, do not fix themA list of the places that look wrong, with where they were found. That list is the basis for a business conversation, not for an immediate fix.
- Align coverage with the changeDo not secure the whole system, secure the area you are touching plus its edges. Completeness is not a goal here.
- Put them in the pipelineThe test only helps if it runs on every change. A script executed locally is forgotten after two weeks.
- Replace them after the modernizationAs soon as real tests exist, the characterization tests may go. Maintaining them permanently cements behaviour nobody wants any more.
Frequently asked
What if the recorded behaviour is wrong?
Record it anyway and note it separately. The net is there to make unintended changes visible, not to guarantee correctness. Whether a behaviour gets corrected is a business decision with its own consequences, and sometimes the answer is: it stays, because customers have built on it.
How much coverage do you need?
As much as the planned intervention touches, plus the edges. Spanning a complete net across a large legacy system takes months and never finishes. The tests are written where the work happens next.
Do characterization tests replace real tests?
No, they bridge. Once an area is modernized and it is clear what it should do, proper tests with real expectations belong there. The characterization tests may then disappear, otherwise they cement behaviour that has just been abolished.
Does this work without a test framework?
It also works with a script that sends inputs through and writes outputs into files under version control. All that matters is that it runs automatically on every change. A comparison somebody has to start by hand does not happen under deadline pressure.