Someone wants to buy a company whose product runs on a twelve-year-old PHP application. An investor is asked to fund another round and wants to know what that means technically. A group acquires a competitor and inherits its platform. In all three cases the same question is asked: what exactly are we buying here?
And in all three cases the person asking gets the same inadequate answer. A developer spends two days reading the source and reports that it is not pretty. That is almost always true and helps nobody. Whether the code is pretty does not decide the purchase price. What decides it is the cost of the next change, and what happens when the database stops tomorrow.
This article describes how I assess a legacy system in five days and what ends up in the report. It is written for the situation where the assessment is planned, with access and time. If the developer has just resigned and you do not know how the system is deployed at all, the order is a different one; that is covered in its own article.
Three questions the report has to answer
A technical due diligence easily produces forty pages that nobody reads. To stop that happening, I write down the three questions the client actually has before I touch anything. It is always the same three.
First: what does it cost to run if we change nothing? Servers, licences, suppliers, and above all the people who keep the system alive. That number is rarely written down in full anywhere, because it is spread across three cost centres.
Second: what does the next change cost? Not any change, but the one in the business plan. A new pricing model, a second tenant, an interface to a partner. The price of that single change says more about the system than any code quality metric.
Third: what can kill us? These are the risks that are not gradual. A database whose backup has never been restored. A library with a known vulnerability and no path forward. A single person who knows how to deploy. That list is short and it belongs on page one.
Everything I do in those five days serves one of those three questions. If something serves none of them, it is interesting, but it belongs in an appendix.
Why five days and not two weeks
Five days sounds too short for a system with two hundred thousand lines. It is not, for a reason that has nothing to do with the system: the insight gained per day drops steeply after day five.
The first days produce the structural findings. Whether there is a pipeline. Whether the schema has foreign keys. Whether the dependencies are still maintained. Those are properties of the system, and you see them quickly because they look the same everywhere. After that the detailed work begins: understanding individual modules, tracing individual bugs. That work is valuable, but it does not answer any of the three questions better, only more precisely.
For a purchase decision, the order of magnitude is enough. Whether modernization takes six or eight months rarely changes the price. Whether it takes six months or three years changes everything. That difference is visible in five days.
What you need for it has to be arranged in advance, or two of the five days go on getting access: read access to the repository with its full history, a copy of the production database or at least its schema, access to the running environment, and two hours with somebody who knows the system.
Day 1: can the system be built at all?
The first day has exactly one goal: the system runs on my machine, built from what is in the repository. That sounds trivial. It is the single most informative finding of the whole assessment.
If it succeeds within a day, the system is reproducible. Then a description of the environment exists, it is correct, and somebody has used it in recent months. If it does not succeed, the most important number in the report is already known: every new developer needs the same amount of time before they can change a single line, and every change is tested directly in production.
I start with the history, not the code. It says more about the state of a project in five minutes than an hour of reading:
# Who has committed in the last two years, and how much?
git shortlog -sn --since="2 years ago"
# When was something last shipped?
git log --first-parent -20 --date=short --pretty="%ad %an %s"
# Which files change most often? That is where the work sits,
# and that is where the risk will sit later.
git log --since="2 years ago" --name-only --pretty=format: \
| sort | uniq -c | sort -rn | head -30The third query is the one I look at longest. It shows the files the business actually keeps busy. If there is a four-thousand-line file in there that has been changed three hundred times, you have found the place where every future change becomes expensive. That is not a judgement about the developer who wrote it. It is a statement about your next two years.
Then the build. What it takes is written down in a file in the best case, and lives in somebody's head in the worst: PHP version, extensions, database version, search index, message queue, a directory of files that is not in the repository. Every item on that list that is not described in code is a finding of its own.
At the end of day 1 there is a number for the report: time to the first running copy. Under four hours is good, one day is normal, more than a week is a chapter of its own.
Day 2: the database outlives every modernization
The code of a legacy system can be replaced. The schema cannot, at least not in one step, because everything hangs off it: reports, interfaces, exports, and a handful of direct queries from other systems that nobody remembers. So the database gets a full day.
Three things interest me. How big it is, how it is cut, and how it gets changed.
Size first, because it decides the migration strategy. An eight gigabyte database moves in a maintenance window. A four hundred gigabyte one does not.
-- Size per table, largest first. Also shows where the history lives.
SELECT table_name,
ROUND((data_length + index_length) / 1024 / 1024) AS mb,
table_rows
FROM information_schema.tables
WHERE table_schema = DATABASE()
ORDER BY data_length + index_length DESC
LIMIT 20;
-- Tables without a primary key. Every single one is a problem for
-- replication, for migrations and for tracing changes.
SELECT t.table_name
FROM information_schema.tables t
LEFT JOIN information_schema.table_constraints c
ON c.table_schema = t.table_schema
AND c.table_name = t.table_name
AND c.constraint_type = 'PRIMARY KEY'
WHERE t.table_schema = DATABASE() AND c.constraint_name IS NULL;Then the cut. Are there foreign keys? If not, referential integrity lives in the application, which in practice means it lives wherever somebody remembered it. For an assessment that is one of the more expensive discoveries, because it makes every data migration harder. You can no longer trust that the data fits together, you have to measure it.
And finally: how do changes get into the schema? There are three answers and they differ a lot. A migration tool with versioned steps in the repository is the good case. A directory of numbered SQL files is the normal case. "A colleague does that by hand in the evening" is the case that belongs on the risk list, because then nobody knows whether test and production share a schema. You can check that in ten minutes by exporting both and comparing them:
mysqldump --no-data --skip-comments production > /tmp/prod.sql
mysqldump --no-data --skip-comments staging > /tmp/test.sql
diff /tmp/prod.sql /tmp/test.sqlI have never seen that comparison come back empty. What matters is not whether there are differences, but how many and whether anybody can explain them.
Day 3: dependencies and their expiry dates
Day three is about how long the system may be operated in its present form at all. That is the question of end of life, and it comes with a date that is not negotiable.
Taking stock is quick:
php -v # Is this version still in security support?
composer show --direct # The packages the project itself chose
composer audit # Known vulnerabilities in everything installed
composer outdated --direct # How far behind the maintained version is it?The interesting part is the interpretation, and there are three categories that feel very different and cost very different amounts.
- Outdated but maintained. The package is two major versions behind, but it is still developed and has an upgrade guide. That is work: plannable, with a known end.
- Abandoned, with a successor. The package is no longer maintained, but somebody has taken it over or there is an accepted replacement. More expensive, because interfaces differ, but the path is clear. Zend Framework 1 is the best known case of this kind.
- Abandoned, without a successor. A package from 2014 that does exactly one thing the system needs, that nobody maintains and for which no replacement exists. That becomes in-house work, and it belongs in the report with an estimate.
For every finding in the third category I note how deep it sits. An abandoned package called from three places is a week of work. The same package, whose classes are imported in two hundred files, is a quarter. You see that with a single search, and that number is worth more in the report than the name of the package:
grep -rl "Old\Namespace" src/ | wc -lDay 4: what does nobody know any more?
Day four is the least comfortable one, because it is not technical. It is about how much knowledge of the system exists outside people's heads, and the answer decides what a handover costs.
Two measurements and one conversation.
The first measurement is test coverage, and it is almost always read wrongly. Forty per cent test coverage sounds like half of eighty. It does not work that way. What matters is whether the places where the business happens are covered. So I compare coverage against the list of most frequently changed files from day 1. If the twenty most-changed files have no tests, the overall number is meaningless, however high it is.
The second measurement is the bus factor, and it can be read off the history. For every substantial area of the system: how many people have changed anything there in the last two years, and are they still around? An area with exactly one active author is not a staffing problem, it is part of the price.
The conversation takes two hours and has three questions. What would you change first if you had the time? Which part of the system worries you? What should I know that I have not asked about? The third question produces the answers that make the meeting worth having, and it only works if everybody understands beforehand that this is not an assessment of people.
What you should not do on that day is take the documentation as evidence. A wiki with two hundred pages says nothing about whether anything current is in it. Sampling helps: open three pages, follow the steps they describe, count how many still hold.
Day 5: operations, access, the way back
The last day covers the questions that only hurt once something has happened. They are the part of the assessment that gets skipped most often, and the only part that finds risks capable of actually hitting a company.
Three of them are not negotiable.
A backup is only a backup once it has been restored. I do not ask whether backups exist, I ask when one was last restored and how long that took. If there is no answer with a date on it, the report says: restore untested. That is not a formality. A backup nobody has restored is an assumption, and I have seen backups that had been writing an empty file for months without anybody noticing.
Who can reach production? Not who should, but who can. That list is regularly longer than expected and contains former suppliers, shared accounts and a key on a machine that no longer exists. For a handover this is one of the first pieces of clean-up work and belongs in the report with an estimate.
What happens when a deployment goes wrong? Is there a way back, and has anybody taken it? A system without a way back forces every change into a maintenance window, and that limits how fast anything can change after the handover. In one project I was involved in, that was precisely what made the modernization possible: a motorcycle magazine platform with twenty years of history was rebuilt without a single day of downtime, because every step could be shipped and rolled back on its own.
On top of that come the costs. Servers, services, licences, external suppliers. For systems in the cloud a closer look pays off here, because a substantial part of the bill often comes not from load but from architecture; where that money usually sits, I have written up elsewhere.
The numbers that come out of these five days are not yet a measurement series. Which three should run permanently afterwards is covered in Three measurements before you refactor a single line.
What goes into the report
The report has four parts and fits on ten pages. Longer reports do not get read, and what does not get read changes no decision.
One page of findings. The three questions from the start, answered, with numbers. No introduction, no description of the method.
The risk list. Every risk with three entries: what can happen, how likely it is, what it costs to remove. Sorted by the product of the last two, not by technical severity. An untested restore ranks above an outdated library, even when the library lights up red in every scanner.
The price of the next change. The plan from the business case, costed against this system. This is where day 1 pays off: if the affected files are among the most-changed and have no tests, the estimate is a different one than if they sit in a quiet corner.
What I would recommend, in order. Three to five items for the first six months. In many cases the first item is not a rebuild but a pipeline or a tested backup, because without those two every further step is riskier than it needs to be.
What does not belong in the report is a grade. "Code quality: satisfactory" is not a statement anybody can derive a decision from, and it hides the fact that one part of a system can be very good and another very bad.
Anyone who has to turn the findings into a number will find the five quantities that drive the price in What a modernization costs.
Three things a due diligence does not deliver
So that the report does not promise more than it can, I say up front what it does not contain.
It does not find security holes. Five days of assessment are not a penetration test. Whatever stands out gets noted, but the absence of findings is no evidence of security.
It does not say whether the system models the business logic correctly. Whether the commission calculation is right is known by the department, not by the assessor. I can say whether it is testable and whether anybody can explain it.
It does not replace the first change. The most honest estimate comes from having somebody make a small, real change to the system and measuring how long it took. Where that is possible, I recommend it: one week of assessment plus one real change beats two weeks of assessment.
If you take one thing from this article, let it be the order. Write down the three questions first, then assess. An assessment that begins with the source code ends with a judgement about source code, and that was never the question. And if a handover follows: how an orderly project handover works is covered on its own page.
This article belongs to a series about systems that already exist. The retrospective orders every article in it by situation.
