Resilience
Resilience is not a setting you switch on, and it is not a second availability zone in your Terraform. It is a property that shows itself in how a system behaves when part of it stops answering. That is where I work: make the dependencies visible, walk the recovery path once for real, and leave the changes behind as code so they survive the next deployment.
Remote, directly with me. Assessment and rehearsal as a bounded two-week engagement at a fixed price, implementation afterwards according to scope. On-call and day-to-day operations stay with your team, which is why the handover is part of the work.
Remote from Germany. Straight with me, no agency in between.
The starting point
The most common finding has nothing to do with missing hardware: the second availability zone is there, but the path to it has never been walked. A database with Multi-AZ fails over when AWS fails it over. Whether the application survives that depends on connection pools, timeouts and whether it notices the switch at all. Until somebody has tried it, Multi-AZ is a line on the bill rather than a promise.
The second finding is the control plane, and there is a textbook case for it. On 20 October 2025 an outage in us-east-1 began with a latent race condition in the automated DNS management of DynamoDB. Because internal AWS services build on it themselves, more than 140 services were affected in the end, and full recovery took around fifteen hours. It was not a datacentre fire: no second datacentre helps against a dependency in the control plane. Only one question does, namely what your application does when a service stops answering, and how much of its job it can still do without it.
The third finding sits in the backup policy. Almost every company has backups, RTO and RPO are written down somewhere, and almost nobody has ever restored into an empty environment while watching the clock. That single exercise is where the gaps surface: a key, a piece of configuration, an order in which services must start, a DNS record with a long time to live. A recovery plan that has never been executed is a claim, and claims fail during incidents.
Does any of this sound familiar?What I do
The map comes first: which services, databases, queues and third parties hang off which path, and what happens when one of them stops answering. That includes the invisible dependencies, such as DNS, certificates, the control plane of individual AWS services, an identity provider, a single payment provider. The result is a list of single points of failure ranked by severity rather than alphabetically.
Spreading across availability zones is the cheapest win when it is done properly: stateless services in at least two zones, databases with a rehearsed switch, load balancing with health checks that check the right state rather than just the port. Alongside that, autoscaling that is as careful scaling in as scaling out, because most self-inflicted outages happen while shrinking.
Partial failure is the normal case, not the exception. So it gets decided: timeouts on every call, retries with jitter rather than in lockstep, a breaker that cuts off a broken dependency for a while, queues as a buffer and idempotent processing so a retry does nothing twice. Whatever cannot work works in a reduced form instead of not at all.
A restore into an empty environment with a clock next to it: that produces the real values for RTO and RPO, and the list of what the policy is missing. Alongside it a rehearsed failover on a live service. Once that runs twice without surprises, your recovery plan has evidence for the first time.
After that we rehearse rather than wait: remove an instance, make a zone unreachable, make a dependency slow. The AWS Fault Injection Service does this with a bounded blast radius and a stop condition, and the Resilience Hub suggests matching scenarios. First on a staging tier, then under control in production, and only when the observability is strong enough to abort the experiment early.
Every change lives as Terraform in the repository so it survives the next deployment. Alongside it a runbook per scenario, the measured values in writing, and a handover session with your team. On-call stays with you. That is not a saving, it is the goal: during an incident the knowledge has to be inside the company.
When did you last execute your recovery plan?
Tell me what happened during your last incident and how long it took. You get an assessment before you commission anything.
How it runs
Four steps. After the second one you know where you stand, with numbers. No step assumes you will commission the next one.
I read the infrastructure, the dependencies and the incident history. Along with the question that orders everything else: which business path may be unavailable, and for how long. Read access is enough.
A restore and a failover are carried out and measured. The result is your real RTO and RPO, the list of single points of failure by severity, and the effort per measure. That lets you decide, including against me.
Ordered by benefit, one slice at a time, each useful on its own and reversible on its own. All of it as Terraform, so it stays.
A failure triggered under control with your team watching, then the handover: runbooks, ownership, the measured values in writing.
Entry offer
Before anyone buys redundancy, it should be clear what actually happens today. That is what this package does: the recovery path is walked once, measured and written down, at a fixed price, with a result you can keep working from without me.
What you get
What you do not get
The outcome
One service fails and the rest keeps working, reduced rather than gone. That is the whole difference between a disruption and an outage with an audience.
You can state both numbers without opening a document, and you know where they came from. That also gives you something to negotiate with when a customer asks for more than is possible today.
Anyone who has done a restore twice does it for the third time during the incident. That is the difference between two hours and two days.
Resilience costs money, but not double. What lies between a second zone and a second region gets decided rather than bought, with the price per step written next to it.
Technologies I use
From practice
Revenue reporting in the nine-figure range, grown code without tests, no documented infrastructure, regular incidents. A test net, infrastructure as code and recovery came together, and the rebuild ran while the system kept running. Afterwards incidents were 90 per cent lower.
Read the case studyAlso from real projects
With payments every outage is revenue directly. Go services were placed next to the existing system, integrated over gRPC and instrumented, without stopping operations: 15 per cent fewer payment drop-offs, 75 per cent shorter deployments.
Read the case studyWhat actually matters when a region fails: which multi-region strategies exist, what a realistic failover costs, and why for most applications the answer sits below multi-region.
Read the article
Who you are talking to
I am Tim Rutte. More than 20 years in software development, and the most expensive outages were never the most complicated ones. They were the ones nobody had rehearsed. You talk to the person who touches your code, from the first call to the handover.
Working together
You do not need to bring a particular setup. I adapt the engagement to how your company works and how much responsibility you want to hand over.
If knowledge and responsibilities already sit with you, I join wherever additional experience is needed. Inside your workflows, in direct contact, without creating a parallel track.
If you lack time or capacity internally, I take a clearly scoped piece of work from technical clarification through to production. You set the objective and constraints; I take care of the path there.
Whichever model you choose, you always know what is being built, which decisions are pending and what happens next.
Continuity
Independent delivery does not create dependency on me. Everything required to develop and operate the result stays in your environment from day one and remains ready for handover throughout the engagement.
Code, cloud accounts, pipelines and secrets live in your systems. Operations never depend on a personal account or credentials that only I control.
Architecture decisions, operating procedures and known risks are documented where your team will find them and kept current as the work progresses.
Reproducible environments, automated deployments and regular handovers allow your team or another provider to continue without starting again.
The goal is not for you to depend on me permanently. The goal is for you to remain free to decide who develops the system next.
Common questions
Probably not, and that answer often saves you a six-figure sum. Multi-region means holding data twice, resolving conflicts, rehearsing the switch regularly and operating everything twice. It is right when an outage of hours threatens the business or a contract demands it. For most applications the bigger win sits below that: distribution across zones that has genuinely been rehearsed, a recovery with measured values, and behaviour under partial failure that does not take everything down. The article on regional outages works through the numbers.
No, and the two pages still belong together. Observability answers whether and where something is broken, which is seeing. This page answers what happens while it is broken, which is surviving. A system can be exemplary instrumented and still be fragile, and it can be robust without anyone noticing. On order of work: without a minimum of visibility no outage can be rehearsed under control, so sometimes I start there.
With a bounded blast radius and a stop condition, and in this order: first on a staging tier, then under control in production, and only when the observability is strong enough to abort early. The AWS Fault Injection Service makes exactly that plannable, because an experiment declares up front what it touches, for how long, and which condition stops it. No experiment in production without your explicit approval and a window you choose.
A backup is half the answer; the other half is the restore, and that rarely gets practised. The rehearsal regularly surfaces what the policy does not mention: a key that lived in the account you just lost, an order in which services have to start, configuration that only ever existed on the old machine, a DNS record with a long time to live. That is why the executed restore is the core of this offer rather than a line in a report.
No, and that is a decision rather than a capacity question. I build it, rehearse it with you and hand it over. Resilience whose knowledge lives outside the company is in exactly the wrong place during an incident. If you need permanent on-call, you need a team or a provider with shift cover, and I am neither.
Less than its reputation suggests, once the steps are separated. Spreading stateless services across availability zones costs almost nothing; for databases it roughly doubles the storage. What gets expensive is the second region, and that is rarely necessary. So the findings state the monthly cost next to each measure. The cost page is the neighbour to this one, and the same person writes both.
Yes, and often more easily than with a distributed application. A monolith has fewer paths along which a fault can spread; its problem is usually the database and a deployment that briefly takes it away entirely. Both can be worked on without taking the application apart: a rehearsed database switch, two instances instead of one, schema changes in two steps. A rebuild into microservices is not a precondition and rarely the answer.
The patterns apply everywhere, the tools do not. Timeouts, retries with jitter, idempotent processing and a rehearsed restore are independent of the provider. What I set up concretely I know best on AWS, and that is where I work daily. If your system runs elsewhere with only part of it on AWS, I will tell you in the first conversation which part is worth the effort.
Other services
Backends for SaaS and platforms that hold under real load: Go and PHP 8, event-driven, with tenant isolation and recovery designed in.
Learn moreModernizing grown systems step by step: strangler fig instead of a rewrite, operations untouched, every step reversible.
Learn moreFrom your own data centre or another cloud onto AWS, with a cost model before the move and a rollback path for every step.
Learn moreI find where your AWS budget leaks away, and cut it measurably without giving up performance or availability.
Learn moreThe infrastructure behind AI systems that actually ship: MCP servers, controlled tool access, LLM integration with real permissions and cost control.
Learn moreReplacing manual workflows with real systems: wired into the software you already run, with permissions and an audit log instead of a chain of tools.
Learn more