11 June 2026
13 min read

Bad architecture sends no alerts. It just keeps costing.

FinOps on AWS, hidden architecture costs article image

FinOps on AWS beyond Savings Plans: architecture costs have no dashboard. Why the most expensive spots on your bill send no alerts, how to find them, and when looking for them is not worth the effort.

A setup I came across recently: logs went from the ECS tasks to CloudWatch, and from there via a Lambda forwarder on to Datadog. Technically clean, running for months, nobody had a problem with it. Cost: around 700 euros a month, over eight months. For a detour nobody needed.

Nobody had built this architecture to burn money. It had grown that way. At some point someone took the obvious route, CloudWatch was there anyway, the forwarder was written quickly, and after that the system simply worked. That is exactly the problem.

A crashed service sends an alert. An oversized instance does not. A broken cost architecture keeps running quietly and writes the same number on the bill every month, until someone finally takes a close look. Usually that takes years.

Why nobody looks

When a company grows, one number counts: growth. New customers, new features, more traffic. The engineering team builds, deploys, keeps building. Who is going to ask about AWS costs at that point?

Nobody. And that is the right call.

If you grow by double digits every month, you do not spend capacity on S3 storage classes or CloudWatch retention. A developer hour that goes into optimizing a 200-euro line item instead of the feature that lands the next major customer is badly invested at that stage. Cost awareness at the wrong time is just as expensive as cost blindness at the right one.

But at some point the curve flattens. The focus shifts from faster to more efficient. And then someone really looks at the AWS bill for the first time, not the total at the bottom but the structure behind it. That is the moment when years of accumulation become visible.

The actual problem is not that these costs exist. It is that they have no visibility. A service with high latency shows up in tracing. An error rate lands on a dashboard. But the decision to send logs on an expensive detour, or to let traffic cross AZ boundaries, leaves no trace in the systems a team looks at every day. It only shows up on the bill, and nobody reads that line by line.

The detour nobody sees

Back to the logging example, because it is typical.

The awslogs driver sends container logs to CloudWatch Logs. From there a subscription filter picks them up and a Lambda forwards them to Datadog. Every stop costs: CloudWatch charges ingestion per gigabyte, the Lambda charges invocations and runtime, and the outbound traffic costs as well. CloudWatch is a pure transit stop here, one you pay for in full.

The most common reflex at this point is the wrong one: cut the retention. That saves nothing relevant. With CloudWatch Logs the dominant cost factor is ingestion, so taking the data in, not keeping it. Dropping retention from 365 to 30 days reduces storage costs, which are only a fraction anyway. The ingestion has long been paid for by then.

The real lever is the architecture. FireLens is a log router that runs as a sidecar in the ECS task, usually on Fluent Bit internally. It takes the container logs and sends them straight to Datadog, with no CloudWatch in between. The ingestion costs disappear, the forwarder Lambda becomes redundant, the data path gets shorter.

It is important to be precise about what this saves and what it does not. The roughly 700 euros a month are the AWS-side costs of the detour. FireLens changes nothing about the Datadog bill, because Datadog also charges by ingested volume. FireLens removes the toll on the route, not the destination. And it is a trade-off: taking CloudWatch out of the path entirely means losing native queryability via Logs Insights and a buffer layer that some teams rely on for ad-hoc analysis. For a team that evaluates everything in Datadog anyway, that is a pure win. For a team that actively uses CloudWatch, it is a deliberate decision. Either way, over the eight months the setup ran, that was 5,600 euros for a configuration that could have been corrected in half a day. And that was one single spot.

The usual suspects

Grown AWS setups almost always have several spots like this. No single one blows the budget. Together they regularly add up to four- and five-figure amounts a year that quietly disappear.

NAT Gateway

A NAT Gateway costs twice: an hourly price for provisioning and a price per gigabyte of processed data. The hourly price is well known. The data processing rarely is.

The expensive mistake happens when traffic to other AWS services runs through the NAT Gateway even though it does not have to. A service in a private subnet that talks to S3, DynamoDB or ECR sends that traffic through the NAT Gateway by default. Every gigabyte is charged, even though the destination is inside AWS.

VPC gateway endpoints for S3 and DynamoDB solve this. They are free and keep that traffic away from the NAT Gateway entirely. For services like ECR or Secrets Manager there are interface endpoints, which do cost something themselves but are considerably cheaper than NAT data processing at high volume. In setups where most of the NAT volume is that kind of AWS-internal traffic, the lion's share of the cost disappears. And if you run a separate gateway per availability zone without needing one, you pay the hourly price several times over.

Data transfer

The NAT Gateway is only the most visible slice of a bigger block, and that block is often the most expensive of all: data transfer. It shows up in no architecture diagram and on no dashboard, but it is on every bill.

Three areas are worth a look. First, traffic between availability zones. It is charged in both directions, per gigabyte each way. A service that constantly talks to a database in another AZ, or replication across AZ boundaries, adds up exactly like the NAT story, except that nobody has a single culprit in mind. Second, cross-zone load balancing: on the ALB it is included, on the NLB it is off by default and produces precisely these inter-AZ costs when you turn it on. Enable it without a reason and you pay for it. Third, egress to the internet. Outbound traffic costs per gigabyte, inbound is free. For workloads that serve a lot, putting CloudFront in front is often cheaper than direct egress, because cached delivery is priced lower and the load drops at the same time.

The common denominator is topology. Data transfer costs are almost always the receipt for where components sit relative to each other. Keep tightly communicating parts in the same AZ and know which traffic crosses zones or regions, and you have already solved most of the problem.

S3 without a lifecycle

S3 buckets fill up with data that nobody touches any more. Logs, old exports, backups of systems that were shut down long ago. All of it sits in S3 Standard, the most expensive class, and nobody cleans up.

If the access pattern is known, the bucket needs a lifecycle policy: transition to Standard-IA, then into a Glacier class, and deletion at the end if the retention period allows it. There is a trap here, though, one that can make the naive switch more expensive than the original. Standard-IA bills every object at a minimum of 128 kilobytes and a minimum of 30 days of storage, plus a small fee per transition. For a bucket full of tiny objects, moving to IA ends up costing more, not less. If the access pattern is unknown or irregular, Intelligent-Tiering is the better default: it moves objects automatically and charges no retrieval fees.

CloudWatch, the volume problem

I called out retention above as a fake lever. The real lever on logging costs is volume: what gets logged at all, and at what granularity.

The most common single line item is debug logging that accidentally stayed active in production after a deployment and has been recording every request ever since. Alongside that there are logs AWS generates itself that nobody ever reads, first and foremost VPC Flow Logs at full granularity on subnets that hardly anyone analyses. Both drive ingestion, and neither shows up anywhere as a deliberate decision. If you want to save here, you do not lower retention, you lower the volume at the source.

Dev and stage running around the clock

Non-production environments run 24 hours a day, seven days a week, exactly like production, surprisingly often. The maths behind it is simple and worth making visible once. A week has 168 hours. A non-prod environment gets used maybe from 8am to 6pm on working days, so around 50 hours. Shut it down outside those hours and you pay the runtime-dependent costs for 50 hours instead of 168. That is roughly 70 percent less, without anyone doing anything differently than before.

A scheduler that shuts down EC2 instances, auto scaling groups and stoppable RDS instances in the evening and brings them back up in the morning costs a few hours of work. With RDS, watch the seven-day limit after which a stopped instance restarts automatically; a pausable Aurora Serverless configuration is often a better fit here. After that the saving runs by itself every month.

Oversized instance classes

Instances get chosen generously during the first setup, because nobody knows at the start what the workload really needs. That is sensible. What is missing is the second look, months later, once the real metrics are in.

Compute Optimizer and the CloudWatch metrics quickly show which instances sit permanently at 10 percent CPU. One size smaller, or a move to a more modern generation, lowers the bill immediately. Graviton-based instances are often the single biggest improvement here, provided the workload supports ARM.

Orphaned resources

The classic to finish with, because it turns up in almost every grown account: resources nobody needs any more and that get paid for every month regardless. EBS volumes that are no longer attached after an instance was deleted keep running as storage without anything accessing them. EBS snapshots pile up over years, created automatically, never cleaned up. Unused elastic IPs are charged by the hour, and since 2024 so are all public IPv4 addresses. Add load balancers without targets and old AMIs together with the snapshots behind them. Individually cents, in total a quiet line item that only grows. A script that reports orphaned resources regularly finds all of this in minutes.

This is architecture, not couponing

What stands out about this list is what is not on it: Reserved Instances and Savings Plans.

Both have their place, but they are the second measure, not the first. A Savings Plan lowers the price per unit. It does not lower the quantity. Commit to a level of consumption that is 30 percent waste and you freeze that waste for one to three years and get a discount on it. It feels like saving and is still the more expensive option.

So the order is clear: clean up usage first, then secure the cleaned-up base load with commitments. Do it the other way round and you optimize the price of a problem instead of solving the problem.

That is exactly where FinOps differs from plain cost hunting. The items on the list above are not tariff questions, they are architecture and lifecycle decisions. And they have a pleasant side effect: almost all of them can be fixed without touching a single line of production code. A VPC endpoint, a lifecycle policy, a scheduler, a changed task definition. That is infrastructure, not application logic. I have worked with teams that saved four- and five-figure amounts a month this way, without anything changing about how their systems function.

When it is not worth it

In case this leaves the impression that every fix is always right: it is not.

FinOps is worth it when the recurring saving clearly exceeds the one-off cost of finding and fixing, including maintaining the fix afterwards. Below that line, you leave it alone. For a small account, in the phase before product-market fit, or when the developer hour costs more than the line item it saves, cost optimization is the waste. The same applies when a fix introduces complexity or operational risk out of all proportion to the saving, or when the system in question is being rebuilt next quarter anyway. Then you are optimizing something that is about to disappear.

So the honest answer is not "always save", it is "know where the line is". Most mature setups sit well above it. But the question belongs at the start, before you begin.

Give your costs a dashboard

The title of this piece is also the solution. Architecture costs have no dashboard, so you give them one.

The first step is cost allocation tags, used consistently. Without tags an AWS bill is a total per service across the entire account. With tags by team, environment and service, it becomes visible where the money goes. The breakdown by environment alone exposes the 24/7 stage systems immediately. If you need more detail, enable the Cost and Usage Report and analyse it with Athena. That answers questions Cost Explorer does not, such as how much NAT data processing is down to S3 traffic, or which log groups drive ingestion.

The second step is keeping it going. Cost Anomaly Detection catches new outliers before they run for eight months. And the most effective thing costs nothing at all: making cost a fixed item in the architecture review, on equal footing with latency and error rate. A one-off cleanup gets you to a clean baseline. The practice keeps you there.

Bad architecture sends no alerts. It just keeps costing, quietly, every month, until someone decides to look. Unlike technical debt in application code, most of these problems are infrastructure decisions with a clear, local fix and no impact on functionality. That makes FinOps one of the few optimizations with high impact and low risk.

Which leaves one open question, and it is not a technical one: when did you last do an architecture review, not out of obligation, but because you wanted to know what is really running?

The best time for that was two years ago. The second best is now.