Glossary
S3 storage classes
Also: Lifecycle rules · Intelligent-Tiering · Glacier
Tiers of storage graded by access frequency, from instantly available to archive, at very different prices.
S3 does not have one price, it has several at once: storage per gigabyte and month, requests per thousand accesses, retrieval fees in the colder classes, and egress when data leaves the region. Looking only at the storage price means optimizing in the wrong place.
Standard is the class for everything read regularly. Standard-IA and One Zone-IA are cheaper to store and more expensive to retrieve, meant for data you rarely need but need immediately. Glacier Instant, Flexible and Deep Archive step further down, with retrieval times from milliseconds to hours.
Intelligent-Tiering takes the decision off your hands and moves objects automatically based on access. It charges a small monitoring fee per object, and that is where the catch sits: objects below 128 kilobytes are not monitored and stay permanently in the most expensive class. With many small files you pay for monitoring without benefit.
Lifecycle rules are the direct route: after X days move to a colder class, after Y days delete. They are one line of infrastructure code and apply to the whole estate. The hard part is not the rule but the question before it: what does the business actually need, what does a retention obligation require, and what is only habit.
The quietest item is aborted multipart uploads. The fragments appear in no bucket view but cost storage like whole objects, and grown accounts often hold terabytes of them. A lifecycle rule removing them after seven days belongs in every bucket.
How you notice it
- The bucket grows monotonically and there is no lifecycle rule.
- Nobody can say how long which data has to be kept.
- Versioning is active without a rule for old versions.
- Test data is kept as long as production data.
Not to be confused with
- Intelligent-Tiering
- Automation instead of a rule. Good for unknown or changing access patterns, bad for many small objects, because the monitoring fee then eats the benefit.
- Lifecycle rules
- Fixed rules by age. Cheaper than Intelligent-Tiering when the access pattern is known, and the only route to deletion.
- Versioning
- Keeps previous object states. It quickly doubles the estate when no lifecycle rule clears out old versions. A common unrecognized cost driver.
- Glacier Deep Archive
- The cheapest storage, with retrieval times of hours and noticeable retrieval fees. For data you keep for legal reasons and hopefully never read.
When it fits
- The estate is several terabytes and still growing.
- There is data with a recognizable pattern: logs, raw data, exports, old backups.
- Before any retention decision, because deleting gets harder afterwards.
When it does not
- For small estates below a terabyte the effort rarely pays off.
- For objects read in full every quarter: archive classes are then more expensive than Standard.
- With many objects below 128 kilobytes, Intelligent-Tiering is the wrong choice.
How to approach it
- Throw away first, then moveA considerable share of the estate is no longer needed for any analysis. Deleting saves storage, requests and future transition cost in one go.
- Measure access patternsS3 Storage Lens and access logging for two weeks before writing rules. Without that data you move objects into classes you then pay dearly to read from.
- Clear out aborted uploadsA lifecycle rule deleting multipart uploads after seven days belongs in every bucket. The item is invisible and often substantial in grown accounts.
- Limit old versionsWhere versioning is active, set a rule for non-current versions. Otherwise the estate grows twice over without showing up in the bucket view.
- Clarify retention, then write the ruleWhat the business needs, what an obligation requires, what is habit. That clarification takes longer than the Terraform line afterwards and is the actual work.
Frequently asked
Intelligent-Tiering or your own lifecycle rules?
Intelligent-Tiering where the access pattern is unknown or changing. Your own rules where it is known, because they carry no monitoring fee and can also delete. With many objects below 128 kilobytes, Intelligent-Tiering is the wrong choice, because those are not monitored at all.
How much does a transition typically save?
Between 40 and 80 per cent of the storage cost for the affected data, depending on the target class. Deleting usually brings considerably more, though: in one project a substantial share of the estate was no longer needed for any analysis, and less data means less storage, less scanning and less of everything.
What does a retrieval from archive cost?
Depending on the class, a retrieval fee per gigabyte plus waiting time, up to several hours for Deep Archive. Hence the rule of thumb: first check how often the data is actually read. Quarterly full retrievals make any archive class more expensive than Standard.
Why does my storage usage not appear in the bucket view?
Because aborted multipart uploads are not listed there. They still occupy storage and are billed. A lifecycle rule removing them after seven days is the standard move and regularly finds terabytes in grown accounts.