All terms

Glossary

Sharding

Also: Horizontal partitioning

Data is split across several databases by a key, when a single one no longer carries the volume or the load.

The split follows a criterion such as customer, region or time range. Each shard holds part of the data, and the application has to know where to look.

The price is queries across shard boundaries. What used to be a join becomes several queries with merging in the application.

How you notice it

  • A single database can no longer hold the data volume.
  • Indexes, archiving and bigger machines are exhausted.
  • The data can be split along a clear criterion.

Frequently asked

What comes before sharding?

Four things: missing indexes, archiving old data, read replicas for reporting, and a bigger machine. In practice those four handle most cases. Sharding is the answer when they do not.

Where this comes up in my work

Building SaaS platforms

SaaS platforms that still hold at the tenth customer: a decided tenant model, billing with usage metering you can prove, and a list of what breaks first at ten times the volume.

To the service

Backend development

Backends for SaaS and platforms that hold under real load: Go and PHP 8, event-driven, with tenant isolation and recovery designed in.

To the service

Freelance Go Developer

Go services with production experience: millions of requests a day, single-digit milliseconds, multi-region on AWS. New services alongside your existing system instead of a rewrite.

To the page

Related terms

Multi-tenancyExpand and contract