Scaling and Automatic Sizing
You never choose how much capacity a storage gets. The cluster decides it from what the storage actually holds, revisits the decision every night, and keeps a number of copies that fits what the data is.
None of this needs configuring, and none of it changes how you write a blueprint. A storage is created before it holds anything, so nobody can know whether it will end up with forty rows or forty million. The platform does not ask.
Decided for you
How much capacity a storage gets, how many copies are kept, when that decision is revisited, and what happens when a storage outgrows its shape. All of it measured and live: no downtime, no migration, no maintenance page.
Still yours
How many storages you create and what goes in them, whether data has a natural end date, and telling your operator before you load fifty times more than usual. The cluster reacts to growth quickly, but it cannot know about tomorrow's import.
A new storage starts at its smallest useful size
Every storage is created at the smallest shape that works, and grows from measurement. That is the cheaper direction to be wrong in.
Capacity set aside for a storage is paid for whether documents arrive or not, on every machine that holds a copy. An empty storage given room for a million documents costs memory, recovery time after a restart, and work on every query it answers, for as long as it exists. Growing a storage that turned out to be popular is a solved problem, covered below. Reclaiming capacity from a hundred storages that never filled is not.
How many copies your data keeps depends on what it is
Every storage is held more than once so that losing a machine does not lose data. The number of spare copies is decided per storage, because the cost of losing it and the cost of keeping it both differ.
| Kind of data | Spare copies | Reasoning |
|---|---|---|
| Your business data: the storages you create | One | Survives a machine failing. Two copies is what failover requires; a third is a copy nobody ever reads, paid for on every write. |
| Sign-in, sessions, tokens, page definitions | Two | Read on essentially every request, so the extra copy buys read throughput, and keeps logins working while machines are restarted one by one during an upgrade. |
| Change history and audit trails | One, never zero | A history cannot be rebuilt from the current documents. It is kept for years and never treated as disposable, whatever the storage it belongs to is used for. |
| Caches and derived data | None | Rebuildable by definition, so a spare copy buys nothing. |
| Usage statistics, logs, request history | One | Very high volume, low value per individual record. Worth keeping, not worth keeping three times. |
A storage the platform does not otherwise recognise is treated as business data, which is the careful end of that table. Nothing lands in the cache row by accident.
A nightly pass keeps the decision current
Once a day, in a quiet window, the cluster looks at every storage it holds and asks four questions:
- How large is it?
- How many documents does it hold?
- How fast did it grow over the last week and the last month?
- How much is it actually read and written?
From those answers it changes one thing itself and only recommends the other:
| Change | Who does it | Why |
|---|---|---|
| The number of copies | The nightly pass, immediately | Adding or dropping a copy is a live background operation. Nothing is rewritten, nothing is taken offline, and it is reversible the following night. |
| The capacity a storage is spread over | Written down as a recommendation for a person | Reshaping a storage moves its contents. That belongs in a window somebody chose, on a cluster somebody is watching. |
Each run leaves a report behind: what it changed, what it recommends, what it skipped and the reason it skipped it. On a new deployment the pass starts in a mode where it reports and changes nothing, so the recommendations can be read against reality for a week or two before anybody lets it act.
What the nightly pass refuses to do
These are the standing rules, and none of them has an override that runs unattended.
| Rule | What it prevents |
|---|---|
| Never fewer than one spare copy of anything that cannot be rebuilt | An automated pass quietly leaving your only copy of something on one machine. One copy is not a backup and it is not failover. |
| Never reshapes a storage on its own | Contents being moved without anyone deciding it was a good moment. |
| Never runs during business hours | Even adding a copy sends data across the network. It happens while nobody is waiting on a page. |
| Never starts while the cluster is unhealthy or already moving data | Piling work onto a cluster that is already recovering, including from the pass's own previous action. |
| One storage at a time | A run touching everything at once and making its own effects impossible to read. |
| Never more copies than the cluster has machines to hold | Copies that can never be placed, which report as a permanently unhealthy cluster rather than as extra safety. |
| A cooldown of about two days after changing a storage, holding the previous value | A storage being changed back and forth night after night, and a change nobody can undo. Every action carries its own undo record. |
| Reports before it acts, and reports what it did afterwards | Anything happening to your data that is not written down somewhere you can read. |
A storage grows without going offline
When a storage really has outgrown its capacity, it can be grown in place. A failure at any point leaves you where you started.
Hold writes
Writes to the storage are held briefly.
Build a larger copy
It is built alongside the original, which stays readable throughout.
Count and compare
Both sides are counted. A mismatch stops everything here, with the original untouched.
Move the name
The name moves from the old to the new in one single step, so there is never a moment where the name resolves to nothing and writes fail.
Carry the other names across
Every other name that pointed at the old storage moves with it, so nothing anyone wrote down stops resolving.
- It runs on request, by a cluster admin, in a window they picked. It previews by default: you have to explicitly say "do it" for anything to move.
- It refuses outright on a cluster that is not fully healthy or is already moving data around.
- If a step fails before the switchover, the partial work is removed and the original is unblocked.
- If the platform cannot be certain what happened, it deletes nothing and says so loudly.
Data that only grows is bounded, not enlarged
Usage statistics, request logs and event histories have no natural end. Whatever room you give them, they will eventually fill.
The platform cuts its own such data into time buckets with a horizon behind it. Statistics are written into a bucket per month and buckets past the horizon are dropped whole, so history stays bounded at roughly the last year plus the current month. Dropping a whole month is close to free; deleting a year of records one at a time out of one enormous storage is one of the most expensive things you can ask a cluster to do.
Apply the same rule to your own high-volume storages: put a date in the design and decide how far back you actually need. Document expiry covers the per-document version of the same idea.
The few things that are still yours
These are the decisions automatic sizing cannot make for you, roughly in order of how much difference they make.
| Do this | Because |
|---|---|
| Keep one storage per kind of thing, not one per customer, tenant or day | The biggest lever you hold. A hundred storages holding forty documents each cost far more than one storage holding four thousand with a field naming the customer, because the cost is per storage rather than per document. Filter on a field instead. |
| Delete storages you stopped using | An empty storage is not free. It is observed, copied and kept healthy exactly like a full one. |
| Give data with a natural end an actual end | Sessions, one-time codes, drafts and imports mostly have an obvious expiry. Setting it is cheaper than any capacity decision made later. |
| Warn your operator before a large import | The nightly pass measures what happened yesterday. It handles steady growth well and a fifty-fold jump overnight less well, and a heads-up turns that into a scheduled change instead of a surprise. |
| Leave the recommendations alone unless you are the one running them | The report is advisory. Anything it recommends moves data, and it is written for a person with a maintenance window, not for the next deploy. |
What automatic sizing does not do
- It does not shrink a storage on its own. Making a storage smaller is more disruptive than making it larger, so it is always a human decision.
- It does not change your document design. How many fields a document has and what they contain is a modelling question. See Storage Examples.
- It does not tune individual queries. Capacity is a reliability decision. A slow query is a query problem, and Common Pitfalls is the better starting point.
- It never touches another tenant. The pass is scoped to the entities it is running for, and a recommendation is never derived from anybody else's data.
Next steps
Storages The document model underneath all of this: _meta, mutations, queries and the document lifecycle. Protected Storages Keeping a storage usable by a flow without handing its contents to whoever builds it. RUAL Core Nano The single-machine shape, where sizing is a hardware decision you make once.Frequently asked
Do I have to size a storage in RUAL?
No. Every storage is created at the smallest shape that works and grows from measurement. A nightly pass looks at how large each storage is, how many documents it holds, how fast it grew over the last week and month, and how much it is read and written, then adjusts the number of copies it keeps.
How many copies of my data does a RUAL cluster keep?
It depends on what the data is. Business storages keep one spare copy, sign-in and session data keeps two because it is read on nearly every request, change history never drops below one, caches keep none because they can be rebuilt, and high volume telemetry keeps one. The count is also capped at what the cluster has machines to hold.
Does RUAL reshape a storage automatically?
No. The nightly pass changes the number of copies itself, because that is a live background operation, and writes down capacity changes as a recommendation for a person. Reshaping a storage moves its contents, so it belongs in a window somebody chose on a cluster somebody is watching.
What is the biggest thing I can do to help a RUAL cluster scale?
Keep one storage per kind of thing rather than one per customer, tenant or day. A hundred storages holding forty documents each cost far more than one storage holding four thousand with a field naming the customer, because the cost is per storage rather than per document. Filter on a field instead.