State and recovery
How long you have decided the business can be down — and the only version of the number worth having is one somebody has timed.
Also called Recovery time objective · Целевое время восстановления
Formula
RTO = Detection + Decision + Restoration + Verification
- Detection
- from failure to somebody knowing, which is often the largest term and is never in the estimate
- Decision
- from knowing to somebody with authority saying restore — out of hours, this is a phone tree
- Restoration
- the part everyone measures: data back, service running
- Verification
- confirming it is correct rather than merely up, which is what prevents restoring the damage twice
Estimates usually contain only the third term. A rehearsal measures all four, and the difference between the estimate and the rehearsal is routinely a factor of several.
The distinction from RPO is worth keeping sharp: RPO is what you lose, RTO is how long you are stopped. They are bought separately and traded against each other — a warm standby buys hours off the RTO and does nothing for the RPO, more frequent copies do the reverse.
Detection deserves its own attention because it is the term nobody plans. A failure at 02:00 on a Friday discovered by a customer at 09:00 on Monday has already spent three days of RTO before anyone has done anything, and no amount of restoration speed recovers it. That is a monitoring decision wearing a recovery cost.
“We can restore in two hours” is a statement about one of four terms. The number that matters to the business starts when the system broke and ends when somebody has confirmed the service is correct. Rehearse it end to end, unannounced, including finding the person who is allowed to authorise it — and use the measured figure, because the estimate is always the optimistic term alone.
The ratios a decision is taken on. They inherit every error below them, which is why they are the last thing to trust and the first thing quoted.
Cannot be computed without
- Recoverable copy
- A copy somebody has restored from. Until then it is a hypothesis, and the word backup is what the hypothesis is called before it is tested.
Knowing the definition is not the same as being able to check the figure. These are the procedures that do the second thing.
- Testing a claim that you are covered
- “We have backups. Everything is backed up nightly.” · 40 minutes, 6 questions.
The definitions are the easy part. Whether the figure on your dashboard was computed this way is a different question, and usually the more expensive one.