Data reconstruction

When there is nothing to restore from, the state is rarely gone entirely; it is scattered. Logs, third-party records, exports somebody kept, caches, search indexes, the counterparties who hold their half of every transaction, and in commerce the suppliers whose catalogues you sell. Reconstruction is the work of establishing which of those is authoritative for which field and assembling a defensible version from them.

The order of operations matters more than the tooling. Identify an authoritative source per field before writing anything, because a reconstruction that merges two sources without a rule produces a dataset nobody can later reason about. Record the provenance of every reconstructed value. And reconstruct the invariants first — the constraints that must hold — so the rebuilt state cannot be internally contradictory even where it is incomplete.

This practice has done it at scale once: after a cyberattack destroyed a retailer's site, product database, pricing and every backup, 8,500 catalogue entries were rebuilt programmatically rather than by hand, from the sources that remained. It is the least glamorous engineering there is and the only thing that helps on the day.

Rebuilding by hand under time pressure

A team facing a lost catalogue starts re-typing, because it feels like progress and the first hundred rows go quickly. At a few thousand rows it is months of work with a typo rate nobody is measuring, and no record of where any value came from. Establish sources and write the pipeline; it is slower for two days and finished in a fraction of the time, with provenance attached.

A definition sits still. This does not.

The term is a reading taken at a point in a sequence that is already running. These are the sequences, with the place each of them jams.

From something breaking to somebody confirming it works

An outage, from the moment the system stopped being correct to the moment a person has checked that it is correct again — including the parts that happen before anybody has touched a keyboard. 6 steps, monitoring whether the site is up is the arrangement that fails while looking reasonable.

Somebody will quote this number at you.

Knowing the definition is not the same as being able to check the figure. These are the procedures that do the second thing.

Testing a claim that you are covered

“We have backups. Everything is backed up nightly.” · 40 minutes, 6 questions.

Here is the number, and here is the page it came off.

A definition proves nothing. These are the places on this site where this quantity was actually computed, with the period and whose figure it is stated beside each one.

Somebody is accountable for this number.

A definition is free. Being answerable for the figure it produces is the part that is bought, and this term is a working part of the engagements below.

Anti-crisis engineering and recovery

Stabilise before you improve. A system under stress cannot absorb a redesign.

Recoverable copy

A copy somebody has restored from. Until then it is a hypothesis, and the word backup is what the hypothesis is called before it is tested.

---

Domain invariant

A statement about your data that must never be false — enforced by the system that stores it, not by the code that happens to be running.

---

RPO

How much recent work you have decided you can afford to lose — expressed in time, and settled by the business rather than by the infrastructure.

---

Migration

Moving data from one shape to another while the business keeps running — and the risk is never the data, it is the interval where two shapes are both live.

All ninety-six termsWhat this is regulated by in the UAE

Is this one right in your own numbers?

The definitions are the easy part. Whether the figure on your dashboard was computed this way is a different question, and usually the more expensive one.