Skip to main content

Incident response

First, capture

Evidence disappears the moment you start fixing. Before touching anything: what you see, when it started, who is affected, and the request id or error text verbatim.

Severity

MeaningResponse
1Data exposed across tenants, or records lostStop. Preserve. Escalate immediately
2A paying board cannot workFix now
3A feature degraded, workaround existsSame day
4Cosmetic or single-userNext release

Any suspicion of cross-tenant exposure is severity 1, including "it looks like it might have". The cost of over-reacting is an hour; the cost of under-reacting is the product's entire claim.

Containment before diagnosis

For a suspected exposure, remove access before working out why. A board can be suspended — records stay readable, writes stop — which is almost always better than leaving a leak open while you investigate.

Do not

  • Delete anything, including apparently-bad rows. audit_log and decision_seal are append-only for this reason.
  • Edit a migration that has run.
  • Fix in production without the change also going through the repository.

Afterwards

A postmortem for anything severity 1 or 2 — use templates/postmortem.md. The most valuable section is why it was not caught sooner, not what broke.