Write the guesses down
The first ten minutes of an incident decide the next two hours. Before changing anything, write down what you know, what you think, and what would prove each guess wrong. It feels slow; it is the fastest way to stop chasing the same idea twice.
A guess you can't disprove isn't a lead. It's a mood.
Logs you can query
A log line is only useful in an incident if you can filter on it. Log events as objects with a request id, a duration and the few fields you will want to group by later.
const started = performance.now();try { return await handler(request);} finally { log.info({ event: "request", id: request.headers.get("x-request-id"), route: new URL(request.url).pathname, ms: Math.round(performance.now() - started), });}Bisect in time
Most production problems start at a moment. Line the graph up with deploys, config changes and traffic, and the list of suspects usually shrinks to one or two. If it doesn't, the cause is probably outside your code: a dependency, a certificate, a quota.
- When did it start, to the minute?
- What shipped or changed within an hour before?
- Is it every request, one route, or one customer?


