From: Sven Holm, on-call lead, Fernhill Garden Supply
"Payments started failing just after ten this morning. I've put this morning's access log in logs/ and today's deploys in deploys.txt. Please don't edit the originals: copy what you need into incident/.
I need to know how many server errors there were, when they started, which endpoint they hit, and which deploy to roll back."
Your answer: an incident folder with the evidence, the counts, the first error, the suspect deploy and a four-line summary.
A checkout starts failing a few minutes after a deploy, and the on-call engineer has a raw access log and a list of today's deploys. This lab works the incident the way you would on a real server: copy the evidence first, select the server errors, count them per minute and per endpoint, find when they started, match that to a deploy, and write a summary someone can act on.
Each line of the log is time, method, path, status and duration. A status of 500 or above is a server error; a slow request with a 2xx status is not. The traps are the ones that cost time at 3 a.m.: counting with uniq before sorting, grepping for "50" and catching durations, and confusing the first request with the first error.
All eight steps share one workspace, and every check also protects the earlier results.
Preserve the evidence before you change anything.