What a service level review is
A service level objective (SLO) is a target a feature must meet, such as "95 percent of requests succeed". A weekly review compares what happened with the targets and decides what to do.
Earlier modules give you costs, traces and the design of a feature. This lesson goes further: a measured week, an error budget and a runbook step for each alarm.
By the end of this lesson
Note
I can check a feature against a service level over a week.
Key words
| Word | What it means | Example |
|---|---|---|
| Target | the level a feature must reach | 95 percent |
| Breach | a target that was missed | |
| Error budget | the errors a target allows in a period | |
| Runbook | a list that says what to do for each alarm | |
| Alarm | a signal that a target is missed | |
| Freeze | to stop making changes for a while |
The cells students fill in are left blank in this preview.
Three targets for the week
Here the feature has three targets. At least 95 percent of requests must succeed. At least 90 percent must be fast. The cost for the week must stay under a cap of 1800000 units.
Percents are rounded down. Meeting a target exactly is fine. Only missing it is a breach.
Worked example: a recorded week
| Day | Requests | Errors | Slow | Cost |
|---|---|---|---|---|
| Mon | 200 | 6 | 14 | 300000 |
| Tue | 210 | 8 | 15 | 320000 |
| Wed | 220 | 9 | 20 | 340000 |
| Thu | 205 | 30 | 40 | 310000 |
| Fri | 215 | 7 | 18 | 330000 |
| Sat | 90 | 2 | 5 | 140000 |
| Sun | 80 | 1 | 4 | 120000 |
| Week | 1220 | 63 | 116 | 1860000 |
The cells students fill in are left blank in this preview.
A common mistake
Warning
Many people think an average that looks fine means the week was fine. A bad day can hide inside a good week. Check each day as well as the total, and say which day went wrong first.
The error budget
A 95 percent target allows 5 percent of requests to fail. That allowance is the error budget. For 1220 requests it is 61 errors, working in whole numbers.
The week had 63 errors, so the budget left is 61 minus 63, which is -2. The budget is spent, so the team freezes changes until it is back.
Worked example: the budget
| Step | Sum | Result |
|---|---|---|
| Success percent | (1220 minus 63) times 100 divided by 1220 | 94 |
| Allowed errors | 1220 times 5 divided by 100 | 61 |
| Errors used | From the week | 63 |
| Budget left | 61 minus 63 | -2 |
The cells students fill in are left blank in this preview.
One step for each alarm
A runbook is a short list: for this alarm, do this. It is written before the bad day, so whoever is on call does not have to invent a plan.
If a breach has no step in the runbook, escalate it to the owner. Add a step for it afterwards.
Write the model helper, part 1
docker exec -w /home/node/project agent sh -c "printf '%s\n' 'const base = process.env.AI_URL || \"http://ai:4000\";' '// The token counts of a reply, or null when the reply carries none.' 'function count(data, name) {' ' return data.usage && typeof data.usage[name] === \"number\" ? data.usage[name] : null;' '}' > model.js"Shown here, not run. Create a free account to press Run and type it into your own lab’s terminal.
Write the model helper, part 2
docker exec -w /home/node/project agent sh -c "printf '%s\n' 'async function ask(messages, sample, max) {' ' try {' ' const res = await fetch(base + \"/v1/chat/completions\", {' ' method: \"POST\",' ' headers: { \"content-type\": \"application/json\", authorization: \"Bearer lab\" },' ' body: JSON.stringify({ model: \"lab\", messages: messages, max_tokens: max || 100 }),' ' signal: AbortSignal.timeout(30000),' ' });' ' if (res.ok === false) throw new Error(\"status \" + res.status);' ' const data = await res.json();' ' const text = data.choices[0].message.content;' ' return { text, inTokens: count(data, \"prompt_tokens\"), outTokens: count(data, \"completion_tokens\"), recorded: false };' ' } catch (error) {' ' console.log(\"Recorded sample: no model answered (\" + error.message + \").\");' ' return { text: sample, inTokens: null, outTokens: null, recorded: true };' ' }' '}' 'module.exports = { ask };' >> model.js"Shown here, not run. Create a free account to press Run and type it into your own lab’s terminal.
Write the review script, part 1
docker exec -w /home/node/project agent sh -c "printf '%s\n' 'const { ask } = require(\"./model\");' 'const days = [' ' [200, 6, 14, 300000], [210, 8, 15, 320000], [220, 9, 20, 340000],' ' [205, 30, 40, 310000], [215, 7, 18, 330000], [90, 2, 5, 140000], [80, 1, 4, 120000],' '];' 'const targets = { success: 95, fast: 90, cap: 1800000 };' > review.js"Shown here, not run. Create a free account to press Run and type it into your own lab’s terminal.
Write the review script, part 2
docker exec -w /home/node/project agent sh -c "printf '%s\n' 'function totals() {' ' const t = { n: 0, bad: 0, slow: 0, cost: 0 };' ' for (const d of days) { t.n += d[0]; t.bad += d[1]; t.slow += d[2]; t.cost += d[3]; }' ' return t;' '}' 'function pct(part, whole) { return Math.floor((whole - part) * 100 / whole); }' >> review.js"Shown here, not run. Create a free account to press Run and type it into your own lab’s terminal.
Write the review script, part 3
docker exec -w /home/node/project agent sh -c "printf '%s\n' 'async function main() {' ' const t = totals();' ' const ok = pct(t.bad, t.n);' ' const fast = pct(t.slow, t.n);' ' const allowed = Math.floor(t.n * (100 - targets.success) / 100);' ' console.log(\"success \" + ok + \" percent, target \" + targets.success);' ' console.log(\"fast \" + fast + \" percent, target \" + targets.fast);' ' console.log(\"cost \" + t.cost + \", cap \" + targets.cap);' ' console.log(\"error budget: allowed \" + allowed + \", used \" + t.bad + \", left \" + (allowed - t.bad));' ' const reply = await ask([{ role: \"user\", content: \"In one short sentence, what is a runbook for?\" }], \"It tells whoever is on call exactly what to do for each alarm.\", 60);' ' console.log(\"model said: \" + reply.text);' '}' 'main();' >> review.js"Shown here, not run. Create a free account to press Run and type it into your own lab’s terminal.
Run the review
docker exec -w /home/node/project agent node review.jsShown here, not run. Create a free account to press Run and type it into your own lab’s terminal.
What the Run prints
| Line | What it shows |
|---|---|
| success 94 percent, target 95 | A breach, one point under |
| fast 90 percent, target 90 | Met exactly, so no breach |
| cost 1860000, cap 1800000 | A breach, over the cap |
| error budget: allowed 61, used 63, left -2 | The budget is spent |
| model said: ... | A sentence from the lab. A recorded sample if no model answers |
The cells students fill in are left blank in this preview.