Testing an MCP server with scripted clients
A scripted client is a short program that sends a fixed list of messages to a server and checks every reply. It never gets tired and it never skips a step.
It matters because a server is easy to break by accident. A one-character edit can change a reply, and nobody notices until a real client fails.
By the end of this lesson
Note
I can test a server with scripted clients.
Key words
| Word | What it means | Example |
|---|---|---|
| Transcript | a list of messages to send, each with the reply expected | |
| Expected | the reply the test wants | |
| Actual | the reply the server really sent | |
| Difference | the first place the two do not match | |
| Wildcard | a marker that matches any value | <any> |
The cells students fill in are left blank in this preview.
A transcript pairs each message with its reply
A transcript is a list of steps. Each step has a message to send and the reply expected. A notification step expects no reply.
Some values change, such as a version number. The test marks them with a wildcard, so a change there is not a failure. Everything else must match exactly.
Worked example: a transcript for an adding tool
| Step | Sent | Expected reply |
|---|---|---|
| 1 | initialize | The version asked for, tools announced, any server version |
| 2 | The ready notification | No reply |
| 3 | tools/list | One tool called add |
| 4 | add 2 and 3 | The text 5, isError false |
| 5 | add "2" and 3 | A tool error, isError true |
The cells students fill in are left blank in this preview.
Report the first difference, with its place
A test that only says "failed" wastes time. A good one says where. The path result.content[0].text points to one value in the reply.
The checker walks the expected reply in a fixed order and stops at the first difference. It says what it expected and what it got.
A common mistake
Warning
Many people run a test only against a server that works. A test that has never failed has never been shown to catch anything. Break the server on purpose and check that the test fails. Then undo the break.
Write the model helper
docker exec -w /home/node/project agent sh -c "printf '%s\n' 'const base = process.env.AI_URL || \"http://ai:4000\";' 'async function ask(messages, sample, max) {' ' try {' ' const res = await fetch(base + \"/v1/chat/completions\", { method: \"POST\", headers: { \"content-type\": \"application/json\", authorization: \"Bearer lab\" }, body: JSON.stringify({ model: \"lab\", messages: messages, max_tokens: max || 80 }), signal: AbortSignal.timeout(30000) });' ' if (res.ok === false) throw new Error(\"status \" + res.status);' ' const data = await res.json();' ' return { text: data.choices[0].message.content, recorded: false };' ' } catch (error) {' ' console.log(\"Recorded sample: no model answered (\" + error.message + \").\");' ' return { text: sample, recorded: true };' ' }' '}' 'module.exports = { ask };' > model.js"Shown here, not run. Create a free account to press Run and type it into your own lab’s terminal.
Write the server, part 1
docker exec -w /home/node/project agent sh -c "printf '%s\n' 'function send(m) { process.stdout.write(JSON.stringify(Object.assign({ jsonrpc: \"2.0\" }, m)) + \"\n\"); }' 'const SCHEMA = { type: \"object\", properties: { a: { type: \"integer\" }, b: { type: \"integer\" } }, required: [\"a\", \"b\"] };' 'function words(text, bad) { return { content: [{ type: \"text\", text: text }], isError: bad }; }' > server6.js"Shown here, not run. Create a free account to press Run and type it into your own lab’s terminal.
Write the server, part 2
docker exec -w /home/node/project agent sh -c "printf '%s\n' 'function handle(msg) {' ' if (msg.id === undefined) return;' ' const p = msg.params || {};' ' const args = p.arguments || {};' ' if (msg.method === \"initialize\") return send({ id: msg.id, result: { protocolVersion: p.protocolVersion, capabilities: { tools: {} }, serverInfo: { name: \"adder\", version: \"1.0\" } } });' >> server6.js"Shown here, not run. Create a free account to press Run and type it into your own lab’s terminal.
Write the server, part 3
docker exec -w /home/node/project agent sh -c "printf '%s\n' ' if (msg.method === \"tools/list\") return send({ id: msg.id, result: { tools: [{ name: \"add\", description: \"Add two whole numbers\", inputSchema: SCHEMA }] } });' ' if (msg.method === \"tools/call\" && p.name === \"add\") {' ' if (Number.isInteger(args.a) && Number.isInteger(args.b)) return send({ id: msg.id, result: words(String(args.a + args.b), false) });' ' return send({ id: msg.id, result: words(\"a and b must be whole numbers\", true) });' ' }' ' if (msg.method === \"tools/call\") return send({ id: msg.id, error: { code: -32602, message: \"Unknown tool: \" + p.name } });' ' send({ id: msg.id, error: { code: -32601, message: \"Method not found: \" + msg.method } });' '}' 'require(\"readline\").createInterface({ input: process.stdin }).on(\"line\", function (line) { handle(JSON.parse(line)); });' >> server6.js"Shown here, not run. Create a free account to press Run and type it into your own lab’s terminal.
Write the checker, part 1
docker exec -w /home/node/project agent sh -c "printf '%s\n' 'const { ask } = require(\"./model\");' 'const server = require(\"child_process\").spawn(\"node\", [process.argv[2] || \"server6.js\"]);' 'const any = \"<any>\";' 'const one = function (id, method, params) { return { jsonrpc: \"2.0\", id: id, method: method, params: params }; };' 'const add = function (id, args) { return one(id, \"tools/call\", { name: \"add\", arguments: args }); };' 'const ok = function (id, result) { return { jsonrpc: \"2.0\", id: id, result: result }; };' 'const text = function (t, bad) { return { content: [{ type: \"text\", text: t }], isError: bad }; };' > check6.js"Shown here, not run. Create a free account to press Run and type it into your own lab’s terminal.
Write the checker, part 2
docker exec -w /home/node/project agent sh -c "printf '%s\n' 'const steps = [' ' [one(1, \"initialize\", { protocolVersion: \"2025-03-26\" }), ok(1, { protocolVersion: \"2025-03-26\", capabilities: { tools: {} }, serverInfo: { name: \"adder\", version: any } })],' ' [{ jsonrpc: \"2.0\", method: \"notifications/initialized\" }, null],' ' [one(2, \"tools/list\"), ok(2, { tools: [{ name: \"add\", description: any, inputSchema: any }] })],' ' [add(3, { a: 2, b: 3 }), ok(3, text(\"5\", false))],' ' [add(4, { a: \"2\", b: 3 }), ok(4, text(\"a and b must be whole numbers\", true))],' ' [one(5, \"tools/call\", { name: \"nope\" }), { jsonrpc: \"2.0\", id: 5, error: { code: -32602, message: \"Unknown tool: nope\" } }],' '];' >> check6.js"Shown here, not run. Create a free account to press Run and type it into your own lab’s terminal.
Write the checker, part 3
docker exec -w /home/node/project agent sh -c "printf '%s\n' 'function isObj(x) { return Boolean(x) && typeof x === \"object\"; }' 'function name(path, k, list) { return list ? path + \"[\" + k + \"]\" : (path ? path + \".\" + k : k); }' 'function diff(want, got, path) {' ' if (want === any) return null;' ' const both = isObj(want) && isObj(got) && Array.isArray(want) === Array.isArray(got);' ' const said = \" expected \" + JSON.stringify(want) + \", got \" + JSON.stringify(got);' ' if (both === false) return JSON.stringify(want) === JSON.stringify(got) ? null : (path || \"(root)\") + \":\" + said;' >> check6.js"Shown here, not run. Create a free account to press Run and type it into your own lab’s terminal.
Write the checker, part 4
docker exec -w /home/node/project agent sh -c "printf '%s\n' ' const list = Array.isArray(want);' ' if (list && (want.length === got.length) === false) return (path || \"(root)\") + \": expected \" + want.length + \" items, got \" + got.length;' ' const keys = Object.keys(want).sort();' ' for (const k of keys) {' ' if ((k in got) === false) return name(path, k, list) + \": missing\";' ' const d = diff(want[k], got[k], name(path, k, list));' ' if (d) return d;' ' }' ' const extra = Object.keys(got).sort().filter(function (k) { return keys.indexOf(k) < 0; });' ' return extra.length > 0 ? name(path, extra[0], list) + \": unexpected\" : null;' '}' >> check6.js"Shown here, not run. Create a free account to press Run and type it into your own lab’s terminal.
Write the checker, part 5
docker exec -w /home/node/project agent sh -c "printf '%s\n' 'const lines = [];' 'const waiting = [];' 'require(\"readline\").createInterface({ input: server.stdout }).on(\"line\", function (l) {' ' if (waiting.length > 0) waiting.shift()(JSON.parse(l)); else lines.push(JSON.parse(l));' '});' 'function next() { return lines.length > 0 ? Promise.resolve(lines.shift()) : new Promise(function (r) { waiting.push(r); }); }' >> check6.js"Shown here, not run. Create a free account to press Run and type it into your own lab’s terminal.
Write the checker, part 6
docker exec -w /home/node/project agent sh -c "printf '%s\n' 'async function main() {' ' let passed = 0, total = 0, first = null;' ' for (let i = 0; i < steps.length; i++) {' ' server.stdin.write(JSON.stringify(steps[i][0]) + \"\n\");' ' if (steps[i][1] === null) continue;' ' total += 1;' ' const d = diff(steps[i][1], await next(), \"\");' ' console.log((d ? \"FAIL step \" + (i + 1) + \" at \" + d : \"PASS step \" + (i + 1)));' ' if (d === null) passed += 1; else if (first === null) first = d;' ' }' ' server.stdin.end();' ' console.log(\"passed \" + passed + \" of \" + total);' ' if (first === null) return;' ' const r = await ask([{ role: \"user\", content: \"A test of my server failed with this difference. In one sentence, what is the likely cause? \" + first }], \"The server gave a different text from the one the test expected.\", 60);' ' console.log(\"model: \" + r.text);' '}' 'main();' >> check6.js"Shown here, not run. Create a free account to press Run and type it into your own lab’s terminal.
Run the checker on the good server
docker exec -w /home/node/project agent node check6.jsShown here, not run. Create a free account to press Run and type it into your own lab’s terminal.
Make a copy with a bad edit
docker exec -w /home/node/project agent sh -c "sed 's/args.a + /args.a - /' server6.js > bad6.js"Shown here, not run. Create a free account to press Run and type it into your own lab’s terminal.
Run the checker on the bad copy
docker exec -w /home/node/project agent node check6.js bad6.jsShown here, not run. Create a free account to press Run and type it into your own lab’s terminal.
The two runs side by side
| Run | Steps checked | Result |
|---|---|---|
| Good server | 1, 3, 4, 5 and 6 | passed 5 of 5 |
| Bad copy | 1, 3, 4, 5 and 6 | passed 4 of 5 |
| Bad copy, step 4 | add 2 and 3 | result.content[0].text: expected "5", got "-1" |
The cells students fill in are left blank in this preview.