A confident README can still be wrong
Documentation written with an AI assistant is text that describes your code, such as a README or a comment. It matters because an assistant writes a fluent description whether or not the code does what it says.
A wrong document is worse than none, because readers trust it.
By the end of this lesson
Note
I can have an assistant write documentation and check it against the code.
Key words
| Word | What it means | Example |
|---|---|---|
| Claim | one statement in a document about what code does | |
| README | the file that introduces a project to a reader | |
| Comment | a note in the code, written for the next reader | |
| Signature | a function's name and its parameters | truncate(text, n) |
| Unsupported | a claim the code does not back up |
The cells students fill in are left blank in this preview.
Every sentence in a doc is a claim
A README says that a function exists, what it takes and what it gives back. Each of those is a claim you can check by reading the code or running it.
An assistant writes from the code you show it and from what is usual. It can describe a function that is not there, or an argument under the wrong name.
One possible request
Tip
Write a README for text.js. For each function give its name, its parameters and what it returns. Describe only what the code does. Do not mention any function that is not in the file.
Write a small file of helpers
docker exec -w /home/node/project agent sh -c "mkdir -p docs-demo && printf '%s\n' '// Small text helpers.' 'function slugify(text) {' ' return text.toLowerCase().split(\" \").join(\"-\");' '}' 'function truncate(text, n) {' ' return text.slice(0, n) + \"...\";' '}' 'module.exports = { slugify, truncate };' > docs-demo/text.js"Shown here, not run. Create a free account to press Run and type it into your own lab’s terminal.
Write a README an assistant might produce
docker exec -w /home/node/project agent sh -c "mkdir -p docs-demo && printf '%s\n' '# text.js' 'slugify(text) makes the text lower case and joins the words with hyphens.' 'slugify(text) also removes punctuation.' 'truncate(text, n) cuts the text to at most n characters.' 'capitalise(text) makes the first letter a capital.' > docs-demo/README.md"Shown here, not run. Create a free account to press Run and type it into your own lab’s terminal.
List the functions that really exist
docker exec -w /home/node/project agent grep -n function docs-demo/text.jsShown here, not run. Create a free account to press Run and type it into your own lab’s terminal.
Check each claim against the code
For every claim ask three things. Does the function exist? Do the parameters match? Does it do what the sentence says on a real example?
The next Runs write a short script with one test for each claim in the README and print which claims the code supports.
Write a test for each claim, part 1
docker exec -w /home/node/project agent sh -c "mkdir -p docs-demo && printf '%s\n' 'const t = require(\"./text\");' 'const claims = [' ' [\"slugify joins the words with hyphens\", () => t.slugify(\"Hot Chocolate\") === \"hot-chocolate\"],' ' [\"slugify removes punctuation\", () => t.slugify(\"Hi, there\") === \"hi-there\"],' ' [\"truncate gives at most n characters\", () => t.truncate(\"chocolate\", 4).length <= 4],' ' [\"capitalise makes a capital\", () => t.capitalise(\"tea\") === \"Tea\"],' '];' > docs-demo/check-docs.js"Shown here, not run. Create a free account to press Run and type it into your own lab’s terminal.
Write a test for each claim, part 2
docker exec -w /home/node/project agent sh -c "printf '%s\n' 'for (const [claim, test] of claims) {' ' let result;' ' try {' ' result = test() ? \"supported\" : \"NOT supported\";' ' } catch (error) {' ' result = \"NOT supported (\" + error.message + \")\";' ' }' ' console.log(result + \": \" + claim);' '}' >> docs-demo/check-docs.js"Shown here, not run. Create a free account to press Run and type it into your own lab’s terminal.
Test every claim in the README
docker exec -w /home/node/project agent node docs-demo/check-docs.jsShown here, not run. Create a free account to press Run and type it into your own lab’s terminal.
What the script printed
| Claim | Result |
|---|---|
| slugify joins the words with hyphens | supported |
| slugify removes punctuation | NOT supported |
| truncate gives at most n characters | NOT supported |
| capitalise makes a capital | NOT supported (t.capitalise is not a function) |
The cells students fill in are left blank in this preview.
A comment should say why, not what
A comment that repeats the line is noise. count = count + 1 with the note "add one to count" tells the reader nothing new.
A comment that says why helps: "the screen counts from 1, the list from 0". Assistants write the first kind by default. Ask for the reason, then check that the reason is true.
A common mistake
Warning
Many people think a fluent, well-organised document must be accurate. The README read well, and three of its four claims were false. Fluent is not checked.
Try it on a file of your own
The lab's own job is the shop's fault, and this lesson leaves it alone. It is a good time to try the ideas on a different file.
Ask an assistant to document a small file you wrote. List every claim it makes, test each one by running the code, and change the document where the code disagrees.