A fix counts only when you retest it
Iterative design is the habit of testing a design, fixing the worst problem, and testing again. It matters because a change that seems obvious can fail, or can fix one problem and cause another.
This lesson shows how to compare a page before and after a change, so that you can say what improved and how you know.
By the end of this lesson
Note
I can compare a design before and after a change and say whether it improved.
Key words
| Word | What it means |
|---|---|
| Iteration | one round of test, fix and test again |
| Prototype | a cheap early version made to test an idea |
| Baseline | the numbers before a change, kept to compare with |
| Retest | to run the same test again after a change |
| Regression | a fix that makes something else worse |
The cells students fill in are left blank in this preview.
Keep the numbers from before the change
You cannot say a change helped unless you know where you started. The numbers you record before you change anything are the baseline.
Run the same checks and tasks afterwards, the same way. A retest that uses different tasks or a different test shows nothing.
Record the baseline of the shop
docker exec jupyter python3 steps/fix.pyShown here, not run. Create a free account to press Run and type it into your own lab’s terminal.
The script prints the audit, then contrast
The first line says that nothing is fixed here. The script does not change the shop. It runs the audit, then the contrast check, so the numbers move only if the pages change.
Run it now and keep the output. It is your baseline: the faults counted across the pages and the style sheet, and five colour pairs. Only one of them reaches 4.5.
Worked example: a before and after table
| Problem counted | Before | After (an example) | Change |
|---|---|---|---|
| Pictures with no alt text | 3 | 0 | Fixed |
| Inputs without a label | 3 | 3 | Not touched |
| Links with vague text | 5 | 5 | Not touched |
| Rules that remove the focus outline | 1 | 0 | Fixed |
| Body text contrast ratio | 2.85 | 7.00 | Passes 4.5 |
The cells students fill in are left blank in this preview.
Fix one thing at a time, then test
If you change five things at once and the test improves, you do not know which change helped. If it gets worse, you do not know which one hurt.
Pick the biggest problem the test found, change that, test again, and keep or undo the change. Then pick the next problem.
Test a prototype before you build the page
A prototype is a cheap early version of an idea. It can be a sketch on paper, a set of linked images, or a rough page with no code behind it.
It costs minutes to change a sketch and weeks to change a finished page. So test the idea with a prototype first, and build only what people could use.
A common mistake
Warning
Many people think that a fix which looks right is fixed. A fix can fail, or cause a regression, which is a new problem elsewhere. Make the change, run the same checks and tasks again, and compare the numbers. Only then say it is fixed.
Say what changed and how sure you are
Compare the same measure before and after. A rise in the success rate from 50 to 75 per cent is a change of 25 points, or half as much again.
With a small test, say what the numbers suggest. A rise of 25 points with twelve people is a good sign. A rise of 5 points could be luck, so call it no clear change.
Going further
Pick a page you know and write four short paragraphs. Give the problem, the baseline numbers, the one change you would make, and the numbers that would show it worked.
Say what result would make you keep the change, and what result would make you undo it.