You cannot test every input, so choose well
Equivalence partitioning and boundary value analysis are two ways to choose test cases. They let a handful of tests stand in for millions of inputs.
A ticket machine takes any age from 0 to 120. Nobody can try them all, and nobody needs to. Faults gather at the edges, and that is where you test.
By the end of this lesson
Note
I can pick test cases from equivalence classes and boundaries.
Key words
| Word | What it means | Example |
|---|---|---|
| Equivalence class | a group of inputs the code treats alike | ages 16 to 64 |
| Boundary | the value where one class ends and the next begins | |
| Valid input | an input the code accepts | |
| Invalid input | an input the code must refuse | |
| Off-by-one | a fault where a limit is one too high or too low |
The cells students fill in are left blank in this preview.
An equivalence class groups similar inputs
If the code treats every age from 16 to 64 the same way, then one age in that group stands for all of them. If 30 works, 40 almost certainly does too.
So you split the inputs into equivalence classes and test one value from each class. Do not forget the invalid classes: ages below 0 and above 120.
Worked example: the classes of a ticket machine
| Class | Ages | Price in pence | One test value |
|---|---|---|---|
| Invalid, too low | Below 0 | Refused | -5 |
| Under 5 | 0 to 4 | 0 | 2 |
| Child | 5 to 15 | 400 | 10 |
| Adult | 16 to 64 | 900 | 30 |
| Senior | 65 to 120 | 500 | 80 |
| Invalid, too high | Above 120 | Refused | 130 |
The cells students fill in are left blank in this preview.
Faults gather at the boundaries
A boundary is where one class ends and the next starts: 4 and 5, 15 and 16, 64 and 65. Programmers often write < where they meant <=, which is an off-by-one fault.
A middle value never shows it. So for each boundary test the last value of one class and the first value of the next.
Make a folder for the tests
mkdir -p ~/tickettestShown here, not run. Create a free account to press Run and type it into your own lab’s terminal.
Go into it
cd ~/tickettestShown here, not run. Create a free account to press Run and type it into your own lab’s terminal.
Write the ticket machine
printf 'def ticket_price(age):\n if age < 0 or age > 120:\n raise ValueError("age out of range")\n if age < 5:\n return 0\n if age < 16:\n return 400\n if age < 65:\n return 900\n return 500\n' > ticket.pyShown here, not run. Create a free account to press Run and type it into your own lab’s terminal.
Write a table of boundary cases
printf 'import unittest\nfrom ticket import ticket_price\n\nCASES = [(0, 0), (4, 0), (5, 400), (15, 400), (16, 900), (64, 900), (65, 500), (120, 500)]\n\n\nclass TicketTest(unittest.TestCase):\n def test_table(self):\n for age, expected in CASES:\n self.assertEqual(ticket_price(age), expected, "age " + str(age))\n\n def test_out_of_range(self):\n for age in [-1, 121]:\n with self.assertRaises(ValueError):\n ticket_price(age)\n' > test_ticket.pyShown here, not run. Create a free account to press Run and type it into your own lab’s terminal.
Run the boundary tests
python3 -m unittest test_ticketShown here, not run. Create a free account to press Run and type it into your own lab’s terminal.
A table of cases keeps a test short
The test file holds a list of pairs: an age and the price expected. One loop checks them all, and the message after the comma names the age that failed.
assertRaises checks that the code refuses an invalid age with a ValueError. A refusal is behaviour too, so it needs a test.
See a boundary test catch an off-by-one fault
Now put a fault in on purpose. Change age < 5 to age <= 5, so a five year old is wrongly free.
First try two middle values, 30 and 10. Then run the boundary tests.
Put the off-by-one fault in
sed -i 's/age < 5/age <= 5/' ticket.pyShown here, not run. Create a free account to press Run and type it into your own lab’s terminal.
Try two middle values
python3 -c "from ticket import ticket_price; print(ticket_price(30), ticket_price(10))"Shown here, not run. Create a free account to press Run and type it into your own lab’s terminal.
Run the boundary tests again
python3 -m unittest test_ticketShown here, not run. Create a free account to press Run and type it into your own lab’s terminal.
The middle values passed and the boundary failed
The two middle values still gave 900 and 400, the right prices, so a test that used only those would pass. The boundary table failed with AssertionError: 0 != 400 : age 5.
The test named the age that failed. That is why the table passes a message to the assertion.
A common mistake
Warning
Many people test a few ordinary values and call it done. Ordinary values sit in the middle of a class, where faults are rare. Test the edges first, on both sides of every boundary, and the middle with one value.