A quiet Tuesday and Black Friday

A shop website works well on a quiet Tuesday. On Black Friday 20,000 people arrive at once.

The site is slow. Then it stops. The tests on Tuesday never saw this. Different problems need different kinds of testing. In this lesson you meet six more.

Key words

Word	What it means
Performance test	Checks how fast the system is and how much it uses.
Load test	Checks the system with the number of users you expect on a busy day.
Stress test	Pushes the system past its limit to see how and when it breaks.
Usability test	Real people try the product. You watch how easy it is to use.
Regression test	Runs the old tests again after a change, to check nothing broke.
Closed box test	You test inputs and outputs. You cannot see the code.
Open box test	You can see the code and choose tests that use every path.
Automation	Letting a computer run the tests for you.
Macro	A recorded list of actions that a program can repeat.
Script	A small program that runs a job, such as running tests.
Function	A named piece of code that does a job.
Unit test	A test of one small part on its own.
Prototype	A simple early model of a product, used to try out an idea.

The cells students fill in are left blank in this preview.

Six more testing methods

Method	Purpose	When it is used
Performance	Is it fast enough, and does it use too much memory?	When speed matters, and after big changes
Usability	Can real people use it easily?	On a prototype and on the finished product
Regression	Did a change break something that worked?	After every change or fix
Load	Does it cope with the expected number of users?	Before a busy time, such as a sale
Stress	What happens when it is pushed past its limit?	To find the breaking point and see how it fails
Closed box and open box	Test with no view of the code, or with the code in view	Closed: acceptance and system tests. Open: unit tests

The cells students fill in are left blank in this preview.

Closed box and open box

In closed box testing you treat the program like a sealed box. You put data in and check what comes out. You do not care how it works inside.

In open box testing you can see the code. You choose tests so that every path is used: every if, every elif and every else.

Try it: cover every path

Code: python

def fare(age):
    if age < 5:
        return 0.0
    elif age < 16:
        return 1.0
    else:
        return 2.5

# This function has three paths, so an open box tester needs three tests.
assert fare(3) == 0.0
assert fare(10) == 1.0
assert fare(30) == 2.5
print("All three paths tested")

# Delete the last assert. Which path is now never tested?

Benefits and drawbacks

Method	Benefit	Drawback
Performance	Finds slow parts before users do	Needs a realistic machine and data
Usability	Shows real problems that developers miss	Slow and costly: needs real people
Regression	Catches broken old features	The set of tests grows and takes time to run
Load and stress	Finds the limit before a busy day	Needs special tools and a copy of the live system
Closed box	Tests like a real user, and the code does not guide the tests	Some paths in the code may never be tried
Open box	Can test every path in the code	The tester needs to understand the code

The cells students fill in are left blank in this preview.