The Friday fix

On Friday a developer fixes a small bug in the shop app. Only that bug is tested.

On Saturday, the busiest day, customers cannot pay. The fix broke the payment page.

A small change can break an old, working part. How do teams stop this?

Key words

Word	What it means	Example
Regression testing	Run the old tests again after a change, to see that nothing else broke.	Run all 200 tests
Release	Giving the new version to its users.	Version 2.1 reaches the users
Planned release	A release on a planned date, after all the planned work and tests.	Every first Monday
Reactive release	An urgent release, made because of a serious problem.	A fix for a security weakness
Rollback	Going back to the last good version.	Undo version 2.1
Version number	Numbers that name each release.	2.4.1
Critical	So important that the product cannot go out without it.	Payment must work
Control	Deciding who may release, and when.	The lead developer says yes

The cells students fill in are left blank in this preview.

Planned and reactive releases

	Planned release	Reactive release
Why	Planned changes are ready	Something is wrong now
When	On a set date	As soon as it is safe
Tests	The full set of tests	At least the critical tests
Example	Version 2.1 with dark mode, every first Monday	A fix for a weak place that attackers use today
Risk	Low: there was time to check	Higher: there is less time

The cells students fill in are left blank in this preview.

Try it: a fix that breaks something

Code: python

def price(age):
    if age < 5:
        return 0
    if age < 16:
        return 1.0
    return 2.5

def run_tests():
    assert price(3) == 0, "free test failed"
    assert price(10) == 1.0, "child test failed"
    assert price(30) == 2.5, "adult test failed"
    print("All tests passed")

run_tests()

# A developer wants children under 6 to travel free. On line 2 they change 5, but type 60 instead of 6.
# Make that change and run again. (< means "is smaller than")
# Which old test now fails? The free test still passes.

Regression testing

After a change, run the old tests again. This is regression testing. A regression is when something that used to work stops working.

The more tests are automatic, the cheaper this is. Teams often run them after every change.

Do not skip tests when in a hurry

data-callouts.tip.label

In a reactive release the team is in a hurry and may skip tests. That is when new problems appear. At least run the critical tests.