How to test a web API
An API test sends a request and checks the answer. It does not look at the code behind the API. It only asks what a caller would ask, and compares the answer with what the API promised.
You will write a few checks by hand, see one fail, and test what happens when a request is sent twice.
By the end of this lesson
Note
I can test an API with checks for good requests, bad requests and retries.
Key words
| Word | What it means |
|---|---|
| Test case | one request and the answer you expect to get back |
| Expected status | the status code the API should answer with |
| Test data | rows made only for a test, so they can be found and removed |
| Retry | sending the same request again, because the first answer was lost |
| Idempotent | sending it twice leaves things as they were after sending it once |
The cells students fill in are left blank in this preview.
A check is a request and a status
Typing curl and reading numbers by eye does not scale. Write the check once: a name, the status you want, and the request.
First give psql the password and make the books table. Then define a small function called check in your terminal, which sends the request and says PASS or FAIL.
Give psql the password
export PGPASSWORD=shopShown here, not run. Create a free account to press Run and type it into your own lab’s terminal.
Make the books table
psql -h db -U shop -c "drop table if exists books; create table books (id serial primary key, title text not null, year int not null, available boolean not null default true); insert into books (title, year, available) values ('Maps of the Sea', 1998, true), ('Rivers of Europe', 2004, false), ('Night Trains', 2011, true); notify pgrst, 'reload schema'"Shown here, not run. Create a free account to press Run and type it into your own lab’s terminal.
Define the check function
check() { want=$1; name=$2; shift 2; got=$(curl -s -o /dev/null -w '%{http_code}' "$@"); if [ "$got" = "$want" ]; then echo "PASS $name"; else echo "FAIL $name, wanted $want and got $got"; fi; }Shown here, not run. Create a free account to press Run and type it into your own lab’s terminal.
Check a good request
check 200 "list books" "http://api:3000/books"Shown here, not run. Create a free account to press Run and type it into your own lab’s terminal.
Check something that should fail
check 200 "wrong on purpose" "http://api:3000/nothing"Shown here, not run. Create a free account to press Run and type it into your own lab’s terminal.
Test what the API refuses
The second check was wrong on purpose, to prove the function can say FAIL. A check that always prints PASS is worse than none.
A good test suite spends as much time on bad requests as on good ones. Each one should come back as a 4xx with the right code, and should not change the data.
Check a book with no title
check 400 "no title" -X POST -H "Content-Type: application/json" -d '{"year":2019}' "http://api:3000/books"Shown here, not run. Create a free account to press Run and type it into your own lab’s terminal.
Check a body that is not JSON
check 400 "broken json" -X POST -H "Content-Type: application/json" -d 'this is not json' "http://api:3000/books"Shown here, not run. Create a free account to press Run and type it into your own lab’s terminal.
Check adding a test book
check 201 "add a book" -X POST -H "Content-Type: application/json" -d '{"id":900,"title":"Test book","year":2020}' "http://api:3000/books"Shown here, not run. Create a free account to press Run and type it into your own lab’s terminal.
Check the same id again
check 409 "same id again" -X POST -H "Content-Type: application/json" -d '{"id":900,"title":"Test book","year":2020}' "http://api:3000/books"Shown here, not run. Create a free account to press Run and type it into your own lab’s terminal.
Check the body as well as the status
curl -s "http://api:3000/books?id=eq.900" | grep -q '"title":"Test book"' && echo "PASS the body has the title" || echo "FAIL the body has the title"Shown here, not run. Create a free account to press Run and type it into your own lab’s terminal.
Test a request that is sent twice
Networks lose answers, so clients send the same request again. A request is idempotent if sending it twice leaves the data as it was after sending it once.
Test it by counting rows. First a change that sets a value, then a POST with no id, which should make a new book each time.
Send the same change twice
for i in 1 2; do curl -s -o /dev/null -w "%{http_code}\n" -X PATCH -H "Content-Type: application/json" -d '{"available":false}' "http://api:3000/books?id=eq.1"; doneShown here, not run. Create a free account to press Run and type it into your own lab’s terminal.
Send the same new book twice
for i in 1 2; do curl -s -o /dev/null -X POST -H "Content-Type: application/json" -d '{"title":"Twice","year":2020}' "http://api:3000/books"; doneShown here, not run. Create a free account to press Run and type it into your own lab’s terminal.
Count the books called Twice
psql -h db -U shop -At -c "select count(*) from books where title = 'Twice'"Shown here, not run. Create a free account to press Run and type it into your own lab’s terminal.
Make a create safe to retry
Two books called Twice means a retry made a duplicate. A caller who chooses the id can make the create safe. Send the same id, with a header that says "if it is already there, update it".
The first call answers 201 and the second answers 200. Both leave one row.
Send a safe create twice
for i in 1 2; do curl -s -o /dev/null -w "%{http_code}\n" -X POST -H "Content-Type: application/json" -H "Prefer: resolution=merge-duplicates" -d '{"id":902,"title":"Once","year":2020}' "http://api:3000/books"; doneShown here, not run. Create a free account to press Run and type it into your own lab’s terminal.
Count the rows with that id
psql -h db -U shop -At -c "select count(*) from books where id = 902"Shown here, not run. Create a free account to press Run and type it into your own lab’s terminal.
Worked example: which retries are safe
| Request sent twice | Rows after the second | Safe to retry? |
|---|---|---|
| PATCH one book to available false | The same as after the first | Yes, it sets a value |
| POST a book with no id | One more row than after the first | No, it makes a duplicate |
| POST with an id and merge-duplicates | The same as after the first | Yes, the id names the row |
| DELETE book 900 | Gone after the first, still gone | Yes, but the lab answers 204 both times |
The cells students fill in are left blank in this preview.
Remove the test rows
psql -h db -U shop -c "delete from books where id >= 900 or title = 'Twice'"Shown here, not run. Create a free account to press Run and type it into your own lab’s terminal.
A common mistake
Warning
Many people think testing the good request is enough. It is not: test what is refused too, and see a check fail. Clean up your own test rows as well. Otherwise the second run finds the data the first left, and fails for a reason that has nothing to do with the API.