Easier to read with cream paper, a tint, more contrast or larger text? Choose your colours and text, once, for every page.

Free lesson

What data an app collects, and why

After this lesson: I can say what data a service collects and why.

45 min lesson2 commandsJupyterLab data science lab: pandas, Matplotlib and scikit-learn in the browser

The app that knows your Saturday

You hire a bike with an app on your phone. Before the bike unlocks, the app has your name, your email and a bank card.

When you hand the bike back, it knows where you rode and for how long. Why does it need all of that? And who else gets to see it?

By the end of this lesson

Note

I can say what data a service collects and why.

Key words

WordWhat it means
Datafacts that are recorded about something
Personal datadata that can be linked to one person
Purposethe reason data is collected
Consenta free and clear yes to one use of your data
Privacyyour control over who knows things about you
Anonymiseto remove what links data to a person

The cells students fill in are left blank in this preview.

Before you start

Note

Press Start the lab on the bar at the top and wait until it says it is running. Each Run button types a command into the lab's terminal. docker exec jupyter runs it inside the lab's JupyterLab, which has Python. If a Run says the lab is still starting, wait and press it again.

Every piece of data needs a reason

A service should collect data only for a purpose it can name. The bike app needs your card to charge you. It needs the bike's place to find it again.

Does it need your date of birth? Perhaps, to check you are old enough to ride. Does it need your phone's contacts? No. Collecting only what a purpose needs is called data minimisation.

Worked example: what a bike app collects

DataWhy the app wants itNeeded?
Name and emailto make your account and send receiptsYes
Bank cardto charge you for the rideYes
Where the bike isto find it and to work out the priceYes
Date of birthto check you are old enoughYes, if the rules need it
Your contactsnothing the service doesNo
Your photosnothing the service doesNo

The cells students fill in are left blank in this preview.

See what the bike shop records

Command
docker exec jupyter head -4 hires.csv

Shown here, not run. Create a free account to press Run and type it into your own lab’s terminal.

This file has no person in it

The first line names the columns: the date, the month, the weekday, the weather, the temperature, the rain and the hires. Each line after it is one day.

Nothing says who rode. The shop wants to know how many bikes to put out each morning. A count for each day answers that, so it never needs a name.

Count the rows and columns

Command
docker exec jupyter python3 steps/load.py

Shown here, not run. Create a free account to press Run and type it into your own lab’s terminal.

Personal data can point to you

Personal data is any data that can be linked to one person. A name is personal data. So is an email address, a face in a photo and a phone's location.

Some data only points to a person when it is put together. A postcode alone covers many homes. A postcode, a birth date and the time somebody always rides can point to one person.

The law, in our own words

In the UK and the EU, the law says a service needs a fair reason to use your personal data. Consent is one reason. A contract is another, such as an address for a delivery.

The service must tell you what it collects and why, in plain words. It must keep the data safe, and only for as long as it needs it. You can ask to see what it holds about you.

Anonymising removes the link to a person

To study data safely, anonymise it. Delete names and emails. Make exact values rougher: an age of 17 becomes the band 10 to 19, and a postcode keeps only its first part.

Swapping names for codes is not enough. That is called pseudonymised data, because whoever keeps the list of codes can put the names back.

Anonymise one record

Code: python
# One rider's record, as the app stored it.
rider = {"name": "Priya Shah", "postcode": "LS6 2AB", "age": 17, "rides": 12}

# Keep only what the study needs, and make it rougher.
low = rider["age"] // 10 * 10
safe = {}
safe["area"] = rider["postcode"].split(" ")[0]
safe["age_band"] = str(low) + "-" + str(low + 9)
safe["rides"] = rider["rides"]
print(safe)

# Change the age to 34 and the postcode to "M1 4BT".
# Press Run again. What changes in the answer?

A common mistake

Warning

Many people think deleting the name makes data anonymous. It does not, if what is left still points to one person. A full postcode and a birth date can pick out one rider. Make values rougher, or leave them out, until no row stands for one person.

In your notebook: anonymise a table

Code: python
import pandas as pd

riders = pd.DataFrame({"name": ["Priya", "Tom", "Ana"],
                       "postcode": ["LS6 2AB", "LS1 4DY", "M1 4BT"],
                       "age": [17, 34, 17],
                       "rides": [12, 3, 8]})
# Keep the first part of each postcode, then drop what names a person.
riders["area"] = riders["postcode"].str.split(" ").str[0]
riders.drop(columns=["name", "postcode"])

Going further: read a real privacy notice

Open JupyterLab from the bar, make a new notebook and type the code above into a cell. Press Shift and Enter.

Then pick an app you use. Find its privacy notice and list three pieces of data it collects. Next to each, write the purpose it gives. Is any item one it does not need?

About this lab and its source: JupyterLab data science lab: pandas, Matplotlib and scikit-learn in the browser

Teaching a class? Set this module for your students with a teacher guide, marked assignments and a lab for every student.