The fair-looking machine
A small lender trains a program on its past decisions. The program learns that people in the South area are refused more often. It then refuses more people from the South. Nobody typed a rule about area.
The machine copied the past. If the past was unfair, the machine is unfair too.
Key words
| Word | What it means |
|---|---|
| Personal data | Information about a living person who can be identified. |
| Bias | A leaning that treats some people or results unfairly. |
| Automation | Using a computer to do a task with no person doing each step. |
| Machine learning | A way for a program to learn patterns from past data. |
| Privacy | The right to keep your own information to yourself. |
| Impact | The effect that an action has on people or an organisation. |
| Ownership | Who holds the rights over a piece of work or a dataset. |
| Licence | Permission to use work, with rules about how. |
| Dataset | A collection of data about one topic. |
The cells students fill in are left blank in this preview.
By the end of this lesson
Note
I can spot bias in automation and in myself, and explain privacy, impact and ownership in an analysis.
Starter: last lesson
Have a go
Starter: last lesson
3 questionsYour answers are marked here and are not kept. A college account keeps every student’s marks.
Where bias comes from
Bias can come from three places. The data may leave out some people. The automation or machine learning may copy unfair patterns from old data. The analyst may look only for results that fit what they already believe.
To avoid bias, check who is missing, compare groups and ask a colleague to review.
Three sources of bias
| Source | What goes wrong | A way to reduce it |
|---|---|---|
| Data | A survey asks only one area | Check who is missing and widen the sample |
| Automation or machine learning | A model copies old unfair decisions | Test results for each group |
| The analyst | Keeping only results that fit a belief | Write the question first and have a review |
The cells students fill in are left blank in this preview.
Worked example: the approval rate by area
| Area | Applications | Approved | Approved out of every 100 |
|---|---|---|---|
| North | 10 | 8 | 80 |
| South | 10 | 3 | 30 |
| Gap between areas | 50 out of every 100 |
The cells students fill in are left blank in this preview.
Try it: compare the areas
SELECT area, COUNT(*) AS applications, SUM(approved) AS approved
FROM applications
GROUP BY area
ORDER BY area;Check: bias
Have a go
Check: bias
3 questionsYour answers are marked here and are not kept. A college account keeps every student’s marks.