Cleaning by hand does not scale
The club list has 5 rows. You can fix it with a pencil. Next term the list has 5,000 rows, and the term after that it arrives again with new mistakes.
A cleaning process is a set of steps written down once, then run on any list.
Key words
| Word | What it means |
|---|---|
| Process | a set of steps carried out in order |
| Cleaning process | steps that turn unfit data into data that is fit for purpose |
| Decompose | split data into its smaller parts |
| Recompose | join the parts back together into a whole |
| Cleaning log | a record of what was changed or removed, and why |
| Dataset | a collection of records held together for analysis |
| Fit for purpose | good enough for the job the data is needed for |
The cells students fill in are left blank in this preview.
By the end of this lesson
Note
I can design a process that handles errors, removes unusable records, splits and joins data, and use it to clean a dataset.
Starter
Have a go
Starter
3 questionsYour answers are marked here and are not kept. A college account keeps every student’s marks.
Four jobs every process covers
- Handle errors in values: spaces, capitals, wrong types and formats.
- Handle records that are unfit: blank, incomplete, duplicate or old.
- Decompose values that hold two facts.
- Recompose them in the shape the job needs.
The order matters. Fix values first, so that duplicates show up as identical.
Worked example: five rows in, two rows out
| Row | What is wrong | What the process does |
|---|---|---|
| 1: " amy REED", 8, 07/09/2026 | Spaces and capitals | Keeps it as Amy Reed, 8, 2026-09-07 |
| 2: empty | Blank record | Drops it |
| 3: Bo Jones, "7 ", 25/12/2025 | A space after the year | Keeps it as Bo Jones, 7, 2025-12-25 |
| 4: "AMY reed ", 8, 07/09/2026 | Same as row 1 after cleaning | Drops it as a duplicate |
| 5: Cy Dean, empty year, 01/01/2026 | Incomplete record | Drops it |
The cells students fill in are left blank in this preview.
Check the order
Have a go
Check the order
3 questionsYour answers are marked here and are not kept. A college account keeps every student’s marks.