Twelve Amys
A club collects sign-ups from three classes. When the lists are joined, one pupil appears as "Amy Reed", "amy reed" and "Amy Red". Another pupil has no year group at all.
The club wants to count members in each year. Can it trust this list yet? Not until it is cleaned.
Key words
| Word | What it means |
|---|---|
| Fit for purpose | good enough for the job the data is needed for |
| Unfit for purpose | not good enough for that job, so it must be cleaned or not used |
| Cleaning | finding and fixing or removing faults in data |
| Incomplete | a value is missing |
| Erroneous | a value is present but wrong |
| Transposition | two characters swapped by mistake, such as 1024 for 1042 |
| Dataset | a collection of records held together for analysis |
The cells students fill in are left blank in this preview.
By the end of this lesson
Note
I can say whether data is fit for a purpose, and name the kinds of error that need cleaning.
Starter
Have a go
Starter
3 questionsYour answers are marked here and are not kept. A college account keeps every student’s marks.
Fit depends on the job
Data is not good or bad on its own. It is fit or unfit for a purpose.
A list with some misspelt names is fit for counting members in each year, if the year column is complete. It is unfit for sending letters, where every name must be right.
Worked example: kinds of error to clean
| Error | What it looks like | Why it hurts |
|---|---|---|
| Incomplete value | Year group left empty | Counts by year are too small |
| Erroneous value | Year group 99 | A wrong number hides in the totals |
| Different type or format | Year as 8 in one list and eight in another; dates as 07/09 and 2026-09-07 | Sorting and joining fail |
| Different field name | Column called Year in one list and Yr in another | Lists will not join up |
| Case error | amy reed and Amy Reed | Counted as two pupils |
| Transposition | Code 1024 for 1042 | A different record is matched |
| Spelling error | Amy Red for Amy Reed | Counted as two pupils |
The cells students fill in are left blank in this preview.
Try it: the computer sees two names
a = "Amy Reed"
b = "amy reed"
print(a == b)
print(a.lower() == b.lower())
c = "Amy Reed "
print(a == c)
print(a == c.strip())
# Capitals and spare spaces make two values that look the same compare as different.
Check: name the error
Have a go
Check: name the error
4 questionsYour answers are marked here and are not kept. A college account keeps every student’s marks.