An export nobody can use
A drinks shop exports its orders. One line says "12; Large ;north". The next says "13;LARGE;North ". Another says "14;huge;east".
The same size is spelt three ways, and one word is not on the menu. Before anybody can report on this, somebody must tidy it. That job has five steps.
Key words
| Word | What it means | Example |
|---|---|---|
| Data wrangling | Turning messy raw data into data that is ready to use. | Tidying an order export |
| Structure | To put data into fields and rows. | Split a line at each semicolon |
| Clean | To fix or remove wrong, odd or repeated values. | Large and LARGE become large |
| Validate | To check that data is sensible and allowed. | Size must be small, medium or large |
| Enrich | To add useful extra data from somewhere else. | Add the price of each size |
| Output | To save or show the finished data. | A report line |
The cells students fill in are left blank in this preview.
By the end of this lesson
Note
I can put the five steps for tidying messy data in order and say what each does.
Starter: what you already know
Have a go
Starter: what you already know
3 questionsYour answers are marked here and are not kept. A college account keeps every student’s marks.
Wrangling makes raw data ready
Data wrangling turns messy raw data into data that is ready to use. Use it when data comes from many places, or in different shapes, or with mistakes.
It is not one job. It is five steps, done in a set order.
What each step does
- Structure. Put the data in fields and rows.
- Clean. Fix spaces, capitals, repeats and gaps.
- Validate. Check each value is allowed. Flag or remove the rest.
- Enrich. Add useful facts, such as the price of each size.
- Output. Save or show the finished data in the form people need.
Worked example: one messy line
| Step | What we have | What happened |
|---|---|---|
| Raw line | 13;LARGE;North | Messy export |
| Structure | 13 | LARGE | North | Three fields |
| Clean | 13 | large | north | Spaces and capitals fixed |
| Validate | large is allowed | Passes the check |
| Enrich | large: 4 pounds | Extra fact added |
| Output | 13: large (4 pounds) | Ready for a report |
The cells students fill in are left blank in this preview.