Easier to read with cream paper, a tint, more contrast or larger text? Choose your colours and text, once, for every page.

Preview

DA 3.8: Gather, cleanse and standardise

This is the first page of the lesson. Read it and have a go at the activities. The rest is in the full lesson.

60 minutes

One date, two spellings

A made-up web shop sends an export. One row says 2026-09-01. Another says 02/09/2026. Both are real dates.

Sort the file by date and the second row lands in the wrong place. Today we turn a messy export into data that sorts, adds up and joins.

Key words

WordWhat it meansExample
RequirementWhat the finished data must be able to do or show.One row per sale in September
GatherCollect the data you need from each source.Export the web orders and the till sales
RemoveTake out rows that should not be there.Test orders and exact repeats
EditChange a value so it is tidy and correct.Take spaces off a name
ParseSplit or read a piece of text so its parts can be used.Read 02/09/2026 as a date
StandardiseWrite every value in one agreed style.Every date as year, month, day
Test rowA made-up row used to check a system, not a real sale.TEST ORDER

The cells students fill in are left blank in this preview.

By the end of this lesson

Note

I can pick the sources a requirement needs, gather the data, and cleanse and standardise it by removing, editing and parsing.

Starter: what you already know

Have a go

Starter: what you already know

3 questions
Have a go: your answers are marked here and then forgotten. Sign in if you would like your score kept, and to pick up where you left off.Sign in to keep your score

Your answers are marked here and are not kept. A college account keeps every student’s marks.

Start from the requirement

The shop owner writes the requirement: "One row for every sale in September, with the date, who bought, what and the price in pounds."

Read it for the fields it needs. Then ask which source holds each field. Gather only what the requirement needs.

Worked example: requirement to sources

Field the requirement needsSourceGathered as
Date of the saleWeb export and till exportText, in two styles
Who boughtWeb export onlyText with extra spaces
What was boughtTill exportText in mixed case
Price in poundsBoth exportsText such as GBP 4.50

The cells students fill in are left blank in this preview.

Try it: look at the raw export

Code: duckdb
select * from web_raw;

-- Spot four problems: spaces, capitals, two date styles, and a test row.

Three cleansing jobs

Remove. Take out rows that do not belong: test rows and exact repeats.

Edit. Fix values in place: take off extra spaces, make letters one case.

Parse. Read text so its parts can be used: turn 02/09/2026 into a real date, and GBP 12.00 into the number 12.00.

Then standardise: every value in a column ends up in one style.

The lesson goes on

5 more parts of this lesson are in the full curriculum, with the tasks, the final check and the labs. A college account gives your students all of it.