The report nobody can use
An analyst sends a manager one line: "The average is 13." The manager asks: from how many rows? After what changes? Is it going up? How spread out is it?
A number on its own is not a description. A good description answers all of these.
Key words
| Word | What it means |
|---|---|
| Characterise | describe a dataset fully, with its size, shape, patterns and measures |
| Dataset | a collection of records held together for analysis |
| Input of cleaning | the data as it arrived, before any fixing |
| Output of cleaning | the data after cleaning: what it is now fit for |
| Statistical measure | a number that summarises data, such as the mean or the range |
| Trend | the general direction over time: up, down or flat |
The cells students fill in are left blank in this preview.
By the end of this lesson
Note
I can describe a dataset using its cleaning inputs and outputs, its patterns and trends, and its measures.
Starter
Have a go
Starter
3 questionsYour answers are marked here and are not kept. A college account keeps every student’s marks.
Four parts of a good description
| Part | Say | Example |
|---|---|---|
| Input of cleaning | What arrived | 7 rows, 2 with problems, 1 outlier |
| Output of cleaning | What is left, and what was removed | 4 rows kept, 3 removed and why |
| Patterns and trends | What you see over time or groups | Visits rise by about 2 a day |
| Statistical measures | The numbers that summarise it | Lowest 10, highest 16, median 13, mean 13 |
The cells students fill in are left blank in this preview.
Why describe the cleaning too
A mean of 13 from 4 rows is different from a mean of 13 from 4,000 rows. The reader needs to know what was removed to trust the numbers.
Say what came in, what was removed and why, then what the cleaned data shows.
Try it: build a description
raw_count = 7
kept = [10, 12, 14, 16]
low = min(kept)
high = max(kept)
mean = sum(kept) / len(kept)
print("Cleaned " + str(raw_count) + " rows to " + str(len(kept)) + ".")
print("Lowest " + str(low) + ", highest " + str(high) + ", mean " + str(mean) + ".")
# Add one more value to kept, such as 18, and run it again.
Worked example: a described dataset
| Part | Description |
|---|---|
| Input | Seven daily visit counts, one empty, one a word, one of 300 for a year group visit |
| Output | Four usable counts: 10, 12, 14, 16. Three were removed and the reasons are logged |
| Pattern and trend | Visits rise by about 2 a day. No weekly pattern can be seen in four days |
| Measures | Lowest 10, highest 16, range 6, median 13, mean 13 |
The cells students fill in are left blank in this preview.
Check
Have a go
Check
4 questionsYour answers are marked here and are not kept. A college account keeps every student’s marks.