"Then it just sort of works"
A design says: "The file is loaded, and then the points are worked out." A builder asks: loaded how? From where? What if a row is bad? What exactly goes into the points step?
A good design leaves no gap for a guess.
Key words
| Word | What it means |
|---|---|
| Process | a series of steps that turns an input into an output |
| Input | what goes into a step |
| Output | what a step gives out |
| Data assurance | the checks that make data accurate and reliable |
| Flowchart | a diagram of steps joined by arrows |
| Dependency | when one step needs the result of another |
The cells students fill in are left blank in this preview.
By the end of this lesson
Note
I can define each step of a data process with its input, action, output and check, so that every step uses the one before.
Starter
Have a go
Starter
3 questionsYour answers are marked here and are not kept. A college account keeps every student’s marks.
This is practice
Important
The real task comes from the exam board. This lesson uses an invented organisation and its own made-up data. Your teacher tells you how the real tasks are set and what you may bring.
Worked example: each step defined
| Step | Input | Action | Output | Data assurance |
|---|---|---|---|---|
| 1 Read | The sales file | Read every line into a list | A list of raw rows | Count the lines read |
| 2 Check | The raw rows | Test type, range and date | Good rows and a problem list | Every row is in one list or the other |
| 3 Clean | The good rows | Trim spaces, fix capitals | Clean rows | Re-run the checks |
| 4 Load | The clean rows | Insert into the tables | Rows in the tables | Rows loaded equals clean rows |
| 5 Points | The loaded sales | One point for each whole pound | Points for each customer | Total points equals total pounds |
| 6 Report | The points | Sort and print | A one page report | Check one customer by hand |
The cells students fill in are left blank in this preview.
Run it: each step uses the last
raw = [["Maya", "12"], ["Ben", "oops"], ["Maya", "8"], ["Ben", "5"]]
problems = []
good = []
for row in raw:
if row[1].isdigit():
good.append([row[0], int(row[1])])
else:
problems.append(row)
points = {}
for row in good:
if row[0] in points:
points[row[0]] = points[row[0]] + row[1]
else:
points[row[0]] = row[1]
print("read", len(raw), "good", len(good), "problems", len(problems))
print(points)Rules for a good step
Each step has one job. Its input is the output of an earlier step, so you can follow the data from the start to the end.
Say what happens to a bad value. The problem list is an output too: nothing is thrown away without a record.
End every step with a check that you could run.