A hammer for every job?
A club leader wants to know if rain affects visits. A manager wants one screen of daily numbers. A researcher wants to test a theory.
One tool will not fit all three. Choosing well means asking what you need to find, what result you expect, and what you have to work with.
Key words
| Word | What it means | Example |
|---|---|---|
| Exploratory data analysis | Looking through data to see what it holds before testing ideas. | Draw graphs and count values |
| Time series forecasting | Using data over time to predict what comes next. | Next month's visits |
| Hypothesis testing | Checking if data supports or rejects an idea. | Does rain cut visits? |
| Data mining | Searching large data for hidden patterns. | Which items are bought together |
| Histogram | A graph of bars showing how many values fall in each band. | How many ages in 10 to 19 |
| Box plot | A graph showing the middle, the spread and the odd values of a set of numbers. | Spread of daily spend |
| Scatter graph | A graph of dots showing if two measures move together. | Rain against visits |
| Query | A question you ask a store of data. | Sales for each day |
| SQL | The language used to write a query. | select item from sales |
| Big data | Data so large, fast or varied that ordinary tools struggle. | Every till sale in a country |
| KPI | A key performance indicator: a number showing how well something is going. | Visits each day |
| Dashboard | One screen of the main numbers and charts. | A KPI board |
The cells students fill in are left blank in this preview.
By the end of this lesson
Note
I can give criteria for choosing tools and techniques, and match common ones to a brief.
Starter: what you already know
Have a go
Starter: what you already know
3 questionsHave a go: your answers are marked here and then forgotten. Sign in if you would like your score kept, and to pick up where you left off.Sign in to keep your score
Your answers are marked here and are not kept. A college account keeps every student’s marks.
Three selection criteria
| Criterion | Ask |
|---|---|
| Purpose | Why do I need the trend or pattern? |
| Expected outcome | Gains, trends, regressions or derived data? |
| Resources | What tools, skills, time and money do I have? |
The cells students fill in are left blank in this preview.
Tools
| Tool | Good for |
|---|---|
| Programming language: Python, R, Scala | Flexible, repeatable analysis of any size |
| Dashboard: KPIs, financial | Watching key numbers at a glance |
| Query tool: SQL | Asking questions of stored tables |
| Scatter graph and trend line | Seeing if two measures are linked |
| Histogram and box plot | Seeing the spread of values and odd ones |
The cells students fill in are left blank in this preview.
Techniques
| Technique | Use it to |
|---|---|
| Exploratory data analysis | Get to know the data first |
| Time series forecasting | Predict the future from the past |
| Hypothesis testing | Test an idea against the data |
| Data mining | Find hidden patterns in big data |
The cells students fill in are left blank in this preview.
Try it: a text histogram
ages = [11, 12, 14, 15, 15, 16, 19, 22]
bands = [0, 0, 0]
for age in ages:
if age < 10:
bands[0] = bands[0] + 1
elif age < 20:
bands[1] = bands[1] + 1
else:
bands[2] = bands[2] + 1
print("0-9 ", "#" * bands[0])
print("10-19", "#" * bands[1])
print("20-29", "#" * bands[2])
# Try it: add three ages and press Run.