A million readings before lunch
A city has 2,000 fitness trackers lent out to volunteers. Each one sends a reading every second. That is over 172 million readings a day.
Some are numbers, some are text notes, some are photos. Some are wrong. A spreadsheet cannot cope. This kind of problem needs new tools and new thinking.
Key words
| Word | What it means | Example |
|---|---|---|
| Big Data | Data so large, fast or varied that ordinary tools struggle with it. | Readings from 2,000 trackers |
| Dimension | One way to measure or describe something. | Volume |
| Volume | How much data there is. | Over 172 million readings a day |
| Variety | How many different kinds of data there are. | Text, numbers, pictures |
| Variability | How much the data changes in shape or meaning. | The same word means different things |
| Velocity | How fast the data arrives. | One reading a second per tracker |
| Veracity | How true and reliable the data is. | A faulty sensor gives wrong numbers |
| Value | How useful the data is for a decision. | Spotting an unwell patient early |
| Gather | To collect data from its sources. | Receive a reading |
| Maintain | To keep data accurate and up to date. | Fix wrong records |
The cells students fill in are left blank in this preview.
By the end of this lesson
Note
I can name the six Vs and say how each changes the way data is kept and changed.
Starter: what you already know
Have a go
Starter: what you already know
3 questionsYour answers are marked here and are not kept. A college account keeps every student’s marks.
What makes data "big"?
Big Data is data that is too large, too fast or too varied for ordinary tools. It is not only about size.
Big Data has several dimensions. There are six, and each begins with V: volume, variety, variability, velocity, veracity and value.
The six Vs and what they change
| V | Question | Effect on keeping the data |
|---|---|---|
| Volume | How much? | Needs big, cheap storage and ways to spread the work |
| Variety | How many kinds? | Needs ways to store text, numbers and pictures together |
| Variability | How much does it change? | Needs rules that cope when meaning or shape shifts |
| Velocity | How fast does it arrive? | Needs fast collection and quick processing |
| Veracity | How true is it? | Needs checks and cleaning before use |
| Value | How useful is it? | Decides if the cost of keeping it is worth paying |
The cells students fill in are left blank in this preview.
How each V changes gathering and keeping
Each V changes how data is gathered and maintained.
High volume means more disks. High velocity means collecting and processing in real time. High variety means many formats to read. Low veracity means more time cleaning. Low value means asking if the data should be kept at all.
Variability means the same word can mean different things on different days, so the rules must be updated.
How each V changes transforming the data
| V | What it does to the changes you make | Example |
|---|---|---|
| Volume | A change must run over every row, so it needs to be quick or run in parts | Add up over 172 million readings |
| Variety | Each kind needs its own change, then all must be joined | Turn photos into tags, text into numbers |
| Variability | The change must cope when the shape or meaning moves | A new column appears in the feed |
| Velocity | The change must keep up as data arrives | Average each minute as readings come in |
| Veracity | Cleaning must come before any analysis | Remove impossible heart rates first |
| Value | Spend effort only on data that is worth the change | Skip a feed nobody uses |
The cells students fill in are left blank in this preview.