Where does all of it go?
A shop group has till records, website visits, loyalty cards, camera clips and finance records. They sit in a dozen systems.
To study them together, an analyst needs them in one place. There are three common places, and each suits a different job.
Key words
| Word | What it means | Example |
|---|---|---|
| Data warehouse | A large store of cleaned, structured data from many systems, kept for analysis. | Sales, finance and stock joined together |
| Data lake | A large store of raw data in any form, kept as it arrived. | Logs, photos and CSV files |
| Data mart | A small slice of a warehouse for one team. | North shop figures only |
| Big Data | Data too large, fast or varied for ordinary tools. | Millions of till records |
| Structured data | Data in fixed fields. | A table of sales |
| Raw data | Data kept as it arrived, with no changes. | A log file |
| Scale | How big the data is and how fast it grows. | Ten rows or ten million |
The cells students fill in are left blank in this preview.
By the end of this lesson
Note
I can tell three big stores of data apart and say when each is used.
Starter: what you already know
Have a go
Starter: what you already know
3 questionsYour answers are marked here and are not kept. A college account keeps every student’s marks.
A warehouse keeps clean, structured data
A data warehouse collects data from many systems. The data is cleaned and put into a set structure before it goes in.
It suits reports and questions you know in advance, such as "sales per month by shop". It is fast to search, but takes work to load.
A lake keeps raw data as it is
A data lake keeps data in any form, raw, exactly as it arrived: logs, photos, sound and files.
You decide the structure later, when you read it. It is cheap and flexible, and good for new, unplanned questions. But without care it becomes a swamp that nobody can search.
A mart is a small slice
A data mart is a small slice of a warehouse for one team or topic. The north shop gets only north shop figures.
It is quicker and simpler for that team, and the team sees only what it needs.
Check: which store?
Have a go
Check: which store?
3 questionsYour answers are marked here and are not kept. A college account keeps every student’s marks.