Where does all of it go?
A college has ticket data, door logs, Wi-Fi logs, camera clips and finance records. They sit in a dozen systems.
To study them together, somebody must collect them in one place. There are three common places, and each suits a different job.
By the end of this lesson
Note
I can tell three big stores of data apart and say when each is used.
Key words
| Word | What it means | Example |
|---|---|---|
| Data warehouse | A large store of cleaned, structured data from many systems, kept for analysis. | Tickets, finance and HR joined together |
| Data lake | A large store of raw data in any form, kept as it arrived. | Logs, photos and CSV files |
| Data mart | A small slice of a warehouse for one team. | Support team figures only |
| Big Data | Data too large, fast or varied for ordinary tools. | Millions of log lines |
| Structured data | Data in fixed fields. | A table of tickets |
| Raw data | Data kept as it arrived, with no changes. | A log file |
The cells students fill in are left blank in this preview.
Starter: what you already know
Have a go
Starter: what you already know
3 questionsYour answers are marked here and are not kept. A college account keeps every student’s marks.
A warehouse keeps clean, structured data
A data warehouse collects data from many systems. The data is cleaned and put into a set structure before it goes in.
It suits reports and questions you know in advance, such as "tickets per month by team". It is fast to search, but takes work to load.
A lake keeps raw data as it is
A data lake keeps data in any form, raw, exactly as it arrived: logs, photos, sound and files.
You decide the structure later, when you read it. It is cheap and flexible, and good for new, unplanned questions. But without care it becomes a swamp that nobody can search.
A mart is a small slice
A data mart is a small slice of a warehouse for one team or topic. The support team gets only support figures.
It is quicker and simpler for that team, and the team sees only what it needs.
Check: which store?
Have a go
Check: which store?
3 questionsYour answers are marked here and are not kept. A college account keeps every student’s marks.