How can a sample describe a whole group?
Sampling means measuring a small part of a group to learn about the whole group. A confidence interval is a range that shows how sure you can be of what you learned.
You need both because nobody can measure every customer or every day. You measure some, and then you say how far you trust the answer.
By the end of this lesson
Note
I can say how a sample estimates a population and how sure we are.
Key words
| Word | What it means |
|---|---|
| Population | the whole group you want to know about |
| Sample | the part of the group you actually measure |
| Bias | a way of choosing that favours some values |
| Standard error | how far a sample mean usually sits from the true mean |
| Confidence interval | a range that catches the true value in most samples |
The cells students fill in are left blank in this preview.
A sample mean estimates the population mean
The population is every value you care about, such as all 365 days. A sample is some of them. The mean of the sample is your estimate of the population mean.
An estimate is only as good as the way the sample was chosen. If the choice favours some values, the sample has bias.
Describe the first 30 days of the year
docker exec jupyter python3 -c "import pandas as pd; s = pd.read_csv('hires.csv')['hires'].head(30); print(len(s), round(s.mean(), 1), round(s.std() / len(s) ** 0.5, 1))"Shown here, not run. Create a free account to press Run and type it into your own lab’s terminal.
Describe every twelfth day of the year
docker exec jupyter python3 -c "import pandas as pd; s = pd.read_csv('hires.csv')['hires'].iloc[::12]; print(len(s), round(s.mean(), 1), round(s.std() / len(s) ** 0.5, 1))"Shown here, not run. Create a free account to press Run and type it into your own lab’s terminal.
Worked example: two samples of one year
| Sample | Days | Mean | Standard error | Verdict |
|---|---|---|---|---|
| The first 30 days | 30 | 190.7 | 11.3 | All winter days, too low |
| Every twelfth day | 31 | 266.9 | 12.5 | Spread over the year, close |
| All 365 days | 365 | 270.3 | The population mean |
The cells students fill in are left blank in this preview.
The standard error shrinks as the sample grows
Take another sample and you get another mean. The standard error says how far such means usually sit from the true mean. You find it by dividing the standard deviation by the square root of the sample size.
A bigger sample gives a smaller error. But the square root means you need four times the data to halve it.
Worked example: error against sample size
| Sample size | Square root | Standard deviation 79.7 divided by it |
|---|---|---|
| 25 | 5 | 15.9 |
| 100 | 10 | 8.0 |
| 400 | 20 | 4.0 |
The cells students fill in are left blank in this preview.
Mean plus or minus two standard errors
A confidence interval goes from the mean minus two standard errors to the mean plus two. If you took many samples and made an interval from each, about 95 in 100 would catch the true mean.
This course uses 2 as the multiplier. A very small sample needs a slightly bigger one.
Make an interval from each sample
docker exec jupyter python3 -c "import pandas as pd; h = pd.read_csv('hires.csv'); ci = lambda x: print(round(x.mean() - 2 * x.std() / len(x) ** 0.5, 1), round(x.mean() + 2 * x.std() / len(x) ** 0.5, 1)); ci(h['hires'].head(30)); ci(h['hires'].iloc[::12])"Shown here, not run. Create a free account to press Run and type it into your own lab’s terminal.
A common mistake
Warning
Many people think a narrow interval means a trustworthy answer. A narrow interval from a biased sample is precise and wrong. The first 30 days gave 168.0 to 213.4, which misses the true mean of 270.3 by a long way.
Build an interval in Python
# Hires on eight days chosen to describe one station.
values = [190, 240, 210, 260, 300, 220, 250, 230]
total = 0
for one in values:
total = total + one
mean = total / len(values)
squares = 0
for one in values:
squares = squares + (one - mean) ** 2
spread = (squares / (len(values) - 1)) ** 0.5
error = spread / len(values) ** 0.5
print("mean:", mean)
print("standard error:", round(error, 1))
print("interval:", round(mean - 2 * error, 1), "to", round(mean + 2 * error, 1))
# Add the same eight numbers to the list again, so it holds sixteen.
# Does the interval get wider or narrower?
Going further
In your own notebook, take every seventh day with iloc[::7]. Is its interval narrower than the interval from every twelfth day? Does it catch 270.3?
Then take every seventh day starting from the 3rd, iloc[2::7]. Say why a sample that is the same weekday every time could mislead you.