Where bias in data comes from

Bias in data is a pattern that treats some groups unevenly or leaves them out. A model learns from the rows it is given, so it copies whatever bias those rows hold.

A bank trains a loan model on its last ten years of decisions. If those decisions were uneven, the model learns the unevenness. In this lesson you find where bias enters, before any model is trained.

The population is everyone a model will be used on. The sample is the part you managed to collect.

If the sample holds few people from group B, the model sees few examples of them and learns less about them. So count each group in the sample, count each group in the population, and compare the shares.

The Run below builds an invented sample of 100 applicants: 90 from group A and 10 from group B. It prints both counts and the share of group B.

docker exec jupyter python3 -c 'a=["A"]*90+["B"]*10;print(a.count("A"),a.count("B"),a.count("B")/len(a))'

# Applicants in a sample, by group.
sample = ["A"] * 90 + ["B"] * 10
area_share_b = 0.5
share_b = sample.count("B") / len(sample)
print("Group B in the sample:", share_b)
print("Group B in the area:", area_share_b)
print("Gap:", area_share_b - share_b)
# Change 90 and 10 so the sample matches the area, then run again.

A label is the answer a model learns to predict. Often it stands in for what you really want to know.

A bank wants to know who will repay a loan. What it has is who it approved. A refused applicant never got a loan to repay, so their outcome is missing. The model learns only from people the old process let through.

Historical bias is unevenness in past decisions. Suppose staff approved fewer applicants from group B who had similar incomes.

The approved column records that. A model trained to match it will match the unevenness too, and the match will look like accuracy. The data is only repeating what the staff did.

The table shows invented numbers, and the Run below works out the two approval rates.

docker exec jupyter python3 -c 'r=[("A",1)]*72+[("A",0)]*18+[("B",1)]*4+[("B",0)]*6;print([sum(x[1] for x in r if x[0]==g)/sum(1 for x in r if x[0]==g) for g in "AB"])'

Open hires.csv in your notebook. Count how many days each weather value has, and how many of the 365 days fall on a weekend.

A model learning from this file would see some conditions far more often than others. Write two sentences on which conditions it would know least about.