Fairness and bias in AI

Bias in AI means a program treats some groups of people less fairly than others. A cinema uses an AI tool to pick who gets an interview for a weekend job.

Twenty people apply. Almost everyone it picks lives on the east side of town. Nobody told it to do that, so how did it happen?

The cinema tool learned from past staff. Most of them lived on the east side, because the old bus route came from there.

So the tool learned "east side" as part of a good worker. That is bias. The training data leaned one way, so the results lean too.

You can often spot bias with simple counting. Split the people into groups, then compare how often each group was picked.

If two groups applied in the same way but one is picked far more, ask why.

# Each person is [group, picked]. True means picked.
people = [
    ["east", True], ["east", True], ["east", False], ["east", True],
    ["west", False], ["west", True], ["west", False], ["west", False],
]

applied = {}
picked = {}
for person in people:
    group = person[0]
    if group not in applied:
        applied[group] = 0
        picked[group] = 0
    applied[group] = applied[group] + 1
    if person[1]:
        picked[group] = picked[group] + 1

for group in applied:
    print(group, picked[group], "out of", applied[group])

# Picked out of applied for each group, from the Run above.
east_picked = 3
east_applied = 4
west_picked = 1
west_applied = 4

east_rate = round(east_picked * 100 / east_applied)
west_rate = round(west_picked * 100 / west_applied)
print("East:", east_rate, "%")
print("West:", west_rate, "%")
print("Gap:", east_rate - west_rate, "%")

Say what you noticed, with the numbers. Tell somebody who can change it: a teacher, a manager, or the report button in an app. Ask for a person to check the decision.

Check your own use too. If you ask AI for pictures or stories about people, look at who is shown and who is missing.

Think of a picture tool asked for "a scientist" or "a nurse". If it always shows the same kind of person, that is bias in its examples.

Write down two questions you would ask the people who made such a tool.