What might you borrow next?
A recommendation system suggests items a person has not yet tried, using what other people with similar tastes liked. Shops, video sites and libraries all run one.
The method needs no knowledge of the items themselves. You will learn to measure how alike two people are and turn that into a score for each unseen item.
By the end of this lesson
Note
I can recommend items by scoring them with the likes of similar users.
Key words
| Word | What it means |
|---|---|
| Recommender | a system that suggests items to a person |
| Collaborative filtering | recommending from what similar users liked |
| Similarity | a number for how alike two users are |
| Score | a number for how strongly to suggest an item |
| Cold start | having no history for a new user or item |
The cells students fill in are left blank in this preview.
Similar users like the same things
Take a library where each member has marked the books they liked. To compare two members, count the books they both liked, and divide by the number of different books either one liked.
This share is their similarity. It runs from 0 (nothing in common) to 1 (exactly the same likes).
Worked example: five members and what they liked
| Member | Liked | Shared with Asha | Similarity to Asha |
|---|---|---|---|
| Asha | Dune, Emma | - | - |
| Ben | Dune, Emma, Persuasion | 2 of 3 different books | 2 / 3 = 0.667 |
| Cai | Emma, Persuasion, Hamlet | 1 of 4 different books | 1 / 4 = 0.25 |
| Dee | Hamlet, Ulysses | 0 of 4 different books | 0 / 4 = 0 |
| Eli | Dune, Ulysses | 1 of 3 different books | 1 / 3 = 0.333 |
The cells students fill in are left blank in this preview.
Measure how alike two members are
likes = {
"Asha": ["Dune", "Emma"],
"Ben": ["Dune", "Emma", "Persuasion"],
"Cai": ["Emma", "Persuasion", "Hamlet"],
"Dee": ["Hamlet", "Ulysses"],
"Eli": ["Dune", "Ulysses"],
}
def similarity(first, second):
shared = 0
for book in first:
if book in second:
shared = shared + 1
everything = len(first) + len(second) - shared
return shared / everything
for name in ["Ben", "Cai", "Dee", "Eli"]:
print("Asha and", name, round(similarity(likes["Asha"], likes[name]), 3))
# Add Fay, who likes Dune and Hamlet. How alike are Asha and Fay?
Check Asha and Eli with Python
docker exec jupyter python3 -c "print(round(len({'Dune', 'Emma'} & {'Dune', 'Ulysses'}) / len({'Dune', 'Emma'} | {'Dune', 'Ulysses'}), 3))"Shown here, not run. Create a free account to press Run and type it into your own lab’s terminal.
Score an item by who liked it
Now suggest a book to Asha. Look at each book Asha has not read. Add up the similarity of every member who liked it.
A book liked by close neighbours gets a high score. A book liked only by people unlike Asha stays low. The book with the top score is the recommendation.
Worked example: scores for Asha
| Book | Liked by | Sum | Score |
|---|---|---|---|
| Persuasion | Ben (0.667), Cai (0.25) | 0.667 + 0.25 | 0.917 |
| Ulysses | Dee (0), Eli (0.333) | 0 + 0.333 | 0.333 |
| Hamlet | Cai (0.25), Dee (0) | 0.25 + 0 | 0.25 |
The cells students fill in are left blank in this preview.
Score every unseen book for Asha
likes = {
"Asha": ["Dune", "Emma"], "Ben": ["Dune", "Emma", "Persuasion"],
"Cai": ["Emma", "Persuasion", "Hamlet"], "Dee": ["Hamlet", "Ulysses"], "Eli": ["Dune", "Ulysses"],
}
def similarity(first, second):
shared = 0
for book in first:
if book in second:
shared = shared + 1
return shared / (len(first) + len(second) - shared)
scores = {}
for other in likes:
if other != "Asha":
weight = similarity(likes["Asha"], likes[other])
for book in likes[other]:
if book not in likes["Asha"]:
if book not in scores:
scores[book] = 0
scores[book] = scores[book] + weight
for book in scores:
print(book, round(scores[book], 3))
# Which book would you suggest to Asha? Change Eli's list to see the scores move.
Check the score of Persuasion
docker exec jupyter python3 -c "print(round(2 / 3 + 1 / 4, 3))"Shown here, not run. Create a free account to press Run and type it into your own lab’s terminal.
A common mistake
Warning
Many people think a recommender knows what a person likes. It only knows what people with a similar history did. A new member has no history at all, which is the cold start problem, so a shop first falls back on its most popular items.
The idea also works on items
You can also compare items. Two books are alike if the same members liked both. To recommend, take a book a person liked and suggest the books most like it.
Large shops often prefer this, because items change less often than people. The counting is the same, with rows and columns swapped.
Worked example: two books compared
| Book | Liked by | Shared with Dune | Similarity to Dune |
|---|---|---|---|
| Dune | Asha, Ben, Eli | - | - |
| Emma | Asha, Ben, Cai | Asha, Ben | 2 / 4 = 0.5 |
| Ulysses | Dee, Eli | Eli | 1 / 4 = 0.25 |
| Hamlet | Cai, Dee | none | 0 / 5 = 0 |
The cells students fill in are left blank in this preview.
Going further
Add a sixth member to the likes in the first code block and watch the scores for Asha change. Then explain why a book that every member liked gives no useful information about taste.
In a real shop, ratings from 1 to 5 hold more detail than liked or not. How would you change the similarity to use them?