Cosine similarity between texts

Cosine similarity is a number from 0 to 1 that says how alike two texts are. It compares the direction of their word-count vectors, so a long text and a short one about the same thing still match.

A search engine uses an idea like this to rank pages against your query. You will build it from a list of numbers.

To compare texts with arithmetic, turn each one into a vector, a list of numbers. First fix a vocabulary, a list of words in a set order. Then count each word of the vocabulary in the text.

Both texts use the same vocabulary, so place 1 of each vector means the same word.

vocabulary = ["bus", "late", "exam", "monday", "library"]
text = "the bus is late the bus is full"
counts = []
for word in vocabulary:
    counts.append(text.split().count(word))
print(counts)
# Change the text to "the exam is on monday". What are the counts now?

The dot product multiplies two vectors place by place and adds the results. A place where both have the word adds a positive number. A place where one has 0 adds nothing.

A and C: 2 times 1, plus 1 times 1, plus zeros, makes 3. A and B share no word, so their dot product is 0.

A long text has a big dot product just for being long. To remove that, divide by the two lengths. The length of a vector is the square root of its dot product with itself.

A has length the square root of 5, about 2.236. C has the square root of 2, about 1.414. So 3 divided by 2.236 times 1.414 is 0.949: very alike.

docker exec jupyter python3 -c "import math; a=[2,1,0,0,0]; c=[1,1,0,0,0]; print(round(sum(x*y for x,y in zip(a,c))/(math.sqrt(5)*math.sqrt(2)),3))"

docker exec jupyter python3 -c "import math; print(round(math.sqrt(5), 3), round(math.sqrt(2), 3))"

import math

def cosine(a, b):
    top = 0
    for n in range(len(a)):
        top = top + a[n] * b[n]
    return top / (math.sqrt(sum_squares(a)) * math.sqrt(sum_squares(b)))

def sum_squares(v):
    total = 0
    for x in v:
        total = total + x * x
    return total

print(round(cosine([2, 1, 0, 0, 0], [1, 1, 0, 0, 0]), 3))
print(round(cosine([1, 2], [2, 4]), 3))
# Try [1, 0] and [0, 1]. What does the answer mean?

Turn a search query into a vector, compare it with every page, and show the highest cosines first. Compare a new support ticket with old ones to find a likely answer.

Counts can be replaced by TF-IDF weights, so rare words count for more. The method does not change.

Write three short texts about your own week and put them in a table like the one above. Work out which two are the most alike, by hand, then check with the function.

Then explain why two texts can score 1 while meaning opposite things, such as "the bus is late" and "the bus is not late".