What is the dot product?
The dot product multiplies two vectors to give one number that says how far they point the same way. Search tools and recommendation tools use it to score how alike two things are.
This lesson shows how to work it out, and how to turn it into a score between -1 and 1.
By the end of this lesson
Note
I can use the dot product to say how much two vectors agree.
Key words
| Word | What it means | Example |
|---|---|---|
| Dot product | multiply matching components, then add | [1, 2] and [3, 4] give 11 |
| Orthogonal | at right angles, with a dot product of 0 | |
| Angle | the turn between two vectors | |
| Cosine similarity | the dot product divided by both lengths | |
| Similarity | a score for how alike two things are |
The cells students fill in are left blank in this preview.
The dot product multiplies and adds
Multiply the matching components, then add the products. For [1, 2, 3] and [4, 5, 6] that is 1 x 4 + 2 x 5 + 3 x 6 = 4 + 10 + 18 = 32.
The answer is a single number, not a vector. Both vectors need the same dimension.
Worked example: three bike stations
| Time of day | Station A | Station B | Station C | A x B | A x C |
|---|---|---|---|---|---|
| Morning | 8 | 4 | 1 | 32 | 8 |
| Noon | 1 | 0 | 6 | 0 | 6 |
| Afternoon | 1 | 1 | 7 | 1 | 7 |
| Evening | 7 | 4 | 1 | 28 | 7 |
| Dot product | 61 | 28 |
The cells students fill in are left blank in this preview.
Work out a dot product in NumPy
docker exec jupyter python3 -c "import numpy as np; a = np.array([1, 2, 3]); b = np.array([4, 5, 6]); print(a @ b, np.dot(a, b))"Shown here, not run. Create a free account to press Run and type it into your own lab’s terminal.
The sign of the dot product shows agreement
A large positive dot product means the vectors point the same way. A dot product of 0 means they are at right angles, which is called orthogonal. A negative one means they point against each other.
The dot product also grows with length. A long vector gives a big number even when the direction is only a little alike.
Worked example: reading the sign
| Pair | Dot product | What it says |
|---|---|---|
| [1, 0] and [1, 0] | 1 | Same direction |
| [1, 0] and [5, 0] | 5 | Same direction, a longer vector |
| [1, 0] and [0, 1] | 0 | Right angles |
| [1, 0] and [-1, 0] | -1 | Opposite directions |
The cells students fill in are left blank in this preview.
A common mistake
Warning
Many people think the bigger dot product always means the more alike pair. It also grows with length. Compare directions with cosine similarity, which divides the length out, and use the plain dot product only when size matters.
Cosine similarity compares direction only
Cosine similarity is the dot product divided by the two lengths multiplied together. It cancels the size and leaves the direction.
The score is 1 for the same direction, 0 for right angles and -1 for opposite directions. It is the cosine of the angle between the vectors.
Worked example: which stations are alike?
| Pair | Dot product | Lengths multiplied | Cosine similarity |
|---|---|---|---|
| A and B | 61 | 10.72 x 5.74 = 61.6 | 0.99 |
| A and C | 28 | 10.72 x 9.33 = 100.0 | 0.28 |
The cells students fill in are left blank in this preview.
Score the stations by cosine
docker exec jupyter python3 -c "import numpy as np; a = np.array([8, 1, 1, 7]); b = np.array([4, 0, 1, 4]); c = np.array([1, 6, 7, 1]); print(round(float(a @ b / (np.linalg.norm(a) * np.linalg.norm(b))), 2), round(float(a @ c / (np.linalg.norm(a) * np.linalg.norm(c))), 2))"Shown here, not run. Create a free account to press Run and type it into your own lab’s terminal.
Station A is like B, not C
Stations A and B are busy in the morning and the evening, and quiet in between. B has fewer bikes, so its dot product is smaller. The cosine ignores that, and scores 0.99.
Station C is busy at noon and in the afternoon. It scores 0.28 with A. A vector's length is the square root of its dot product with itself.
Score two stations in Python
# Cosine similarity of two stations, in plain Python.
def dot(a, b):
total = 0
for i in range(len(a)):
total = total + a[i] * b[i]
return total
def length(a):
return dot(a, a) ** 0.5
cosine = dot([8, 1, 1, 7], [4, 0, 1, 4]) / (length([8, 1, 1, 7]) * length([4, 0, 1, 4]))
print(round(cosine, 2))
# Replace the second vector with [1, 6, 7, 1], both times.
# Predict whether the score goes up or down first.
Going further
In your own notebook, take the first week of hires and the first week of temperatures as two vectors. Score them by cosine similarity.
Then score week one against week two of the hires. Both scores will be high, because every number is positive. Say why that makes cosine similarity a weak test here.