One operation on a whole array
Vectorising means replacing a loop over the items of a list by one operation on the whole array. It is the first step towards a GPU, because the same small job on many independent items is exactly what a GPU is built for.
A shop that multiplies every price by a quantity does the same sum for each line. Nothing in one line depends on another.
By the end of this lesson
Note
I can replace a loop by one operation on a whole array.
Key words
| Word | What it means | Example |
|---|---|---|
| Array | a row of numbers of one type, stored together | |
| Element-wise | done to each item on its own | a + b |
| Vectorised | written as one operation on a whole array | |
| Independent | not needing the result of another item | |
| Reduction | many numbers combined into one, such as a sum |
The cells students fill in are left blank in this preview.
An element-wise operation treats each item alone
An element-wise operation does the same sum to every item, and each result needs only its own inputs. Adding two lists item by item is one.
Because the items are independent, they can be done in any order, or all at once. That is the property a parallel machine uses.
Worked example: adding two arrays
| Position | a | b | a + b |
|---|---|---|---|
| 0 | 1 | 10 | 11 |
| 1 | 2 | 20 | 22 |
| 2 | 3 | 30 | 33 |
The cells students fill in are left blank in this preview.
Run a loop over order lines
# One order line at a time: price times quantity.
prices = [4, 12, 7, 3]
quantities = [10, 2, 5, 20]
totals = []
operations = 0
for i in range(len(prices)):
totals.append(prices[i] * quantities[i])
operations = operations + 1
print(totals)
print("operations written in Python:", operations)
# Add two more prices and quantities, then run again.
NumPy writes the loop as one operation
NumPy is a Python library for arrays. With it, a * 10 multiplies every item of the array a by 10, and a + a adds two arrays item by item. There is no loop in your code.
The loop still happens, but inside fast compiled code. The first Run below shows two such operations on a three-item array.
Two operations on an array
docker exec jupyter python3 -c "import numpy as np; a = np.array([1, 2, 3]); print(a * 10, a + a)"Shown here, not run. Create a free account to press Run and type it into your own lab’s terminal.
Counting operations instead of seconds
The lab's vectors.py adds up the squares of 1 to 1000 twice: with a loop, and with one NumPy operation. Both give 333833500, and the script says the results are equal.
It counts the operations your Python code writes, not seconds. The loop writes 2000, a multiply and an add for each of 1000 numbers. The vectorised version writes 1. The sums are the same. Fewer instructions come from you.
Compare the loop with one operation
docker exec jupyter python3 steps/vectors.pyShown here, not run. Create a free account to press Run and type it into your own lab’s terminal.
A reduction turns many numbers into one
Some jobs need the items together. A sum is a reduction: many numbers become one. Squaring is element-wise, then a sum reduces the squares.
The Run below squares 1 to 5 and then adds them. Element-wise steps parallelise easily. A reduction needs the partial results merged, which is a small serial part.
Square, then add
docker exec jupyter python3 -c "import numpy as np; a = np.arange(1, 6); print(a * a, int(a.sum()))"Shown here, not run. Create a free account to press Run and type it into your own lab’s terminal.
A common mistake
Warning
Many people think a vectorised version does fewer sums. It does the same sums, with fewer instructions written in Python. Count what the machine does separately from what you write, and remember the speed comes from doing independent items together.