Writing a GPU kernel

A kernel is the small function a GPU runs once for every thread. Writing one means describing the work for a single element, then launching enough threads to cover them all.

This lab has no GPU, so the launch is a pair of loops in Python. The kernel itself looks the same as it would on a GPU: find your index, check it, do one small job.

Every kernel has the same three steps. First it works out its global index. Second it checks that the index is less than the number of elements. Third it does one small job on its element.

The check is the guard. The grid is built from whole blocks, so it can hold more threads than there are elements.

# A kernel that doubles every price, launched over 10 elements.
prices = [4, 9, 2, 7, 5, 8, 1, 6, 3, 10]
n = len(prices)
doubled = [0] * n
block_dim = 4
grid_dim = 3

def kernel(block_idx, thread_idx):
    i = block_idx * block_dim + thread_idx
    if i < n:
        doubled[i] = prices[i] * 2

for block_idx in range(grid_dim):
    for thread_idx in range(block_dim):
        kernel(block_idx, thread_idx)
print(doubled)
# Make grid_dim 2, run again, and see which prices are left at 0.

The grid size is the number of elements divided by the block size, rounded up. For 10 elements in blocks of 4, that is 3 blocks, or 12 threads.

The Run below rounds up with whole numbers. 10 divided by 4 rounded down is 2, with 2 left over, so one more block is needed. A minus sign on each side turns round down into round up.

docker exec jupyter python3 -c "print(-(-10 // 4))"

The lab's kernel.py adds two vectors of 10 numbers, a and b. Its kernel is a function of a block number and a thread number.

Print the script first and find the guard. Then run it: it prints what every thread did and ends with the result, 11, 22 and so on up to 110.

docker exec jupyter cat steps/kernel.py

docker exec jupyter python3 steps/kernel.py

A kernel is safe to run in any order when each thread writes only its own element. The threads of the doubling kernel never touch each other's results.

If two threads wrote the same cell, the answer would depend on which came last. On a real GPU that order changes from run to run. So give each thread its own output.