A digital image is a grid of numbers
A digital image is a grid of tiny dots called pixels, and each pixel is stored as a number or a few numbers. A program never sees a face or a coin. It sees the numbers, and computer vision is the work of turning them into an answer.
In this lesson you read those numbers with Pillow and NumPy, and learn where each one sits.
By the end of this lesson
Note
I can read a picture as an array of numbers and find a pixel by its row and column.
Key words
| Word | What it means | Example |
|---|---|---|
| Pixel | one dot of a picture, stored as numbers | |
| Array | a grid of numbers with rows and columns | [[0, 9], [9, 0]] |
| Mode | how the numbers of a pixel are laid out | RGB |
| Channel | one of the numbers in a pixel, such as red | |
| Grey level | a brightness from 0 (black) to 255 (white) |
The cells students fill in are left blank in this preview.
A grey picture is rows of grey levels
Think of a tiny grey picture, three rows tall and four columns wide. Each pixel holds one grey level: 0 is black and 255 is white.
The picture is a list of rows, and each row is a list of pixels. The size is its width and its height: here 4 by 3.
Worked example: a 4 by 3 grey picture
| Row | Column 0 | Column 1 | Column 2 | Column 3 |
|---|---|---|---|---|
| 0 | 12 | 40 | 200 | 35 |
| 1 | 90 | 255 | 60 | 0 |
| 2 | 33 | 17 | 255 | 80 |
The cells students fill in are left blank in this preview.
Read the grey ramp with Pillow
docker exec jupyter python3 -c "from PIL import Image; p = Image.open('images/gradient.png'); print(p.size, p.mode, p.getpixel((0, 0)), p.getpixel((100, 5)), p.getpixel((255, 0)))"Shown here, not run. Create a free account to press Run and type it into your own lab’s terminal.
Pillow reports a picture's size and mode
Pillow is the Python library that opens image files. The Run above opens gradient.png, a ramp from black on the left to white on the right.
It prints the size, which is (256, 64): 256 wide and 64 high. The mode is L, one grey number per pixel. Then three pixels: 0, 100 and 255. Pixel 100 along the ramp has grey level 100.
A colour pixel holds one number per channel
A colour picture needs more than one number. In mode RGB a pixel holds three channels: how much red, green and blue light. Each runs from 0 to 255.
Mixing them makes every colour. Red 200, green 40 and blue 40 is a strong red. All three at 255 is white, and all three at 0 is black.
Worked example: three common modes
| Mode | Numbers per pixel | Meaning | Example pixel |
|---|---|---|---|
| L | 1 | one grey level | 88 |
| RGB | 3 | red, green, blue | (200, 40, 40) |
| RGBA | 4 | red, green, blue and see-through | (200, 40, 40, 255) |
The cells students fill in are left blank in this preview.
Turn a colour picture into a NumPy array
docker exec jupyter python3 -c "import numpy as np; from PIL import Image; a = np.array(Image.open('images/shapes.png')); print(a.shape, a.dtype, a[30, 30].tolist(), a[0, 0].tolist())"Shown here, not run. Create a free account to press Run and type it into your own lab’s terminal.
NumPy holds the whole picture in one array
NumPy stores a picture as one array. The Run above prints (200, 300, 3): 200 rows, 300 columns and 3 channels, so a colour picture is a stack of three grids.
The type uint8 means each number is a whole number from 0 to 255. At row 30, column 30 the pixel is [200, 40, 40], a red shape. At row 0, column 0 it is [255, 255, 255], the white background.
A common mistake
Warning
Many people think the first number of a position is always the row. Pillow asks for the column first: getpixel((x, y)), where x counts across. NumPy asks for the row first: a[y, x]. Whenever you move a position between the two, swap the order.
Find the brightest pixel with loops
grid = [
[12, 40, 200, 35],
[90, 255, 60, 0],
[33, 17, 255, 80],
]
best = grid[0][0]
best_row = 0
best_column = 0
for row in range(len(grid)):
for column in range(len(grid[row])):
if grid[row][column] > best:
best = grid[row][column]
best_row = row
best_column = column
print("brightest:", best, "at row", best_row, "column", best_column)
# Change one number to 256 or more and press Run.
# Real pixels stop at 255, so what would a program do with 300?
Going further
Open start.ipynb in JupyterLab and add a cell that loads road.png with np.array. Print its shape and its smallest and largest numbers.
Is the picture grey or colour? Is there any pixel that is pure black or pure white?