How many coins are in the picture?
A person counts the coins in a photo in a second. A computer is given only a long list of numbers, and has to be told how to turn them into "nine coins". Doing that, a step at a time, is what computer vision is.
By the end of this lesson
Note
I can read the pixels of a picture and use a threshold to count the objects in it.
Key words
- Pixel: one dot of a picture, stored as numbers.
- Mode: how the numbers are laid out, for example RGB (red, green, blue) or L (one grey level).
- Grey level: a number from 0 (black) to 255 (white).
- Threshold: a grey level used to split pixels into dark and bright.
- Filter: a rule that changes every pixel using the pixels around it, such as a blur.
- Label: a number given to each separate object found in a picture.
Starter: what you already know
Note
Answer in your head. 1. What are the three colours mixed to make every colour on a screen? 2. Which is brighter, 40 or 200, on a scale from 0 to 255? 3. What does > do in a comparison? Answers: red, green and blue; 200; it asks whether the left side is bigger than the right.
Before you start
Note
Press Start the lab on the bar at the top of this lesson and wait until it says it is running. There is one container, jupyter, and the terminal reaches its Python with docker exec jupyter, which runs one command inside it. The JupyterLab link opens its web page, port 8888, where you can open the pictures and see them. Each Run button below types its command into your lab's terminal, one after another in the same session. To watch them, or to type your own, press Open terminal on the bar.
A picture is a grid of numbers
ls images lists the five pictures the lab drew. The first script opens coins.png and prints what is in it. The answer says the picture is 320 pixels wide and 240 high, in mode RGB. The pixel at 10, 10 is (35, 35, 35), almost black: the background. The pixel at 50, 50 is (197, 157, 62), a gold colour: part of a coin. The mean grey level of the whole picture is 57.6, dark, because most of it is background.
docker exec jupyter ls imagesShown here, not run. Create a free account to press Run and type it into your own lab’s terminal.
docker exec jupyter python3 steps/look.pyShown here, not run. Create a free account to press Run and type it into your own lab’s terminal.
Grey keeps the brightness and drops the colour
convert("L") makes every pixel one number. The coin pixel (197, 157, 62) becomes 158. The script then counts the pixels in four bands of grey. 63231 are dark, from 0 to 63, none are in the middle and 13566 are from 128 to 191: two clear groups, the background and the coins. It also saves coins_grey.png, which you can open in JupyterLab.
docker exec jupyter python3 steps/grey.pyShown here, not run. Create a free account to press Run and type it into your own lab’s terminal.
A threshold splits dark from bright
A threshold turns a grey picture into a yes or no for every pixel: is it brighter than this level? The script tries three levels on 76800 pixels. A level of 30 keeps 58806, nearly the whole picture, because the background is brighter than 30 in many places. A level of 100 keeps 13569, which are the coins. A level of 160 keeps 10062 and cuts into the coins. It saves the level-100 result as coins_mask.png: white where there is a coin.
docker exec jupyter python3 steps/threshold.pyShown here, not run. Create a free account to press Run and type it into your own lab’s terminal.
Check
Tip
Which of the three levels separated the coins from the background, and why did 30 fail? Answer: 100, because it sits in the empty gap between the dark background and the bright coins; 30 is inside the background's own noise, so it keeps most of the background too.
A blur calms noise
The page scan, scan.png, has speckles, like a dirty photocopy. A blur replaces each pixel with an average of its neighbours. The script measures how much an empty patch of the page varies, with no writing in it. Before the blur it varies by 25.3 grey levels. After, only 4.9. The noise is mostly gone, and the writing will look softer too. It saves scan_blurred.png so you can compare.
docker exec jupyter python3 steps/blur.pyShown here, not run. Create a free account to press Run and type it into your own lab’s terminal.
Counting objects
To count the coins, make the mask with the level that worked, then ask scikit-image to label it: every group of touching white pixels gets its own number. The answer is 9 objects, the nine coins the lab drew. The areas come in three sizes, three of 657 pixels, three of 1313 and three of 2553: small, medium and large coins.
docker exec jupyter python3 steps/count.pyShown here, not run. Create a free account to press Run and type it into your own lab’s terminal.
A common mistake
Warning
Many people think the computer finds the right threshold by itself. It does not: you chose 100 because you looked at the two groups of grey levels. On a picture with uneven light, such as scan.png, one level that works at the left may fail at the right. Always check the mask by eye before you trust a count.
The same work in a notebook
Open JupyterLab from the bar and double-click start.ipynb. It does what the scripts did, one cell at a time. Click a cell and press Shift and Enter to run it. The picture appears under the cell. The last cell is where your own question goes: open road.png or scan.png and ask it something.
Exit check
Tip
Answer without looking back. 1. What range of numbers does a grey level use? 2. What does a threshold decide for each pixel? 3. Why did the blur lower the spread of the empty patch? 4. How many coins did the labelling find, and how did you know it was right? Answers: 0 to 255; whether it is brighter than the level; each pixel became an average of its neighbours, so the odd bright and dark dots were pulled towards the middle; 9, the same as the number drawn.
For teachers
Tip
This lesson is a small example of a lesson worked in a lab: the steps and the Run buttons are the lesson, and the lab is each student's own copy. The lab is one service, JupyterLab with Pillow, NumPy and scikit-image, and nothing here marks itself: the student hands in a notebook of their own over one of the pictures, downloaded from JupyterLab, and a sentence on what it showed. Key learning points: 1. A picture is a grid of numbers. 2. A threshold splits pixels in two. 3. A filter changes a pixel using its neighbours. Common mistake: trusting a threshold or a count without looking at the mask. Equipment: the lab and a browser. Tip: ask students to guess each number before they press Run. The pictures are drawn from a fixed seed, so every copy has the same numbers, and the lab has no neural networks and no OpenCV. Copy it into your own module and change it, or build one from scratch with the Lab menu in the lesson builder.