Labelling a point by its neighbours (kNN)

Each morning the shop decides whether to call in a second worker. It does not need an exact number of hires. It needs one word: busy or quiet.

A good guess is to look at the days most like today and see what they were. That is the whole idea of this lesson.

Each day is a point placed by its features, here temperature and rain. The distance between two days is the straight-line gap between their points.

To label a new day, find the k training days closest to it. Each votes with its own label, and the most votes wins. With two labels, an odd k means there is never a tie.

# A new day: 11 degrees and 3 mm of rain.
new = [11.0, 3.0]
days = [[10.0, 5.0, "quiet"], [12.0, 0.0, "busy"], [15.0, 0.0, "busy"],
        [6.0, 0.0, "quiet"], [8.0, 9.0, "quiet"]]
for day in days:
    gap = ((day[0] - new[0]) ** 2 + (day[1] - new[1]) ** 2) ** 0.5
    print(day[2], round(gap, 2))
# The three smallest gaps vote. Which label wins?
# Change new to [7.0, 1.0] and vote again.

import pandas as pd

hires = pd.read_csv("hires.csv")
day = hires.loc[0, ["temp_c", "rain_mm"]]
# The gap from 1 January to every day of the year.
gap = ((hires[["temp_c", "rain_mm"]] - day) ** 2).sum(axis=1) ** 0.5
hires.assign(gap=gap).sort_values("gap").head(6)

knn.py labels a day busy at 300 hires or more. It trains on three quarters of the year and guesses the rest with k = 5.

Accuracy is the share of guesses that are right. On its own it says little. So the script also prints the score of a lazy rule that always says quiet. A model is only useful if it beats that.

docker exec jupyter python3 steps/knn.py

Two models with the same accuracy can make different mistakes. A confusion matrix counts each pair of truth and guess.

In knn.py the rows are the truth and the columns the guess, busy first. Calling a busy day quiet leaves one worker alone in a queue. Calling a quiet day busy pays a worker to wait. The shop cares which.

With k = 1 the model copies the single nearest day, noise and all. It looks perfect on days it learnt from and worse on new ones.

A large k smooths the noise, but too large and every guess drifts toward the common label. knnk.py tries six values of k and scores each on both sets of days.

docker exec jupyter python3 steps/knnk.py

Add a weekend feature of 1 and 0 to the kNN in your notebook. Score it on the same split as knn.py. Then scale every feature with StandardScaler first and score it again.

Explain why scaling can matter for kNN. Hint: a weekend can only move a point by 1, but the temperature moves it by up to 26.