Machine LearningBeginner1:21 video5 chapters

k-Nearest Neighbours: Learning by Similarity — lecture notes

Classify a new point by asking its closest neighbours to vote. Simple, intuitive and a great first classifier.

0:001. Introduction

Introduction — k-Nearest Neighbours: Learning by Similarity

If you want to guess whether you will like a film, you might ask friends with similar taste. The k nearest neighbours algorithm works exactly like that.

0:122. How k-NN classifies

How k-NN classifies — k-Nearest Neighbours: Learning by Similarity

Here are labelled pink and blue points. A new point arrives with no label. We measure its distance to every example, find the five nearest, and let them vote. Three are blue and two are pink, so the new point is classified as blue.

0:313. Changing k

Changing k — k-Nearest Neighbours: Learning by Similarity

With k equals three, the vote is two blue to one pink, still blue. Small k is sensitive to noisy points. Large k is smoother but can blur real boundaries. We choose k by testing on validation data, and an odd k avoids ties between two classes.

0:514. Practical tips

Practical tips — k-Nearest Neighbours: Learning by Similarity

k nearest neighbours has no training step: it just stores the data. That makes prediction slow on big datasets. Always scale your features, or large numbers like income will dominate the distance. And it works for regression too, by averaging neighbours.

1:085. Recap

Recap — k-Nearest Neighbours: Learning by Similarity

To recap. Find the k closest examples and take a vote. Choose k carefully, normalise your features, and remember it is simple and explainable but slow on large data.

Key takeaways

k-NN predicts by majority vote among the k most similar examples.
In the animation, k = 5 gives 3 blue vs 2 pink → blue.
Feature scaling matters because distances drive everything.
There is no training phase, but predictions can be slow.

Check yourself

With k = 5, three neighbours are blue and two are pink. The prediction is…
Show answer
Blue — The majority of the five neighbours are blue.
Why normalise features for k-NN?
Show answer
So one large-scale feature does not dominate the distance — Distance treats all features equally only if their scales are comparable.
What is a downside of k-NN on very large datasets?
Show answer
Predictions are slow because it compares with many points — Each prediction needs distances to the stored examples.

Go deeper

k-Nearest Neighbours: Learning by Similarity · The AI Lecture Hall
Feature Scaling and Normalisation: Standardisation, Min–Max and Robust Scaling · The AI Lecture Hall