Linear Classifiers: Logistic Regression and the Perceptron — lecture notes
Draw a line that separates two classes. Watch logistic regression learn a probabilistic boundary and the perceptron nudge its line towards mistakes.
0:001. Introduction
Classification means predicting a category: spam or not, pass or fail. The simplest classifiers separate the classes with a straight line.
0:092. Logistic regression
Logistic regression learns a boundary and a probability for every point. The background shading shows how confident it is. Watch the boundary move as training runs. Accuracy climbs from 59 percent to 93 percent, with a few overlapping points still misclassified.
0:273. The sigmoid
The model computes a linear score, like linear regression, and passes it through the sigmoid, which squeezes any number into a probability between zero and one. Above one half, we predict the positive class.
0:424. The perceptron
The perceptron, from 1958, is even simpler. It looks at a training point. If the point is on the wrong side, it nudges the line towards it. If it is correct, it does nothing. Repeat, and the line settles into a good position.
1:005. Comparison
Logistic regression gives probabilities and trains smoothly. The perceptron only outputs a class and only learns from mistakes. But it is historically important: it is the ancestor of every neural network.
1:146. Recap
To recap. Linear classifiers draw a straight boundary. Logistic regression outputs probabilities. The perceptron learns from mistakes. And neither can handle problems that need a curved boundary, which is where more powerful models come in.
Key takeaways
- Linear classifiers separate classes with a straight line (or plane).
- Logistic regression applies the sigmoid to a linear score to get a probability.
- The perceptron updates only when it misclassifies a point.
- In the animation both reach 41 of 44 points correct (93%).
Check yourself
- What does the sigmoid function do?
Show answer
Squeezes any number into the range 0–1 — Sigmoid outputs can be read as probabilities.
- When does the perceptron change its line?
Show answer
Only when a point is misclassified — It learns only from mistakes.
- What is a limitation of both models?
Show answer
They can only draw straight boundaries — Linear models cannot represent curved boundaries without extra features.
Go deeper
- Logistic Regression: Probabilistic Classification Done Right · The AI Lecture Hall
- The Perceptron: The First Learning Machine · The AI Lecture Hall
- Softmax Regression and Multiclass Classification Strategies · The AI Lecture Hall
© 2026 Janin A Apurba, CSE, AUST · Advanced ICT Officer, CNRS-UNHCR. All rights reserved. Notes for the animated lecture at https://ai-in-motion.vercel.app/watch/logistic-regression-and-the-perceptron.html