Deep Learning
Neurons, networks, activations, training, backpropagation, optimisers, RNNs, embeddings, autoencoders and GANs.
14 animated lectures · 40 minutes
▶ Start with lecture 1Artificial Neurons and the Perceptron
Inside a single artificial neuron: weighted inputs, a sum, a bias and an activation — plus the perceptron that learns from its mistakes.
Neural Networks: The Forward Pass
Stack neurons into layers and connect them. Watch signals flow from input to output and become class probabilities.
Activation Functions: Sigmoid, Tanh and ReLU
Why networks need non-linearity, and how sigmoid, tanh, ReLU, Leaky ReLU and GELU differ — drawn live.
Loss Functions and the Training Loop
How a network measures its mistakes and improves: mini-batches, forward pass, loss, backward pass and update — repeated thousands of times.
Backpropagation, Step by Step
The chain rule in action: compute values forward, then pass gradients backward through a tiny computational graph — with real numbers.
Optimisers: SGD, Momentum and Adam
Compare optimisers racing across the same loss surface: noisy SGD, plain gradient descent, momentum and Adam.
Vanishing Gradients and Residual Connections
Why very deep networks used to be untrainable — gradients shrinking layer by layer — and how skip connections fixed it.
Dropout and Regularisation
Big networks memorise. Dropout, weight decay, early stopping and augmentation keep them honest — see dropout flicker neurons on and off.
Recurrent Networks and LSTMs
Networks with memory: an RNN reads a sentence word by word, carrying a hidden state; an LSTM adds gates to remember for longer.
Embeddings: Meaning as Vectors
Words become points in space where similar meanings sit close together — and directions carry meaning, as in king − man + woman ≈ queen.
Autoencoders: Compress and Reconstruct
Squeeze an image through a tiny bottleneck and rebuild it. Autoencoders learn compact representations — and can clean up noisy inputs.
Generative Adversarial Networks (GANs)
A forger and a detective train together. Watch the generator’s fake distribution move until it matches real data.
Training Deep Neural Networks: A Practical Deep Dive
The full recipe for training a network well: forward pass, softmax and cross-entropy, backpropagation, initialisation, vanishing gradients and skip connections, normalisation, optimisers, regularisation and a debugging checklist.
From RNNs to Transformers: Sequence Models in Depth
How neural networks learned to handle sequences: recurrent networks and backpropagation through time, vanishing gradients, LSTM and GRU gates, encoder–decoder translation, attention, and why transformers replaced recurrence.