Explain the Architecture and Mathematics of Artificial Neural Networks and Backpropagation

Learning Objectives

At the end of this theory note, trainees must be able to:

1. Describe the architecture of an artificial neural network, including neurons, layers, and activation functions.
2. Explain and manually compute forward propagation for a small network.
3. Explain the backpropagation algorithm and compute gradients using the chain rule.
4. Explain loss functions and gradient descent optimisation.
5. Describe how weights, biases, and learning rate affect network behaviour and convergence.