Backpropagation

concepts · 7 notes linked

Related: Neural Networks · Gradient Descent · Deep Learning · Michael Nielsen · Geoffrey Hinton · Stochastic Gradient Descent · Automatic Differentiation · Activation Functions

Notes

• A Simple Neural Network - With Numpy in Python — Implementing neural network forward pass and backpropagation from scratch with NumPy
• Automatic Differentiation: The most criminally underused tool in the potential machine learning toolbox? — Reverse-mode autodiff computes gradients at same cost as the forward function
• Calculus on Computational Graphs: Backpropagation — Backpropagation as reverse-mode differentiation on computational graphs
• Facebook's Quest to Build an Artificial Brain Depends on This Guy — Yann LeCun pioneers convolutional neural networks and leads Facebook AI lab
• Neural Networks and Deep Learning — Chapter 2 derivation of the four backpropagation equations
• The Matrix Calculus You Need For Deep Learning — Matrix calculus tutorial covering gradients and Jacobians for neural network training
• neural-networks-and-deep-learning/src/network.py at master · mnielsen/neural-networks-and-deep-learning — Michael Nielsen's minimal readable feedforward neural network with SGD and backpropagation in Python