neural-networks-and-deep-learning/src/network.py at master · mnielsen/neural-networks-and-deep-learning

neural-networksbackpropagationpythonstochastic-gradient-descentdeep-learning

Abstraction: Michael Nielsen's minimal readable feedforward neural network with SGD and backpropagation in Python

Key points:

Implements a fully connected feedforward network in ~141 lines of Python (numpy only); prioritizes readability over performance
Network class takes a sizes list (e.g. [784, 30, 10]); weights and biases initialized as random Gaussians (mean 0, variance 1)
SGD method: mini-batch stochastic gradient descent with configurable epochs, batch size, and learning rate eta
backprop method: computes gradient of cost C_x via forward pass storing activations and z-vectors, then backward pass using sigmoid_prime
update_mini_batch: applies averaged gradients across the mini-batch to update weights and biases
Companion code for Michael Nielsen's free online book "Neural Networks and Deep Learning"

Connections: Michael Nielsen · Github · Neural Networks · Backpropagation · Stochastic Gradient Descent