A Simple Neural Network - With Numpy in Python

Abstraction: Implementing neural network forward pass and backpropagation from scratch with NumPy

Key points:

• Builds a multi-layer neural network using only NumPy — no deep learning frameworks
• Sigmoid activation function implemented with a Derivative=True flag for reuse in backprop
• Weights initialized from a normal distribution with scale 0.1, shaped as (l2, l1+1) to include bias
• Forward pass uses np.vstack with a row of ones to append bias node for each input example
• Backpropagation iterates layers in reverse with reversed(), computing deltas and pulling them back through the weight matrix
• Toy XOR example converges to error ~0.000291 after 100,000 iterations with learning rate 0.2

Connections: Neural Networks · Backpropagation · Gradient Descent · Activation Functions

Source: https://mlnotebook.github.io/post/nn-in-python/