Gradient descent for linear regression using PyTorch

Abstraction: PyTorch tutorial implementing minibatch SGD for least-squares linear regression

Key points:

• PyTorch Tensor tracks computation history for automatic gradient computation (requires_grad=True); backward() computes gradients for all contributing tensors
• optimizer.zero_grad() → loss.backward() → optimizer.step() is the canonical PyTorch training loop pattern
• Two implementations shown: manual weight tensor approach, and idiomatic torch.nn.Module + torch.nn.Linear approach — identical results
• torch.optim.SGD used; Adam noted as alternative but not an improvement for this simple objective
• PyTorch uses separate operations for vector-vector (dot), matrix-vector (mv), and matrix-matrix (mm) products, unlike NumPy's unified dot
• Chalmers University lecture demo using synthetic 1000-sample dataset with slope -0.9

Connections: Pytorch · Gradient Descent · Linear Regression · Automatic Differentiation

Source: http://www.cse.chalmers.se/~richajo/dit866/backup_2019/lectures/l5/PyTorch%20linear%20regression%20demo.html