Automatic Differentiation

concepts · 8 notes linked

Related: Pytorch · Deep Learning · Gradient Descent · Google · Backpropagation · Neural Networks · Google Colab · GPU Computing

Notes

• Automatic Differentiation: The most criminally underused tool in the potential machine learning toolbox? — Reverse-mode autodiff computes gradients at same cost as the forward function
• Beyond automatic differentiation — Google's AutoBound computes polynomial function bounds enabling hyperparameter-free optimizers
• Calculus on Computational Graphs: Backpropagation — Backpropagation as reverse-mode differentiation on computational graphs
• Getting Started With Pytorch In Google Collab With Free GPU — Beginner tutorial on PyTorch tensors and autograd using free Colab GPU
• Gradient descent for linear regression using PyTorch — PyTorch tutorial implementing minibatch SGD for least-squares linear regression
• JAX: Is This Google's NumPy killer? — JAX library combining NumPy API with autodiff and GPU/TPU acceleration
• PyTorch Explained: From Automatic Differentiation to Training Custom Neural Networks | Towards Data Science — PyTorch core abstractions from tensors and autograd to transformer encoder
• Swift: Google's bet on differentiable programming — Google's Swift for TensorFlow project integrating native differentiable programming