activation functions
Activation functions are mathematical functions applied to nodes in a neural network that determine whether a neuron should be activated or not, influencing the network's capacity to learn complex patterns.
- Block Coordinate Descent for Neural Networks Provably Finds Global Minima
- Curvature Tuning: Provable Training-free Model Steering From a Single Parameter
- Generalization Bounds for Rank-sparse Neural Networks
- Gompertz Linear Units: Leveraging Asymmetry for Enhanced Learning Dynamics
- Jacobian-Based Interpretation of Nonlinear Neural Encoding Model
- Measure gradients, not activations! Enhancing neuronal activity in deep reinforcement learning
- Optimal Minimum Width for the Universal Approximation of Continuously Differentiable Functions by Deep Narrow MLPs
- SAD Neural Networks: Divergent Gradient Flows and Asymptotic Optimality via o-minimal Structures