Optimization

concepts · 10 notes linked

Related: Convex Optimization · Gradient Descent · Google Deepmind · Prompt Engineering · Large Language Models · D Wave · Quantum Computing · Quantum Annealing

Notes

• Beyond automatic differentiation — Google's AutoBound computes polynomial function bounds enabling hyperparameter-free optimizers
• Gradient descent for linear regression¶ — GD, SGD, and minibatch SGD implementations for least-squares linear regression
• How long should a lockdown-relaxation cycle last? — Mathematical optimization of lockdown-relaxation cycle triggers and lengths
• Large Language Models as Optimizers — OPRO uses LLMs as gradient-free optimizers via natural language prompts
• Machine Learning Research Blog — Francis Bach's blog on optimization theory and ML fundamentals
• NOX Metals — AI-driven aluminum plate cut to size (Detroit) — AI-optimized aluminum cutting and supply, Detroit
• Other Versions — Practical Python performance optimization techniques and profiling tools
• Quantum effects of D-Wave's hardware boost its performance — D-Wave quantum annealer demonstrating hardware advantage over classical simulations
• Risky Giant Steps Can Solve Optimization Problems Faster | Quanta Magazine — Oversized gradient descent steps with cyclical patterns converge nearly 3x faster
• Why does Deep Learning work? — Deep learning energy landscape as spin funnel not spin glass