REINFORCEMENT LEARNING AND OPTIMAL CONTROL

Abstraction: Bertsekas textbook unifying RL, dynamic programming, and optimal control

Key points:

• ~500-page textbook by Dimitri Bertsekas (MIT/ASU), 2nd edition covers 2019-2025 course material; downloadable free for instructional use.
• Organized in two parts: a foundational platform (Chapter 1 on approximate DP/RL landscape) and in-depth coverage (Chapters 2-3 on rollout, value approximation, policy gradient).
• Newton's method interpretation of Bellman equation approximation is a central conceptual framework — on-line play amounts to a Newton step on top of off-line training.
• AlphaZero and TD-Gammon are analyzed as examples of off-line training combined with on-line policy improvement via lookahead/rollout.
• 2nd edition adds connections to transformers, LLMs, and HMM inference; MPC application to computer chess; and expanded policy gradient material.
• Companion books include "Rollout, Policy Iteration, and Distributed Reinforcement Learning" and "Lessons from AlphaZero" (Athena Scientific).

Connections: Dimitri Bertsekas · Mit · Arizona State University · Reinforcement Learning · Dynamic Programming · Optimal Control · Model Predictive Control

Source: http://web.mit.edu/dimitrib/www/RLbook.html