Reinforcement Learning algorithms — an intuitive overview

Abstraction: Survey of model-free and model-based RL algorithm families

Key points:

• Model-free RL splits into policy optimization (PG, A3C, TRPO, PPO) and value-iteration methods (DQN, C51, QR-DQN, HER); model-free is statistically less efficient than model-based
• PPO limits policy updates each training step to prevent performance collapse; OpenAI used it to defeat top Dota 2 players
• A3C (Asynchronous Advantage Actor-Critic), published by DeepMind, trains multiple parallel agents in independent environment copies and aggregates into a master agent
• DQN uses neural networks to approximate Q-values when a full Q-table is impractical for large state spaces
• Model-based RL learns or uses a known environment model (AlphaGo Zero used given model); world models let agents learn from "dreams" via variational autoencoders
• Hybrid methods (DDPG, SAC, TD3) combine policy-gradient and Q-learning strengths

Connections: Openai · Deepmind · Reinforcement Learning · Policy Gradient · Q Learning · Model Based RL

Source: https://smartlabai.medium.com/reinforcement-learning-algorithms-an-intuitive-overview-904e2dff5bbc