DeepMind's Collect & Infer: A Fresh Look at Data-Efficient Reinforcement Learning | Synced

Abstraction: DeepMind Collect and Infer paradigm separates RL data collection from inference

Key points:

• DeepMind proposes "Collect & Infer" (C&I): a conceptual framework that separates RL into two distinct processes — collecting data into a transition memory and inferring knowledge from that data
• Optimal inference goal: given a fixed data batch, find the learning setup that yields the maximally performing policy
• Optimal collection goal: given an inference process, find the minimal data set needed for a maximally performing policy
• The framework spans the spectrum from pure offline (batch) RL to online RL and supports multi-task, transfer, meta-learning, and life-long learning scenarios
• Unlike exploration–exploitation trade-off frameworks (e.g., Bayesian methods), C&I separates a learning phase from a deployment phase for evaluation
• C&I can integrate skills, model-based methods, and unsupervised exploration without biasing the final task solution

Connections: Deepmind · Reinforcement Learning · Offline RL · Data Efficiency

Source: https://syncedreview.com/2021/08/31/deepmind-podracer-tpu-based-rl-frameworks-deliver-exceptional-performance-at-low-cost-94/