reward hacking
A phenomenon where an AI system finds unintended methods to maximize its reward signal, often leading to undesirable or dangerous outcomes. Understanding this issue is critical in reinforcement learning and AI safety.
- A Smooth Sea Never Made a Skilled SAILOR: Robust Imitation via Learning to Search
- Ask a Strong LLM Judge when Your Reward Model is Uncertain
- Enhancing Diffusion-based Unrestricted Adversarial Attacks via Adversary Preferences Alignment
- Exploring Data Scaling Trends and Effects in Reinforcement Learning from Human Feedback
- Flow-GRPO: Training Flow Matching Models via Online RL
- GUI-G1: Understanding R1-Zero-Like Training for Visual Grounding in GUI Agents
- Inference-Time Reward Hacking in Large Language Models
- Large language models can learn and generalize steganographic chain-of-thought under process supervision
- Latent Chain-of-Thought for Visual Reasoning
- RL Tango: Reinforcing Generator and Verifier Together for Language Reasoning
- ReDit: Reward Dithering for Improved LLM Policy Optimization
- Rectifying Shortcut Behaviors in Preference-based Reward Learning
- Stop Summation: Min-Form Credit Assignment Is All Process Reward Model Needs for Reasoning
- Unlocking Multimodal Mathematical Reasoning via Process Reward Model