Cognitive Predictive Processing: A Human-inspired Framework for Adaptive Exploration in Open-World Reinforcement Learning

Qing Li (Beijing Institute of Technology) · Ziyu Li (Beijing Institute of Technology) · Xiuxing Li (Beijing Institute of Technology) · Xia Wu (Beijing Institute of Technology) · boheng liu (Beijing Institute of Technology) · Chenghua Duan (Beijing Institute of Technology) · YuTian Liu (Beijing Information Science and Technology University) · Zhuo Wang (Beijing Institute of Technology)

adaptive uncertainty regulationcognitive predictive processingdual-memory integration systemdual-modal memoryenvironmental predictionsimagination horizonsls-imagineneural-inspired modelsopen-world reinforcement learningphase-adaptive cognitive controllerresource collection tasksselective memory utilizationstrategic adaptationtask decompositionuncertainty-modulated prediction regulator

Open-world reinforcement learning challenges agents to develop intelligent behavior in vast exploration spaces. Recent approaches like LS-Imagine have advanced the field by extending imagination horizons through jumpy state transitions, yet remain limited by fixed exploration mechanisms and static jump thresholds that cannot adapt across changing task phases, resulting in inefficient exploration and lower completion rates. Humans demonstrate remarkable capabilities in open-world decision-making through a chain-like process of task decomposition, selective memory utilization, and adaptive uncertainty regulation. Inspired by human decision-making processes, we present Cognitive Predictive Processing (CPP), a novel framework that integrates three neurologically-inspired systems: a phase-adaptive cognitive controller that dynamically decomposes tasks into exploration, approach, and completion phases with adaptive parameters; a dual-memory integration system implementing dual-modal memory that balances immediate context with selective long-term storage; and an uncertainty-modulated prediction regulator that continuously updates environmental predictions to modulate exploration behavior. Comprehensive experiments in MineDojo demonstrate that these human-inspired decision-making strategies enhance performance over recent techniques, with success rates improving by an average of 4.6\% across resource collection tasks while reducing task completion steps by an average of 7.1\%. Our approach bridges cognitive neuroscience and reinforcement learning, excelling in complex scenarios that require sustained exploration and strategic adaptation while demonstrating how neural-inspired models can solve key challenges in open-world AI systems.