Embodied Web Agents: Bridging Physical-Digital Realms for Integrated Agent Intelligence

Ying Nian Wu (University of California, Los Angeles) · Kai-Wei Chang (UCLA) · Rui Sun (University of California, Los Angeles) · Yining Hong (University of California, Los Angeles) · Bingxuan Li (University of Illinois Urbana-Champaign) · Xingcheng Yao (University of California, Los Angeles) · Maxine Wu (UCLA Computer Science Department, University of California, Los Angeles) · Alexander Chien (University of California, Los Angeles) · Da Yin (Meta FAIR) · Zhecan Wang (UCLA | Columbia University)
coordinated reasoningcross-domain intelligenceembodied cognitionembodied perceptionembodied web agentsembodied web agents benchmarkfunctional web interfaceshuman capabilitiesintegrated physical and digital intelligenceperformance gapsrealistic 3d environmentssimulation platformsystematic assessmenttask environmentsweb-scale reasoning

AI agents today are mostly siloed — they either retrieve and reason over vast amount of digital information and knowledge obtained online; or interact with the physical world through embodied perception, planning and action — but rarely both. This separation limits their ability to solve tasks that require integrated physical and digital intelligence, such as cooking from online recipes, navigating with dynamic map data, or interpreting real-world landmarks using web knowledge. We introduce \textsc{Embodied Web Agents}, a novel paradigm for AI agents that fluidly bridge embodiment and web-scale reasoning. To operationalize this concept, we first develop the \textsc{Embodied Web Agents} task environments, a unified simulation platform that integrates realistic 3D indoor and outdoor environments with functional web interfaces. Building upon this platform, we construct and release the \textsc{Embodied Web Agents} Benchmark, which encompasses a diverse suite of tasks including cooking, navigation, shopping, tourism, and geolocation — all requiring coordinated reasoning across physical and digital realms for systematic assessment of cross-domain intelligence. Experimental results reveal significant performance gaps between state-of-the-art AI systems and human capabilities, establishing both challenges and opportunities at the intersection of embodied cognition and web-scale knowledge access.