What Do Latent Action Models Actually Learn?

Jiang Bian (Microsoft Research) · Chuheng Zhang (Microsoft) · Tim Pearce (Microsoft Research) · Pushi Zhang (Microsoft) · Kaixin Wang (Microsoft Research) · Xiaoyu Chen (Tsinghua University, Tsinghua University) · Wei Shen (Shanghai Jiao Tong University) · Li Zhao (Microsoft Research)
action-relevant changesactions structureauxiliary action-predictioncontrollable changesdata augmentationdata cleaningdata-generating policyexogenous noiselatent action modelslinear modelnoise influencenumerical simulationobservations structureprincipal component analysisunlabeled videos

Latent action models (LAMs) aim to learn action-relevant changes from unlabeled videos by compressing changes between frames as latents. However, differences between video frames can be caused by \textit{controllable changes} as well as exogenous noise, leading to an important concern -- do latents capture the changes caused by actions or irrelevant noise? This paper studies this issue analytically, presenting a linear model that encapsulates the essence of LAM learning, while being tractable. This provides several insights, including connections between LAM and principal component analysis (PCA), desiderata of the data-generating policy, and justification of strategies to encourage learning controllable changes using data augmentation, data cleaning, and auxiliary action-prediction. We also provide illustrative results based on numerical simulation, shedding light on the specific structure of observations, actions, and noise in data that influence LAM learning.