Towards Identifiability of Hierarchical Temporal Causal Representation Learning

Kun Zhang (CMU & MBZUAI) · Zijian Li (Mohamed bin Zayed University of Artificial Intelligence) · Minghao Fu (University of California, San Diego) · Guangyi Chen (MBZUAI&CMU) · Ruichu Cai (Guangdong University of Technology) · Junxian Huang (Guangdong University of Technology) · Yifan Shen (Mohamed bin Zayed University of Artificial Intelligence) · Yuewen Sun (Mohamed bin Zayed University of Artificial Intelligence)

causally hierarchical latent dynamicconditionally independent observationscontextual encoderempirical evaluationsflow-based hierarchical prior networkshierarchical latent dynamicsindependent noise conditionjoint distributionlatent variable identificationmulti-layer latent variablesnatural sparsitytemporal causal representation learningtemporal contextual variablestime series generative modelvariational inference

Modeling hierarchical latent dynamics behind time series data is critical for capturing temporal dependencies across multiple levels of abstraction in real-world tasks. However, existing temporal causal representation learning methods fail to capture such dynamics, as they fail to recover the joint distribution of hierarchical latent variables from \textit{single-timestep observed variables}. Interestingly, we find that the joint distribution of hierarchical latent variables can be uniquely determined using three conditionally independent observations. Building on this insight, we propose a Causally Hierarchical Latent Dynamic (CHiLD) identification framework. Our approach first employs temporal contextual observed variables to identify the joint distribution of multi-layer latent variables. Sequentially, we exploit the natural sparsity of the hierarchical structure among latent variables to identify latent variables within each layer. Guided by the theoretical results, we develop a time series generative model grounded in variational inference. This model incorporates a contextual encoder to reconstruct multi-layer latent variables and normalize flow-based hierarchical prior networks to impose the independent noise condition of hierarchical latent dynamics. Empirical evaluations on both synthetic and real-world datasets validate our theoretical claims and demonstrate the effectiveness of CHiLD in modeling hierarchical latent dynamics.