Autoregressive Motion Generation with Gaussian Mixture-Guided Latent Sampling

Hefei Ling (Huazhong University of Science and Technology) · Zongyi Li (Fudan University) · Linnan Tu (Huazhong University of Science and Technology) · Lingwei Meng (The Chinese University of Hong Kong) · Shijuan Huang (Huazhong University of Science and Technology)

autoregressive modelcausal transformerconditional probability distributioncontinuous representationsdiffusion models gaussian mixture modelgenerative transformersinference processmotion errorsmotion synthesisresidual vector quantizationsampling process state-of-the-art modelstraining processvae latent space

Existing efforts in motion synthesis typically utilize either generative transformers with discrete representations or diffusion models with continuous representations. However, the discretization process in generative transformers can introduce motion errors, while the sampling process in diffusion models tends to be slow. In this paper, we propose a novel text-to-motion synthesis method GMMotion that combines a continuous motion representation with an autoregressive model, using the Gaussian mixture model (GMM) to represent the conditional probability distribution. Unlike autoregressive approaches relying on residual vector quantization, our model employs continuous motion representations derived from the VAE's latent space. This choice streamlines both the training and the inference processes. Specifically, we utilize a causal transformer to learn the distributions of continuous motion representations, which are modeled with a learnable Gaussian mixture model. Extensive experiments demonstrate that our model surpasses existing state-of-the-art models in the motion synthesis task.