Boosting Skeleton-based Zero-Shot Action Recognition with Training-Free Test-Time Adaptation

Jun Liu (Xi&#x27;an Jiaotong University) · Mohammed Bennamoun (University of Western Australia) · Qiuhong Ke (Monash University) · Jingmin Zhu (Monash University) · Anqi Zhu (Monash University) · Hossein Rahmani (Lancaster University)

class-specific importance weightsdescriptor-wise predictionsgeneralized zero-shot settingsglobal descriptorsllm-guided semantic priorslocal descriptorsmodel generalizationnon-parametric cachentu rgb+d.semantic reasoningskeleton-cachestructured skeleton representationstest-time adaptationzero-shot action recognitionzero-shot settings

We introduce Skeleton-Cache, the first training-free test-time adaptation framework for skeleton-based zero-shot action recognition (SZAR), aimed at improving model generalization to unseen actions during inference. Skeleton-Cache reformulates inference as a lightweight retrieval process over a non-parametric cache that stores structured skeleton representations, combining both global and fine-grained local descriptors. To guide the fusion of descriptor-wise predictions, we leverage the semantic reasoning capabilities of large language models (LLMs) to assign class-specific importance weights. By integrating these structured descriptors with LLM-guided semantic priors, Skeleton-Cache dynamically adapts to unseen actions without any additional training or access to training data. Extensive experiments on NTU RGB+D 60/120 and PKU-MMD II demonstrate that Skeleton-Cache consistently boosts the performance of various SZAR backbones under both zero-shot and generalized zero-shot settings. The code is publicly available at https://github.com/Alchemist0754/Skeleton-Cache.