Multi-Scale Finetuning for Encoder-based Time Series Foundation Models

Xudong Jiang (Nanyang Technological University) · Ming Jin (Griffith University) · Qingsong Wen (Head of AI Research @ Squirrel AI) · Chenghao Liu (Datadog) · Yiming Zhang (Nanyang Technological University) · Zhongzheng Qiao (Nanyang Technological University) · Quang Pham (A*STAR) · Ponnuthurai Suganthan (University of Qatar) · Savitha Ramasamy (Institute for Infocomm Research, Agency for Science, Technology and Research, Singapore)

causal perspective deep learning methods downstream tasksencoder-based modelsfinetuningmulti-scale forecastingmultiscale finetuningoverfittingparameter efficient methodsperformance gainssampling scalestemporal patterns time series foundation models zero-shot performance

Time series foundation models (TSFMs) demonstrate impressive zero-shot performance for time series forecasting. However, an important yet underexplored challenge is how to effectively finetune TSFMs on specific downstream tasks. While naive finetuning can yield performance gains, we argue that it falls short of fully leveraging TSFMs' capabilities, often resulting in overfitting and suboptimal performance. Given the diverse temporal patterns across sampling scales and the inherent multi-scale forecasting capabilities of TSFMs, we adopt a causal perspective to analyze finetuning process, through which we highlight the critical importance of explicitly modeling multiple scales and reveal the shortcomings of naive approaches. Focusing on encoder-based TSFMs, we propose Multiscale finetuning (MSFT), a simple yet general framework that explicitly integrates multi-scale modeling into the finetuning process. Experimental results on three different backbones (Moirai, Moment and Units) demonstrate that TSFMs finetuned with MSFT not only outperform naive and typical parameter efficient finetuning methods but also surpass state-of-the-art deep learning methods. Codes are available at https://github.com/zqiao11/MSFT.