TimeXL: Explainable Multi-modal Time Series Prediction with LLM-in-the-Loop

Wei Cheng (NEC Labs America) · Haifeng Chen (NEC Labs America) · Yushan Jiang (University of Connecticut) · Dongjin Song (University of Connecticut) · Wenchao Yu (NEC Laboratories America) · Geon Lee (Korea Advanced Institute of Science and Technology) · Kijung Shin (Korea Advanced Institute of Science and Technology) · Yanchi Liu (NEC-Labs)

auc improvementauxiliary modalitiescase-based rationalesclosed-loop workflowempirical evaluationsencoder retraininghuman-centric explanationsinterpretabilityllm-driven reasoningmulti-modal predictionprediction refinementprototype-based encodertextual inconsistenciestime series analysis

Time series analysis provides essential insights for real-world system dynamics and informs downstream decision-making, yet most existing methods often overlook the rich contextual signals present in auxiliary modalities. To bridge this gap, we introduce TimeXL, a multi-modal prediction framework that integrates a prototype-based time series encoder with three collaborating Large Language Models (LLMs) to deliver more accurate predictions and interpretable explanations. First, a multi-modal prototype-based encoder processes both time series and textual inputs to generate preliminary forecasts alongside case-based rationales. These outputs then feed into a prediction LLM, which refines the forecasts by reasoning over the encoder's predictions and explanations. Next, a reflection LLM compares the predicted values against the ground truth, identifying textual inconsistencies or noise. Guided by this feedback, a refinement LLM iteratively enhances text quality and triggers encoder retraining. This closed-loop workflow