Pu Zhang, Zhaowen Li, Zongxin Gao, Shiming Xiang. A Single Frozen Transformer in Language Models is Simple yet Powerful Time Series ForecasterJ. Machine Intelligence Research, 2026, 23(6): 1318-1339. DOI: 10.1007/s11633-025-1622-x
Citation: Pu Zhang, Zhaowen Li, Zongxin Gao, Shiming Xiang. A Single Frozen Transformer in Language Models is Simple yet Powerful Time Series ForecasterJ. Machine Intelligence Research, 2026, 23(6): 1318-1339. DOI: 10.1007/s11633-025-1622-x

A Single Frozen Transformer in Language Models is Simple yet Powerful Time Series Forecaster

  • Time series forecasting (TSF) is critical in various real-world applications, and has been extensively studied in the research community. Recent state-of-the-art (SOTA) methods for this task largely rely on fine-tuning or prompt engineering within the context of large language models (LLMs). However, fine-tuning can compromise the original language capabilities of the models, thereby affecting their generalization. Meanwhile, incorporating prompts significantly increases computational overhead. To address these challenges, we propose Frozen-Time, a simple, lightweight, and efficient approach leveraging LLMs for effective TSF across various domains. Technically, Frozen-Time first divides the input time series into non-overlapping patches with uniform temporal resolution. This naturally facilitates the capture of local semantics and the modeling of internal correlations within the time series. After the divided patches pass through the adaptive input embedding linear layer, we propose a frozen module as the forecaster, which consists of a single frozen transformer layer from the LLM. Finally, the forecasting results are generated through a lightweight output projection layer. This adaptive design avoids the overlapping time series misinterpretation, significantly reduces computational costs, and maintains high performance. Extensive experiments on popular public datasets demonstrate the robustness, computational efficiency, and superior generalization capabilities of Frozen-Time, consistently outperforming SOTA methods in long-term, short-term, few-shot, and zero-shot forecasting tasks. In addition, our method significantly reduces parameters and computational requirements, making it feasible for deployment on less advanced GPUs.
  • loading

Catalog

    /

    DownLoad:  Full-Size Img  PowerPoint
    Return
    Return