# Are Language Models Actually Useful for Time Series Forecasting?

**Authors:** Mingtian Tan, Mike A. Merrill, Vinayak Gupta, Tim Althoff, Thomas Hartvigsen  
**arXiv:** [2406.16964](https://arxiv.org/abs/2406.16964)  
**Date:** 2024-06-22  
**Summary page:** https://www.summarizepaper.com/en/paper/2406.16964v2/

## Key points

- Authors: Mingtian Tan, Mike A. Merrill, Vinayak Gupta, Tim Althoff, Thomas Hartvigsen
- Study focus: Application of large language models (LLMs) in time series forecasting
- Removing or replacing LLM component does not degrade forecasting performance and may even improve it
- Pretrained LLMs do not outperform models trained from scratch
- Pretrained LLMs fail to capture sequential dependencies in time series data and offer no advantage in few-shot settings
- Patching and attention structures yield similar performance to LLM-based forecasters
- Acceptance: Presented at NeurIPS 2024 as a Spotlight paper

## Summary

In their paper titled "Are Language Models Actually Useful for Time Series Forecasting? ", authors Mingtian Tan, Mike A. Merrill, Vinayak Gupta, Tim Althoff, and Thomas Hartvigsen explore the application of large language models (LLMs) in time series forecasting. Through a series of ablation studies on three popular LLM-based forecasting methods, the authors investigate the impact of removing the LLM component or replacing it with a basic attention layer. Surprisingly, their findings reveal that these modifications do not degrade forecasting performance; in fact, in most cases, the results even improve. Despite the significant computational cost associated with pretrained LLMs, the study shows that they do not outperform models trained from scratch. Furthermore, pretrained LLMs fail to capture sequential dependencies in time series data and offer no advantage in few-shot settings. The authors also delve into alternative time series encoders and discover that patching and attention structures yield similar performance to LLM-based forecasters. This insightful research was accepted for presentation at NeurIPS 2024 as a Spotlight paper. The comprehensive analysis provided by Tan et al. sheds light on the effectiveness of language models in time series forecasting and offers valuable insights for future developments in this field.