# Transformers in Time Series: A Survey

**Authors:** Qingsong Wen, Tian Zhou, Chaoli Zhang, Weiqi Chen, Ziqing Ma, Junchi Yan, Liang Sun  
**arXiv:** [2202.07125](https://arxiv.org/abs/2202.07125)  
**Date:** 2022-02-15  
**Summary page:** https://www.summarizepaper.com/en/paper/2202.07125v1/

## Key points

- Transformers have achieved superior performance in natural language processing and computer vision tasks.
- Interest has grown in applying transformers to time series modeling.
- Transformers are attractive for analyzing time series data due to their ability to capture long-range dependencies and interactions.
- The paper provides a comprehensive survey of transformer schemes for time series modeling.
- A new taxonomy is introduced, categorizing time series transformers based on network modifications and applications.
- Network modifications include adaptations at the module and architecture levels, incorporating techniques like positional encoding and attention mechanisms.
- Applications of time series transformers include forecasting, anomaly detection, and classification tasks.
- Empirical analyses are conducted to evaluate the performance of transformers in different scenarios, including robust analysis, model size analysis, and seasonal-trend decomposition analysis.
- Insights into the strengths and limitations of transformers in time series analysis are provided.
- Future research directions and potential areas for improvement are discussed.
- This survey serves as a valuable resource for researchers interested in applying transformers to time series data.

## Summary

Transformers have emerged as a powerful tool in natural language processing and computer vision, achieving superior performance in various tasks. This success has sparked interest in applying transformers to time series modeling. The ability of transformers to capture long-range dependencies and interactions makes them particularly attractive for analyzing time series data. In this paper, the authors provide a comprehensive survey of transformer schemes for time series modeling. They highlight the strengths and limitations of existing time series transformers through a new taxonomy. The taxonomy is based on two perspectives: network modifications and applications. From the perspective of network modifications, the authors summarize the adaptations made at both the module level and architecture level of time series transformers. These modifications enhance the ability of transformers to handle temporal data by incorporating techniques such as positional encoding and attention mechanisms. From the perspective of applications, the authors categorize time series transformers based on common tasks including forecasting, anomaly detection, and classification. They discuss how transformers have been applied to these tasks and highlight their advantages compared to traditional methods. To evaluate the performance of transformers in time series analysis, the authors conduct empirical analyses including robust analysis, model size analysis, and seasonal-trend decomposition analysis. These analyses provide insights into how transformers perform in different scenarios and shed light on their strengths and limitations. Finally, the authors discuss future directions for research in time series transformers. They suggest potential areas for improvement and identify research gaps that need to be addressed. Overall, this survey serves as a valuable resource for researchers interested in applying transformers to time series data. This paper is considered to be the first comprehensive and systematic summary of recent advances in using transformers for modeling time series data. The authors hope that this survey will inspire further research interest in exploring the potential of transformers in analyzing time series data.