# MST-GAT: A Multimodal Spatial-Temporal Graph Attention Network for Time Series Anomaly Detection

**Authors:** Chaoyue Ding, Shiliang Sun, Jing Zhao  
**arXiv:** [2310.11169](https://arxiv.org/abs/2310.11169)  
**Date:** 2023-10-17  
**Summary page:** https://www.summarizepaper.com/en/paper/2310.11169v1/

## Key points

- Multimodal time series (MTS) are important for anomaly detection in various working devices
- Deep learning methods often fail to capture spatial-temporal relationships between different modalities, leading to higher false negatives and false positives
- The proposed approach, MST-GAT, combines a multimodal graph attention network (M-GAT) and a temporal convolution network
- M-GAT utilizes multi-head attention and relational attention modules to model correlations between modalities
- MST-GAT optimizes both reconstruction and prediction modules for accurate modeling of normal patterns and anomaly detection
- Experimental results show that MST-GAT outperforms state-of-the-art baselines in anomaly detection performance
- MST-GAT accurately locates the most anomalous univariate time series within the multimodal data, enhancing interpretability.

## Summary

, , , , 
In the field of anomaly detection, multimodal time series (MTS) play a crucial role in ensuring the safety and stability of various working devices such as water treatment systems and spacecraft. MTS data are characterized by multivariate time series with diverse modalities. While deep learning methods have shown promise in anomaly detection, they often fail to explicitly capture the spatial-temporal relationships between univariate time series of different modalities, leading to higher rates of false negatives and false positives. To address this limitation, this paper proposes a novel approach called the Multimodal Spatial-Temporal Graph Attention Network (MST-GAT). The MST-GAT framework leverages a multimodal graph attention network (M-GAT) and a temporal convolution network to effectively capture the spatial-temporal correlations present in multimodal time series data. The M-GAT component of MST-GAT utilizes a multi-head attention module along with two relational attention modules (intra- and inter-modal attention) to explicitly model the correlations between different modalities. By doing so, it enables the network to better understand how different modalities interact with each other over time. Furthermore, MST-GAT optimizes both reconstruction and prediction modules simultaneously, allowing for more accurate modeling of normal patterns in the data while also enhancing the ability to detect anomalies. Experimental results on four benchmark datasets demonstrate that MST-GAT outperforms state-of-the art baselines in terms of anomaly detection performance. Additionally, further analysis reveals that MST-GAT enhances interpretability by accurately locating the most anomalous univariate time series within the multimodal data, providing valuable insights into which specific aspects or modalities contribute most significantly to detected anomalies. In conclusion, this paper presents MST-GAT as an effective solution for multimodal time series anomaly detection due its ability to explicitly capture spatial temporal relationships between different modalities and enhance interpretability.