# GraphWeaver: Billion-Scale Cybersecurity Incident Correlation
**Authors:** Scott Freitas, Amir Gharib
**arXiv:** [2406.01842](https://arxiv.org/abs/2406.01842)
**Date:** 2024-06-03
**Summary page:** https://www.summarizepaper.com/en/paper/2406.01842v1/
## Key points
- Large enterprise cybersecurity faces challenges in accurately correlating billions of security alerts to form comprehensive incidents
- Traditional correlation techniques struggle with maintenance, scalability, and adapting to emerging threats and diverse telemetry sources
- GraphWeaver is an industry-scale framework that revolutionizes incident correlation by leveraging a data-optimized, geo-distributed graph-based approach
- Key features of GraphWeaver include a geo-distributed database, PySpark analytics engine for large-scale data processing, minimum spanning tree algorithm for optimized correlation storage, integration of security domain knowledge and threat intelligence, and a human-in-the-loop feedback system
- Integrated into Microsoft Defender XDR product globally with proven capability in managing billions of correlations at 99% accuracy rate
- GraphWeaver reduces traditional correlation storage requirements by 7.4 times while upholding high levels of accuracy
- The framework sets a new standard by providing transparency into its advanced methodologies for handling billion-scale incident correlations effectively
## Summary
In the constantly evolving landscape of large enterprise cybersecurity, accurately and efficiently correlating billions of security alerts to form comprehensive incidents is a significant challenge. Traditional correlation techniques often face obstacles in terms of maintenance, scalability, and adapting to emerging threats and diverse sources of telemetry. To address these challenges, GraphWeaver emerges as an industry-scale framework that revolutionizes the incident correlation process by leveraging a data-optimized, geo-distributed graph-based approach. GraphWeaver introduces a range of innovative solutions specifically designed to tackle the complexities associated with correlating vast amounts of shared evidence alerts across hundreds of thousands of enterprises. Among its key features are a geo-distributed database and PySpark analytics engine for handling large-scale data processing, a minimum spanning tree algorithm that optimizes correlation storage, integration of security domain knowledge and threat intelligence, and a human-in-the-loop feedback system that continuously refines crucial correlation processes and parameters. Integrated into the Microsoft Defender XDR product and deployed globally,GraphWeaver has proven its capability by successfully managing billions of correlations with an impressive 99% accuracy rate. This achievement has been validated through customer feedback as well as extensive evaluations conducted by security experts. Furthermore,this integration not only upholds high levels of correlation accuracy but also significantly reduces traditional correlation storage requirements by 7.4 times. This detailed overview sheds light on the fundamental design principles and operational functionalities that underpin GraphWeaver's success in addressing the intricate demands of large-scale cybersecurity incident correlation. By openly discussing these critical capabilities at such depth,GraphWeaver sets a new standard as the first cybersecurity entity to provide transparency into its advanced methodologies for handling billion-scale incident correlations effectively.