# Generating Fake Cyber Threat Intelligence Using Transformer-Based Models

**Authors:** Priyanka Ranade, Aritran Piplai, Sudip Mittal, Anupam Joshi, Tim Finin  
**arXiv:** [2102.04351](https://arxiv.org/abs/2102.04351)  
**Date:** 2021-02-08  
**Summary page:** https://www.summarizepaper.com/en/paper/2102.04351v3/

## Key points

- Cyber-defense systems can automatically ingest Cyber Threat Intelligence (CTI) containing semi-structured data and/or text to populate knowledge graphs
- Fake CTI can be generated and spread through Open-Source Intelligence (OSINT) communities or on the Web to effect a data poisoning attack on these systems
- Adversaries can use fake CTI examples as training input to subvert cyber defense systems, forcing the model to learn incorrect inputs to serve their malicious needs
- Researchers have conducted a study in which they automatically generate fake CTI text descriptions using transformers and demonstrate that given an initial prompt sentence, a public language model like GPT-2 with fine-tuning can generate plausible CTI text with the ability of corrupting cyber-defense systems
- The researchers then utilize the generated fake CTI text to perform a data poisoning attack on a Cybersecurity Knowledge Graph (CKG) and a cybersecurity corpus, introducing adverse impacts such as returning incorrect reasoning outputs, representation poisoning and corruption of other dependent AI-based cyber defense systems
- Professional threat hunters were equally likely to consider their fake generated CTI as true based on human evaluation study results
- This study highlights the need for increased vigilance when it comes to cyber defense systems' vulnerability to data poisoning attacks.

## Summary

Cyber-defense systems are becoming increasingly sophisticated, with the ability to automatically ingest Cyber Threat Intelligence (CTI) that contains semi-structured data and/or text to populate knowledge graphs. However, this development comes with potential risks as fake CTI can be generated and spread through Open-Source Intelligence (OSINT) communities or on the Web to effect a data poisoning attack on these systems. Adversaries can use fake CTI examples as training input to subvert cyber defense systems, forcing the model to learn incorrect inputs to serve their malicious needs. To explore this issue further, researchers Priyanka Ranade, Aritran Piplai, Sudip Mittal, Anupam Joshi, and Tim Finin have conducted a study in which they automatically generate fake CTI text descriptions using transformers. They demonstrate that given an initial prompt sentence, a public language model like GPT-2 with fine-tuning can generate plausible CTI text with the ability of corrupting cyber-defense systems. The researchers then utilize the generated fake CTI text to perform a data poisoning attack on a Cybersecurity Knowledge Graph (CKG) and a cybersecurity corpus. The poisoning attack introduced adverse impacts such as returning incorrect reasoning outputs, representation poisoning and corruption of other dependent AI-based cyber defense systems. To evaluate their findings, the researchers utilized traditional approaches and conducted a human evaluation study with cybersecurity professionals and threat hunters. Based on the study's results professional threat hunters were equally likely to consider their fake generated CTI as true. This study highlights the need for increased vigilance when it comes to cyber defense systems' vulnerability to data poisoning attacks. As adversaries become more sophisticated in their methods of attack it is crucial that those responsible for developing these systems remain vigilant in identifying potential vulnerabilities and taking steps to mitigate them.