# LLMs Killed the Script Kiddie: How Agents Supported by Large Language Models Change the Landscape of Network Threat Testing

**Authors:** Stephen Moskal, Sam Laney, Erik Hemberg, Una-May O'Reilly  
**arXiv:** [2310.06936](https://arxiv.org/abs/2310.06936)  
**Date:** 2023-10-10  
**Summary page:** https://www.summarizepaper.com/en/paper/2310.06936v1/

## Key points

- Large Language Models (LLMs) have the potential to enhance threat analysis, tool generation, and automation of cyber campaigns.
- The study explores how LLMs can support specific actions and decisions related to threats through manual exploration.
- Authors present prompt engineering strategies for a plan-act-report loop for individual threat actions and a prompt chaining design for sequential decision-making in multi-action campaigns.
- The research evaluates LLM's knowledge depth in cybersecurity through a demonstrated short campaign and insights into designing prompts for actionable responses.
- Ethical considerations are raised regarding the use of LLMs in accelerating threat actor capabilities, along with concerns about handling complex networks, sophisticated vulnerabilities, and prompt sensitivity.
- This study is expected to stimulate discussions on the evolving role of LLM-supported technologies in shaping the cyber adversarial landscape.

## Summary

In their paper titled "LLMs Killed the Script Kiddie: How Agents Supported by Large Language Models Change the Landscape of Network Threat Testing," authors Stephen Moskal, Sam Laney, Erik Hemberg, and Una-May O'Reilly delve into the potential of Large Language Models (LLMs) in enhancing threat analysis, tool generation, and automation of cyber campaigns. The study begins with a manual exploration of how LLMs can support specific actions and decisions related to threats. Subsequently, the authors automate the decision-making process within a cyber campaign by presenting prompt engineering strategies for a plan-act-report loop for individual threat actions and a prompt chaining design that guides sequential decision-making in multi-action campaigns. The research evaluates the depth of LLM's knowledge in cybersecurity through a demonstrated short campaign and offers insights into designing prompts that elicit actionable responses. Additionally, the authors discuss the potential implications of LLMs on the threat landscape and raise ethical considerations regarding their use in accelerating threat actor capabilities. While acknowledging the promising application of generative AI in addressing cyber threats, they also highlight concerns about LLMs' ability to handle complex networks, sophisticated vulnerabilities, and prompt sensitivity. This study is expected to stimulate discussions on the evolving role of LLM-supported technologies in shaping the cyber adversarial landscape. By shedding light on both opportunities and challenges associated with leveraging LLMs for cybersecurity purposes, this research underscores the need for ongoing deliberations on advancing capabilities in this domain.