# Adversarial Deep Reinforcement Learning for Cyber Security in Software Defined Networks

**Authors:** Luke Borchjes, Clement Nyirenda, Louise Leenen  
**arXiv:** [2308.04909](https://arxiv.org/abs/2308.04909)  
**Date:** 2023-08-09  
**Summary page:** https://www.summarizepaper.com/en/paper/2308.04909v1/

## Key points

- The paper explores the impact of leveraging autonomous offensive approaches in Deep Reinforcement Learning (DRL) for autonomous security in Software Defined Networks (SDN).
- Two algorithms, Double Deep Q-Networks (DDQN) and Neural Episodic Control to Deep Q-Network (NEC2DQN or N2D), are compared.
- The attacker has full observability of the environment and access to a causative attack that manipulates state.
- Two games are played with roles reversed: DDQN as defender and N2D as attacker, once without an active causative attack and once with an active causative attack.
- Three sets of game results are recorded, each consisting of 10 game runs.
- Small parameter changes made to the algorithms result in growth in the attacker's capability.
- Both algorithms demonstrate their ability to defend against adversarial attacks according to their strengths.
- Leveraging autonomous offensive approaches can enhance deep reinforcement learning for autonomous security in SDNs.

## Summary

This paper explores the impact of leveraging autonomous offensive approaches in Deep Reinforcement Learning (DRL) to train more robust agents for autonomous security in Software Defined Networks (SDN). The authors compare two algorithms, Double Deep Q-Networks (DDQN) and Neural Episodic Control to Deep Q-Network (NEC2DQN or N2D), with NEC2DQN being a new member of the deep q-network (DQN) family of algorithms proposed in 2018. The attacker in this study has full observability of the environment and access to a causative attack that manipulates state in an attempt to disrupt the learning process. The attack is implemented under a white-box setting, where the attacker has access to the defender's model and experiences. Two games are played: in the first game, DDQN acts as the defender while N2D plays as the attacker, and in the second game, their roles are reversed. The games are played twice: once without an active causative attack and once with an active causative attack. Three sets of game results are recorded, with each set consisting of 10 game runs. The before and after results are compared to determine if there was any improvement or degradation. The results show that even with small parameter changes made to the algorithms, there is growth in the attacker's capability as it becomes capable of winning games. However, despite these adversarial attacks both algorithms demonstrate their ability to defend against them according to their strengths. In conclusion, this study highlights how leveraging autonomous offensive approaches can enhance deep reinforcement learning for autonomous security in SDNs by providing insights into how adversarial learning affects DRL algorithms and demonstrating their resilience against causative attacks when defending networks.