# HarmAug: Effective Data Augmentation for Knowledge Distillation of Safety Guard Models

**Authors:** Seanie Lee, Haebin Seong, Dong Bok Lee, Minki Kang, Xiaoyin Chen, Dominik Wagner, Yoshua Bengio, Juho Lee, Sung Ju Hwang  
**arXiv:** [2410.01524](https://arxiv.org/abs/2410.01524)  
**Date:** 2024-10-02  
**Summary page:** https://www.summarizepaper.com/en/paper/2410.01524v2/

## Key points

- Safety guard models are crucial for detecting malicious queries and ensuring responsible deployment of large language models (LLMs) in real-world applications.
- Deploying safety guard models alongside LLMs on mobile devices faces challenges due to high memory requirements and latency issues.
- Researchers have developed a novel approach to distill large teacher safety guard models into smaller, more efficient versions using a labeled dataset of instruction-response pairs with harmfulness labels.
- Limited diversity of harmful instructions in existing datasets leads to underperformance of naively distilled models compared to larger ones.
- HarmAug is introduced as a data augmentation method that prompts LLMs to generate harmful instructions by jailbreaking them with specific prompts, enhancing diversity and quality of generated content.
- Models trained with HarmAug outperform other baselines, achieving comparable F1 scores and surpassing larger models in AUPRC while operating at significantly lower computational costs.
- The study proposes distilling large safety guard models into smaller sub-billion parameter models for efficient deployment and introduces HarmAug as a technique bridging the performance gap between small and large safety guard models.
- Open-source release of synthetic datasets, safety guard models, and code enables further research in improving detection capabilities for harmful conversations and enhancing computational efficiency in safety guard models.

## Summary

In the realm of large language models (LLMs), safety guard models play a crucial role in detecting malicious queries and ensuring the responsible deployment of LLMs in real-world applications. However, deploying these safety guard models alongside LLMs on mobile devices poses challenges due to high memory requirements and latency issues. To address this issue, researchers have developed a novel approach to distill a large teacher safety guard model into a smaller, more efficient version using a labeled dataset of instruction-response pairs with harmfulness labels. One of the key challenges faced during this distillation process is the limited diversity of harmful instructions in the existing labeled dataset, leading to underperformance of naively distilled models compared to larger ones. To bridge this performance gap, the researchers introduce HarmAug, a data augmentation method that involves prompting an LLM to generate harmful instructions by jailbreaking it with specific prompts. By adding an affirmative prefix to the LLM's response and encouraging it to continue generating offensive content, HarmAug effectively enhances the diversity and quality of harmful instructions generated. Empirical results demonstrate that models trained with HarmAug outperform other relevant baselines. Notably, a 435-million-parameter safety guard model trained using HarmAug achieves an F1 score comparable to larger models with over 7 billion parameters and even surpasses them in terms of Area Under the Precision-Recall Curve (AUPRC). Furthermore, this model operates at less than 25% of their computational cost, showcasing its efficiency and effectiveness. The contributions and findings outlined in this study include proposing a method for distilling large safety guard models into smaller sub-billion parameter models for efficient deployment. The introduction of HarmAug as a data augmentation technique bridges the performance gap between small and large safety guard models while significantly reducing computational costs. The release of synthetic datasets, safety guard models, and code as open-source resources enables further research and development in improving detection capabilities for harmful conversations and enhancing computational efficiency in safety guard models. Overall, this refined summary highlights the innovative approach taken by researchers to enhance the security and efficiency of deploying safety guard models alongside LLMs in real-world scenarios.