# Do Users Write More Insecure Code with AI Assistants?

**Authors:** Neil Perry, Megha Srivastava, Deepak Kumar, Dan Boneh  
**arXiv:** [2211.03622](https://arxiv.org/abs/2211.03622)  
**Date:** 2022-11-07  
**Summary page:** https://www.summarizepaper.com/en/paper/2211.03622v3/

## Key points

- Investigated how users interact with an AI Code assistant for security-related tasks in various programming languages
- Focused on the impact of using OpenAI's codex-davinci-002 model on code security
- Participants with access to the AI assistant produced code with significantly more security vulnerabilities compared to those without it
- Emphasizes potential over-reliance on AI assistants for security tasks and the need for a balanced approach
- Participants who exhibited lower trust in the AI assistant and actively adjusted parameters tended to produce code with fewer vulnerabilities
- Critical approach towards utilizing AI tools can lead to better outcomes in terms of code security
- Authors provided an in-depth analysis of participants' language and interaction behavior to inform future design of AI-based Code assistants
- Released user interface as a tool for conducting similar studies in the future

## Summary

investigated how users interact with an AI Code assistant when solving security-related tasks in various programming languages. The study focused on the impact of using OpenAI's codex-davinci-002 model on code security. The findings revealed that participants who had access to the AI assistant produced code with significantly more security vulnerabilities compared to those who did not use the assistant. This discrepancy highlights a potential over-reliance on AI assistants for security tasks and emphasizes the importance of fostering a balanced approach towards leveraging AI technology for software development. The study also found that participants who exhibited lower levels of trust in the AI assistant and engaged more actively in rephrasing prompts or adjusting parameters (such as temperature) tended to produce code with fewer security vulnerabilities. This suggests that a critical approach towards utilizing AI tools can lead to better outcomes in terms of code security. To further inform the design of future AI-based Code assistants, the authors provided an in-depth analysis of participants' language and interaction behavior. Additionally, they released their user interface as a tool for conducting similar studies in the future. , , , , .