# IDA: Breaking Barriers in No-code UI Automation Through Large Language Models and Human-Centric Design

**Authors:** egev Shlomov, Avi Yaeli, Sami Marreed, Sivan Schwartz, Netanel Eder, Offer Akrabi, Sergey Zeltyn  
**arXiv:** [2407.15673](https://arxiv.org/abs/2407.15673)  
**Date:** 2024-07-22  
**Summary page:** https://www.summarizepaper.com/en/paper/2407.15673v1/

## Key points

- Business users spend a lot of time on repetitive tasks in enterprise digital platforms, highlighting the need for automation solutions.
- Low-code tools have advanced UI automation but are complex for non-technical users.
- Large language models (LLMs) offer powerful yet simpler programming environments tailored to human needs.
- IDA (Intelligent Digital Apprentice) is a cutting-edge no-code Web UI automation tool designed for business users without technical backgrounds.
- IDA features guided programming by demonstration, semantic programming models, and a teacher-student learning metaphor to empower users.
- By leveraging LLMs, IDA overcomes technical barriers traditionally limiting no-code solutions' feasibility.
- A user study showed that real-world business users could effectively use IDA to create automation processes with positive feedback on its user-friendly interface and performance.
- The study marks progress in using AI assistants to enhance productivity through intuitive and efficient no-code user interface automation.

## Summary

In the realm of enterprise digital platforms, business users often find themselves spending a considerable amount of time on repetitive tasks. This underscores the urgent need for automation solutions. While low-code tools have made strides in UI automation, their complexity poses a significant hurdle for non-technical users. However, recent advancements in large language models (LLMs) have opened up new possibilities by offering more powerful yet simpler programming environments that are centered around human needs. Introducing IDA (Intelligent Digital Apprentice), a cutting-edge no-code Web UI automation tool specifically crafted to empower business users with no technical background. IDA is built on human-centric design principles such as guided programming by demonstration, semantic programming models, and a teacher-student learning metaphor tailored to the skill set of business users. By harnessing LLMs, IDA effectively overcomes key technical barriers that have traditionally limited the feasibility of no-code solutions. The development of a prototype of IDA was followed by a user study involving real-world business users and enterprise applications. The results were promising, showcasing that users could effectively leverage IDA to create automation processes. Qualitative feedback from participants highlighted IDA's user-friendly interface and trustworthy performance. This study marks an important step towards unlocking the potential of AI assistants in enhancing the productivity of business users through intuitive and efficient no-code user interface automation. The collaborative efforts of authors Egev Shlomov, Avi Yaeli, Sami Marreed, Sivan Schwartz, Netanel Eder, Offer Akrabi, and Sergey Zeltyn have paved the way for a transformative approach to streamlining workflows within enterprise settings.