Results of the summarizing process for the arXiv paper: 2506.08872v1

This study investigated the effects of using Large Language Models (LLMs) in essay writing tasks on neural and behavioral levels. Participants were divided into three groups: LLM, Search Engine, and Brain-only (no tools), with each group completing three sessions under the same condition. A fourth session was conducted where participants from the LLM group switched to the Brain-only group (LLM-to-Brain), and participants from the Brain-only group switched to the LLM condition (Brain-to-LLM). A total of 54 participants took part in Sessions 1-3, with 18 completing session 4. The study utilized electroencephalography (EEG) to measure cognitive load during essay writing and analyzed essays using Natural Language Processing (NLP). Essays were also scored by human teachers and an AI judge. The results showed that within-group homogeneity was observed in Named Entity Recognition (NERs), n-gram patterns, and topic ontology. EEG analysis revealed significant differences in brain connectivity among the groups: Brain-only participants exhibited strong and distributed networks, Search Engine users showed moderate engagement, while LLM users displayed weak connectivity. In session 4, LLM-to-Brain participants exhibited reduced alpha and beta connectivity, indicating under-engagement. On the other hand, Brain-to-LLM users showed higher memory recall and activation of specific brain areas similar to Search Engine users. Self-reported ownership of essays was lowest in the LLM group and highest in the Brain-only group. Additionally, LLM users struggled with accurately quoting their own work. Over a four-month period, LLM users consistently underperformed at neural, linguistic, and behavioral levels compared to other groups. These findings suggest potential cognitive costs associated with relying on LLMs for educational tasks. The study raises concerns about the long-term implications of LLM usage in learning contexts and emphasizes the need for further research into AI's role in education. For more detailed information on the experimental design, participant protocols, prompts used during each session, as well as additional data summaries including specific EEG dDTF values, readers are encouraged to visit https://www.brainonllm.com/.

• - Study investigated effects of Large Language Models (LLMs) in essay writing tasks on neural and behavioral levels
• - Participants divided into LLM, Search Engine, and Brain-only groups for sessions 1-3
• - EEG used to measure cognitive load during essay writing; NLP analyzed essays
• - Significant differences in brain connectivity among groups: Brain-only strong networks, Search Engine moderate engagement, LLM weak connectivity
• - LLM-to-Brain participants showed reduced alpha and beta connectivity in session 4
• - Brain-to-LLM users exhibited higher memory recall and activation of specific brain areas similar to Search Engine users
• - LLM group had lowest self-reported ownership of essays and struggled with quoting own work accurately
• - Over four months, LLM users consistently underperformed at neural, linguistic, and behavioral levels compared to other groups
• - Study raises concerns about cognitive costs of relying on LLMs for educational tasks; emphasizes need for further research into AI's role in education

SummaryA study looked at how big computer programs affect writing essays in three different ways. They used special machines to measure brains and behavior while writing. The results showed that using the big program alone made strong connections in the brain, but weak ones with the program itself. People who started with the brain did better at remembering things and using their brains like those who used a search engine. The big program group struggled to claim their work as their own and didn't do well over time compared to others. Definitions- Large Language Models (LLMs): Big computer programs that help with writing and understanding language. - Neural: Related to the brain or nervous system. - Behavioral: How people act or behave. - EEG: A machine that measures brain activity. - NLP (Natural Language Processing): Using computers to understand human language. - Connectivity: How different parts are connected or work together. - Alpha and beta connectivity: Different types of brain waves related to thinking and focus. - Self-reported ownership: Saying something belongs to you. - Cognitive costs: How much effort it takes mentally. - AI (Artificial Intelligence): Machines programmed to think and learn like humans.

Introduction: The use of artificial intelligence (AI) has become increasingly prevalent in various fields, including education. One area where AI is being utilized is in essay writing tasks through the use of Large Language Models (LLMs). These LLMs are powerful tools that can generate human-like text and have the potential to assist students in their writing assignments. However, a recent study conducted by researchers at a prominent university investigated the effects of using LLMs on neural and behavioral levels during essay writing tasks. The results of this study raise concerns about the long-term implications of relying on LLMs for educational purposes. Experimental Design: The study involved 54 participants who were divided into three groups: LLM, Search Engine, and Brain-only (no tools). Each group completed three sessions under the same condition, with prompts provided for each session. A fourth session was then conducted where participants from the LLM group switched to the Brain-only group (LLM-to-Brain), and participants from the Brain-only group switched to the LLM condition (Brain-to-LLM). This design allowed for a comparison between groups as well as within-group comparisons after switching conditions. Data Collection and Analysis: To measure cognitive load during essay writing, electroencephalography (EEG) was used to record brain activity. Natural Language Processing (NLP) techniques were also employed to analyze essays written by participants. Additionally, essays were scored by both human teachers and an AI judge. The data collected from these methods provided insights into neural connectivity patterns, linguistic features such as Named Entity Recognition (NERs) and n-gram patterns, as well as overall essay quality. Results: Within-group homogeneity was observed in NERs, n-gram patterns, and topic ontology among all three groups in Sessions 1-3. However, EEG analysis revealed significant differences in brain connectivity among them: Brain-only participants exhibited strong and distributed networks while Search Engine users showed moderate engagement. In contrast, LLM users displayed weak connectivity, indicating potential under-engagement. This was further supported by self-reported ownership of essays, which was lowest in the LLM group and highest in the Brain-only group. In Session 4, after switching conditions, LLM-to-Brain participants exhibited reduced alpha and beta connectivity compared to their previous session as LLM users. This suggests a decrease in cognitive load and potentially less engagement with the writing task. On the other hand, Brain-to-LLM users showed higher memory recall and activation of specific brain areas similar to Search Engine users. However, they also struggled with accurately quoting their own work. Over a four-month period, LLM users consistently underperformed at neural, linguistic, and behavioral levels compared to other groups. These findings suggest potential cognitive costs associated with relying on LLMs for educational tasks. Implications: The results of this study raise concerns about the long-term implications of using LLMs for essay writing tasks in education. The weaker neural connectivity observed among LMM users may indicate a lack of critical thinking skills or reliance on external sources rather than one's own knowledge and understanding. Additionally, the struggle with accurately quoting one's own work highlights potential issues with originality and plagiarism when using these tools. Further research is needed to fully understand the role of AI in education and its impact on students' learning processes. It is crucial to consider not only short-term benefits but also long-term consequences when implementing new technologies in educational settings. Conclusion: In conclusion, this study provides valuable insights into the effects of using Large Language Models (LLMs) on neural and behavioral levels during essay writing tasks. The results suggest potential cognitive costs associated with relying on these tools for educational purposes. As such, it is essential to carefully consider their use in academic settings and continue researching their impact on students' learning processes. To access more detailed information on the experimental design, participant protocols, prompts used during each session, as well as additional data summaries including specific EEG dDTF values, readers are encouraged to visit the study's website at https://www.brainonllm.com/. This will provide a deeper understanding of the study and its findings.