# ChatGPT in the classroom. Exploring its potential and limitations in a Functional Programming course

**Authors:** Dan-Matei Popovici  
**arXiv:** [2401.11166](https://arxiv.org/abs/2401.11166)  
**Date:** 2024-01-20  
**Summary page:** https://www.summarizepaper.com/en/paper/2401.11166v1/

## Key points

- OpenAI introduced ChatGPT in November 2022
- ChatGPT is based on supervised and reinforcement learning
- ChatGPT can answer questions with human-like responses and generate code from scratch or complete coding templates
- A survey at POLITEHNICA University of Bucharest (UPB) found over 40% of students had been using generative AI tools like ChatGPT for at least six months for coding assignments
- ChatGPT provided correct answers in 68% of cases, but only around half were deemed legible solutions beneficial to students
- ChatGPT excelled in performing code reviews on student programming homework
- ChatGPT scored approximately 7 out of 10 for correctness in a study on 72 programming tasks from the Functional Programming course at UPB, but struggled with efficiency and comprehensibility
- The evaluation highlighted that errors pointed out by students could improve ChatGPT's score to 8.6, requiring a certain level of expertise from students
- Generating tests for solutions proved challenging as only 70% were accurate, potentially causing confusion when tests failed for students
- Generative AI tools like ChatGPT are commonly used by students for homework and exams according to survey results
- Further assessment is needed to determine the accuracy and practicality of these tools in education, especially regarding providing correct and useful results for students
- The dataset used for evaluation consisted of exercises from the Functional Programming course at UPB categorized into easy, medium, and hard levels based on teaching experience
- While promising in assisting with coding assignments and providing feedback on student work, limitations exist regarding efficiency and comprehension that need addressing for optimal educational use
- Further research is necessary to explore effective integration of generative AI tools like ChatGPT into programming-focused lectures while ensuring they enhance learning outcomes

## Summary

In November 2022, OpenAI introduced ChatGPT, a chatbot based on supervised and reinforcement learning that can answer questions with human-like responses and generate code from scratch or complete coding templates. The release of ChatGPT sparked a debate about its use in academia, particularly by students. Surprisingly, a survey conducted at POLITEHNICA University of Bucharest (UPB) revealed that over 40% of students had been using generative AI tools like ChatGPT for at least six months for coding assignments. Further exploration into the tool's capabilities showed that ChatGPT provided correct answers in 68% of cases but only around half were deemed legible solutions beneficial to students. However, ChatGPT excelled in performing code reviews on student programming homework. A study on 72 programming tasks from the Functional Programming course at UPB found that ChatGPT scored approximately 7 out of 10 for correctness but struggled with efficiency and comprehensibility. Additionally, the evaluation highlighted that ChatGPT could improve its score to 8.6 when errors were pointed out, although this required a certain level of expertise from students. Generating tests for solutions also proved challenging as only 70% were accurate, leading to potential confusion for students when tests failed. The survey results indicated that generative AI tools like ChatGPT are commonly used by students for homework and exams. To gain a better understanding of their accuracy and practicality in education, further assessment is needed to determine how these tools can provide correct and useful results for students. The dataset used for evaluation consisted of exercises from the Functional Programming course at UPB categorized into easy, medium, and hard levels based on teaching experience. Overall, while ChatGPT shows promise in assisting with coding assignments and providing feedback on student work, there are limitations regarding efficiency and comprehension that need to be addressed for optimal educational use. Further research is necessary to explore ways to effectively integrate generative AI tools like ChatGPT into programming-focused lectures while ensuring they enhance learning outcomes for students.