Creating Large Language Model Resistant Exams: Guidelines and Strategies
AI-generated Key Points
- Large Language Models (LLMs) like ChatGPT may impact academic integrity in exams due to their lack of genuine comprehension.
- Educators need to adapt examination design to maintain assessment integrity and promote essential skills development in students.
- Traditional evaluation methods based on long-form essays may need reconsideration due to advancements in LLMs.
- Exams that do not incorporate modern tools can become inauthentic and fail to represent real-life problem-solving workflows.
- Guidelines for creating LLM-resistant exams include content moderation, misdirecting AI models with deliberate inaccuracies, evaluating real-world scenarios beyond the LLM’s knowledge base, developing effective distractor options, and incorporating non-textual information in examinations.
- Soft skills such as communication, collaboration, leadership, and critical thinking should be emphasized as they are beyond the scope of LLM capabilities.
- By implementing these strategies and shifting the focus towards students’ engagement and comprehension rather than just their ability to produce long-form essays or answer multiple-choice questions accurately while using an LLM tool; educators can create exams that are more resistant to LLM intervention while ensuring a fair and accurate assessment of students’ knowledge and skills.
Authors: Simon kaare Larsen
Abstract: The proliferation of Large Language Models (LLMs), such as ChatGPT, has raised concerns about their potential impact on academic integrity, prompting the need for LLM-resistant exam designs. This article investigates the performance of LLMs on exams and their implications for assessment, focusing on ChatGPT's abilities and limitations. We propose guidelines for creating LLM-resistant exams, including content moderation, deliberate inaccuracies, real-world scenarios beyond the model's knowledge base, effective distractor options, evaluating soft skills, and incorporating non-textual information. The article also highlights the significance of adapting assessments to modern tools and promoting essential skills development in students. By adopting these strategies, educators can maintain academic integrity while ensuring that assessments accurately reflect contemporary professional settings and address the challenges and opportunities posed by artificial intelligence in education.
Ask questions about this paper to our AI assistant
You can also chat with multiple papers at once here.
Assess the quality of the AI-generated content by voting
Why do we need votes?
Votes are used to determine whether we need to re-run our summarizing tools. If the count reaches -10, our tools can be restarted.
Similar papers summarized with our AI tools
Navigate through even more similar papers through atree representation
Look for similar papers (in beta version)
By clicking on the button above, our algorithm will scan all papers in our database to find the closest based on the contents of the full papers and not just on metadata. Please note that it only works for papers that we have generated summaries for and you can rerun it from time to time to get a more accurate result while our database grows.
Disclaimer: The AI-based summarization tool and virtual assistant provided on this website may not always provide accurate and complete summaries or responses. We encourage you to carefully review and evaluate the generated content to ensure its quality and relevance to your needs.