How Useful are Educational Questions Generated by Large Language Models?

AI-generated keywords: Controllable Text Generation

AI-generated Key Points

  • Controllable text generation (CTG) has the potential to revolutionize education for teachers and students.
  • Prompt engineering is an emerging paradigm for text generation that involves prompting LLMs for a desired output by feeding an input prompt or query.
  • CTG requires a control element in the prompt, such as a keyword, to guide the generation.
  • By combining CTG and question taxonomies like Bloom's taxonomy and difficulty-level taxonomy, different types of questions can be generated from a given context that teachers think are appropriate for use in the classroom.
  • A human evaluation with teachers across two domains and nine question types showed that the questions generated are high quality and sufficiently useful, demonstrating their promise for widespread use in the classroom setting.
  • Long context passages (6-9 sentences) were found to improve generation empirically, while few-shot setting outperformed zero-shot setting with five-shot performing best.
  • Few-shot generation consists of an instruction, examples, and desired task.
  • Controllable text generation by large language models shows great potential for transforming education through high-quality and diverse question generation that can reduce teacher workload while improving educational content quality.
Also access our AI generated: Comprehensive summary, Lay summary, Blog-like article; or ask questions about this paper to our AI assistant.

Authors: Sabina Elkins, Ekaterina Kochmar, Jackie C. K. Cheung, Iulian Serban

Accepted to AIED Late Breaking Results 2023 - to be published in their proceedings
License: CC BY 4.0

Abstract: Controllable text generation (CTG) by large language models has a huge potential to transform education for teachers and students alike. Specifically, high quality and diverse question generation can dramatically reduce the load on teachers and improve the quality of their educational content. Recent work in this domain has made progress with generation, but fails to show that real teachers judge the generated questions as sufficiently useful for the classroom setting; or if instead the questions have errors and/or pedagogically unhelpful content. We conduct a human evaluation with teachers to assess the quality and usefulness of outputs from combining CTG and question taxonomies (Bloom's and a difficulty taxonomy). The results demonstrate that the questions generated are high quality and sufficiently useful, showing their promise for widespread use in the classroom setting.

Submitted to arXiv on 13 Apr. 2023

Ask questions about this paper to our AI assistant

You can also chat with multiple papers at once here.

AI assistant instructions?

Results of the summarizing process for the arXiv paper: 2304.06638v1

Controllable text generation (CTG) by large language models has the potential to revolutionize education for both teachers and students. Prompt engineering is an emerging paradigm for text generation that involves prompting LLMs for a desired output by feeding an input prompt or query. For CTG, a control element must be included in the prompt, such as a keyword that will guide the generation. By combining CTG and question taxonomies like Bloom's taxonomy and difficulty-level taxonomy, we can generate different types of questions from a given context that teachers think are appropriate for use in the classroom. To assess the quality and usefulness of outputs from combining CTG and question taxonomies, we conducted a human evaluation with teachers across two domains and nine question types. The results demonstrate that the questions generated are high quality and sufficiently useful, showing their promise for widespread use in the classroom setting. The parameter settings used in this study were guided by preliminary experimentation. Long context passages (6-9 sentences) were found to improve generation empirically, while few-shot setting outperformed zero-shot setting with five-shot performing best. Few-shot generation consists of an instruction, examples, and desired task. Overall, controllable text generation by large language models shows great potential for transforming education through high-quality and diverse question generation that can reduce teacher workload while improving educational content quality.
Created on 02 May. 2023

Assess the quality of the AI-generated content by voting

Score: 0

Why do we need votes?

Votes are used to determine whether we need to re-run our summarizing tools. If the count reaches -10, our tools can be restarted.

Similar papers summarized with our AI tools

Navigate through even more similar papers through a

tree representation

Look for similar papers (in beta version)

By clicking on the button above, our algorithm will scan all papers in our database to find the closest based on the contents of the full papers and not just on metadata. Please note that it only works for papers that we have generated summaries for and you can rerun it from time to time to get a more accurate result while our database grows.

Disclaimer: The AI-based summarization tool and virtual assistant provided on this website may not always provide accurate and complete summaries or responses. We encourage you to carefully review and evaluate the generated content to ensure its quality and relevance to your needs.