# How Useful are Educational Questions Generated by Large Language Models?

**Authors:** Sabina Elkins, Ekaterina Kochmar, Jackie C. K. Cheung, Iulian Serban  
**arXiv:** [2304.06638](https://arxiv.org/abs/2304.06638)  
**Date:** 2023-04-13  
**Summary page:** https://www.summarizepaper.com/en/paper/2304.06638v1/

## Key points

- Controllable text generation (CTG) has the potential to revolutionize education for teachers and students.
- Prompt engineering is an emerging paradigm for text generation that involves prompting LLMs for a desired output by feeding an input prompt or query.
- CTG requires a control element in the prompt, such as a keyword, to guide the generation.
- By combining CTG and question taxonomies like Bloom's taxonomy and difficulty-level taxonomy, different types of questions can be generated from a given context that teachers think are appropriate for use in the classroom.
- A human evaluation with teachers across two domains and nine question types showed that the questions generated are high quality and sufficiently useful, demonstrating their promise for widespread use in the classroom setting.
- Long context passages (6-9 sentences) were found to improve generation empirically, while few-shot setting outperformed zero-shot setting with five-shot performing best.
- Few-shot generation consists of an instruction, examples, and desired task.
- Controllable text generation by large language models shows great potential for transforming education through high-quality and diverse question generation that can reduce teacher workload while improving educational content quality.

## Summary

Controllable text generation (CTG) by large language models has the potential to revolutionize education for both teachers and students. Prompt engineering is an emerging paradigm for text generation that involves prompting LLMs for a desired output by feeding an input prompt or query. For CTG, a control element must be included in the prompt, such as a keyword that will guide the generation. By combining CTG and question taxonomies like Bloom's taxonomy and difficulty-level taxonomy, we can generate different types of questions from a given context that teachers think are appropriate for use in the classroom. To assess the quality and usefulness of outputs from combining CTG and question taxonomies, we conducted a human evaluation with teachers across two domains and nine question types. The results demonstrate that the questions generated are high quality and sufficiently useful, showing their promise for widespread use in the classroom setting. The parameter settings used in this study were guided by preliminary experimentation. Long context passages (6-9 sentences) were found to improve generation empirically, while few-shot setting outperformed zero-shot setting with five-shot performing best. Few-shot generation consists of an instruction, examples, and desired task. Overall, controllable text generation by large language models shows great potential for transforming education through high-quality and diverse question generation that can reduce teacher workload while improving educational content quality.