# Proficiency assessment of L2 spoken English using wav2vec 2.0

**Authors:** Stefano Bannò, Marco Matassoni  
**arXiv:** [2210.13168](https://arxiv.org/abs/2210.13168)  
**Date:** 2022-10-24  
**Summary page:** https://www.summarizepaper.com/en/paper/2210.13168v1/

## Key points

- Demand for learning English as a second language is increasing
- Interest in methods for automatically assessing spoken language proficiency is growing
- Most approaches rely on hand-crafted features that may discard potentially salient information about proficiency
- Transcriptions produced by ASR systems may not provide a faithful rendition of a learner's utterance in specific scenarios and do not yield information about relevant aspects such as intonation, rhythm or prosody
- Researchers investigated the use of wav2vec 2.0 for assessing overall and individual aspects of proficiency on two small datasets: ICNALE and TLT-school
- The ICNALE dataset comprises written and spoken responses of English learners ranging from A2 to B2 of the CEFR for languages and partially of native speakers, while the TLT-school dataset consists of recordings from non-native children aged between 7-12 years old.
- Experiments were conducted using a small quantity of training data but still managed to achieve promising results on both datasets.
- The wav2vec 2.0 approach significantly outperformed the BERT-based baseline system trained on ASR and manual transcriptions used for comparison.
- The approach could assess individual aspects of proficiency such as pronunciation accuracy and fluency.
- This study highlights the potential effectiveness of using wav2vec 2.0 for automatic spoken language proficiency assessment even with limited training data.

## Summary

The demand for learning English as a second language has increased, leading to a growing interest in methods for automatically assessing spoken language proficiency. However, most approaches rely on hand-crafted features that may discard potentially salient information about proficiency. Additionally, transcriptions produced by ASR systems may not provide a faithful rendition of a learner's utterance in specific scenarios and do not yield information about relevant aspects such as intonation, rhythm or prosody. In this study, the researchers investigate the use of wav2vec 2.0 for assessing overall and individual aspects of proficiency on two small datasets: ICNALE and TLT-school. The ICNALE dataset is publicly available and comprises written and spoken responses of English learners ranging from A2 to B2 of the CEFR for languages and partially of native speakers. The TLT-school dataset consists of recordings from non-native children aged between 7-12 years old. The experiments were conducted using a small quantity of training data but still managed to achieve promising results on both datasets. The researchers divided the ICNALE data into a training set, development set, and test set with 3898 answers, 217 answers each. For the experiments on this dataset, proficiency assessment is treated as a classification task with five classes: A2, B1 1, B1 2, B2, and native speakers. The results show that the wav2vec 2.0 approach significantly outperforms the BERT-based baseline system trained on ASR and manual transcriptions used for comparison. Furthermore, the researchers found that their approach could assess individual aspects of proficiency such as pronunciation accuracy and fluency. Overall, this study highlights the potential effectiveness of using wav2vec 2.0 for automatic spoken language proficiency assessment even with limited training data.