# How Good are Commercial Large Language Models on African Languages?

**Authors:** Jessica Ojo, Kelechi Ogueji  
**arXiv:** [2305.06530](https://arxiv.org/abs/2305.06530)  
**Date:** 2023-05-11  
**Summary page:** https://www.summarizepaper.com/en/paper/2305.06530v1/

## Key points

- Recent advancements in Natural Language Processing (NLP) have led to the widespread use of large pretrained language models.
- Effectiveness of these models on African languages has not been extensively studied.
- Preliminary analysis conducted on commercial large language models for eight African languages across different language families and geographical regions.
- Evaluation focused on machine translation and text classification tasks.
- Findings show subpar performance of commercial language models on African languages.
- Better performance observed on text classification compared to machine translation for these languages.
- Urgent need to ensure adequate representation of African languages in commercial large language models due to their increasing popularity and usage.
- Study presented at AfricaNLP Workshop at ICLR 2023 by Jessica Ojo and Kelechi Ogueji from Masakhane.
- Call-to-action emphasizes improving inclusivity of these models to better serve diverse linguistic communities worldwide.

## Summary

Recent advancements in Natural Language Processing (NLP) have led to the widespread use of large pretrained language models. However, their effectiveness on African languages has not been extensively studied. To address this gap, we conducted a preliminary analysis of commercial large language models on eight African languages across different language families and geographical regions. Specifically, we evaluated their performance on machine translation and text classification tasks. Our findings revealed that these commercial language models exhibit subpar performance when applied to African languages. Interestingly, we observed that they perform better on text classification compared to machine translation for these languages. Overall, our results underscore the urgent need to ensure that African languages are adequately represented in commercial large language models given their increasing popularity and usage. This study was presented at the AfricaNLP Workshop at ICLR 2023 by Jessica Ojo and Kelechi Ogueji from Masakhane. The call-to-action highlighted in our findings emphasizes the importance of improving the inclusivity of these models to better serve diverse linguistic communities worldwide.