# Is ChatGPT A Good Translator? Yes With GPT-4 As The Engine

**Authors:** Wenxiang Jiao, Wenxuan Wang, Jen-tse Huang, Xing Wang, Shuming Shi, Zhaopeng Tu  
**arXiv:** [2301.08745](https://arxiv.org/abs/2301.08745)  
**Date:** 2023-01-20  
**Summary page:** https://www.summarizepaper.com/en/paper/2301.08745v4/

## Key points

- Comprehensive evaluation of ChatGPT as a machine translator
- Focus on translation prompt, multilingual translation, and translation robustness
- ChatGPT performs well with recommended prompts for translation ability
- Competes competitively with Google Translate for high-resource European languages
- Falls behind significantly for low-resource or distant languages
- Good results for spoken language but not as good for biomedical abstracts or Reddit comments
- Introduction of "pivot prompting" strategy for distant languages, resulting in improved performance
- GPT-4 engine enhances translation performance, even for distant languages
- ChatGPT with GPT-3.5 generates more hallucinations and mis-translations compared to GPT-4
- GPT-4 has better overall accuracy and fewer errors than GPT-3.5
- ChatGPT has become a reliable translator due to advancements brought by GPT-4

## Summary

This report presents a comprehensive evaluation of ChatGPT as a machine translator, focusing on translation prompt, multilingual translation, and translation robustness. The study finds that ChatGPT performs well with minor performance differences when using the prompts recommended by ChatGPT to trigger its translation ability. It competes competitively with commercial translation products like Google Translate for high-resource European languages but falls behind significantly for low-resource or distant languages. While ChatGPT does not perform as well as commercial systems for biomedical abstracts or Reddit comments, it exhibits good results for spoken language. The report also introduces an interesting strategy called "pivot prompting" for distant languages. This strategy involves translating the source sentence into a high-resource pivot language before translating it into the target language, resulting in noticeable improvements in translation performance. With the launch of the GPT-4 engine, the translation performance of ChatGPT is significantly enhanced and becomes comparable to commercial translation products even for distant languages. Human analysis comparing Google Translate and ChatGPT reveals that ChatGPT with GPT-3.5 tends to generate more hallucinations and mis-translation errors compared to GPT-4 which makes fewer errors and has better overall accuracy. Overall, this study concludes that ChatGPT has become a reliable translator due to the advancements brought by GPT-4 and can compete effectively with commercial systems in terms of accuracy and performance.