# DeepSeek-R1 Outperforms Gemini 2.0 Pro, OpenAI o1, and o3-mini in Bilingual Complex Ophthalmology Reasoning

**Authors:** Pusheng Xu, Yue Wu, Kai Jin, Xiaolan Chen, Mingguang He, Danli Shi  
**arXiv:** [2502.17947](https://arxiv.org/abs/2502.17947)  
**Date:** 2025-02-25  
**Summary page:** https://www.summarizepaper.com/en/paper/2502.17947v1/

## Key points

- Study evaluated performance of four large language models (LLMs) in bilingual complex ophthalmology cases
- DeepSeek-R1 emerged as top performer with overall accuracy of 0.862 in Chinese MCQs and 0.808 in English MCQs
- Gemini 2.0 Pro, OpenAI o1, and OpenAI o3-mini achieved accuracies of 0.715, 0.685, and 0.692 in Chinese MCQs respectively
- In English MCQs, they achieved accuracies of 0.746, 0.723, and 0.577 respectively
- DeepSeek-R1 excelled particularly in management questions conducted in Chinese
- All four LLMs shared similar reasoning logic but had common causes of reasoning errors such as ignoring key positive history or signs, misinterpretation of medical data, and being too aggressive
- DeepSeek-R1 showcased promising results in reasoning tasks compared to other state-of-the-art LLMs

## Summary

A recent study by Xu et al. evaluated the performance of four large language models (LLMs) - DeepSeek-R1, Gemini 2.0 Pro, OpenAI o1, and OpenAI o3-mini - in bilingual complex ophthalmology cases. The researchers collected 130 multiple-choice questions from the Chinese ophthalmology senior professional title examination and translated them into English using DeepSeek-R1. From February 15 to February 20, 2025, the responses of these LLMs were generated under default configurations and their accuracy was calculated based on correctly answered questions. DeepSeek-R1 emerged as the top performer with an overall accuracy of 0.862 in Chinese MCQs and 0.808 in English MCQs. Gemini 2.0 Pro, OpenAI o1, and OpenAI o3-mini also achieved accuracies of 0.715, 0.685, and 0.692 in Chinese MCQs respectively (all statistically significant compared to DeepSeek-R1). In English MCQs, they achieved accuracies of 0.746, 0.723, and 0.577 respectively. DeepSeek-R1 demonstrated superior performance across five topics in both Chinese and English MCQs and excelled particularly in management questions conducted in Chinese. Reasoning ability analysis revealed that all four LLMs shared similar reasoning logic but also had common causes of reasoning errors such as ignoring key positive history or signs, misinterpretation of medical data, and being too aggressive. In conclusion,showcased promising results in  reasoning tasks compared to other state-of-the-art LLMs.may pose challenges, but it holds potential for supporting diagnosis and clinical decision-making in the field of ophthalmology.