Capabilities of Gemini Models in Medicine
Auteurs : Khaled Saab, Tao Tu, Wei-Hung Weng, Ryutaro Tanno, David Stutz, Ellery Wulczyn, Fan Zhang, Tim Strother, Chunjong Park, Elahe Vedadi, Juanma Zambrano Chaves, Szu-Yeu Hu, Mike Schaekermann, Aishwarya Kamath, Yong Cheng, David G. T. Barrett, Cathy Cheung, Basil Mustafa, Anil Palepu, Daniel McDuff, Le Hou, Tomer Golany, Luyang Liu, Jean-baptiste Alayrac, Neil Houlsby, Nenad Tomasev, Jan Freyberg, Charles Lau, Jonas Kemp, Jeremy Lai, Shekoofeh Azizi, Kimberly Kanada, SiWai Man, Kavita Kulkarni, Ruoxi Sun, Siamak Shakeri, Luheng He, Ben Caine, Albert Webson, Natasha Latysheva, Melvin Johnson, Philip Mansfield, Jian Lu, Ehud Rivlin, Jesper Anderson, Bradley Green, Renee Wong, Jonathan Krause, Jonathon Shlens, Ewa Dominowska, S. M. Ali Eslami, Claire Cui, Oriol Vinyals, Koray Kavukcuoglu, James Manyika, Jeff Dean, Demis Hassabis, Yossi Matias, Dale Webster, Joelle Barral, Greg Corrado, Christopher Semturs, S. Sara Mahdavi, Juraj Gottweis, Alan Karthikesalingam, Vivek Natarajan
Résumé scientifique
Excellence in a wide variety of medical applications poses considerable challenges for AI, requiring advanced reasoning, access to up-to-date medical knowledge and understanding of complex multimodal data. Gemini models, with strong general capabilities in multimodal and long-context reasoning, offer exciting possibilities in medicine. Building on these core strengths of Gemini, we introduce Med-Gemini, a family of highly capable multimodal models that are specialized in medicine with the ability to seamlessly use web search, and that can be efficiently tailored to novel modalities using custom encoders. We evaluate Med-Gemini on 14 medical benchmarks, establishing new state-of-the-art (SoTA) performance on 10 of them, and surpass the GPT-4 model family on every benchmark where a direct comparison is viable, often by a wide margin. On the popular MedQA (USMLE) benchmark, our best-performing Med-Gemini model achieves SoTA performance of 91.1% accuracy, using a novel uncertainty-guided search strategy. On 7 multimodal benchmarks including NEJM Image Challenges and MMMU (health & medicine), Med-Gemini improves over GPT-4V by an average relative margin of 44.5%. We demonstrate the effectiveness of Med-Gemini's long-context capabilities through SoTA performance on a needle-in-a-haystack retrieval task from long de-identified health records and medical video question answering, surpassing prior bespoke methods using only in-context learning. Finally, Med-Gemini's performance suggests real-world utility by surpassing human experts on tasks such as medical text summarization, alongside demonstrations of promising potential for multimodal medical dialogue, medical research and education. Taken together, our results offer compelling evidence for Med-Gemini's potential, although further rigorous evaluation will be crucial before real-world deployment in this safety-critical domain.
Les articles autour de lui, par date et par proximité
Plus haut, c’est plus proche de cet article ; à gauche, c’est plus ancien. Les traits relient les articles proches entre eux : les groupes de traits sont des sous-thèmes. Sélectionnez un point pour voir pourquoi il est là. Retour au résumé de cet article
Les articles autour de celui-ci
Classés selon la proximité de leur titre et de leur résumé avec ceux de cet article. Sélectionnez-en pour les lire ensemble.
Les articles les plus proches
-
MedGemma Technical ReportAndrew B. Sellergren, S. Kazemzadeh, Tiam Jaroensri et al. · 2025 · cité 532 fois · Pas encore résumé
Résumé scientifique
Artificial intelligence (AI) has significant potential in healthcare applications, but its training and deployment faces challenges due to healthcare's diverse data, complex tasks, and the need to preserve privacy. Foundation models that perform well on medical tasks and require less task-specific tuning data are critical to accelerate the development of healthcare AI applications. We introduce MedGemma, a collection of medical vision-language foundation models based on Gemma 3 4B and 27B. MedGemma demonstrates advanced medical understanding and reasoning on images and text, significantly exceeding the performance of similar-sized generative models and approaching the performance of task-spe…
-
Application of Gemini in Public Health Amid the Artificial Intelligence EraMoosa Tatar, Soheila Farokhi, Arash Azizian Foumani et al. · 2026 · IEEE Engineering Management Review · cité 4 fois · Pas encore résumé
Résumé scientifique
In December 2023, Google DeepMind unveiled its large language model, Gemini, which is shown to be capable of understanding and processing multimodal information. This article explores the applications of Gemini in public health. Unlike traditional models, Gemini integrates and analyzes information through diverse modalities (text, image, video, etc.) fostering new applications. Notably, Google Gemini is presented as the first model to surpass human experts on the Massive Multitask Language Understanding benchmark, which combines multiple subjects including science, technology, engineering, and mathematics, humanities, and others to test the knowledge and problem-solving abilities of artifici…
-
Potential of multimodal large language models for data mining of medical images and free-text reportsYutong Zhang, Yi Pan, Tianyang Zhong et al. · 2024 · Meta-Radiology · cité 52 fois · Pas encore résumé
Résumé scientifique
Medical images and radiology reports are essential for physicians to diagnose medical conditions . However, the vast diversity and cross-source heterogeneity inherent in these data have posed significant challenges to the generalizability of current data-mining methods for clinical decision-making. Recently, multimodal large language models (MLLMs), especially Gemini-Vision-series (Gemini) and GPT-4-series (GPT-4) models, have revolutionized numerous domains, significantly impacting the medical field. In this study, we conducted a detailed evaluation of the performance of the Gemini series models (including Gemini-1.0-Pro-Vision, Gemini-1.5-Pro, and Gemini-1.5-Flash) and GPT series models (i…
-
An open vision-language model for diverse medical applicationsAndrew B. Sellergren, S. Kazemzadeh, Fereshteh Mahvar et al. · 2026 · Nature Medicine · Pas encore résumé
Résumé scientifique
Artificial intelligence has high potential for impact in healthcare applications, but its training and deployment are challenging due to diverse data, a complex spectrum of possible tasks and important privacy needs. High-performing foundation models that enable data-efficient fine-tuning for diverse downstream tasks can meaningfully accelerate development in this domain. We introduce MedGemma, a collection of medical vision-language foundation models based on Gemma 3. MedGemma demonstrates advanced medical understanding and reasoning across images and text and multiple medical imaging domains, exceeding the performance of similarly sized generative models while maintaining the general capab…
-
Towards Generalist Biomedical AITao Tu, Shekoofeh Azizi, Danny Driess et al. · 2023 · Computation and Language · preprint, non relu par les pairs · cité 647 fois · Pas encore résumé
Résumé scientifique
Medicine is inherently multimodal, with rich data modalities spanning text, imaging, genomics, and more. Generalist biomedical artificial intelligence (AI) systems that flexibly encode, integrate, and interpret this data at scale can potentially enable impactful applications ranging from scientific discovery to care delivery. To enable the development of these models, we first curate MultiMedBench, a new multimodal biomedical benchmark. MultiMedBench encompasses 14 diverse tasks such as medical question answering, mammography and dermatology image interpretation, radiology report generation and summarization, and genomic variant calling. We then introduce Med-PaLM Multimodal (Med-PaLM M), ou…
-
A Modular Multi-Agent Multimodal AI Framework Integrating Vision–Language Models and IoT Data for Explainable Clinical Decision SupportEt-Tousy Jamal, Chtara Malek, Zyane Abdellah · 2026 · 2026 6th International Conference on Innovative Research in Applied Science, Engineering and Technology (IRASET) · Pas encore résumé
Résumé scientifique
Clinical decision support systems now provide more options because of the integration of medical imaging, clinical text, and contextual data utilizing contemporary visionlanguage models (VLMs). Performance, explainability, accessibility, and regulatory compliance are all trade-offs in any model. In this study, we assess four VLMs using multimodal medical benchmarks: MedGemma-4B-IT, LLaVA-Med, MedVLMR1, and GPT-4V. Because of its high multimodal capabilities, reasoning ability, integration of external knowledge, and modifiability, we have chosen MedGemma-4B-IT as the best opensource choice. MedGemma-4B-IT provides competitive baseline performance on standard medical VQA benchmarks by building…
-
Towards conversational artificial intelligence for disease managementValentin Liévin, Anil Palepu, Wei-Hung Weng et al. · 2026 · Nature · cité 18 fois · Pas encore résumé
Résumé scientifique
Although large language models have shown promise in diagnostic dialogue1, their capabilities for effective management reasoning, including disease progression, therapeutic response and safe medication prescription, have remained underexplored. We have advanced the previously demonstrated diagnostic capabilities of the Articulate Medical Intelligence Explorer (AMIE)1, 2–3 using a new large-language-model-based agentic system optimized for multivisit clinical management and dialogue. To ground the reasoning of AMIE in authoritative clinical knowledge, we leveraged the long-context capabilities of Gemini4, combining in-context retrieval with structured reasoning to align its output with up-to-…
-
Meissa: Multi-modal Medical Agentic IntelligenceYi-Xiong Chen, Xin-Yi Bai, Yue Pan et al. · 2026 · arXiv.org · preprint, non relu par les pairs · cité 6 fois · Pas encore résumé
Résumé scientifique
Multi-modal large language models (MM-LLMs) have shown strong performance in medical image understanding and clinical reasoning. Recent medical agent systems extend them with tool use and multi-agent collaboration, enabling complex decision-making. However, these systems rely almost entirely on frontier models (e.g., GPT), whose API-based deployment incurs high cost, high latency, and privacy risks that conflict with on-premise clinical requirements. We present Meissa, a lightweight 4B-parameter medical MM-LLM that brings agentic capability offline. Instead of imitating static answers, Meissa learns both when to engage external interaction (strategy selection) and how to execute multi-step i…
Les articles qui le citent
-
Large Language Models lack essential metacognition for reliable medical reasoningMaxime Griot, Coralie Hemptinne, Jean M. Vanderdonckt et al. · 2025 · Nature Communications · cité 126 fois · Pas encore résumé
Résumé scientifique
Large Language Models have demonstrated expert-level accuracy on medical board examinations, suggesting potential for clinical decision support systems. However, their metacognitive abilities, crucial for medical decision-making, remain largely unexplored. To address this gap, we developed MetaMedQA, a benchmark incorporating confidence scores and metacognitive tasks into multiple-choice medical questions. We evaluated twelve models on dimensions including confidence-based accuracy, missing answer recall, and unknown recall. Despite high accuracy on multiple-choice questions, our study revealed significant metacognitive deficiencies across all tested models. Models consistently failed to rec…
-
Multimodal generative AI for interpreting 3D medical images and videosJung-Oh Lee, Hong-Yu Zhou, T. Berzin et al. · 2025 · npj Digital Medicine · cité 27 fois · Pas encore résumé
Résumé scientifique
This perspective proposes adapting video-text generative AI to 3D medical imaging (CT/MRI) and medical videos (endoscopy/laparoscopy) by treating 3D images as videos. The approach leverages modern video models to analyze multiple sequences simultaneously and provide real-time AI assistance during procedures. The paper examines medical imaging’s unique characteristics (synergistic information, metadata, and world model), outlines applications in automated reporting, case retrieval, and education, and addresses challenges of limited datasets, benchmarks, and specialized training.
-
Advancing conversational diagnostic AI with multimodal reasoningKhaled Saab, Chunjong Park, Tim Strother et al. · 2026 · Nature Medicine · cité 9 fois · Pas encore résumé
Résumé scientifique
Real-world clinical practice is inherently multimodal, relying on the synthesis of patient history with visual information such as medical imagery and clinical documents. Although large language models (LLMs) have shown promise in diagnostic dialogue, their evaluation has been largely restricted to text-only interactions, failing to capture the complexity of modern remote care delivery. Here we introduce a multimodal extension of the Articulate Medical Intelligence Explorer (multimodal AMIE), capable of gathering, interpreting and reasoning about multimodal data within a diagnostic conversation. To achieve this, we developed a state-aware dialogue framework that dynamically guides history-ta…
-
Agentic Medical Knowledge Graphs Enhance Medical Question Answering: Bridging the Gap Between LLMs and Evolving Medical KnowledgeMohammad R. Rezaei, Reza Saadati Fard, J. Parker et al. · 2025 · Conference on Empirical Methods in Natural Language Processing · cité 29 fois · Pas encore résumé
Résumé scientifique
Large Language Models (LLMs) have significantly advanced medical question-answering by leveraging extensive clinical data and medical literature. However, the rapid evolution of medical knowledge and the labor-intensive process of manually updating domain-specific resources pose challenges to the reliability of these systems. To address this, we introduce Agentic Medical Graph-RAG (AMG-RAG), a comprehensive framework that automates the construction and continuous updating of medical knowledge graphs, integrates reasoning, and retrieves current external evidence, such as PubMed and WikiSearch. By dynamically linking new findings and complex medical concepts, AMG-RAG not only improves accuracy…
-
Foundation Model for Medical Imaging: A Comprehensive ReviewLi-Cheng Jiao, Jingyi Yang, Rui-Yang Li et al. · 2026 · IEEE Transactions on Artificial Intelligence · cité 8 fois · Pas encore résumé
Résumé scientifique
The rapid development of highly flexible and reusable foundation models has introduced a new paradigm shift in the field of medical artificial intelligence (AI). Medical foundation models (MFMs) are built upon large-scale, diverse datasets and utilize multistage training strategies to thoroughly understand task representations, fitting the differences between various patterns, thereby achieving contextual reasoning abilities and broad applicability. MFMs can perform zero-shot inference on a variety of downstream tasks and also enhance the understanding of new tasks with task-specific prompts, without the need for extensive labeled data and retraining. To achieve more transparent and universa…
Aller plus loin
Comment ces articles sont-ils trouvés ?
Nous comparons le titre et le résumé de cet article avec ceux de tous les articles résumés sur ce site, et avec les articles que Semantic Scholar recommande, ceux qu’il cite et ceux qui le citent (pour un article très cité, ses citations les plus récentes et, depuis OpenAlex, les plus citées). Parmi les travaux antérieurs et les suites, les articles qui s’utilisent en profondeur, une méthode qu’il utilise et les suites très citées remontent un peu. Les niveaux (très proche, proche, apparenté) viennent de la similarité des textes, pas d’un jugement de qualité ; * signale un article sans résumé, comparé par son titre. La phrase sous un article est écrite par une IA à partir des deux résumés : vérifiez-la dans les articles. Un article jugé hors sujet par trois lecteurs quitte la liste.
Mis à jour le 3 novembre 2026.