Summarize Paper logo SummarizePaper.com

AI summary · arXiv: 2610.41437v1 · cs.HC

A prospective randomized study of patient-centered AI for clinical visit preparation

Xincheng Tan, Matthew Thompson, Xiang Ji, Angelica Willis, Will Vaughan, Roma Ruparel, Michael A. Sanchez, Sean Chang,
and 28 more authorsJoseph Xu, Bhavna Daryani, Preeti Singh, Pinal Bavishi, Kejia Chen, Fan Zhang, Priya Gupta, Ernest Lim, Christina Chen, Bob Lou, Dimitrios Antos, Zach Wasson, Ray Luo, Fadi Yousif, Florence Thng, Mark Malhotra, Joseph Breda, Daniel McDuff, Kavi Goel, Katherine Chou, Dale R. Webster, Avinatan Hassidim, Yossi Matias, Po-Hsuan Cameron Chen, Yun Liu, Mike Schaekermann, Quang Duong, Jackie M. Barr
Original paper Submitted to arXiv on 30 Oct. 2026 50 pages (16 main text, 34 appendix), 6 figures License: CC BY 4.0 Also in: fr

How to read this summary? Illustrated guide

The essentials

Structured AI visit preparation improved patient-reported readiness and clinician-rated comprehensiveness and personalization versus a minimally prompted chatbot, without significant differences in rated safety.

Question

Does a structured conversational AI workflow help patients prepare for clinical visits better than a minimally prompted chatbot using the same underlying model? The study assessed both patient-reported experience and clinician-rated quality.

Method

In a prospective online study, 5,960 U.S. adults were randomized 1:2:2 to a control chatbot or standard and concise structured preparation agents, all using Gemini 2.5 Flash in the Google Health app. The modified intention-to-treat analysis included 802 participants contributing 866 health-related conversations, with the structured arms pooled for the main comparison. Outcomes included 460 immediate patient surveys, blinded clinician reviews of 272 completed sessions, and 95 exploratory 14-day follow-up responses.

Findings

  • Result Structured preparation increased clinician-rated comprehensiveness from 50.6% to 67.2% and patient alignment from 90.4% to 98.9%.
    See the passage in the paper
    Aggregated across all steps using a strict overall metric (all ratings positive), conversations and summaries generated by the visit preparation agents were significantly more likely to be comprehensive (67.2% vs. 50.6%; $P<0.05$) and closely aligned with the patient’s clinical context and goals (98.9% vs. 90.4%; $P<0.01$).
  • Result Clinician-rated medical safety and appropriateness were high in both groups, without significant between-group differences.
    See the passage in the paper
    Based on blinded clinician evaluation, we found that high baseline clinical reliability was maintained across both arms, with no significant differences in medical safety or clinical appropriateness (Figure 4, top).
  • Result Patients receiving structured preparation more often felt prepared for their visit and confident about what to ask their clinician.
    See the passage in the paper
    In the immediate post-survey, 73.8% of the visit preparation agent users reported feeling prepared for their upcoming visit, compared with 57.0% of Control users ($P<0.01$), a 16.8-percentage-point increase. These participants were also more confident in knowing what to ask their clinician (81.4% vs. 67.1%; $P<0.01$) and in making the most of the visit (81.4% vs. 69.6%; $P<0.05$).
  • Result Patient survey responses did not significantly correlate with clinician ratings after correction for multiple comparisons.
    See the passage in the paper
    After correcting for multiple comparisons, we found no significant correlation between any survey item and the clinical ratings of any preparation step, across all clinical dimensions ().

Limitations

  • Restricting clinical reviews to completed sessions may overestimate conversation quality through survivorship bias.
    See the passage in the paper
    Because clinical evaluation was restricted to sessions that completed the flow, it may also be subject to survivorship bias: if some participants withdrew prematurely because of agent shortcomings (e.g., repetitive questioning or unaddressed concerns), clinical ratings may overestimate the quality of all conversations.
  • Limited follow-up participation and few intervening clinical visits constrained assessment of benefits during actual encounters.
    See the passage in the paper
    Another limitation is that our 14-day follow-up survey was intended to capture whether the effectiveness of visit preparation carried over to the actual encounter, but a relatively limited number of participants responded and even fewer had an encounter in that window.
  • The self-selected Google Health app cohort may not represent the broader patient population.
    See the passage in the paper
    Because the participants were Android users who actively used the Google Health app and opted into a research study, they were likely more technologically engaged and health literate than the general patient population.

Why it matters

The authors argue that structured interaction design, rather than model capability alone, can improve patient-facing AI visit preparation. They also argue that evaluation requires both independent clinical review and patient-reported outcomes, while benefits for actual encounters and downstream health outcomes remain to be established.

Written by AI from the paper. Passages are checked word for word against the text we analysed; the rest may contain errors.

AI-generated keywords: Conversational AI Appointment preparation Structured prompting Patient readiness Randomized study

Read further

  • Comprehensive Summary
  • Layman's Summary
  • Blog article
  • Original abstract
This paper examines whether a structured conversational AI tool helps patients prepare for medical appointments more effectively than a minimally prompted chatbot using the same underlying language model. Rather than diagnosing conditions or recommending treatment, the tool gathers relevant context, explains what to expect during a visit, provides general health information, helps prioritize questions, and generates a preparation summary. The central question is whether explicitly organizing these activities improves patients’ perceived readiness and the clinical quality of their preparation. The researchers conducted a prospective, three-arm randomized study through the Google Health Android app in the United States between November 2025 and May 2026. Participants were assigned in a 1:2:2 ratio to a control chatbot, a standard structured preparation agent, or a concise structured agent. All three used Gemini 2.5 Flash and the same interface; only their system prompts differed. The two structured arms were combined for the primary analysis. Eligible participants were English-speaking adults; exclusions included known mental health disorders, sexually transmitted infections, and pregnancy. Of the 5,960 people randomized, 802 contributed 866 eligible health-related conversations. Immediate patient surveys covered 460 conversations, and independent clinicians evaluated 272 completed sessions. Positive ratings were defined as scores of 4 or 5 on five-point scales. The main comparisons were adjusted for multiple testing. Structured preparation improved several clinician-rated measures. Across applicable preparation steps, 67.2% of structured-agent sessions received positive comprehensiveness ratings, compared with 50.6% of control sessions—a difference of 16.6 percentage points. Positive patient-alignment ratings were 98.9% versus 90.4%, a difference of 8.5 percentage points. Both differences were statistically significant. Comprehensiveness improved significantly in context gathering, expectation setting, health information, and summarization, but not in agenda generation. Safety and appropriateness ratings were high, with no statistically significant differences between groups. Positive safety ratings were 93.7% for structured preparation and 96.4% for control, while positive appropriateness ratings were 93.7% and 91.6%, respectively. The proportion of sessions meeting every reliability and utility criterion increased from 50.6% to 64.6%. Patients also reported better experiences with structured preparation. The proportion who felt prepared for their appointment increased from 57.0% in the control group to 73.8% in the structured group, an improvement of 16.8 percentage points. Confidence about what to ask increased from 67.1% to 81.4%, and confidence in making the most of the visit increased from 69.6% to 81.4%. Satisfaction was 74.0% versus 62.0%, and willingness to reuse the tool was 76.6% versus 65.8%. All five immediate survey differences were statistically significant. Follow-up evidence, however, was limited. Only 95 conversations had 14-day survey responses, and just 59 respondents reported attending an appointment. Neither alignment between expected and actual visits nor use of prepared questions differed significantly between groups. After correction for multiple testing, clinician ratings and patient reports showed no significant correlations. The findings suggest that structured prompting can improve the breadth and personalization of appointment preparation, as well as patients’ perceived readiness, without a detected reduction in clinician-rated safety. They do not establish improved clinical outcomes or demonstrate equivalent safety. Important limitations include substantial nonparticipation after randomization, incomplete conversations, and selective survey response. Clinical review included only completed sessions, potentially overestimating overall quality, as 42.3% of eligible conversations ended before completion. The small follow-up sample prevented firm conclusions about actual appointments. Generalizability is also limited by recruitment of English-speaking, technologically engaged Android users and the exclusion of several clinical populations. The Alphabet-funded study supports further investigation, particularly research assessing clinician perspectives, communication during appointments, and downstream health outcomes.
Created on 02 Nov. 2026

Was this synthesis useful?

Score: 0

Why do we need votes?

Votes are used to determine whether we need to re-run our summarizing tools. If the count reaches -10, our tools can be restarted.

Ask questions about this paper to our AI assistant

How to chat with this paper? Illustrated guide

Compare with the PDF

You can also chat with multiple papers at once here.

Figures and plots

Ask about a figure by number or use its question button to request visual analysis. Up to two figures and four image panels per question. Values read from plots are approximate; check the original data.
AI assistant instructions?

Tools for this paper

Explain it differently 3 free rewrites left today
Save to a collection Group papers in collections to compare them, chat with them and export them. Create a free account Log in
Export BibTeX · RIS (Zotero, Mendeley, EndNote) · Obsidian (Markdown) · Notion (Markdown)

Advanced tools Premium

For researchers who follow a field closely: citation alerts, journal-club slides, version comparison and Zotero export. Discover Premium

Zotero
Citation alerts Citation alerts guide
Who works on it? The people and institutions who publish on its topic
Weekly letter Presented in "Computing beyond AI this week" of 2 November 2026. Computing beyond AI this week: Better algorithms, smaller machines, harder tests
Journal club Premium+ Slides to present it Propose for the journal club
Versions What changed between versions
Talk about it in public Premium+ Press kit: release, questions, hype check Press kit guide

Papers around this one

See them on a map

Ranked by how close their title and abstract are to this paper’s. Select some to read them together.

Closest papers

  1. Very close Another approach to the problem
    Towards Conversational Diagnostic AI
    Tao Tu, Anil Palepu, Mike Schaekermann et al. · 2024 · Artificial Intelligence · preprint, not peer reviewed · cited 167 times · AI summary available

    AMIE trains an LLM for diagnostic dialogue, whereas this study tests a structured workflow for preparing patients for clinical visits.

    Abstract

    At the heart of medicine lies the physician-patient dialogue, where skillful history-taking paves the way for accurate diagnosis, effective management, and enduring trust. Artificial Intelligence (AI) systems capable of diagnostic dialogue could increase accessibility, consistency, and quality of care. However, approximating clinicians' expertise is an outstanding grand challenge. Here, we introduce AMIE (Articulate Medical Intelligence Explorer), a Large Language Model (LLM) based AI system optimized for diagnostic dialogue. AMIE uses a novel self-play based simulated environment with automated feedback mechanisms for scaling learning across diverse disease conditions, specialties, and cont…

  2. Close Review
    Leveraging Large Language Models for Patient Engagement: The Power of Conversational AI in Digital Health
    Bo Wen, Raquel Norel, Julia Liu et al. · 2024 · Artificial Intelligence · cited 31 times · AI summary available

    This overview surveys LLM applications for patient engagement, including conversational analysis, chatbots, summarization, and patient engagement systems.

    Abstract

    The rapid advancements in large language models (LLMs) have opened up new opportunities for transforming patient engagement in healthcare through conversational AI. This paper presents an overview of the current landscape of LLMs in healthcare, specifically focusing on their applications in analyzing and generating conversations for improved patient engagement. We showcase the power of LLMs in handling unstructured conversational data through four case studies: (1) analyzing mental health discussions on Reddit, (2) developing a personalized chatbot for cognitive engagement in seniors, (3) summarizing medical conversation datasets, and (4) designing an AI-powered patient engagement system. Th…

  3. Close Another approach to the problem
    ProAI: Proactive Multi-Agent Conversational AI with Structured Knowledge Base for Psychiatric Diagnosis
    Yuqi Wu, Guangya Wan, Jingjing Li et al. · 2025 · Artificial Intelligence · cited 9 times · AI summary available

    ProAI uses proactive, knowledge-guided multi-agent dialogue for psychiatric diagnosis, rather than visit preparation.

    Abstract

    Most LLM-driven conversational AI systems operate reactively, responding to user prompts without guiding the interaction. Most LLM-driven conversational AI systems operate reactively, responding to user prompts without guiding the interaction. However, many real-world applications-such as psychiatric diagnosis, consulting, and interviews-require AI to take a proactive role, asking the right questions and steering conversations toward specific objectives. Using mental health differential diagnosis as an application context, we introduce ProAI, a goal-oriented, proactive conversational AI framework. ProAI integrates structured knowledge-guided memory, multi-agent proactive reasoning, and a mul…

  4. Close Broader context
    AI as a Medical Ally: Evaluating ChatGPT's Usage and Impact in Indian Healthcare
    Aryaman Raina, Prateek Mishra, Harshit goyal et al. · 2024 · Human-Computer Interaction · preprint, not peer reviewed · cited 3 times · AI summary available

    This study surveys Indian users and clinicians about ChatGPT in healthcare, including concerns about reliability and trust.

    Abstract

    This study investigates the integration and impact of Large Language Models (LLMs), like ChatGPT, in India's healthcare sector. Our research employs a dual approach, engaging both general users and medical professionals through surveys and interviews respectively. Our findings reveal that healthcare professionals value ChatGPT in medical education and preliminary clinical settings, but exercise caution due to concerns about reliability, privacy, and the need for cross-verification with medical references. General users show a preference for AI interactions in healthcare, but concerns regarding accuracy and trust persist. The study underscores the need for these technologies to complement, no…

  5. Close Broader context
    Performance of ChatGPT on USMLE: Unlocking the Potential of Large Language Models for AI-Assisted Medical Education
    Prabin Sharma, Kisan Thapa, Dikshya Thapa et al. · 2023 · Computers and Society · preprint, not peer reviewed · cited 28 times · Not summarized yet

    This study evaluates ChatGPT's answers to medical and USMLE questions, rather than its role in preparing patients for visits.

    Abstract

    Artificial intelligence is gaining traction in more ways than ever before. The popularity of language models and AI-based businesses has soared since ChatGPT was made available to the general public via OpenAI. It is becoming increasingly common for people to use ChatGPT both professionally and personally. Considering the widespread use of ChatGPT and the reliance people place on it, this study determined how reliable ChatGPT can be for answering complex medical and clinical questions. Harvard University gross anatomy along with the United States Medical Licensing Examination (USMLE) questionnaire were used to accomplish the objective. The paper evaluated the obtained results using a 2-way A…

  6. Close Another approach to the problem
    ChatDoctor: A Medical Chat Model Fine-Tuned on a Large Language Model Meta-AI (LLaMA) Using Medical Domain Knowledge
    Yunxiang Li, Zihan Li, Kai Zhang et al. · 2023 · Computation and Language · cited 812 times · AI summary available

    ChatDoctor fine-tunes LLaMA on patient-doctor dialogues and adds retrieval to generate medical advice, rather than guiding visit preparation.

    Abstract

    The primary aim of this research was to address the limitations observed in the medical knowledge of prevalent large language models (LLMs) such as ChatGPT, by creating a specialized language model with enhanced accuracy in medical advice. We achieved this by adapting and refining the large language model meta-AI (LLaMA) using a large dataset of 100,000 patient-doctor dialogues sourced from a widely used online medical consultation platform. These conversations were cleaned and anonymized to respect privacy concerns. In addition to the model refinement, we incorporated a self-directed information retrieval mechanism, allowing the model to access and utilize real-time information from online …

  7. Close Same method, another problem
    CoachAI: A Conversational Agent Assisted Health Coaching Platform
    Ahmed Fadhil, Gianluca Schiavo, Yunlong Wang · 2019 · Artificial Intelligence · preprint, not peer reviewed · cited 10 times · Not summarized yet

    CoachAI uses a conversational healthcare agent to guide users, applying chatbot-based interaction to health coaching rather than visit preparation.

    Abstract

    Poor lifestyle represents a health risk factor and is the leading cause of morbidity and chronic conditions. The impact of poor lifestyle can be significantly altered by individual behavior change. Although the current shift in healthcare towards a long lasting modifiable behavior, however, with increasing caregiver workload and individuals' continuous needs of care, there is a need to ease caregiver's work while ensuring continuous interaction with users. This paper describes the design and validation of CoachAI, a conversational agent assisted health coaching system to support health intervention delivery to individuals and groups. CoachAI instantiates a text based healthcare chatbot syste…

  8. Close Same method, another problem
    Psy-LLM: Scaling up Global Mental Health Psychological Services with AI-based Large Language Models
    Tin Lai, Yukun Shi, Zicong Du et al. · 2023 · Computation and Language · preprint, not peer reviewed · cited 86 times · AI summary available

    Psy-LLM applies LLM-based question answering to psychological consultations, rather than helping patients prepare for clinical visits.

    Abstract

    The demand for psychological counselling has grown significantly in recent years, particularly with the global outbreak of COVID-19, which has heightened the need for timely and professional mental health support. Online psychological counselling has emerged as the predominant mode of providing services in response to this demand. In this study, we propose the Psy-LLM framework, an AI-based assistive tool leveraging Large Language Models (LLMs) for question-answering in psychological consultation settings to ease the demand for mental health professions. Our framework combines pre-trained LLMs with real-world professional Q\&A from psychologists and extensively crawled psychological articles…

Go further

How are these papers found?

We compare the title and abstract of this paper with those of every paper summarized on this site, and with the papers Semantic Scholar recommends, those it cites and those citing it (for a much-cited paper, its most recent citations and, from OpenAlex, the most cited ones). Among the prior work and the follow-ups, the papers that use each other in depth, a method it uses and the much-cited follow-ups come a little higher. The levels (very close, close, related) come from the similarity of the texts, not from a judgment of quality; * marks a paper without an abstract, compared by its title. The sentence under a paper is written by an AI from the two abstracts: check it in the papers. A paper marked not relevant by three readers leaves the list.

Updated on 2 November 2026.

Disclaimer: The AI-based summarization tool and virtual assistant provided on this website may not always provide accurate and complete summaries or responses. We encourage you to carefully review and evaluate the generated content to ensure its quality and relevance to your needs.