# Dynamic Q&A of Clinical Documents with Large Language Models

**Authors:** Ran Elgedawy, Sudarshan Srinivasan, Ioana Danciu  
**arXiv:** [2401.10733](https://arxiv.org/abs/2401.10733)  
**Date:** 2024-01-19  
**Summary page:** https://www.summarizepaper.com/en/paper/2401.10733v1/

## Key points

- Study focuses on developing a natural language interface using large language models (LLMs) for dynamic question-answering on clinical notes in EHRs
- Introduces a chatbot powered by Langchain and transformer-based LLMs for querying in natural language and receiving relevant answers from clinical notes
- Preprocessing steps include extraction of relevant fields to streamline the dataset and focus on pertinent information
- Single document evaluations using synthetic data for comparisons with OpenAI's GPT-4, multi-document evaluations emphasize performance metrics and inference time
- Model optimization strategies explored to enhance Wizard Vicuna's inference speed, focusing on quantization techniques to reduce computational overhead
- Limitations encountered include GPU RAM constraints, fine-tuning overheads, and evaluation challenges
- Future work includes application containerization for enhanced deployment and scalability, refining evaluation strategies for consistent results, expanding dataset scope for comprehensive testing
- Addressing challenges such as model hallucinations and diverse medical case evaluations crucial for unlocking full value of clinical notes in healthcare settings

## Summary

This study focuses on developing a natural language interface using large language models (LLMs) to enable dynamic question-answering on clinical notes stored in electronic health records (EHRs). The increasing volume and complexity of clinical notes make manual extraction challenging, highlighting the need for innovative solutions. To address this issue, the research introduces a chatbot powered by Langchain and transformer-based LLMs that allows users to query in natural language and receive relevant answers from clinical notes. The study delves into the preprocessing steps undertaken, including the extraction of relevant fields to streamline the dataset and focus on pertinent information. Single document evaluations using synthetic data closely mirroring the structure of the MIMIC dataset enable comparisons between model outputs and OpenAI's GPT-4. Multi-document evaluations further highlight performance metrics, with a particular emphasis on inference time as a key factor. Additionally, model optimization strategies are explored to enhance Wizard Vicuna's inference speed without compromising accuracy, with a focus on quantization techniques to reduce computational overhead. However, limitations encountered during the research include GPU RAM constraints, fine-tuning overheads, and evaluation challenges. Future work includes application containerization for enhanced deployment and scalability, refining evaluation strategies for consistent results across settings, and expanding dataset scope for comprehensive testing. Overall,, highlighting potential advancements in AI-driven clinical decision-making. Addressing challenges such as model hallucinations and diverse medical case evaluations remains crucial for unlocking the full value of clinical notes in healthcare settings.