# Can LLMs' Tuning Methods Work in Medical Multimodal Domain?

**Authors:** Jiawei Chen, Yue Jiang, Dingkang Yang, Mingcheng Li, Jinjie Wei, Ziyun Qian, Lihua Zhang  
**arXiv:** [2403.06407](https://arxiv.org/abs/2403.06407)  
**Date:** 2024-03-11  
**Summary page:** https://www.summarizepaper.com/en/paper/2403.06407v1/

## Key points

- Large language models (LLMs) have exceptional ability to comprehend world knowledge
- Tailoring LLMs to specific subfields requires precise adjustments due to their vast scale
- Traditional global fine-tuning methods for large models are computationally expensive and may impact generalization capabilities
- Parameters-Efficient Fine-Tuning (PEFT) methods have emerged as a solution, showing success in LLMs and LVLMs
- Fine-tuning a medical Vision-Language Pretrained (VLP) model is crucial for customizing it for specific tasks in the medical domain
- Research explores transferring fine-tuning methods from large models to medical field for enhanced transfer learning efficiency
- Extensive experiments conducted on how fine-tuning methods affect multimodal models in the medical domain at training data and model structure levels
- Study aims to optimize training costs associated with VLMs in healthcare fields by developing efficient ways to fine-tune medical VLP models
- Code and dataset used in research will be made available for further exploration and validation

## Summary

In the realm of large language models (LLMs), their exceptional ability to comprehend world knowledge is undeniable. However, tailoring these models to specific subfields necessitates precise adjustments that can be challenging due to the vast scale of the models. Traditional global fine-tuning methods for large models often come with a hefty computational cost and may impact their generalization capabilities. To tackle this issue, a new wave of innovative Parameters-Efficient Fine-Tuning (PEFT) methods has emerged, showcasing remarkable success in both LLMs and Large Vision-Language Models (LVLMs). Within the medical domain, fine-tuning a medical Vision-Language Pretrained (VLP) model is crucial for customizing it to perform specific tasks effectively. The question arises: can the fine-tuning methods developed for large models be seamlessly transferred to the medical field to enhance transfer learning efficiency? This paper delves into the intricate details of fine-tuning methods for LLMs and conducts extensive experiments to explore how these methods impact existing multimodal models in the medical domain at both the training data level and model structure level. Through rigorous experimentation, this research sheds light on the diverse impacts of fine-tuning methods designed for large models on medical Vision-Language Models (VLMs). By identifying and developing efficient ways to fine-tune medical VLP models, this study aims to guide researchers in optimizing training costs associated with VLMs and ultimately foster broader applications of these advanced models within healthcare fields. Upon acceptance, the code and dataset utilized in this research will be made available for further exploration and validation.