# RAG vs Fine-tuning: Pipelines, Tradeoffs, and a Case Study on Agriculture

**Authors:** Aman Gupta, Anup Shirgaonkar, Angels de Luis Balaguer, Bruno Silva, Daniel Holstein, Dawei Li, Jennifer Marsman, Leonardo O. Nunes, Mahsa Rouzbahman, Morris Sharp, Nick Mecklenburg, Rafael Padilha, Ranveer Chandra, Renato Luiz de Freitas Cunha, Roberto de M. Estevão Filho, Ryan Tsang, Sara Malvar, Swati Sharma, Todd Hendry, Vijay Aski  
**arXiv:** [2401.08406](https://arxiv.org/abs/2401.08406)  
**Date:** 2024-01-16  
**Summary page:** https://www.summarizepaper.com/en/paper/2401.08406v1/

## Key points

- The paper explores two approaches for incorporating proprietary and domain-specific data into applications of Large Language Models (LLMs): Retrieval-Augmented Generation (RAG) and Fine-Tuning.
- RAG involves augmenting the prompt with external data, while Fine-Tuning incorporates additional knowledge directly into the model.
- The authors propose a pipeline that combines fine-tuning and RAG techniques for multiple popular LLMs, including Llama2-13B, GPT-3.5, and GPT-4.
- The pipeline consists of stages such as extracting information from PDFs, generating questions and answers, using them for fine-tuning, and leveraging GPT-4 to evaluate the results.
- Metrics are introduced to assess the performance of different stages in the RAG and fine-tuning pipeline.
- An in-depth study on an agricultural dataset demonstrates how these techniques can be applied to provide location-specific insights to farmers.
- Results show that the proposed dataset generation pipeline effectively captures geographic-specific knowledge.
- Using both RAG and fine-tuning together leads to increased accuracy compared to using either approach alone.
- A specific experiment shows that the fine-tuned model leverages information from various geographies to answer specific questions more accurately.
- The research showcases how systems built using LLMs can incorporate industry-specific knowledge across critical dimensions.
- The findings open up possibilities for further applications of LLMs in other industrial domains where AI has not seen much penetration yet.

## Summary

In this paper, the authors explore two common approaches for incorporating proprietary and domain-specific data into applications of Large Language Models (LLMs): Retrieval-Augmented Generation (RAG) and Fine-Tuning. RAG involves augmenting the prompt with external data, while Fine-Tuning incorporates additional knowledge directly into the model. However, the advantages and disadvantages of both methods are not well understood. To address this gap, the authors propose a pipeline that combines fine-tuning and RAG techniques for multiple popular LLMs, including Llama2-13B, GPT-3.5, and GPT-4. The pipeline consists of several stages such as extracting information from PDFs, generating questions and answers, using them for fine-tuning, and leveraging GPT-4 to evaluate the results. The authors also introduce metrics to assess the performance of different stages in the RAG and fine-tuning pipeline. They conduct an in-depth study on an agricultural dataset to demonstrate how these techniques can be applied in a potentially disruptive application: providing location-specific insights to farmers. The results show that the proposed dataset generation pipeline effectively captures geographic-specific knowledge. Additionally, they highlight both quantitative and qualitative benefits of using RAG and fine-tuning together. Fine-tuning alone leads to an accuracy increase of over 6 percentage points (p.p. ), which is further enhanced by 5 p.p. when combined with RAG. Furthermore,a specific experiment demonstrates that the fine-tuned model leverages information from various geographies to answer specific questions more accurately,increasing answer similarity from 47% to 72%. Overall,this research showcases how systems built using LLMs can be adapted to incorporate industry-specific knowledge across critical dimensions.The findings pave the way for further applications of LLMs in other industrial domains where AI has not seen much penetration yet.