# RAG vs Fine-tuning: Pipelines, Tradeoffs, and a Case Study on Agriculture

**Authors:** Angels Balaguer, Vinamra Benara, Renato Luiz de Freitas Cunha, Roberto de M. Estevão Filho, Todd Hendry, Daniel Holstein, Jennifer Marsman, Nick Mecklenburg, Sara Malvar, Leonardo O. Nunes, Rafael Padilha, Morris Sharp, Bruno Silva, Swati Sharma, Vijay Aski, Ranveer Chandra  
**arXiv:** [2401.08406](https://arxiv.org/abs/2401.08406)  
**Date:** 2024-01-16  
**Summary page:** https://www.summarizepaper.com/en/paper/2401.08406v3/

## Key points

- The paper explores two approaches for incorporating proprietary and domain-specific data into applications of Large Language Models (LLMs): Retrieval-Augmented Generation (RAG) and Fine-Tuning.
- RAG involves augmenting the prompt with external data, while Fine-Tuning incorporates additional knowledge directly into the model.
- The authors propose a pipeline that combines both fine-tuning and RAG techniques to address the limited understanding of their pros and cons.
- The pipeline is evaluated using popular LLMs such as Llama2-13B, GPT-3.5, and GPT-4.
- The study focuses on applying this pipeline to an agricultural dataset in order to provide location-specific insights to farmers.
- The dataset generation pipeline effectively captures geographic-specific knowledge.
- Fine-tuning the model improves accuracy by over 6 percentage points (p.p.), which is further enhanced by RAG with an additional 5 p.p. increase in accuracy.
- The fine-tuned model leverages information from different geographies to answer specific questions, increasing answer similarity from 47% to 72%.
- This research demonstrates how systems built using LLMs can incorporate industry-specific knowledge across different dimensions.
- The findings pave the way for further applications of LLMs in various industrial domains beyond agriculture.

## Summary

The authors of this paper explore two common approaches for incorporating proprietary and domain-specific data into applications of Large Language Models (LLMs): Retrieval-Augmented Generation (RAG) and Fine-Tuning. RAG involves augmenting the prompt with external data, while Fine-Tuning incorporates additional knowledge directly into the model. However, there is limited understanding of the pros and cons of these approaches. To address this gap, the authors propose a pipeline that combines both fine-tuning and RAG techniques. They evaluate the tradeoffs of using this pipeline with popular LLMs such as Llama2-13B, GPT-3.5, and GPT-4. The pipeline consists of several stages including extracting information from PDFs, generating questions and answers, utilizing them for fine-tuning, and leveraging GPT-4 for evaluation. The study focuses on applying this pipeline to an agricultural dataset in order to provide location-specific insights to farmers. Agriculture is an industry that has not fully embraced AI technologies yet, making it a potentially disruptive application area. The authors demonstrate the effectiveness of their dataset generation pipeline in capturing geographic-specific knowledge. The results show significant improvements in accuracy when fine-tuning the model, with an increase of over 6 percentage points (p.p.). This improvement is further enhanced by RAG, which increases accuracy by an additional 5 p.p. In one experiment, the authors also highlight how the fine-tuned model leverages information from different geographies to answer specific questions, increasing answer similarity from 47% to 72%. Overall, this research demonstrates how systems built using LLMs can be adapted to incorporate industry-specific knowledge across different dimensions. The findings pave the way for further applications of LLMs in various industrial domains beyond agriculture.