# RAFT: Adapting Language Model to Domain Specific RAG

**Authors:** Tianjun Zhang, Shishir G. Patil, Naman Jain, Sheng Shen, Matei Zaharia, Ion Stoica, Joseph E. Gonzalez  
**arXiv:** [2403.10131](https://arxiv.org/abs/2403.10131)  
**Date:** 2024-03-15  
**Summary page:** https://www.summarizepaper.com/en/paper/2403.10131v1/

## Key points

- Pretraining Large Language Models (LLMs) on extensive textual data is a standard practice in natural language processing.
- Incorporating new knowledge into pretrained LLMs is crucial for various downstream applications.
- Methods for integrating new knowledge include RAG-based prompting and fine-tuning.
- Retrieval Augmented FineTuning (RAFT) offers a novel training recipe to enhance reasoning capabilities within specific domains.

## Summary

, , , , 

In the realm of natural language processing, pretraining Large Language Models (LLMs) on extensive textual data has become a standard practice. These LLMs are often utilized in various downstream applications, where incorporating new knowledge into the pretrained model is crucial. This additional knowledge could range from time-sensitive news updates to domain-specific information. The methods commonly employed for integrating this new knowledge include RAG-based prompting and fine-tuning. However, the most effective approach for imbuing these models with fresh knowledge remains an open question. <nl>

Natural language processing has become a standard practice for pretraining Large Language Models (LLMs). Incorporating new knowledge into these models is crucial for their use in downstream applications. Retrieval Augmented FineTuning (RAFT) offers a novel training recipe to enhance reasoning capabilities within specific domains by disregarding irrelevant documents and extracting relevant information verbatim from appropriate sources.