Running summarizing tools on a new article
Check the paper below, then start its synthesis.
Enhancing Code Transformation in Large Language Models Through Retrieval-Augmented Fine-Tuning
Jing-Ming Guo (National Taiwan University of Science and Technology), P. L. Liu (National Taiwan University of Science and Technology), Yi‐Chong Zeng (National Taiwan University of Science and Technology), Ting-Ju Chen (National Taiwan University of Science and Technology)
Abstract: Large language models (LLMs) have made substantial advancements in knowledge reasoning and are increasingly utilized in specialized domains such as code completion, legal analysis, and medical transcription, where accuracy is paramount. In such applications, document-specific precision is more critical than general reasoning capabilities. This paper proposes a novel approach based on Retrieval-Augmented Fine-Tuning (RAFT) to enhance model-generated outputs, particularly in code transformation tasks. RAFT integrates domain-specific knowledge, optimizing in-domain retrieval-augmented generation by training the model to discern the relationship between prompts, retrieved documents, and target outputs. This enables the model to extract relevant information while mitigating the impact of noise. Experimental results demonstrate that the proposed method improves accuracy of 2.4% and CodeBLEU of 1.3% for VB-to-C# code conversion, highlighting its effectiveness in domain-specific applications.