The False Promise of Imitating Proprietary LLMs

The development of language models (LMs) has been a significant area of research in natural language processing (NLP). An emerging method to improve weaker LMs is to fine-tune them on outputs from stronger models, such as proprietary systems like ChatGPT. This approach aims to imitate the capabilities of proprietary models using weaker open-source models at a lower cost. In this study, the authors critically analyze this approach by finetuning a series of LMs that imitate ChatGPT using varying base model sizes, data sources, and imitation data amounts. The models are evaluated using crowd raters and canonical NLP benchmarks. Initially, the output quality of the imitation models surprised the researchers as they appeared far better at following instructions, and crowd workers rated their outputs as competitive with ChatGPT. However, when conducting more targeted automatic evaluations, they found that imitation models closed little to none of the gap from the base LM to ChatGPT on tasks that are not heavily supported in the imitation data. The researchers show that these performance discrepancies may slip past human raters because imitation models are adept at mimicking ChatGPT's style but not its factuality. Overall, the authors conclude that model imitation is a false promise since there exists a substantial capabilities gap between open and closed LMs that can only be bridged using an unwieldy amount of imitation data or by using more capable base LMs. They argue that improving open-source models' highest leverage action is tackling the difficult challenge of developing better base LMs rather than taking shortcuts by imitating proprietary systems. This study sheds light on the limitations of model imitation and highlights the need for continued efforts towards developing better base LMs for NLP applications.