# Better Synthetic Data by Retrieving and Transforming Existing Datasets

**Authors:** Saumya Gandhi, Ritu Gala, Vijay Viswanathan, Tongshuang Wu, Graham Neubig  
**arXiv:** [2404.14361](https://arxiv.org/abs/2404.14361)  
**Date:** 2024-04-22  
**Summary page:** https://www.summarizepaper.com/en/paper/2404.14361v1/

## Key points

- Need for abundant and dependable data remains a significant bottleneck in AI research
- Manual curation of task-specific annotated data is labor-intensive
- Previous studies have explored prompt-driven synthetic data generation using large language models
- DataTune method introduced to enhance automatic dataset generation by leveraging existing datasets more effectively
- DataTune facilitates dataset transformation to align with specific task requirements
- Experiments show that finetuning language models using DataTune outperformed few-shot prompting baselines by 49%
- Dataset transformation increased diversity and difficulty of generated data across tasks
- DataTune integrated into an open-source repository for broader research community access

## Summary

In the realm of , the need for  remains a significant bottleneck despite recent advancements in large language models. Obtaining abundant and dependable data is crucial for building and deploying  effectively. However, many specialized or emerging tasks lack task-specific annotated data, making the manual curation of such data a labor-intensive process. To address this challenge, previous studies have explored prompt-driven synthetic data generation using large language models. Yet, these generated datasets often lack complexity and diversity. In response to these limitations, a novel method called \textit{DataTune} has been introduced to enhance automatic dataset generation by leveraging existing publicly available datasets more effectively. facilitates dataset transformation, enabling the repurposing of existing datasets into formats that align directly with specific task requirements. Through experiments on a diverse range of language-based tasks from the BIG-Bench benchmark, it was found that finetuning language models using  outperformed few-shot prompting baselines by 49% and surpassed existing methods utilizing synthetic or retrieved training data by 34%. Notably, dataset transformation significantly increased the diversity and difficulty of generated data across various tasks. Furthermore,  has been integrated into an open-source repository to make this innovative method accessible to the broader research community at https://github.com/neulab/prompt2model. This advancement holds promise for improving the quality and efficiency of dataset generation for specialized tasks in AI research, offering a valuable tool for researchers seeking to optimize model performance in low-resource settings where task-specific annotated data is limited.