# Efficient Adaptation of Pretrained Transformers for Abstractive Summarization

**Authors:** Andrew Hoang, Antoine Bosselut, Asli Celikyilmaz, Yejin Choi  
**arXiv:** [1906.00138](https://arxiv.org/abs/1906.00138)  
**Date:** 2019-06-01  
**Summary page:** https://www.summarizepaper.com/en/paper/1906.00138v1/

## Key points

- Authors Andrew Hoang, Antoine Bosselut, Asli Celikyilmaz, and Yejin Choi explore adapting pretrained transformer language models for text summarization tasks
- Proposed solutions: source embeddings and domain-adaptive training to address challenges in integrating learned representations into existing neural text production architectures
- Experiments on three abstractive summarization datasets show new state-of-the-art performance on two of them
- Improvements lead to more focused summaries with fewer unnecessary details, especially benefiting more abstractive datasets
- Efficiently leveraging pretrained transformer models through source embeddings and domain-adaptive training enhances summarization tasks using large-scale learning techniques
- Findings contribute to advancing the field of abstractive summarization by demonstrating effective strategies for leveraging pretrained language models in text summarization applications

## Summary

In their paper titled "Efficient Adaptation of Pretrained Transformers for Abstractive Summarization," authors Andrew Hoang, Antoine Bosselut, Asli Celikyilmaz, and Yejin Choi explore the potential of adapting pretrained transformer language models for text summarization tasks. The authors propose two solutions to address challenges in integrating learned representations into existing neural text production architectures: source embeddings and domain-adaptive training. Through experiments on three abstractive summarization datasets, the authors demonstrate that their proposed solutions lead to new state-of-the-art performance on two of them. These improvements result in more focused summaries with fewer unnecessary details, particularly benefiting more abstractive datasets. By efficiently leveraging pretrained transformer models through source embeddings and domain-adaptive training, the authors showcase the potential for enhancing summarization tasks using large-scale learning techniques. Their findings contribute to advancing the field of abstractive summarization by demonstrating effective strategies for leveraging pretrained language models in text summarization applications.