# MusicLM: Generating Music From Text

**Authors:** Andrea Agostinelli, Timo I. Denk, Zalán Borsos, Jesse Engel, Mauro Verzetti, Antoine Caillon, Qingqing Huang, Aren Jansen, Adam Roberts, Marco Tagliasacchi, Matt Sharifi, Neil Zeghidour, Christian Frank  
**arXiv:** [2301.11325](https://arxiv.org/abs/2301.11325)  
**Date:** 2023-01-26  
**Summary page:** https://www.summarizepaper.com/en/paper/2301.11325v1/

## Key points

- MusicLM is a model that generates high-fidelity music from text descriptions
- It uses hierarchical sequence-to-sequence modeling to achieve consistent music production at 24 kHz over several minutes
- MusicLM surpasses previous systems in terms of audio quality and adherence to the given text description
- It can be conditioned on both text and melody, transforming whistled and hummed melodies according to the style described in a text caption
- The creators of MusicLM have released a dataset called MusicCaps consisting of 5.5k music-text pairs with rich text descriptions for further research
- MusicLM offers improved audio quality and fidelity compared to previous systems
- Its ability to incorporate both text and melody conditioning opens up new possibilities for creative applications in music composition and production.

## Summary

MusicLM is a model that can generate high-fidelity music from text descriptions. It takes in text inputs like "a calming violin melody backed by a distorted guitar riff" and produces music at 24 kHz that remains consistent over several minutes. This process is achieved through hierarchical sequence-to-sequence modeling. The experiments conducted show that MusicLM surpasses previous systems in terms of audio quality and adherence to the given text description. One interesting aspect of MusicLM is its ability to be conditioned on both text and a melody. It can transform whistled and hummed melodies according to the style described in a text caption, demonstrating its flexibility and versatility. To support further research, the creators of MusicLM have publicly released a dataset called MusicCaps which consists of 5.5k music-text pairs with rich text descriptions provided by human experts. Researchers can use this dataset to explore and enhance the capabilities of music generation models. Overall, MusicLM presents an innovative approach to generating music from textual descriptions, offering improved audio quality and fidelity compared to previous systems. Its ability to incorporate both text and melody conditioning opens up new possibilities for creative applications in music composition and production.