SongGen: A Single Stage Auto-regressive Transformer for Text-to-Song Generation

Abstract

Text-to-song generation, the task of creating vocals and accompaniment from textual inputs, poses significant challenges due to domain complexity and data scarcity. Existing approaches often employ multi-stage generation procedures, leading to cumbersome training and inference pipelines, as well as suboptimal overall generation quality due to error accumulation across stages. In this paper, we propose SongGen, a fully open-source, single-stage auto-regressive transformer designed for controllable song generation. The proposed model facilitates fine-grained control over diverse musical attributes, including lyrics and textual descriptions of instrumentation, genre, mood, and timbre, while also offering an optional three-second reference clip for voice cloning. Within a unified auto-regressive framework, SongGen supports two output modes: mixed mode, which generates a mixture of vocals and accompaniment directly, and dual-track mode, which synthesizes them separately for greater flexibility in downstream applications. We explore diverse token pattern strategies for each mode, leading to notable improvements and valuable insights. Furthermore, we design an automated data preprocessing pipeline with effective quality control. To foster community engagement and future research, we will release our model weights, training code, annotated data, and preprocessing pipeline. The code is available at https://github.com/LiuZH-19/SongGen.

Author abstract · arXiv abstract · v2

Publication

International Conference on Machine Learning (ICML), 2025

Paper and resources

• Paper · arXiv v2 (PDF)
• arXiv:2502.13128
• Google Scholar
• Code · LiuZH-19/SongGen

Research topics

SongGen · Text-to-song generation · Autoregressive audio generation · Controllable music generation · Voice cloning · Vocal synthesis · Dual-track generation · Audio tokenization

Research problem and approach

Multi-stage song generation introduces pipeline complexity and can accumulate errors. SongGen unifies generation in one transformer while supporting lyrics, musical descriptions and an optional three-second reference for voice cloning. arXiv abstract · v2

Main contributions

• Supports mixed and dual-track generation within a unified autoregressive framework. arXiv abstract · v2
• Studies audio token patterns and develops an automated, quality-controlled preprocessing pipeline. arXiv abstract · v2

Method comparison

arXiv abstract · v2

Selected results

• SongGen’s Mixed-pro configuration obtains FAD 1.71 versus 2.18 for the multi-stage baseline, with CLAP 0.35 versus 0.29. Its phoneme error rate is 40.58 versus 38.80, so improved audio-distribution and text-alignment scores do not imply better lyric accuracy in every comparison. Table 1 · song-generation comparison · arXiv v2
• Removing curriculum learning increases FAD from 1.71 to 2.35 and phoneme error rate from 40.58 to 55.71. Removing high-quality fine-tuning gives FAD 2.01 and phoneme error rate 43.68. Table 4 · training-scheme ablation · arXiv v2

Cite this paper

Zihan Liu, Shuangrui Ding, Zhixiong Zhang, Xiaoyi Dong, Pan Zhang, Yuhang Zang, Yuhang Cao, Dahua Lin, Jiaqi Wang. SongGen: A Single Stage Auto-regressive Transformer for Text-to-Song Generation. Proceedings of the 42nd International Conference on Machine Learning, 267, 2025, pp. 38351–38364.

@inproceedings{arxiv250213128,
  title     = {{SongGen: A Single Stage Auto-regressive Transformer for Text-to-Song Generation}},
  author    = {Zihan Liu and Shuangrui Ding and Zhixiong Zhang and Xiaoyi Dong and Pan Zhang and Yuhang Zang and Yuhang Cao and Dahua Lin and Jiaqi Wang},
  booktitle = {Proceedings of the 42nd International Conference on Machine Learning},
  year      = {2025},
  volume    = {267},
  publisher = {PMLR},
  pages     = {38351--38364},
  url       = {https://proceedings.mlr.press/v267/liu25m.html}
}