ACE-Step

ACE-Step
A Step Towards Music Generation Foundation Model

    Project |
    Hugging Face |
    ModelScope |
    Space Demo |
    Discord |
    Technical Report |
    ACE-Step v1.5

Table of Contents

- ✨ Features
- 📦 Installation
- 🚀 Usage
- 📱 User Interface Guide
- 🔨 Train

📝 Abstract

We introduce ACE-Step, a novel open-source foundation model for music generation that overcomes key limitations of existing approaches and achieves state-of-the-art performance through a holistic architectural design. Current methods face inherent trade-offs between generation speed, musical coherence, and controllability. For instance, LLM-based models (e.g., Yue, SongGen) excel at lyric alignment but suffer from slow inference and structural artifacts. Diffusion models (e.g., DiffRhythm), on the other hand, enable faster synthesis but often lack long-range structural coherence.

ACE-Step bridges this gap by integrating diffusion-based generation with Sana’s Deep Compression AutoEncoder (DCAE) and a lightweight linear transformer. It further leverages MERT and m-hubert to align semantic representations (REPA) during training, enabling rapid convergence. As a result, our model synthesizes up to 4 minutes of music in just 20 seconds on an A100 GPU—15× faster than LLM-based baselines—while achieving superior musical coherence and lyric alignment across melody, harmony, and rhythm metrics. Moreover, ACE-Step preserves fine-grained acoustic details, enabling advanced control mechanisms such as voice cloning, lyric editing, remixing, and track generation (e.g., lyric2vocal, singing2accompaniment).

Rather than building yet another end-to-end text-to-music pipeline, our vision is to establish a foundation model for music AI: a fast, general-purpose, efficient yet flexible architecture that makes it easy to train sub-tasks on top of it. This paves the way for developing powerful tools that seamlessly integrate into the creative workflows of music artists, producers, and content creators. In short, we aim to build the Stable Diffusion moment for music.

📢 News and Updates

- 🎉 2026.01.28: Released ACE-Step v1.5 - Our latest and most advanced model is now available!

- 📃 2025.06.02: Released ACE-Step Technical Report (PDF).

- 🎮 2025.05.14: Add Stable Audio Open Small sampler pingpong. Use SDE to achieve better music consistency and quality, including lyric alignment and style alignment. Use a better method to re-implement Audio2Audio

- 🎤 2025.05.12: Release RapMachine and fix lora training issues
    - See ZH_RAP_LORA.md for details. Audio Examples: https://ace-step.github.io/#RapMachine
    - See TRAIN_INSTRUCTION.md for detailed training instructions.

- 🔥 2025.05.10: Memory Optimization Update
  - Reduced Max VRAM to 8GB, making it more compatible with consumer devices
  - Recommended launch options:
    bash
    acestep --torch_compile true --cpu_offload true --overlapped_decode true
    Windows need to install triton:
    pip install triton-windows

- 📢 2025.05.09: Graidio Demo support Audio2Audio. ComfyUI: Ace_Step_4x_a2a.json

- 🚀 2025.05.08: ComfyUI_ACE-Step node is now available! Explore the power of ACE-Step within ComfyUI. 🎉

- 🚀 2025.05.06

……（后略，见原文）