AI Music Generator

Author: Andrea D'Ambrosio — github.com/andrebuils

---

What is AI Music Generator?

AI Music Generator is an open-source SaaS platform that generates original music using artificial intelligence. Starting from a plain-text description, custom lyrics, or a style prompt, the system produces a complete track — including audio and a cover image.

Built with a production-ready stack: user authentication, a credit/payment system with Polar.sh, background processing queues with Inngest, and serverless GPU inference with Modal.

All platforms used offer a free tier — you can run this project at zero cost during development.

---

Features

• 🎵 AI music generation powered by ACE-Step
• 🧠 Automatic lyrics and prompt generation with Qwen2-7B
• 🖼️ AI cover image generation with SDXL-Turbo
• 🎤 Three generation modes: free description, custom lyrics, described lyrics
• 🎸 Instrumental mode (no vocals)
• ⚡ Serverless GPU processing via Modal
• 📊 Background job queue with Inngest
• 💳 Credit system
• 💰 Polar.sh integration for purchasing credit packs
• 👤 User authentication with BetterAuth
• 🎧 Community feed to discover, listen to, and like generated tracks
• 🎛️ Personal dashboard to manage, listen to, and publish your own tracks
• 🐍 Python backend on Modal for all AI generation logic
• 📱 Modern UI built with Next.js, Tailwind CSS, and ShadCN

---

Tech Stack

Area	Technologies
Frontend	Next.js 15, React 19, TypeScript, Tailwind CSS 4, ShadCN
Authentication	BetterAuth
Payments	Polar.sh
Database	PostgreSQL (Neon) + Prisma
Queue / Workflow	Inngest
Storage	AWS S3
AI Backend	Python, Modal (GPU L40S)
Music model	ACE-Step v1 3.5B
Text model	Qwen2-7B-Instruct
Image model	SDXL-Turbo

---

Backend Setup

Clone the repository

git clone --recurse-submodules https://github.com/andrebuils/ai-music-generator.git
cd ai-music-generator

Install Python 3.12

Download and install Python 3.12 from python.org/downloads.

Create a virtual environment:

python -m venv venv
venv\Scripts\activate   # Windows
source venv/bin/activate  # macOS / Linux

Install dependencies

cd backend
pip install -r requirements.txt

Configure Modal

modal setup

Create a Modal secret named ai-music-generator-secret containing the environment variables listed in the Backend Environment Variables section below.

Run the backend locally

modal run main.py

Deploy the backend

modal deploy main.py

---

Backend Environment Variables

Create a Modal secret named ai-music-generator-secret with the following keys:

Variable	Description
AWS_ACCESS_KEY_ID	AWS IAM access key with S3 upload permissions
AWS_SECRET_ACCESS_KEY	Corresponding AWS secret key
AWS_REGION	S3 bucket region (e.g. us-east-1)
S3_BUCKET_NAME	Name of your S3 bucket

---

AWS S3 Configuration

Two separate IAM users are required.

Backend IAM user policy (upload + read):

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "AllowUploadAndRead",
            "Effect": "Allow",
            "Action": ["s3:PutObject", "s3:GetObject"],
            "Resource": "arn:aws:s3:::YOUR_BUCKET_NAME/*"
        },
        {
            "Sid": "AllowList",
            "Effect": "Allow",
            "Action": "s3:ListBucket",
            "Resource": "arn:aws:s3:::YOUR_BUCKET_NAME"
        }
    ]
}

Frontend IAM user policy (read-only + presigned URLs):

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "AllowRead",
            "Effect": "Allow",
            "Action": "s3:GetObject",
            "Resource": "arn:aws:s3:::YOUR_BUCKET_NAME/*"
        },
        {
            "Sid": "AllowList",
            "Effect": "Allow",
            "Action": "s3:ListBucket",
            "Resource": "arn:aws:s3:::YOUR_BUCKET_NAME"
        }
    ]
}

---

Frontend Setup

Install dependencies

cd frontend
npm install

Configure environment variables

Copy the example file and fill in the values:

cp .env.example .env

See Frontend Environment Variables below for the full list.

Run the development server

npm run dev

Start the Inngest queue (in a second terminal)

npx inngest-cli@latest dev

Apply database migrations

npx prisma migrate dev

---

Frontend Environment Variables

Variable	Description
DATABASE_URL	PostgreSQL connection string (e.g. from Neon)
BETTER_AUTH_SECRET	Secret key for BetterAuth session signing
MODAL_KEY	Modal API key (from modal token new)
MODAL_SECRET	Modal API secret
AWS_ACCESS_KEY_ID	AWS IAM access key (frontend read-only user)
AWS_SECRET_ACCESS_KEY_ID	Corresponding AWS secret key
AWS_REGION	S3 bucket region
S3_BUCKET_NAME	Name of your S3 bucket
GENERATE_FROM_DESCRIPTION	Modal endpoint URL for description-based generation
GENERATE_FROM_DESCRIBED_LYRICS	Modal endpoint URL for described-lyrics generation
GENERATE_WITH_LYRICS	Modal endpoint URL for custom-lyrics generation
POLAR_ACCESS_TOKEN	Polar.sh API access token
POLAR_WEBHOOK_SECRET	Polar.sh webhook signing secret

---

Contributing

Contributions, issues, and feature requests are welcome. Feel free to open a pull request or an issue.

---

License

MIT © Andrea D'Ambrosio