Pixel is an intelligent platform for creating comics and videos from scripts using AI. It streamlines the workflow from scriptwriting to asset management, storyboard generation, and final video production, leveraging the power of Multi-Agent Systems and advanced generative models.
- Intelligent Script Analysis:
- Automatically parse scripts to identify characters, scenes, and props using LLMs.
- Agent-based Workflow: Utilizes specialized agents (powered by AgentScope) for deep script understanding and breakdown.
- Multi-Provider AI Support:
- LLM: DashScope (Qwen), Google (Gemini), VolcEngine (Doubao).
- Image Generation: Flux 1.1 Pro, Wanx, Kolors (ModelScope).
- Video Generation: Kling 1.5, Hailuo (MiniMax), CogVideoX, Wan 2.1.
- Asset Management: Centralized "Material Center" to manage and edit creative assets.
- AI Storyboarding: Generate visual storyboards from script descriptions using AI image generation with style consistency control.
- Video Generation & Editing:
- Transform static storyboards into dynamic videos.
- Fine-grained Control: First/Last frame control, Camera Motion (Zoom, Pan, Tilt), and Motion Bucket settings.
- Infinite Canvas: A node-based free creation workspace (powered by React Flow) supporting multi-selection, smooth zooming, and flexible node connections.
- Project Management: Organize your creative works in a structured workspace with support for Episodes and Scenes.
The project is structured as a monorepo:
backend/: A NestJS backend service using Prisma ORM, BullMQ for task queues, and supporting multiple model providers via a plugin system.frontend/: A Next.js 15 (App Router) application using React 19, Tailwind CSS, and@xyflow/reactfor the canvas interface.
Comprehensive documentation is available:
- docs/API.md: Complete API reference with examples
- docs/development-guide.md: Development best practices and guidelines
- docs/FRONTEND_OPTIMIZATION.md: Frontend optimization strategies
- Node.js: v20 or higher (Required for Next.js 15)
- Package Manager:
pnpm(Backend and Frontend) - Redis: v7+ (Required for BullMQ task queues)
- PostgreSQL: v15+ (Required for Prisma)
- API Keys: Aliyun DashScope, VolcEngine, or Google AI Studio keys depending on models used
git clone <repository-url>
cd pixelcd backend
# Install dependencies
pnpm install
# Configure environment variables
cp .env.example .env
# Edit .env with your API keys
# Generate Prisma client
pnpm prisma:generate
# Run database migrations
pnpm prisma:migrate
# Start the backend server
pnpm start:devThe backend will be available at http://localhost:8001
cd frontend
# Install dependencies
pnpm install
# Start the development server
pnpm devThe frontend will be available at http://localhost:3000
Create a .env file in the backend/ directory:
# AI Providers
DASHSCOPE_API_KEY=your_dashscope_key
VOLCENGINE_ACCESS_KEY=your_volcengine_key
VOLCENGINE_SECRET_KEY=your_volcengine_secret
GOOGLE_API_KEY=your_google_key
KLING_API_KEY=your_kling_key
# Database (Required)
DATABASE_URL=postgresql://user:pass@localhost:5432/pixel
# Redis (Required for BullMQ)
REDIS_URL=redis://localhost:6379# Build and start all services
docker-compose up -d
# View logs
docker-compose logs -f
# Stop services
docker-compose downThe backend provides RESTful APIs:
- API Documentation: Complete API reference with examples
- Interactive Docs: http://localhost:8001/docs (Swagger UI)
- Image Generation:
POST /api/v1/generations/image - Video Generation:
POST /api/v1/generations/video - Script Analysis:
POST /api/v1/script/analyze - Task Status:
GET /api/v1/tasks/{task_id} - Project Management:
/api/v1/projects/* - Canvas Operations:
/api/v1/canvas/*
All generation endpoints support an extra_params field to pass model-specific arguments directly to the underlying SDK.
Backend:
cd backend
pnpm test # Run all tests
pnpm test:watch # Run in watch modeFrontend:
cd frontend
pnpm test:run # Run all tests (Vitest)
pnpm test:coverage # Run with coverage
pnpm e2e # Run E2E tests (Playwright)Backend:
pnpm lint # Lint (ESLint)
npx tsc --noEmit # Type checkFrontend:
pnpm lint # Lint (ESLint)
pnpm type-check # Type check
pnpm format # Formatcd backend
# Create a new migration
pnpm prisma:migrate --name description
# Apply migrations
pnpm prisma:migrate
# Generate Prisma client
pnpm prisma:generateWhen backend API changes, regenerate frontend types:
cd frontend
pnpm gen:apicurl http://localhost:8001/health- Fork the repository
- Create a feature branch (
git checkout -b feature/amazing-feature) - Commit your changes (
git commit -m 'Add amazing feature') - Push to the branch (
git push origin feature/amazing-feature) - Open a Pull Request
- Follow NestJS conventions (Modules, Controllers, Services)
- Write tests for new features
- Update documentation
- Follow code style guidelines
- Use conventional commits
Redis Connection Error:
# Check if Redis is running
redis-cli ping
# Should return: PONGDatabase Connection Error:
# Check database connection
psql -h localhost -U user -d pixelDependency Issues:
# Reinstall dependencies
pnpm installModule Not Found:
# Clear cache and reinstall
rm -rf node_modules .next
pnpm installAPI Type Mismatch:
# Regenerate API types
pnpm gen:apipixel/
├── backend/ # NestJS backend
│ ├── src/
│ │ ├── modules/ # Feature modules
│ │ ├── common/ # Shared utilities
│ │ └── config/ # Configuration
│ ├── prisma/ # Database schema & migrations
│ └── tests/ # Test suite
├── frontend/ # Next.js frontend
│ ├── src/
│ │ ├── app/ # Next.js pages (App Router)
│ │ ├── components/ # React components
│ │ ├── lib/ # Utilities and services
│ │ └── store/ # State management (Zustand)
│ └── tests/ # Test suite
├── cli/ # CLI tool
├── docs/ # Documentation
└── docker-compose.yml # Docker configuration
MIT