Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
52 changes: 52 additions & 0 deletions .github/copilot-instructions.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,52 @@
# Copilot Instructions — GalaxyQuest

Diese Datei gilt für alle Python-Komponenten in GalaxyQuest.
Sie folgt dem Standard aus `makr-code/RespoTemplate-Python`.

---

## Python-Coding-Standards (3.11+)

- **Typing**: Alle öffentlichen Funktionen erhalten vollständige Type Hints.
Verwende `type | None` (Python 3.10+ Union-Syntax), nicht `Optional[type]`.
- **Async**: Alle I/O-Operationen (HTTP, Dateisystem, Subprozesse) über
`async/await`. Blocking-Operationen immer via `run_in_threadpool` ausführen.
- **HTTP-Client**: `httpx` mit explizitem Timeout. Kein `urllib.request`.
- **Validation**: `pydantic.BaseModel` an API-Grenzen; `dataclass` für interne Strukturen.
- **Config**: `pydantic-settings` mit `.env`-Fallback. Kein `os.getenv()` im App-Code.
- **Logging**: `structlog` mit strukturierten Key-Value-Events. Kein `print()`, kein `logging.basicConfig`.
- **Ressourcen**: `with` / `async with` für alle I/O-Ressourcen.
- **Fehlerbehandlung**: Spezifische Exception-Typen; kein nacktes `except Exception`.
- **Globaler Zustand**: Mutable globals nur mit `asyncio.Lock` absichern.

## Toolchain

| Tool | Befehl | Zweck |
|------|--------|-------|
| Lint | `ruff check tts_service/ scripts/` | Muss sauber laufen |
| Format | `ruff format --check tts_service/ scripts/` | Muss sauber laufen |
| Typen | `mypy tts_service/main.py` | Keine neuen Fehler |
| Security | CodeQL Python (`.github/workflows/codeql-python.yml`) | Keine High/Critical-Findings |

## Python-Komponenten in diesem Repo

| Verzeichnis | Beschreibung |
|-------------|-------------|
| `tts_service/` | FastAPI TTS Microservice (Piper / Coqui XTTS) |
| `scripts/trellis2_*.py` | CLI-Tools für 3D-Asset-Generierung (TRELLIS2) |
| `scripts/prepend_trigger.py` | LoRA Dataset Helper |

## Dockerfile-Standards (nach RespoTemplate-Python)

- Multi-stage Build (builder + runtime)
- Non-root User (`useradd` + `USER`)
- `PYTHONUNBUFFERED=1` + `PYTHONDONTWRITEBYTECODE=1`
- `HEALTHCHECK` mit `curl -f /health`
- `curl` im Runtime-Image für den Health Probe

## Architekturprinzipien

- Dependency Injection statt globaler Singletons.
- YAGNI – keine Abstraktionen ohne konkreten Nutzen.
- Kleine, fokussierte Funktionen mit klaren Ein- und Ausgaben.
- `Protocol` für Interfaces – Domänenlogik frei von Infrastruktur-Imports.
35 changes: 35 additions & 0 deletions .github/workflows/codeql-python.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,35 @@
name: codeql-python

on:
push:
branches:
- main
- master
pull_request:
branches:
- main
- master
schedule:
- cron: "30 6 * * 1"

permissions:
contents: read
security-events: write

jobs:
analyze:
name: CodeQL Python Security Scan
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- name: Initialize CodeQL
uses: github/codeql-action/init@v3
with:
languages: python
paths: |
tts_service
scripts
- name: Perform CodeQL Analysis
uses: github/codeql-action/analyze@v3
with:
category: "/language:python"
59 changes: 59 additions & 0 deletions .github/workflows/python-quality.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,59 @@
name: python-quality

on:
pull_request:
push:
branches:
- main
- master

permissions:
contents: read

jobs:
lint:
name: Ruff lint + format
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- uses: actions/setup-python@v5
with:
python-version: "3.11"
- name: Install ruff
run: pip install ruff
- name: ruff check
run: ruff check tts_service/ scripts/
- name: ruff format check
run: ruff format --check tts_service/ scripts/

typecheck:
name: mypy
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- uses: actions/setup-python@v5
with:
python-version: "3.11"
- name: Install tts_service dependencies
run: pip install -r tts_service/requirements.txt
- name: mypy – tts_service
working-directory: tts_service
run: mypy main.py config.py cache.py audio.py auth.py engines/
- name: mypy – scripts
run: |
pip install pathlib2 huggingface_hub 2>/dev/null || true
mypy scripts/prepend_trigger.py scripts/trellis2_download_models.py scripts/trellis2_generate.py

test:
name: pytest
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- uses: actions/setup-python@v5
with:
python-version: "3.11"
- name: Install dependencies
run: pip install -r tts_service/requirements.txt
- name: Run tests
working-directory: tts_service
run: pytest tests/ -v
9 changes: 9 additions & 0 deletions .pre-commit-config.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
repos:
- repo: https://github.com/astral-sh/ruff-pre-commit
rev: v0.4.10
hooks:
- id: ruff
args: [--fix]
files: ^(tts_service|scripts)/
- id: ruff-format
files: ^(tts_service|scripts)/
1 change: 1 addition & 0 deletions docker-compose.yml
Original file line number Diff line number Diff line change
Expand Up @@ -88,6 +88,7 @@ services:
TTS_DEFAULT_VOICE: de_DE-thorsten-high
TTS_MAX_CHARS: 2000
# TTS_SECRET: change_me_in_production
# TTS_CORS_ORIGINS: http://localhost:8080 # restrict CORS in production
ports:
- "5500:5500"
volumes:
Expand Down
16 changes: 16 additions & 0 deletions ruff.toml
Original file line number Diff line number Diff line change
@@ -0,0 +1,16 @@
# Root ruff configuration – applies to tts_service/ and scripts/.
# Per-directory overrides can be placed in tts_service/pyproject.toml.
target-version = "py311"
line-length = 100

[lint]
select = ["E", "W", "F", "I", "B", "C4", "UP"]
ignore = [
"E501", # line too long – handled by formatter
"B008", # do not perform function calls in default arguments (FastAPI Depends)
"C901", # function complexity
"EXE001", # shebang without exec bit – expected on non-Unix checkouts
]

[lint.isort]
known-first-party = []
85 changes: 47 additions & 38 deletions scripts/prepend_trigger.py
Original file line number Diff line number Diff line change
Expand Up @@ -20,25 +20,31 @@
python scripts/prepend_trigger.py datasets/vortak/filtered/ "vortak_race" --force
"""

import os
import sys
from __future__ import annotations

import argparse
import sys
from pathlib import Path


def prepend_trigger(directory: str, trigger_word: str, dry_run: bool = False, force: bool = False) -> None:
"""
Prepend trigger_word to every .txt caption file in directory.
def prepend_trigger(
directory: Path,
trigger_word: str,
dry_run: bool = False,
force: bool = False,
) -> None:
"""Prepend trigger_word to every .txt caption file in directory.

:param directory: Path to directory containing .txt caption files
:param directory: Path to directory containing .txt caption files
:param trigger_word: The LoRA trigger word to prepend (e.g. "vortak_race")
:param dry_run: If True, print actions without writing any files
:param force: If True, re-prepend even if trigger_word already present
:param dry_run: If True, print actions without writing any files
:param force: If True, re-prepend even if trigger_word already present
:raises FileNotFoundError: When *directory* does not exist
"""
if not os.path.isdir(directory):
print(f"[ERROR] Directory not found: {directory}", file=sys.stderr)
sys.exit(1)
if not directory.is_dir():
raise FileNotFoundError(f"Directory not found: {directory}")

txt_files = sorted(f for f in os.listdir(directory) if f.endswith('.txt'))
txt_files = sorted(directory.glob("*.txt"))

if not txt_files:
print(f"[WARN] No .txt files found in: {directory}")
Expand All @@ -47,30 +53,23 @@ def prepend_trigger(directory: str, trigger_word: str, dry_run: bool = False, fo
updated = 0
skipped = 0

for filename in txt_files:
path = os.path.join(directory, filename)

with open(path, 'r', encoding='utf-8') as fh:
content = fh.read().strip()
for file_path in txt_files:
content = file_path.read_text(encoding="utf-8").strip()

# Check if trigger word is already at the start
already_present = content.startswith(trigger_word)

if already_present and not force:
print(f" [SKIP] {filename} (trigger already present)")
if content.startswith(trigger_word) and not force:
print(f" [SKIP] {file_path.name} (trigger already present)")
skipped += 1
continue

new_content = f"{trigger_word}, {content}" if content else trigger_word

if dry_run:
print(f" [DRY] {filename}")
print(f" [DRY] {file_path.name}")
print(f" Before: {content[:80]}{'...' if len(content) > 80 else ''}")
print(f" After: {new_content[:80]}{'...' if len(new_content) > 80 else ''}")
else:
with open(path, 'w', encoding='utf-8') as fh:
fh.write(new_content)
print(f" [ OK ] {filename}")
file_path.write_text(new_content, encoding="utf-8")
print(f" [ OK ] {file_path.name}")

updated += 1

Expand All @@ -83,12 +82,18 @@ def prepend_trigger(directory: str, trigger_word: str, dry_run: bool = False, fo

def main() -> None:
parser = argparse.ArgumentParser(
description='Prepend a LoRA trigger word to all .txt caption files in a dataset directory.'
description="Prepend a LoRA trigger word to all .txt caption files in a dataset directory."
)
parser.add_argument("directory", help="Path to dataset directory containing .txt caption files")
parser.add_argument("trigger_word", help='LoRA trigger word to prepend (e.g. "vortak_race")')
parser.add_argument(
"--dry-run", action="store_true", help="Preview changes without modifying files"
)
parser.add_argument(
"--force",
action="store_true",
help="Re-prepend even if trigger already present",
)
parser.add_argument('directory', help='Path to dataset directory containing .txt caption files')
parser.add_argument('trigger_word', help='LoRA trigger word to prepend (e.g. "vortak_race")')
parser.add_argument('--dry-run', action='store_true', help='Preview changes without modifying files')
parser.add_argument('--force', action='store_true', help='Re-prepend even if trigger already present')

args = parser.parse_args()

Expand All @@ -98,13 +103,17 @@ def main() -> None:
print("[INFO] Mode: DRY-RUN (no files will be modified)")
print()

prepend_trigger(
directory=args.directory,
trigger_word=args.trigger_word,
dry_run=args.dry_run,
force=args.force,
)
try:
prepend_trigger(
directory=Path(args.directory),
trigger_word=args.trigger_word,
dry_run=args.dry_run,
force=args.force,
)
except FileNotFoundError as exc:
print(f"[ERROR] {exc}", file=sys.stderr)
sys.exit(1)


if __name__ == '__main__':
if __name__ == "__main__":
main()
24 changes: 12 additions & 12 deletions scripts/trellis2_download_models.py
Original file line number Diff line number Diff line change
Expand Up @@ -10,21 +10,24 @@
from __future__ import annotations

import argparse
import json
import pathlib
import sys
from typing import Sequence
from collections.abc import Sequence

AVAILABLE_MODELS: dict[str, str] = {
"image-large": "microsoft/TRELLIS-image-large",
"text-base": "microsoft/TRELLIS-text-base",
"text-large": "microsoft/TRELLIS-text-large",
"text-xlarge": "microsoft/TRELLIS-text-xlarge",
"image-large": "microsoft/TRELLIS-image-large",
"text-base": "microsoft/TRELLIS-text-base",
"text-large": "microsoft/TRELLIS-text-large",
"text-xlarge": "microsoft/TRELLIS-text-xlarge",
}
DEFAULT_MODELS = ["image-large"]


def parse_args(argv: Sequence[str] | None = None) -> argparse.Namespace:
parser = argparse.ArgumentParser(description="Laedt TRELLIS2-Modelle von HuggingFace lokal herunter.")
parser = argparse.ArgumentParser(
description="Laedt TRELLIS2-Modelle von HuggingFace lokal herunter."
)
parser.add_argument(
"--models",
default=",".join(DEFAULT_MODELS),
Expand Down Expand Up @@ -76,8 +79,7 @@ def download_model(

if model_key not in AVAILABLE_MODELS:
raise ValueError(
f"Unbekannter Model-Schluessel '{model_key}'. "
f"Erlaubt: {', '.join(AVAILABLE_MODELS)}"
f"Unbekannter Model-Schluessel '{model_key}'. Erlaubt: {', '.join(AVAILABLE_MODELS)}"
)

repo_id = AVAILABLE_MODELS[model_key]
Expand All @@ -102,8 +104,6 @@ def download_model(

def write_model_registry(cache_dir: pathlib.Path, downloaded: dict[str, pathlib.Path]) -> None:
"""Schreibt models.json mit lokalen Pfaden fuer schnellen Zugriff durch generate-Skripte."""
import json

registry: dict = {}
for key, local_dir in downloaded.items():
registry[key] = {
Expand Down Expand Up @@ -152,14 +152,14 @@ def main(argv: Sequence[str] | None = None) -> int:
try:
local_dir = download_model(model_key, cache_dir, args.token, args.revision)
downloaded[model_key] = local_dir
except Exception as exc:
except (OSError, ValueError, RuntimeError) as exc:
print(f"[TRELLIS2] ✗ Fehler bei '{model_key}': {exc}")
failed.append(model_key)

if downloaded:
write_model_registry(cache_dir, downloaded)

print("")
print()
print(f"[TRELLIS2] Abgeschlossen: {len(downloaded)} OK, {len(failed)} Fehler.")
if failed:
print(f"[TRELLIS2] Fehlgeschlagen: {failed}")
Expand Down
Loading