Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 8 additions & 0 deletions .dockerignore
Original file line number Diff line number Diff line change
Expand Up @@ -104,3 +104,11 @@ coverage.xml

# UV cache (keep .python-version and uv.lock - they're needed in Docker)
.uv/

# DragonTTS — self-contained sub-service (own Dockerfile/pyproject/venv under
# dragontts/). The clairvoyance Dockerfile does `COPY . .` (NOT `COPY app ./app`),
# so this exclusion is LOAD-BEARING: without it, the entire dragontts/ service
# (its own Dockerfile, pyproject, and app/ subpackage) would be baked into the
# clairvoyance image. Do NOT remove it thinking it's redundant. dragontts ships
# its own image via dragontts-docker-publish.yml (context: dragontts).
dragontts/
62 changes: 62 additions & 0 deletions .github/workflows/dragontts-docker-publish.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,62 @@
name: Build & Push DragonTTS Image to Artifact Registry

# Pushes the DragonTTS image (the self-contained app under dragontts/, with
# its own Dockerfile/pyproject/uv.lock) to its own Artifact Registry repo.
# Independent of the clairvoyance image workflows: it only runs when DragonTTS
# files change on `release`. Build + push only — the GKE rollout is done
# manually with kubectl. Reuses clairvoyance's WIF identity (same repo/project).
#
# Prerequisite (GCP side, not CI): the github-deployer@breeze-automatic-prod SA
# must have Artifact Registry Writer on the `dragontts` repo in
# breeze-automatic-prod (it already has it on `clairvoyance`).

on:
push:
branches:
- release
paths:
- dragontts/**

permissions:
id-token: write
contents: read

concurrency:
group: dragontts-docker-publish-${{ github.ref }}
cancel-in-progress: false

jobs:
build-and-push:
runs-on: ubuntu-latest

steps:
- name: Checkout code
uses: actions/checkout@v4

- name: Authenticate to Google Cloud
uses: google-github-actions/auth@v1

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🎯 Functional Correctness | 🟠 Major | ⚡ Quick win

🧩 Analysis chain

🏁 Script executed:

#!/usr/bin/env bash
set -euo pipefail
gh api repos/google-github-actions/auth/releases/latest --jq .tag_name
gh api 'repos/google-github-actions/auth/contents/action.yml?ref=v1' \
  --jq .content | base64 -d | rg -n 'using:|node'

Repository: juspay/clairvoyance

Length of output: 182


🏁 Script executed:

#!/usr/bin/env bash
set -euo pipefail

echo "Workflow references:"
rg -n "google-github-actions/auth|runs-on:|node:" .github/workflows/dragontts-docker-publish.yml || true

echo
echo "Workflows using node24/ubuntu latest/github-actions:"
rg -n "node24|runs-on:|google-github-actions/auth" .github/workflows || true

echo
echo "actionlint availability/build config:"
(rg -n "actionlint|google-github-actions/auth|google-github-actions/setup-gcloud" -S .github/workflows actionlint.* 2>/dev/null || true)

echo
echo "latest auth action.yml runtime header:"
gh api 'repos/google-github-actions/auth/contents/action.yml?ref=v3' \
  --jq .content | head -n 25 | base64 -d || true

echo
echo "latest auth action.yml node/runtime mentions:"
gh api 'repos/google-github-actions/auth/contents/action.yml?ref=v3' \
  --jq .content | base64 -d | rg -n 'using:|node|node24|runner-target' || true

Repository: juspay/clairvoyance

Length of output: 2599


🌐 Web query:

google-github-actions/auth v3 node24 GitHub Actions runner compatibility

💡 Result:

The google-github-actions/auth v3 action requires the use of Node 24 [1][2]. This version was introduced in v3.0.0, which updated the action's execution environment to Node 24 [1][3]. If you encounter an "argument was out of the range of valid values" error when using v3, it indicates that your GitHub Actions runner does not support Node 24 [2]. To resolve this compatibility issue, you must use a runner environment that includes support for Node 24 [4][2]. If you are using self-hosted runners, you need to ensure the runner software is updated to a version capable of executing Node 24 actions [2]. As of July 2026, GitHub has been migrating its hosted runners to support Node 24 as part of the deprecation of Node 20 [5]. If you are experiencing issues on specific infrastructure, verify that your runner environment is fully updated and compatible with the latest GitHub Actions requirements [2][5]. In cases where an upgrade is not immediately possible, you may need to pin your workflow to a previous version of the action that still supports Node 20 (e.g., v2.x) until your environment can be upgraded [2].

Citations:


Update the Google authentication action.

google-github-actions/auth@v1 runs under node16, while the current release (v3) targets node24. Update this workflow to @v3 and ensure the runner image supports Node 24; otherwise keep it pinned to a Node 20-compatible version.

Suggested change
-uses: google-github-actions/auth@v1
+uses: google-github-actions/auth@v3
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
uses: google-github-actions/auth@v1
uses: google-github-actions/auth@v3
🧰 Tools
🪛 actionlint (1.7.12)

[error] 37-37: the runner of "google-github-actions/auth@v1" action is too old to run on GitHub Actions. update the action's version to fix this issue

(action)

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In @.github/workflows/dragontts-docker-publish.yml at line 37, Update the Google
authentication step using google-github-actions/auth to `@v3` and ensure the
workflow’s runner image supports Node 24; if the runner cannot support Node 24,
retain the action at a Node 20-compatible version instead.

Source: Linters/SAST tools

with:
workload_identity_provider: projects/840384202515/locations/global/workloadIdentityPools/github-pool/providers/github-provider
service_account: github-deployer@breeze-automatic-prod.iam.gserviceaccount.com

- name: Configure Docker for Artifact Registry
run: |
gcloud auth configure-docker asia-south1-docker.pkg.dev --quiet

- name: Set up Docker Buildx
uses: docker/setup-buildx-action@v3

- name: Extract short SHA
run: echo "SHORT_SHA=${GITHUB_SHA:0:8}" >> $GITHUB_ENV

- name: Build and Push DragonTTS Image to Artifact Registry
uses: docker/build-push-action@v5
with:
context: dragontts
push: true
platforms: linux/amd64
tags: |
asia-south1-docker.pkg.dev/breeze-automatic-prod/dragontts/dragontts:${{ env.SHORT_SHA }}
asia-south1-docker.pkg.dev/breeze-automatic-prod/dragontts/dragontts:latest
provenance: false
sbom: false
15 changes: 15 additions & 0 deletions dragontts/.dockerignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,15 @@
.git
.github
.venv
.pytest_cache
.claude
.env
.env.*
!.env.example
data
tests
scripts
README.md
*.md
__pycache__
*.pyc
31 changes: 31 additions & 0 deletions dragontts/.env.example
Original file line number Diff line number Diff line change
@@ -0,0 +1,31 @@
# --- Provider credentials (names mirror clairvoyance) ---
CARTESIA_API_KEY=
SARVAM_API_KEY=
ELEVENLABS_API_KEY=
ELEVENLABS_BASE_URL=https://api.elevenlabs.io
ELEVENLABS_INDIAN_RESIDENCY_API_KEY=
ELEVENLABS_INDIAN_RESIDENCY_BASE_URL=
GOOGLE_CREDENTIALS_JSON=
GOOGLE_CREDENTIALS_PATH=

# --- Storage (point both at a persistent volume in k8s) ---
DB_PATH=data/dragontts.db
BLOB_DIR=data/blobs

# --- Cache policy ---
MAX_CACHE_BYTES=0 # 0 = unlimited
TTL_SECONDS=0 # 0 = no expiry
ENABLE_WRITE_THROUGH=true
Comment on lines +16 to +18

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🩺 Stability & Availability | 🟠 Major | ⚡ Quick win

Bound cache growth in the default configuration.

MAX_CACHE_BYTES=0 plus TTL_SECONDS=0 makes write-through cached audio permanent and unlimited. On the documented persistent volume, this can eventually exhaust disk space and break SQLite/blob writes. Set finite defaults or require explicit capacity and retention settings before deployment.

🧰 Tools
🪛 dotenv-linter (4.0.0)

[warning] 16-16: [ValueWithoutQuotes] This value needs to be surrounded in quotes

(ValueWithoutQuotes)


[warning] 17-17: [ValueWithoutQuotes] This value needs to be surrounded in quotes

(ValueWithoutQuotes)


[warning] 18-18: [UnorderedKey] The ENABLE_WRITE_THROUGH key should go before the MAX_CACHE_BYTES key

(UnorderedKey)

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@dragontts/.env.example` around lines 16 - 18, Update the default cache
configuration in .env.example by replacing unlimited MAX_CACHE_BYTES and
non-expiring TTL_SECONDS values with finite, bounded defaults, while preserving
ENABLE_WRITE_THROUGH=true. Ensure the documented defaults cap both cache
capacity and retention so persistent cached audio cannot grow indefinitely.


# --- Sentence splitting ---
# Split a transcript into sentences BEFORE synth/serve so each is synthesized +
# cached under its own key (recurring sentences get reused). Parts are then pure-
# concatenated — no edge trim, no inserted gap (nothing is cut; appended exactly
# as synthesized). Empty = OFF (the whole transcript is one cache entry, the
# default). Set to the symbols to split AFTER, e.g. .:?! The synth and stream
# paths are controlled INDEPENDENTLY:
SPLIT_AT_SYMBOLS= # /tts/bytes (one-shot synth) path; empty = OFF
SPLIT_AT_SYMBOLS_STREAM= # /tts/stream path; empty = OFF
# Only split when EVERY resulting part has >= this many words (a lone single-word
# part isn't worth caching alone, so the whole phrase stays one entry).
SPLIT_MIN_WORDS_PER_PART=2
41 changes: 41 additions & 0 deletions dragontts/.gitignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,41 @@
# Python
__pycache__/
*.py[cod]
*.egg-info/
.venv/
venv/
env/

# Node
node_modules/

# Env / secrets
.env
.env.*
!.env.example
*.pem
*.key

# OS
.DS_Store
Thumbs.db

# IDE
.vscode/
.idea/

# Claude (local tooling state — never commit)
.claude/

# DragonTTS runtime storage (SQLite db + audio blobs)
data/
*.db

# Audio artifacts
*.wav
*.mp3
*.ogg
/samples/

# Local live smoke tests + their audio output (never commit)
/smoke/
1 change: 1 addition & 0 deletions dragontts/.python-version
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
3.11
29 changes: 29 additions & 0 deletions dragontts/Dockerfile
Original file line number Diff line number Diff line change
@@ -0,0 +1,29 @@
FROM python:3.12-slim AS base

# Cap glibc malloc arenas so freed memory returns to the OS instead of fragmenting
# across dozens of pools. The default (8 x host-cores) lets the 4x32 worker
# threads strand freed audio buffers across ~100 arenas -> anon RSS creeps. 4 is a
# small fixed cap; zero perf cost on this 1-CPU, GIL-bound, async-I/O pod.
ENV PYTHONUNBUFFERED=1 \
PYTHONDONTWRITEBYTECODE=1 \
UV_LINK_MODE=copy \
UV_COMPILE_BYTECODE=1 \
MALLOC_ARENA_MAX=4

COPY --from=ghcr.io/astral-sh/uv:0.5.11 /uv /usr/local/bin/uv

WORKDIR /app

COPY pyproject.toml uv.lock ./
RUN uv sync --frozen --no-install-project

COPY app ./app

RUN uv sync --frozen

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🗄️ Data Integrity & Integration | 🟠 Major | ⚡ Quick win

🧩 Analysis chain

🏁 Script executed:

#!/bin/bash
set -euo pipefail

echo "== repo files matching Dockerfile/uv.lock =="
git ls-files | rg '(^|/)Dockerfile$|(^|/)uv\.lock$|(^|/)pyproject\.toml$' || true

echo
echo "== dockerfile context =="
if [ -f dragontts/Dockerfile ]; then
  nl -ba dragontts/Dockerfile | sed -n '1,80p'
else
  fd -a 'Dockerfile' dragontts 2>/dev/null | while read -r f; do
    echo "--- $f"
    nl -ba "$f" | sed -n '1,120p'
  done
fi

echo
echo "== uv lock/pyproject =="
if [ -f pyproject.toml ]; then nl -ba pyproject.toml | sed -n '1,160p'; fi
if [ -f uv.lock ]; then wc -l uv.lock && head -80 uv.lock; fi

echo
echo "== search for uv sync usage =="
rg -n "uv sync|uv run|uv pip|lockfile|locked|frozen" .

Repository: juspay/clairvoyance

Length of output: 367


🏁 Script executed:

#!/bin/bash
set -euo pipefail

echo "== dockerfile context =="
sed -n '1,120p' dragontts/Dockerfile

echo
echo "== project files =="
for f in pyproject.toml dragontts/pyproject.toml uv.lock dragontts/uv.lock; do
  if [ -f "$f" ]; then
    echo "--- $f ($(wc -l "$f" | awk '{print $1}') lines)"
    sed -n '1,180p' "$f"
  fi
done

echo
echo "== uv sync/frozen/locked usage =="
rg -n "uv sync|uv run|uv pip|lockfile|locked|frozen" -S .

Repository: juspay/clairvoyance

Length of output: 50376


Validate the DragonTTS lockfile in the final image layer.

COPY --from=ghcr.io/astral-sh/uv:0.5.11 makes uv available in the Dockerfile, but there is no uv sync --locked after copying dragontts/; after dependencies change, a stale dragontts/uv.lock can still build an environment that no longer matches dragontts/pyproject.toml.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@dragontts/Dockerfile` at line 22, Update the DragonTTS dependency
installation step using uv sync so the final image validates that uv.lock
matches pyproject.toml, replacing the frozen-only mode with the locked
validation mode after dragontts/ is copied.


RUN useradd --uid 1000 --create-home appuser && chown -R appuser:appuser /app
USER appuser

EXPOSE 8000

CMD ["uv", "run", "uvicorn", "app.main:app", "--host", "0.0.0.0", "--port", "8000", "--workers", "4", "--loop", "uvloop", "--no-access-log", "--limit-concurrency", "1024"]
48 changes: 48 additions & 0 deletions dragontts/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,48 @@
# dragonTTS

TTS caching proxy (FastAPI): caches text-to-speech audio to cut latency and cost
on repeated synthesis. Multi-provider (Cartesia / ElevenLabs / Sarvam / Gemini),
SQLite + filesystem store, write-through caching with length-scaled TTL.

## Run locally

The folder is self-contained — a venv, deps, and the `app` package all live
inside it. Always `cd` here and run from this folder.

### 1. Configure env
```bash
cp .env.example .env # then fill in provider API keys in .env
```
`.env` is gitignored — it is the only place real keys go locally (in prod they
are kubectl-injected, never in the image).

Key knobs (see `.env.example` for the full list):
- `SPLIT_AT_SYMBOLS` — split `/tts/bytes` (one-shot) transcripts into per-
sentence cache entries (e.g. `.:?`). Empty = off.
- `SPLIT_AT_SYMBOLS_STREAM` — same, for `/tts/stream`.
- `ENABLE_WRITE_THROUGH=true` — store every synth into the cache.
- `CACHE_TTL_*` — length-scaled TTL (longer phrases live longer).
Comment on lines +19 to +24

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🗄️ Data Integrity & Integration | 🟡 Minor | ⚡ Quick win

Document the actual TTL variables.

The README tells operators to configure CACHE_TTL_*, but dragontts/.env.example exposes only TTL_SECONDS=0 in its full configuration list. Update the README to the actual variables or add the documented keys so operators can configure the intended TTL behavior.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@dragontts/README.md` around lines 19 - 24, Update the README’s CACHE_TTL_*
documentation to match the actual TTL configuration exposed by
dragontts/.env.example, specifically TTL_SECONDS, or add the documented
CACHE_TTL_* keys to the environment configuration. Ensure operators can identify
and configure the variables controlling cache TTL behavior.


### 2. Install + run (uv)
```bash
uv sync
uv run uvicorn app.main:app --reload
```
Plain venv (no uv): `python3.11 -m venv .venv && source .venv/bin/activate &&
pip install -e . && uvicorn app.main:app --reload`

Server starts on http://127.0.0.1:8000 → try `GET /health`.

## Endpoints
- `POST /tts/bytes` — one-shot synthesis (cached, returns μ-law 8 kHz).
- `POST /tts/stream` — chunked streaming synthesis (raw pcm_s16le @ 16 kHz).
- `POST /cache/clear` — clear the cache at runtime (don't delete `data/` while
the server runs).
- `GET /health` — liveness + configured providers.
- `GET /stats`, `GET /stats/daily`, `GET /stats/latency` — cache economics.
- `POST /slack-summary` — manually trigger the daily Slack summary.

## Notes
- Cache store lives in `data/` (SQLite db + audio blobs) — gitignored; keep it
on a persistent volume in prod.
- Split parts are pure-concatenated (no audio is trimmed/cut).
3 changes: 3 additions & 0 deletions dragontts/app/__init__.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
"""DragonTTS — caching TTS proxy compatible with clairvoyance providers."""

__version__ = "0.1.0"
1 change: 1 addition & 0 deletions dragontts/app/alerts/__init__.py
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
"""Slack alerting (incoming-webhook Block Kit sender + daily cache summary)."""
Loading
Loading