Self-hosted setup: multiple env validation and configuration issues

[gfirem]

During self-hosted deployment, we encountered several issues that blocked getting the platform running. All are related to env validation being too strict for self-hosted deployments where optional integrations (S3, GitHub, Kubernetes) are not configured.

1. SDK Protocol Mismatch - discover Response Format
   The documentation at docs.agent.autonoma.app/guides/environment-factory shows the discover response should return { environments: [...] }, but the actual SDK validation (SdkDiscoverResponseSchema) requires { schema: { models, edges, relations, scopeField } }. We had to update our endpoint to return both formats for compatibility.

2. Onboarding Flow Blocked at scenario_dry_run
   After a successful discover call ("Scenarios discovered successfully"), the onboarding gets stuck at the scenario_dry_run step. The system requires ScenarioRecipes to run up/down validation, but these recipes are only generated after connecting GitHub (which comes later in the flow). This creates a chicken-and-egg problem where you can't proceed to GitHub without completing dry run, but you can't complete dry run without GitHub-generated scenarios.

3. Missing /api/ Proxy Route in UI Nginx Config
   The nginx.conf.template in apps/ui/ only proxies /v1/ and /ingest/ routes to the backend. The /api/trpc/* routes were not configured, causing all tRPC API calls to return the frontend HTML instead of JSON responses. We added the missing /api/ location block to fix this.

4. Required env vars crash API on startup
   The following vars are marked z.string().min(1) (required) but are only needed for optional integrations. If not set, the API throws at startup and enters a crash loop.

Variable	Package	Integration
S3_BUCKET, S3_REGION, S3_ACCESS_KEY_ID, S3_SECRET_ACCESS_KEY	packages/storage	File storage
NAMESPACE	packages/k8s, packages/workflow	Kubernetes
GITHUB_APP_ID, GITHUB_APP_PRIVATE_KEY, GITHUB_APP_WEBHOOK_SECRET, GITHUB_APP_SLUG	apps/api	GitHub integration
GROQ_KEY, OPENROUTER_API_KEY	packages/ai	AI providers

Expected behavior: API starts normally. Features that depend on these vars throw a descriptive error only when invoked.

Affected files:

• packages/storage/src/env.ts
• packages/k8s/src/env.ts
• packages/workflow/src/env.ts
• packages/workflow/src/k8s/k8s-client.ts
• packages/k8s/src/index.ts
• packages/storage/src/providers/s3-storage.ts
• apps/api/src/env.ts
• apps/api/src/context.ts

5. Prisma v7 migration Dockerfile fails due to prisma.config.ts
packages/db/Dockerfile runs prisma migrate deploy but Prisma v7 uses prisma.config.ts which imports TypeScript dependencies (@t3-oss/env-core, zod, tsx). A minimal Docker image without these fails with:
   Error: The datasource.url property is required in your Prisma config file when using prisma migrate deploy.
   Fix: The migration Dockerfile must install prisma, tsx, @t3-oss/env-core, and zod, and copy both prisma.config.ts and src/env.ts into the image.

6. Missing /api/ proxy in nginx.conf.template
apps/ui/nginx.conf.template only proxied /v1/ and /ingest/. Requests to /api/ routes returned the SPA's index.html instead of reaching the API.

Fix: Add a location /api/ block proxying to $API_UPSTREAM.

7. BETTER_AUTH_URL expected format not documented
   BETTER_AUTH_URL must be set to the root domain (e.g. https://yourdomain.com), not including basePath. Since betterAuth is configured with basePath: "/v1/auth", setting BETTER_AUTH_URL=https://yourdomain.com/v1 causes all auth endpoints to return 404.

This is not obvious from .env.example and caused significant debugging time.

Fix: Add a comment to .env.example:
Root URL of the server only — do NOT include /v1 or any path suffix.
BETTER_AUTH_URL=https://yourdomain.com

After I make it work in our self-hosted

1. Dashboard is not detecting the GitHub app
2. The onboarding process was stuck in all steps that I was able to move from the database
3. I can't monitor any other branch for PR except main, that is not a good starting point to me
4. The autonoma agent is still running for 2 days with no way to stop it or see why it's blocked during the generation process

Environment
• Docker Compose v2
• AWS EC2 (us-east-2)
• Ploi server management
• Node 24, pnpm 10.33.0