Skip to content

Affordability: raise from 3/5 to 5/5 (enterprise quality audit) #122

Description

@Kravalg

Description

This issue was produced by the automated enterprise quality & NFR audit (2026-07-02), which audited the repository against the quality attributes from Wikipedia's "List of system quality attributes" plus AI-native development readiness. Attribute: affordability (cluster: Governance, Process & Compliance), current score 3/5. Audit justification: The template is cost-conscious in places: the CrossGuard policy pack (policy/guardrails.py, policy/vilnacrm_guardrails.yaml) makes CostCenter/Owner tagging mandatory for cost allocation, the entire toolchain is OSS (uv, ruff, ty, gitleaks, pip-audit) with no paid SaaS dependency, and all workflows use concurrency cancellation plus bounded timeout-minutes so stale runs do not burn minutes. However, there is no cloud cost-estimation gate even though docs/sre-operations.md:142 lists 'Infracost' as a CI check that does not exist anywhere in .github/workflows/, and every one of the ~25 CI jobs rebuilds the large multi-stage Dockerfile from scratch via 'make start' with zero layer caching or prebuilt image, multiplying CI compute spend on every PR (python-quality.yml alone spawns 6 rebuild jobs; pulumi-mutation.yml runs a 45-minute-budget job on every PR). Identified gaps: No infrastructure cost-estimation gate despite docs claiming Infracost exists — docs/sre-operations.md line 142 maps 'Infracost' to a 'GitHub-native only' CI check, but no Infracost workflow, config file, or action exists anywhere in .github/workflows/ or the repo root. Downstream teams consuming this template get no PR-time visibility into the dollar impact of Pulumi changes, and the documentation makes a false claim about an enforced control — a governance red flag in an enterprise audit. Every CI job rebuilds the Docker workspace from scratch with no caching — grep for 'cache' across .github/workflows/*.yml returns nothing, while 'make start' (compose build of the ~300-line multi-stage Dockerfile downloading Pulumi, AWS CLI, uv, actionlint, gitleaks, hadolint) runs in at least 16 workflow jobs per PR push (6 in python-quality.yml, 6 in security-scans.yml, 4 in pulumi-pr-guardrails.yml, plus unit/integration/policy/structural/mutation/bats/local-battery). This is pure redundant compute cost for the org and for every downstream repo created from the template. Policy pack enforces no cost-relevant sizing or budget guardrails — policy/guardrails.py covers tagging, region allowlists, S3 privacy, encryption, logging, wildcard IAM, open admin ports, and production-database safety, but nothing prevents accidentally expensive resources (e.g. oversized RDS/EC2 instance classes, unthrottled provisioned IOPS, NAT gateway sprawl) and no AWS Budgets/Cost Anomaly Detection pattern is provided in pulumi/app/ for downstream teams.

Tasks

  • Add an Infracost (or pulumi-cost-estimation equivalent) job to .github/workflows/pulumi-pr-guardrails.yml that consumes the existing .artifacts/pulumi-preview/*.json artifacts and posts a cost diff comment on PRs
  • Add an infracost.yml (or equivalent) config at the repo root and document the required INFRACOST_API_KEY secret in docs/github-actions-secrets.md
  • Until implemented, remove or correct the 'Infracost' row in docs/sre-operations.md line 142 so documented controls match reality
  • Publish the dev image to GHCR from a main-branch workflow and make CI jobs pull it by digest, falling back to build only when Dockerfile/uv.lock change
  • Alternatively add docker/build-push-action with cache-from/cache-to (type=gha) to the shared bootstrap path invoked by 'make start' in CI
  • Update docs/ci-architecture.md 'Prepared Docker Context' section to describe the caching contract
  • Add an advisory or mandatory CrossGuard policy in policy/guardrails.py + policy/pack.py that flags instance classes/storage settings above a configurable threshold in policy/vilnacrm_guardrails.yaml
  • Add a documented example (pulumi/app/ or docs/) of provisioning an AWS Budget with alert thresholds tied to the enforced CostCenter tag
  • Cover the new guardrail in tests/policies/test_policy_pack.py to keep the 100% policy coverage gate green

Acceptance Criteria

  • A PR that changes pulumi/ resources receives an automated cost-estimate summary (comment or job summary) before merge
  • Every check listed in docs/sre-operations.md's CI troubleshooting map corresponds to an actual workflow job in .github/workflows/
  • make test / make ci-pr documentation and CI check inventory are consistent with grep of the workflows directory
  • CI jobs on a docs-only or Python-only change reuse cached image layers or a prebuilt image instead of running the full Dockerfile tool-download stages
  • Median per-job 'Start development environment' step time drops measurably (e.g. below 2 minutes) and is documented
  • Downstream template consumers inherit the caching behavior without extra setup beyond documented variables
  • A Pulumi preview containing an instance class above the configured threshold produces a policy diagnostic in make test-policy and make pulumi-preview
  • policy/vilnacrm_guardrails.yaml exposes the cost threshold as configuration with a documented default
  • tests/policies suite covers the new rule and make test-policy passes at 100% coverage
  • Attribute re-scores 5/5 in a follow-up enterprise quality audit

Metadata

Metadata

Assignees

No one assigned

    Labels

    devopsdocumentationImprovements or additions to documentationenhancementNew feature or request

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions