Fix async scan state recovery (#168)

[ritiksah141]

Summary

Hardens async scan execution so scan state survives Render restarts and stale running scans recover through the PostgreSQL-backed queue. Addresses #168 by making interrupted worker runs retryable before they are marked failed.

Before merging

No repo-admin action is required for this PR.

This change adds a schema migration through the existing startup-safe run_migrations() path. Deployments should run the normal startup flow so the attempt_count column is added before workers claim scans.

What changed

Persistent scan retry state (#168)

• Added attempt_count to the scans table for fresh databases and existing deployments.
• New scans start with attempt_count = 0.
• Worker claims now increment attempt_count and clear stale error_message values when a pending scan moves to running.
• Completed scans clear error_message so old retry messages do not remain attached to successful scan records.

Render restart recovery

• recover_stale_scans() now returns stale running scans to pending while retry attempts remain.
• Scans only move to failed after reaching the max attempt count, preventing infinite retry loops for persistent Azure credential or permission failures.
• Retried scans clear claimed_at, allowing the worker to claim them again with the existing FOR UPDATE SKIP LOCKED flow.

Tests

• Added regression coverage proving POST /api/scans/trigger persists a pending row through DatabaseManager.
• Added coverage proving GET /api/scans/<scan_id> reads durable database status instead of consulting memory.
• Added coverage for attempt counting, retry-before-limit behavior, and final failure after the retry limit.

Docs

• Updated docs/async-scan-architecture.md to explain how queued and running scans behave across Render web or worker restarts.

Notes for reviewers

• This keeps the existing PostgreSQL-backed queue design rather than introducing Redis, Celery, or another queue service. That matches the current architecture and avoids new infrastructure for long-running scan jobs.
• The API response shape is unchanged. Frontend polling continues to use GET /api/scans/<scan_id> and sees the same pending, running, completed, or failed statuses.
• In-flight scans are not resumed from the exact rule where they stopped. They are safely re-queued and re-run by the worker, which is the simpler and more reliable behavior for the current scan engine.

Local verification

• ruff check .
• ruff format --check .
• bandit -r api/ scanner/ ai/ -ll
• pip-audit -r requirements.txt --ignore-vuln PYSEC-2025-217 --ignore-vuln CVE-2026-1839 --ignore-vuln CVE-2026-4372
• Full backend suite with coverage: 114 passed, 3 skipped, 32.14% coverage
• Frontend npm run lint and npm run build pass with existing warnings only
• Rule, playbook, and compliance validation passed for 45 rules
• SBOM generation with Syft passed locally

Local caveat

gitleaks detect --source . --no-git fails locally because an ignored .env file exists in the working tree with real local secrets. The file is not tracked and is not part of this PR. GitHub CI runs on a clean checkout, so this local-only file should not affect the PR secret-scan gate.

Files to review

File	Why
api/models/finding.py	Adds retry state and stale scan recovery behavior
tests/test_async_scan_persistence.py	Regression coverage for DB-backed async state and retries
docs/async-scan-architecture.md	Documents Render restart and retry behavior

[Vishnu2707]

Exactly what was flagged when merging #129. postgres backed retry is the right call to take from what i understand, no need for Redis at this scale. approving this!