Add ConsumerSupervisor for background task management

[aidankhogg]

Summary

This PR introduces a ConsumerSupervisor class to manage long-running consumer tasks (like DNS update processing) with automatic restart on failure. It refactors the DNS consumer from an ad-hoc background task into a supervised, managed consumer that runs after all phases complete.

Key Changes

• New ConsumerSupervisor class (netengine/core/consumer_supervisor.py):

  • Manages registration and lifecycle of long-running consumer coroutines
  • Provides automatic restart on failure with 5-second backoff
  • Graceful shutdown with cancellation handling
  • Status monitoring for all registered consumers

• Orchestrator integration (netengine/core/orchestrator.py):

  • Initialize ConsumerSupervisor in orchestrator constructor
  • Pass supervisor to PhaseContext for access by phase handlers
  • Add start_consumers() method to start all registered consumers after phases complete

• Phase context updates (netengine/handlers/context.py):

  • Add consumer_supervisor field to PhaseContext with TYPE_CHECKING import to avoid circular dependencies

• DNS consumer refactoring (netengine/phases/phase_registries.py):

  • Replace direct asyncio.create_task() with supervisor registration
  • Improve error handling with nested try-except blocks
  • Add logging for DNS update processing and error cases
  • Implement dead-letter queue (DLQ) archival for failed messages
  • Add backoff delay on consumer loop errors

• OIDC handler enhancement (netengine/handlers/oidc_handler.py):

  • Add add_token_mapper() method to configure protocol mappers for JWT claims

• Platform identity phase updates (netengine/phases/phase_platform_identity.py):

  • Create platform API client with OIDC
  • Add token mapper to include organization claim in JWT tokens
  • Store platform client ID in runtime state

• CLI updates (netengine/cli/main.py):

  • Start background consumers after all phases complete
  • Keep event loop alive to allow consumers to run
  • Graceful shutdown on Ctrl+C with consumer cleanup

Implementation Details

The ConsumerSupervisor wraps consumer coroutines in a supervised loop that automatically restarts them on failure. This provides resilience for long-running background tasks without requiring manual restart logic. The DNS consumer now properly handles message processing errors by archiving failed messages to a DLQ, improving observability and error recovery.

https://claude.ai/code/session_017q7TcBiinkpr66zwYFgjHB