Integrate model, service, layer, and API changes into the core backend module

[maraxen]

Core Modules:

• asset_lock_manager.py - Asset synchronization
• asset_manager.py - Asset lifecycle management
• celery_tasks.py - Task definitions
• celery.py - Celery configuration
• container.py - DI container (already exists!)
• decorators.py - Cross-cutting concerns
• orchestrator.py - High-level coordination
• protocol_code_manager.py - Protocol management
• protocol_execution_service.py - Execution logic
• run_context.py - Execution context management
• scheduler.py - Task scheduling
• workcell_runtime.py - Runtime management
• workcell.py - Workcell abstraction

Dependency Analysis Required

Phase 0: Dependency Mapping

• Analyze import statements in each core module
• Create dependency graph showing actual relationships
• Identify circular dependencies (if any)
• Document global state usage patterns
• Assess existing container.py implementation

Refactoring Strategy

Phase 1: Foundation Layer

Start with most isolated modules

• decorators.py - Should be dependency-free
• container.py - Enhance existing DI container
• run_context.py - Remove global state, use DI

Phase 2: Core Services

Modules with minimal dependencies

• asset_lock_manager.py - Inject database/redis dependencies
• protocol_code_manager.py - Inject storage dependencies
• celery.py - Configuration as injectable service

Phase 3: Business Logic

Core domain services

• asset_manager.py - Depends on asset_lock_manager
• workcell.py - Core domain model
• scheduler.py - Depends on celery config

Phase 4: Runtime & Execution

Complex orchestration modules

• workcell_runtime.py - Depends on workcell, asset_manager
• protocol_execution_service.py - Depends on multiple services
• celery_tasks.py - Task implementations with DI

Phase 5: Orchestration

Highest-level coordination

• orchestrator.py - Coordinates all services

Implementation Guidelines

Dependency Injection Patterns
Container Enhancement Strategy

• Audit existing container.py capabilities
• Add scoped lifetime management (singleton, transient, scoped)
• Implement interface binding
• Add configuration injection support

Success Criteria

Technical Goals:

• Zero global state in core modules
• All dependencies injected via constructor
• Clean separation of concerns
• No circular dependencies
• 100% type annotations with protocols
• All tests pass*
  *the current testing suite is out of date and incomplete, for now pyright and ruff linting errors should be resolved as much as possible

Quality Gates:

• Dependency graph is acyclic
• Each module has single responsibility
• Interfaces are well-defined with protocols
• Configuration is externalized
• Error handling is consistent

Risk Mitigation
Potential Issues:

• Existing circular dependencies may require interface extraction
• Global state removal might break existing integrations
• Celery task registration may need special handling

Mitigation Strategies:

Create feature flags for gradual rollout
Comprehensive integration testing at each phase
Testing Strategy

• Unit tests for each refactored module
• Integration tests for service interactions
• End-to-end tests for critical workflows
• Performance benchmarking before/after

Acceptance Criteria

• All core modules use dependency injection
• No global state variables
• Clean dependency graph (no cycles)
• Type-safe interfaces with protocols
• Comprehensive test coverage maintained
• Documentation updated for new architecture

Priority: High Labels: refactoring, architecture, dependency-injection, core

Next Steps:

• Approve this refactoring plan
• Complete Phase 0 dependency analysis
• Begin Phase 1 implementation with foundation layer
• Begin Phase 2
• Phase 3
• Phase 4
• Phase 5
• Final review and merge