Update Testing Suite

[maraxen]

Issue: Update Testing Suite for Reworked Backend

Overview

The backend has undergone significant changes, introducing a new architecture with a clear separation of concerns into models, services, API, and core layers. The existing testing suite is no longer sufficient to provide confidence in the system's functionality and stability. This issue outlines a comprehensive plan to create a robust and high-coverage testing suite that reflects the new backend structure.

Our goal is to achieve near-full test coverage, ensuring that each component is individually validated through unit tests and that the interactions between components work as expected through integration tests. This phased approach will allow us to tackle the most critical parts of the application first, providing a solid foundation before moving to the central business logic.

Current Codebase Structure

The primary backend code lives in praxis/backend, with the following key directories:

• praxis/backend/models: Contains SQLAlchemy ORM and Pydantic models.
  • enums/: Enumeration classes for various models.
  • orm/: The SQLAlchemy models that define the database schema.
  • pydantic_internals/: Pydantic models for data validation and serialization.
• praxis/backend/services: Contains the business logic for interacting with models and external resources.
• praxis/backend/api: The FastAPI-based API layer that exposes endpoints.
• praxis/backend/core: The central orchestration logic, including the scheduler and asset management.
• praxis/backend/utils: Lower-level utility functions.

Existing tests are located in the tests/ directory and will be used as a starting point.

Testing Strategy and Phased Plan

We will proceed with testing in the following order:

Phase 1: Models (High Priority)

Objective: Validate the integrity of our data structures. Ensure all ORM models map correctly to the database schema and that Pydantic models correctly handle data serialization and validation, including edge cases.

Tasks:

• praxis/backend/models/orm/:
  • asset.py: Verify Asset model's columns, types, and relationships.
  • deck.py: Test Deck and DeckPosition models, including relationships.
  • machine.py: Validate Machine model's columns and relationships.
  • outputs.py: Test FunctionDataOutput and WellDataOutput models.
  • plr_sync.py: Verify PLRSync model's columns and relationships.
  • protocol.py: Test Protocol and ProtocolAsset models.
  • resource.py: Validate Resource model's columns.
  • schedule.py: Test Schedule and related models.
  • user.py: Verify User model's columns.
  • workcell.py: Test Workcell model and its relationships.
• praxis/backend/models/pydantic_internals/:
  • asset.py: Test Pydantic models for asset creation and updates.
  • deck.py: Validate Pydantic models for deck layout and configuration.
  • filters.py: Test filter models for correct query parameter handling.
  • machine.py: Verify Pydantic models for machine data.
  • outputs.py: Test Pydantic models for output data.
  • plr_sync.py: Validate Pydantic models for PLR sync.
  • protocol.py: Test Pydantic models for protocol creation and updates.
  • pydantic_base.py: Verify custom base model functionality.
  • resource.py: Validate Pydantic models for resources.
  • runtime.py: Test runtime configuration models.
  • scheduler.py: Verify Pydantic models for scheduling.
  • user.py: Test user-related Pydantic models.
  • workcell.py: Validate Pydantic models for workcell configuration.
• praxis/backend/models/enums/:
  • Create a single test file to confirm all enumerations in this directory are correctly defined and used.

Phase 2: Services & Utilities (High Priority)

Objective: Test the core business logic. Each service should be tested in isolation by mocking dependencies like the database session or external API calls.

Tasks:

• praxis/backend/services/:
  • deck.py: Test all CRUD operations, mocking the database session.
  • deck_type_definition.py: Verify deck type definition logic.
  • discovery_service.py: Test protocol discovery mechanisms.
  • entity_linking.py: Validate entity linking logic.
  • machine.py: Test machine service functions.
  • machine_type_definition.py: Verify machine type definition logic.
  • outputs.py: Test service functions for handling outputs.
  • plate_parsing.py: Verify plate parsing logic.
  • plate_viz.py: Test plate visualization service.
  • plr_type_base.py: Test base class for PLR types.
  • praxis_orm_service.py: Test the base ORM service.
  • protocol_definition.py: Verify protocol definition logic.
  • protocol_output_data.py: Test service for protocol output data.
  • protocols.py: Test protocol creation, validation, and saving.
  • resource.py: Test resource service functions.
  • resource_type_definition.py: Verify resource type definition logic.
  • scheduler.py: Test scheduling service logic.
  • state.py: Verify state management service.
  • type_definition_base.py: Test the base class for type definitions.
  • well_outputs.py: Test service for well outputs.
  • workcell.py: Test workcell service functions.
• praxis/backend/utils/:
  • accession_resolver.py: Test the accession resolver.
  • db.py: Test database utility functions.
  • db_decorator.py: Verify database decorator logic.
  • errors.py: Test custom error classes.
  • logging.py: Verify logging configuration and utilities.
  • notify.py: Test notification utilities.
  • plr_inspection.py: Verify PLR inspection utilities.
  • redis_lock.py: Test Redis lock implementation.
  • run_control.py: Test run control utilities.
  • sanitation.py: Verify sanitation functions.
  • type_inspection.py: Test type inspection utilities.
  • uuid.py: Verify UUID generation functions.

Phase 3: API (Medium Priority)

Objective: Verify that our API endpoints correctly handle requests, call the appropriate services, and return the expected responses. This phase focuses on integration testing.

Tasks:

• praxis/backend/api/:
  • decks.py: Test all endpoints for decks.
  • dependencies.py: Test API dependencies.
  • discovery.py: Test discovery-related endpoints.
  • global_dependencies.py: Verify global API dependencies.
  • machines.py: Test all endpoints for machines.
  • outputs.py: Test endpoints for outputs.
  • protocols.py: Test all endpoints for protocols.
  • resources.py: Test all endpoints for resources.
  • scheduler.py: Test scheduler-related endpoints.
  • workcell.py: Test workcell-related endpoints.
  • utils/crud_router_factory.py: Test the CRUD router factory.

Phase 4: Core Module (Ongoing)

Objective: As the core module is still in development, we'll focus on foundational tests that lay the groundwork for future development.

Tasks:

• praxis/backend/core/:
  • asset_lock_manager.py: Write initial tests for the AssetLockManager.
  • asset_manager.py: Test the AssetManager.
  • celery.py, celery_base.py, celery_tasks.py: Test Celery configuration and tasks.
  • container.py: Test the DI container.
  • decorators.py: Test core decorators.
  • orchestrator.py: Write initial tests for the Orchestrator.
  • protocol_code_manager.py: Test the ProtocolCodeManager.
  • protocol_execution_service.py: Test the ProtocolExecutionService.
  • run_context.py: Test the RunContext.
  • scheduler.py: Write initial tests for the Scheduler.
  • workcell.py: Test the Workcell class.
  • workcell_runtime.py: Test the WorkcellRuntime.

Definition of Done

• All code in praxis/backend/models/, praxis/backend/services/, praxis/backend/utils/, and praxis/backend/api/ achieves at least 95% line and branch coverage.
• The tests/ directory is well-structured and mirrors the praxis/backend/ directory.
• All new tests pass successfully in the CI/CD pipeline.
• Documentation is updated to reflect any new testing procedures or test-related changes.