[chore] flakey test in backend

[hholb]

In PR #149 we had CI fail and the first round in check-and-test due to a failing backend test. Rerunning the CI job without any changes it passed indicating the failing test is non-deterministic and somewhat flakey. This is the failing run: https://github.com/AI-Almanac/ai-almanac/actions/runs/31727665978/attempts/1

Here are the relevant logs:

=================================== FAILURES ===================================
___________________ test_supervisor_cancels_running_workload ___________________

user_id = '5136c53a-4d87-458b-824d-04d2c17bcf12'

    @pytest.mark.asyncio
    async def test_supervisor_cancels_running_workload(user_id: str) -> None:
        from ai_almanac.server.services.job_manager import execute_job, request_cancel
    
        job_id = await _insert_job(
            user_id,
            "queued",
            config={"model_name": "cancel-test"},
        )
        supervisor = asyncio.create_task(asyncio.to_thread(execute_job, job_id))
    
        from ai_almanac.server.db import get_db
    
        for _ in range(50):
            async with get_db() as conn:
                status = (
                    await conn.execute(
                        text("SELECT status FROM jobs WHERE id = :id"),
                        {"id": job_id},
                    )
                ).scalar_one()
            if status == "running":
                break
            await asyncio.sleep(0.1)
        else:
>           pytest.fail("job did not enter running state")
E           Failed: job did not enter running state

tests/test_job_manager.py:257: Failed
---------------------------- Captured stderr setup -----------------------------
INFO  [alembic.runtime.migration] Context impl SQLiteImpl.
INFO  [alembic.runtime.migration] Will assume non-transactional DDL.
=============================== warnings summary ===============================
tests/test_assistant_compare.py: 134 warnings
tests/test_chat_api.py: 26 warnings
tests/test_chat_blend_endpoints.py: 2 warnings
tests/test_turn_log.py: 4 warnings
  /home/runner/work/ai-almanac/ai-almanac/.pixi/envs/dev/lib/python3.14/site-packages/aiosqlite/core.py:63: DeprecationWarning: The default datetime adapter is deprecated as of Python 3.12; see the sqlite3 documentation for suggested replacement recipes
    result = function()

tests/test_data_sources.py::test_source_validation_does_not_persist_and_returns_inferred_metadata
  <frozen importlib._bootstrap>:491: RuntimeWarning: numpy.ndarray size changed, may indicate binary incompatibility. Expected 16 from C header, got 96 from PyObject

tests/test_data_sources.py: 2 warnings
tests/test_forecast_pipeline.py: 5 warnings
tests/test_metrics_service.py: 12 warnings
  /home/runner/work/ai-almanac/ai-almanac/.pixi/envs/dev/lib/python3.14/site-packages/xarray/backends/netCDF4_.py:96: DeprecationWarning: Setting the shape on a NumPy array has been deprecated in NumPy 2.5.
  As an alternative, you can create a new view using np.reshape (with copy=False if needed).
    data[key] = value

-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
=========================== short test summary info ============================
FAILED tests/test_job_manager.py::test_supervisor_cancels_running_workload - Failed: job did not enter running state
=========== 1 failed, 482 passed, 5 skipped, 186 warnings in 47.55s ============
Error: Process completed with exit code 1.