Skip to content

test: split e2e tests into integration + e2e pairs #729

Description

@planetf1

Context

Part of the Testing Infrastructure & Strategy Overhaul epic (item 2a). Depends on granularity markers (#727) and marker audit (#728).

Objective

Audit current e2e tests and create mocked integration variants for CI and local dev, keeping the original e2e tests for nightlies. This follows the agreed "split, don't just downgrade" principle — both variants may diverge over time to focus on their respective concerns.

The pattern to follow is #567 (test_astream_mock.py).

Work

  • Inventory all current e2e tests and identify candidates for integration variants
  • Create mocked integration variants for each candidate (using the test_astream_mock.py pattern)
  • Ensure integration variants run in PR CI and local dev (<15 min budget)
  • Ensure original e2e tests continue to run in nightlies
  • Apply correct granularity markers (integration / e2e) to both variants per test: define and implement granularity marker taxonomy #727
  • Update contributor documentation on when to create both variants vs. a single test

Notes

  • This covers both test/ and example scripts
  • Do this before 2b (parametrisation) — it delivers immediate CI coverage value and builds understanding of the test landscape
  • Integration variants should use mocked backends, not just lighter models

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    p2Medium/low: minor bugs, niche features, polish, docs, tests, cleanup. Scoped, lower urgency.testing

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions