Skip to content

E2E: order-dependent row assertion fails ~1 spec per run and blocks 9 PRs #1441

Description

@rubenvdlinde

Nine open PRs are red on quality / E2E Tests (Playwright) for the same reason, none of them related to the change they carry — including a phpstan-only dependency bump (#1424) and an app-id resolution fix (#1437).

Symptom

Consistently 1 failed / ~306-308 passed. The failure is always an assertion that a positionally addressed row contains a type keyword:

Error: expect(locator).toContainText(expected) failed
Locator: locator('#content-vue').locator('table tbody tr').nth(10)
Expected pattern: /request/i
Timeout: 15000ms
Locator: locator('#content-vue').locator('table tbody tr').nth(4)
Expected pattern: /complaint/i

The failing row index changes between runsnth(10) in one, nth(4) in another. That is the signature of an order-dependent assertion, not a stable defect in the code under test.

Where

tests/e2e/spec-coverage/request-management.spec.ts (and the sibling contactmoment/complaint specs) assert that every visible row matches the subtype:

const rows = content.locator('table tbody tr')
const count = await rows.count()
for (let i = 0; i < count; i++) {
    await expect(rows.nth(i)).toContainText(/request/i)
}

The quick filter itself looks correct — src/manifest.json maps the Tickets tab to {"ticketType": "request"}, so the helper's name and comment are accurate. So one of these is true, and the spec cannot tell them apart:

  1. the filter is applied server-side but the rendered page includes a row it should not (paging/ordering), or
  2. a seeded row genuinely has no ticketType text in any visible column, or
  3. the filter is applied client-side over server-paged data, so the page boundary leaks a row.

Option 3 has bitten this fleet before.

Why it matters beyond the flake

The assertion is all-rows-must-match, which couples the test to how many rows fit on page 1 and in what order they arrive. It passed while the seed happened to contain nothing else on the first page. It is not measuring "the filter works" — it is measuring "the first page happens to be homogeneous".

A version that asserts the actual requirement would be stable and stricter:

  • assert at least one row matches the subtype (the list is not empty and is on the right type), and
  • assert no row matches a different subtype (the filter excludes), rather than iterating positions.

Blocked PRs

#1424, #1425, #1427, #1429, #1430, #1431, #1434, #1436, #1437

development itself is green (309 passed on the 11:28 run), which is consistent with an order-dependent assertion rather than a regression on any branch.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

triageAwaiting triage

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions