Fix e2e tests flake connected with tooltips and task creation - #11011
Fix e2e tests flake connected with tooltips and task creation#11011archibald1418 wants to merge 11 commits into
Conversation
archibald1418
changed the title
|
app.cvat.ai nightly base_e2e https://github.com/cvat-ai/app.cvat.ai/actions/runs/31171703770 |
Codecov Report✅ All modified and coverable lines are covered by tests. 📢 Thoughts on this report? Let us know! |
|
/check |
|
✔️ All checks completed successfully |
|
@codex review |
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 30362a0026
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| }]).flat(); | ||
|
|
||
| cy.window().then((window) => { | ||
| cy.intercept('PUT', `/api/jobs/${jobId}/annotations**`).as('putAnnotations'); |
There was a problem hiding this comment.
When this setup hook calls window.cvat.server.request, the request begins asynchronously while the newly added cy.intercept is still only queued for execution inside the current cy.window().then() callback. Consequently the PUT can pass before the route is registered, leaving cy.wait('@putAnnotations') to time out and making this spec fail instead of removing the flake. Register the intercept before cy.window(), then return or wrap the request promise.
Useful? React with 👍 / 👎.
There was a problem hiding this comment.
makes sense, moved intercept to the previous line
| cy.window().then((window) => { | ||
| window.cvat.server.request(`/api/jobs/${jobId}/annotations`, { | ||
| method: 'PUT', | ||
| data: { shapes, tracks, tags }, | ||
| }); |
There was a problem hiding this comment.
Do we really need to intercept request that is made out of the test? This feature should be used against requests that are made by app itself.
We can wait for it like this, i guess it will fail if its not 2xx:
cy.window().then((window) => window.cvat.server.request(
`/api/jobs/${jobId}/annotations`,
{ method: 'PUT', data: { shapes } },
));
There was a problem hiding this comment.
We can just use cy.request(...).its('status').should('eq', 200). cy.request will always wait since it's a command, not an invocation of async function. Also, it will show in command log, adding visibility to setup
There was a problem hiding this comment.
or at least cy.wrap it
|
TODO: check features2/new_organization_pipeline.js from #11020 UPD:
attempt3 - success |
|
/check |
1 similar comment
|
/check |
|
✔️ All checks completed successfully |
|
/check |
|
❌ Some checks failed |
|
✔️ All checks completed successfully |
2 participants
(https://github.com/cvat-ai/cvat/actions/runs/30721083352/job/91425080789)
Tooltip-related failures (from cvat nightly):
case_114_backup_restore_project - after clicking on save icon, tooltip doesn’t disappear, obscures ‘Projects’ tab from view, further tests crash because of this
https://github.com/cvat-ai/cvat/actions/runs/30721083352/job/91425080802
actions_projects_models/case_114_backup_restore_project.js - after creating a task with tags, UI checks can’t find tag annotations in job, probably the render was done before write to database happened. Fix: guard PUT annotation request with intercept.
If this is not enough, will further harden this with retry-able checks of annotations' insides. This is an extra API GET request to jobs/annotations
https://github.com/cvat-ai/cvat/actions/runs/31115486464/job/92665432017
Task create failure in annotations_actions.js
A complicated task is being created in the setup, with non-trivial annotations.
annotations_actions.jsopened a job and expected to see tags, but couldn't, failing the test. This adds an intercept to PUT request, it was not guarded before and was called through a s call tocvat.server.request