Skip to content

Collect nf-test runtime reports - #2265

Draft
FriederikeHanssen wants to merge 2 commits into
devfrom
codex/ci-test-observability
Draft

Collect nf-test runtime reports#2265
FriederikeHanssen wants to merge 2 commits into
devfrom
codex/ci-test-observability

Conversation

@FriederikeHanssen

@FriederikeHanssen FriederikeHanssen commented Aug 12, 2026

Copy link
Copy Markdown
Contributor

What changed

  • Collect each nf-test job's CSV report as a short-lived artifact.
  • Merge successful observations into stable per-test median, P75 and maximum runtimes.
  • Publish a concise slow-test table in the workflow summary and a machine-readable artifact for later shard planning.

Why

The current suite has a long and uneven tail, but the workflow does not retain enough timing data to tune it safely. This PR adds measurement only; it does not remove tests or change required coverage.

Impact

This is the first PR in a stack. The next PR consumes the generated timing format to balance CPU shards by expected runtime.

Generated by Codex

Checks

  • python -m unittest tests/ci/test_summarise_runtimes.py
  • python -m py_compile tests/ci/summarise_runtimes.py tests/ci/test_summarise_runtimes.py
  • prek run
  • nf-core pipelines lint
  • Full nf-test execution is intentionally delegated to this draft PR's matrix because the local all-tests run did not complete in a useful time window.

PR checklist

  • This comment contains a description of changes (with reason).
  • If you've fixed a bug or added code that should be tested, add tests!
  • Make sure your code lints (nf-core pipelines lint).
  • Ensure the test suite passes (nextflow run . -profile test,docker --outdir <OUTDIR>).
  • Check for unexpected warnings in debug mode (nextflow run . -profile debug,test,docker --outdir <OUTDIR>).

@github-actions

github-actions Bot commented Aug 12, 2026

Copy link
Copy Markdown

nf-core pipelines lint overall result: Passed ✅ ⚠️

Posted for pipeline commit 920e5ef

+| ✅ 237 tests passed       |+
#| ❔  15 tests were ignored |#
!| ❗   8 tests had warnings |!
Details

❗ Test warnings:

  • nextflow_config - Config manifest.version should end in dev: 3.10.0
  • pipeline_todos - TODO string in main.nf: Optionally add in-text citation tools to this list.
  • pipeline_todos - TODO string in main.nf: Optionally add bibliographic entries to this list.
  • pipeline_todos - TODO string in main.nf: Only uncomment below if logic in toolCitationText/toolBibliographyText has been filled!
  • pipeline_todos - TODO string in base.config: Check the defaults for all processes
  • schema_lint - Input mimetype is missing or empty
  • schema_description - No description provided in schema for parameter: markduplicates_pixel_distance
  • schema_description - No description provided in schema for parameter: gatk_pcr_indel_model

❔ Tests ignored:

✅ Tests passed:

Run details

  • nf-core/tools version 4.1.0
  • Run at 2026-08-12 10:38:26

maxulysse
maxulysse reviewed Aug 12, 2026
View reviewed changes

@maxulysse maxulysse left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LOVING IT

This was referenced Aug 12, 2026
Draft
Draft
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@maxulysse maxulysse maxulysse left review comments

At least 1 approving review is required to merge this pull request.

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@FriederikeHanssen @maxulysse