Skip to content

**PATCH** Update to Jaffa Parsing - Bump Docker to Python 3.12 - Correct "Directory not empty: 'tmp_dir'" - #95

Open
MikeWLloyd wants to merge 11 commits into
Clinical-Genomics:masterfrom
MikeWLloyd:master
Open

**PATCH** Update to Jaffa Parsing - Bump Docker to Python 3.12 - Correct "Directory not empty: 'tmp_dir'"#95
MikeWLloyd wants to merge 11 commits into
Clinical-Genomics:masterfrom
MikeWLloyd:master

Conversation

@MikeWLloyd

Copy link
Copy Markdown
Contributor

Summary of changes

  1. JAFFA parser updates
  • Updated JAFFA parsing to support both legacy and newer JAFFA output headers.
  • Updated JAFFA parsing to support both ":" and "::" gene name delimiters in fusion names.
  • Added fallback handling for known status when the old known column is absent, using known mitelman and known cosmic fields.
  • Added optional extraction of newer metadata fields when present, including geneCounts1, geneCounts2, known_mitelman, known_cosmic, cosmic_tier, and gtex_samples.
  • Changes are in jaffa.py.
  1. JAFFA test coverage updates
  • Expanded parser tests to validate compatibility with both old and newer JAFFA formats.
  • Added fixture coverage for new JAFFA headers and expected field mapping behavior.
  • Changes are in test_jaffa_parser.py.
  1. Dockerfile fixes and runtime compatibility improvements
  • Updated base image to Python 3.12 slim to match package metadata and current code syntax requirements.
  • Added WORKDIR /app so repository contents are copied into an application directory instead of filesystem root.
  • Installed procps to provide ps, which is required by Nextflow task metrics collection.
  • Kept sqlite3 installation and pip-based package install flow.
  • Changes are in Dockerfile.
  1. Fix for Directory not empty: tmp_dir
    Stated was fixed in [MAJOR] Update COSMIC API access, add createdb command, improve tests/docs, and remove unused sync #90, but I was still seeing the error in the most recent code base via the 3.12 docker container. So to fix:
  • Hardened download cleanup logic so temporary directory handling is path-safe and idempotent.
  • Avoided fragile cleanup behavior tied to relative working directories.
  • Ensured cleanup runs from a stable cwd and tmp cleanup is done with absolute paths.
  • Also aligned QIAGEN download function signature/call path in the same flow.
  • Changes are in download.py and net.py.

Why this is needed

  • JAFFA output format changes were causing a missing field to be extraction without compatibility handling.
  • Docker builds and runtime behavior were impacted by older Python base usage and root-level copy behavior.
  • Nextflow execution requires ps; missing procps caused runtime failures.
  • Download workflow could fail during cleanup with Directory not empty: tmp_dir in containerized or repeated runs.

Validation performed

  1. Verified fusion_report entrypoint works from a local Python 3.12+ virtual environment.
  2. Confirmed CLI help command executes successfully.
  3. Confirmed Dockerfile now uses application workdir and includes Nextflow-related ps dependency.
  4. Confirmed cleanup code paths now use absolute paths and safer deletion behavior.

Notes

  • This PR does not change report-generation semantics outside JAFFA field mapping compatibility and download cleanup robustness.
  • Existing workflows using old erJAFFA output remain supported while adding support for newer JAFFA outputs.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant