From 7b41af60df56cb9e0954da294f1dd6fe8746fe8f Mon Sep 17 00:00:00 2001 From: Laurent Guerard Date: Wed, 24 Jun 2026 16:25:51 +0200 Subject: [PATCH] =?UTF-8?q?refactor(workflow):=20=E2=99=BB=EF=B8=8F=20guar?= =?UTF-8?q?d=20the=20stage=20store=20with=20a=20touch()=20sentinel?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Replace the prepare rule's directory(STAGE) output with a touch()ed STAGE_OK marker (sopa's sentinel discipline). Snakemake then tracks a tiny leaf file instead of a directory, so it no longer deletes/recreates (or "moves") the stage store on a re-run and is immune to directory-mtime quirks. segment depends on the sentinel; scripts open the store by path. With this + marker outputs + rerun-triggers: mtime, touching any script yields "Nothing to be done", and losing one tile re-runs only that tile and the merge — convert/prepare stay cached. Co-Authored-By: Claude Opus 4.8 --- workflow/rules/common.smk | 4 ++++ workflow/rules/segment.smk | 4 ++-- 2 files changed, 6 insertions(+), 2 deletions(-) diff --git a/workflow/rules/common.smk b/workflow/rules/common.smk index ac92e65..03f6c3e 100644 --- a/workflow/rules/common.smk +++ b/workflow/rules/common.smk @@ -9,6 +9,10 @@ IMAGE = f"{WORK}/image.zarr" IMAGE_OK = f"{IMAGE}/zarr.json" TILES = f"{WORK}/tiles.json" STAGE = f"{WORK}/stage.zarr" +# Completion sentinel for the stage store. Tracking a touch()ed marker instead +# of directory(STAGE) keeps Snakemake from deleting/recreating the store on a +# re-run and avoids directory-mtime quirks (same touch() discipline as sopa). +STAGE_OK = f"{STAGE}.done" def occupied_done(wildcards): diff --git a/workflow/rules/segment.smk b/workflow/rules/segment.smk index 3d3328f..2111868 100644 --- a/workflow/rules/segment.smk +++ b/workflow/rules/segment.smk @@ -5,7 +5,7 @@ checkpoint prepare: IMAGE_OK, output: tiles=TILES, - stage=directory(STAGE), + stage=touch(STAGE_OK), script: "../scripts/prepare_tiles.py" @@ -14,7 +14,7 @@ rule segment: """Segment one tile on a GPU and write it into the stage store.""" input: tiles=TILES, - stage=STAGE, + stage=STAGE_OK, image=IMAGE_OK, output: f"{WORK}/seg/{{index}}.done",