osl-pathfinder

A tiny two-way mapping between a short id and the files it owns.

You write the filename of each kind of file once, as a template with {field} placeholders. Pathfinder then gives you clean conversions in both directions — and groups every kind under one short id:

fields  <->  id (a short label)  <->  path (per file kind)

So you stop hand-writing the code that slices an id out of a filename, or rebuilds a filename from an id, in every script. One source of truth.

from osl_pathfinder import Pathfinder

pf = Pathfinder(paths={
    "eeg":  "/data/sub-{subject}/ses-{session}/sub-{subject}_ses-{session}_run-{run}_block-{block}_eeg.fif",
    "fmri": "/data/sub-{subject}/ses-{session}/sub-{subject}_ses-{session}_run-{run}_block-{block}_bold.nii.gz",
    "t1w":  "/data/sub-{subject}/anat/sub-{subject}_T1w.nii.gz",
}, anchor="eeg")

pf.field2id(subject="007", session="1", run="2", block="3")  # -> "007-1-2-3"
pf.id2path("eeg", "007-1-2-3")                              # -> Path(".../_eeg.fif")  (must exist)
pf.id2path("t1w", "007-1-2-3")                              # -> shared T1w (subject only)
pf.path2id("eeg", some_path)                               # -> "007-1-2-3"
pf.id2path(file_id="007-1-2-3")                            # all kinds -> {"eeg": Path, "fmri": Path, "t1w": Path}

Run python example.py for a complete, self-contained tour.

Why an id?

The id is a compact, readable label for one entity (subject/session/run/block). Use it as a plot title, a dict key, a CSV column — without dragging a full path around, and without re-deriving it from a filename each time.

Pass id="..." to control exactly how ids look. For a tight, separator-free id like 7111 / 10121 (no dashes, no zero-padding), use integer fields with numeric width specs — the trailing single-digit fields keep a contiguous id unambiguous, and it still round-trips:

pf = Pathfinder(
    id="{subject:d}{session:1d}{run:1d}{block:1d}",     # -> 7111, 10121
    paths={"eeg": ".../sub-{subject:03d}/ses-{session:02d}/..._block-{block:02d}_eeg.fif"},
)
pf.field2id(subject=7, session=1, run=1, block=1)   # "7111"
pf.id2field("10121")                               # {subject:10, session:1, run:2, block:1}

The bare default joins fields with - ("007-1-2-3") only because, without a separator or fixed widths, an id can't be split back unambiguously — so supply an explicit id= (as above) whenever you want the compact form.

Ambiguity check. The constructor rejects an id template where two variable-width fields touch with no separator, because id2field couldn't recover them — e.g. "{subject:d}{session:d}{run:1d}{block:1d}" raises ValueError (subject/session are both variable-width and adjacent). Fix it by adding a separator or giving all but one a fixed width ({session:1d}).

Mixed granularity (one T1w for many blocks)

A template only uses the fields it names. t1w mentions just {subject}, so every block of a subject resolves to the same T1w file. No special-casing.

Filtering a hierarchy

Because ids are structured, you can ask hierarchical questions. Discover what's on disk with scan(), then filter:

pf.scan()                                    # enumerate ids by globbing the anchor
pf.field2id(subject="007", fuzzy=True)       # every block of subject 007
pf.field2id(session=["1", "2"], fuzzy=True)  # ids in either session
pf.id2path(file_id="007-1-1-1")              # {kind: Path} for every file that exists

(field2id builds one id from complete fields by default; fuzzy=True turns it into a disk-backed filter returning the matching id list.)

The whole API

convert — no disk	what
field2id(**fields)	complete fields → the one id
id2field(id)	id → fields
path2field(kind, path) / path2id(kind, path)	a path you already have → fields / id

entity → path	what
id2path(kind, id) / field2path(kind, fields)	→ a kind Path (see mode below)
id2path(file_id=id)	omit kind → {kind: Path} for all kinds of one id
exists(kind, key)	does at least one such file exist? (never raises)

discover — globs disk	what
scan(kind=anchor)	enumerate ids found on disk (cached; raises on duplicate)
field2id(**constraints, fuzzy=True)	filter ids by field (scalar or iterable; unknown field raises)

id2path/field2path take a mode:

• mode="find" (default) — finds an existing file. Globs the rendered path, so any placeholder you don't supply (a timestamp/hash/derived {foo}) is filled from disk; returns the single existing Path, raising if it's missing or ambiguous. The all-kinds form (id2path(file_id=id)) skips kinds with no file. This is the point of a pathfinder.
• mode="build" — builds an output path for a file you're about to write. Renders the exact path, touches no disk, and returns a Path that need not exist. Every placeholder must be supplied — an unresolved one raises (never a * in an output path).

A key is either a short id string or a dict of fields (or keyword args).

Pipelines: define once, read and write

Construction touches no disk, so define every kind up front in one place and import it in each step. Read inputs with the default find; write outputs with build — the path logic lives in exactly one spot:

# config.py
pf = Pathfinder(paths={"raw": ..., "src": ..., "power": ...}, id="...")

# 1.py   (src doesn't exist yet)
for fid in pf.ids:
    raw = pf.id2path("raw", fid)                 # find existing input
    save(transform(raw), pf.id2path("src", fid, mode="build"))   # build output

# 2.py   (now both exist)
for fid in pf.ids:
    raw, src = pf.id2path("raw", fid), pf.id2path("src", fid)    # both found

Placeholders that aren't id fields (e.g. {foo})

A path template may contain placeholders beyond the id fields — a timestamp, hash, or any varying part you don't want to track. They are simply globbed:

pf = Pathfinder(
    paths={"preproc": "/deriv/sub-{subject}/sub-{subject}_{foo}_preproc.fif"},
    id="{subject}",
)
pf.id2path("preproc", "007")   # globs sub-007_*_preproc.fif -> the real Path

There's no special naming rule — any placeholder that isn't an id field (and that you don't pass in) becomes *. path2field can still recover its value from a real path if you want it; it just never enters the id.

Differently-named segments are not derived fields

A path segment named differently from your id field is still a plain template — it's the same field wrapped in different literal text. Say your fMRI dirs are sub-001_ses-01_scan-01_part-01, where scan == run and part == block. Just write them as literals:

"fmri": ".../sub-{subject:03d}_ses-{session:02d}_scan-{run:02d}_part-{block:02d}/bold.nii.gz"

No derive needed — it's an ordinary template.

derive — only for a computed value (rare)

Use derive solely when a path needs a value it must compute from the id and that appears in no other filename:

def derive(fields, kind):
    if kind != "clean":
        return {}
    return {"group": "odd" if int(fields["subject"]) % 2 else "even"}

pf = Pathfinder(paths={...}, anchor="raw", derive=derive)

Notes & contracts

• The anchor defines an entity and is what scan() globs; it defaults to the first kind in paths (insertion order), so use the finest-grained modality or pass anchor= explicitly.
• scan() results are cached, not live. ids/field2id(fuzzy=True) scan once on first use and reuse that result; call scan() again after files appear to refresh.
• Disk globs are re-run every call by default. find-mode lookups, exists, and scan hit the filesystem each time so they always see current state. Pass Pathfinder(..., cache=True) to memoise globs (keyed by rendered pattern) when you query the same entities repeatedly; call refresh() to drop the cache (and the scan id cache) after files change on disk.
• Keep each field in one canonical form (e.g. always "007") and it round-trips trivially. Format specs are applied on render only — a fill/align spec like {subject:0>3} pads 007 but parsing returns the literal text, so it does not strip back to 7. If you genuinely need stripped⇄padded, use a numeric d type ({subject:03d} / {subject:d}) with integer field values — then id2field/path2field return ints for those fields (the value type follows the spec).
• One physical file can back many ids (shared T1w, polhemus, etc.).
• id2path(file_id=id, mode="build") (all kinds) only works when every template's placeholders are id fields (or passed in extra) — build won't invent an untracked {foo}.

Install

pip install -e .          # editable, for local dev
pip install -e .[test]    # + pytest
pytest -q

Requires Python ≥ 3.8 and parse.