Skip to content

Releases: HallquistLab/BrainGnomes

0.9-2

Choose a tag to compare

@michaelhallquist michaelhallquist released this 02 Sep 19:47
4523637
  • Make run_project() explain its pre-submission work in plain language, including subject discovery, run-record creation, and the first full read of a large container file. Large cohort submissions now report bounded progress through the subject list, and explicit subject filters are applied before session directories are inspected so small requested subsets do not scan the full project tree.
  • Extend the established setup_project() -> run_project() workflow across the complete R and command-line lifecycle without adding required setup steps. Optional inspection tooling now provides non-mutating configuration validation, doctor() preflight checks, and serializable execution plans; direct runs resolve the same execution model exposed by plans. Run handles, tracked-run views, log discovery, non-interactive diagnosis, failed-job retry planning, and guarded scheduler cancellation support observation and recovery. Flywheel controller snapshots are now run-specific so concurrent submissions cannot overwrite one another.
  • Record complete provenance before each run begins submission. Every run now retains its resolved request and subject scope, exact configuration, planned resources and dependencies, software/R/host/scheduler identity, and content fingerprints for selected containers and other execution-driving files. get_run_provenance() and the BrainGnomes provenance command expose the record together with current job-tracking rows; plan and retry origins are retained explicitly.
  • Expand applied-user recovery documentation around the established workflow. The Quickstart and diagnosis guide now show how to select one run, inspect its failed jobs, logs, and provenance, preview a retry, submit it as a separate new run, decide whether to include downstream blocked work, and safely preview cancellation. Function and CLI help use the same plain-language behavior and safety guidance.
  • Make the R recovery APIs preview-first as well: retry_project_run() and cancel_project_run() now default to dry_run = TRUE, and require an explicit dry_run = FALSE to submit or cancel work. Retry planning preserves postprocessing and extraction stream names containing underscores.
  • Add an executable motion-QC vignette for calculate_motion_outliers(). It demonstrates raw and filtered FD summaries, strict threshold interpretation, unavailable-filter handling, run-level QC/exclusion exports, and the boundary between interactive QC summaries and postprocessing scrubbing.
  • Make guided setup_project() and edit_project() saves use the same atomic configuration writer as non-interactive lifecycle tooling after the user confirms replacement of an existing file.
  • Recalibrate masked-SUSAN validation on real fMRIPrep BOLD data for the distinct no-input-mask, fMRIPrep-mask, and TemplateFlow-mask conditions. Validation now enforces the selected detrending-plus-MAD estimator, uses up to 96 timepoints deterministically distributed over the complete run (or all timepoints in shorter runs), reads only those volumes with RNifti, and cannot pass by extrapolating across input-mask, kernel-size, voxel-size, or sampling support. Calibration retains the full-run SUSAN threshold, temporal mean, and extents while estimating smoothness from the selected timepoints.
  • Validate the promoted 3--8 mm masked-SUSAN models on independent fMRIPrep 25.2.5 derivatives and a 96-volume postprocessing E2E fixture drawn across the complete run. The 10 mm stress kernel remains outside the supported calibration range.
  • Allow Slurm and PBS fsaverage setup to copy with GNU cp when newer fMRIPrep containers do not provide rsync, while retaining the existing rsync path when present.
  • Strengthen postprocessing validation so masking is replayed exactly on 32 deterministic volumes distributed across the complete run (or every volume in shorter runs), interpolation preserves retained volumes and matches sampled natural-spline values, removed volumes match the censor vector in order, and AROMA/confound regression samples are deterministic, pre-selected, and spatially balanced across image resolutions.
  • Make temporal-filter validation deterministic and pre-selected, require finite per-voxel stopband and passband evidence, verify that no-noise-IC AROMA output is actually unchanged, reject wholly invalid AROMA component requests, and fail every image validator when spatial NIfTI grid metadata change unexpectedly.
  • Install multitaper as a required runtime dependency so strict temporal-filter validation is available in ordinary package installations. Preserve target qform/sform matrices and coordinate-system codes when resampling TemplateFlow masks, and automatically rebuild incompatible cached masks rather than failing later validation.
  • Reduce peak memory use in image_quantile() by avoiding a second full image copy and whole-image inclusion bitmap; masked quantile values and numerical behavior are unchanged.
  • Route postprocessing checks through a common validation runner: validator errors now obey the configured continue/stop policy, reused intermediates are checked, structured results are retained in a JSON audit beside the subject log, and final images remain staged until last-step validation completes.

0.9

0.9

Choose a tag to compare

@michaelhallquist michaelhallquist released this 26 Aug 12:50
  • Document stage-specific runtime requirements in package metadata, the README, and Quickstart. Add an installed miniature project configuration and an executable local-onboarding vignette that clearly separates no-cluster helpers, submission-free dry runs, and scheduler/container-dependent project execution.
  • Correct ROI-connectivity provenance and execution: estimator-specific filenames are now unique, scheduled extraction honors nested correlation settings, cor.shrink has a stable BIDS entity, and time-series-only extraction supports cor_method = "none".
  • Preserve the one-to-one association between multiple postprocessing input streams and their BIDS descriptions during ROI extraction. Ambiguous vector lengths now fail explicitly instead of selecting or combining unintended inputs.
  • Apply interactively configured ROI masks, preserve every atlas label when an ROI is fully masked, and return schema- and dimension-stable time-series, connectivity, and diagnostic outputs for empty ROIs and entirely masked atlases.
  • Report per-stream ROI-extraction state in project and subject status. Scheduled extraction now writes an explicit manifest of its actual time-series, connectivity, and diagnostic outputs so completion checks do not depend on a directory-wide snapshot.
  • Honor save_ts = FALSE in scheduled extraction and reject contradictory extraction configurations before output is created.
  • Align the CLI, run_project() help, examples, and vignettes with the seven supported submitted stages. BIDS validation remains project-configured but is submitted separately with run_bids_validation(); stream selection and dry-run output now expose resolved settings.
  • Harden public and native interfaces with complete help signatures, working examples, stable empty data-frame schemas, finite image-quantile validation, zero-length and dimension checks, and clearer argument errors.
  • Keep development-only calibration resources, local Codex files, audit reports, prior build products, and Python bytecode caches out of source packages and installed-package tests. Remove the obsolete ROI_TempCorr.R entry point after migrating its useful voxel-retention diagnostics into supported ROI extraction.
  • Stabilize empty-result contracts: extract_bids_info(character()) now returns its complete typed BIDS schema, and get_project_status() returns configured status columns even before any subject jobs exist. image_quantile() now rejects empty, missing, NaN, and infinite probability vectors before reading image data.
  • Improve onboarding and release hygiene: generate Quickstart CLI help from the installed command, document the standalone BIDS-validation boundary consistently, show resolved postprocessing and extraction stream settings during dry runs, and remove the obsolete ROI_TempCorr.R installed entry point after migrating its useful diagnostics into extract_rois().
  • Add optional per-ROI voxel-retention diagnostics to ROI extraction. Reports
    separately track atlas size, optional-mask survival, BOLD-valid voxels,
    minimum-voxel requirements, retention status, and exclusion reasons; the
    scheduled extraction workflow includes requested diagnostics in its explicit
    output manifest.
  • Preserve source FD in notch-filtered calculated confounds and write the
    recomputed series immediately beside it as framewise_displacement_filtered.
    The output respects the configured header setting and logs column order when
    headers are disabled; filtering-based scrubbing and confound regression use
    the filtered FD. Clearly reversed notch bounds are repaired automatically;
    skipped filters are logged and never produce a misleading filtered-FD label.
    When FD is selected as a processed confound, both the source-derived and
    notch-derived FD columns retain all configured BOLD-matched confound
    processing.
  • Add a voxel_psc intensity-normalization mode that uses the existing robust reference-core and eligible-frame policy and applies denominator-guarded baseline-to-100 scaling after spatial processing. Reliable local baselines use ordinary PSC scaling, very low positive baselines use a lower denominator bound, and invalid baselines or those with too few eligible frames use a conservative run-level fallback. The guards do not clip observations or mask voxels; the user's apply_mask decision is preserved, and the multiplier map and guard counts are saved for provenance. Guard counts and percentages within the conservative automask are logged at info level, with complete-grid counts at debug level.
  • Add a user-oriented intensity-normalization vignette documenting the target convention, robust reference-core policy, provenance outputs, QA, and limitations.
  • Replace automask()'s background-sensitive positive-voxel quantile interpolation with an iterative AFNI-style clip estimator and a smoothly varying local threshold field.
  • Match AFNI's automask() peeling more closely with a 17-of-18 NN2 survival rule, layer-aware restoration, and post-peel face-connected reclustering.
  • Replace postprocessing's late 4D-median intensity estimate with an automask-based robust reference core selected from the original positive-scale BOLD image; measure and apply the run factor after masking/smoothing but before AROMA, temporal filtering, confound regression, or timepoint removal, and save the core mask and JSON provenance.
  • Accept postprocess/intensity_normalize/target as the simplified normalization setting while retaining global_median as a backward-compatible alias.
  • Preserve each voxel's pre-AROMA temporal mean during both aggressive and non-aggressive AROMA denoising, retaining the positive baseline intensity used for cross-run scaling.
  • Refactor postprocessing to use job arrays and sentinels for cleanup
  • Add additional templates to prefetch needed by MRIQC
  • Preserve user-specified metadata/sqlite_db values and expose sqlite_db in edit_project().
  • Clean postprocessing scratch workspaces and temporary automask files on errors as well as successful exits.
  • Use exit-time cleanup for temporary FSL postprocessing files generated during temporal filtering, smoothing, confound regression, and brain-mask computation.
  • Add regression tests for editable SQLite database configuration and postprocessing temp-file cleanup after failures.
  • Prefetch resolution-1 T1w and brain-mask assets used by fMRIPrep anatomical reports, including when an output space explicitly requests another resolution.
  • Recalibrate spatial-smoothing validation on held-out BOLD runs from three datasets and resolutions; expected post-smoothing FWHM now conditions on baseline smoothness, program/mask mode, and the voxel-to-kernel ratio.

0.8-2

Choose a tag to compare

@michaelhallquist michaelhallquist released this 14 Jul 18:42
  • Add a voxel_psc intensity-normalization mode that uses the existing robust reference-core and eligible-frame policy and applies denominator-guarded baseline-to-100 scaling after spatial processing. Reliable local baselines use ordinary PSC scaling, very low positive baselines use a lower denominator bound, and invalid baselines or those with too few eligible frames use a conservative run-level fallback. The guards do not clip observations or mask voxels; the user's apply_mask decision is preserved, and the multiplier map and guard counts are saved for provenance. Guard counts and percentages within the conservative automask are logged at info level, with complete-grid counts at debug level.
  • Add a user-oriented intensity-normalization vignette documenting the target convention, robust reference-core policy, provenance outputs, QA, and limitations.
  • Replace automask()'s background-sensitive positive-voxel quantile interpolation with an iterative AFNI-style clip estimator and a smoothly varying local threshold field.
  • Match AFNI's automask() peeling more closely with a 17-of-18 NN2 survival rule, layer-aware restoration, and post-peel face-connected reclustering.
  • Replace postprocessing's late 4D-median intensity estimate with an automask-based robust reference core selected from the original positive-scale BOLD image; measure and apply the run factor after masking/smoothing but before AROMA, temporal filtering, confound regression, or timepoint removal, and save the core mask and JSON provenance.
  • Accept postprocess/intensity_normalize/target as the simplified normalization setting while retaining global_median as a backward-compatible alias.
  • Preserve each voxel's pre-AROMA temporal mean during both aggressive and non-aggressive AROMA denoising, retaining the positive baseline intensity used for cross-run scaling.
  • Refactor postprocessing to use job arrays and sentinels for cleanup
  • Add additional templates to prefetch needed by MRIQC
  • Preserve user-specified metadata/sqlite_db values and expose sqlite_db in edit_project().
  • Clean postprocessing scratch workspaces and temporary automask files on errors as well as successful exits.
  • Use exit-time cleanup for temporary FSL postprocessing files generated during temporal filtering, smoothing, confound regression, and brain-mask computation.
  • Add regression tests for editable SQLite database configuration and postprocessing temp-file cleanup after failures.
  • Prefetch resolution-1 T1w and brain-mask assets used by fMRIPrep anatomical reports, including when an output space explicitly requests another resolution.

0.8-1

Choose a tag to compare

@michaelhallquist michaelhallquist released this 12 Mar 17:57
  • Improve CLI interface to support --help or BrainGnomes help
  • Add CLI status command to get project status from command line
  • Add dry_run option to run_project to see what would be run without executing it
  • Refactor prefetch to accept cohort specifications and extend them to T2w fetch.
  • Refactor prefetch to fall back to no desc field if desc:brain fails
  • Expand fMRIPrep TemplateFlow defaults to include MNI152NLin2009cAsym boldref, res-2 brain mask, brain probseg, and carpet dseg assets observed during workflow construction
  • Add conditional CIFTI TemplateFlow defaults so prefetch only stages MNI152NLin6Asym and fsLR sphere assets when fMRIPrep CLI options request --cifti-output
  • Harden prefetch caching and validation checks so that later failures invalidate skip logic
  • Make prefetch state query-specific so that an exact snapshot of templateflow files is retained
  • Move prefetch state files out of templateflow_home and into hashed project log paths; legacy state files in templateflow_home are now migrated and removed to avoid poisoning TemplateFlow standard-space discovery.
  • Expand TemplateFlow default to desc=None for T1w to mirror some versions of fmriprep.
  • Harden check on flywheel location to accommodate missing fw command.
  • Update RSQLite connections to default to synchronous=NULL to prevent spurious warnings
  • Included OASIS30 as a default template space for prefetch because it is used by fmriprep
  • bugfix: preserve cohort-<n> in BIDS parsing/reconstruction so postprocessing can resolve cohort-qualified fMRIPrep outputs such as space-MNIPediatricAsym_cohort-2

0.8

0.8

Choose a tag to compare

@michaelhallquist michaelhallquist released this 25 Feb 22:32
  • Add optional low-pass filtering of motion parameters before FD recomputation; rename notch config fields to
    bandstop_min_bpm/bandstop_max_bpm (deprecated: band_stop_min/band_stop_max).
  • All HPC jobs are now tracked in detail by an SQLite database
  • Job failures and other errors can now be investigated using diagnose_pipeline
  • Added a new vignette, "Diagnosing Pipeline Runs", that walks through get_project_status(), get_subject_status(),
    and interactive use of diagnose_pipeline()
  • Improved error logging in HPC scripts so that success and failure are indicated more clearly
  • Stale .fail files are removed when a newer .complete file exists, clarifying status of processing steps
  • Jobs now write a manifest of files and times to the job tracking database for more thorough completeness tests
  • Added optional low-pass filtering of motion parameters, matching Gratton
  • Gracefully adjust motion filtering parameters if they fall above Nyquist at this TR
  • Modify extract ROIs config to avoid input_regex and always generate it internally from postproc stream
  • Add optional header row for postprocessed confounds TSVs, configurable via postprocess YAML and validated during setup
  • Added extensive checks on write/permission issues with directories and files
  • bugfix: Get CSF probseg image for MRIQC during prefetch
  • run_project() now skips TemplateFlow prefetch only when a prior successful prefetch covers requested spaces and the TemplateFlow manifest in job tracking still verifies; missing/deleted template files trigger re-prefetch.
  • bugfix: preserve user-specified metadata/log_directory (including external paths) instead of always resetting to <project_directory>/logs.
  • During postprocess setup, confound_calculate now offers guided prompts to add framewise_displacement when omitted, including whether to use motion-filtered FD and whether FD should be processed vs kept as noproc for QC/exclusion workflows.
  • Increase consistency of instructions and formatting in setup_project()
  • bugfix: avoid spurious "Already disconnected" warnings on exit from diagnose_pipeline()
  • bugfix: diagnose_pipeline() now respects configured metadata/log_directory instead of assuming <project_directory>/logs
  • bugfix: diagnose_pipeline() now matches subjects by exact sub-<id> tokens to avoid accidental partial matches
  • bugfix: run_bg_and_wait() now suppresses and restores ERR trap handling around wait, so non-zero container exits can be reconciled against success tokens before jobs are marked failed.
  • bugfix: shell trap handlers now attempt a best-effort SQLite status update to FAILED before exit, reducing _fail/DB mismatch after abrupt failures.
  • bugfix: update_tracked_job_status() now warns when no tracking rows are updated for a job_id (instead of failing silently).
  • bugfix: check_status_reconciliation() now checks .fail markers against DB status and reports mismatch details.

0.7-5

Choose a tag to compare

@michaelhallquist michaelhallquist released this 13 Jan 18:30

v0.7-5

0.7-4

Choose a tag to compare

@michaelhallquist michaelhallquist released this 25 Nov 14:29

V0.7-4

0.7-3

Choose a tag to compare

@michaelhallquist michaelhallquist released this 11 Nov 20:59

Version 0.7-3

0.7-2

Choose a tag to compare

@michaelhallquist michaelhallquist released this 22 Oct 00:20
Support multisession directories for AROMA checks

0.7-1

Choose a tag to compare

@michaelhallquist michaelhallquist released this 24 Sep 13:48

v0.7-1 of package, with small tweaks to temporal filter documentation and fixes to edit_project for postprocess and roi extraction.