Dealing with output for applytrasform

[md06451]

Hi Noah,
ANTs: Apply Transforms is an analysis gear and saves the output under analyses, which makes it very tedious to download the data.
I will need to download the scans. Considering the big size of the data-set, what would you suggest to facilitate the download of the data? could changing the gear to utility gear be a solution? could we otherwise make a script to download the data from analyses section?

Thanks in advance
Martina

[noah10]

We could make a script to do this. How would you want to specify which analysis you want to download the output of? Something like this?

download-analysis-outputs --project my-project-name --subject 120 --session pre --analysis-label myRecentApplyTransformAnalysis --output transformOutput.tar

(Which would download all of the output files for the analysis "myRecentApplyTransformAnalysis" for subject 120, session pre, in the project "my-project-name" and save them into a tar file named transformOutput.tar.)

[md06451]

Hi Noah,
few questions:

1. Would myRecentApplyTransformAnalysis" be equivalent to "Step6_Dag_Pipline_FSEnative_movingto_FseMNI_applytrasforms" in this example https://uscdni.flywheel.io/#/dashboard/collection/67356b12b53624e7f982b12c/session/6738f6154a27f89b4282b0e7?tab=analyses?
2. I guess then we could easily specify all the subject names and project names where the scans belong to. in this way we can esily download evreythign. correct? if yes, this works great.
3. Random question why would you use tar rather than zip? In my understanding, zip compresses each file individually, so files can be accessed randomly without decompressing the entire archive. But you know better, either works for me.

[noah10]

1. Yes
2. Maybe. If you want to download all outputs from all analyses named "Step6_Dag_Pipline_FSEnative_movingto_FseMNI_applytrasforms" then it might be better to do something like this:

download-analysis-outputs --project my-project-name --analysis-label Step6_Dag_Pipline_FSEnative_movingto_FseMNI_applytrasforms

...which would walk through every session in the project named "my-project-name", see if it has any analyses named "Step6_Dag_Pipline_FSEnative_movingto_FseMNI_applytrasforms" and, if so, download the output files from them. The name of the tar file would be generated automatically based on the subject and session labels.

Do you mainly want to use this tool to download the output of individual analyses or of lots of different analyses all at once?

3. The flywheel API has a convenient method to download all of the output files as a single tar file.

[md06451]

I foresse I need this for only the one step/analysis. Also would it be possible to make it such as that we can specify what file we want to download? applytrasforms saves both json and nii.gz files. I think we will be interested only in nii.gz files

[md06451]

Also when we saved/download things, does it save the data based on the project in different sub-folders? just to make sure it is easy to identify to which study the scan belongs as some participant from different studies have the same id

[noah10]

Let's take Step6_Dag_Pipline_FSEnative_movingto_FseMNI_applytrasforms as an example. There will be about 700 different analyses with that name, correct? And you want to use this tool to download the outputs of all 700 of them, correct?

[md06451]

yes exactly. Step6_Dag_Pipline_FSEnative_movingto_FseMNI_applytrasforms will have two outputs: 1 in .nii.gz format and 1 in .json format. I would be interested in downloading only .nii.gz. If not possible we can download everything