tsv-gz header info to be cleared

[karl-koschutnig]

Hi Marcel,

I hope you’re doing well!

I have a quick question. We use physio-files acquired with CMRR and convert them using your bidscoiner. That works really well, but we encountered a minor issue with the tsv.gz header (warning in bids-validator)

We discovered that in this specific header (not for instance in nii.gz), there’s information about the acquisition date (or better, zip-date) and the name of the sequence is also present in the gz-header.

We’ve already made a quick fix, but it would be more efficient to directly handle this in your pipeline.

Here’s the relevant file:
https://github.com/MRI-Lab-Graz/Nextflow-dcm-converter

The file is called TSV_HeaderCleaner_v2.py.

Best, Karl

[marcelzwiers]

Hi Karl,
A little busy with a new project, so sorry for this late reply. Hope you are well too. I wanted to take a look, but if I follow the link to the relevant file, I get a 404 error....
I'll be happy to take your fix :-)
Cheers,
Marcel

[karl-koschutnig]

Hi Marcel,

Sorry, the script has been removed. Here is the mentioned script (gz_header_cleaner.py) to be found

Best, Karl

PS: Some old (bidscoiner-converted) studies also have the mtime and sequence name in the gz header (I mean in the nii.gz). This is not the case for our latest studies. Not sure what changes?

[marcelzwiers]

Hi Karl,
I don't have time now to work on bidscoin, but I had another quick look. I'm not sure exactly what you mean? In the tsv.gz file there shouldn't be a problem; and in the json there is indeed acquisition time info, but not date? And I don't see anything about sequence name at all? I'm confused with what the issue is you describe...

PS: Some old (bidscoiner-converted) studies also have the mtime and sequence name in the gz header (I mean in the nii.gz). This is not the case for our latest studies. Not sure what changes?

Probably your dcm2niix version?

Update: I found it!

        # Add some (BIDS) metadata from the DICOM header
        for key in ('Modality',
                    'Manufacturer',
                    'ManufacturerModelName',
                    'StationName',
                    'DeviceSerialNumber',
                    'SoftwareVersions',
                    'InstitutionName',
                    'InstitutionAddress',
                    'PatientPosition',
                    'BodyPartExamined',
                    'ImageType',
                    'ProtocolName',
                    'SeriesDescription',
                    'SeriesNumber',
                    'AcquisitionNumber'):

What fields exactly do you want to be removed?

[karl-koschutnig]

Thanks for getting back.
No worries, we can fix it locally - I thought it's related to bidscoin, maybe it's rather a dcm2niix-problem. I don't have much time either. Just for the record, here (one) bids-validator warning:

[WARNING] GZIP_HEADER_MTIME The gzip header contains a non-zero timestamp.

This may leak sensitive information or indicate a non-reproducible conversion process.

	/sub-107BFJM012034/ses-2/func/sub-107BFJM012034_ses-2_task-pa_acq-Fs2_run-02_bold.nii.gz
	/sub-107BFJM012034/ses-2/func/sub-107BFJM012034_ses-2_task-rest_acq-Fs2_run-01_bold.nii.gz

	286 more files with the same issue

You can close this - maybe we find the time to make a pull request.

PS: I don't think it's related to the keys you have provided. I think it's a gzip-behaviour (golang/go#17663)

[marcelzwiers]

For the record, tsv.gz files are created with pandas, so I now replaced:

physiotable.to_csv(tsvfile, header=False, index=False, sep='\t', compression='infer')

with:

    kwargs = {'compresslevel':9, 'mtime':0, 'name' if sys.version_info >= (3, 12) else 'filename':''}
    with gzip.open(tsvfile, 'wt', **kwargs) as f:
        physiotable.to_csv(f, sep='\t', index=False, header=False)

That should solve the warnings for the tsv-files. You got those MTIME warnings for them as well, right? I'm still confused how you get them for your dcm2niix produced gz-files?