mriconvert: Support for exporting BIDS compatible JSON files
Roman Fleysher
roman.fleysher at einstein.yu.edu
Mon Oct 19 13:58:00 PDT 2015
Dear Chris,
According to Webster, compromise:
: a way of reaching agreement in which each person or group gives up something that was wanted in order to end an argument or dispute
: something that combines the qualities of two different things
: a change that makes something worse and that is not done for a good reason
Which of these you think is relevant and "everyone is happy with"?
Roman
________________________________
From: mriconvert-bounces at lists.uoregon.edu [mriconvert-bounces at lists.uoregon.edu] on behalf of Chris Filo Gorgolewski [krzysztof.gorgolewski at gmail.com]
Sent: Monday, October 19, 2015 4:11 PM
To: MRIConvert user's list
Subject: Re: mriconvert: Support for exporting BIDS compatible JSON files
Thanks! I'm really glad we managed to reach a compromise everyone is happy with!
Best,
Chris
On Mon, Oct 19, 2015 at 2:51 PM, Roman Fleysher <roman.fleysher at einstein.yu.edu<mailto:roman.fleysher at einstein.yu.edu>> wrote:
Thank you, Jolinda!!!
Roman
________________________________
From: Jolinda [jolinda at uoregon.edu<mailto:jolinda at uoregon.edu>]
Sent: Monday, October 19, 2015 3:39 PM
To: MRIConvert user's list; Roman Fleysher
Subject: Re: mriconvert: Support for exporting BIDS compatible JSON files
Hi Roman,
Here is the code. It requires DCMTK and boost. I build with cmake, so I've attached the cmakelists.txt file. If you use cmake, you may need to replace the FindDCMTK.cmake file, so I attached one that should work. It combs through dicom files in a directory, recursively, pulls out fields, and writes the results to a comma separated text file. Fields are specified according to the DCMTK dicom dictionary: for example, you could call GetDicomFields mydirectory myoutputfile PatientName StudyDate StudyDescription. Duplicates are discarded. Hopefully you can make use of it, or at least use it as a starting point.
It is my fervent hope that someday the DICOM people and the analysis people will talk to each other long enough that all these other formats can go away, and I can put MRIConvert on the shelf!
Jolinda
On Mon, 19 Oct 2015 11:36:39 -0700, Roman Fleysher <roman.fleysher at einstein.yu.edu<mailto:roman.fleysher at einstein.yu.edu>> wrote:
Dear Jolinda,
I agree with you that synchronization between NIFTI produced by converter and parameters produced by a separate extractor is a point that may require integration. This argument, not the speed nor "others do it too", is what I would accept in favor of integration.
Even with the meta file that mcverter produces already, I have to read DICOM separately to extract proprietary DICOM fields. I am writing from experience of making a standalone extractor, one who faced and solved the synchronization problem. Since I do not consider myself an expert on DICOM I can not be sure my solution is of general enough value to offer it to others. However, I know you are far more knowledgable than I in these matters and thus I will gladly accept your offer --- the 150 line code to read DICOM parameters --- to see if this is something I can use with trust.
I am also trying to point out that ASL data needs accompanying info about labeling but does not need b-values. DTI data needs info about b-values, but not ASL labeling. Inventing a common JSON format file that is capable of dealing with all possibilities will lead to a new DICOM format. If DICOM were so useful and easy to use we would not be talking about JSON.
Roman
________________________________
From: mriconvert-bounces at lists.uoregon.edu<mailto:mriconvert-bounces at lists.uoregon.edu> [mriconvert-bounces at lists.uoregon.edu<mailto:mriconvert-bounces at lists.uoregon.edu>] on behalf of Jolinda [jolinda at uoregon.edu<mailto:jolinda at uoregon.edu>]
Sent: Monday, October 19, 2015 2:18 PM
To: MRIConvert user's list
Subject: Re: mriconvert: Support for exporting BIDS compatible JSON files
Roman,
If it required a lot of additional work on the part of MRIConvert/mcverter to extract this info, I might agree. However, much of this information has to extracted anyway in order to correctly convert the data, so it isn't much more work to keep track of it. It's important that the sidecar files "line up" with the nifti output files, and it would be tricky to ensure that happened with separate modules. In any case, at this point it would be far more work to split it into separate modules that to keep it all in one program.
If the JSON format expects information not found in the DICOM file I have a simple answer. MRIConvert won't write that info and it will be up to the user to edit the files later and add it.
Incidentally, if you are interested in a more general extractor for DICOM fields I have one I can share. It's about 150 lines of code and pulls out whatever fields are specified by the user from every DICOM file in a directory, using DCMTK.
Jolinda
On Mon, 19 Oct 2015 10:04:39 -0700, Roman Fleysher <roman.fleysher at einstein.yu.edu<mailto:roman.fleysher at einstein.yu.edu>> wrote:
Thank you, Jolinda, for allowing this discussion.
I would like to make sure I am understood correctly. I am not advocating against existence of meta data in JSON format, nor do I advocate against MRIConvert developers writing code to produce meta data in JSON format. I am advocating for such meta data extractor to be a separate module with its user interface.
The reason for this is that I expect different extractors will be needed for different purposes. Writing general one will be very complicated and potentially useless because many parameters are not part of DICOM but are part of meta data. The extractor could ask user for such additional info or some other input mechanism.
Thank you,
Roman
________________________________
From: mriconvert-bounces at lists.uoregon.edu<mailto:mriconvert-bounces at lists.uoregon.edu> [mriconvert-bounces at lists.uoregon.edu<mailto:mriconvert-bounces at lists.uoregon.edu>] on behalf of Jolinda [jolinda at uoregon.edu<mailto:jolinda at uoregon.edu>]
Sent: Monday, October 19, 2015 12:52 PM
To: MRIConvert user's list
Subject: Re: mriconvert: Support for exporting BIDS compatible JSON files
Hello,
This has been a great discussion, and please continue, but I'd like to take a moment to address the initial request. MRIConvert already creates a text file for each series during conversion, so it should be a fairly simple matter to format that file to meet the JSON specification. If other users would like to weigh in on the pros/cons of this approach, I'd like to hear from you.
Jolinda
On Fri, 16 Oct 2015 10:39:17 -0700, Roman Fleysher <roman.fleysher at einstein.yu.edu<mailto:roman.fleysher at einstein.yu.edu>> wrote:
Dear Chris,
You are, of course, correct that error code can be used to flag the exit condition (as I indicated myself as well). The issue is not whether it is possible or impossible. The issue is complexity of dealing with it.
Your version:
status = mcverter()
if( status == 1) then
print "could not produce NIFTI file, meta data is OK"
end
if( status == 2) then
print "could not produce meta data, NIFTI file is OK"
end
if( status == 3) then
print "could produce neither meta data nor NIFTI file"
end
All OK if I got here
My version:
if( mcverter() !=0 ) then
print "could not produce NIFTI file "
end
if( getMetaData() !=0 ) then
print "could not produce meta data "
end
All OK if I got here
My version is simpler and easier to program and understand. More understandable is usually less buggy. These are not just questions of style.
As to your second point about being "unnecessarily slow because you would have to load the same data twice" this heavily depends on the important word "unnecessarily". I agree, separating functionality will cause some slow down. How much can be tolerated? Using dcmdump is rather fast, a second or two. In my opinion this slowdown is OK.
I can imagine there are applications when even 1 or 2 second slow down is not acceptable. But then, you might wonder why spend time writing out NIFTI file (which takes 1 or 2 seconds at least) in order to read it right away (also a second or two delay) to perform motion correction for fMRI data or eddy current correction for DTI? In these applications, you will wonder, why not combine DICOM to NIFTI conversion not only with meta data extraction but also with motion correction and eddy current correction. That will be your next request to MRIConvert (and dcm2nii, dicm2nii, dcmstack to name only a few) developers?
Do you see the point? The point is integration versus modularization. If things need to be fast --- they have to be integrated. The software will necessarily get far more complicated and far more buggy, but in the end will be the fastest possible at execution. Or, if user can wait a little, the software can be split into modular components which are easy to interchange and compose. This necessarily makes each software component simpler to write, less buggy and gives ability to pair with any motion correction algorithm.
I am for modular software construction and I urge MRIConvert developers (as well as others) to adopt modular rather than integrated approach. I do not see the small gains in performance being valuable to warrant complexity of the integrated software and its user interface for the data processing tasks I am familiar with.
Roman
________________________________
From: mriconvert-bounces at lists.uoregon.edu<mailto:mriconvert-bounces at lists.uoregon.edu> [mriconvert-bounces at lists.uoregon.edu<mailto:mriconvert-bounces at lists.uoregon.edu>] on behalf of Chris Filo Gorgolewski [krzysztof.gorgolewski at gmail.com<mailto:krzysztof.gorgolewski at gmail.com>]
Sent: Thursday, October 15, 2015 7:55 PM
To: MRIConvert user's list
Subject: Re: mriconvert: Support for exporting BIDS compatible JSON files
Dear Roman,
Thank you for the clarification. I respectfully disagree. There are many ways to clarify the reason for which a given command line fails. For once there is the return code you mentioned. It's a common practice to associate different non-zero codes with different types of failures (for example -1 for conversion failure, -2 for metadata extraction failure). In addition one can use standard error channel of the console output to communicate the nature of the error in plain English.
Using a separate command line to extract metadata would be unnecessarily slow because you would have to load the same data twice. For this reason other dicom converters (dcm2nii, dicm2nii, dcmstack to name only a few) also perform metadata extraction.
I agree that that this is not the best place to discuss this. I will try other ways to reach mriconvert developers to work on this feature.
Best regards,
Chris Gorgolewski
On Wed, Oct 7, 2015 at 12:38 PM, Roman Fleysher <roman.fleysher at einstein.yu.edu<mailto:roman.fleysher at einstein.yu.edu>> wrote:
Dear Chris,
I am not sure this forum is the right place to discuss general programming stylistics. I will give a small example.
Typically, a well written code (and I regard mcverter as one of these) sets an exit code to indicate success or error. If success (conventionally code=0), the user knows conversion was good. If fail (code not zero) indicates NIFTI is ether corrupt or not present and code may be interpreted as to the reason: no input data, wrong flags, no space on disk to output etc. Suppose conversion was OK, but the text file you propose could not be generated because DICOM fields were not filled. Is that a success or a failure? Which code should be given? By convention there is only one code for success: 0. Programmers have this joke that makes it easy to remember: Why did Roman Empire die? Because they did not have numeric code for successful completion of operations. (Roman numerals --- I, II, III, IV... --- do not have notation for zero.)
The reason I talk exit codes is because as our analyses are getting more complicated, and they do, humans are less and less involved in the routine tasks. Tasks are scripted. I call mcverter from within a script that checks exit codes and tells me where error occurred. If the task of conversion and creation of the parameter file are split into two commands, then each gets to flag its exit status and makes it easier to trace successful conversion and successful parameter extraction.
Moreover, it will be easy to pair DTI data with mcverter and b-value extractor, fMRI data with mcverter and timing extractor, ASL with mcverter and label extractor.... Conversion is always the same. Parameters depend on the context.
Roman
________________________________
From: mriconvert-bounces at lists.uoregon.edu<mailto:mriconvert-bounces at lists.uoregon.edu> [mriconvert-bounces at lists.uoregon.edu<mailto:mriconvert-bounces at lists.uoregon.edu>] on behalf of Chris Filo Gorgolewski [krzysztof.gorgolewski at gmail.com<mailto:krzysztof.gorgolewski at gmail.com>]
Sent: Friday, October 02, 2015 11:18 PM
To: MRIConvert user's list
Subject: Re: mriconvert: Support for exporting BIDS compatible JSON files
Hi Roman,
Thank you for you feedback. Could you elaborate a bit more why do you consider command lines producing more than one file so dangerous?
Best,
Chris
On Fri, Oct 2, 2015 at 8:00 PM, Roman Fleysher <roman.fleysher at einstein.yu.edu<mailto:roman.fleysher at einstein.yu.edu>> wrote:
Dear MRIConvert developers,
Dear MRIConvert users,
I am strictly against such addition to MRIConvert. This functionality must be implemented in a separate command. There are other parameters that might be needed and other ways to get them. I consider producing such files as part of DICOM to NIFTI conversion a side effect. As we know in programming, side effects is one of the biggest causes of errors. User interfaces get clumsy. Please do not add side effects to MRIConvert. Not this, not any other.
Thank you,
Roman
________________________________
From: mriconvert-bounces at lists.uoregon.edu<mailto:mriconvert-bounces at lists.uoregon.edu> [mriconvert-bounces at lists.uoregon.edu<mailto:mriconvert-bounces at lists.uoregon.edu>] on behalf of Chris Filo Gorgolewski [krzysztof.gorgolewski at gmail.com<mailto:krzysztof.gorgolewski at gmail.com>]
Sent: Friday, October 02, 2015 9:26 PM
To: mriconvert at lists.uoregon.edu<mailto:mriconvert at lists.uoregon.edu>
Subject: mriconvert: Support for exporting BIDS compatible JSON files
Dear developers of MRIConvert,
Brain Imaging Data Structure (BIDS)<https://docs.google.com/document/d/1HFUkAEE-pB-angVcYe6pf_-fVf4sCpOHKesUvfb8Grc/edit#> is a new specification describing how a neuroimaging dataset should be organized and described. Part of the standard are JSON sidecar files with acquisition parameters essential for performing data analysis that are not present (or reliably reported) in the NIFTI header (see here<https://docs.google.com/document/d/1HFUkAEE-pB-angVcYe6pf_-fVf4sCpOHKesUvfb8Grc/edit#heading=h.r8mrcau3kkcq> for details). Such fields include but are not limited to:
* EffectiveEchoSpacing
* RepetitionTime
* PhaseEncodingDirection
* SliceTiming
* SliceEncodingDirection
* EchoTime
Some of those fields are part of DICOM Ontology and are directly accessible from standard DICOM headers (such as RepetitionTime and EchoTime) and some are not part of standard DICOM nomenclature and require extraction using vendor and sequence specific heuristics (for example PhaseEncodingDirection or EffectiveEchoSpacing). We aded them to the BIDS standard because they are necessary for data processing.
This is how an example BIDS compatible JSON file looks like (more examples here<https://github.com/INCF/BIDS-examples>):
{
"EchoTime": 0.017,
"EffectiveEchoSpacing": 0.0003333262223739227,
"PhaseEncodingDirection": "y-",
"RepetitionTime": 3.0,
"SliceEncodingDirection": "z",
"SliceTiming": [
1.508,
0.0,
1.55,
0.043,
1.592,
0.087,
1.635,
0.13,
1.677,
0.173,
1.722,
0.215,
1.765,
0.26,
1.808,
0.302,
1.85,
0.345,
1.893,
0.388,
1.938,
0.43,
1.98,
0.475,
2.022,
0.518,
2.065,
0.56,
2.11,
0.603,
2.152,
0.645,
2.195,
0.69,
2.238,
0.733,
2.28,
0.775,
2.325,
0.818,
2.367,
0.86,
2.41,
0.905,
2.453,
0.948,
2.495,
0.99,
2.54,
1.032,
2.583,
1.075,
2.625,
1.12,
2.668,
1.163,
2.71,
1.205,
2.755,
1.248,
2.798,
1.293,
2.84,
1.335,
2.883,
1.378,
2.925,
1.42,
2.97,
1.462
]
}
I've heard a lot of good things about MRIConvert. It would be great if the future release supported saving BIDS style JSON files along the converted NIFTI files. In addition you can consider embedding this JSON inside the header (similar to what dcmstack does). From looking at your code base I see that you already calculate many of the required parameters. I'm more than happy to provide any help in implementing this feature.
I have reached out to other developers of DICOM to NIFTI converters with similar proposals. I hope that soon we will have a common way of describing NIFTI metadata!
Best regards,
Chris Gorgolewski
_______________________________________________
mriconvert mailing list
mriconvert at lists.uoregon.edu<mailto:mriconvert at lists.uoregon.edu>
https://lists-prod.uoregon.edu/mailman/listinfo/mriconvert
_______________________________________________
mriconvert mailing list
mriconvert at lists.uoregon.edu<mailto:mriconvert at lists.uoregon.edu>
https://lists-prod.uoregon.edu/mailman/listinfo/mriconvert
--
Using Opera's mail client: http://www.opera.com/mail/
--
Using Opera's mail client: http://www.opera.com/mail/
--
Using Opera's mail client: http://www.opera.com/mail/
_______________________________________________
mriconvert mailing list
mriconvert at lists.uoregon.edu<mailto:mriconvert at lists.uoregon.edu>
https://lists-prod.uoregon.edu/mailman/listinfo/mriconvert
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists-prod.uoregon.edu/pipermail/mriconvert/attachments/20151019/f372eb58/attachment-0001.html>
More information about the mriconvert
mailing list