mriconvert: Support for exporting BIDS compatible JSON files
Chris Filo Gorgolewski
krzysztof.gorgolewski at gmail.com
Mon Oct 19 13:11:29 PDT 2015
Thanks! I'm really glad we managed to reach a compromise everyone is happy
with!
Best,
Chris
On Mon, Oct 19, 2015 at 2:51 PM, Roman Fleysher <
roman.fleysher at einstein.yu.edu> wrote:
> Thank you, Jolinda!!!
>
> Roman
> ------------------------------
> *From:* Jolinda [jolinda at uoregon.edu]
> *Sent:* Monday, October 19, 2015 3:39 PM
> *To:* MRIConvert user's list; Roman Fleysher
>
> *Subject:* Re: mriconvert: Support for exporting BIDS compatible JSON
> files
>
> Hi Roman,
>
> Here is the code. It requires DCMTK and boost. I build with cmake, so I've
> attached the cmakelists.txt file. If you use cmake, you may need to replace
> the FindDCMTK.cmake file, so I attached one that should work. It combs
> through dicom files in a directory, recursively, pulls out fields, and
> writes the results to a comma separated text file. Fields are specified
> according to the DCMTK dicom dictionary: for example, you could call
> GetDicomFields mydirectory myoutputfile PatientName StudyDate
> StudyDescription. Duplicates are discarded. Hopefully you can make use of
> it, or at least use it as a starting point.
>
> It is my fervent hope that someday the DICOM people and the analysis
> people will talk to each other long enough that all these other formats can
> go away, and I can put MRIConvert on the shelf!
>
> Jolinda
>
> On Mon, 19 Oct 2015 11:36:39 -0700, Roman Fleysher <
> roman.fleysher at einstein.yu.edu> wrote:
>
> Dear Jolinda,
>
> I agree with you that synchronization between NIFTI produced by converter
> and parameters produced by a separate extractor is a point that may require
> integration. This argument, not the speed nor "others do it too", is what I
> would accept in favor of integration.
>
> Even with the meta file that mcverter produces already, I have to read
> DICOM separately to extract proprietary DICOM fields. I am writing from
> experience of making a standalone extractor, one who faced and solved the
> synchronization problem. Since I do not consider myself an expert on DICOM
> I can not be sure my solution is of general enough value to offer it to
> others. However, I know you are far more knowledgable than I in these
> matters and thus I will gladly accept your offer --- the 150 line code to
> read DICOM parameters --- to see if this is something I can use with trust.
>
> I am also trying to point out that ASL data needs accompanying info about
> labeling but does not need b-values. DTI data needs info about b-values,
> but not ASL labeling. Inventing a common JSON format file that is capable
> of dealing with all possibilities will lead to a new DICOM format. If DICOM
> were so useful and easy to use we would not be talking about JSON.
>
>
> Roman
>
>
> ------------------------------
> *From:* mriconvert-bounces at lists.uoregon.edu [
> mriconvert-bounces at lists.uoregon.edu] on behalf of Jolinda [
> jolinda at uoregon.edu]
> *Sent:* Monday, October 19, 2015 2:18 PM
> *To:* MRIConvert user's list
> *Subject:* Re: mriconvert: Support for exporting BIDS compatible JSON
> files
>
> Roman,
>
> If it required a lot of additional work on the part of MRIConvert/mcverter
> to extract this info, I might agree. However, much of this information has
> to extracted anyway in order to correctly convert the data, so it isn't
> much more work to keep track of it. It's important that the sidecar files
> "line up" with the nifti output files, and it would be tricky to ensure
> that happened with separate modules. In any case, at this point it would be
> far more work to split it into separate modules that to keep it all in one
> program.
>
> If the JSON format expects information not found in the DICOM file I have
> a simple answer. MRIConvert won't write that info and it will be up to the
> user to edit the files later and add it.
>
> Incidentally, if you are interested in a more general extractor for DICOM
> fields I have one I can share. It's about 150 lines of code and pulls out
> whatever fields are specified by the user from every DICOM file in a
> directory, using DCMTK.
>
> Jolinda
>
>
> On Mon, 19 Oct 2015 10:04:39 -0700, Roman Fleysher <
> roman.fleysher at einstein.yu.edu> wrote:
>
> Thank you, Jolinda, for allowing this discussion.
>
> I would like to make sure I am understood correctly. I am not advocating
> against existence of meta data in JSON format, nor do I advocate against
> MRIConvert developers writing code to produce meta data in JSON format. I
> am advocating for such meta data extractor to be a separate module with its
> user interface.
>
> The reason for this is that I expect different extractors will be needed
> for different purposes. Writing general one will be very complicated and
> potentially useless because many parameters are not part of DICOM but are
> part of meta data. The extractor could ask user for such additional info or
> some other input mechanism.
>
>
> Thank you,
>
> Roman
>
> ------------------------------
> *From:* mriconvert-bounces at lists.uoregon.edu [
> mriconvert-bounces at lists.uoregon.edu] on behalf of Jolinda [
> jolinda at uoregon.edu]
> *Sent:* Monday, October 19, 2015 12:52 PM
> *To:* MRIConvert user's list
> *Subject:* Re: mriconvert: Support for exporting BIDS compatible JSON
> files
>
> Hello,
>
> This has been a great discussion, and please continue, but I'd like to
> take a moment to address the initial request. MRIConvert already creates a
> text file for each series during conversion, so it should be a fairly
> simple matter to format that file to meet the JSON specification. If other
> users would like to weigh in on the pros/cons of this approach, I'd like to
> hear from you.
>
> Jolinda
>
>
> On Fri, 16 Oct 2015 10:39:17 -0700, Roman Fleysher <
> roman.fleysher at einstein.yu.edu> wrote:
>
> Dear Chris,
>
> You are, of course, correct that error code can be used to flag the exit
> condition (as I indicated myself as well). The issue is not whether it is
> possible or impossible. The issue is complexity of dealing with it.
>
> Your version:
>
> status = mcverter()
> if( status == 1) then
> print "could not produce NIFTI file, meta data is OK"
> end
>
> if( status == 2) then
> print "could not produce meta data, NIFTI file is OK"
> end
>
> if( status == 3) then
> print "could produce neither meta data nor NIFTI file"
> end
>
> All OK if I got here
>
> My version:
>
> if( mcverter() !=0 ) then
> print "could not produce NIFTI file "
> end
> if( getMetaData() !=0 ) then
> print "could not produce meta data "
> end
>
> All OK if I got here
>
> My version is simpler and easier to program and understand. More
> understandable is usually less buggy. These are not just questions of style.
>
> As to your second point about being "unnecessarily slow because you would
> have to load the same data twice" this heavily depends on the important
> word "unnecessarily". I agree, separating functionality will cause some
> slow down. How much can be tolerated? Using dcmdump is rather fast, a
> second or two. In my opinion this slowdown is OK.
>
> I can imagine there are applications when even 1 or 2 second slow down is
> not acceptable. But then, you might wonder why spend time writing out NIFTI
> file (which takes 1 or 2 seconds at least) in order to read it right away
> (also a second or two delay) to perform motion correction for fMRI data or
> eddy current correction for DTI? In these applications, you will wonder,
> why not combine DICOM to NIFTI conversion not only with meta data
> extraction but also with motion correction and eddy current correction.
> That will be your next request to MRIConvert (and dcm2nii, dicm2nii,
> dcmstack to name only a few) developers?
>
> Do you see the point? The point is integration versus modularization. If
> things need to be fast --- they have to be integrated. The software
> will necessarily get far more complicated and far more buggy, but in the
> end will be the fastest possible at execution. Or, if user can wait a
> little, the software can be split into modular components which are easy to
> interchange and compose. This necessarily makes each software component
> simpler to write, less buggy and gives ability to pair with any motion
> correction algorithm.
>
> I am for modular software construction and I urge MRIConvert developers
> (as well as others) to adopt modular rather than integrated approach. I do
> not see the small gains in performance being valuable to warrant complexity
> of the integrated software and its user interface for the data processing
> tasks I am familiar with.
>
>
> Roman
>
>
>
>
> ------------------------------
> *From:* mriconvert-bounces at lists.uoregon.edu [
> mriconvert-bounces at lists.uoregon.edu] on behalf of Chris Filo Gorgolewski
> [krzysztof.gorgolewski at gmail.com]
> *Sent:* Thursday, October 15, 2015 7:55 PM
> *To:* MRIConvert user's list
> *Subject:* Re: mriconvert: Support for exporting BIDS compatible JSON
> files
>
> Dear Roman,
> Thank you for the clarification. I respectfully disagree. There are many
> ways to clarify the reason for which a given command line fails. For once
> there is the return code you mentioned. It's a common practice to associate
> different non-zero codes with different types of failures (for example -1
> for conversion failure, -2 for metadata extraction failure). In addition
> one can use standard error channel of the console output to communicate the
> nature of the error in plain English.
>
> Using a separate command line to extract metadata would be unnecessarily
> slow because you would have to load the same data twice. For this reason
> other dicom converters (dcm2nii, dicm2nii, dcmstack to name only a few)
> also perform metadata extraction.
>
> I agree that that this is not the best place to discuss this. I will try
> other ways to reach mriconvert developers to work on this feature.
>
> Best regards,
> Chris Gorgolewski
>
> On Wed, Oct 7, 2015 at 12:38 PM, Roman Fleysher <
> roman.fleysher at einstein.yu.edu> wrote:
>
>> Dear Chris,
>>
>> I am not sure this forum is the right place to discuss general
>> programming stylistics. I will give a small example.
>>
>> Typically, a well written code (and I regard mcverter as one of these)
>> sets an exit code to indicate success or error. If success (conventionally
>> code=0), the user knows conversion was good. If fail (code not zero)
>> indicates NIFTI is ether corrupt or not present and code may be interpreted
>> as to the reason: no input data, wrong flags, no space on disk to output
>> etc. Suppose conversion was OK, but the text file you propose could not be
>> generated because DICOM fields were not filled. Is that a success or a
>> failure? Which code should be given? By convention there is only one code
>> for success: 0. Programmers have this joke that makes it easy to remember:
>> Why did Roman Empire die? Because they did not have numeric code for
>> successful completion of operations. (Roman numerals --- I, II, III, IV...
>> --- do not have notation for zero.)
>>
>> The reason I talk exit codes is because as our analyses are getting more
>> complicated, and they do, humans are less and less involved in the routine
>> tasks. Tasks are scripted. I call mcverter from within a script that checks
>> exit codes and tells me where error occurred. If the task of conversion and
>> creation of the parameter file are split into two commands, then each gets
>> to flag its exit status and makes it easier to trace successful conversion
>> and successful parameter extraction.
>>
>> Moreover, it will be easy to pair DTI data with mcverter and b-value
>> extractor, fMRI data with mcverter and timing extractor, ASL with mcverter
>> and label extractor.... Conversion is always the same. Parameters depend on
>> the context.
>>
>>
>> Roman
>>
>>
>> ------------------------------
>> *From:* mriconvert-bounces at lists.uoregon.edu [
>> mriconvert-bounces at lists.uoregon.edu] on behalf of Chris Filo
>> Gorgolewski [krzysztof.gorgolewski at gmail.com]
>> *Sent:* Friday, October 02, 2015 11:18 PM
>> *To:* MRIConvert user's list
>> *Subject:* Re: mriconvert: Support for exporting BIDS compatible JSON
>> files
>>
>> Hi Roman,
>> Thank you for you feedback. Could you elaborate a bit more why do you
>> consider command lines producing more than one file so dangerous?
>>
>> Best,
>> Chris
>>
>> On Fri, Oct 2, 2015 at 8:00 PM, Roman Fleysher <
>> roman.fleysher at einstein.yu.edu> wrote:
>>
>>> Dear MRIConvert developers,
>>> Dear MRIConvert users,
>>>
>>> I am strictly against such addition to MRIConvert. This functionality
>>> must be implemented in a separate command. There are other parameters that
>>> might be needed and other ways to get them. I consider producing such files
>>> as part of DICOM to NIFTI conversion a side effect. As we know in
>>> programming, side effects is one of the biggest causes of errors. User
>>> interfaces get clumsy. Please do not add side effects to MRIConvert. Not
>>> this, not any other.
>>>
>>>
>>> Thank you,
>>>
>>> Roman
>>>
>>> ------------------------------
>>> *From:* mriconvert-bounces at lists.uoregon.edu [
>>> mriconvert-bounces at lists.uoregon.edu] on behalf of Chris Filo
>>> Gorgolewski [krzysztof.gorgolewski at gmail.com]
>>> *Sent:* Friday, October 02, 2015 9:26 PM
>>> *To:* mriconvert at lists.uoregon.edu
>>> *Subject:* mriconvert: Support for exporting BIDS compatible JSON files
>>>
>>> Dear developers of MRIConvert,
>>>
>>> Brain Imaging Data Structure (BIDS)
>>> <https://docs.google.com/document/d/1HFUkAEE-pB-angVcYe6pf_-fVf4sCpOHKesUvfb8Grc/edit#> is
>>> a new specification describing how a neuroimaging dataset should be
>>> organized and described. Part of the standard are JSON sidecar files with
>>> acquisition parameters essential for performing data analysis that are not
>>> present (or reliably reported) in the NIFTI header (see here
>>> <https://docs.google.com/document/d/1HFUkAEE-pB-angVcYe6pf_-fVf4sCpOHKesUvfb8Grc/edit#heading=h.r8mrcau3kkcq> for
>>> details). Such fields include but are not limited to:
>>>
>>> - EffectiveEchoSpacing
>>> - RepetitionTime
>>> - PhaseEncodingDirection
>>> - SliceTiming
>>> - SliceEncodingDirection
>>> - EchoTime
>>>
>>> Some of those fields are part of DICOM Ontology and are directly
>>> accessible from standard DICOM headers (such as RepetitionTime and
>>> EchoTime) and some are not part of standard DICOM nomenclature and require
>>> extraction using vendor and sequence specific heuristics (for example
>>> PhaseEncodingDirection or EffectiveEchoSpacing). We aded them to the BIDS
>>> standard because they are necessary for data processing.
>>>
>>> This is how an example BIDS compatible JSON file looks like (more
>>> examples here <https://github.com/INCF/BIDS-examples>):
>>>
>>> {
>>> "EchoTime": 0.017,
>>> "EffectiveEchoSpacing": 0.0003333262223739227,
>>> "PhaseEncodingDirection": "y-",
>>> "RepetitionTime": 3.0,
>>> "SliceEncodingDirection": "z",
>>> "SliceTiming": [
>>> 1.508,
>>> 0.0,
>>> 1.55,
>>> 0.043,
>>> 1.592,
>>> 0.087,
>>> 1.635,
>>> 0.13,
>>> 1.677,
>>> 0.173,
>>> 1.722,
>>> 0.215,
>>> 1.765,
>>> 0.26,
>>> 1.808,
>>> 0.302,
>>> 1.85,
>>> 0.345,
>>> 1.893,
>>> 0.388,
>>> 1.938,
>>> 0.43,
>>> 1.98,
>>> 0.475,
>>> 2.022,
>>> 0.518,
>>> 2.065,
>>> 0.56,
>>> 2.11,
>>> 0.603,
>>> 2.152,
>>> 0.645,
>>> 2.195,
>>> 0.69,
>>> 2.238,
>>> 0.733,
>>> 2.28,
>>> 0.775,
>>> 2.325,
>>> 0.818,
>>> 2.367,
>>> 0.86,
>>> 2.41,
>>> 0.905,
>>> 2.453,
>>> 0.948,
>>> 2.495,
>>> 0.99,
>>> 2.54,
>>> 1.032,
>>> 2.583,
>>> 1.075,
>>> 2.625,
>>> 1.12,
>>> 2.668,
>>> 1.163,
>>> 2.71,
>>> 1.205,
>>> 2.755,
>>> 1.248,
>>> 2.798,
>>> 1.293,
>>> 2.84,
>>> 1.335,
>>> 2.883,
>>> 1.378,
>>> 2.925,
>>> 1.42,
>>> 2.97,
>>> 1.462
>>> ]
>>> }
>>>
>>> I've heard a lot of good things about MRIConvert. It would be great if
>>> the future release supported saving BIDS style JSON files along the
>>> converted NIFTI files. In addition you can consider embedding this JSON
>>> inside the header (similar to what dcmstack does). From looking at your
>>> code base I see that you already calculate many of the required parameters.
>>> I'm more than happy to provide any help in implementing this feature.
>>>
>>>
>>> I have reached out to other developers of DICOM to NIFTI converters with
>>> similar proposals. I hope that soon we will have a common way of describing
>>> NIFTI metadata!
>>>
>>>
>>> Best regards,
>>>
>>> Chris Gorgolewski
>>>
>>> _______________________________________________
>>> mriconvert mailing list
>>> mriconvert at lists.uoregon.edu
>>> https://lists-prod.uoregon.edu/mailman/listinfo/mriconvert
>>>
>>>
>>
>> _______________________________________________
>> mriconvert mailing list
>> mriconvert at lists.uoregon.edu
>> https://lists-prod.uoregon.edu/mailman/listinfo/mriconvert
>>
>>
>
>
>
> --
> Using Opera's mail client: http://www.opera.com/mail/
>
>
>
>
> --
> Using Opera's mail client: http://www.opera.com/mail/
>
>
>
>
> --
> Using Opera's mail client: http://www.opera.com/mail/
>
> _______________________________________________
> mriconvert mailing list
> mriconvert at lists.uoregon.edu
> https://lists-prod.uoregon.edu/mailman/listinfo/mriconvert
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists-prod.uoregon.edu/pipermail/mriconvert/attachments/20151019/b14faaaf/attachment-0001.html>
More information about the mriconvert
mailing list