mriconvert: Support for exporting BIDS compatible JSON files
Jolinda
jolinda at uoregon.edu
Mon Oct 19 12:39:44 PDT 2015
Hi Roman,
Here is the code. It requires DCMTK and boost. I build with cmake, so I've
attached the cmakelists.txt file. If you use cmake, you may need to
replace the FindDCMTK.cmake file, so I attached one that should work. It
combs through dicom files in a directory, recursively, pulls out fields,
and writes the results to a comma separated text file. Fields are
specified according to the DCMTK dicom dictionary: for example, you could
call GetDicomFields mydirectory myoutputfile PatientName StudyDate
StudyDescription. Duplicates are discarded. Hopefully you can make use of
it, or at least use it as a starting point.
It is my fervent hope that someday the DICOM people and the analysis
people will talk to each other long enough that all these other formats
can go away, and I can put MRIConvert on the shelf!
Jolinda
On Mon, 19 Oct 2015 11:36:39 -0700, Roman Fleysher
<roman.fleysher at einstein.yu.edu> wrote:
> Dear Jolinda,
> I agree with you that synchronization between NIFTI produced by
> converter and parameters produced by a separate extractor is a point
> that may require integration. This >argument, not the speed nor "others
> do it too", is what I would accept in favor of integration.
> Even with the meta file that mcverter produces already, I have to read
> DICOM separately to extract proprietary DICOM fields. I am writing from
> experience of making a >standalone extractor, one who faced and solved
> the synchronization problem. Since I do not consider myself an expert on
> DICOM I can not be sure my solution is of general >enough value to offer
> it to others. However, I know you are far more knowledgable than I in
> these matters and thus I will gladly accept your offer --- the 150 line
> code to >read DICOM parameters --- to see if this is something I can use
> with trust.
>
> I am also trying to point out that ASL data needs accompanying info
> about labeling but does not need b-values. DTI data needs info about
> b-values, but not ASL labeling. >Inventing a common JSON format file
> that is capable of dealing with all possibilities will lead to a new
> DICOM format. If DICOM were so useful and easy to use we would not >be
> talking about JSON.
>
>
> Roman
>
>
> From: mriconvert-bounces at lists.uoregon.edu
> [mriconvert-bounces at lists.uoregon.edu] on behalf of Jolinda
> [jolinda at uoregon.edu]
> Sent: Monday, October 19, 2015 2:18 PM
> To: MRIConvert user's list
> Subject: Re: mriconvert: Support for exporting BIDS compatible JSON files
>
> Roman,
>
> If it required a lot of additional work on the part of
> MRIConvert/mcverter to extract this info, I might agree. However, much
> of this information has to >extracted anyway in order to correctly
> convert the data, so it isn't much more work to keep track of it. It's
> important that the sidecar files "line up" with the >nifti output files,
> and it would be tricky to ensure that happened with separate modules. In
> any case, at this point it would be far more work to split it into
> >separate modules that to keep it all in one program.
>
> If the JSON format expects information not found in the DICOM file I
> have a simple answer. MRIConvert won't write that info and it will be up
> to the user >to edit the files later and add it.
>
> Incidentally, if you are interested in a more general extractor for
> DICOM fields I have one I can share. It's about 150 lines of code and
> pulls out whatever >fields are specified by the user from every DICOM
> file in a directory, using DCMTK.
>
> Jolinda
>
>
> On Mon, 19 Oct 2015 10:04:39 -0700, Roman Fleysher
> <roman.fleysher at einstein.yu.edu> wrote:
>
>> Thank you, Jolinda, for allowing this discussion.
>> I would like to make sure I am understood correctly. I am not
>> advocating against existence of meta data in JSON format, nor do I
>> advocate against MRIConvert developers >>writing code to produce meta
>> data in JSON format. I am advocating for such meta data extractor to be
>> a separate module with its user interface.
>> The reason for this is that I expect different extractors will be
>> needed for different purposes. Writing general one will be very
>> complicated and potentially useless because >>many parameters are not
>> part of DICOM but are part of meta data. The extractor could ask user
>> for such additional info or some other input mechanism.
>>
>>
>> Thank you,
>>
>> Roman
>>
>> From: mriconvert-bounces at lists.uoregon.edu
>> [mriconvert-bounces at lists.uoregon.edu] on behalf of Jolinda
>> [jolinda at uoregon.edu]
>> Sent: Monday, October 19, 2015 12:52 PM
>> To: MRIConvert user's list
>> Subject: Re: mriconvert: Support for exporting BIDS compatible JSON
>> files
>>
>> Hello,
>>
>> This has been a great discussion, and please continue, but I'd like to
>> take a moment to address the initial request. MRIConvert already
>> creates a text file for >>each series during conversion, so it should
>> be a fairly simple matter to format that file to meet the JSON
>> specification. If other users would like to weigh in >>on the pros/cons
>> of this approach, I'd like to hear from you.
>>
>> Jolinda
>>
>>
>> On Fri, 16 Oct 2015 10:39:17 -0700, Roman Fleysher
>> <roman.fleysher at einstein.yu.edu> wrote:
>>
>>> Dear Chris,
>>> You are, of course, correct that error code can be used to flag the
>>> exit condition (as I indicated myself as well). The issue is not
>>> whether it is possible or impossible. The >>>issue is complexity of
>>> dealing with it.
>>>
>>> Your version:
>>>
>>> status = mcverter()
>>> if( status == 1) then
>>> print "could not produce NIFTI file, meta data is OK"
>>> end
>>>
>>> if( status == 2) then
>>> print "could not produce meta data, NIFTI file is OK"
>>> end
>>>
>>> if( status == 3) then
>>> print "could produce neither meta data nor NIFTI file"
>>> end
>>>
>>> All OK if I got here
>>>
>>> My version:
>>>
>>> if( mcverter() !=0 ) then
>>> print "could not produce NIFTI file "
>>> end
>>> if( getMetaData() !=0 ) then
>>> print "could not produce meta data "
>>> end
>>>
>>> All OK if I got here
>>>
>>> My version is simpler and easier to program and understand. More
>>> understandable is usually less buggy. These are not just questions of
>>> style.
>>>
>>> As to your second point about being "unnecessarily slow because you
>>> would have to load the same data twice" this heavily depends on the
>>> important word >>>"unnecessarily". I agree, separating functionality
>>> will cause some slow down. How much can be tolerated? Using dcmdump is
>>> rather fast, a second or two. >>>In my opinion this slowdown is OK.
>>>>>> I can imagine there are applications when even 1 or 2 second slow
>>>>>> down is not acceptable. But then, you might wonder why spend time
>>>>>> writing out NIFTI >>>file (which takes 1 or 2 seconds at least) in
>>>>>> order to read it right away (also a second or two delay) to perform
>>>>>> motion correction for fMRI data or eddy >>>current correction for
>>>>>> DTI? In these applications, you will wonder, why not combine DICOM
>>>>>> to NIFTI conversion not only with meta data extraction but >>>also
>>>>>> with motion correction and eddy current correction. That will be
>>>>>> your next request to MRIConvert (and dcm2nii, dicm2nii, dcmstack
>>>>>> to name only a >>>few) developers?
>>>
>>>>>> Do you see the point? The point is integration versus
>>>>>> modularization. If things need to be fast --- they have to be
>>>>>> integrated. The software will necessarily >>>get far more
>>>>>> complicated and far more buggy, but in the end will be the fastest
>>>>>> possible at execution. Or, if user can wait a little, the software
>>>>>> can be split >>>into modular components which are easy to
>>>>>> interchange and compose. This necessarily makes each software
>>>>>> component simpler to write, less buggy and >>>gives ability to pair
>>>>>> with any motion correction algorithm.
>>>
>>>>>> I am for modular software construction and I urge MRIConvert
>>>>>> developers (as well as others) to adopt modular rather than
>>>>>> integrated approach. I do not >>>see the small gains in performance
>>>>>> being valuable to warrant complexity of the integrated software and
>>>>>> its user interface for the data processing tasks I >>>am familiar
>>>>>> with.
>>>
>>>
>>> Roman
>>>
>>>
>>>
>>>
>>> From: mriconvert-bounces at lists.uoregon.edu
>>> [mriconvert-bounces at lists.uoregon.edu] on behalf of Chris Filo
>>> Gorgolewski [krzysztof.gorgolewski at gmail.com]
>>> Sent: Thursday, October 15, 2015 7:55 PM
>>> To: MRIConvert user's list
>>> Subject: Re: mriconvert: Support for exporting BIDS compatible JSON
>>> files
>>>
>>> Dear Roman,Thank you for the clarification. I respectfully disagree.
>>> There are many ways to clarify the reason for which a given command
>>> line fails. For once there is >>>the return code you mentioned. It's a
>>> common practice to associate different non-zero codes with different
>>> types of failures (for example -1 for conversion >>>failure, -2 for
>>> metadata extraction failure). In addition one can use standard error
>>> channel of the console output to communicate the nature of the error
>>> in >>>plain English.
>>>
>>> Using a separate command line to extract metadata would be
>>> unnecessarily slow because you would have to load the same data twice.
>>> For this reason >>>other dicom converters (dcm2nii, dicm2nii, dcmstack
>>> to name only a few) also perform metadata extraction.
>>> I agree that that this is not the best place to discuss this. I will
>>> try other ways to reach mriconvert developers to work on this feature.
>>>
>>> Best regards,
>>> Chris Gorgolewski
>>>
>>> On Wed, Oct 7, 2015 at 12:38 PM, Roman Fleysher
>>> <roman.fleysher at einstein.yu.edu> wrote:
>>>> Dear Chris,
>>>> I am not sure this forum is the right place to discuss general
>>>> programming stylistics. I will give a small example.
>>>> Typically, a well written code (and I regard mcverter as one of
>>>> these) sets an exit code to indicate success or error. If success
>>>> (conventionally code=0), the user knows >>>>conversion was good. If
>>>> fail (code not zero) indicates NIFTI is ether corrupt or not present
>>>> and code may be interpreted as to the reason: no input data, wrong
>>>> flags, >>>>no space on disk to output etc. Suppose conversion was OK,
>>>> but the text file you propose could not be generated because DICOM
>>>> fields were not filled. Is that a >>>>success or a failure? Which
>>>> code should be given? By convention there is only one code for
>>>> success: 0. Programmers have this joke that makes it easy to
>>>> remember: >>>>Why did Roman Empire die? Because they did not have
>>>> numeric code for successful completion of operations. (Roman numerals
>>>> --- I, II, III, IV... --- do not have >>>>notation for zero.)
>>>>
>>>> The reason I talk exit codes is because as our analyses are getting
>>>> more complicated, and they do, humans are less and less involved in
>>>> the routine tasks. Tasks are >>>>scripted. I call mcverter from
>>>> within a script that checks exit codes and tells me where error
>>>> occurred. If the task of conversion and creation of the parameter
>>>> file are >>>>split into two commands, then each gets to flag its exit
>>>> status and makes it easier to trace successful conversion and
>>>> successful parameter extraction.
>>>>
>>>> Moreover, it will be easy to pair DTI data with mcverter and b-value
>>>> extractor, fMRI data with mcverter and timing extractor, ASL with
>>>> mcverter and label extractor.... >>>>Conversion is always the same.
>>>> Parameters depend on the context.
>>>>
>>>> Roman
>>>>
>>>> From: mriconvert-bounces at lists.uoregon.edu
>>>> [mriconvert-bounces at lists.uoregon.edu] on behalf of Chris Filo
>>>> Gorgolewski [krzysztof.gorgolewski at gmail.com]
>>>> Sent: Friday, October 02, 2015 11:18 PM
>>>> To: MRIConvert user's list
>>>> Subject: Re: mriconvert: Support for exporting BIDS compatible JSON
>>>> files
>>>>
>>>> Hi Roman,Thank you for you feedback. Could you elaborate a bit more
>>>> why do you consider command lines producing more than one file so
>>>> dangerous?
>>>>
>>>> Best,
>>>> Chris
>>>>
>>>> On Fri, Oct 2, 2015 at 8:00 PM, Roman Fleysher
>>>> <roman.fleysher at einstein.yu.edu> wrote:
>>>>> Dear MRIConvert developers,
>>>>> Dear MRIConvert users,
>>>>> I am strictly against such addition to MRIConvert. This
>>>>> functionality must be implemented in a separate command. There are
>>>>> other parameters that might be needed >>>>>and other ways to get
>>>>> them. I consider producing such files as part of DICOM to NIFTI
>>>>> conversion a side effect. As we know in programming, side effects is
>>>>> one of >>>>>the biggest causes of errors. User interfaces get
>>>>> clumsy. Please do not add side effects to MRIConvert. Not this, not
>>>>> any other.
>>>>>
>>>>>
>>>>> Thank you,
>>>>>
>>>>> Roman
>>>>>
>>>>> From: mriconvert-bounces at lists.uoregon.edu
>>>>> [mriconvert-bounces at lists.uoregon.edu] on behalf of Chris Filo
>>>>> Gorgolewski [krzysztof.gorgolewski at gmail.com]
>>>>> Sent: Friday, October 02, 2015 9:26 PM
>>>>> To: mriconvert at lists.uoregon.edu
>>>>> Subject: mriconvert: Support for exporting BIDS compatible JSON files
>>>>>
>>>>>
>>>>> Dear developers of MRIConvert,
>>>>>
>>>>> Brain Imaging Data Structure (BIDS) is a new specification
>>>>> describing how a neuroimaging dataset should be organized and
>>>>> described. Part of the >>>>>standard are JSON sidecar files with
>>>>> acquisition parameters essential for performing data analysis that
>>>>> are not present (or reliably reported) in the >>>>>NIFTI header (see
>>>>> here for details). Such fields include but are not limited to:
>>>>> EffectiveEchoSpacing
>>>>> RepetitionTime
>>>>> PhaseEncodingDirection
>>>>> SliceTiming
>>>>> SliceEncodingDirection
>>>>> EchoTime
>>>>>
>>>>> Some of those fields are part of DICOM Ontology and are directly
>>>>> accessible from standard DICOM headers (such as RepetitionTime and
>>>>> EchoTime) >>>>>and some are not part of standard DICOM nomenclature
>>>>> and require extraction using vendor and sequence specific heuristics
>>>>> (for example >>>>>PhaseEncodingDirection or EffectiveEchoSpacing).
>>>>> We aded them to the BIDS standard because they are necessary for
>>>>> data processing.
>>>>>
>>>>> This is how an example BIDS compatible JSON file looks like (more
>>>>> examples here):
>>>>> {
>>>>> "EchoTime": 0.017,
>>>>> "EffectiveEchoSpacing": 0.0003333262223739227,
>>>>> "PhaseEncodingDirection": "y-",
>>>>> "RepetitionTime": 3.0,
>>>>> "SliceEncodingDirection": "z",
>>>>> "SliceTiming": [
>>>>> 1.508,
>>>>> 0.0,
>>>>> 1.55,
>>>>> 0.043,
>>>>> 1.592,
>>>>> 0.087,
>>>>> 1.635,
>>>>> 0.13,
>>>>> 1.677,
>>>>> 0.173,
>>>>> 1.722,
>>>>> 0.215,
>>>>> 1.765,
>>>>> 0.26,
>>>>> 1.808,
>>>>> 0.302,
>>>>> 1.85,
>>>>> 0.345,
>>>>> 1.893,
>>>>> 0.388,
>>>>> 1.938,
>>>>> 0.43,
>>>>> 1.98,
>>>>> 0.475,
>>>>> 2.022,
>>>>> 0.518,
>>>>> 2.065,
>>>>> 0.56,
>>>>> 2.11,
>>>>> 0.603,
>>>>> 2.152,
>>>>> 0.645,
>>>>> 2.195,
>>>>> 0.69,
>>>>> 2.238,
>>>>> 0.733,
>>>>> 2.28,
>>>>> 0.775,
>>>>> 2.325,
>>>>> 0.818,
>>>>> 2.367,
>>>>> 0.86,
>>>>> 2.41,
>>>>> 0.905,
>>>>> 2.453,
>>>>> 0.948,
>>>>> 2.495,
>>>>> 0.99,
>>>>> 2.54,
>>>>> 1.032,
>>>>> 2.583,
>>>>> 1.075,
>>>>> 2.625,
>>>>> 1.12,
>>>>> 2.668,
>>>>> 1.163,
>>>>> 2.71,
>>>>> 1.205,
>>>>> 2.755,
>>>>> 1.248,
>>>>> 2.798,
>>>>> 1.293,
>>>>> 2.84,
>>>>> 1.335,
>>>>> 2.883,
>>>>> 1.378,
>>>>> 2.925,
>>>>> 1.42,
>>>>> 2.97,
>>>>> 1.462
>>>>> ]
>>>>> }
>>>>>
>>>>> I've heard a lot of good things about MRIConvert. It would be great
>>>>> if the future release supported saving BIDS style JSON files along
>>>>> the converted >>>>>NIFTI files. In addition you can consider
>>>>> embedding this JSON inside the header (similar to what dcmstack
>>>>> does). From looking at your code base I >>>>>see that you already
>>>>> calculate many of the required parameters. I'm more than happy to
>>>>> provide any help in implementing this feature.
>>>>>
>>>>>
>>>>>
>>>>> I have reached out to other developers of DICOM to NIFTI converters
>>>>> with similar proposals. I hope that soon we will have a common way
>>>>> of >>>>>describing NIFTI metadata!
>>>>>
>>>>>
>>>>>
>>>>> Best regards,
>>>>>
>>>>> Chris Gorgolewski
>>>>>
>>>>> _______________________________________________
>>>>> mriconvert mailing list
>>>>> mriconvert at lists.uoregon.edu
>>>>> https://lists-prod.uoregon.edu/mailman/listinfo/mriconvert
>>>>>
>>>>
>>>>
>>>> _______________________________________________
>>>> mriconvert mailing list
>>>> mriconvert at lists.uoregon.edu
>>>> https://lists-prod.uoregon.edu/mailman/listinfo/mriconvert
>>>>
>>>
>>
>>
>>
>> --Using Opera's mail client: http://www.opera.com/mail/
>
>
>
> --Using Opera's mail client: http://www.opera.com/mail/
--
Using Opera's mail client: http://www.opera.com/mail/
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists-prod.uoregon.edu/pipermail/mriconvert/attachments/20151019/d0c03d23/attachment-0001.html>
-------------- next part --------------
An embedded and charset-unspecified text was scrubbed...
Name: CMakeLists.txt
URL: <http://lists-prod.uoregon.edu/pipermail/mriconvert/attachments/20151019/d0c03d23/attachment-0001.txt>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: FindDCMTK.cmake
Type: application/octet-stream
Size: 12138 bytes
Desc: not available
URL: <http://lists-prod.uoregon.edu/pipermail/mriconvert/attachments/20151019/d0c03d23/attachment-0002.obj>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: GetDicomFields.cpp
Type: application/octet-stream
Size: 3675 bytes
Desc: not available
URL: <http://lists-prod.uoregon.edu/pipermail/mriconvert/attachments/20151019/d0c03d23/attachment-0003.obj>
More information about the mriconvert
mailing list