DDLm, dREL, images and NeXus

[Brian H. Toby]

I will not address the specific question of dREL, since I am not  
certain on its implications, but I want support strongly greater  
interoperability between CIF and NeXus. In fact each standard lacks  
the strongest features of the others: CIF has an extensive and  
rigorous data dictionary while NeXus has a flexible and compact  
structure for complex and large data objects.

NeXus is the only standard that synchrotron and neutron facilities  
can adopt for data collection, since much of the data simply cannot  
be handled in CIF. It may take a while for this to filter into the  
biological diffraction and scattering community at these facilities,  
but I think it will happen as the facilities build new instruments.

We will have at best one standard for data collection (NeXus) and one  
for structural analysis reporting (CIF). The more they can be  
integrated and interoperate the better.

Brian

On Dec 10, 2008, at 6:32 PM, Herbert J. Bernstein wrote:

> Dear Colleagues,
>
>    This is a distillation of an email conversation James Hester and I
> have been having since the Osaka meeting.  We both feel that it
> would be helpful if others were to join in and express their views.
> We have been discussing the interaction among DDLm, dREL, images and
> NeXus.  We agree on most points, and disagree on a few, and hope,
> by opening up the discussion, to arrive at a consensus.
>
>    What is driving this discussion is a need to understand how best to
> manage image data in the context of both imgCIF and NeXus, and to do
> so in a way that is consistent with the recent adoption of DDLm as
> the target framework for new work on CIF dictionaries.
>
>    It must be clearly understood that it is highly unlikely that a
> single standard will ever be adopted for crystallographic diffraction
> images, much less for the broader context of pixel-based data in
> structural biology.  The best we can hope for right now is to have
> some number of clearly defined image data frameworks, and agreed
> algorithms for conversion among them.  There are many frameworks
> to consider, but two that are very close to achieving the goal of  
> becoming
> inter-operable in the immediate future are imgCIF and NeXus.  What is
> missing is a formal language within which to specify how to move  
> between
> them.
>
>    We could, of course, just come up with a verbal description of how
> to move between imgCIF and NeXus and a couple of example conversion
> programs written ad hoc in whatever language might come to mind.   
> However,
> the effort being expended on dREL, the supporting language for DDLm,
> suggests the possibility of building on dREL as a base to do this job
> by extending dREL to have the capability of working with NeXus (dREL
> is already capable of dealing with CIF).  James has made the counter
> proposal of leaving dREL as just a CIF-specific language and keeping
> the CIF-NeXus conversion algorithm specification as a matter for a
> different language and/or API.  James has also suggested the further
> step of stripping out the built-in functions from dREL and dealing  
> with
> just a very stable dREL language specification in one instance and a
> perhaps evolving API (list of builtin functions available in dREL)  
> on the
> other:
>
> "My comment at this stage would be that defining a coupling mechanism
> between CIF and a given language is not a large task, due to the
> simplicity of the CIF syntax, whereas adding lots of stuff to dREL
> would be a serious task and has some important downsides (loss of
> simplicity being an important one). Apropros the
> simplicity of the coupling mechanism, I (predictably) quite like my
> Python model of a CIF file as a hash table of CIF data block objects
> indexed by datablock name, and the datablock objects are themselves
> hash tables of strings/lists of strings indexed by dataname.  This
> model would appear to translate pretty easily into most other
> languages.  What then remains is some syntactic sugar (the use of
> square brackets to do key-based lookup is nice in dREL) which can be
> replaced in another language by a few standard methods."
>
> There was a lot more to the discussion, but let us try to settle a
> direction:
>
> Should we be trying to extend dREL to support more than just CIF,
> specifically NeXus, making something we might call dREL++, or  
> should the
> language for this broader task be something distinct from dREL with a
> distinct name.  In practice, in either case, I suspect all of this  
> will be
> built on a python base, or something similar, as James suggests, but
> names do matter,
>
> Comments please.
>
> Regards,
>    Herbert
>
> =====================================================
>   Herbert J. Bernstein, Professor of Computer Science
>     Dowling College, Kramer Science Center, KSC 121
>          Idle Hour Blvd, Oakdale, NY, 11769
>
>                   +1-631-244-3035
>                   yaya at dowling.edu
> =====================================================
> _______________________________________________
> comcifs mailing list
> comcifs at iucr.org
> http://scripts.iucr.org/mailman/listinfo/comcifs

********************************************************************
Brian H. Toby, Ph.D.                            office: 630-252-5488
Materials Characterization Group Leader, Advanced Photon Source
9700 S. Cass Ave, Bldg. 433/D003             work cell: 630-327-8426
Argonne National Laboratory         secretary (Marija): 630-252-5453
Argonne, IL 60439-4856         e-mail: brian dot toby at anl dot gov
********************************************************************


-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://scripts.iucr.org/pipermail/comcifs/attachments/20081211/9b324898/attachment-0001.html