[Cif2-encoding] Splitting of imgCIF and other sub-topics. .. .

Mon Sep 13 11:22:30 BST 2010

I questioned:

"For example, if mandatory, does that mean it becomes impossible to create a 
non-UTF8 CIF without using
CIF2-aware software?"

In some respects this might not be a bad idea - i.e.restricting the use of 
non-UTF8 to CIF2-aware systems...

Simon (thinking aloud)

________________________________
From: SIMON WESTRIP <simonwestrip at btinternet.com>
To: Group for discussing encoding and content validation schemes for CIF2 
<cif2-encoding at iucr.org>
Sent: Monday, 13 September, 2010 11:05:12
Subject: Re: [Cif2-encoding] Splitting of imgCIF and other sub-topics. .. .

Yes - I beleive that such a declaration should be mandatory for all non-UTF8 
CIF2 files,
and agree that a supporting checksum mechanism would be very useful to 
CIF2-aware
programs. Until I've revisited the checksum scheme, I can not say that the 
checksum should be mandatory too.
For example, if mandatory, does that mean it becomes impossible to create a 
non-UTF8 CIF without using
CIF2-aware software?

I need to review the discussions on checksums and indeed the various forms that 
such a declaration might take,
but I do beleive in the principle that it should be mandatory for all 
'stand-alone' non-UTF8 CIF2 files.
If a CIF is packaged in a container, then it will be the job of non-CIF software 
to retreive it from the container
and deliver it in its original form. So a non-UTF8  CIF packaged in a non-UTF8 
container (or even a UTF8 container)
should still carry its non-UTF8 declaration.

Cheers

Simon

________________________________
From: James Hester <jamesrhester at gmail.com>
To: Group for discussing encoding and content validation schemes for CIF2 
<cif2-encoding at iucr.org>
Sent: Monday, 13 September, 2010 6:24:42
Subject: Re: [Cif2-encoding] Splitting of imgCIF and other sub-topics. .. .

Hi Simon: the issue with such an encoding declaration is that it is
not supported by generic text tools, and so would not be  automatically
inserted, updated or respected when creating, editing (ie open in one
encoding, save in another) or transcoding a CIF2 file.  This means it
has no status beyond a hint that could cause as many problems as it
solves. Such a declaration becomes more robust if accompanied by the
checksum that John B suggested.  The checksum gives some guarantee
that the encoding has been checked by a CIF-aware program.

If you are proposing that such a declaration and checksum be mandatory
for all non-UTF8 CIF2 files (not only during transfer), I agree with
you that this would be acceptable.

On Sat, Sep 11, 2010 at 6:59 PM, SIMON WESTRIP
<simonwestrip at btinternet.com> wrote:
> Dear all
>
> I have found recent exchanges, especially Herbert's contributions regarding
> the real-world use of  imgCIF, very
> enlightening. Primarily for reasons of flexibility, I now find myself
> inclined to support a CIF specification
> that allows a variety of encodings, provided that such are "clearly and
> unambiguously defined".
>
> To me, the clear and unambiguous definition should encompass a clear and
> unambiguous *declaration*
>  of the encoding; in the absence of such a declaration in the CIF or in its
> container, a default encoding
> should be assummed, either the default CIF encoding (which I think most
> agree should be UTF8) or inherited
> from the container?
>
> Though CIF1 has been successful without such a declaration (largely because
> of the ASCII restriction),
> I beleive it is essential in the case of CIF2.
>
> Cheers
>
> Simon
>
>
>
>
>
>
>
>  ________________________________
> From: "Bollinger, John C" <John.Bollinger at STJUDE.ORG>
> To: Group for discussing encoding and content validation schemes for CIF2
> <cif2-encoding at iucr.org>
> Sent: Friday, 10 September, 2010 19:24:05
> Subject: Re: [Cif2-encoding] Splitting of imgCIF and other sub-topics. .. .
>
>
> On Friday, September 10, 2010 11:02 AM, Herbert J. Bernstein wrote:
>>As I have said before, we went through this approach
>>in 1997 and ended up going the other way -- treating the
>>text-based CIF and the binary CBF as parts of the _same_
>>format, not two different formats, not one being a serialization
>>of the other, but the same format.  This may seem  like a
>>minor distinction, but it actually has strong implications
>>for software design and implementation, ensuring that
>>binaries in a CIF context are just a particular type of data
>>handled with all the same mecnahisms as ASCII data, allowing,
>>for example, multiple diffraction images and thumbnails in
>>one file in an order-independent way.
>>
>>You may be interested to know that the false dichotomy between
>>binary and text-based representations is not starting
>>to imapct HDF5, requiring some significant effort to now
>>work in database access, an aspect CIF1 supports -- why
>>throw it away for CIF2?
>
> Herb,
>
> Perhaps you're reading more into my comments than I intended to put there.
> In particular, I did not aim to suggest one on-disk/wire format should be a
> serialization of another, but  rather that *all* on-disk/wire formats be
> characterized in terms of serialization of the Unicode character sequences
> described by most of the spec.  I meant "text" in that sense -- a sequence
> of Unicode characters -- not in the sense of a sequence of bytes conforming
> to some particular set of local conventions for text.  I meant
> "serialization" in the general sense of any reversible transformation of CIF
> text into a byte sequence, including those that rely on interpreting the CIF
> syntax.  That's aimed primarily at recognizing the use case in which CIF2 is
> embedded in or transformed into some other format, such as XML.
>
> I postulate, but do not specify, a serialization form defining the CIF2
> version of what we have conventionally called "a CIF."  The details of that
> form are exactly what this list was established to discuss, and I did  not
> intend to imply a particular resolution of our ongoing debate.  It was
> perhaps a mistake to include imgCIF/CBF on the list of possible alternative
> serialization forms, as it is far from settled whether it will fit under the
> umbrella of the 'CIF File' serialization form.  I apologize if that caused
> confusion.
>
> [... I wrote:]
>>> I think this matter would be best addressed by explicitly adopting an
>>> idea that we have discussed before: a formal separation between the
>>> definition of CIF text (i.e. James's "CIF2-conformant character stream") and
>>> the particular kind of packaging that we are accustomed to calling "a CIF"
>>> or "a CIF file".  James's suggestion implies such a separation anyway, so
>>> let's not do it halfway.  Given such a separation, the explanatory comment
>>> could be as  simple as:
>>>
>>> "This specification's definition of the 'CIF File' serialization form for
>>> CIF2 text is not intended to preclude definition or use of other
>>> serialization forms, such as HDF5-based forms, XML-based forms, or
>>> imgCIF/CBF."
>>>
>>> I choose the term "serialization form" because it puts primary emphasis
>>> on the CIF text (which after all is the subject of the bulk of the
>>> specification).  Every correct serialization of CIF text is, by definition,
>>> transformable into CIF text form.
>
>
> Regards,
>
> John
> --
> John C. Bollinger, Ph.D.
> Department of Structural Biology
> St. Jude Children's Research Hospital
>
>
>
> Email Disclaimer:  www.stjude.org/emaildisclaimer
>
> _______________________________________________
> cif2-encoding mailing list
> cif2-encoding at iucr.org
> http://scripts.iucr.org/mailman/listinfo/cif2-encoding
>
> _______________________________________________
> cif2-encoding mailing list
> cif2-encoding at iucr.org
> http://scripts.iucr.org/mailman/listinfo/cif2-encoding
>
>

-- 
T +61 (02) 9717 9907
F +61 (02) 9717 3145
M +61 (04) 0249  4148
_______________________________________________
cif2-encoding mailing list
cif2-encoding at iucr.org
http://scripts.iucr.org/mailman/listinfo/cif2-encoding
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://scripts.iucr.org/pipermail/cif2-encoding/attachments/20100913/4d4b820f/attachment.html 