[Cif2-encoding] How we wrap this up

[Bollinger, John C]

On Tuesday, September 28, 2010 4:54 AM, SIMON WESTRIP wrote:
[...]
>So I think the 'As for CIF1...' proposals with this explicit default encoding is certainly
>heading towards a workable compromise. Herbert is unhappy to mandate a particular encoding
>for non-ASCII use, but has agreed to recommend UTF8 and UTF16 in such cases.
>Such recommendations along with a default encoding that should be adopted in the absence of
>any pointers to the contrary could boil down to UTF8/16 + local in all intents and purposes,
>and could boil down to UTF8/16 if you want to use non-ASCII text.

Recommending UTF-8 and / or UTF-16 without mandating support for one or both does not get us where I insist we need to be.  In particular, the point of requiring support for at least one specific encoding applicable to the entire CIF2 character repertoire is to provide a means *wholly within the standard* by which conforming parties can be certain of communicating arbitrary CIF content accurately.  The various Unicode Transformation Formats have additional desirable properties in that regard that we have covered extensively.

If establishing UTF-8 as the default encoding confers a mandate to support it then where indeed is the great distinction between 'As for CIF1...' and UTF-8 (+- UTF-16) + local?  If there is one then it can only be in the definition of "text" on the one hand and "local" on the other, which is to say in the details of support for non-UTF-x encodings.  That is an area where perhaps we could find a consensus, or at least a strong majority opinion.  For that to happen, I require definitions of "text" and "text file" sufficient to program to.  James has asked for the same.  "local" already provides such definitions, intended to cover the cases that CIF1 allows and UTF-8 +- UTF-16 does not.  Are there cases it misses that should be covered?  Are there cases it covers that should be missed?


Regards,

John
--
John C. Bollinger, Ph.D.
Department of Structural Biology
St. Jude Children's Research Hospital


Email Disclaimer:  www.stjude.org/emaildisclaimer