[Cif2-encoding] How we wrap this up

[Bollinger, John C]

Dear Simon,

It is exactly this sort of issue that drove me to support more permissive encoding rules and ultimately to devise the UTF-8 + UTF-16 + local proposal.

Do please think about the considerations Herb raised.  As you reconsider your votes, I urge you also to ask yourself what, *precisely*, a "text file" is, and to consider whether your answer is functionally different from my "local".  If you decide not, then please consider what that answer implies about CIF2 support of UTF-8 and UTF-16 (which evidently you favor) under each option on the table, especially for CIFs containing non-ASCII characters.  Whatever you decide about the meaning of "text file", please consider whether reasonable people might reach a different conclusion, as I assert they might do, and to what extent the standard needs to address that.


Regards,

John
--
John C. Bollinger, Ph.D.
Department of Structural Biology
St. Jude Children's Research Hospital


>From: cif2-encoding-bounces at iucr.org [mailto:cif2-encoding-bounces at iucr.org] On Behalf Of SIMON WESTRIP
>Sent: Friday, September 24, 2010 7:53 AM
>To: Group for discussing encoding and content validation schemes for CIF2
>Subject: Re: [Cif2-encoding] How we wrap this up. .
>
>Dear Herbert
>
>Not for the first time, I find your arguement persuasive. Brian's vote and explanation have also raised some
>questions that I would like to look into.
>
>I will confirm or otherwise my vote as soon as possible, assuming that is OK with James and assuming that
>this round of votes might wrap this up.
>
>Cheers
>
>Simon
>
>________________________________________
>From: Herbert J. Bernstein <yaya at bernstein-plus-sons.com>
>To: Group for discussing encoding and content validation schemes for CIF2 <cif2-encoding at iucr.org>
>Sent: Friday, 24 September, 2010 13:17:14
>Subject: Re: [Cif2-encoding] How we wrap this up
>
>If he ignores the standard, in most cases all he has to do to comply with CIF2 is to run whatever applications he currently runs to produce CIF1 and, perhaps, in some cases, run a minor edit pass at the end, to convert for the minor syntactive differences and/or changed tags required to comply with CIF2 and the new dictionaries, but he is unlikely to have to do anything to deal with the messy business of whether his encoding is really a proper UTF8 encoding or not.

>The punishment if he tries to comply, is that he has to totally uproot and reconfigure the environment in which he produces CIFs from whatever he is currently doing to create an enviroment in which he can reliably create and, more importantly, transmit compliant UTF8 files.  This can be very tricky if he does only a partial job, say fudging in one special application (yet to be written), because if he stays with his old system, all kinds of tools will keep trying to transcode whatever he has produced back to whatever his system considers a standard. Those of us who have files, applications and tools that have lived through several generations of macs are living proof of the problem. Macs now have excellent UTF8/16 unicode support, but every once in a while in working with a unicode file I find it has been strangely and unexpectedly converted to something else, and it can be really tricky to spot when the unaccented roman text part has been left untouched but just a few accented letters have gotten different accents.

>Mandating UTF8 is simply trying to shift a serious software problem from the central handlers of CIF (IUCr, PDB, etc.) to the external users. Most users will probably have the good sense to simply ignore the demand and leave the burden just where it is now.  A few sophisticated users will probably adapt with no trouble, but the punishment for those users who blindly follow orders before we have a complete multiplatform supporting infrastructure in place by mandating UTF8 is severe, expensive and undeserved.  Until and unless we have developed solid support, we will just be alienating people from CIF.  I will continue to oppose such a move.

[...]


Email Disclaimer:  www.stjude.org/emaildisclaimer