[Cif2-encoding] Splitting of imgCIF and other sub-topics. .. .

[James Hester]

The original concept was to edit the non UTF8 files in the text editor
of choice, then run a simple checksumming application (that
understands CIF2 syntax) to update the checksum.  This application
would also pick out sections of text that would be displayed
incorrectly in the wrong encoding, and ask the user to confirm that
the text was displayed correctly.  Such an application could be made
freely available by the IUCr.

On Mon, Sep 13, 2010 at 8:22 PM, SIMON WESTRIP
<simonwestrip at btinternet.com> wrote:
> I questioned:
>
> "For example, if mandatory, does that mean it becomes impossible to create a
> non-UTF8 CIF without using
> CIF2-aware software?"
>
> In some respects this might not be a bad idea - i.e.restricting the use of
> non-UTF8 to CIF2-aware systems...
>
> Simon (thinking aloud)
>
> ________________________________
> From: SIMON WESTRIP <simonwestrip at btinternet.com>
> To: Group for discussing encoding and content validation schemes for CIF2
> <cif2-encoding at iucr.org>
> Sent: Monday, 13 September, 2010 11:05:12
> Subject: Re: [Cif2-encoding] Splitting of imgCIF and other sub-topics. .. .
>
> Yes - I beleive that such a declaration should be mandatory for all non-UTF8
> CIF2 files,
> and agree that a supporting checksum mechanism would be very useful to
> CIF2-aware
> programs. Until I've revisited the checksum scheme, I can not say that the
> checksum should be mandatory too.
> For example, if mandatory, does that mean it becomes impossible to create a
> non-UTF8 CIF without using
> CIF2-aware software?
>
> I need to review the discussions on checksums and indeed the various forms
> that such a declaration might take,
> but I do beleive in the principle that it should be mandatory for all
> 'stand-alone' non-UTF8 CIF2 files.
> If a CIF is packaged in a container, then it will be the job of non-CIF
> software to retreive it from the container
> and deliver it in its original form. So a non-UTF8 CIF packaged in a
> non-UTF8 container (or even a UTF8 container)
> should still carry its non-UTF8 declaration.
>
> Cheers
>
> Simon
>
> ________________________________
> From: James Hester <jamesrhester at gmail.com>
> To: Group for discussing encoding and content validation schemes for CIF2
> <cif2-encoding at iucr.org>
> Sent: Monday, 13 September, 2010 6:24:42
> Subject: Re: [Cif2-encoding] Splitting of imgCIF and other sub-topics. .. .
>
> Hi Simon: the issue with such an encoding declaration is that it is
> not supported by generic text tools, and so would not be automatically
> inserted, updated or respected when creating, editing (ie open in one
> encoding, save in another) or transcoding a CIF2 file.  This means it
> has no status beyond a hint that could cause as many problems as it
> solves. Such a declaration becomes more robust if accompanied by the
> checksum that John B suggested.  The checksum gives some guarantee
> that the encoding has been checked by a CIF-aware program.
>
> If you are proposing that such a declaration and checksum be mandatory
> for all non-UTF8 CIF2 files (not only during transfer), I agree with
> you that this would be acceptable.
>
>