[Imgcif-l] Reading CBF files from Python

Tue Jun 22 16:49:32 BST 2010

Graeme,

Vanilla image decompression is straightforward with cctbx:

from cbflib_ext import MiniCBFAdaptor
adaptor = MiniCBFAdaptor("image.cbf")
rawdata = adaptor.optimized_read_data(2048,2048)
#The data can be accessed by x,y address:
for x in xrange(2048):
  for y in xrange(2048):
    print rawdata[x,y]

I don't think there's anything in cctbx that will compress the data to 
file.  I'll work on this later today & see if I can wrap up the basic 
compression routine.

Nick

On 6/22/2010 2:03 AM, Graeme.Winter at Diamond.ac.uk wrote:
> Hi Nick,
>
> Thanks, I will put it somewhere useful. Would it be straightforward to
> have these C++ methods boosted to Python to perhaps read in a string of
> the compressed image, return a flex array of integers and the inverse?
> This would make them directly available in a most convenient manner!
>
> Image at: http://www.ccp4.ac.uk/xia/image.cbf
>
> Best wishes,
>
> Graeme
>
> -----Original Message-----
> From: imgcif-l-bounces at iucr.org [mailto:imgcif-l-bounces at iucr.org] On
> Behalf Of Nicholas K. Sauter
> Sent: 21 June 2010 16:51
> To: The Crystallographic Binary File and its imgCIF application to image
> data
> Subject: Re: [Imgcif-l] Reading CBF files from Python
>
> Graeme,
>
> The cctbx::cbflib_adaptbx has been optimized for decompression of
> Pilatus images--should be about 90 ms on newer Linux systems.  
> Compression hasn't been tested but could potentially be optimized along
> similar lines. 
>
> The code currently in cctbx::iotbx could be subclassed to read whatever
> CBF image you have--if you'd like to post your data somewhere I'll have
> a look.
>
> Nick
>
> On 6/21/2010 2:26 AM, Graeme.Winter at Diamond.ac.uk wrote:
>   
>> My rate limiting step right now is the reading and writing of the byte
>>     
>
>   
>> offset compression. I am going to guess that if I messed with the 
>> python paths and such I should be able to find your implementation. Do
>>     
>
>   
>> you have a method in there which will allow me to just pack and unpack
>>     
>
>   
>> the byte_offset compressed regions?
>>
>> This is the data in question:
>>
>> ###CBF: Version July 2008 generated by XDS
>>
>> data_FRAME.cbf
>>
>> _array_data.header_convention "XDS special"
>> _array_data.header_contents
>> ;
>> ;
>>
>> _array_data.data
>> ;
>> --CIF-BINARY-FORMAT-SECTION--
>> Content-Type: application/octet-stream;
>>      conversions="x-CBF_BYTE_OFFSET"
>> Content-Transfer-Encoding: BINARY
>> X-Binary-Size:   9450352
>> X-Binary-ID: 1
>> X-Binary-Element-Type: "signed 32-bit integer"
>> X-Binary-Element-Byte-Order: LITTLE_ENDIAN
>> X-Binary-Number-of-Elements:   9437184
>> X-Binary-Size-Fastest-Dimension:    3072
>> X-Binary-Size-Second-Dimension:    3072
>> (massive binary blob)
>>
>> The bit which is causing me problems is that the cbflib adaptor raises
>>     
>
>   
>> an exception as most of the "proper" cbf stuff (i.e. the CIF) is not 
>> present. Erk.
>>
>> Any suggestions will be pounced apon!
>>
>> Many thanks,
>>
>> Graeme
>>
>> -----Original Message-----
>> From: imgcif-l-bounces at iucr.org [mailto:imgcif-l-bounces at iucr.org] On 
>> Behalf Of Jon Wright
>> Sent: 21 June 2010 10:06
>> To: The Crystallographic Binary File and its imgCIF application to 
>> image data
>> Subject: Re: [Imgcif-l] Reading CBF files from Python
>>
>> Hi Graeme,
>>
>> There is a swig based python binding included with CBFlib in the pycbf
>>     
>
>   
>> subfolder. It should work for reading files (see pycbf_test1.py).
>> Further work was needed for writing. Let me know if you run into 
>> problems. Improvements are most welcome.
>>
>> Best
>>
>> Jon
>>
>> Graeme.Winter at Diamond.ac.uk wrote:
>>   
>>     
>>> Hi Folks,
>>>  
>>> Does anyone know if CBF images can be read from Python? I coded up 
>>> something in Python to do this but it's a spot slow to say the least.
>>> My interest is partly in reading the cbf "image" files calculated by 
>>> XDS to explain what it has been up to. I could code something up in 
>>> 'c' or a bridge to CBFlib but that would be hardish to maintain, so a
>>>       
>
>   
>>> solution using existing Python libraries would suit me better.
>>>  
>>> I looked quickly at PyCIFrw but this appears to be mostly about the 
>>> CIF side of things rather than the MIME encoded chunks. Please 
>>> correct
>>>     
>>>       
>>   
>>     
>>> me if I am wrong. I also tried iotbx.detectors.cbf.CBFImage but hit 
>>> an
>>>     
>>>       
>>   
>>     
>>> exception which probably relates to this being more of a picture than
>>>       
>
>   
>>> a diffraction image.
>>>  
>>> Thanks,
>>>  
>>> Graeme
>>>  
>>> Dr. Graeme Winter
>>> Software and MX Support Scientist
>>> Diamond Light Source
>>>  
>>> +44 1235 778091 (work)
>>> +44 7786 662784 (work mobile)
>>>  
>>>  
>>>
>>>
>>>
>>>     
>>>       
>> _______________________________________________
>> imgcif-l mailing list
>> imgcif-l at iucr.org
>> http://scripts.iucr.org/mailman/listinfo/imgcif-l
>>
>>   
>>     
> _______________________________________________
> imgcif-l mailing list
> imgcif-l at iucr.org
> http://scripts.iucr.org/mailman/listinfo/imgcif-l
>
>   