Data Standards Committee Report 2006

The committee provided partial sponsorship for a workshop held as part of the ACA annual meeting in July 2006.  The title of the workshop was Management of Synchrotron Image Data: imgCIF File System and Beyond.  It was organized by Herbert Bernstein and Robert Sweet.  Bernstein had worked very hard to obtain outside funding (NSF, DOE, and Advanced Detectors Systems Corp.) that not only is funding travel expenses and the workshop fee for every participant of this workshop, and provided lunch and refreshments, but also would provide initial funding for two more workshops.  These two workshops are envisioned to occur roughly six months from now, and then at the next ACA summer meeting.  We are very grateful to Bernstein for his hard work in bringing this important workshop together; it wouldn't have happened without him.

The charge to the workshop was to "Organize the adoption of a standard, self-documenting format for diffraction image files."  Rather immediately, the focus was on adoption of the current imgCIF/CBF standard that has been approved by the IUCr COMCIFS.  The format of the meeting included over ten short talks, presenting a little history and background, and then descriptions by investigators, with various backgrounds and interests, of their own goals or apprehensions about adoption of the imgCIF standard.

The second segment of the meeting was open discussion on a range of topics: Consider the whole flow of information/knowledge in the experiment; where does imgCIF fit? How does one deal with corrected vs uncorrected data?  What are the impediments to immediate adoption: can we define a time line?  What about Intellectual Property, esp. as concerns compression algorithms.  What will be the mechanism for an open-source protocol for continuing development of imgCIF?  And finally, we should talk about the dictionary to understand whether to add new items now.

The conclusions and recommendations, annotated slightly by this committee, were these:

1. SR sources should start writing imgCIF image files as soon as possible, employing the already-approved dictionary.  These will be no worse than any of the file headers written now, in terms of being complete or accurate, but they will stimulate conversion of programs both that feed the experimental parameters, and that reduce the data.

2. An important but easy prerequisite for this is that someone must provide certified conversion programs that will convert from imgCIF to the existing standard native file format used for each detector system in the field.

3. The files should include both the corrected and the raw images, probably together in the same file.

4. Those interested in imgCIF should look for a way to assure continuity in development. Maintenance and development of the dictionary and software library need to be taken up by a consortium where continued interest, and especially continued funding, are assured.  There are several possibilities, and some study will be done by this committee and others in the coming year.

5.  There are some other ideas: should the dictionary, include parameters that might shift with time, like the beam center, each to be recorded with a time stamp to identify the history of these parameters.  Should the imgCIF community develop common cause with the NEXUS community to assure interoperability?  Should imgCIF include a highly compressed image of dead pixels with each data image?  These are all second-order questions that must not interfere with the initial effort at adoption.

This committee is neutral on the question of the usefulness of further workshops in the near term.

This committee would like to commend whoever is responsible for making arrangements for the lecture room for this workshop, especially the audio/visual equipment and WiFi, which worked well, the food service, and the view out the back window.  There were small misunderstandings about the food, but it was ample, high quality, on time, and served cheerfully.