Revitalising COMCIFS
Herbert J. Bernstein
yayahjb at gmail.com
Mon Aug 16 10:43:32 BST 2021
Dear James,
I understand the good intentions behind the idea of the separation you
propose, which is exactly the same as the
separation between "line" and "management" in corporate america, but in
software engineering at least, despite
the best of intentions it has not worked here. The tip-off as to what goes
wrong is right there in the name "Technical
Advisory Committee" making it advisory rather than operational. If the
people with the technical skills are not
full members of the decision making entity it is much too easy to
downweight, ignore or over-ride their after-the-fact
advice. It is not that the decision makers do not intend to follow the
advice they promised to respect, it is that
the process of having two separate bodies forces too long a communication
path in the course of the decision making,
so that the necessary technical input is coming too late for it to derail
decisions that have already been made. If the
people with the technical knowledge and the users of that technical
knowledge do not talk to one-another directly
during the formative stages of making decisions, those decisions will be
very hard to undo. In software engineering
there is an approach called "participatory design" or the
"Scandanavian Method" in which the users of a proposed
software product and the implementers work together without their managers,
talking directly and exchanging plans
until they get down to something that both satisfies the needs of the users
and is something that the implementers
can really implement. This is a very old method, but is one of the very
few software engineering approaches
that actually works. I suggest we follow the same approach, and instead of
having two bodies -- COMCIFS and
a TAC, have one CIF Dictionary Task Force (CDTF) using the RFC approach
developed by the Internet Engineering
Task Force (IETF) and try to emulate the success of the original IETF in
doing very complex design tasks that
managed to satisfy a huge range of user communities. Yes, this may well
violate IUCr rules on how to structure
committees and commissions, but we already have COMMDAT if we need a
formally proper entity. For this work
we don't need a formally correct hierarchical structure -- we need results
that work for our multiple user communities.
Regards,
Herbert
On Mon, Aug 16, 2021 at 5:02 AM James H <jamesrhester at gmail.com> wrote:
> Hi Herbert,
> Thanks for this thoughtful contribution. Regarding meeting dates, if
> people cannot make any of the suggested dates that will be clear from the
> doodle poll and we'll try again.
> I take your point about "needing deep knowledge of CIF". That is why I
> have proposed a technical advisory committee (TAC) (the current ddlm-group)
> that has the final say on any technical issues and would indeed require
> good knowledge of CIF. There are nevertheless broad "management" style
> issues that do not need a deep knowledge of CIF. For example, our current
> conversation requires understanding of committees and standards processes> but not CIF in particular. So I think there is room to at least discuss a
> separation between management and technical functions, and doing it right> unlike the corporate models you describe. For example, one might give the
> TAC a veto on any COMCIFS decision or require the chair to be a
> past/present member of the TAC. This separation gives people who might no> feel so confident about their CIF knowledge, but have other skills, the
> chance to contribute and allow the technically-skilled people to focus on
> what they do best.
> all the best,
> James.
> PS: watching a software-oriented stream from the IUCr meeting at the
> moment. So, even if I'm not across the technical side of particular
> software, I can still ask a question around long-term succession planning
> as I know that's an issue with software in general. These are the sorts o> non-technical questions that COMCIFS could address without needing deep
> knowledge of CIF.
> On Mon, 16 Aug 2021 at 17:31, Herbert J. Bernstein <yayahjb at gmail.com>
> wrote:
>> Dear James,
>> Let me pick up one important point of "5 new contributors" and suggest
>> adding Aaron Brewster of LBL. He has
>> been actively involved in work related both to imgCIF and to the
>> interaction with NeXus and has been involved
>> with the DIALS effort and with XFEL data collection and processing
>> worldwide for many years.
>> In terms of a meeting date, Please note the combination of Labor Day
>> and the Jewish high holidays will make
>> organizing anything between 1 September and 16 September difficult, and,
>> more depressingly, in the US at
>> least the CoVID numbers are getting worse and worse. I would suggest no>> waiting too long after COMMDAT.
>> Early September is not looking like a good time to schedule anything.
>> I am concerned about the approach of "not requir[ing] a deep knowledge
>> of CIF". This very much reminds me of
>> the U.S. approach to both business organization and to software
>> engineering in which there is a deliberate separation
>> of "line" from "management" with managers talking to other managers and
>> making decisions about what the line
>> people will do. The result is almost always very poor decision-making,
>> making many US workplaces into Dilbert
>> cartoons and making US software engineering into a world-wide joke for
>> more than 6 decades. If we are going to
>> continue to be a body responsible for "maintenance of the CIF standard",
>> at the very least, we need to insist that
>> our members and decision makers go to the trouble of gaining a "deep
>> knowledge of CIF" or what results may well be
>> as much of a non-functional product as ISO OSI was in the 1970s, and
>> agile programming in most corporate environments is
>> today in the US today. I am not saying everybody has to be an expert on
>> every aspect of CIF, databases, NeXus,
>> HDF5, etc., but that everybody has to respect the need for such expertis>> in the heart of our decision-making and
>> make sure those with a deep knowledge of CIF are intimately involved in
>> the decisions about CIF.
>> Regards,
>> Herbert
>>
>> On Sun, Aug 15, 2021 at 10:12 PM James H <jamesrhester at gmail.com> wrote:
>>
>>> Dear All,
>>>
>>> A COMCIFS meeting is certainly a good idea. I would prefer the week
>>> after Commdat to give us time to have some preliminary discussions here
>>> about topics raised. I will ask Brian to arrange a doodle poll.
>>>
>>> Herbert - thanks for the reminder about other standards organisations -
>>> the minimalist RFC approach does seem remarkably successful.
>>>
>>> John - good to have some insight into how things work in the MM world.
>>> Any particular suggestions on ways to improve COMCIFS operations based on
>>> the clearly successful MM experiences would be welcomed, I think. As far as
>>> bureaucracy goes I'd opt for "no more than necessary" rather than "none at
>>> all". At the moment in the "none at all" situation, if you aren't prepared
>>> to wade through COMCIFS discussions for the last decade (or more) it can be
>>> difficult to have any idea as to how we do things. So it may be that th>>> final resting place of my proposals is that we prepare a few RFC style
>>> documents that explain how we do the important things, together with an RFC
>>> that describes how to make new RFCs. I would be keen to delegate the actual
>>> dictionary construction process that you describe to separate groups that
>>> can figure out for themselves the best way to work - if we provide them
>>> with a suggested roadmap and offer them a place on Github to get starte>>> with that would probably be enough.
>>>
>>> Again, an important goal of this initiative is to increase the number o>>> people contributing in the CIF space. There is a serious bottleneck at the
>>> moment as can be seen from the number of unresolved threads in coreDMG, the
>>> page of unresolved issues on the Github respository, and various dictionary
>>> initiatives languishing. Many of these do not require deep knowledge o>>> CIF. I'm not expecting a broad influx of people but even 5 new contributors
>>> who are willing to take on one-off tasks would make an enormous difference.
>>>
>>> all the best,
>>> James.
>>>
>>> On Sun, 15 Aug 2021 at 08:19, john.westbrook at rcsb.org <
>>> jdwestbrook at gmail.com> wrote:
>>>
>>>> I was just inquiring of James if there was an IUCr COMCIFS meeting
>>>> scheduled. 10ET on the 25th would
>>>> work for me.
>>>>
>>>> John
>>>>
>>>> On 8/14/21 5:47 PM, Herbert J. Bernstein wrote:
>>>> > Shouldn't we have a zoom meeting just after the IUCr meeting to
>>>> discuss this and any other open
>>>> > COMCIFS issues? I believe that the CommDat meeting is scheduled for
>>>> 8am NY time on Wedneday,
>>>> > 25 August, 1 pm London Time, 2 pm Prague time, 10 pm Sydney time.
>>>> Might it be sensible for us to
>>>> > have a COMCIFS meeting a little later the same day, say 10 am NY
>>>> time, 3 pm London Time, 4 pm
>>>> > Prague time, midnight Sydney time?
>>>> >
>>>> > In general, I think the most important step we could make in
>>>> revitalizing COMCIFS would be to meet
>>>> > regularly, certainly at least once a year.
>>>> >
>>>> > For the moment the agenda could be:
>>>> > revitalizing COMCIFS
>>>> > report of current activities
>>>> > ITVG
>>>> > old business
>>>> > new business
>>>> >
>>>> >
>>>> > On Sat, Aug 14, 2021 at 5:18 PM john.westbrook at rcsb.org <mailto:
>>>> john.westbrook at rcsb.org> <jdwestbrook at gmail.com
>>>> > <mailto:jdwestbrook at gmail.com>> wrote:
>>>> >
>>>> > Hi all,
>>>> >
>>>> > I agree with Herbert’s friend’s opinion that adding process and
>>>> bureaucracy will have a negative impact on productivity. In my
>>>> > view,
>>>> > fancy standards processes are wonderful for managing large group>>>> or well-supported and highly motivated individuals. In contrast,
>>>> > I believe that such approaches do little but impede the efforts
>>>> of smaller groups of volunteers with only limited free cycles to
>>>> > bring to a project.
>>>> >
>>>> > While the dynamics of MM dictionary development may not be
>>>> representative of the overall COMCIFS development effort, we have had
>>>> > success working with standing committees of developers and key
>>>> stakeholders in particular domain areas. wwPDB team members try to
>>>> > facilitate discussion and generally reduce the friction of movin>>>> innovation in science, technology, methodology into dictionary
>>>> > semantics that works and plays well with the rest of the MM data
>>>> ecosystem. This work is conducted in regular virtual meetings and
>>>> > with the help of common software development collaboration
>>>> channels available on GitHub. Discussions typically center around
>>>> > evaluating if prototype dictionary extensions and the viability
>>>> of implementing these within the most widely used software tools
>>>> > and
>>>> > packages. Our focus is always on trying to achieve a standard
>>>> data representation coupled with a consensus implementation that can
>>>> > move new data into the repository.
>>>> >
>>>> > In our experience, the success of any dictionary development
>>>> effort centrally depends on getting the key stakeholders together in
>>>> > regular face-to-face or now virtual meetings. As I am sure, you
>>>> appreciate, nailing down semantics in almost any domain is always
>>>> > more complicated than initially anticipated, and discussions
>>>> often evolve to an unanticipated outcome. Such discussions are
>>>> > tedious, and in my view, inefficiently conducted in protracted
>>>> e-mail exchanges. Getting a virtual consensus on a trial set of
>>>> > semantics in periodic zoom meetings, followed by some intervenin>>>> time to develop and test prototype implementations, is the
>>>> > process
>>>> > that we currently find successful in the MM space.
>>>> >
>>>> > Best -
>>>> >
>>>> > John
>>>> >
>>>> > On 8/13/21 4:56 AM, Herbert J. Bernstein via comcifs wrote:
>>>> > > Dear Colleagues,
>>>> > >
>>>> > > James is right about the need for change, and I support hi>>>> suggestions. In
>>>> > > addition, I would suggest taking a look at the RFC process as
>>>> described in
>>>> > >
>>>> > > https://en.wikipedia.org/wiki/Request_for_Comments <
>>>> https://en.wikipedia.org/wiki/Request_for_Comments>
>>>> > <https://en.wikipedia.org/wiki/Request_for_Comments <
>>>> https://en.wikipedia.org/wiki/Request_for_Comments>>
>>>> > >
>>>> > > which has been very successful in achieving some remarkable
>>>> results over many
>>>> > > decades,
>>>> > >
>>>> > > the ISO standardization process
>>>> > >
>>>> > >
>>>> https://en.wikipedia.org/wiki/International_Organization_for_Standardization
>>>> > <
>>>> https://en.wikipedia.org/wiki/International_Organization_for_Standardization
>>>> >
>>>> > > <
>>>> https://en.wikipedia.org/wiki/International_Organization_for_Standardization
>>>> > <
>>>> https://en.wikipedia.org/wiki/International_Organization_for_Standardization
>>>> >>
>>>> > >
>>>> > > which has also produced many important results, but also some
>>>> serious mistakes,
>>>> > > and the IEEE Standards Association process
>>>> > >
>>>> > > https://en.wikipedia.org/wiki/IEEE_Standards_Association <
>>>> https://en.wikipedia.org/wiki/IEEE_Standards_Association>
>>>> > <https://en.wikipedia.org/wiki/IEEE_Standards_Association <
>>>> https://en.wikipedia.org/wiki/IEEE_Standards_Association>>
>>>> > >
>>>> > > which fits somewhere between the two others in success of its
>>>> efforts.
>>>> > >
>>>> > > I have a friend who insists that it is a terrible mistake
>>>> for an organization to become
>>>> > > "process driven", and he is, of course, right. What should
>>>> drive our activities should
>>>> > > be the effectiveness of the results we achieve, but well
>>>> defined, strong processes
>>>> > > used as a tool, not as an end in themselves, can be very
>>>> helpful in achieving those
>>>> > > results.
>>>> > >
>>>> > > I look forward to this discussion.
>>>> > >
>>>> > > Regards,
>>>> > > Herbert
>>>> > >
>>>> > > On Fri, Aug 13, 2021 at 1:51 AM James H via comcifs <
>>>> comcifs at iucr.org <mailto:comcifs at iucr.org> <mailto:comcifs at iucr.org
>>>> > <mailto:comcifs at iucr.org>>> wrote:
>>>> > >
>>>> > > Dear COMCIFS members,
>>>> > >
>>>> > > I believe it is time to assess how we do things on
>>>> COMCIFS, and to take some incremental steps towards streamlining our
>>>> > > activities and broadening our base of dictionary
>>>> contributors. To that end I've created a discussion document which you
>>>> > can read
>>>> > > at
>>>> https://github.com/COMCIFS/comcifs.github.io/blob/master/draft/CIF_processes_discussion.md
>>>> > <
>>>> https://github.com/COMCIFS/comcifs.github.io/blob/master/draft/CIF_processes_discussion.md
>>>> >
>>>> > > <
>>>> https://github.com/COMCIFS/comcifs.github.io/blob/master/draft/CIF_processes_discussion.md
>>>> > <
>>>> https://github.com/COMCIFS/comcifs.github.io/blob/master/draft/CIF_processes_discussion.md>>.
>>>> That document is also appended to
>>>> > > this email.
>>>> > >
>>>> > > Please discuss. In the absence of substantial objections,
>>>> I will take this as broad agreement with the document and
>>>> > proceed on
>>>> > > that basis.
>>>> > >
>>>> > > all the best,
>>>> > > James.
>>>> > >
>>>> ==========================================================================
>>>> > >
>>>> > > # Revitalising COMCIFS: Discussion
>>>> > >
>>>> > > DRAFT 2021-08-13
>>>> > >
>>>> > > See "Next Step" at the end for suggested next actions.
>>>> > >
>>>> > > # Introduction
>>>> > >
>>>> > > After a decade as COMCIFS chair I have (finally, some
>>>> might say)
>>>> > > perceived a couple of related issues:
>>>> > >
>>>> > > 1. Most of the work is falling on a few people, which is
>>>> unsustainable
>>>> > > and leads to too narrow a focus
>>>> > >
>>>> > > 2. Dictionary development is not keeping pace with the
>>>> science
>>>> > >
>>>> > > This discussion document contains some ideas for a way
>>>> forward which
>>>> > > I'd like you all to consider and to bring your combined
>>>> experience of
>>>> > > committees and scientists on committees to bear.
>>>> > >
>>>> > > # Current situation
>>>> > >
>>>> > > Formally, COMCIFS is a subcommittee of the IUCr executive..
>>>> While we
>>>> > > are relatively minor compared to the commissions, as a
>>>> result we have
>>>> > > a great deal of flexibility in how we organise ourselves.
>>>> > >
>>>> > > COMCIFS currently operates in a relatively informal
>>>> fashion. Discussions
>>>> > > of policy are held on the official COMCIFS mailing list.
>>>> Discussions
>>>> > > relating to the Core dictionary are held on the core-DMG
>>>> mailing list.
>>>> > > Technical issues, including DDLm development, are
>>>> discussed either on
>>>> > > the DDLm mailing list or within the Github repositories.
>>>> > >
>>>> > > # Suggestions for improvement
>>>> > >
>>>> > > ## Suggestion 1: Document procedures and processes.
>>>> > >
>>>> > > The informal way of doing things is essentially
>>>> exclusionary to all
>>>> > > those "not in the club". In contrast, easy-to-find and
>>>> clear
>>>> > > procedures allow new contributors to feel confident that
>>>> they are
>>>> > > approaching a task correctly and thus lower the barriers to
>>>> > > contribution.
>>>> > >
>>>> > > Additionally, agreed and transparent procedures reduce th>>>> possibility
>>>> > > of mistakes or misunderstandings. I realise that I might
>>>> be sounding
>>>> > > (perhaps frighteningly) bureaucratic to some of you, but
>>>> my plan would
>>>> > > be to document no more than necessary to achieve the abov>>>> goals. It
>>>> > > is likely that most procedures would be a single page, if
>>>> that, and as
>>>> > > you will see below I'm suggesting that the quantity of
>>>> procedures
>>>> > > depends very much on the interest of COMCIFS in having
>>>> them.
>>>> > >
>>>> > > ## Suggestion 2: Technical Advisory Committee
>>>> > >
>>>> > > This would be the group currently called "ddlm-group"
>>>> which consults
>>>> > > on any changes to the foundational standards (DDLm and
>>>> dREL). This
>>>> > > group would become responsible for the detail of
>>>> implementing
>>>> > > procedures using Github, the IUCr website and so on.
>>>> > >
>>>> > > The idea of this group is to remove the (mostly perceived>>>> need for
>>>> > > COMCIFS members to be across the technical detail. Instea>>>> technical
>>>> > > questions/issues can be delegated to the TAC. Membership
>>>> models for
>>>> > > the TAC can be discussed, there are many to choose from i>>>> the open
>>>> > > source world, e.g. Python.
>>>> > >
>>>> > > ## Suggestion 3: Formally involve the relevant IUCr
>>>> commissions
>>>> > >
>>>> > > IUCr commissions have no formal relationship to
>>>> dictionaries that
>>>> > > cover their topics. However, it makes no sense that, for
>>>> example, the
>>>> > > powder diffraction commission has no expected input or
>>>> responsibility
>>>> > > for the powder dictionary.
>>>> > >
>>>> > > The IUCr executive have recently encouraged us to
>>>> formalise links with
>>>> > > commissions. This is important, as the IUCr executive are
>>>> the ones who
>>>> > > have the ability to hold commissions accountable for thei>>>> area of
>>>> > > expertise in the dictionaries.
>>>> > >
>>>> > > ## Suggestion 4: Lower barriers to participation
>>>> > >
>>>> > > All interested parties should be able to join both COMCIF>>>> and any
>>>> > > dictionary management lists that fall under our purview
>>>> > > automatically. If unproductive discussion due to too many
>>>> voices
>>>> > > becomes a problem then we can revisit this.
>>>> > >
>>>> > > ## Suggestion 5: Better information dissemination
>>>> > >
>>>> > > An informal newsletter covering recent developments helps
>>>> all parts
>>>> > > of the community understand what is going on without
>>>> having to visit
>>>> > > the various places in which things are happening.
>>>> > >
>>>> > > # First steps
>>>> > >
>>>> > > Creating and documenting processes takes time and energy.
>>>> However,
>>>> > > before involving the commissions these processes need to
>>>> be set up. So
>>>> > > process number 1 is the process for producing documents
>>>> (sort of like
>>>> > > ddl.dic is the dictionary for dictionaries). I propose th>>>> following
>>>> > > outline for this "procedure number 1".
>>>> > >
>>>> > > ## Creating procedures: procedure number 1
>>>> > >
>>>> > > The type of work that COMCIFS does is similar to the W3C
>>>> and other
>>>> > > standards bodies. I suggest that the International Virtua>>>> Observatory
>>>> > > Alliance documentation standards are a good reference point
>>>> > > (
>>>> https://www.ivoa.net/documents/DocStd/20170517/REC-DocStd-2.0-20170517..pdf
>>>> > <
>>>> https://www.ivoa.net/documents/DocStd/20170517/REC-DocStd-2.0-20170517..pdf
>>>> >
>>>> > > <
>>>> https://www.ivoa.net/documents/DocStd/20170517/REC-DocStd-2.0-20170517..pdf
>>>> > <
>>>> https://www.ivoa.net/documents/DocStd/20170517/REC-DocStd-2.0-20170517..pdf
>>>> >>).
>>>> > > These are themselves based on the W3C documentation
>>>> standards. Given
>>>> > > that our goals are considerably more modest than those
>>>> sprawling
>>>> > > organisations, we can aim for considerable simplification..
>>>> > >
>>>> > > The following three steps and documents should be tracked
>>>> on a
>>>> > > website: either in the IUCr CIF area, or Github
>>>> repository, or both.
>>>> > >
>>>> > > ### Step 1: Proposal
>>>> > >
>>>> > > A short statement outlining the nature, scope and
>>>> objectives of the
>>>> > > procedure is submitted to the COMCIFS mailing list, eithe>>>> directly or
>>>> > > via the COMCIFS secretary or chair. A draft document may
>>>> be provided
>>>> > > but is not necessary.
>>>> > >
>>>> > > ### Step 2: Working group
>>>> > >
>>>> > > If the procedure is seen as worthwhile by COMCIFS, a
>>>> working group is
>>>> > > formed and tasked to produce a Working Draft.
>>>> > >
>>>> > > ### Step 3: Approved document
>>>> > >
>>>> > > The Working draft is presented to COMCIFS for feedback an>>>> approval.
>>>> > > Once approved, the working draft becomes an approved
>>>> document.
>>>> > >
>>>> > > # Other required procedures
>>>> > >
>>>> > > After agreeing on something like the above process, I
>>>> suggest we need
>>>> > > to deal with the following as well:
>>>> > >
>>>> > > - Procedure for COMCIFS approval
>>>> > > - Procedure for adding a dictionary definition
>>>> > > - Procedure for creating a new dictionary
>>>> > >
>>>> > > # Next step
>>>> > >
>>>> > > The "Creating a procedure" proposal is discussed by
>>>> COMCIFS as per
>>>> > > Step 1 above. If COMCIFS agrees, a working group is forme>>>> to document
>>>> > > the process for creating new procedures, as per Step 2
>>>> above.
>>>> > > --
>>>> > > T +61 (02) 9717 9907
>>>> > > F +61 (02) 9717 3145
>>>> > > M +61 (04) 0249 4148
>>>> > > _______________________________________________
>>>> > > comcifs mailing list
>>>> > > comcifs at iucr.org <mailto:comcifs at iucr.org> <mailto:
>>>> comcifs at iucr.org <mailto:comcifs at iucr.org>>
>>>> > > http://mailman.iucr.org/cgi-bin/mailman/listinfo/comcifs <
>>>> http://mailman.iucr.org/cgi-bin/mailman/listinfo/comcifs>
>>>> > <http://mailman.iucr.org/cgi-bin/mailman/listinfo/comcifs <
>>>> http://mailman.iucr.org/cgi-bin/mailman/listinfo/comcifs>>
>>>> > >
>>>> > >
>>>> > > _______________________________________________
>>>> > > comcifs mailing list
>>>> > > comcifs at iucr.org <mailto:comcifs at iucr.org>
>>>> > > http://mailman.iucr.org/cgi-bin/mailman/listinfo/comcifs <
>>>> http://mailman.iucr.org/cgi-bin/mailman/listinfo/comcifs>
>>>> > >
>>>> >
>>>> > --
>>>> > John Westbrook
>>>> > RCSB, Protein Data Bank
>>>> > Rutgers, The State University of New Jersey
>>>> > Institute for Quantitative Biomedicine at Rutgers
>>>> > 174 Frelinghuysen Rd
>>>> > Piscataway, NJ 08854-8087
>>>> > e-mail: john.westbrook at rcsb.org <mailto:john.westbrook at rcsb.org>
>>>> > Ph: (848) 445-4290 Fax: (732) 445-4320
>>>> >
>>>>
>>>> --
>>>> John Westbrook
>>>> RCSB, Protein Data Bank
>>>> Rutgers, The State University of New Jersey
>>>> Institute for Quantitative Biomedicine at Rutgers
>>>> 174 Frelinghuysen Rd
>>>> Piscataway, NJ 08854-8087
>>>> e-mail: john.westbrook at rcsb.org
>>>> Ph: (848) 445-4290 Fax: (732) 445-4320
>>>>
>>>
>>>
>>> --
>>> T +61 (02) 9717 9907
>>> F +61 (02) 9717 3145
>>> M +61 (04) 0249 4148
>>>
>>
> --
> T +61 (02) 9717 9907
> F +61 (02) 9717 3145
> M +61 (04) 0249 4148
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.iucr.org/pipermail/comcifs/attachments/20210816/a47c03f6/attachment-0001.htm>
More information about the comcifs
mailing list