8 February 2000Manuel Delfino / CERN IT Division / CHEP 2000 A3971 Can HEP benefit from Open Source ? Manuel Delfino* European Organization for Nuclear

8 February 2000 Manuel Delfino / CERN IT Division / CHEP 2000 A397 1

Can HEP benefit from Open Source ?

Manuel Delfino*European Organization for Nuclear Research

(CERN)

*Permanent address: Physics Dept., Universitat Autonoma de Barcelona, Spain


What is Open Source ?

• Direct descendent of Internet toolkit and Berkeley Unix “hackers” ?

• A technologically related social movement ?

• An emerging business model ?

Make up your own mind by:1. Reading “The Cathedral and the

Bazaar” by Eric S. Raymond, ISBN 1-56592-724-9

2. Browsing http://www.opensource.org


Why a 20 year gap ?

In my opinion, part of it is technological:• Commoditization of computing

hardware• Huge increases in Internet speed

…but also part of it is socio-political:• Global familiarity brought by CNN,

inexpensive travel, cultural exchanges• Disappearance of many trade barriers• Arrival of a world that WANTED to be

interconnected and was READY to work together


But there are more ingredients…

Perhaps it has taken 20 years to realize that:

• Sharing your ideas and listening to other’s opinions is more valuable than carefully guarding your ideas

• Many people have similar problems• The value is in the ideas, but to fully

collaborate you have to share code• Vision and architecture are more effective

than management to create a following• Humility is a great quality


Bazaar-style development

Some of Raymond’s 19 principles:3. Plan to throw one away; you will, anyhow5. When you loose interest in a program, your last

duty … is to hand it off to a competent successor

6. Treating your users as co-developers is … route to rapid code improvement and … debugging

7. Release Early. Release Often. And listen to your customers

11. The next best thing to having good ideas is recognizing good ideas …

12. Often, the most striking and innovative solutions come from realizing that your concept of the problem was wrong


Role of Project Leader

“… the cutting edge of the open-source software will belong to people who start from individual vision and brilliance, then amplify it through the effective construction of voluntary communities…”

Other characteristics of open-source contributors:

• “…self-promotion … mercilessly criticized”• “…quality…must be left to speak for itself”• “…attacking the author rather than the

code is not done”


So, can HEP benefit from Open Source ?

(Actually, it already has…the real question is should

HEP do some of its development in open-source)


Almost 2/3 of CERN computer center is in Linux PCs


Opportunities…

Why does each major lab write its own management software for tape/disk ?(see E176, C223, C68, C308)

Did you know that Cisco wrote a print server system whose goals are the same as E369, and place it in open-source ?

Is our need for event-oriented statistical analysis similar enough to Migro’s ?


…but no magic wand

Warnings to the “bandwagon jumpers”:• Going Open-source implies a

commitment to deliver and maintain a high quality product.

• The product must be of sufficient interest to others and must be general enough.

• This can often result in the need to increase the resources in the original development team for quite some time before and after open-source release.


Is HEP really ready for open source ?

• It is said that open source projects are tapping the top 5% in quality developers worldwide.

• Certainly the project leaders are impressive.• Just as computing technologies build on the

“commoditization and banalization” of previous ones, the open-source developers count on people being fluent in everything from architecture and design to multiple languages and details of CORBA…

Does HEP have such people ?Are they allowed to contribute to community ?


The importance of vision and architecture

• Linux is by far the biggest successful open-source project. However, it inherited its architecture from Unix.

• Emacs is another great success, but it is a relatively simple toolkit. It’s success seems to be due to its flexibility.

• Mozilla (eg. Open-Netscape) is not doing so well. Perhaps it lacks vision and architecture ?

• The closer we get to experiment’s software, the more we seem to depend on our own HEP architects. But how much does Torvalds know about chips ? Could we still benefit from others?


How about HEP pseudo-open bazaar?

Within an experiment ?• It has been tried and it sort of worked• Sadly, degenerates to personal battles• Seems hard to convince experiment’s

management fond of librarians and “official code”

Between experiments ?• Perhaps there is an opportunity ?• How much can be moved from

experiment-specific to “HEP Foundation”?


What is the bottleneck for HEP ?

My personal opinion is that the HEP community as whole has not really absorbed the “Internet way to do things”

• Over-emphasis of face-to-face meetings• “Tribes” come together at meetings, show

“their stuff”, then go back and continue to do “their own thing” -> “Institutionalized forking”

• Very little open e-discussion• No productization: “Production on

Prototypes”• Management very zealous of people not

working “directly for the experiment”


So, can HEP benefit from Open Source?

My conclusion: Yes, it probably can… Thanks to the large effort made in the last 5 years to “align”

software techniques used in HEP with the rest of the world, at least now we can probably explain ourselves to others (though maybe less so to our own collaborators !!!)

But…• Carefully evaluate on a case-by-case basis

the cost and the benefits• Need project leaders with vision and

humility• Continuous training aligned with the world• Build trust using e-communication• Face investment without clear direct benefit• Convince management of virtues of open-

source

Documents

8 February 2000Manuel Delfino / CERN IT Division / CHEP 2000 A3971 Can HEP benefit from Open Source ? Manuel Delfino* European Organization for Nuclear