HyperNews for LHCb

13th September 2006

Tim Adye / Jim Libby 1

HyperNews for LHCbHyperNews for LHCb

Tim Adye, Rutherford Appleton Laboratorypresented by

Jim Libby, Oxford University

Physics - HLT – Computing PlenaryLHCb Collaboration Week, Heidelberg

13th September 2006

13th September 2006


Outline

1. What is HyperNews?2. Why switch?3. HyperNews in LHCb4. How do we get started?

a. User registration and subscriptionb. Migrating existing lists

5. Concluding thoughts

ReferencesBackup slides

13th September 2006


What is HyperNews?

• We could use it as a replacement for most of our Simba2 mailing lists• Also has features of

• bulletin board systems (like RootTalk) and• news systems (like Usenet)

• Not intended to replace TWiki or web pages

• Widely used in HEP• BaBar, H1, STAR, Geant4, CMS, ATLAS,…

• Heavily used: BaBar 400,000 messages since 1997;CMS 32,000 (December 2005);Atlas 8,000 (January 2006); LHCb test 200

• Continuing development• HyperNews server managed by LCG

• runs independent instances for all four LHC experiments

13th September 2006


Why Change from Simba E-mail Lists?

• Web archive is central to HyperNews• Web access to Simba archives very cumbersome

• Not so bad via Outlook, but only really for CERN local users

• single, integrated, system instead of disjoint mailing lists• can search all fora at once• see all recent posts (all or a subset of fora)

• displays threads• Threads, sub-threads, individual postings have a static URL

for easy reference• Eg. in other postings, TWiki pages, private e-mail

• easy to browse, very fast to search• I searched for “database” in 400,000 BaBar posts. It found

18,000 in under a second

• Can post via the web or e-mail

13th September 2006


13th September 2006


13th September 2006


Keeping up to Date

• Can subscribe to receive posts as e-mails• Simple and powerful control of what you receive

• Whole forum or individual threads

• Reply to post with e-mail, or follow link to web display

• ... or keep up-to-date / catch up via the web• “recent posts” page

13th September 2006


Give it a try in the “HyperNews Test Forum”

13th September 2006


Lots of other nice features/uses

• “on holiday” switch• automatic thread subscription• full thread view for printing• refer to old thread with a simple URL• …• Forum ideas:

• new forum announcements, Q&A, chat, paper reviews, recipes, logbook…

• anyone can ask for a forum, even if of minority interest

Just ask any former BaBarian, and they’ll tell you how much they miss HyperNews

( )

13th September 2006


HyperNews for LHCb

• LHCb HyperNews running since July• Contains a few initial fora

• Tested by migrating lhcb-core-soft e-mail list• No big problems• Some concerns about how best to manage migration

• Can always “catch up” with the “recent postings” web page

• Once we get the go-ahead, new fora can be created on request

• Migrate existing lists adiabatically• Let each group manage their own transition in their own

time• A similar migration has been relatively painless in CMS

and Atlas

13th September 2006


How do I start using HyperNews?1. From https://hypernews.cern.ch/ select LHCb2. The first time (only) you use HyperNews, follow the

“new member” instructions• If you have an LHCb AFS (lxplus) account, you can use this to

register – just takes a minute• If you don’t, send an e-mail to the admin address listed there

([email protected])

• LHCb members without AFS accounts get a manually-created account

• Non-LHCb people can be subscribed to specific fora

3. Subscribe to the fora you are interested in• Just scan the list and check the boxes• If you subscribe to “HyperNews Announcements” then you will

learn about new fora as they are created.

... or just wait for the announcement and follow instructions

13th September 2006


How do I Migrate my E-mail list1. Post a 2-week warning to the old list, so people know

when the list will be migrated2. Add [email protected] as an owner of the old list

• Allows me to manage the transition for you

3. Select “Request a New Forum” and fill in the details• Include the old list name in the description

4. I will take care of the rest...i. Create the HyperNews forumii. post final notice on the old list and in “HyperNews

Announcements”• Will include reminders of how to register and subscribe

iii. block posts and subscriptions on the old list• This is important to prevent people accidentally posting to the old

list, not realising that the message only goes to old members• I will keep an eye out for people accidentally using the old list

13th September 2006


Final Thoughts

• Effective communication is crucial in a large geographically-distributed collaboration like LHCb• Meetings, documentation, TWiki, and e-mail all useful

forms of communication

• But e-mail has problems when collaborating with many people• private e-mail restricted to those in the loop• mailing lists OK for announcements, not so good for

discussion• mailing lists most useful if also easily used after the fact

• HyperNews addresses these issues• makes collaboration e-mail more Wiki-like• encourages open communication• opens up archive – documentation for free

13th September 2006


References

• HyperNews development• D.A.Smith, P.Elmer, T.Hung,

HyperNews - managing discussions in HEP, CHEP06

• http://hypernews.slac.stanford.edu/

• LHCb instance• https://hypernews.cern.ch/HyperNews/LHCb/top.pl

which is linked from• https://hypernews.cern.ch/

13th September 2006


Backup SlidesBackup Slides

13th September 2006


Design choices

• Why not automatically register all LHCb members?Why not automatically transfer subscriptions from the old lists?• Start as we mean to go on• Simba based on e-mail address, HyperNews based on account

name (usually tied to AFS account)• No 1-to-1 correspondence for many people, leading to confusion

and lots of extra work• Besides, self-subscription is trivial

• just check the box• can review what other lists to join on the same page

• Why tie HyperNews account to AFS account?• Allows quick, reliable, and secure initial authentication• Allows easy HN password setting, if forgotten

• Though many people let their web browser remember for them• Compatible with CMS and Atlas – simplifies management

13th September 2006


Why not use <insert-cool-technology> instead?

• I haven’t done a full survey but, for collaboration, I find HyperNews the most convenient• cf. Usenet, BBS (like RootTalk), discussion ‘blogs

• and of course mailing lists and private mails

• It is widely used in HEP, so familiar to some• Ask any former BaBarian, and they’ll tell you how

much they miss HyperNews

• LCG run the server for us• We know the developers

• at SLAC and CERN (Pete Elmer, CMS)

13th September 2006


Limitations of HyperNews

• All fora are open to all collaboration members• private lists should remain in Simba2

• All e-mails to/from HyperNews are converted to plain text• attachments are stripped off – give a URL instead

• good practice anyway – don’t fill up peoples’ inboxes• URLs are automatically turned into links in web view

• Support for HTML and attachments will be added soon

• Used to be a problem with threading posts from Microsoft Exchange• MS hotfix in January (applied at CERN, RAL,...)

• After only 29 years, Microsoft support SMTP RFC!

• Still relies on Simba spam filtering• Next version will have SpamAssassin built-in

13th September 2006


CMS Migration Experience

• ~ 30 CMS lists migrated since December 2005• All of the computing and software lists are in

HyperNews• Most of the big "analysis" and "analysis/reco" lists

migrated• Only some smattering of detector groups, so far.

• Following a similar “adiabatic” procedure as we plan• LHCb have 65 Simba lists in total, though I don't know

how many of those are active.

• Also created ~60 entirely new HyperNews fora

13th September 2006


Discussion fora (1)

• Everyone should subscribe to “HyperNews System Announcements”• New fora announced here, allowing people to subscribe

• Eventually could migrate nearly everything that currently has a mailing list• Eg. one forum for each detector and software subsystem

• In some cases, it is useful to have separate “development discussions” and “user questions” fora• Experts subscribe to, or watch, both• Users post to the latter, receive e-mail responses without

having to subscribe• Can also browse/search this forum to see if it has already been

answered• Soon users start to help other

13th September 2006


Discussion fora (2)

• “Task forces” and system/paper reviews have their own fora• Separates out short-term discussion, while maintaining a

record in the archive

• Can also be used as a shared record of important e-mails• Eg. communications with journals, approvals, minutes of

external meetings, etc.• Just forward them to the appropriate e-mail gateway

address

• I already created some generic fora• Linux, programming, and (La)TeX questions• HyperNews announcements, feedback, and test

• Use “HyperNews Test Forum” to play with the system

• CERN Computing Announcements• Automatic feed from [email protected]

• Chat and discussion that doesn’t fit elsewhere

13th September 2006


Other Use Cases• If a question or discussion has come up before, just

post a link to it• No need to copy text• The questioner can see the full discussion / thread

• Sometimes a discussion strays to another forum’s topic• Post replies in new forum with link to old + quick link

oldnew

• Short-term documentation, recipes, and kicks can be posted as a HyperNews announcement with a link from the documentation• When it becomes obsolete, remove the link

• a record remains of what people recommended

• Sort-term only: not a substitute for proper documentation!• Can easily migrate to Wiki

13th September 2006


Forum categories

• Currently have:• Computing• Computing Operations• General Discussions• HyperNews

• Could add now or later:• Journal Paper Reviews

• Perhaps add:• Online Computing• Offline Computing• Physics Analysis• Trigger• Detector Subsystems

• or one category per subsystem?

• National Discussions• Collaboration Matters• Detector Commissioning /

Operations

• Fora requested by each group, but categories defined in advance• can be changed of course, but not from the web form

Documents

HyperNews for LHCb