L2CAL Pulsar Upgrade Proposal in 24

minutes An Executive Summary

Ted Liu For the L2CAL upgrade team

Pisa/Purdue/Rockefeller/UC/Upenn

/Madrid/Padova/Academica Sinica/

FNAL

Trigger Upgrade Review, July 27 2006

•This is the talk shown at the review on July 27th, 2006.• With some new slides on schedule and people power at the end (backup slides). Updated Aug.3rd, 06.

A brief history of time …

It has been a long way towards this proposal … just like the Pulsar project history Early 2001, 1st time Ted met Pacman while working with Monica on L2CAL

commissioning… impressed with the simplicity of pacman’s algorithm, worried about jet merging at higher luminosity …

Year 2002, Pulsar was designed for general purposes (some CDF people hated it, yet many liked it):CDF/BABAR/CMS/ATLAS/TOTEM/ALICE/…compatible with L2CAL too

Sept. 23rd, 2004: Henry’s email to Ted, asking if we can improve L2 clustering for taus and jets… I was really busy with L2 Pulsar upgrade back then…. Only answered Henry’s email on March 23, 2005. The answer was: YES, but needs physics motivation….

Nov. 22nd, 2004 and Feb. 22, 2005: Viktor Veszpremi showed his study on L1MET and L2MET at exotics meeting, with higher resolution….

April 2005, Ted asked a few others to look for jet merging… lots of hall-way talking… Early 2005: Ming&Mario mentioned the L2CAL upgrade idea at TDWG, a few people were

very much pissed at them…:( June 18th, 2005: Mary showed a few event display with jet merging in ROF. Vadim and Ted confirmed Mary’s finding, and showed the “phase transition”… However, it was really Paola Giannetti’s full support that convinced Ted to go ahead and

push this proposal forward … Paola is willing to put her full team behind this upgrade! Anwar/Mary/Gene developed L2Cone algorithm, Gene did extensive studies on it, and

Laura significantly optimized the L2Cone algorithm speed in the spare L2 decision CPU…

CDF Trigger Performancein 24 seconds…

L1A bandwidth: ~ 30KHz, no longer a bottleneck for RunIIb L2 Decision crate and SVT upgrade XFT upgrade will also improve the L1 purity

L2A bandwidth: ~1 KHz, not enough for RunIIb Tons of work went into EVB/L3 upgrade, DAQ improvements … Significant improvements from ~ 300 Hz to 1 KHz Still not enough for RunIIb, L2A already at ~800Hz at ~180E30 The problem: L2A cross section grows rapidly with luminosity Why? Mostly JUNK L2 does not have enough information to control the background

Most recent run (July 23, 2006)

QuickTime™ and aTIFF (Uncompressed) decompressor

are needed to see this picture.

Slope: > ~100Hz/10E30

QuickTime™ and aTIFF (Uncompressed) decompressor

are needed to see this picture.

QuickTime™ and aTIFF (Uncompressed) decompressor

are needed to see this picture.

Soft Readoutdeadtime wall

HardBusydeadtimewall

The two great Wallscompeting for attention…

L2 triggers with high growth term:two types of L2 triggers gone wild…

. Track/mu related:

e.g. CMX etc

L2CAL related:

e.g. MET+2JETs

Why? Probably Bad names…

JET: JUNK Enhanced TriggerMET: MESS Enhanced Trigger

CMX: Could Match beamX

Examples of L2 triggers gone wild

QuickTime™ and aTIFF (Uncompressed) decompressor

are needed to see this picture.

CJET10_JET10_L1_MET25(MET+2JET HIGGs trigger)

Other examples of JET/MET related triggers will go wild in rate at higher luminosity: MET35, Higgs high-Pt b-jet, Higgs multi-jets …JET40/60/90…

also go wild in inefficiency:Triggers require multi-jets at L2, e.g.

Top_multi_jets (L2_FOUR_JET15_SUMET175)could go TOPLESS at higher luminosity due to jets merging…

QuickTime™ and aTIFF (Uncompressed) decompressor

are needed to see this picture.

L2_CMX6_PT15:

XFT upgrade improves this

Bottom line:

Higgs/Top/QCD physics is at stake. Once bandwidth saturated, ALL physics triggers at stake…

600 nb @ 170E30: ~100 Hz

CDF Trigger Performancein 24 seconds…

L1A bandwidth: ~ 30KHz, no longer a bottleneck for RunIIb L2 Decision crate and SVT upgrade XFT upgrade will also improve the L1 purity

L2A bandwidth: ~1 KHz, not enough for RunIIb Tons of work went into EVB/L3 upgrade, DAQ improvements … Significant improvements from ~ 300 Hz to 1 KHz Still not enough for RunIIb, L2A already at ~800Hz at ~180E30 The problem: L2A cross section grows rapidly with luminosity Why? Mostly JUNK L2 does not have enough information to control the background

The most effective way to solve the L2A bandwidth problem is to:

improve the trigger purity at L2 …

How to improve L2 purity?send more information into L2 CPU

We are already doing it ! Send XFT stereo information from L1

Segment Finders to L2 CPU via Pulsars, to improve CMX etc trigger purity …

The information is made available by designing the new stereo Finder boards …

Today, we are proposing the same thing for JET/MET related triggers (L2CAL path) Send all 10-bit trigger tower information from

L1 DIRACs to L2 CPU via Pulsars, to improve JET/MET trigger purity… The information has already been there in the

system, and we have not made use of them! This is a natural thing to do… Will take some real work for sure …

Wait a second! Do we really have to do this? (I)

Are you sure the machine luminosity will ever go beyond 200E30?

We have every reason to believe that it will (past year experience!) True, every single machine I know is a moving target We are not 100% for sure it will ever reach 300E30 We cannot even say for sure it will not go beyond 300E30 (CESR&PEP-II&KEKB all went way beyond design luminosity at the

end)

It does not matter:

One may hope for the worst (to minimize the work), we will have toprepare for the best. Otherwise, why waste our life in the CDF trailer?

Wait a second! Do we really have to do this?

(II)

OK, let’s assume the luminosity will reach 300E30, but there must be other easy&simple&cheap&lazy way to deal with this! Sure. The simplest way is to DPS all high rate L2 triggers, and call it the final

RunIIb trigger table It will only take a few hours work for TDWG. We have been practicing this

technique a lot in the past few years, mostly to charm and beauty triggers We have been pretty good at this already. Quite a few Higgs high-Pt or multi-

jets triggers are already DPSed in the default trigger table This would minimize the work load for ALL of us …:)

Is this really what you want?

The simplest idea of all: beg the machine folks to keep the luminosity low... It may reduce their work load as well…Want more discussions on luminosity leveling?

Wait a second! Do we really have to do this?

(III)

OK, But what if one could improve the L2A bandwidth to, say, ~5 KHz? Do we still need to do the L2CAL upgrade? Or, what if one could improve the L1MET at L1 with higher resolution, to reduce the MET+2JET rate?

If we had much more L2A bandwidth, much less work online for sure! Jet90, MET35, Jet40/60 etc could be saved this way … However, still much more work offline…. Top_multi_jets efficiency is inst. luminosity dependent due to jet merging MET+2JETs efficiency is inst. luminosity dependent due to jet merging The same is true for any trigger requiring multi-jets… Offline analysis on systematic could be a lot of fun, making many heroes… It doesn’t really minimize the work for ALL of us

Wait a second! Do we really have to do this?

(cont.)

OK, I got it. This L2CAL upgrade thing is all about survival then: to improve the purity of Jet/Met related triggers so that we have extra bandwidth for other triggers… No, it is not just about survival, it is really about how to live gracefully… L2Cone algorithm is much less sensitive to luminosity… the trigger efficiency is more robust against inst. Luminosity… much easier life offline… Finer resolution MET sharpens the turn-on curve: possibility to lower the threshold. L2CAL upgrade allows more sophisticated algorithms to be implemented in software… JET/MET can be made nearly equivalent to offline quality Many new opportunities for additional improvements in trigger purity and performance, most notably for Higgs and exotics triggers, with such information available at L2 as dijet mass, delta-Phi between jets or between a jet and MET, sum Et of the cluster, better jet-SVT matching for b-jets, improvements in tau triggers …. With the L2CAL Pulsar upgrade, we could really push the Higgs/exotics search sensitivity beyond

the baseline … to make sure we have the best shot for discovery Keep in mind: we are not alone in the Tevatron Universe … D0 has just upgraded its L1CAL with more sophisticated capability at L1…

“We need to have trigger datasets with broad bandwidth for discovery” --- Jaco Konigsberg, the CDF spoke, has spoken

Wait another second, Do we have enough man/woman

power to do this?

For long, I have heard horrible stories that nowadays it is very hard to find people to even take care of the CDF operation …

This is so not true with the L2CAL Pulsar upgrade There are people who HATE the L2CAL upgrade, for whatever reasons Still, there are MANY brave people who are highly motivated to work on

this upgrade It has been a “phase transition” in the past few months around the trigger

workshop on May 12th… all started from hall-way talking … no single bottle of beer was even involved

Pisa, Rockefeller, Purdue, UC, Acadmia Sinica, Madrid, Padova, Frascati …

I am TOTALLY impressed that there is still SO MUCH energy within CDF. All it takes is to inspire them…

The man/woman power:only people with > 20% time shown

Engineers: Marco Piendibene (50%): Pisa (Pulsar firmware expert) Lucas Rogondino (100%): Pisa engineer student Mircea Bogdan or Fukun tang, Harold Sanders (part time): UC engineers,

Pulsar hardware experts Richard Northrop (part time): UC mechanical engineer for cabling Davis Pantano (30%): Padoca technicain

Physicists: postdocs Laura Satori (100%): pisa Gene Flanagan (100%): Purdue Giorgio Cortiana (100%): Padova, available soon New Padova postdoc (100%): available end of year New U. Chicago postdoc from Henry (100%): end of year

Physicists: Ph.D students Miguel Vidal (100%): Madrid, available Sept. 2006 Michael Schmidt (30%): U. Chicago (Y.K. Kim), available Sept. 2006

So how much it would cost?

No Pain, No Gain…. As we all learned. We are not cheap… In order to interface with L1 DIRAC, we need to design a new mezzanine card, and we need 72 of them (I.e. build ~100 of them). The total cost is about $50K. We already have 92 Pulsar boards for CDF: 36 for SVT+9 for L2D + 6 for XFT

= 51. Still 41 left. However, L2CAL upgrade takes 24 Pulsars… so only 17 Pulsars left.

This is clearly not enough, given all the test stand needs for the official projects at different places, and the unofficial XFT project needs, hot spares for operation … etc

Also depends on how many other upgrade proposals will be approved… The last thing we want to see is to run out of Pulsar boards. I would strongly suggest that we order ~25 Pulsar boards if the L2CAL upgrade

is approved… This would cost 25 x $4K ~ $100 K, bring the total to $150K Plus ~30% contingency: $50K + $150K ~ $200K. This is the price tag for doing

the dirty work for CDF…

So how long it would take?

Strongly depends on the people involved

It could take > 1 year to just change a simple firmware… We have the best people on the case… with excellent track-record With good people involved, strong leadership is the key…. Laura will be in charge of the technical aspects Gene will be in charge of the physics trigger aspects Paola/Henry/Mary/Matthew/Ted/etc will help We expect ~ 6 months to finish the hardware installation. It typically takes

another few months to fully make use of the new L2CAL trigger capabilities in the official trigger table, based on past experiences.

However, we will seek ways to shorten this time, by parallel processing…

Tasks involvedLVDS signal splitting

Mezzanine card design

Firmware

Readout software

Online monitoring software

Clustering algorithm code optimization

Physics trigger studies

Final Commissioning

Half year, assume all the man/woman power in place

We are familiar with all the tasks…

The Lockyer’s committee-- it is now in your hands

Backup slides

A few on hardware configuration

A few on schedule and people power:

As we speak, there are still more people joining the effort. The schedule shown here does not include all of them. Some of them will join the effort in the new trigger studies and developments, some will help out

the hardware/software.

Mezzanine slotsAUX card

Pulsar Design modular/universal/self-testable

Pulsar

Custommezzanine

Bottom view

Since most of the upgrade proposals are using Pulsar boards,This slide is a backup slide.

S-LINKPCIGbE

worksup to

100MHz

Top view

What’s involved?-- only CAL related shown: concept

L1CAL

L2CAL

Calorimeter10 bits tower energy

10 bits tower energy

288 LVDS cables

Only 8 bits tower energy used by L1CAL

L2CPU

L2 Pulsar crate

L2CAL Pulsar crate

L2 CPU for commission

(1) A copy of input signal(2) New mezzanine: 4 cable/card(3) 18 Pulsars/AUX with new input firmware(4) 6 Pulsar/AUX SLINK mergers(5) Some simple online code(6) New clustering algorithm code

Pulsar Cluster (1 Pulsar: 4 mezzanine x 4 cable = 16) x 18 = 288 input cables total

Pulsar x9

Pulsar x9

Pulsar Crate 1

Pulsar Crate 2

144 cablesfrom DIRAC

one 40-bit word/cable

144 cablesfrom DIRAC

9 slink outputs

9 slink outputs

Pulsar Slink merger x6 PC

The average data transfer latency after L1A~ expected within in ~10 usNote: unlike other L2 paths, CAL data already available at L2 input upon L1A

Raw data size w/o suppression: 288x40/8 ~1.5KB per evt. With some overhead, < ~ 600 slink words maximum

w/ suppression, data size should be much less.

Mezzanine card design concept

FPGA

LVDS/TTL chips

Cable connector

Pulsarside

Rack door

Cost estimate:

72 needed ~ 100

< $300 per card

(dominated by FPGA)

< $30K for mezzanine

~2 weeks engineer time

25 AUX cards

+ long cables

Total cost of the project:

< ~ $50K

40

EM HAD EM HAD

10bits x4 =40 @ cdfclkinput data per cable

JTAG

Pulsar

front panel

How to copy the input signal?-- for parasitic running,

crucial for commissioning

In principle, one could design a special splitter board. Or can use LVDS multi-drop property to get a copy:

DIRACdriver

DCAS

receiver

Pulsar mezzanine Receiver (no termination)

100 ohm

Need to make longer LVDS cablesActual cabling needs to be clean: with help from JDL

DS90C031

New Cabling at trigger room

L2 CAL crates

L1CAL crates

Under the floor

Not terminatedon pulsar mezzanine

Possible new pulsar crate locations

L2 Decision

crate

Commissioning can be done in pure parasitic mode, using the spare decision CPU,along with a copy of all other L2 data paths information

Above the racks

The man/woman power:only people with > 30% time shown

updated on Aug. 3rd, 2006

Engineers: Marco Piendibene (50%): Pisa (Pulsar firmware expert). At FNAL now. Lucas Rogondino (100%): Pisa engineer student. Arrived FNAL Aug. 2nd. Mircea Bogdan or Fukun tang, Harold Sanders (part time): UC engineers, Pulsar

hardware experts. New mezzanine card design. Richard Northrop (part time): UC mechanical engineer for cabling Davis Pantano (30%): Padova technician

Physicists: postdocs Laura Satori (100%): Pisa Gene Flanagan (100%): Purdue Anadi Canepa (100% after Sept 1st): Upenn (N. Lockyer) Giorgio Cortiana (100%): Padova, arriving FNAL Aug. 2006 Massimo Casarsa (50%): Pisa, arrive fall. New Padova postdoc (100%): available end of year New U. Chicago postdoc from Henry (50%): end of year

Physicists: Ph.D students Miguel Vidal (100%): Madrid, available Sept. 2006 Michael Schmidt (30%): U. Chicago (Y.K. Kim), available Sept. 2006 Yanjun Tu (50%): Upenn (N. Lockyer), available Sept. 1st 2006 (after ACE shifts)

L2CAL upgrade team

Overall Project Leader: TBA Technical coordinator: Laura Physics Trigger coordinator: Gene

The rest of the slides are Ted’s initial thoughts on the schedule and people power… poor Ted doesn’t know much at all about project managing, let alone fancy Microsoft project managing tools… so this is in power point format. Project managing experts, please do laugh at me, but not too much…:)

Assume this project will be approved officially in Aug. 06

Basic Tasks involvedLVDS signal splitting

Mezzanine card design

Firmware

Readout software

Online monitoring software

Clustering algorithm code optimization

Physics triggers

Final Commissioning

Half year, with all the man/woman power in place

We are familiar with all the tasks… 20% effort on new stuff, 80% effort on testing…

Summary: L2CAL work plan draft

(from now to system commissioning)

Cabling: design

Northrop/JDL/TL/Dervin

Cable testing: students*/Northrop

Cable/crates installation: Dervin/BB/JDL/TL/Vadim

Aug. Sept. Oct. Nov. 06 - Feb 07

Mezzanine design: Mircea + Laura/Marco/TL

Firmware design/testing: Marco/Davis/Laura + help fromTL

L2CAL Pulsar Testing software: Lucas + Pulsar group/Vadim

Mezzanine/Pulsar testing: students* + Marco + Lucas

Readout software: Laura W/ help from DAQ

Online monitoring: Lucas/Giorgio+students*/Vadim/Wojtek

Algorithm code with isolation and two electron passes: Anwar/Gene/Mary Final overall code optimization: Laura + Yanju + Vadim/TL

Continue work to replace all existing triggers: Gene

New triggers for new L2CAL: Anadi/Giorgio/Massimo/etc/TDWG (could use more help here!)

Systemcommissioningwith parasitic running first (Everyone)

Key person in red. Students* = Yanjun/Miguel/Michael. Some details follow …

LVDS signal splitting

Cabling design: Rich Northrop with help from JDL/Dervin/TL

Aug. 2006 (initial work started in July)

Cable/connectors assembly and testing: Rich Northrop + students*

Sept 2006

Cable/crates installation: Dervin’s team/BB/JDL/Vadim/TL

Oct. 2006

*Students: Yanju/Miguel/Michael

Mezzanine card/firmware The mezzanine card is a simple design… with LVDS signals into a small FPGA,

running at 132 ns clock…

Mezzanine design: Mircea Bogdan + Laura/TLAug.-Sept 2006

Mezzanine card / Pulsar DataIO firmware (simple): Marco with help from Laura/TL

Aug. - Sept. 2006

Mezzanine card testing software: Lucas with help from Pulsar group Aug - Sept. 2006

Mezzanine card production testing: students* + Marco + Lucas Oct. 2006

*Students: Yanju/Miguel/Michael. ~25 AUX cards made during this time

Readout software

Mostly Identical to the current Pulsar VME software. Minor modifications to format the new bank exactly the same as the current DCAS bank.

Laura with help from DAQ group and TL

Aug.-Sept 2006

Commissioning the readout software: Lucas + Laura Oct. 2006

Online monitoring software &

system installation

Mostly Identical to the current PulsarMon software. Cut&paste with minor modifications to compare the new bank to the current DCAS bank bit by bit.

also new TL2D bank vs old TL2D bank…cut&paste with minor mods

Lucas/Giorgio + students* with help from existing Pulsar group (Vadim/Wojtek)

Oct. 2006

Commissioning the online monitoring software: Lucas/Giorgio + Laura + students* Nov. 2006

System installation*: * New Pulsar production on going between Sept. - Nov. No need to wait for them for system

installation… we can use the existing spares.* Testing of the new Pulsars can be done at PREP, like before. They will be the new spares.

Clustering algorithm code

The core code (clustering/Met) already in good shape (optimized for timing)…

Add 2 electron passes and isolation: Anwar/Gene/Mary/TL Aug.-Sept. 2006

Pushing algorithm code optimization for speed: Laura

Oct. - Nov. 2006

Continue work to replace the existing triggers with the new triggers: Gene

Oct. - Dec. 2006

Developing new triggers for the new L2CAL system: Anadi/Giorgio/+ others from physics groupsSept. - Jan. 2006 (to get the best physics out of the new system. Help from TDWG)

System commissioning

People: ALL

Commissioning period: with help from TDWG and existing Pulsar group Nov. 2006 - Feb. 2007

Hardware/software commissioning: Laura/Lucas/Marco/Gene/Yanju/Miguel Offline trigger validation: Gene/Anadi/Giorgio/Massimo/Yanju/Miguel/Michael

System maintenance long term:

Students: Yanju/Miguel/Michael (first - second year grads. More coming)Postdocs: Laura/Gene/Anadi/Giorgio/Massimo…

Some new people(/blood) will be available ~end of 2006 (during final commissioning)