DOE/NSF Review May 8-10, 2001 1 US CMS US CMS M&O Planning Dan Green US CMS Project Manager DOE/NSF Lehman Review, May 8-10, 2001

DOE/NSF Review May 8-10, 2001

1

US CMS US CMS

US CMS M&O Planning

Dan Green

US CMS Project Manager

DOE/NSF Lehman Review, May 8-10, 2001


2

OutlineOutline

• CMS and CERN Planning

• Resource Loaded Schedules– Category A

– Category B

• US CMS - Planning for the Physics - the research program.


3

Planning for the PhysicsPlanning for the Physics• CMS has put “CPT” in place to prepare for the research

program. This is a coherent sum of Core SWC, Physics reconstruction and selection, and high level Triggers.

• CMS will present draft MOU for M&O and SWC at the next RRB meeting at CERN.

• Draft cost profiles for M&O, SWC have been shown to the JOG.

• Upgrade R&D and some fabrication is part of the research plan.

• US CMS has created a draft Management Plan for M&O.

• A resource loaded schedule (WBS) was made in preparation for the Lehman review. FNAL had a PMG review April 11-12, 2001.

• The Lehman review (May 8-10) has requested a complete plan for all costs of the US CMS Research Program - M&O + SWC + Upgrades.


4

US CMS Research Program - M&OUS CMS Research Program - M&O

• Letter of August 17, 2000 from the Joint Oversight Group ( JOG - DOE/NSF) to Fermilab to manage Maintenance and Operations (M&O)– Detector operations and data monitoring

– M&O of US CMS supplied subsystems

– Virtual control room at FNAL for US based physics analysis

– R&D for and fabrication of upgrades to the experiment

• First cost estimate shown to JOG on December 6, 2000.


5

CMS Steering CommitteeCMS Steering CommitteeChanges have been made in CMS to plan for SWC + Physics. CMS is planning for the research program.

M&O + SWC at the next RRB.

US physicists are an integral part of the CMS management.

US CMS


6

““CPT”CPT”

CSC (SWC MOU)

Pimia

Stickland

TRIDAS(Detector MOU)

Cittolin

Smith

PRS

Sphicas

Common Technical Board

CSC IB TRIDAS IB

Tier 1 Tier 1 L. Bauerdick - US CMS SWC


7

PRS and Detector L2PRS and Detector L2Tasks: detector simulation

detector reconstruction

detector calibration

monitoring

physics objects, reconstruction and selection

test beam analysis e - Seez J, - Eno

- Gasparini, Acosta

,b - Mannelli

ECAL - Faure

HCAL - Green

MU - Gasparini

Mitselmakher

Tracker - Rolandi


8

CMS L2 and PRSCMS L2 and PRS


9

Resource Loaded Schedule - M&OResource Loaded Schedule - M&O

• WBS 1-7 are subsystem costs for US built and/or managed systems (Category B)

• WBS 8 contains the Category A (common CMS costs), the Remote Control Room, and US CMS Outreach. Estimates for Cat. A are from CMS/CERN

• WBS 9 gives detail on the estimated costs for Project offices at FNAL and at CERN.

• WBS 10 is the management reserve.


10

WBS Dictionary - NotesWBS Dictionary - Notes

US CMS has begun to create a resource loaded schedule for M&O. The “notes” field is used for the dictionary and BOE.


11

Resource SheetResource Sheet

The base program resources must be called out explicitly, even if there is no explicit M&O cost. The US CMS research program cannot succeed without strong base support.


12

WBS 8.1 - M&O CERN and CMSWBS 8.1 - M&O CERN and CMS

DOE+NSF/CERN MOU is umbrella document - with “general conditions applicable to experiments performed at CERN”. Now work on “MOU for Collaboration in the Exploitation of the CMS Detector” and an IMOU on SWC.

Category A error is large +- 400 k$/yr. US is pro rata ~ 20%. NO costs to US CMS before FY03.

CERN RRB Proposal

0

500

1000

1500

2000

2500

2001 2002 2003 2004 2005 2006

FY

AY

(M$

)

Outreach

Consultancy andmanagementServices

Shops and Labs

Test Beams andCalibrationData Management

On-line computing

Communications

Publications

Detector Services


13

8.1 Category A Subtasks8.1 Category A Subtasks

WBS 8.1.1 is the largest item in WBS 8.1 Explicitly call out the subtasks of 8.1


14

US CMS - at PresentUS CMS - at Present387 Members from 38 Institutions387 Members from 38 Institutions

US CMS - at PresentUS CMS - at Present387 Members from 38 Institutions387 Members from 38 Institutions


15

How Will US CMS Grow?How Will US CMS Grow?

D0 History

0

100

200

300

400

500

600

700

1987

1989

1991

1993

1995

1997

1999

2001

CY

To

tal

Ph

ysic

ists

Grad Students

Physicists

D0 doubled in size around the time of first data taking. US CMS is imagined to do likewise. CDF and D0 are “natural” future additional collaborators. It is expected that US LHC physics will engage a substantial fraction of the US HEP experimental community by the time of first LHC data.


16

8.2 Remote Control Room8.2 Remote Control Room

• Expect (D0 experience) US CMS will grow dramatically in FY05, 06 by a factor ~ 2. Taking US CMS ~ CDF now, we estimate 200 students and 120 university postdocs “in residence” if we adopt this model.

• If all are at CERN, just the incremental costs to the “base program” are ~ 5.4 M$/year. This would also be a major change in the sociology of U.S. HEP.

• An alternative is to create a “critical mass” stationed in the US using the Tier1 UF (SWC) and Tier2 (NSF) as the primary computing resources for US physics analysis. The Remote Control Room (M&O) would be used to debug equipment, teleconference hold physics analysis meetings and take data shifts.

• We will attempt to create a coherent national US CMS physics effort with a real identity. Upper Atmospheric Research Collaboratory (UARC) is a data point (NSF).


17

US CMS - UF + RCRUS CMS - UF + RCR

Raw Data: 1000 Gbit/sRaw Data: 1000 Gbit/s

5 TeraIPS5 TeraIPSEvents:

10 Gbit/sEvents:

10 Gbit/s

10 TeraIPS10 TeraIPS

Controls: 1 Gbit/s

Controls: 1 Gbit/s

To regional centers 622 Mbit/s

To regional centers 622 Mbit/s

CMS networks, farms and data flows

Remote control rooms

Remote control rooms

Controls: 1 Gbit/sControls: 1 Gbit/s

US CMS


18

US CMS Remote ControlUS CMS Remote Control

Reality check: US CMS is using the CDF/KEK remote control room for Run II as a starting point. However, we want to expand the scope to encompass a US based physics group and US LHC accelerator tasks.


19

8.3 US CMS - Outreach8.3 US CMS - OutreachThe US CMS Construction PO now supports proposals for educational initiatives. The US CMS Education Coordinator is responsible for prioritizing the proposals.

The US CMS Project Office for the detector project then funds the agreed upon proposals. This effort would continue smoothly into the research phase of US CMS.

QuarkNet is a powerful example of the educational possibilities inherent in the LHC. We plan to continue to exploit the outreach opportunities of US CMS e.g. CMS web cam in SX5


20

WBS 1-6, US CMS Category BWBS 1-6, US CMS Category B

• The US CMS L2 Managers were asked to estimate their M&O tasks for;

– M&S costs to maintain and operate systems provided by US CMS (Category B).

– Resources needed to maintain technical personnel at CERN.

– Additional costs associated with the major annual shutdown.

– NOTE: Post-doc and graduate student salaries and incremental costs (e.g. per diem) are assumed to be supported by the university base programs. Use annual SOW to define that commitment?


21

WBS 1-6, US CMS - ScalingWBS 1-6, US CMS - Scaling

Reality Check:• For CDF Operations there are 2 FTE engineers

and 14 FTE technicians.• We plan for a similar economy of scale by

putting a “US CMS pool” together for general engineering and technician tasks.

• This team would have 4 FTE engineers and 20 FTE technicians.

• This team in residence costs ~ 1.6 M$/yr.


22

WBS 7 - US CMS POWBS 7 - US CMS PO

Reality Check:• US CMS has experience from the construction project

and we plan a smooth “handoff”.

• There will need to be both a CERN and a FNAL/NEU “branch” of the PO.

• There are ~ 6-8 FTE in the PPD/CDF Ops Dept. We expect ~ 4 FTE at CERN, ~ 4 FTE at Fermilab.

• The M&O PO costs ~ 1 M$/year and scales well with the current PO.


23

US CMS - L2 AccountsUS CMS - L2 Accounts

• US CMS - CERN Branch Project Office (opened on May 1, 2000)

• US CMS L2 CERN Team Accounts

– Funds sent to FNAL

– Allocations by L2 managers

– Transfer to Team Accounts at CERN

– Authorization of expenses by L2 managers

– Invoices sent to FNAL for financial tracking

– PO, EMU, HCAL, and SiTrkr have started L2 accounts

• Serves as a model for M&O (Cat A+B) funding and tracking. FNAL has accepted a “pass through” overhead rate, treating CERN as a collaborating institution. This decision allowed US CMS to plan for the installation and commissioning phase of the detector.


24

US CMS - FY01 RequestsUS CMS - FY01 RequestsUS CMS - FY01 University Request

Texas Tech

UC Davis

Illinois Chicago

Rice

UC San Diego (DOE)

UC Davis

Florida

Caltech

The P.O. already has input to DOE/NSF in regards to a project perspective on base support increments for postdocs. This recommendation refers ONLY to project related activities and not research activities covered by the peer review process. Adequate base support wil, however, be critical to the success of the research program.


25

Management ReserveManagement Reserve• Much of the research program costs will be salaries.

• A full contingency analysis at the lowest task level seems inappropriate.

• A management reserve of 25 % is proposed to be held in the Project Office.

• This level is similar to the R&D reserve held at the start of the detector project. This mechanism worked well in that instance.

• Problems will arise that must be solved, e.g. currency fluctuations. Other possibilities would be an accident at the detector or more frequent than expected repairs in this hostile accelerator environment. We must plan to cover these contingencies.


26

WBS - Cost and ManpowerWBS - Cost and Manpower

Reality Check: The T+E level is close to the scaling estimate. These cost estimates do not contain salaries of research personnel - postdocs and graduate students. However, the tasks are called out and there are ~ 48 “FTE” PD working on M&O. Note also Cat B does not start until ~ FY05

US CMS M&O Cost Estimate

$0

$1,000,000

$2,000,000

$3,000,000

$4,000,000

$5,000,000

$6,000,000

$7,000,000

$8,000,000

$9,000,000

$10,000,000

FY03 FY04 FY05 FY06 FY07 FY08

AY

$

Mgt Reserve

OpsOffice

Common

SiTrk

Fpix

ECAL

DAQ

Trigger

HCAL

EMU

US CMS M&O Resource Usage

0

10

20

30

40

50

60

70

80

FY

03

FY

04

FY

05

FY

06

FY

07

FY

08

Fiscal Year

FT

Es

Post-docs

Engineers

Technicians


27

US CMS Research Program - SWCUS CMS Research Program - SWC

11/98: DOE and NSF Joint Review of ATLAS and CMS Computing/Software

5/99: DOE and NSF Review of US ATLAS and US CMS Computing and Software

1/00: DOE/NSF Review of US LHC Software and Computing Projects

10/00: Hoffmann Review Report (CERN SWC Review)

11/00: US LHC Software and Computing ReviewThere have been several US reviews of SoftWare and Computing needs over more than 2 years. A large investment in SWC is needed to realize the investment in the construction of the CMS detector. Note SUSY can come in the first month at 10% luminosity.


28

US CMS Research ProgramUS CMS Research Program

• Upgrades are estimated - both R&D and fabrication. They should be proposal driven and reviewed in the context of the full US HEP program. Note that SWC also has upgrades as a part of the plan (Moore’s Law).

• Contingency is explicitly held in an annual management reserve. The proposals for allocation of reserve should be reviewed as they are in the construction project.

• Postdocs and students are estimated from CDF and D0 scaling. They are identified as necessary to the success of US CMS research, but their support is assumed to reside in the “base program”.


29

The Cost of US CMS ResearchThe Cost of US CMS Research

The components of the research program are SWC, M&O, Upgrades and scaled estimates for the dislocation costs associated with postdocs and students - base program cost.

US CMS - Construction and Research

0

5

10

15

20

25

30

35

40

19

96

19

98

20

00

20

02

20

04

20

06

20

08

FY

AY

(M$

)

ResearchPersonnel

Upgrades

M&O CostEstimate

SWC CostEstimate

ConstructionProject Funding


30

US CMS - SummaryUS CMS - Summary

• The physics available at the LHC is compelling.• The transition from US CMS construction to

research is being planned.• There are now resource loaded schedules in

place for all the components the US CMS research program.

• Our aim is to enable the US physicists to be a coherent group making a major impact on physics analysis within CMS.


31

Detector Upgrades - IDetector Upgrades - I

• US CMS Scope restoration (8.6 M$)– ME4/2 mechanics– ME4/2 electronics– ME4/1 electronics– Installation and Commissioning the above– Fpix third layer


32

Detector Upgrades - IIDetector Upgrades - II

• CMS Scope Restoration - US Management (4.0 M$) of EMU, HCAL– ME1/1 electronics and cables– HF shielding– HF tooling and supports

• Repairs (5.5 M$)– HE |y|=3 scintillator repair– Fpix replacement for radiation damage


33

Detector Upgrades - IIIDetector Upgrades - III

• R&D EDIA - ( 0.2 M$/year)

• “TRIDAS” Upgrades ( ~1.0 M$/year)– increase in C.M. energy ~ transparent– increase in luminosity --> “smarter” L1

trigger and faster L2, L3 event building– DAQ improvements with commodity

purchases– r.f. changes in bunch crossing time ?


34

JOG and FermilabJOG and Fermilab

• Letter of August 17, 2000 from JOG to Fermilab to manage M&O– Detector operations and data monitoring (Category A, WBS 8)

– M&O of US CMS supplied subsystems (Category B)

– Virtual control room at FNAL for US based physics analysis (WBS 8)

– R&D for and fabrication of upgrades to the experiment (WBS 8)

• First cost estimate shown to JOG on December 6, 2000.


35

M&O PlanningM&O Planning

• CMS has put “CPT” in place to prepare for Physics.

• Draft cost profiles for M&O, SWC have been shown to the JOG, most recently Dec. 6, 2000.

• Upgrade R&D and some fabrication is part of the plan.

• We DO NOT assume that the Project can use M&O funding to complete installation.

• Planning and resource loaded schedules are being created for SWC and M&O which will both be needed to realize the investment in the CMS detector. These efforts need to be globally optimized.

• US CMS has made a draft OMP for M&O.

• The Lehman review has requested a complete plan for all costs of the US CMS Research Program - M&O + SWC.


36

D0 - Demographic GrowthD0 - Demographic Growth

CDF has ~ 406 physicists with 154 students. D0 has ~ 493 physicists with 108 students.

The are now ~ 160 students and university postdocs in residence at D0. There will be an increase when Run IIa is in full operation.

D0 doubled in size just before data taking. US CMS is imagined to do likewise.

D0 History

0

100

200

300

400

500

600

700

1987

1989

1991

1993

1995

1997

1999

2001

CY

To

tal

Ph

ysic

ists

Grad Students

Physicists

Documents

DOE/NSF Review May 8-10, 2001 1 US CMS US CMS M&O Planning Dan Green US CMS Project Manager DOE/NSF Lehman Review, May 8-10, 2001