Planned FCC Power Outage

July 11, 2011

ARRA Construction @ Feynman

• ARRA construction project requires completion by September 30th.

• This constraint requires a outage window on Saturday August 13th to enable subsequent deliverables and activities.

• Date negotiated with experiment D0, CDF and CMS spokespersons to minimize impact.

ARRA Construction @ Feynman

• Cooling Upgrade project requires an outage– Electrical work associated with cooling system for

the FCC2 computer room– EPO upgrade to new system on FCC2 (including

commissioning and testing)– Opportunistic Work• Installation of power monitoring for the original FCC

480V service (DOE requirement)• Small electrical upgrades & work

Impact to Services

• Core Services will be unavailable with the exception of network for the duration.

• Shared Scientific Computing Services will be either unavailable or impacted for the duration.

• We are working in the interim to mitigate the impact on the network and scientific services.

Outage Schedule

• Saturday, August 13• Scheduled for 8 hours• Tentative start of work is 6:00AM

• All power and cooling in Feynman is impacted except for the FCC3 HACC computer room

Questions

Appendix

GCC Cooling Outage

GCC Cooling Outage

• June 7 loss of cooling in Computer Room B– Ambient air to condensers to hot > 120F– Heat build up on condenser pad

14:0514:10

14:1514:20

14:2514:30

14:3514:40

14:4514:50

14:5515:00

15:0515:10

15:1515:20

15:250

20

40

60

80

100

120

140

C-C6-M H-D6-T C-A6-M H-A6-M

Record Setting Temperatures @ Dupage Airport

Short Term Actions

• Understand indicators or predicators • Computing load shed plan• Cooling in the room reviewed– Increased cooling setpoint from 73F to 77F

• Review of existing monitoring data• Water cooling under condensers at the pad• Reduction of vegetation around the pad• Augmentation of monitoring at the pad

FESS Engineering Study

• Engineering Review of the cooling system• Determine Root Cause• Recommendations & proposals

• Deliverables from study in late summer• Implement any modifications – Prior to Summer, 2012