Eddy Jans 0 Vertex2013 Operational aspects of the VELO cooling system of LHCb Eddy Jans (Nikhef) on behalf of the LHCb VELO group Introduction Main components

Eddy Jans 1Vertex2013

Operational aspects of the

VELO cooling system of LHCbEddy Jans (Nikhef)

on behalf of the LHCb VELO group

• Introduction• Main components and operation principle of the system• Issues: how to prevent and to tackle them• Keep the detectors cold 24/7• Summary & outlook

19 September 2013

Eddy Jans 2

Introduction• VELO-module is double-sided (300 mm, oxygenated, n+-on-n) and operated in vacuum,• Strip closest to the beams is at 8.2 mm,• Per double-sided module: 2x16 frontend chips that together dissipate ~20 W,• 4 NTCs give temperature readings• Two movable detector halves with 21 VELO + 2 PileUp modules each (400 W/side)

Vertex2013

NTCs

19 September 2013

Vacuum tank

One detector half

Detectors are operated in a secondary vacuum absolutely no leaks allowed.Orbital welding or vacuum brazing of the pipes. All tested at 170 bars.

cooling block frontend chips

module base iskept at +20 oC

CO2 connections

Eddy Jans 3Vertex201319 September 2013

0.3 mm thick RF-box

• in 1999 it was proposed to use CO2 as refrigerant for the vertex detector of LHCb [LHCb note 99-046], • CO2 is radiation hard,• CO2 has a high latent heat value can use small diameter capillaries small amount of dead material in the acceptance,• stainless steel capillaries: Finner=1 mm, wall thickness 0.25 mm• system uses bi-phase CO2 via the accumulator controlled method.

Some cooling considerations

cooling blocks


Pressure x Enthalpy diagram of CO2

At -30 oC: 300 J/g

mass flow: 10 g/sper module: 0.43 g/sfull evaporation130 W

liquid CO2-speed: 28 cm/s all-gas speed: 240 cm/s

bi-phasearea

pres

sure

[M

Pa]

enthalpy [kJ/kg]

bi-phasearea

-30 oC

gasphase

liquidphase

pres

sure

[ba

r]

criticalpoint

vapour quality 0 1

isothermal cooling


liquid vapor

bi-phase

EnthalpyPr

essu

re

1

2 3

4 56P7

The accumulator controlled cooling cycle

12: pump increases the pressure of the sub-cooled liquid23: heat exchange in the transfer line brings evaporator pre-expansion per definition right above saturation point, since (E2-E3) = -(E5-E6)34: pressure drop in restriction and expansion in capillary brings CO2 in cooling blocks in bi-phase state,45: isothermal cooling via evaporation56: warming up of incoming sub-cooled liquid61: condensation and cooling of the CO2

R507

a ch

iller

s

Cond

ense

rpump

evaporatorrestricti

on

accumulator

Hea

t in

Hea

t in

Hea

t ou

t

Hea

t ou

t

1

2 34

56

P7

55 m transfer line

insulationf=80 mm sub-cooled liquid inbi-phase return

only passivecomponentsin the radiationzone


Main components of the system

Air-cooled SB chiller

Water-cooled SA chiller

Velovacuum tank with silicon modules, module base and CO2

evaporator.

TLR – CO2 system TR – CO2 systemTL – CO2 system

TL – CO2 bridge

TR tr

ansfe

r lin

e

TL tr

ansfe

r lin

e

TR – CO2 bridge

Liquid Pump

Electronic 3-way valve

Electronic 2-way valve

Manual restriction valve

No return valve

Electric heater

Damper with heater

Heat exchanger

PLC and electronics rack

Cooling plant at UXA03

VELO

• evaporative CO2 cooling system• “independent” system for either side • PLC-controlled• 2.5 kW water-cooled chiller at –40 oC• 1 kW air-cooled backup chiller at -25 oC• 55 m CO2 transfer lines• 10 heat exchangers• 8 actuators• 9 heaters• 31 pressure sensors• 192 temperature sensors• 350 parameters monitored in PVSS• only passive components at VELO• 2*400 W heat load of detectors• 2*12 kg CO2

Performance @ detector• main chiller: -28 oC operational CO2 temp. LV on: sensors @ -7 oC• backup chiller: LV off: @ -8 oC• stability < 0.1 oC


Design considerations and operational experience

• redundancy of crucial components• insulation• clogging filters• superheated CO2• dependence on electrical power• dependence on chilled water• safety measures to prevent overheated detectors• keeping the system 24/7 cold


Redundancy in the designTo minimise down time the system has a few redundant crucial components:• 3 CO2 pumps, where 2 are needed,• 2 chillers, water-cooled and an air-cooled as backup,• for controls crucial temperature and pressure sensors are two-fold implemented,• possibility to interconnect the two sides by hand,• PLC is on a 1500 VA/1000 W UPS,• PLC, backup chiller and CO2 pumps are connected to a diesel generator.


InsulationLiquid pumped system cold transfer lines good thermal insulation required. This seems trivial, but turned out not to be so in practice.Originally CERN safety regulations forced us touse Armaflex NH.Glued surfaces started to delaminate after 2 years.

Renewed insulation of the transfer lines and most of the cooling plant during Winter shutdown ‘10-’11.Now foamglass covered by an Aluminium protection shield and Armaflex AF, respectively.


Filters

piece of Armaflexthermalinsulationonce completelyblockeda restrictionvalve

Throughout the system eleven 15 mm filters are installed. (5 (CO2-plant), 2(@VELO), 2(manifold), 1(main chiller), 1(backup chiller)). In one detector half we have experienced a few times clogging filters.

Replacement procedure is tricky and risk of additional dirt in the system due to difficult accessible filter houses.

2 months

pressure [bar]

15

16

17


Post mortem analysis of the filters

0 5 10Energy (keV)

0

5

10

15

20

25

cps

O

Cr

Fe

Si CrCr

Fe

Fe

Ni

Many >15 µm orange objects have been observed inside the filter. They mainly contain Fe and O.Before, particles containing Cl had been observed. Possibly due to connections soldered with flux for a testbeam experiment risk in terms of corrosion. Work extremely clean from construction to installation.

Scanning Electron Microscope imageEnergy Dispersive Spectrometer analysis of an orange particlefound inside the filter

75 mm


Superheated CO2 after startup After startup we occasionally observe in a varying number (a fewall)of cooling blocks the phenomenon of superheated CO2.

Issue: cooling performance is very bad because liquid cooling has much less cooling power than evaporative cooling.


Superheated CO2 after warmup and cooldown

DT=3 oC-30

-22

-14

time

Tsilicon[oC] LV off

warmupof thecoolingplant

cooldown

30 minutes


Superheated CO2 after warmup and cooldown

DT=3 oC-30

-22

-14

time

Tsilicon[oC]

startaddingheat

30 minutesRemedy: add heat by means of a dedicated heater to bring the incoming CO2 in the liquid+gas state.

LV offliquid vapor

bi-phase

1

2 3

4 56

Enthalpy

Eddy Jans 15Vertex201319 September 2013Pr

essu

re

Not all cooling blocks behave the same way:- not all show superheating- when adding heat they don’t start boiling at the same moment

Some more superheated CO2

startaddingheat

silicon temperatures of 4 modules

DT4 oC


Power cutsPLC and backup chiller are connected to the power of a diesel generator of LHCb and the PLC also to its own UPS (1500 VA/1000 W)

When the power gets cut the switch-over from main to backup chiller is handled automatically by the PLC.

After switching back to the mainchiller the system is stable after ~20 minutes.After switching on the LV the sensors are at their operational temperatures after 10 minutes. half an hour recovery time.

-10

-20

-30

sensor temperature

10 minutes

LV onEddy Jans 17Vertex201319 September 2013

Failure of chilled water supplyChilled water supply, that cools the main chiller, sometimes gets interrupted.If so, the PLC switches on the air-cooled backup chiller.However, this causes the LV to be switched off also.


Safety

1. HW-based: interlock system

Beam Conditions

Monitor

Cooling-PLC

Vacuum-PLC

Module temperatur

esTemp-boards

Conditions

LV Off

HV Off

Retract VELO

Cooling Off

Actions2. SW-based: warning

and interlock system

132 cooling parameters monitored

3 levels each: warning, error and fatal

Emergency button

Combined information of 4 NTCs per module are input to the FPGA,which can interlock the LV.

Operation in vacuum requires immediate reacting safety systems. Three levels.


3. Human-based

Emergencybutton inthe LHCb control roomto power offthe VELO.


Keep the detectors cold 24/7At the tip the received fluence is 2x1014 neq/cm2 and type inversion has

taken place, so the sensors should always be kept cold, (below -8 oC), in order to prevent the Vdepletion to increase due to reverse annealing.

The beneficial annealing budget amounts to a handful of weeks at room temperature. We try to save it till we really need it. But the conditions of the LongShutdown1 period at LHCb make it hard to do so.

Vdepl NC + DNeff (=effective space charge density)

• Short term: “Beneficial annealing (NA)” • Long term: “Reverse annealing (NY)” - time constant depends on temperature: ~ 500 years (-10°C) ~ 500 days ( 20°C) ~ 21 hours ( 60°C)

NC

NC0

gC Feq

NYNA

1 10 100 1000 10000annealing time at 60oC [min]

0

2

4

6

8

10

D N

eff [

1011

cm-3

]

[M.Moll, PhD thesis 1999, Uni Hamburg]


The challenge is to keep the system operational 24/7.Under normal conditions the PLC deals with common problems.

Goal: minimize the warm time due to• scheduled maintenance of crucial components,• repair of malfunctioning components,• unexpected problems during LS1 and shutdown periods.


Regular maintenanceYearly maintenance of the R507a chillers is performed by a specialized external company. Downtime ~0.5 day / chiller.

Yearly maintenance of the 3 CO2 pumps is done by Nikhef-technicians. Pump is unavailable for >24 hours.Effective downtime of the system: 2 hours / pump.

Repair of failing componentsSo far no component had to be replaced, although a (redundant) pressure gauge stopped working in 2012, but miraculously reincarnatedafter 6 months.


Unexpected problemsHow do you know a serious problem occured, causing the cooling to go off and the detector to warm up ? Especially during LS1.Can’t rely on a PVSS-script sending a mail or sms.A modem and sms-routine have been installed in the PLC.

modem withSunrise sim card, so worksunderground.

antenna

When a problem occurs every half hour a text message is sent to alist of phones numbers, until the cooling system is again in a proper state.Acted 6 times since Feb. ‘13 due to failing services.


Integrated warm time in 2012: ~1 day

0

-30


year 2012

= 8 hours

18

0

-10

Summary• cooling system is continuously operational since >4 years,• performance is stable and according to specs,• redundancy of crucial components has shown to pay off,• clogging filters are annoying,• good thermal insulation is less simple than it seems,• superheated CO2 can be dealt with,• the warning system that sends sms-es is a great tool,• thus far the integrated warm time has been ~1 day/year, so ……….


Eddy Jans 27

Outlook


lets keep it cool till LS2, when the new VELO pixel detector goes in.

Documents

Eddy Jans 0 Vertex2013 Operational aspects of the VELO cooling system of LHCb Eddy Jans (Nikhef) on behalf of the LHCb VELO group Introduction Main components