1 Machine Checkout and Setup Periods Jan Uythoven Thanks to: O.Aberle, R.Bailey, F.Bordry, O.Brunner,…

1

Machine Checkout and Machine Checkout and Setup PeriodsSetup Periods

Jan Uythoven

Thanks to: O.Aberle, R.Bailey, F.Bordry, O.Brunner, L.Bottura, E.Carlier, P.Charrue, E.Ciapala, S.Claudet, R.Denz, D.Forkel-

Wirth, B.Goddard, M.Jimenez, R.Jones, M. Lamont, V. Mertens, P.Ninin, B.Puccio, S.Ramberger, F.Rodriguez

Mateos, R.Schmidt, L.Serio, B.Todd, M.Zerlauth

Jan Uythoven, AB/BT

Chamonix@CERN 2005, Machine Checkout

Page 2

Machine Checkout: Machine Checkout: WHEN ?WHEN ?

Year 1: Following Hardware Commissioning Assume all equipment tests are done in the past 18 months Most of the system tests are done in the past 18 months

Subsequent years: Following Shutdown Work Assume most equipment tests are done

Machine Checkout First time machine complete cold and closed All systems running together from the control room

Before First Beam

Jan Uythoven, AB/BT


Page 3

Tasks - SummaryTasks - Summary Redo a subset of tests performed during the Hardware

Commissioning period Because some tests more than a year ago Because the final configuration of the system might be different,

especially the interface to other equipment If hardware commissioning period was squeezed in time, this will have

been at the cost of the system tests As a result: will need more time for the Machine Checkout

New Tests Additional system tests All systems together – machine wide, interdepedencies Machine completely closed, last access / final checks

Under the responsibility of the Operations Group Green light from DSO, info for INB

Jan Uythoven, AB/BT


Page 4

First beam

Phases of Machine Phases of Machine CheckoutCheckout

Hardware Commissioning Sectors 1-2 and 2-3 Cooldown of all

sectorsELQA tests Close

machine completely; Contr. Access

Complete Machine Cycle from CCCDSO tests

Tim

e

Equipment tests per 2 sectors:Interlocks,Power ConvertersRF, kickers, BDI,..

Machine Checkout phase I

Machine Checkout phase II

Machine Checkout phase III

Some phases in parallel

Jan Uythoven, AB/BT


Page 5

Phase IPhase I Infrastructure for the complete machine

Water, Ventilation, Power, Controls Network Radiation Protection (RAMSES)

Check response ionisation chambers Interface with other systems

UPS, AUG Cooldown of the complete machine

Sectors T up to 200 K: need 10 days for cooldown If T > 80 K: need to redo Electrical Quality Assurance tests

(ELQA), first time 3 weeks If T < 100 K: cooldown needs about 5 days

Up to about 5 weeks for sectors warmed up > 80 K. Need to start cooldown in parallel with Hardware Commissioning. Manpower.

When full inventory of He available?

Jan Uythoven, AB/BT


Page 6

Phase I, continuedPhase I, continuedAccess SystemAccess System

Close the complete machine Test of LHC Access Control System (LACS) Test of LHC Access Safety System (LASS)

New tasks – before only done per sector (during HC) New tasks: interface adjacent sectors, machine wide

Important Safety Element in point X must interface over the global machine and beam dump in point 6

Interface with Radiation Protection equipment: access veto Interface with HR data base, biometrical data Some equipment was not in final state during Hardware Commissioning tests Exercise procedures for ‘patrouille’, giving access, back from access, …

Start first with sectors needed for Machine Checkout equipment tests phase II

If necessary, time to sort out problems parallel to phase II programme

Ideally only controlled access from phase I onwards

Jan Uythoven, AB/BT


Page 7

Phase IIPhase II Per two sectors

Check interlocks: PIC, WIC, QPS See

Test all power converters, two sectors at a time

See

Simultaneously in other sectors Other equipment checks, see

Determines the pace of tests in Phase II

Jan Uythoven, AB/BT


Page 8

Phase II Phase II Interlock SystemsInterlock Systems

Before power converters can be tested, test interlock system Quench Protection System (QPS)

Visual check of equipment, half day per sector Automatic test procedure, 8 hours (same as monthly procedure): fires the

quench heaters without current in magnets Test interface to BIS -> check in phase III

Powering Interlock Controller (PIC) Takes information from the QPS Automated test procedures and visual control Needs about 8 hours for testing

Warm Interlock Controller (WIC) No automated procedures foreseen Some manual testing and visual control: 8 hours

Jan Uythoven, AB/BT


Page 9

Phase IIPhase IIPower ConvertersPower Converters

Followed by testing of Power Converters two sectors at a time Run all converters to full power, perform tests at full power Calibration already done during Hardware Commissioning 12 h continuous running at full power Need about half a week per two sectors

Proceed around the ring

Jan Uythoven, AB/BT


Page 10

Phase IIPhase IIControl SystemControl System

Control System Communication Mechanisms Field buses operational machine wide. All elements to be

checked: need about 8 days Already start during phase I

FrontEnds, BackEnds and general services Should have been made available during the Hardware

Commissioning period sector by sector and left running Timing generation and distribution

6- 8 days for the first year Databases access

1 day, at least

Jan Uythoven, AB/BT


Page 11

Phase IIPhase IIControl System, ctd.Control System, ctd.

Application software Tested continuously during the last months / years Specific projects need testing

Interlocks, realtime feedback, logging, postmortem, safe beam flag etc.

Estimate 10 days for test …..??? Final tests when complete machine available:

See phase III

Message from many (CO, OP and BDI):

DRY RUNS, dry runs & DRY RUNS

Quoting TI 8 experience

Jan Uythoven, AB/BT


Page 12

Other Equipment tests:Other Equipment tests:RFRF

Basic work of testing cavities, cryo, LLRF and control during hardware commissioning period

Number of tests which would clearly accelerate RF operations and improve performance:

- Reliability of RF system: power & cryogenics- Software and diagnostics developments / improvements- New conditioning hardware for subsequent years- RF noise and ripple measurements- Customisation of klystrons (positioning of resonance)- Linearisation of klystron response- Effect of detuning and coupler position on loop stability- Optimisation of loops for higher intensity- Fine adjustment of trip level for fast transients- Setting-up of the longitudinal damper- Preparation of feed-forward and 1-T feedback

Ed’s Wish List

RF will use the time made available

Jan Uythoven, AB/BT


Page 13

Other Equipment tests: Other Equipment tests: Warm MagnetsWarm Magnets

Visual inspection No spanners lying around… All protection covers in place

Cooling system o.k. All valves open No visible leaks

Expect 1 day work per sector, 8 days total

Jan Uythoven, AB/BT


Page 14

Other Equipment Tests:Other Equipment Tests:VacuumVacuum

Assume machine already cold & closed

Check interlocks Sector valves Connection to BIC

Not to forget Vacuum interlocks on transfer lines: 2 days

3 days per octant2 teams in parallel 12 days

Jan Uythoven, AB/BT


Page 15

Other Equipment tests:Other Equipment tests: Collimators and Beam AbsorbersCollimators and Beam Absorbers

Redo some of the hardware commissioning tests in local & remote Motor control, position switches Temperature sensors, cooling system

Interfaces with more intelligent software from the CCC to manage settings Not defined yet, should include BIC

Couple of days In parallel with other tests

Jan Uythoven, AB/BT


Page 16

Other Equipment tests: Other Equipment tests: Pulsed MagnetsPulsed Magnets

Beam dump kickers (MKD and MKB point 6) Injection kickers (MKI points 2 and 8) Tune and Aperture kickers (MKQA point 4) For all of them

Verification of local interlocks Pulse at full power – MKI some re-conditioning time Interface with many systems across the machine

RF point 4 (rev. freq.) Energy from dipoles either side point 6 Timing, controls, machine protection, post mortem OASIS, logging Cross connections between MKI and MKD

Couple of days per system

Couple of WEEKS

Maximum in HC, start early, profit from RELIABILITY RUN

Jan Uythoven, AB/BT


Page 17

Other Equipment tests: Other Equipment tests: BDIBDI

Large distributed systems: BPMs & BLMs Hardware fully checked during HC, but re-check Main tasks: software and logging, links to other users

Interlocked Systems – check interface BIC (phase III) BLM, BCT point 4, BPMs point 6, LDM (Longitudinal Density Monitor)

Diagnostics systems – no special requirements Q and Q’ measurements, Schottky systems, Sychrotron radiation

monitors, OTR screens, Wire scanners

Experience has shown that BDI ALWAYS wants to have another access

CONTROL SYSTEM DRY RUNS, distributed over about

2 months, so start BEFORE the machine checkout

Jan Uythoven, AB/BT


Page 18

Phase IIIPhase III Status after Phase I & II

All equipment and systems (re-)tested, sector by sector Individual interlocks tested Controlled access

Phase III Machine wide tests from the CCC Go through complete machine cycles without beam Accent will be on control system and application software Interleaved with final tests of access system and green light by DSO In principle no access should be necessary any more Control System Tests: Build on experience !

To be tested throughout the yearS with the maximum number of ‘dry runs’ TI 8 tests with beam Hardware Commissioning of all sectors including system tests Sector test with beam

Jan Uythoven, AB/BT


Page 19

Phase IIIPhase IIIComplete Machine Cycle Complete Machine Cycle

Without BeamWithout Beam Initial machine cycle (45 minutes) to get to initial field conditions

No further tests possible – real magnetic field to be found out with beam Generate functions for the power converters Download all functions to power converters – synchronous ramping of all

power converters including energy ramping of beam dumping system Settings management, trimming of functions

Machine wide timing of the systems, pre-pulses etc. Timing (slow and fast), OASIS to set timing of kickers Interface with RF, injection kickers, beam dump kickers, machine protection system

Applications – communication with all equipment All fixed displays, video, trajectory and orbit correction Feedback applications: trajectory, tune, chromaticity, transverse, longitudinal Logging of all parameters during the cycle Vacuum displays, alarm screens, radiation monitoring

Interface with the Beam Interlock System Safe beam flag depending on energy Post mortem after beam dump

Some kind of sequencer to automate tasks (LHCexec)

Jan Uythoven, AB/BT


Page 20

Phase III, continuedPhase III, continued

Now obligatory to have machine completely closed If search broken during phase II: Complete patrouille with several teams:

2 days Re-test complete access system before DSO test: 2 x 12 hours

Machine protection system has probably been touched upon during phase II Followed by Beam Interlock System (BIS) tests (or BIC)

Need “Green Light” from all systems – towards the end of test period 16 different users, 150 ‘user boxes’

Every box needs to receive an interlock, provoked by the equipment connected to it. Check 4 x dump request per equipment.

Estimate 3 x 8 hours to test – 2 days Rubber stamping of systems by DSO – data for INB

Need about 1 day, if everything goes fine Rubber stamping by ‘Mr. Interlock’

Need formal test procedures and acceptance criteria

Jan Uythoven, AB/BT


Page 21

Time RequiredTime Required

Phase I: 1 week minimum Cooldown and closing machine

Phase II: 3 weeks minimum Interlocks, equipment &

system tests Phase III: 2 weeks minimum

Complete cycles without beam Final test access system and BIS DSO tests

Minimum of 6 weeks on planning!

Partially parallel with hardware commissioning

Equipment setup with beam, see

Jan Uythoven, AB/BT


Page 22

Equipment Tests With Equipment Tests With BeamBeam

BDI Calibration BPMS, BCTs Set-up of tune and chromaticity measurements

RF Low level control (typically 3 weeks) Phasing Feed back

Control system Logging beam data, post mortem Steering – feedback

Machine protection tests Check interlocks of the system Safe beam flag

Beam Dumping System Beam dump to be set up at the different energies Verification of post mortem, logging and analysis

Cryogenics with significant beam load

Continuously during the first

weeks / months of operation

Jan Uythoven, AB/BT


Page 23

Not to underestimate:Not to underestimate:Transfer LinesTransfer Lines

TI 8 will need recommissioning Tested before

With beam up to TI 8 TED end 2004, Not tested yet: last 100 m + collimators Sector test end 2006

But will need to be recommissioned before first beam TI 2: never fully commissioned before

Will probably only be ready for commissioning at about the same time as the last LHC sector (transport of all LHC magnets through the TI 2 tunnel)

Could be done in parallel with hardware commissioning of the last sector: preferred solution but impact on HC schedule

Other option is to do this in parallel with ring 2 beam commissioning: higher risk in case of problems

Jan Uythoven, AB/BT


Page 24

Machine Checkout Machine Checkout For Subsequent YearsFor Subsequent Years

No Hardware Commissioning period but shutdown work to be finished

Outline stays the same with similar tests Some tests should be faster, because systems are better

known Foresee about 5 weeks minimum: longer than SPS / LEP

because systems are more complex, machine protection to be taken more seriously

Compare LEP and SPS:About 2 weeks of pre-closure and 2 weeks of real machine checkout when machine is

for OP 2 weeks of LEP / SPS ‘cold checkout’ similar

to 2 weeks of phase III of what is presented for

the LHC

Jan Uythoven, AB/BT


Page 25

ConclusionsConclusions After the hardware commissioning period the machine is not

ready for beam Confirmed by experience from LEP & SPS, need a period to get ready for

beam under the responsibility of OP Even with all equipment groups 100 % ready on time (!): complete systems

tests remain and their interdependencies need to be verified At least 6 weeks of tests need to figure on the LHC planning for

Machine Checkout Start earlier for cooling down the sectors warmed up > 80 K for ELQA tests,

in parallel with hardware commissioning Start earlier for software and machine protection: DRY RUNS

to be planned and scheduled Many tasks in parallel: need careful planning and good coordination

Some running in of equipment with beam is also required BDI being the obvious candidate, but also RF, beam dumping system

Don’t forget the transfer lines Same applies every year after a shutdown

Documents

1 Machine Checkout and Setup Periods Jan Uythoven Thanks to: O.Aberle, R.Bailey, F.Bordry, O.Brunner,…