1 BROOKHAVEN SCIENCE ASSOCIATES The Path to Accelerator Commissioning F. Willeke Accelerator Systems Director NSLS-II PAC Meeting December 10-11, 2009

1 BROOKHAVEN SCIENCE ASSOCIATES

The Path to Accelerator Commissioning

F. WillekeAccelerator Systems Director

NSLS-II PAC MeetingDecember 10-11, 2009


framework of documents: clearly identifies requirements for • documentation readiness,• technical readiness,• personnel Training and Qualification,• hardware and software testing defines the acceptance criteria for CD-4 at the end of commissioning.

hardware and software testing:•sub-system testing and qualification,•integrated system tests •commissioning with beamthis document is necessary at this stage to start preparing for start-up

Start-Up and Test Plan


Integrated Testing

Integrated Testing Technical test of Accelerator Subsystems

- Check out Safety Systems (this is not the “official” safety test)

-Prepare systems for operations

-Test automatic and semi-automatic turn-o procedures

- Verify Assignment, Polarization, Correct Magnitude

Examples:

Systematic check of the components of the safety systems (door locks, shutter positions, including correct wiring, interfaces to RF, main dipole Power Supply, injection/extraction pulsed magnets

• Check Assignment, polarity and magnitude of the magnet current s in the accelerator tunnel as directed by setting using the application

• Check assignments of thermal protection switches on magnets to PS via application software

• Test of the vacuum system, fine-leak checking and fixing, in-situ bake-out

• Turn on/off all power supplies by automatic procedures


Safety System Verification• Formal technical check, Well documented process

•Performed by an independent group (Laboratory QA and Light Source ES&H personal

• Governed by Laboratory Standards and by DOE requirements

• Technical checks out include (but are not limited to)

functioning of components

- door contacts, door-warning lights

- emergency buttons,

- mechanical systems: fences,

- warning signs, signs indicating emergency buttons

- warning systems: acoustical warning, blink-lights, tunnel lightening, warning-tableaus

- Interfaces to accelerator hardware: RF, main dipole, shutters, triggers for septa, gun,

Interlock logics: check whether any kind of anomalous status will lead to desired effect (turn-off the beam, RF )

training of operators and key personal


BNL DOCUMENTS DEFINING THE REQUIREMENTS

• accelerator readiness review that must be conducted prior to a commissioning“Template for the Accelerator Readiness Review (ARR) Plan of Action”

• BNL interlock requirements that apply:“Physical Access Controls for High and Very High Radiation Areas”

Effective Date: Jan 8, 2004 (Reviewed: Jan 8, 2004) | Periodic Review Due: Dec 31, 2009

• the interlock test program at the NSLShttp://www.nsls.bnl.gov/esh/qa/pps.htm

Requirements Radiation Safety Systems Testing


framework of documents: clearly identifies requirements for

•documentation readiness,• technical readiness,• personnel training and qualification,• hardware and software testing defines the acceptance criteria for CD-4 at the end of commissioning.

hardware and software testing:•sub-system testing and qualification,•integrated system tests,•commissioning with beam

Start-Up and Test Plan


particular mode of accelerator operation with beam aiming:

• test the function of accelerator hardware and software with beam, • verify in particular the proper functioning of the equipment protection system, • verify the adequacy of radiation safety shielding, area radiation monitoring, ALARA system to minimize radiation• check the integrity and the consistency of the subsystems as built• develop refined settings of the hardware components to allow efficient injection, and storage of accelerator beams with good stability, beam intensity, the advertised beam parameters, and good beam lifetime.• condition accelerator hardware for optimum performance (vacuum, RF)• develop and document the procedures which are necessary to operate the accelerator routinely and to perform continuous improvement and development.

Commissioning


Commissioning needs BHSO approval authorization process, addressing all relevant issues

• completeness of documentation, • technical readiness, • training of the staff,• existence of detailed commission procedures, • assessment of hazards, • safety and emergency procedures.

Commissioning Authorization


The NSLS-II Authorization Base is defined by the requirements and the guidance in:

• DOE Order 420.2B “Safety of Accelerator Facilities” • DOE Guide 420.2-1 “Accelerator Facility Safety Implementation Guide”• DOE Order 413.3B “Program and Project Management for the Acquisition of Capital Assets”• BNL SBMS Subject Area “Accelerator Safety”

Guiding Documents


Elements of the Authorization Base

Commissioning Modules• LINAC (including LINAC to Booster Transfer Line) • Booster (including Booster to Storage Ring Transfer Line) • Storage Ring • Full Operations (roll-up of first three modules)

Module Components (concluded by authorizations and reviews)• SAD (Safety Assessment Document)• ASE (Accelerator Safety Envelope)• Authorization Plan (commissioning plan)• ORE (Occupational Readiness Evaluation)• ARR (Accelerator Readiness Review)


Coupling of Schedule Elements

Includes staff trainingIncludes staff training


Commissioning StagesLINAC Frontend (gun pre-buncher) early delivery, commissioning and early tests by NSLS-II staff in RF Lab, Special bunch-modes

LINAC commissioning by the vendor (responsible)LtBTL (LINAC building part ) commissioned by NSLS-II-Staff , primary + secondary beam dump

BoosterLBTL commissioning by NSLS-II staff in parallel to booster integrated testingBooster commissioning by vendor (vendor’s responsibility), participation of NSLS-II staffBSTL (part in booster tunnel) performed by NSLS-II staff with project responsibility

Storage Ring Commissioning Phase 1 (without ID)BSTL commissioning and injection set up ; the storage ring beam optics; adequacy of correction systems; adequacy of safety and ALARA systems; efficient injection; adequacy of beam instrumentation; orbital stability; RF set up and stability; RF conditioning; cryogenic stability; beam stability; vacuum integrity and conditioning; high intensity operations

Storage Ring Commissioning Phase 2 Integrate Insertion devices


Assumptions on Conditions

• accelerator tunnel air conditioned; tunnel temperature, humidity within specified range.

• final survey and alignment shortly before start commissioning (ground settling)

• heavy duty construction activities completed

• global utilities available: electrical, de-ionized water, chilled water, and liquid N2

CF deliverables

installation of girders and magnets completed ~1/2 year start of commissioning. cable conduits and openings in the tunnel walls will be properly closed and shielded


SR Subsystem Status at Commissioning

• integrated testing complete• Injectors and transfer lines commissioned• personal safety system and interfaces thoroughly tested• equipment protection system is fully implemented and tested via the control system.• magnets systems installed well before commissioning, • precision alignment shortly before commissioning • insertion devices will not be installed• power supply systems are complete, technical interlocks have been properly tested• vacuum system has a vacuum of at least 10-7 mbar• two sc RF cavities installed driven by a single 310kW transmitter station. • full suite of beam diagnostics is installed and tested• water cooling systems have been installed tested• timing system implemented• control system is fully implemented• relational database is fully functional• all application programs deemed necessary for commissioning available • injection systems have been installed and tested• fast orbit feedback has been implemented• transverse feedback damper system installed and tested


Radiation Safety during Commissioning

• radiation protection systems installed and fully tested, no provisory solutions allowed, no exception possible.

• initial beam intensity very small compared to nominal (0.1%).

• area monitors readings low intensity will be extrapolated to the next step in beam intensity.

• extrapolation is reviewed and analyzed at each level of intensity before the next step in intensity is made.

• initial high intensity studies: assume losses e much larger than under optimized normal conditions may need special (temporary controlled radiation areas)

• beam containment system needs to be tested with beam, location of beam losses identified, beam is lost dominantly in the extra shielded confirmed

• clear and unambiguous procedures operating staff needs to be trained to follow these procedures before operations with continuous injection of high charge (top-off, high intensity operation) can proceed.


Storage Ring Commissioning Modules

COMMSIONING PART IEstablish Initial Beam Operation, 5 modulsCheck out Beam Instrumentation 4 modulsCheck out Safety Functions 3modulesFine tuning of Beam Optics 7modulesFunctionality Tests 1modulsFine Tuning of Orbit and Emittance 3modulesSynchrotron Radiation Measurements 4mod.High Intensity Studies 10 modules

COMMSIONING PART IISafety Related Measurements 2 modules ID Integration 8 x 7 modules Preparation of User Operation 3 modules

70 commissioning modules

Each needs ~4 shifts on average

~300 shifts

100 days

Operational efficiency is assumed 50%

Need 200 days of commissioning


Storage Ring Commissioning Part IEstablish Initial Beam OperationObtain stored beamAdjust and verify RF parameters first Orbit Correctionfirst iteration of correcting chromaticity and coupling first iteration of obtaining acceptable injection efficiencyCheck out Beam Instrumentationestablish beam monitor calibration with local bumpschecking intensity monitorschecking out loss monitorschecking out emittance monitorsCheck out Safety FunctionsCheckout top-off safety functionsCheck-out machine protection systemCheck out BCS and ALARA functionalityFine tuning of Beam OpticsBeam Optics Checks and correction (Response matrix, phase advance, coupling measurements)Beam based alignmentmeasurement of chromatic distortions and correctionnonlinear dynamics related measurements and corrections (D.A., amplitude dependent tune shift, width of resonances, higher order chromaticity)measure 1st and 2nd order momentum compaction factormeasure damping distribution

Functionality TestsFast orbit feed-back testFine Tuning of Orbit and EmittanceOrbit correction to micron leveldispersion free steering and orbit correctionvertical emittance tuning Synchrotron Radiation Measurements Measurement of power deposition s and power loadcheck of temperature monitor systemcheck of vacuum interlockcheck of assumptions on absorber and mask temperatures, monitoring and coolingHigh Intensity StudiesSet up of high efficient injection Set-up of RF feedback and fine tuning of feedback parameters and LLRFSingle Bunch Intensity Limit measurementsbunch lengthening by 3rd harmonic cavity beam lifetime vs bunch length and bunch intensity studyBCS and ALARA studies with high intensity Vacuum conditioning beamRF conditioning with beamtransverse damper testStudy of high multi-bunch intensity limitations


Commissioning Staffing• commissioning will be organized in 3 8hr shifts per day and seven days per week.

• commissioning labor budgeted within the project

• commissioning requires operation of already commissioned subsystem (injectors, cryogenics, utilities)

• commissioning requires the involvement of all accelerator physicists (budgeted) , subsystem experts (budgeted) and operators

• workload of off-hour shifts will be shared between all high level staff

• engineers and technicians on call during commissioning

• control room manned with 1 accelerator physicist ~33 weeks. @ 16000 hr (budgeted)

• involvement of engineers less regular, will depend on the commissioning program

• initially, one machine operator present every shift will gradually develop into a mode with 2 operators and no accelerator physicists.


Commissioning Information and Documentation

• semi-formal meetings of 15 min at shift change ( coming and going shift crews) seamless transition from one shift to the next, communicate and discuss fine-tuning of program. 3 shifts per day and thus there will be three such meetings.weekly commissioning meeting will summarize the commissioning results.opportunity to optimize the commissioning program and direction. • weekly commissioning meetings will be documented including all material presented. • commissioning carried out in modules (ca 200) each requires written plan:

the purpose, the goal, supporting documents, results of calculations necessary to carry out the module estimate on the needed machine time labor resources required.

• results of each module will be documented• raw data will be stored in an organized and accessible way • evaluation, resulting set-points, procedures, parameters will be documented• commissioning report will be published at the end of commissioning


Applications needed for CommissioningOperation Softwareoverall status pagestatus, alarm and warning monitorpermit system monitor and controldata logger and data displayelectronic logbookOperations Softwareaccelerator store/restoreInjection Controlpower supply controlRF controlfast orbit feedback controlfast transverse damper controlfront-end monitoring and controlmachine protection displ &contrmagnet temp. interlock dspl & contrscraper and collimatorssystem turn-on, system shutdown

Utility Controltunnel temperature and humidity monitormechanical utilities status and controlelectrical utilities status and controlsequipment enclosure monitorwater cooling system displaycontrols network monitorAccelerator physics applications •static orbit corrections, first turn steering, •chromatic correction, •response matrix measurements, •phase advance measurements, •beam base alignment measurement,•bpm test programs, •beam optics measurement, •beam based alignment of sextupoles, •analysis on nonlinearities•dispersion measurement and correction, •closed Orbit bump page

Major Subsystem ControlPower supply page RF monitor and controlVacuum display and control Cryogenics system display and controlpulsed magnet monitor and controlinjection element display and control Insertion device controlFront-end control and status Beam DiagnosticsBeam Orbit page withBeam current history and lifetimebunch intensity display and historybeam emittance displayTiming system display and controlSynchronization system displ & contrTune display and controlTemperature monitoring displaySafety Systemspersonal protection system status equipment protection display and controlbeam containment display and controltop-off status monitor


Accelerator Control Room during Commissioning

accelerator control room is the location where:• operators operate the accelerator • operators coordinate with technical staff about technical difficulties and interventions• accelerator experiment are carried out • a logbook on accelerator operations is kept and maintained, • information in case of an emergency is to be obtained and such information will be delivered to• an access point is established which manned 24hr a day 7 days a week• day-to-day coordination between accelerator and experimental floor are exchanged, • shift-change briefing meetings are held, • first hand information on the status of the accelerator is available• the responsibility for operating the accelerator safely within the safety regulations resides• routine safety measures are coordinated (LOTO)• preparation of the accelerator for operations is coordinated after an interruption• computer provide an optimized selection of delays for quick overview of accelerator status• an emergency stop button and other special installation (LAN, site-wide audio) are installed

functionality very different between commissioning and operation at commissioning assumed that the control room is close to the accelerator hardwarecontrol room LAN is accessible from the accelerator tunnel and from the mezzanine.


Commissioning Schedule


Commissioning Budget


Summary

• We have defined a clear path to commissioning• The associated formal procedures and the regulatory has been taken into account in the planning• There is good understanding of the circumstances under which commissioning will take place• There is a good understanding what needs to be done which is base of the preliminary plan• Commissioning budget and schedule are in place (need to revisit once the plan has matured• Next goal is to present a “Start up and Commissioning Plan” to be presented at the Lehman review in February


Accelerator Commissioning and Operation Authorization