11

Operations OverviewOperations Overview

L. ConeyL. Coney

UC RiversideUC Riverside

MICE CM32 – Glasgow – June 2012MICE CM32 – Glasgow – June 2012

22

OperationsOperations

New developments in OperationsNew developments in Operations Shifter training institutedShifter training instituted Review of May run – from Ops perspectiveReview of May run – from Ops perspective Operations NeedsOperations Needs

33

Developments in OperationsDevelopments in Operations

Standard Operations now includes:Standard Operations now includes: PPS PPS New target controller & user interfaceNew target controller & user interface Beam bump in ISIS Beam bump in ISIS change to 2V loss limit again change to 2V loss limit again Simplified Ops proceduresSimplified Ops procedures

Single key exchange with ISISSingle key exchange with ISIS Start-up 30 min; shut-down 20 minStart-up 30 min; shut-down 20 min MICE controls access to MICE HallMICE controls access to MICE Hall

C&M – see Pierrick’s talkC&M – see Pierrick’s talk DAQ – see Yordan’s talkDAQ – see Yordan’s talk Online Reconstruction Online Reconstruction

completed changeover to MAUScompleted changeover to MAUS InfrastructureInfrastructure

44

PPSPPS Smoothly operated – MICE personnel learning to use Smoothly operated – MICE personnel learning to use

system as designedsystem as designed ISIS requested dedicated Controlled Access sign-in record ISIS requested dedicated Controlled Access sign-in record

now have book in MLCR now have book in MLCR Record of both controlled access and completed Hall searchesRecord of both controlled access and completed Hall searches

No accidental Hall search drops during May RunNo accidental Hall search drops during May Run Must maintain this record for running in 2013!Must maintain this record for running in 2013!

Future:Future: How operations procedures affected by addition of SC magnets in How operations procedures affected by addition of SC magnets in

Hall?Hall? Need assess impact and decide on best practice procedures – Need assess impact and decide on best practice procedures –

requires assistance from expertsrequires assistance from experts

55

TargetTarget All safeguards & interlocks in placeAll safeguards & interlocks in place Integrated into ISIS BPS systemIntegrated into ISIS BPS system Single fortress lock key exchangeSingle fortress lock key exchange Automatic consistency checks done by Target Automatic consistency checks done by Target

system allows us to let go of “Target Calibration”system allows us to let go of “Target Calibration” Now reading out ISIS beam position*Now reading out ISIS beam position*

Future:Future: Still need agreement between ISIS definition of beam Still need agreement between ISIS definition of beam

loss and MICE definition of beam lossloss and MICE definition of beam loss ISIS requires new activation run with beam bump ISIS requires new activation run with beam bump

implementedimplemented

66

Online Reconstruction PlotsOnline Reconstruction Plots

Names: imagetof_hit_x, imagetof_hit_y, imagetof_pmt10, Names: imagetof_hit_x, imagetof_hit_y, imagetof_pmt10, imagetof_sp_ximagetof_sp_x

Shifter has no idea what to choose or what it showsShifter has no idea what to choose or what it shows

77

Online Reconstruction PlotsOnline Reconstruction Plots

Names: imagetof_hit_x, imagetof_hit_y, imagetof_pmt10, Names: imagetof_hit_x, imagetof_hit_y, imagetof_pmt10, imagetof_sp_ximagetof_sp_x

Shifter has no idea what to choose or what it showsShifter has no idea what to choose or what it shows Need to make the entire system more user-friendlyNeed to make the entire system more user-friendly

88

Online Reconstruction PlotsOnline Reconstruction Plots

Names: imagetof_hit_x, imagetof_hit_y, imagetof_pmt10, Names: imagetof_hit_x, imagetof_hit_y, imagetof_pmt10, imagetof_sp_x, imagetof_sp_x, ckov plot is imageTOF_Bckov plot is imageTOF_B

Shifter has no idea what to choose or what it showsShifter has no idea what to choose or what it shows Need to make the entire system more user-friendlyNeed to make the entire system more user-friendly We are aware of this and are working on itWe are aware of this and are working on it

99

Online ReconstructionOnline Reconstruction Good news – lots of progress made – ALL MAUS NOW!Good news – lots of progress made – ALL MAUS NOW!

Milestones:Milestones: Visualization – NOT DONE – primary busy on other workVisualization – NOT DONE – primary busy on other work Reco plots to webserver – DONE Reco plots to webserver – DONE

thanks to Chris R and Matt Rthanks to Chris R and Matt R Prototype Tracker Online Plots – DONEPrototype Tracker Online Plots – DONE

worked great for single station run – thanks to Edward S.worked great for single station run – thanks to Edward S. Prototype KL Online Plots – NOT DONE Prototype KL Online Plots – NOT DONE

Originally aimed at May EMR run – however, EMR delayed, KL not in beam – Originally aimed at May EMR run – however, EMR delayed, KL not in beam – reduced priorityreduced priority

Software On-Call position worked well during running – my Software On-Call position worked well during running – my congratulations to the Software Group!congratulations to the Software Group!

Future: More to be done – we have a good startFuture: More to be done – we have a good start Need tweak current plots – what we have, labels, descriptions, need Need tweak current plots – what we have, labels, descriptions, need

better documentation (is recognized – takes time)better documentation (is recognized – takes time) Want get Tracker plots on webWant get Tracker plots on web Want auto-recognize new runWant auto-recognize new run Need example for those wanting to implement new plotsNeed example for those wanting to implement new plots

1010

Infrastructure Infrastructure New procedures are documentedNew procedures are documented Documentation is up-to-dateDocumentation is up-to-date

Good progress made – Matt Robinson (Sheffield) working very hard Good progress made – Matt Robinson (Sheffield) working very hard for usfor us

Milestones:Milestones: Computing access – NOT done – need David C.Computing access – NOT done – need David C. MLCR backups – DONE – Matt installed NAS drive – backups startedMLCR backups – DONE – Matt installed NAS drive – backups started New OS installed – NEARLY COMPLETENew OS installed – NEARLY COMPLETE

Systematically switching machinesSystematically switching machines Automatic update mechanism decided uponAutomatic update mechanism decided upon

More:More: New portable opi machine (C&M user interface) – used to commission new New portable opi machine (C&M user interface) – used to commission new

DL tracker controls system in HallDL tracker controls system in Hall Set up onrec test machine in PPD – for pre-MLCR testing of online reco Set up onrec test machine in PPD – for pre-MLCR testing of online reco

codecode Installed miceraid4/5 on micenetInstalled miceraid4/5 on micenet Adding capability to onrec machinesAdding capability to onrec machines Setting up new DHCP server for micenetSetting up new DHCP server for micenet Assessment of security in progress – sysadmin, head of Software & Assessment of security in progress – sysadmin, head of Software &

Computing, head of Online GroupComputing, head of Online Group More webcams installed; new/better phone in MLCRMore webcams installed; new/better phone in MLCR

1111

MICE Shifter TrainingMICE Shifter Training

At last CM At last CM clear that improvements to shifter training were clear that improvements to shifter training were

neededneeded

Systematic, universal, consistent, and with Systematic, universal, consistent, and with good coverage for all shifter responsibilitiesgood coverage for all shifter responsibilities

Record of trainingRecord of training New shifter training regime developed for New shifter training regime developed for

and implemented during May Runand implemented during May Run

12

MICE Shifter MICE Shifter TrainingTraining

May 2012May 2012

1313

Official Shifter TrainingOfficial Shifter Training Overview & introduction – given by MOMOverview & introduction – given by MOM

Introduces responsibilities and role of ShiftersIntroduces responsibilities and role of Shifters Notification of safety training requirementsNotification of safety training requirements Description of shifter dutiesDescription of shifter duties

Practical trainingPractical training Training not real until actually start working with the equipmentTraining not real until actually start working with the equipment Each shifter-in-training has list of tasks they must learn how to Each shifter-in-training has list of tasks they must learn how to

completecomplete Specific individuals are certified to train shifters on systemsSpecific individuals are certified to train shifters on systems

Final practical testFinal practical test Shifter & Lead Shifter now meaningful titlesShifter & Lead Shifter now meaningful titles Not certified to take shift unless have completed this processNot certified to take shift unless have completed this process

Future:Future: Determine training lifetime and refresher course procedureDetermine training lifetime and refresher course procedure How accommodate new equipmentHow accommodate new equipment

1414

Shifter Practical TrainingShifter Practical Training

TaskTask When When

trainedtrained Who Who

trainedtrained

1515

Shifter Training PracticalsShifter Training Practicals

PPSPPS Macwaters, Coney, Blackmore, NebrenskyMacwaters, Coney, Blackmore, Nebrensky

TargetTarget Smith, Overton, any BLOCSmith, Overton, any BLOC

DS & BeamlineDS & Beamline Any BLOCAny BLOC

C&MC&M Hanlet, ConeyHanlet, Coney

DAQDAQ Karadzhov, Adey, Lead Shifter, ConeyKaradzhov, Adey, Lead Shifter, Coney

Online Monitoring & ReconstructionOnline Monitoring & Reconstruction Karadzhov, Rogers, ConeyKaradzhov, Rogers, Coney

DatamoverDatamover Nebrensky, previously trained shifterNebrensky, previously trained shifter

1616

Review of May RunReview of May Run Overall very successfulOverall very successful Congratulations to Tracker Team led by David AdeyCongratulations to Tracker Team led by David Adey Preparation in advance went wellPreparation in advance went well

Pre-run to-do list very useful for Hall preparationPre-run to-do list very useful for Hall preparation Micemine Run Plan extremely usefulMicemine Run Plan extremely useful Experimenters provided information in advance (goals, schedule, beam Experimenters provided information in advance (goals, schedule, beam

configurations) – greatly helped operationsconfigurations) – greatly helped operations

1717

Review of May RunReview of May Run New Shifter Training implementedNew Shifter Training implemented

Successfully trained ~12 peopleSuccessfully trained ~12 people However, proved difficult & frustrating to train and run However, proved difficult & frustrating to train and run

simultaneouslysimultaneously In future – training in advance of dedicated runningIn future – training in advance of dedicated running Requires personnel on site in advance of scheduled shiftRequires personnel on site in advance of scheduled shift

MLCR too smallMLCR too small David, Edward S, Ed O, Alan, Paul, Tom, MOM (occasionally), David, Edward S, Ed O, Alan, Paul, Tom, MOM (occasionally),

BLOC (occasionally), Lead Shifter, Shifter, Shifter-in-Training, BLOC (occasionally), Lead Shifter, Shifter, Shifter-in-Training, Occasional expert to address problemOccasional expert to address problem

Overflow area outside of MLCR – still not greatOverflow area outside of MLCR – still not great Need access to internet, equipment, other peopleNeed access to internet, equipment, other people Need options – possibly use nearby space? Need options – possibly use nearby space?

Remote MLCR?Remote MLCR?

Difficulties due to failing miceraid machinesDifficulties due to failing miceraid machines

1818

Operations Operations Analysis Analysis Group Group Need better connection between these groupsNeed better connection between these groups Must be good reason to take data – want it analyzed and fed back Must be good reason to take data – want it analyzed and fed back

into Operationsinto Operations What learn from study? & how does it affect running the What learn from study? & how does it affect running the

beamline?beamline? CKOV analysis of December dataCKOV analysis of December data Pion contamination study – can we improve?Pion contamination study – can we improve? D1 scan – are we running at the ideal spot or not?D1 scan – are we running at the ideal spot or not?

BeamlineBeamline What are “standard” beamline settings?What are “standard” beamline settings? When do we use “standard” settings when not?When do we use “standard” settings when not? Need magic spreadsheet to be right – Or we need to not use itNeed magic spreadsheet to be right – Or we need to not use it Need training on magic spreadsheet – which use when? Difficult to Need training on magic spreadsheet – which use when? Difficult to

guarantee appropriate beamline settings when doing on the fly in the guarantee appropriate beamline settings when doing on the fly in the MLCRMLCR

What proton absorber setting for each beam? Need table.What proton absorber setting for each beam? Need table. Study was done – but only for 3 settingsStudy was done – but only for 3 settings

1919

Operations Operations Beamline Beamline

Need standardize settingsNeed standardize settings What are the currents for a 6-200 beam?What are the currents for a 6-200 beam? Do we have the right ones? Matched beams or not?Do we have the right ones? Matched beams or not? How do we know?How do we know? All access to magic spreadsheet not necessarily goodAll access to magic spreadsheet not necessarily good

It’s wrong – missing material – extra materialIt’s wrong – missing material – extra material Does it handle material in the beam correctly?Does it handle material in the beam correctly? How do we use it/them? Need change “magic” into “well-How do we use it/them? Need change “magic” into “well-

understood”understood”

Glad to have Maria on board!Glad to have Maria on board!

2020

Step IV OperationsStep IV Operations Continue:Continue:

Continue pre-run preparationContinue pre-run preparation Further simplify runningFurther simplify running

Improve:Improve: Automate procedures Automate procedures fewer errors fewer errors Move from spreadsheet Move from spreadsheet only ConfigDB only ConfigDB Increase Online Reconstruction capabilitiesIncrease Online Reconstruction capabilities

Complete suite of detectors, Online analysis, Global recoComplete suite of detectors, Online analysis, Global reco

New equipment arrivingNew equipment arriving Need expertise – do we have a plan for support?Need expertise – do we have a plan for support?

Timing – with ISIS RF for tracker, plan for RF stagesTiming – with ISIS RF for tracker, plan for RF stages Modify procedures to include SC magnets and LH2 Modify procedures to include SC magnets and LH2

systemsystem How staff multitude of shifts?How staff multitude of shifts?

2121

Operations NeedsOperations Needs Standardize & understand beamline settingsStandardize & understand beamline settings

How cooling channel B fields affect How cooling channel B fields affect Operations?Operations? can we still control/monitor/operate the equipment?can we still control/monitor/operate the equipment?

Handover/knowledge retention between MOMs Handover/knowledge retention between MOMs still not ideal still not ideal concerns regarding running in 2013 concerns regarding running in 2013 need permanent overarching positionneed permanent overarching position

Systematic understanding/overview of MICE Systematic understanding/overview of MICE safety systemssafety systems MOM position fleeting – lose track of what doneMOM position fleeting – lose track of what done

2222

ConclusionsConclusions Operations going well – we continue to Operations going well – we continue to

improve for each runimprove for each run

Feb CM Operations Needs:Feb CM Operations Needs: New magic spreadsheetNew magic spreadsheet New takeover of G4BeamlineNew takeover of G4Beamline Proper trainingProper training Documentation to be (remain) currentDocumentation to be (remain) current Documentation of changes to procedureDocumentation of changes to procedure

Much more to do – Step IV is rapidly approaching