CSC Shift Training and Operation + B904 Status

14 March 2010 CSC Training/Operation & B904 by Fred B.

1

CSC Shift Training and Operation

+ B904 Status

Fred Borcherding

Fermi


2

Overview

• Training Changes to date– Training is now being conducted at Pt5

– Training is focused on • LHC operations

• cDCS changes

• Run Registry changes

• Differences for Operation and Procedures– cDCS

– Run Registry & DQM

• B904 Status– Cosmic Ray Runs Taken

– Being used for hardware & firmware testing


3

New Main Shift Station

• Main station now has 4 monitors

– UL > EMU Page 1

– UR > DCS

– LL > Run Registry

– LR > DQM / Run Evaluation


4

Changes to Shifter Training

• Moved training to Pt5– Classroom training in

meeting room– With laptop project slides

and/or real-time views of shifter pages

• Point where training could be improved– Assistance with language

• Often with training in English for non-native speakers question go unasked and misunderstanding is too common

• Set requirements on 1st time shifters and returning shifters

• More formal shifter certification?– Now training is required– Require minimum of training

and hands on shift training for shift to count

• Sit as 2nd shifter for part of a shift to complete certification

• Who certifies? Now> Me, Richard

• Need someone to conduct training who is based at CERN


5

Shift Changes – c(entral)DCS

• Moved CSC Shifter DCS to cDCS– Moved DCS procedures out

of CSCOperations and – Into CSCOperationsDCS for

cDCS• Moving DCS procedures to

pdf file– Hypertext linked file – quick

navigation– In pdf format – fixed and

changeable only by experts– See version 1.0

• Status so far– cDCS has control since

Thursday– All alarms addressed at

cDCS station– cDCS shifter AND CSC

shifter together address alarms and problems

• When is DCS Back to CSC?– CEO and Experts fixing

problems, calibrations and other special operations


6

Top Level of CSC DCS UI

Communication Status

Majority Logic

Subsystem Panel

HV, LV, FEDS

CSC End View

* CSC

* PCrates

Login Panel

Value Monitor

Key

CSC Shifter

CSC CEO

CSC CEO

CSC Shifter

Who to Contact

• CSC Shifter - shifter address problem

• CSC CEO – shifter contact CEO

CSC CEO >>16 1972


7

Top Level Procedures for CSC DCS

• Subsystem Panel– Alarm <RED> indicates problem

in all or one end of CSC– Contact CEO 16 1972– Check if cDCS has infrastructure

alarms• Majority Logic

– Alarm <RED> indicates that a significant percentage of CSC devices are in alarm

– Contact CEO 16 1972• HV, LV, FEDS

– Alarm <RED> indicates that a major component of the HV, LV or FEDS is in alarm

– Contact CEO 16 1972– Do NOT turn these back on –

ONLY expert should correct

• End View– Multiple chambers or crates in Alarm

<RED> – indicates there may be a major problem – check bottom of the display – Contact CEO 16 1972

– Single chamber or crate in Alarm <RED> indicates chamber or crate has a single device in error – CSC Shifter follow detailed procedure

• Communication Status– Alarm <RED> monitor, if it does not

clear itself after 5 minutes contact CEO 16 1972

– NOTE: Question marks ‘?’ will also appear on End View

• Value Monitor– This is a diagnostic panel– Shifter hourly cycle through displays

and follow detailed procedures

CEO Level Alarms Shifter Level Alarms


8

Run Registry

• Run Registry – Same tool as used by

Central DQM shifters• Version for CSC

– Divides CSC into several parts

– Authored by Valdas• Local DQM display

has added features– Known problem list

and comparison– Filtering of severity of

errors– Reference run

comparison

• Status - – Shifters have been putting runs

into RR– Shifters have been grading runs

in RR– NEED expert(s) to review and

sign off each run


9

CSC Run Registry

• Top panel of the CSC version of the Run Registry

• http://pccmsdqm04.cern.ch/runregistry/ - Workspace > CSC

• Each run is graded by – CSC > Overall– CSCTF > Track Finder– DDUs > DDU

operation– Rings > by each

station/ring

http://pccmsdqm04.cern.ch/runregistry/


10

Categories

• CSC >> Overall grade for this run– BAD > tbd

• CSC TF >> Grade and details for TF– tbd

• DDUs >> Grade and details for DDU– BAD > 2 or more CRITICAL

(including known)

• Rings >> – BAD > 2 or more CRITICAL

or DEAD (including known)– Insert details for each

CRITICAL and list DEAD


11

New DQM Features

• The DQM display has new features– Compare to run >

REAPEATING_PROBLEMS > when current has same problem

– KNOWN_PROBLEM > compares to a list of known problems

– Filtering > FILTERS > only shows the severity of interest

• NOTE – The light grey are filtered out errors– The blue are the same as for the reference run– The Hot Pink are KNOWN PROBLEMS


12

Action

• These ‘CRITICAL’ CSC problems are not a ‘Known Problem’– Enter the problem’s

messages into the RR

• NOTE – The example ME-2/1/02 is BLUE because it has the exact same problems as for the reference run. Since the display is filtered we see only the CRITICAL chambers


13

Action (cont.)

• These ‘CRITICAL’ CSC problems are the same as for the reference run– Do not enter into the

ELOG– If these CRITICAL

problems were new for this run it should be entered into the ELOG


14

B904

• CTS changes– Can run CR

– HV connected to CAEN supply

• Duplicate HV source possible

– Computers and base crates connected to wall power

• Duplicate power available

– DSS system has been upgraded

• Future– Set up for LVDB

diagnosis / repair?

– Dedicated 2nd chamber?

– UPS for computers?

– Remove chiller from rack DSS interlock?

– Train more people to operate the DSS rack interface?


15

Summary

• Shift training is more formal• New documentation has been put in place for

cDCS and Run Registry• Is there enough documentation for shifters?

– NO, need to push experts to provide operation manuals.• What is the progress on validation by shifters?

– CSC RR is being used by shifters to grade CSC for runs • Are the main goals of shifts being achieved?

– Yes, CSC is running well. The safety of the detector has been assured.

Documents

CSC Shift Training and Operation + B904 Status