22 October 20151/23Fabio Vinagre -- TileCal DCS TileCal DCS Expert tutorial - DCS 22 October 2015

April 20, 2023 1/23Fabio Vinagre -- TileCal DCS

TileCal DCS

Expert tutorial - DCS

April 20, 2023


Summary

DCS/FSM structure

How to Login to DCS machines

Where to look after login

Critical PVSS managers

RDB, OPC and DIM common problems

How to start devices from Windows Terminal Server (WTS)

Start and Stop FSM

How to change alarms thresholds

Start a PVSS project

Documentation and Questions


DCS/FSM Structure

PCATLTILSCSPCATLTILSCS

PCATLTILLV00PCATLTILLV00 PCATLTILLV01PCATLTILLV01 PCATLTILLV02PCATLTILLV02 PCATLTILLV03PCATLTILLV03 InfrastructureInfrastructure

CAN PSUCAN PSUCOOLING

PLANTCOOLING

PLANT

RACKSRACKS

LASERLASER

PartitionPartition

COOLING(1)COOLING(1)

VME Crates(1)VME Crates(1)

DDCDDC

200V PS(1)200V PS(1)

AUX BoardsAUX Boards

Branch 1Branch 1

Branch 2Branch 2

Branch 3Branch 3

Branch 4Branch 4

Module 9Module 9

Module 10Module 10

….….

Module 23Module 23

Module 24Module 24

LVPS 9LVPS 9

Drawer 9Drawer 9

ROD crateROD crate

TTC crateTTC crate

EBA LBA EBCLBC

XXX

PCATLTIL…PCATLTIL…

► Partition

► Machine

XXXXXXXXXXXX ► LU or DU(1) –controlled by SCS machine


First one have to connect to the windows terminal server (point 1 gateway):If you are in CERN GPN:1.Start the remote desktop application and connect to cerntsatldcs;2.Introduce your NICE user name and password

If you are outside of CERN GPN1.First remote desktop to cernts.cern.ch and then to cerntsatldcs;2.Introduce your NICE user name and password

3.Click in Ask Access and type in a Reason (be as clear as possible)4.Wait for the Atlas Control Room response5.When the access is granted the Ask Access button changes to Allowed and the grey buttons became active6.Click on Remote Desktop, enter the machine name (previous slide) and your NICE credentials or choose the

panel and click Run

How to login to DCS machines (1)


How to login to DCS machines (2)

Possible problems:

I cannot login to one machine›Check if you typed the machine name correctly›The login domain have to be CERN, for example: CERN\fvinagre›Do you have the correct permissions? Check before On call shift and contact a DCS admin to obtain authorization. The DCS admins are: Giorgi Arabidze, Fabio Vinagre, Pedro Almeida and Stanislav Nemecek

I am logged in but there is no project running›The project runs as SYSTEM so you have to click on the Start Menu and open PVSS console

There is a user already logged in›Do not proceed with login before contact the user and see if he doesn’t need the machine any more. If you proceed you will log him off and he will lose all the not saved work.

Advice:After login also start the Windows task manager (right click in the Windows Task Bar and click in Task Manager)


What to do after login

After login and open PVSS console the first thing one should do is to check if the critical managers are running:

In LCS one should confirm if:›1 OPC manager, number 7›1 RDB manager , number 99›2 PVSS00dim, one number 8 and the other any number

In SCS one should confirm if:›1 RDB manager , number 99›2 ModBus manager, number 15 and 2›2 OPC manager, number 7 and 14›6 PVSS00dim, number 2, 3, 4, 5, 9 and any number


What each critical manager do?

Manager Manager Number (machine) Description

RDB 99 (all machines) Oracle DB

OPC 7 (all except SCS) fLVPS ELMBs and AUXboard

PVSS00DIM HV (all except SCS) DIM server (Drawer)

PVSS00DIM 8 (number 2 in SCS) DDC

OPC 14 (SCS) Wieners crates

ModBus 15 (SCS) CaV

ModBus 2 (SCS) 200 V bulk PS


RDB Problems

How to know if there are RDB problems?›Plots are not displaying any data›Alarm screen with “Disk buffer number is already 20” message

How to solve it?›Log in the machine with the RDB problems›Use the Tile_DCS_RDB.PDF as a step-by-step guide

If problem not fixed contact the DCS admin (first) or ATLAS central DCS


OPC Problems

How to know if there are OPC problems?›Corresponding data is not being updated›Alarm screen with “TIL <PART> OPC Server Connection” message

How to solve it?›Log in the machine with the OPC problems›Check if all OPCs managers are running (see table in previous slide)›Use the Tile_DCS_OPC.PDF as a step-by-step guide



DIM Problems

How to know if there are DIM problems?›DAQ not receiving/sending information (DIM number 8)›Drawer data is not being updated for 10 minutes the On-Call phone will receive one SMS›Blinking red/gray drawer in FSM

How to solve it?›Log in the machine with the DIM problems›Check if DIM managers are running (see table in previous slide)›If manager is running:

›Use the Tile_DCS_DIM.PDF as a step-by-step guide ›If manager is not running

›Change the start mode properties to Always›Wait ~5 minutes›If data is not being updated use the Tile_DCS_DIM.PDF as a step-by-step guide



Start tripped fLVPS

There are 3 types of LVPS trips:

1.The MB side trip (most common)I. On call shifter should start LVPS from FSM or instruct shifter to do itII. After the operation the ACR shifter have to call DAQ expert

2.The HV side tripI. On call shifter should start HV side from WTS

3.Hole LVPS trip (rare)I. Same procedure as MB side trip

Use the Tile_DCS_LVPS_Trip.PDF as a step-by-step guide


Tripped fLVPS examples

Use the Tile_DCS_LVPS_Trip.PDF as a step-by-step guide

HV side tripHV side trip MB side tripMB side trip


Start HV from WTS


Start/Stop FSM (1)

If you see the FSM with the STATE DEAD in a grey background it means the FSM is stopped.One can start the FSM using the WTS or doing remote desktop to the corresponding DCS machine


Start/Stop FSM (2)

To start the FSM one have to use the Device Editor Navigator.To start the Device Editor Navigator:


Start/Stop FSM (3)

How to start the FSM:1.Stop All2.Check in Windows task manager if any smi.exe or PVSS00smi.exe process is running, if yes, kill it3.Start/Restart All4.After the restart the FSM state should be with a grey background, this means the Tree is not taken by anyone. Leave it like this so the central DCS desk can take control of the Tile Tree

In case the top node (TIL) is DEAD one should perform this steps in the SCS machine


Change Drawer thresholds (1)

If you have a Drawer in Warning or Error state and the shifter already reported as stable one can increase the range of the Voltage or Temperature.

From WTS:


Change Drawer thresholds (2)

Confirm if you open the correct Module


Change LVPS temp. thresholds (1)

To change the LVPS alarm range one have to remote desktop to the machine in question, open the gedi (PVSS graphical editor) and click on PARA module

1

2

3


Change LVPS temp. thresholds (2)

1

2,4

3

Check if you are working in the correct partition

Check if you are working in the correct partition


Start a PVSS project (1)

After an unexpected reboot/shutdown the DCS machine and the PVSS project should start automatically

If that does not happen one should see the FSM with a purple color


Start a PVSS project (2)

To start a PVSS project one have to remote desktop to the corresponding DCS machineIs VERY important to check if the PVSS00pmon.exe process is working under SYSTEM, if not go to Windows services and start it

After the PVSS00pmon.exe is running as SYSTEM one can check if the there is no running process like:›AtlTileHv.exe›CanOpen25+.exe›PVSS00opc.exe›PVSS00ctr.exe›PVSS00mod.exe›PVSS00smi.exe ›smi.exeAfter all this checks is safe to start the PVSS project using the traffic lights in PVSS console

When all is running, start the DIM server (AtlTileHv.exe) from the windows services.After ~5 minutes check if the HV and LV values are being updated.


Documentation and Questions

Tile DCS wiki ›https://twiki.cern.ch/twiki/bin/view/Atlas/TileCalorimeterDCS

FSM how to›https://twiki.cern.ch/twiki/bin/view/Atlas/TileCalorimeterDCS under Attachments

Tile_DCS_RDB.PDF, Tile_DCS_OPC.PDF, Tile_DCS_DIM.PDF and Tile_DCS_LVPS_Trip.PDF›https://twiki.cern.ch/twiki/bin/view/Atlas/TileCalorimeterDCS under Attachments

LV documentation›http://cdsweb.cern.ch/record/1228921

Questions?

Documents

22 October 20151/23Fabio Vinagre -- TileCal DCS TileCal DCS Expert tutorial - DCS 22 October 2015