Online Summary

MICE CM June 2009 Jean-Sebastien Graulich Slide 1

Online SummaryOnline Summary

o Achievements Since CM23

o Control Room

o Controls

o Detector DAQ

o Open Issues

o Summary of the summary

Jean-Sebastien Graulich, Geneva


Achievements since Achievements since CM23CM23

Network in MLCR redesignedNetwork in MLCR redesigned Target DAQ integrated into Detector DAQTarget DAQ integrated into Detector DAQ EPICS Archiver and Alarm Handler EPICS Archiver and Alarm Handler

implementedimplemented EPICS Client Server installed in MLCREPICS Client Server installed in MLCR

First steps toward Unified Control SystemFirst steps toward Unified Control System

Progress in Control and DAQ Interface Progress in Control and DAQ Interface Progress in CAM and DAQ Interface with Progress in CAM and DAQ Interface with

Configuration DataBaseConfiguration DataBase Interface between DAQ and G4MICE Interface between DAQ and G4MICE

improvedimproved Problem with Event building understoodProblem with Event building understood


MLCR Network upgrade

Craig McWaters and Mike Courthold sorted out the chaotic situation

MiceNet: 172.16.246.XXX used for bothDAQ: 172.16.246.01 -> 172.16.246.99 CAM: 172.16.246.100 -> 172.16.246.253

All DAQ machines moved to MiceNet Structure created for all future

machines Strong safety policy enforced

Strong passwords (ask your MOM) Non standard port for ssh No root ssh

+ We have now a printer in MLCR !


EPICS Client / Server Overview

21/04/23 Imperial College 5

Network Status

• Currently have working prototype

– EPICS server connects to PCs via SSH, checks contents of ‘key’ ID file

– Client displays status of all PCs, scans at user-specified period (with ‘check now’ override)

• Need to add service checking & ‘hard’ IOC support

Responsible for System Responsible for EPICS C&M Date Due

Anyone with a PC/IOC in the MLCR/Hall James Leaver (IC) Aug 2009


CAM/DAQ Special Case:

Target DAQ Target data is really both CAM and

DAQ Double flow is implemented Based on a single I/O software -> No issue

with format change

Waiting for the target to be online for final tests

Will require dedicated testing runs


Archiver and Alarm Handler


EPICS Client Server Installed

All client-side applications run on miceecserv Central installation repository greatly simplifies

configuration/maintenance/backup

MOG collates individual applications, applies updates when available from control system ‘owners’

Client control/monitoring GUIs viewed directly on miceecserv, or one of 2 ‘Operator Interface’ PCs

OPI PCs act as ‘dumb terminals’, running displays from miceecserv via SSH

miceecserv miceopi1 miceopi2

EPICS IOC

Portable CA Server

Portable CA Server

EPICS IOC

EPICS IOC

Controls Network

EPICS server applications

EPICS client applications


Unified User Interface

FNAL BPM Sm all

Large

TO F 0

1

2

CKO V 1

2

Tracker 1 D iffuser

Spectrom eter Solenoid

B-F ie ld P robes

AFEIIts

AFEIIt C ryos

KL Calorim eter

Tracker 2 D iffuser


B-F ie ld P robes

AFEIIts

AFEIIt C ryos

H2 Absorbers Focus Coils

Hydrogen System

RF Cavities Coupling C oils

RF System

Drive

Contro ller

Beam Loss

Target

Beam line M agnets

Decay Solenoid

Absorbers

FNAL BPM Sm all

Large

TO F 0

1

2

CKO V 1

2

Tracker 1 D iffuser


B-F ie ld P robes

AFEIIts

AFEIIt C ryos

KL Calorim eter

Tracker 2 D iffuser


B-F ie ld P robes

AFEIIts

AFEIIt C ryos


Hydrogen System


RF System

Drive

Contro ller

Beam Loss

Target

Beam line M agnets

Decay Solenoid

Absorbers

FNAL BPM Sm all

Large

TO F 0

1

2

CKO V 1

2

Tracker 1 D iffuser


B-F ie ld P robes

AFEIIts

AFEIIt C ryos

KL Calorim eter

Tracker 2 D iffuser


B-F ie ld P robes

AFEIIts

AFEIIt C ryos


Hydrogen System


RF System

Drive

Contro ller

Beam Loss

Target

Beam line M agnets

Decay Solenoid

Absorbers

Large wall-mounted display

Alarm Handler

Message log

Client application

launcher

3 Standard desktop monitors

Client GUIConnected to miceecserv

Connected to miceopi1

Connected to miceopi2


C&M Systems OverviewSystem Owner EPICS Developer EPICS Status

Target: Drive Paul Smith; Paul Hodgeson; Chris Booth (UOS) Adrian Oates; Graham Cox (DL) Complete for current Target, but additions required for upgraded system.

Target: Controller Paul Smith (UOS); James Leaver (IC) James Leaver (IC) Not yet commenced.Target: Beam Loss Paul Smith; Paul Hodgeson (UOS); James Leaver (IC) Pierrick Hanlet (IIT) Functionally complete.Beamline Magnets Martin Hughes (RAL) Peter Owens (DL) Complete, with support requirements.Pion Decay Solenoid Mike Courthold (RAL) Adrian Oates; Graham Cox (DL) Complete, with support requirements.FNAL BPMs Alan Bross (FNAL) James Leaver (IC) Complete.TOF Maurizio Bonesini (INFN) Unknown Unallocated.CKOV Lucien Cremaldi; David Sanders (OLEMISS) Pierrick Hanlet (IIT) Not yet commenced.Tracker: Diffuser Wing Lau (OU) Unknown Unallocated.Tracker: Spectrometer Solenoids Steve Virostek (LBNL) Adrian Oates; Graham Cox (DL) DL have completed most of the design drawings, require

~£18K of capital and 0.4 man years of effort to finish the project. Funding has not yet been allocated.

Tracker: B-Field Probes Frank Filthaut (RUN) Frank Filthaut (RUN) Functionally complete.Tracker: AFEIIts Alan Bross (FNAL) James Leaver (IC); Jean-Sebastien

Graulich (UNIGE)Complete, but integration with DATE to be finalised.

Tracker: AFEIIt Infrastructure Alan Bross (FNAL) James Leaver (IC) Thus far, only Wiener PSUs have been identified. Not yet commenced. May have additional requirements.

Calorimeter Virgilio Chimenti (INFN) Unknown Unallocated.H2 Absorbers: Focus Coils Wing Lau (OU) Pierrick Hanlet (IIT) Not yet commenced, manpower unconfirmed.H2 Absorbers: Hydrogen System Yury Ivanyushenkov; Tom Bradshaw (RAL) Adrian Oates; Graham Cox (DL) DL have acquired necessary safety training and have

started evaluating PLC systems. Early stages of development, but fully budgeted for by Tom Bradshaw.

RF Cavities: Coupling Coils Derun Li; Steve Virostek (LBNL) Pierrick Hanlet (IIT) Not yet commenced, manpower unconfirmed.RF Cavities: RF System Andy Moss (ASTeC) Dimity Tettyleman (LBNL); Adrian

Oates; Graham Cox (DL)Early stages of development, but fully budgeted/accounted for by Andy Moss.

DATE Status Jean-Sebastien Graulich (UNIGE) James Leaver (IC); Jean-Sebastien Graulich (UNIGE)

EPICS server + client GUI complete, but client-side integration with DATE required.

Network Status Anyone with a PC/IOC in the MLCR/Hall James Leaver (IC) Prototype complete, requires increased functionality.


Tracker: Spectrometer Solenoids

• Work currently halted due to budget constraints

• 3 options– Allow DL to complete project

• Requires ~£18K capital + 0.4 man years effort

– Take DL’s current design & complete within the collaboration

• Requires ~£18K capital + ~£15.2K vxWorks developer licence + 0.6-0.8 man years effort

• Insufficient MICE manpower available…

– Discard DL’s design & start over within the collaboration

• Unknown capital requirements (likely ~£18K)

• Requires ~1.5 man years effort

• Insufficient MICE manpower available…

Responsible for System Responsible for EPICS C&M Date Due

Steve Virostek (LBNL) Adrian Oates; Graham Cox (DL) Possibly Sep 2009


Detector DAQ Issues

Three priorities were defined in CM23

Limit on Data Size Event Building CAM/DAQ Interface

Limit on Data Size From both DAQ and Monitoring First action on the data size itself ! Simply reducing the number of samples per

fADC channel -> factor of ~5 Requires fADC firmware upgrade First attempt failed


Event building problem

Vassil Verguilov developed software to investigate the problem

The Trigger Time Tag and Bunch ID from the TDC can be used to understand what happened

Each TDC records Trigger Time Tag (TTT)

27 bits 800 ns LSB ~ 100 s full range

Bunch ID (BID) 12 bits 25 ns LSB ~ 50 s full range Clock running Independently

It is possible to retrieve the absolute time of the particle trigger

W.r.t. to board reset (boards not yet synchronized) With some ambiguities… and complications …


CAM/DAQ Interface

Two ways link EPICS should know the DATE status A summary of EPICS data should be inserted

in the online data stream The run should stop automatically when the

CAM goes in severe alarm state

Date Status Monitoring James Leaver has provided

EPICS Server, EPICS Client,Test Software, Status Display

Next step: A small piece of code reading real status from DATE


CAM data Online

Principle re-discussed CAM data appended at the end of the run file

vs CAM data inside the Spill Data

The issue: backward/forward compatibility of data format

Solved by James Leaver An XML format description file will be provided

and appended to the run file Require some “intelligent” unpacking code

CAM data will be stored Spill by Spill(Took just ~4 years to understand that AB was

right…:-)


Trigger IssuesTrigger Issues

No modification since last CMNo modification since last CM The effect of trigger condition choice The effect of trigger condition choice

on TOF calibration has been clearly on TOF calibration has been clearly understoodunderstood

Flexibility has a huge cost ! Flexibility has a huge cost ! TOF Calibration constants depend on the TOF Calibration constants depend on the

trigger sourcetrigger source Do we really need so many trigger conditions Do we really need so many trigger conditions

??

Decision on the future of GVA1Decision on the future of GVA1 It will stay in place at least until Step 2 is

complete


Online SoftwareOnline Software

Unpacking library has been upgradedUnpacking library has been upgraded New fADC (V1731) for CKOV Some bug fixed

Interface with G4MICEInterface with G4MICE Vassil Verguilov has optimized the Date Reader for both

clarity and efficiency Spill structure implemented in G4MICE (for scalar data,

number of particle triggers per DAQ event, etc.) David Adey reproduced online histograms with G4MICE.

Analysis is under way See Software session for more details

Know issues (No change since CM23)Know issues (No change since CM23) Online monitoring GUI tends to get stalled at the end of a

run Need better Online plots for fADC

Current plots found not efficient in detecting broken channels


Front End ElectronicsFront End Electronics

TOFTOF Cabling for TOF TDCs synchronization is finished (Internal

clock and trigger distribution). Will need some dedicated beam time for testing TOF2: We miss some electronics.

Shapers/Splitters: Production finished. Under evaluation TDC: One board missing FADC: 3 boards missing

EMR: Electron Muon Ranger (aka SW)EMR: Electron Muon Ranger (aka SW) Prototype being tested by Michela Prest and Erik Valaza


Schedule Schedule MilestonesMilestones

For DDAQFor DDAQ Target DAQ included in Detector DAQTarget DAQ included in Detector DAQ mid Februarymid February DAQ migration to the DAQ/Control networkDAQ migration to the DAQ/Control network March 06March 06 Installation of the Online Reconstruction FarmInstallation of the Online Reconstruction Farm March 06March 06 Upgrade of fADC firmwareUpgrade of fADC firmware March 06March 06 -> July 09 -> July 09 TOF2 Shaper/Splitter ProductionTOF2 Shaper/Splitter Production mid Marchmid March need need

testingtesting CAM data in Online Data StreamCAM data in Online Data Stream end of April end of April -> June -> June

0909 Tracker integrated in DAQ and OLMTracker integrated in DAQ and OLM end of April end of April -> July -> July

0909 TOF TDC Clock SynchronizationTOF TDC Clock Synchronization end of April end of April -> July 09-> July 09 DAQ reviewDAQ review May 2009May 2009 ->-> June 09June 09 Burst Gate Signal in the Trigger SystemBurst Gate Signal in the Trigger System need beamneed beam

Approval of SW/EMR Front End ElectronicsApproval of SW/EMR Front End Electronics In Proto phaseIn Proto phase Production of SW/EMR Front End ElectronicsProduction of SW/EMR Front End Electronics September 2009September 2009


Items Which Require Action!

• Must find resources within MICE community to complete EPICS C&M systems for– Time of Flight System

– Diffuser

– Calorimeter system

• Must resolve issue of funding for DL’s work on the Spectrometer Solenoids

• PH’s contract expires very soon…– He is essential to success of Online Group

– If he is not reemployed, we won’t have: Alarm Handler, Channel Archiver, remote parameter monitoring, C&M systems for CKOV, Focus Coils, Coupling Coils, etc.


SummarySummary-------------- DDAQ --------------------------------- DDAQ -------------------

Network in MLCR is under ControlNetwork in MLCR is under Control Target DAQ is integrated into online data streamTarget DAQ is integrated into online data stream Interface between DAQ and G4MICE improvedInterface between DAQ and G4MICE improved Problem with Event building understoodProblem with Event building understood DAQ review will happen on Thursday June 4 DAQ review will happen on Thursday June 4

*2009**2009*

-------------- CAM ----------------------------------- CAM --------------------- Infrastructure design has been re-exanimatedInfrastructure design has been re-exanimated EPICS Client Server installed in MLCREPICS Client Server installed in MLCR EPICS Archiver and Alarm Handler implementedEPICS Archiver and Alarm Handler implemented

First steps toward Unified Control SystemFirst steps toward Unified Control System Progress in Control and DAQ InterfaceProgress in Control and DAQ Interface Man power is critical – we should not let go the Man power is critical – we should not let go the

rare & precious expertise we have nowrare & precious expertise we have now

Documents

Online Summary