2008/04/07 - Overview of DOOCS and DAQ

7. April 2008 UCL 2008 London Olaf Hensler DESY-MCS

XFELThe EuropeanX-Ray Laser Project

X-Ray Free-Electron Laser

An Overview ofDOOCS

andDAQ

O. Hensler




Olaf HenslerDESY – MCS

[email protected]

Working in the controls groupfor FLASH and XFEL

mailto:[email protected]




FLASH

Linac is 280 m long up to 1GeV energy 2.5 kA peak current 5 Kryo Modules with eight 9-cell cavities 4 Klystrons to drive Gun and Modules VUV FEL light down to 6nm wavelength Undulator length is almost 30 m shared between Users, XFEL and ILC




3.3 km

HERA

DESY

Hamburg City Centre (7 km)

European X-Ray Free Electron Laser (XFEL)




Accelerator schematic layoutIn

ject

or13

0 M

eVB

C I

500

MeV

BC

II2

GeV

Mai

nLi

nac

10-2

0 G

eV

Und

ulat

ors

Pho

ton

Bea

mlin

es

Col

limat

ion

Dia

gnos

ticF

eedb

ack

Bea

m D

istr

ibut

ion

installed modules 101

installed cavities 808

acc. gradient 23.6 MV/m

installed klystrons 26

RF pulse length 1.4 ms

beam pulse length 0.65 ms

repetition rate 10 Hz

max. average beam power 600 kW




X-ray Free Electron Laser an European project at DESY Linac will 1500 m long up to 25 GeV energy ~100 Kryo Modules with

eight 9-cell cavities each 1 Klystrons to drive 4 Modules 1.3 GHz operation XFEL light down to 0.1 nm wavelength German government announced to spend

50% of the budget == ~350 Mio Euro start of civil engineering in 2008




Some DOOCS features Distributed Object Oriented Control System Client / Server system with ONC-RPC Implemented mainly in C++ Local HDD for archiving and configuration Every server has its local .conf and .log file Hierarchical name space Name calls -> query all locations or properties Most source of a server is inside a central C++ library No central DataBase For Solaris, Linux and Win2000

http://doocs.desy.de

http://doocs.desy.de/




DOOCS features con,t A commen client API for

DOOCS ONC-RPC

TINE

EPICS

On top of this API

DOOCS Data Display DDD

Generic Applications and command line tools

MatLab

LabView

ROOT




DOOCS features con,t

Native Java API : jDOOCS on top of Java

jddd

Alarm & Info System

jDTool ( generic access )




DOOCS architecture• 3 layers• common APIs• modular design• multi protocol

– RPC– TINE– EPICS– Shared Memory

• Device level– ~ 200 server types– Local cfg, arch

• Middle layer– FSM, FB– DAQ

• User applications

– GUI, Web, etc.

http://doocs.desy.de




Nomenclature

FACILITY == Accelerator

DEVICE == Type of a device

LOCATION == position inside the accelerator

PROPERTY == list of properties

Example:

TTF2.VAC/ION_PUMP/1ACC1.WG/STS.ONLINE




DOOCS implementations, tools• API in C++, Java, MATLAB, LABView• SolarisOs, Linux• Multi threaded• Long time archiving (>30000 channels)• Devices (servers > 300)

– synchronized (> 50 VME crates + ~ 60 ICCD & CCD cameras)– not synchronized (CAN, ProfiBus, SEDAC, GPIB, Ethernet etc.)

• ML servers– DAQ (~ 20 Feedback and Monitoring ML servers)– Alarm system– Web Log book

• Applications– DDD, jDDD, jDTool– MATLAB, LabView, ROOT– Electronic Log book

•




DOOCS implementations, toolsDOOCS Data Display (DDD)




DOOCS implementations, tools

Transition from DDD (X-Window and OpenLook) to jDDD (Java)




DOOCS implementations, tools jDDD

view layers

http://jddd.desy.de




DOOCS implementations, toolsjDTool




Concept Overview

XML backup

Alarm and Info server

Device IO

RPC call for device info

XML viaRPC

DOOCS device server

Info

error

status

GUIXML based Alarm and Info Server Architecture

Req / resp.

Dev. Sev. Msg.Time

Dev. Sev. Msg.Time

Fac.Dev.

Loc.Prop.

JMS

JMSServer

XML device history

CentralDOMTree

XML

Collectthread

JNI

Event&session

Queue

ApacheTomcat

JNIServlets

1.2.

3.

jDOOCS(opt.)




The Java GUI

Device Tree

Online Ticker

History




Some pictures : LLRF DSP




Some pictures : LLRF ADC




Some pictures : Diagnostic




Some Pictures : CPU

500 Mhz SPARC CPU

512 MB RAM

SCSI HD Interface

no fan

PMC

1 VME Slot

Solaris8




Some pictures : HardDisk

SCSI HardDisk

5V und 12V from VME Bus

2 VME Slots needed




Fast ADC

Build at DESY-Zeuthen

8 Channels

14Bit ADC

10Mhz

parallel clocked

memory for 16 bunches

170 items at FLASH




Some pictures : Industry Pack

IP-Stepper

IP-DIGI IO

IP-Timer

IP-ADC




DOOCS server design idea

DOOCS modulates a certain hardware into a C++ class

This means : the class describes the properties of one hardware device

Example : DOOCS writes software for one ion-pump





read

convert

Voltage

archive

read

convert

Pressure

archive

read

convert

Voltage

archive

read

convert

Pressure

archive

Powersupply

HV I

Powersupply

HV I

Ion pump at location A Ion pump at location B

...

Ethernet

HardwareDevice

DOOCSServer

To program

classes ofServer lib

Location





does all data taking, calculations, process logic and storing independent from the network, cause of local harddisk for

operating system server binaries configuration files archiving

At startup all objects, locations are created from the configuration file Two main classes

EqFct class => the container class for the location D_fct classes => types for the properties

Most functionality is inside a central server library easy to extend good debugged/tested

The server :





Ser

ver

Pro

gram

Ethernet

Archive Configuration

VME access

Hardware I/O

Enet, ProfiBus, CAN, Sedac

Overload

Specific

Ethernet layer TCP/IP

RPC/XDR (Remote ProcedureCalls,External Data Rep.)Server RPC Interface

Address and Data handling

Equipment Functions standard and

Data Functions (D_float, D_int, D_hist ...

DO

OC

S c

lass

esO

S TINE calls





_SVR 1BC1 2BC2 UND1

List of EqFct == locations

List

of D

_fct

==

pro

pert

ies

SVR.NAME

ERROR

Log

Rate

STS

NAME

ERROR

SIN

Value

STS

NAME

ERROR

SIN

Value

STS

NAME

ERROR

SIN

Value

STS

Standard Properties derivedfrom the base class EqFct

Properties added toEqFct<MY>





MemoryBuffer

.FILT

RingBuffer

.EGU

.DESC

D_hist class

Value

set




Timing Sequence FLASH

} Events

Injector Laser




Fast ADC

ADC

ADC

TIMER

SH

AR

ED

ME

MO

RYDMA

TIMER DRIVER

SCOPE_SERVER

BPM_SERVER

RF_SERVER

CH_DATA

DAQ

VM

E B

usVME Crate

Trigger

DMA_SERVER




Shared Memory

FAST ADC CHANNELS

1 2 NBUF INFO NEXT ADC

CURRENT BUFFERSHM INFO NEXT BUF






DAQ integrationinterrupt_usr1( )TIMER

LOC THREADD_adcdaq

FILL_BUF()

DEVICE LOCDEVICE DATA

SENDER LOCEqSender

CHECK DAQ EVENT

SEND BUF

READY

BUF PAR

LOC THREADD_adcdaq

FILL_BUF()

DEVICE LOCDEVICE DATA

A6




ATCA

ATCA == Advanced Telecom Computer Architecture

probably new crate standard XFEL

serial, switched bus (GB Enet, PCI-x)

redundant, monitored 48V powersupply

standardised monitoring and management (IPMI)

complete redundant setup possible

no IO from industry yet

too expensive for now




Hardware: xTCATiming receiver

board size




Future Shelfs : ATCA




FLASH DAQ Design

BM

FCFC SC

FCDS FCML

OPERATOR GUI

DAQ Server

EVB

dCache

LOCAL GUI

REMOTE GUI

Storage

RC

DB

RC GUI

multicast

Fast data (every shot)Beam relevant measurements: ADCs (BPM, BLM, TOR, etc) CAMERAs

DOOCS(TINE)

Slow data (max 1Hz) Data from slow ADCs(MAG, V, etc.) DOOCS channels (Masks, params, etc.)

45 days of linac data+ GMD , Exp Data

LINAC

FAST

ADC

IMAGE SLOW

ADC

DISK

DCCP DISKROOT

data streams

OracleXML files

48 hours of linac data




FLASH DAQ Design

DISK

dCache

Storage

BM

FC SC

DS FCML

OPERATOR GUI

DAQ Server

EVB

LOCALE GUI

REMOTE GUI

RC SA

RC SA GUI

BM

FC SC

DS ML

OPERATOR GUIEVB

LOCALE GUI

REMOTE GUI

RC

DB

RC GUI

DAQ for LINACDAQ for experiments

FAST

ADC

IMAGE SLOW

ADC

LINAC

FAST

ADC

IMAGE SLOWADC

Experiment




FLASH DAQ Design

8 double core SPARC CPUs32 GB common memory6 (3used by DAQ) x 1Gbit Ethernet1.7 TB local storagefully redundant fan, PS, …

• 2 dual-core AMD Opteron processors 2.6 GHz• 16 GB common memory• 24 TB local storage; 48 SATA HDD • 4 x 1Gbit Ethernet• 1 GBps from disks to network • 2 GBps from disk to memory

DAQ server hardware




Experiments at FLASH

Hardware (custom)

C1class

C2class

C3class

C4class

C5class

C6class

Basic Camera Class

USB FireWire Ethernet FrameGrabber

DOOCSCamera

Server

DAQ

Andor PCO

Acqiris DC282 cPCI High-Speed 10-bit PXI/CompactPCI 4-ch, 2-8 GS/s,256-1024 kpt, 32-128 Mptor 256-1024 Mpt




Experiments at FLASH

Control, monitoring Beam diagnostics panels

DDD panels MATLAB panels

Experiment panels DDD panels




Hints for a hardware design

try to find a one to one relation between hardware and software

try to define an independent device

allow to read back all register, avoid write only

Engineering

2008/04/07 - Overview of DOOCS and DAQ