IPHC - DRS Gilles CLAUS 04/04/20061/20 EUDET JRA1 Meeting, April 2006 MAPS Test & DAQ Strasbourg OUTLINE Summary of MimoStar 2 Workshop CCMOS DAQ Status

04/04/2006 1/20IPHC - DRS Gilles CLAUS

EUDET JRA1 Meeting, April 2006MAPS Test & DAQ Strasbourg

OUTLINE

Summary of MimoStar 2 Workshop

CCMOS DAQ Status

Ideas about Demonstrator DAQ

MimoStar 2

USB Imager Board

04/04/2006 2/20EUDET Meeting, Geneva April 2006 IPHC - DRS Gilles CLAUS


MimoStar 2 Slow Control&

CCMOS DAQ

Strasbourg 14-17 March 2006



One Common Session ( 1 day )

MimoStar 2 readout : Digital control sequence – Analogue frame

MAPS group USB DAQ and JTAG software

Two Parallel Sessions ( 2 days )

MAPS test and calibration procedure

Software Development Kits : JTAG and USB DAQ

At the end « Christmas-Box »

2 USB Imager boards + Software + Documentation

3 MimoStar 2 chips calibrated

SDK DAQ & JTAG + Template Application + All Source code + Documentation



How ?

MimoStar Test Bench


CCMOS DAQ Status

Why a DAQ status report ?

We have done tests in order to

Check if USB Imager board can be use in demonstrator DAQ

Find current event rate of our DAQ in demonstrator configuration

Evaluate maximum event rate we may be able to reach

We should think about Imager board integration in global DAQ

TLU

Planning for next weeks


CCMOS DAQ Status

USB Bandwidth : Imager Board RAM / PC RAM

One Imager Board ~ 13 MB/s => Dead Time on PC side

Parallel readout of 6 Boards => ~ 60 MB/s ( 1 PC - 6 USB links - 2 USB controllers )

Interrupt Handling

HW : PC Parallel port ACK line

SW : Commercial driver => at 10 KHz Less than 0,1 % IRQ lost

USB Overheads …

Data transfer synchronized to 8 KHz clock …

Not efficient for single register access 125 µS, 250 µS … !

Board « event control » can cost more than pixels readout for small event size …

Board event control = few bits => We can use parallel port => 1- 2 µS


CCMOS DAQ Status

Simulation of 6 MimoStar 3 readout

6 USB Imager Boards in DAQ system

One MimoStar 3 @ 10 Mhz / Board

CDS on board - 12 bits – 1 W16 / Pixel

256 x 256 pixels / Board = 128 KB / Event

Free running system ( self triggered ) => Event rate ~ 40 Hz


More than 40 Hz ?

Maximal event rate on DAQ side ?

If on board zero suppression is ready ? When ???

Read 10 % of total pixels number => ~ 100 Hz

Read 1 % of total pixel number => ~ 250 Hz

Maximal event rate MimoStar 3 M side ?

Sync + 2 Frames @ 10 MHz => 3,2 mS => ~300 Hz

No « Continuous » readout ( 600 Hz ) for demonstrator

From 300 Hz to KHz level ?

Single board for all planes

Events storage on board => Firmware development

Not for demonstrator

CCMOS DAQ Status


CCMOS DAQ Status

Trigger Handling

Interface from TLU / USB Imager board

Trigger input from TLU

Reset input from TLU

Busy output to TLU : Set on trigger reception – Reset when ready to next event

Interrupt request output to PC

Conclusion No interface problem seen after first check

but it must be confirmed.

All inputs / outputs are in NIM standard.


CCMOS DAQ Status

To Do List for next months …

We must move our Telescope DAQ from VME to USB

On board CDS is implemented => Must be Tested

Trigger handling is implemented => Must be Tested

6 Imager Boards synchronization - Master / Slave => Must be implemented & tested

On board zero suppression will not be implemented in 2006

Software interface to implement « Geneva CPU zero suppression » may be this year ?



Why Ideas about Demonstrator DAQ ?

A DAQ development is on the way in Strasbourg

For our Silicon Strip reference Telescope ( DUT = MAPS )

We have a short time scale 12/2006 ( Shorter than demonstrator )

It will be the same technology, if Imager board is used for demonstrator

Short time scale – Reduce development time – Modularity

Have quickly a running system => To test hardware side

Pragmatic DAQ Ideas = Not state of the art DAQ … but do we need it ?

Show our ideas to solve this problem …

Reduce development time

Keep modularity

Open system – Upgrade possible

Normally it’s not possibleReduce development time

=> less modularity=> close system



Moving Our DAQ Systems from VME to USB

Current system : VME – LynxOS – Linux ( 2001 – 2006 )

VME Sequencer & ADC boards : « VME Imager »

DAQ : VME Crate - RIO 2 CPU Board – Processor PPC 400 MHz – LynxOS

Control & Monitoring : PC – Linux – LabView ( GUI ) – ROOT ( Monitoring )

New system : USB 2.0 – Windows ( 2007 … )

USB 2.0 Sequencer & ADC boards : « USB Imager »

DAQ : PC – Windows – DAQ Application ( C++ Builder )

Monitoring : PC – Windows – ROOT

Why ?

VME to USB 2.0 => Reduce system cost : CPU Board, Operating system license

Linux To Windows => Reduce PC installation cost « System engineer job »

But Windows can increase development cost / Linux => “ Think in a different way ”



“ Thinking in a different way ”… a list of ideas … in order to avoid to fight with Windows

Use system programming only when it is absolutely required

Each time it is possible

Use simple implementation, not state of the art programming for fun only …

Don’t “ play to software engineer ” for fun only … ( I will try, but can promise ;-)

Avoid to use “ Windows Technologies – DDE - OLE ” … if you decide to use them

Don’t use direct calls : build an interface library

If they modify or stop this technology … you will only need to modify your library

When it is possible … “ try to buy it ” …try to find a company who has already done the job



First example Remote Monitoring Protocol

Windows USB DAQ system ( CCMOS ) ready, since the end of 2004

On-line Monitoring using ROOT available BUT under Linux DAQ

No time to port on-line monitoring from Linux to Windows

Build a “ bridge ” between Windows & Linux … Ready in January 2005

Without any system programming : files exchange on a common network drive

It works … it is more robust than “ LynxOS / Linux version ”

It costs few development time …

Conclusion

We will try this RMP on our Beam Telescope USB DAQ



Second example CCMOS DAQ Upgrade for EUDET

We must read 6 boards in parallel

We must be able to handle interrupt requests

In this case System programming Is Required

Multi threading

Interrupt handler

It has cost development time, but there was no other way

We plan to buy // port interrupt handler ( with source code )

Conclusion

This is the “ heart ” of DAQ … In this case learning more about Windows is not loosing time



How to synchronize MAPS, DUT, … DAQ ? Remote Run Control Protocol ( RRCP )

Each DAQ is controlled either

In local, GUI acts as a master => Stand alone application in lab or User Telescope

By a commands interpreter, GUI acts as a slave => EUDET Beam Telescope

Each DAQ handles his configuration files ( stored in a working directory )

How it works ?

EUDET Master Run Control application ( MRC )

Copy configurations in each DAQ working directory

Send command Start, Stop … with parameters by RRCP ( write command file )

Slave DAQ applications

Interpreter receive commands by RRCP ( read command file in polling )

Execute command

Return status file to MRC ( write status file )



Remote Run Control Protocol ... RRCP ?

This RRCP is not a genial idea ;-)

It’s the standard way to control DAQ systems

But implementation can be done in a basic way

The implementation

Using command and status files on a shared network drive

Command files from Master Run Control to slave DAQ

Status files from slave DAQ to Master Run Control

Checking command / status files by slow polling – remove file after processing

A library provides RRCP function calls and files format

Only need to add Remote Control & Status report features on each DAQ software

Each team can maintain his own DAQ software


One proposition for Demonstrator DAQ

1 Plane High Resolution

6 Imager Boards

6 Planes Mi3M

Config Files

DUT

DAQ

1 or 2 Imager Boards

USB

PC LocalStorage

Data ~ 30 MB/s

40 Events / s

Data ~ 20 MB/s

40 Events / s

DUT

USB

PC LocalStorage

PC LocalStorage

Off-line Analysis

Master Run Control & On-line Analysis

Status FilesCommand Files

Monitoring Files

Ethernet Link

Monitoring 10 % ~ 3 MB/s

Monitoring 10 % ~ 2 MB/s

Monitoring

Data

Ethernet Link

Shared RRCP Disk Shared RMP Disk



Conclusion

Remote Run Control & Remote Monitoring Protocols = Top layer + 2 implementations

First - basic version with files : easy to program & debug

Second - with standard network Protocols : RPC, Socket pipe …

No need to develop a global DAQ system Application

Master Run Control application

Each DAQ is independent BUT can be controlled as a slave system

Allow to use the same DAQ in User or EUDET context

It will be easier for DAQ upgrade => few modifications of MRC

On-line Monitoring and Off-line analysis

Provide libraries for event building (EVB) from all Data Sources ( MAPS, DUT … )

The same EVB libraries should be used for On-line & Off-line analysis


Two Ideas to build Telescope global DAQ

Idea N° 1 : DAQ HW API & Global DAQ Application

Each group provides HW API libraries ( eg : USB Board SDK, JTAG SDK )

Need a global DAQ Application ( “ Active Application ” : Real Time, SDR, EVB …)

Idea N° 2 : Slave DAQ & Master Run Control Application

Each group implements Slave Remote Control in his DAQ Application

Need a Master Run Control Application ( “ Passive Application ”: Control, Supervision )

Not so far ? from Peter Fischer proposition “ Option 3 : Integration on data level ”

DAQ decoupled, less system programming – Upgrade with Peter ideas possible

Idea N°2 … How to do : EVent Building ( EVB ) & Software Data Reduction ( SDR ) ???

Pseudo on-line EVB from data files

SDR ? in each DAQ Application => 2 Outputs : RAW ( keep as reference ) + after DR

SDR ? after final event building

Documents

IPHC - DRS Gilles CLAUS 04/04/20061/20 EUDET JRA1 Meeting, April 2006 MAPS Test & DAQ Strasbourg OUTLINE Summary of MimoStar 2 Workshop CCMOS DAQ Status