SHINE Control System Design and Development

Yingbing YanAccelerator Control Group

Shanghai Synchrotron Radiation Facility (SSRF)

Shanghai soft Xray Free Electron Laser (SXFEL)

Shanghai HIgh repetition rate XFEL aNd Extreme light facility (SHINE)

Shanghai Advanced Research Institute

Chinese Academy of Sciences

Projects Introduction Accelerator Control System Device Control LLRF of SC Linac Data Acquisition Machine Protection Timing System Fast Feedback System Runtime Environment High Level Software Multi-function Testing Platform Summary

Outline

Shanghai Synchrotron Radiation Facility (SSRF) One of the advanced third generation light sources in the world Support and push the cutting-edge scientific research and innovation in China 3.5 GeV, 432 m, ~ 3 nm·rad, 260 mA, Top-Up Open since May 2009 7+1 Phase-I beamlines 7 dedicated beamlines 16 Phase-II beamlines (coming soon)

SSRF

The construction of the Shanghai soft Xray Free Electron Laser Test Facility (SXFEL-TF) started at the end of

2014. Its accelerator tunnel and klystron gallery were ready for equipment installation in April 2016. The

installation was completed by the end of 2016.

The SXFEL-UF, with a designated wavelength in the water window region, began construction in November

2016. It was based on upgrading the Linac energy to 1.5 GeV, and the building of a second undulator line

and five experimental end-stations.

SXFEL

J. Ullrich, A. Rudenko, etc. Ann. Rev. Phys. Chem. 63, 635 (2012) * Carried out jointly by DICP (experimental end stations) and SINAP (Linac + FEL)

SDUV-FEL DCLS * SXFEL SXFEL

Test facility User facility Test facility User facility

Status Operating Operating Commisioning Construction

Wavelength 150-350 nm 50-150 nm 8.8 nm 2.0 nm

Length 65 m 150 m 300 m 540 m

Accelerator S band S band S+C band S+C band

Beam energy 100-200 MeV 300 MeV 840 MeV 1.5 GeV

FEL Principle HGHG, EEHG HGHG HGHG, EEHG SASE

Location Shanghai Dalian Shanghai Shanghai

Shanghai HIgh repetition rate XFEL aNd Extreme light facility (SHINE)

SHINE

e-beam : 8 GeV Photon Energy : 0.4～25 keV Pulse duration : 1～100fs Repetition : 1MHz Total length : 3.1km Underground ~ 29m

The control system will provide operators, engineers and physicists with a comprehensive and easy-to-

use tool to control the machine components to produce high quality electron beam and free electron

laser.

According to the experience of SSRF and SXFEL, the control system of SHINE will be mainly based on

EPICS to reach the balance between the high performance and costs of maintenance.

More large scale, more hardware devices

High repetition rate, high data throughput

Time-stamps timing and feedback capabilities

Powerful data acquisition, processing and storage

High reliability, stability and maintainability

Complicated system integration and management

Accelerator Control System

Device Control

Data Acquisition

Machine Protection

Timing System

Fast Feedback System

Network

Server Cluster

Data Storage

User Interface

High Level Software

Central Control Rooms

···

Accelerator Control System

Controlled Objects ～9000, Embedded IOCs ～1200

- SXFEL-TF ～700, Embedded IOC ～60

EPICS PVs ～1M (Estimated 100 PVs per device)

- SXFEL-TF ～100K

Control Network Switches ～370, including dedicated switches ～230

- SXFEL-TF ～10 + 40; SSRF ～30 + 10

Serial Port Servers ～200

Machine Protection Signals ~ 6000

- SXFEL-TF～1000; SSRF ～3000

Cryomodule Cryogenic Local Control Signals ~ 6000

Timing Signals ~ 1200, Nodes ~ 600

- SXFEL-TF～55; SXFEL-UF～140

Accelerator Control System

It is responsible for the device control and monitoring of magnet power supply, vacuum gauge, ion pump,

stepper motor, equipment cabinet and so on. The IOCs hardware must be COTS (Commercial Off-The-Shelf) products with the features of high

performance and cost effective. The fanless industrial embedded controllers will be the best option. The IOCs connect the local controllers via backplane bus, field buses or Ethernet. By using the protocol

converters, the devices with field buses such as RS-232/RS-485 can be reached via Ethernet. Several dedicated network will be constructed.

Device Control

RF station : LLRF, SSA, Circulator, Motor driver, Piezo driver cavity

SSA : 5.2kW@1.3GHz

LLRF: based on ATCA or Stand-alone architecture

ATCA board layout

LLRF of SC Linac Courtesy of Prof. Yubin Zhao of RF Group

LLRF signal flow

LLRF of SC Linac Courtesy of Prof. Yubin Zhao of RF Group

Module layout

Front view Lateral view

LLRF of SC Linac Courtesy of Prof. Yubin Zhao of RF Group

Image acquisition for beam profile measurement and laser diagnostics system

Waveform data acquisition for beam instrumentation system (position, charge, length, ... )

General LXI devices (oscilloscope and spectrum analyzer)

Due to high repetition rate, the high data throughput of data acquisition will be challenges

Data Acquisition

For the majority of low fps image acquisition, the GigE Vision cameras will be adopt.

The dedicated network will be employed to transmit the images to the servers. The

software will based on the areaDetector and aravisGigE modules.

For the minority of high fps image acquisition, the Camera Link cameras will be

adopt. The software mainly focus on the image acquisition and storage. The image

processing and analysis will be completed offline.

Data Acquisition The Camera Link range extender can be used to extend the data transmission distance over optical fiber links.

The ZeroMQ and HDF5 technologies are being evaluated for image transmission and storage.

ZeroMQ is a high-performance asynchronous messaging library, aimed at use in distributed or concurrent

applications. It provides a message queue, but unlike message-oriented middleware, a ZeroMQ system can

run without a dedicated message broker.

The system is designed to protect the important machine components from damage when serious abnormal

situations occur.

The master node receives and summarizes the signals from slave nodes of the injector, superconducting

Linac, undulators, beamlines and experimental stations.

The slave nodes control the executive devices, such as the vacuum valve and so on.

Response time: 20/100ms | 10/100us

Machine Protection System (MPS)

PXI Digital Reconfigurable I/O Module - PXIe‑7822, Kintex 7 325T FPGA, 128 DIO, 512 MB DRAM

PXI Multifunction Reconfigurable I/O Module - PXIe‑7857, Kintex 7 160T FPGA, 8AI, 8AO, 48DIO

CompactRIO Controller - cRIO‑9039, features a FPGA and a real-time processor running Linux RTOS

Yokogawa Programmable High Speed I/O Modules

Machine Protection System (MPS)

PXIe‑7822 PXIe‑7857

Provides trigger and time-stamps (or Bunch-ID)

White Rabbit technology will be evaluated

Fully deterministic Ethernet-based network for general

purpose data transfer and synchronization

sub-nanosecond accuracy and picoseconds precision

connecting thousands of nodes

typical distances of 10 km between nodes

Ethernet-based gigabit rate reliable data transfer

fully open hardware, firmware and software

multi-vendor commercially produced hardware

Timing System

White Rabbit Trigger Distribution

White Rabbit RF signals Distribution

FMC Node : Trigger + Bunch-ID

FANOUT Node : Trigger

FMC WR Node (Planned)

Timing System

The feasibility of White Rabbit technology for fast feedback system

will be evaluated.

Fast feedback system based on White Rabbit technology can take

advantage of clock synchronization and data transmission

The system architecture is simple, extensible and time determined.

Fast Feedback System

EPICS Services: include the base, support modules, extensions, cross-

compile support for different platforms. All console computers register

on the server and use the shared resources.

Network file service, directory services, archive services, time

synchronization service, web service, software version control services,

syslog service and so on.

The most services run on the virtual servers, which are highly scalable

and available servers built on the cluster of physics servers. The

architecture of the server cluster is fully transparent to the users.

Software development process management

Process variables information and access security

Network online management and monitoring

Runtime Environment

It is composed of three layers: database, service and application. It can

simplify the application development, offload the heavy database

query and shield from the database structure changes.

The database is a general data storage container for the machine

configuration, lattice, state, alarm and so on.

The service implements a series of control and physical related logic

operations, and provides standard interfaces for the application layer,

including local interfaces and remote interfaces.

The application layer consists of kinds of tools and components used to

display the information to users. It accesses the database through the

service layer.

doi:10.1016/j.nima.2018.08.047

High Level Software

Architecture of the high level application

Motivation:

Saving parameter data in relational database for

physicists and researchers

Easy to track the modified history

Architecture:

Based on standard J2EE Glassfish platform with

MySQL database as backend data storage

Database - general data storage container

Business layer - handled by enterprise beans

Web layer - JavaServer Faces and services

Client layer - PC, Mobile phone and other

Database Schema:

Built a universal database schema

Saving properties as Name/Value pairs

High Level SoftwareWeb-based Parameter Management System

Lifespan of the tuner at low temperature

Vacuum leakage rate and the electrical performance of the cold BPM at low temperature

Working stability of the superconducting magnet

Current lead at low temperature

Heat load in each temperature zone

Vacuum leakage rate of the coupler at the low temperature

Cold BPM Couple 2420 × 2114 × 2340mm

Multi-function Testing Platform

Dell PowerEdge R740xd

MOXA DA682B

Yokogawa PLC FA-M3

EPICS V3.16.2

Asyn 4.33

StreamDevice 2.8

netDev 3.3

procServControl 1.9

···

CSS Display Builder

Archiver Appliance

CSS ALarm

Multi-function Testing Platform

The first hard X-ray FEL light source in China, the so-called Shanghai HIgh repetition rate XFEL aNd Extreme

light facility (SHINE), is under construction.

The control system is responsible for the facility-wide device control, data acquisition, machine protection,

timing, high level applications as well as network and computing platform.

The high repetition rate and the high data throughput will be great challenges. The latest hardware and

software technologies need be adopted.

Some research projects have been approved, including system development platform, high frame rate

image acquisition, machine protection system, timing and feedback system.

Technical discussions and cooperation are welcome.

Summary

Thanks for your attention !