Jeff Hill
LANSCE Requirements – a ReviewEPICS Paradigm ShiftMagic PipesData Access, is it Easy CA?Database CA ServiceServer UpgradesOn How We Can Move Forward,
IMHOConclusion
• LANSCE, a versatile machine– Originally producing H+, H-, and polarized H-
• Each with different intensities, duty factors, and even energies – depending on experimental and medical isotope production needs
• LANSCE timing and flavoring of data– Flavoring
• Selection based on - logical combinatorial of beam gates
– Timing • Selection based on - time window sampling
• Many permutations – Too many to, a-priori, install records for all of them– Subscription update filtering is needed
Distributed Control
Data Acquisitio
n
Physics Control
Open Source
Vendor Neutral
OS Neutra
l
Small Footprint
EPICS EPICS
What is a Data Acquisition System? Replacing …
What is a Data Acquisition System? Must efficiently filter, and archive,
copious amounts of data ▪ selecting interesting occurrences
▪ Saving them for later detailed processing/evaluation
Must be easily reconfigurable
What is a Distributed Data Acquisition System? Must be runtime reconfigurable by clients
▪ Don’t expect to know initially, ▪ When designing/compiling runtime system▪ What experiments/filters might be devised later on
▪ Experiments/filters configured when client subscribes▪ Don’t expect to know initially▪ When designing/compiling runtime system▪ What data aggregations will be on different data
branches to different clients
Current weaknesses deploying EPICS into Data Acquisition situations? Record processing provides good flexibility
to create event filters, but …▪ Frequently, it isn't possible to know all of the
experiments when the IOC’s database is designed▪ A distributed data acquisition system needs
runtime reconfiguration, initiated by client side tools
▪ Limited data model▪ No runtime aggregation, or user defined types
Current weaknesses deploying EPICS into Data Acquisition situations? No support for site specific tagging of
the data▪ If a site needs to filter for LANSCE H- beam
▪ Filtering based on process control attributes such as the time-stamp is awkward
▪ Filtering based on site specific parasitic PV attribute data (the LANSCE flavor) leads to better structured control room applications
Before EPICS, a process control system by
design, sometimes used for data acquisition
Now EPICS, a process control and data
acquisition system by design Not an upgrade, but a leap forward in
terms of the general utility of the system
Alarm State
PV ValueSignal Data
Time Stamp
DeviceSupport
RecordSupport
DBCommon
CA Server
Record Specific ValuesDevice Specific Values
Issues transporting data through software layers Independency Data Lifecycle Concurrency Efficiency
Internal code changes in one of Device Support, Record Support, CA Server shouldn’t require matching changes in one of the others
Need runtime data introspection
Data Access provides runtime introspection Catalog, an abstract interface to a
structured data▪ traverse reveals all of the fields and their
purposes▪ find locates a field of a particular purpose
Clerk provides simple get interface▪ clerk.get ( id_units, unitsString );▪ Range errors detected during conversion (an
upgrade)
Data is created during record processing, but must not be destroyed until the last per-client thread in the server is done filtering / copying it
Reference counting smart pointers manage data lifecycle When the last pointer reference is
destroyed the data are destroyed
Data is modified during record processing, but must not be modified at the same time that a server’s per-client thread is filtering / copying it
Auto-locking smart pointers manage concurrency
Smart Pointers work through flexible abstract handle interface Application chooses locking strategy
▪ Locks can be shared between objects▪ If the data are immutable then no locking is required
Applications choose reference counting strategy▪ Reference counter might be shared between objects▪ Reference counter might be embedded with the data▪ Immortal data? no reference counter required
Smart pointer reference counting Uses new atomic operations library
▪ Much faster than a mutexData Access
Arrays transferred in moderate, fixed, sized chunks
• In the EPICS community we have – Application developers (well adjusted,
etc)• EPICS database, screens, matlab, python, tcl,
etc
– System programmers (geeks)• Device drivers, EPICS internals, etc
• Implementing Data Access interface for particular data structure / class • System programmer job
• Once interfaced with Data Access – A high level (i.e. easy ca interface is
available)– DA is not the data manipulation interface used
by application level users– Users use the public interface of the data
structure / class which has been interfaced
– Communities develop around the data structures / classes standardized by particular applications, industries, and instruments
CA Server
Database
Device Support
Database CA Service
General strategy Database service – part of the database
implementation▪ Therefore can be (should be) intimately aware of
database internals Improvements
Eliminate subscription list protection mutex allocated in every record
Allow communication via non-contiguous arrays▪ Eliminate EPICS_CA_MAX_ARRAY_SIZE parameter
• Eliminate EPICS_CA_MAX_ARRAY_SIZE• dbGetField, dbPutField, dbPutCallback array
API– void pointer, number of elements, type code– Contiguous buffer greater than equal largest array must exist in ca server
• In contrast, Data Access – Arrays passed as a sequence of compile-time-
typed contiguous blocks with multi-dimensional bounds
– Sever now uses moderate-sized, fixed-sized, communication buffers
• Somehow Data Access interfaced data must be the source/sink for a database field
Eliminate subscription list protection mutex allocated in every record Database service can protect its
subscription list using DB scan lock Smart pointer handle for the db service
▪ Uses the db scan lock for synchronization
General strategy Minimal changes to existing database access
code which can be easily verified▪ No architectural changes
Refactor dbGetField, dbPutField, dbPutCallback code onto lowest common denominator interface▪ Presuming db scan lock already owned
▪ Having been taken at a higher level
▪ Field modification callback function pointer parameter▪ Lowest common denominator interface is private
Lock, in every record for subscription list is eliminated Opportunities to eliminate locking
▪ Consolidate CA service locking and db scan locking
dbGetField, dbPutField, dbPutField API retained exactly backwards compatible Functionally and almost line-by-line
equivalent, but refactored, code in dbGetField, dbPutField, dbPutField
Designed to transport polymorphic data Event queue carrying polymorphic/parasitic
data New API
▪ Identical server-to-service and client-side-application-to-client-lib
Designed for SMP Eliminated from server
EPICS_CA_MAX_ARRAY_SIZE Multicast listener Binding to specific network interfaces
Inherited from PCAS
Event filtering
>camonitor "fred$F $(PV:)>30 && $(PV)<40"fred$F $(PV:)>30 && $(PV)<40 2010-06-03 07:58:47.224969 36.6466fred$F $(PV:)>30 && $(PV)<40 2010-06-03 07:58:47.227964 37.1654fred$F $(PV:)>30 && $(PV)<40 2010-06-03 07:58:47.267460 33.9427fred$F $(PV:)>30 && $(PV)<40 2010-06-03 07:58:47.276013 33.9976fred$F $(PV:)>30 && $(PV)<40 2010-06-03 07:58:47.299041 37.8033fred$F $(PV:)>30 && $(PV)<40 2010-06-03 07:58:47.319065 33.549
>camonitor "fred$F $(PV:flavor)==30 "fred$F $(PV:flavor)==30 2010-06-03 07:58:18.906049 44.1145fred$F $(PV:flavor)==30 2010-06-03 07:58:21.899019 39.2743fred$F $(PV:flavor)==30 2010-06-03 07:58:24.885000 54.3352fred$F $(PV:flavor)==30 2010-06-03 07:58:27.855063 93.9634fred$F $(PV:flavor)==30 2010-06-03 07:58:30.811997 97.7081
pv { timeStampalarm {
acknowledge { pending }condition { status, severity }
}limits {
display { upper, lower }control { upper, lower }alarm {
major { upper, lower }minor { upper, lower }
}}labelsunitsprecision
class { name }}
Green indicates that a value is stored. In a DA tree a node does not need to be a leaf node in order to carry a value. This allows for less hierarchy traversal when doing a basic fetch. For example.
Catalog & someData;Clerk clerk (someData );double value;clerk.get (pi_pv, value );
pv {signal {
devicdNametimeStampwaveform {
valuesampleRate
}LANSCE {
flavor}
}
}
Addressing ConfigurationRouting issues
One to many communication, but unlike IP broadcasting Designed to make IP routers transparent
IP Multicasting is mature, widely deployed Commercial stock exchanges Multimedia content delivery industries
Administratively scoped multicast groups
Multicasting has good potential to simplify configuring of Channel Access in large systems with multiple subnets
Scope IPV4 Range
link-local 224.0.0.0 to 224.0.0.255
Site-local 239.255.0.0 to 239.255.255.255
Org-local 239.192.0.0 to 239.195.255.255
Global 224.0.1.0 to 238.255.255.255
Search requests Clients send to a special IP v4 address
▪ No client side code changes Server listens to a
▪ multicast group, or▪ multicast group on a specified interface
Beacon messages Ditto, but visa-versa Can eliminate need for CA repeater
Client side EPICS_CA_ADDR_LIST
▪ If address is multicast group ▪ Search messages sent to that mc group
EPICS_CA_BEACON_ADDR_LIST▪ If not defined
▪ Monitor mc groups in EPICS_CA_ADDR_LIST for beacons
▪ If defined and not empty▪ Specifies mc group set to be monitored for beacon
messages
▪ if defined and empty - will not receive mc beacons
Server side EPICS_CAS_INTF_ADDR_LIST isn't defined
▪ Server listens on all network interfaces▪ For messages sent to EPICS_CAS_SERVER_PORT
Unicast and broadcast messages Multicast messages, sent to any multicast group
address found in EPICS_CA_ADDR_LIST
Server side EPICS_CAS_INTF_ADDR_LIST is defined
▪ If address is a multicast group then multicasts sent to that group are received on all configured interfaces
▪ Addresses of the form ▪ {multicast group address, interface address}▪ Multicasts sent to specified mc group will be received
Only on specified interface▪ Unicast and broadcast traffic
Will not be received on specified interface (unless enabled elsewhere)
EPICS_CAS_BEACON_ADDR_LIST Specify beacon destinations (mc or
otherwise)▪ When EPICS_CAS_INTF_ADDR_LIST isn't
defined then▪ Defaults to EPICS_CA_BEACON_ADDR_LIST, or if that
isn't defined, EPICS_CA_ADDR_LIST
Logical names for multicast groups Recommended, but
▪ Just use DNS, local host files, etc
EPICS_CA_BEACON_PORT Defaults to EPICS_CA_REPEATER_PORT
▪ EPICS_CA_REPEATER_PORT becomes deprecated
Just an appropriate name change
Network admin must decide where the boundaries will lie for different levels of administratively scoped IPV4 multicasting A multicast will not be auto-forwarded
outside of its scope
Conservative approach is nature state of control system community Don’t fix it if it isn't broken Its very simple, robust, and efficient now
▪ Will new features only detract from the fundamentals responsible for success?
Hazards of conservative approach New features only added as evolution
instead of architectural upgrades There can be a patchwork of new
features instead of a grand design Learning the system becomes more
difficult because of a need to▪ Carefully absorb a chapter on each patchwork▪ Carefully deciding which patchwork will be
best
Perhaps we can afford some new capabilities We started out on 20MHz 68020 with 2MB
of memory Quad core SMP will soon be the default With a larger user base we can beat the
bugs out of the carpet faster, and amortize the expense over more projects
Low cost of embedded processors ▪ only a few signals per processor is cost effective
Basic principals of software quality New features, only in new-feature
release Patches, only in patch releases
Perhaps the path is easily navigated Provide backwards compatibility New features, only in new-feature
releases Patches only, in patch releases
Full disclosure Users of new-features release pay a price
▪ Document new bugs so that they can be fixed▪ New programming Interfaces need user
feedback▪ They should not be locked down early on
Until a new release reaches sufficient maturity ▪ Not a good choice for essential production
systems
Next generation CA server library substantially complete Provides many advantages
Database CA Service nearing completion
Multicasting Simplified configuration of large
systems!