View
386
Download
2
Category
Tags:
Preview:
Citation preview
CoreDX DDSTechnical Brief
Twin Oaks Computing, Inc.
Jul 2012
CoreDX DDS is a Communications Middleware Eases development and deployment of
Distributed Applications Supports interoperable communications High Throughput with Low Latency Insulates Application from Communication
Details
What is CoreDX DDS?
Publisher
Subscriber
Subscriber
Subscriber
Message Oriented Middleware A Publisher Generates Data A Subscriber Accesses Data The Middleware Manages the
Connections and the Data Publisher and Subscriber are
Loosely Coupled
CoreDX DDS Publish / Subscribe Middleware
Data Type
Data
QoS
Topic
Publisher
DDS extends MOM concepts Adds Strong Data Typing Adds Quality of Service Policies
▪ Reliability / History / Durability / Filtering
Formalized Standard (API and Wire Protocol)▪ Portable / Interoperable between
Vendors
A DataWriter publishes data on a Topic A DataReader subscribes to a Topic Each Topic has a Defined Data Type
CoreDX DDS Architecture
DomainParticipant Associated with a
Domain Communicates with
other DomainParticipants in the same Domain
Contains DataReaders, DataWriters, and Topics
Object Management Group manages DDS standards: DDS API: Data Distribution Service
▪ Entities
▪ QoS Policies
▪ Asynchronous and Synchronous notifications
▪ Data Types
DDS Language Bindings: DDS Wire Protocol: Real-Time Publish Subscribe (RTPS) Wire Protocol
▪ Discovery
▪ Reliability
▪ Data Encoding
DDS-Related Technologies▪ Extensible Types
▪ Web Enabled DDS (in work)
▪ DDS Security (in work)
Standards Based Data Communication
Technical Features
Dynamic Discovery
Reliability
Durability
Historical Data Caches
Filtering
Ownership
Cross-platform
Interoperability
Portability
High Performance
Highly Configurable
DDS Dynamic Discovery is powerful and useful Built-in to all CoreDX DDS Applications
▪ CoreDX DDS applications publish and subscribe to data without configuring endpoints
▪ CoreDX DDS Entities do not require knowledge of other DDS Entities they may be communicating with
Interoperable between DDS implementations▪ Applications built on compliant DDS implementations will
automatically discover each other
Automatic▪ No special configuration required
▪ Simply create DDS Entities
CoreDX DDS Features:Dynamic Discovery
CoreDX DDS Applications (DomainParticipants) Automatically discover each other Exchange network information
▪ IP addresses for multicast and unicast data transfers
Producers and Consumers (DataWriters, DataReaders) Automatically discover each other Exchange configuration information Determine if data exchange is appropriate
▪ Do Topics match?
▪ Do Data Types match?
▪ Do communication behavior expectations match?
If so, enable data exchange
CoreDX DDS Features:Dynamic Discovery (cont)
Twin Oaks Computing, Inc. PROPRIETARY
QoS Policies tailor the behavior of data communications What are the reliability requirements? How long is data saved for possible future publication or
future access by the application?▪ What are the storage requirements?
What are the timing requirements? How should data be presented to subscribers? Should data be filtered? Are there redundancy or failover requirements?
DDS specifies 22 distinct QoS Policies
CoreDX DDS Features:Quality of Service Policies
CoreDX DDS provides configurable Reliable data communications Even on unreliable networks (e.g. wireless) Using UDP (MULTICAST and UNICAST)
CoreDX DDS Reliability Addresses Dropped Packets Out of Order Samples Communication Disconnects Application Re-Starts
CoreDX DDS Features:Efficient Reliability: Overview
CoreDX DDS Features:Efficient Reliability: Protocol
Configurable Heartbeat and ACK/NACK interval Avoid packet storms
Configuration items to balance latency and overhead Heartbeats, delays, resource limits
RELIABLE Guarantees:• Delivery and Order
BEST EFFORT Guarantees: • Order
07 06 05 04 03 02 01
Performance monitored by readers and writers
Flexible handling of slow readers Dynamically remove slow readers
Data Reader
07
0506
04030201
07
0506
04030201
Data Writer
CoreDX DDS Features:Efficient Reliability: Example
Writer sends Heartbeats Heartbeats can be sent with DATA
messages Reader returns ACK/NACK
Missed samples detected by Reader
NACK Indicate specific missing samples
Combined with ACK to acknowledge other samples
Writer can send Data using MULTICAST Repairs are sent UNICAST
Reader delivers samples to the application in order
01
02
03
04
01
02
03 + HB
04
01
05
06
07
05
06 + HB
07
02
04
05
06
07
Writer Reader
03ACK (1,3), NACK [2]
ACK (6)
02
One dropped sample
CoreDX DDS provides configurable Durable data communications
CoreDX DDS Durability addresses How long data is saved by Data Writers If previously published data is sent to ‘late joining’ DataReaders
Durability Options Volatile: Published data is saved just long enough to be sent to currently
matched Readers Transient Local: Published data is saved for as long as the Data Writer is
alive (until application exit) Transient, Persistent: Published data is saved by an external service,
through application restarts, machine reboots
CoreDX DDS Features:Durability
CoreDX DDS Features:Durability: Example
Durability Example
Data Producer
Data published withDurability = TRANSIENT_LOCAL
Data Consumer 1
Data Consumer 2
Subscribing to data with Durability = VOLITILE
Subscribing to data with Durability = TRANSIENT_LOCAL
S2
S3
S4
S4
S1
S2S4
S2 S1
Each Data Reader and Data Writer maintains a Data Cache Data Reader Cache
Holds received data to be delivered to the application Can be sized and configured to save received data for later use Multiple ways to access received data CoreDX DDS ‘loans’ data to the application
▪ Improves performance
▪ Eliminates extra data copy
Data Writer Cache Holds published data to be written on the wire Can be sized and configured to save historical data to be
delivered to late joining Readers
CoreDX DDS Features:Historical Data Caches
Kinds of Filters: Content Filters are specified by an SQL-like query
▪ Same syntax as the “WHERE” clause in an SQL query
Time Based Filters are specified by a duration for minimum separate between samples
Read Filters allow the application to ‘read’ specific kinds of data from the Data Cache▪ Previously seen / Not seen
▪ New data / Old data
▪ Alive / Dead
▪ Matching a content-based query
CoreDX DDS Features:Filtering
Filters are configurable By each Reader Applied at the Writer or Reader
Filters can reduce network and CPU load
CoreDX DDS Features:Filtering (cont)
Filter applied at the Writer▪ Writer can send only the data requested by a Reader (UNICAST)
▪ Other Readers do not ‘wake up’ to receive/process unwanted data
Filter applied at the Reader▪ Writer can MULTICAST data to all Readers
▪ All Readers will ‘wake up’ to receive / apply filters to data
CoreDX DDS provides configurable Ownership of published data Shared: Readers receive (duplicate) data from all Writers Exclusive: Readers receive one copy of data, even when multiple Writers
are publishing (duplicate) data
CoreDX DDS Features:Ownership
Data Producer 1
Ownership.kind = EXCLUSIVEOwnership.strength = 10
The consumer will see data from only the strongest producer.
Data Consumer
Data Producer 2
S1
S1
S2
S1
S2
Ownership.kind = EXCLUSIVEOwnership.strength = 5
Cross-Platform communications: x86 (32 & 64 bit), UltraSPARC, ARMv5,
ARVMv7, PPC, MIPS, Microblaze, FPGAs
Cross-Operating System communications: Linux, Windows, Solaris, QNX, VxWorks,
NexusWare, LynxOS, Android, Unison
Cross-Language communications: C, C++, C#, Java
CoreDX DDS Features:Cross-platform, and more
Portable across languages Common API defined for multiple languages
Portable across Operating Systems Same programming experience on all
operating systems Portable across vendors
Standardized API
CoreDX DDS Features:Portability
What is it? The ability of DDS implementations from different
vendors/sources to communicate Why do we want it?
Increased flexibility in implementation decisions Vendor independence
Open Standards from the OMG allow for Interoperability DDS API RTPS Wire protocol
Vendors commit to and test Interoperability▪ Twin Oaks Computing, RTI, (partial) Prism Tech, Gallium
CoreDX DDS Features:Interoperability
Features: Minimal Lines of Source Code
▪ Entire ‘C’ CoreDX DDS library < 35K SLOC
▪ Safety Critical Edition < 13K SLOC
Completely Native Source Code▪ No 3rd party products or packages
Made in the USA▪ All CoreDX product development performed in the United States, by United States Citizens
Minimal dependencies on Operating System libraries Benefits:
Eases analysis and certification of baseline EASY TO PORT TO ANY PLATFORM: Enterprise to Bare-Metal
operating systems
CoreDX DDS:Source Code: Portable, Certifiable
22Twin Oaks Computing, Inc
PROPRIETARY
No Performance Sacrifice
Performance Summary:• Throughput > 900 Mbps• Latency < 70 usec
CoreDX DDS:Performance: Throughput, Latency
23Twin Oaks Computing, Inc
PROPRIETARY
Long running throughput test• TX1024 byte messages
• Total throughput of over 700 Megabits/sec
• CPU utilization of a single core
• doesn’t exceed 30%
• machine has 4 cores
• Equates to less than 10% of total CPU
Twin Oaks Computing, Inc PROPRIETARY24
CoreDX DDS:Performance: CPU, Network
CoreDX DDS run-time memory requirements
For example:An application with
▪ 1 DomainParticipant,
▪ 6 DataWriters, and
▪ 4 DataReaders
will require <100KB Heap MemoryTwin Oaks Computing, Inc
PROPRIETARY25
CoreDX DDS:Small Run-time Memory Requirements
Local Entity Local Entity Size (bytes)
Remote (Discovered) Entity Size (bytes)
DomainParticipant * 12000 2100
DataReader 3200 2500
DataWriter 3200 2400
Topic 750 0
* DomainParticipant size includes all built-in Topics, DataWriters, and DataReaders
Minimal Binary Code Size Measured in Kilobytes (not Megabytes!)
CoreDX “DDS Ping” application: 250 KB
Complete CoreDX DDS C library < 500KB
Conserves valuable Static RAM / Flash Critical for embedded devices
Examples Gumstix COMs
OMAP L138
Android based phones and tablets
FPGA’sTwin Oaks Computing, Inc
PROPRIETARY26
CoreDX DDS:Small Library Code Size
DDS API includes specific policies for tailoring communication behavior.
Deadline Latency Budget Reliability
Destination Order Lifespan Resource Limits
Durability Liveliness Time Based Filter
Entity Factory Ownership Topic Data
Group Data Partition User Data
History Reader Data Lifecycle Writer Data Lifecycle
In addition, CoreDX DDS allows configuration of:Transport Threads
Discovery Memory Usage
Reliability Logging
CoreDX DDS Features:Configurability
Comparisons
DDS
JMS
Web Services
XMPP
Comparison:DDS and JMS
lookup ConnectionFactorycreate Connectioncreate Sessioncreate MessageProducerstart the Connectionwrite data
lookup ConnectionFactorycreate Connection
create Sessioncreate MessageConsumer
start the Connectionread data
JMS Server
create a DDS Participantcreate a Topiccreate a Publishercreate a DataWriterwrite data
create a DDS participantcreate a Topic
create a Subscribercreate a DataReader
read dataEach DDS application contains all necessary code for discovery and communications, no server required
JMS requires a Server that must be configured and connected
JMS Server often requires significant resources, and creates a single point of failure
JNDI
DDS and JMS
Feature JMS DDS
Architecture Publish subscribe Publish subscribe (multicast)
Platform Independence
Same API is exposed for all HW, OS, and languages supported
Same API is exposed for all HW, OS, and languages supported
Discovery of endpoints
JNDI and JMS servers must be specified and configured
Dynamic Discovery, no need to specify where endpoints reside
Type Safety Generic Objects and XML are not type safe
Strong type safety, application calls write() and read() with a specific data type
Tailoring communication behavior
Limited ability to tailor communications
QoS policies allow for easy tailoring of communication behaviors
Interoperability None Open standard with proven interoperability
Comparison:DDS and SOA Web Services
create a DDS Participantcreate a Topiccreate a Publishercreate a DataWriterwrite data
create a DDS participantcreate a Topic
create a Subscribercreate a DataReader
read data
Each DDS application contains all necessary code for discovery and communications, no server or end point configuration required
create Serviceconfigure Server Address
call Serviceget response
Web Server
enable web servicesinstall web service
A web server must be configured to provide web services, and the web service must be installed
The client must be configured with the address of the web server before the service can be called
Web Service
DDS and SOA Web Services
Feature Web Services DDS
Architecture Message oriented Middleware Publish subscribe (multicast)
Platform Independence
Same API is exposed for all HW, OS, and languages supported.Browser clients can run anywhere
Same API is exposed for all HW, OS, and languages supported
Discovery of endpoints
Web server must be specified and configured
Dynamic Discovery, no need to specify where endpoints reside
Type Safety Use of HTTP and XML does not provide strong type safety
Strong type safety, application calls write() and read() with a specific data type
Tailoring communication behavior
Limited ability to tailor communications
QoS policies allow for easy tailoring of communication behaviors
Interoperability Some interoperability with multiple protocols used: SOAP, REST, WSDL
Open standard with proven interoperability
Visit our Websitehttp://www.twinoakscomputing.com for: Quick Start Guides Free Evaluations Free Demonstrations (with source code) Programmer’s Guide Reference Manuals White Papers
Contact Us Phone: 1-855-671-8754 Email: contact@twinoakscomputing.com Facebook / Linked In / Twitter / Pinterest / Google+
For More Information
Common, Complex Communication Requirements: Where do I get my data?
How often?
What happens if you don’t send it?
What happens if I don’t receive it?
How much received data do I need to store for future use?
How do I select (filter) only the data I need to receive?
CoreDX DDS addresses all these requirements
Summary
34Twin Oaks Computing, Inc
PROPRIETARY
Questions?
Recommended