Upload
angel-jesus-martinez-bernal
View
300
Download
5
Tags:
Embed Size (px)
Citation preview
TRABAJO FIN DE MÁSTER: AUTOMATED MULTIPLATFORM COMPILATION AND VALIDATION OF A COLLABORATIVE REPOSITORY OF
EXAMPLES USING RTI CONNEXT DDS
Author: Ángel Jesús Martínez Bernal Advisor: María Sara Granados Cabeza Granada, December 2014
Content Introduction
Rationale State of the Art
Problem definition
Goals
Development Common Example Structure
Examples Repository
Automation Task
Results
Conclusions
Content Introduction
Rationale State of the Art
Problem definition
Goals
Development Common Example Structure
Examples Repository
Automation Task
Results
Conclusions
Introduction (I) Automation
Automation in Computer Science
Programming Languages
Punched cards
Assembly programming language
Different paradigms (functional programming, logic programming…)
Oriented Object Programming
Scripting
GPU
Introduction (I) Automation
Automation in Computer Science
Programming Languages
Punched cards
Assembly programming language
Different paradigms (functional programming, logic programming…)
Oriented Object Programming
Scripting
GPU
Introduction (II) Continuous Integration
Revision control -> Repositories
Content Introduction
Rationale State of the Art
Problem definition
Goals
Development Common Example Structure
Examples Repository
Automation Task
Results
Conclusions
Rationale - State of the Art (I) Distributed System Communication
Client/Server Paradigm
Rationale - State of the Art (II) Publish/Subscribe Paradigm
Rationale - State of the Art (III) Data Distribution Service (DDS)
OMG Standard
Features:
QoS-enabled
Data-centric
High scalability
More dynamic network topology
Rationale - State of the Art (IV) Data Distribution Service (DDS)
Rationale - State of the Art (V) RTI Connext DDS
Fully compliant with the OMG DDS standard, API and RTPS protocol
Interoperable: languages, operating systems and network types
Low latency and high throughput
Deterministic even under load and at scale
Reliable multicast
Peer-to-peer architecture: no brokers or servers
Non-stop availability.
Different approaches depending on the system.
Rationale - State of the Art (VI)
DDS Bus / RTI DataBus™
Connext
Micro
Pub/Sub API
Small Device Apps
Connext
DDS
Pub/Sub API
DDS Apps
Connext
Messaging
Messaging API
General-Purpose Real-Time Apps
Connext
Integrator
Adapters
Discrete OT & IT Apps/Systems
Administration
Monitoring
Logging
Recording
Replay
Federation
Transformation
Persistence Visualization
Rationale - State of the Art (VII) Continuous Integration (Bamboo)
Local agent
Remote agent
Virtual agent
Content Introduction
Rationale State of the Art
Problem definition
Goals
Development Common Example Structure
Examples Repository
Automation Task
Results
Conclusions
Problem Definition (I) RTI’s user community unmaintainable
Out-of-date examples
Not compiling
Not working in latest versions
Using deprecated features
Only available for one language or platform
Many collaborators Different patterns
More time understanding the example than the feature.
No continuous integration!
Problem Definition (II) QoS used to be
changed programmatically.
Now, XML files are recommended.
We needed to commented them out and create XML profiles. E.G.: Asynchronous
publication:
Problem Definition (III) Use USER_QOS_PROFILES.xml rather than code.
Problem Definition (IV) Needed an automation process able to:
Manage the copyright header
Not necessary for external examples
Build all examples in different platforms
Validate QoS files (XML extension)
Manage <dds> tag
Bundle examples for uploading to a website
Solution needs to be scalable and maintainable.
Content Introduction
Rationale State of the Art
Problem definition
Goals
Development Common Example Structure
Examples Repository
Automation Task
Results
Conclusions
Goals (I) Complete & update the RTI Connext DDS
example repository
Refactor examples to follow a common same pattern
Add new examples
Integrate the examples in the Continuous Integration methodology:
Automated process
Create configurable scripts for scalability
Goals (II) Initial state of the repository
Content Introduction
Rationale State of the Art
Problem definition
Goals
Development Common Example Structure
Examples Repository
Automation Task
Results
Conclusions
Common Example Structure (I) Example Git repository:
https://github.com/rticommunity/rticonnextdds-examples
Outside structure: Example directory:
README.txt /C
README.txt Foo_publisher.c Foo_subscriber.c USER_QOS_PROFILES.xml Foo.idl
/C++ /Java /CS
Create participant
Create publisher
Get type_name
Create topic
Create DataWriter
Narrow DataWriter
Create Instance
Delete data
Common Example Structure (II) Publisher
Modify data
(instance)
Write the data
(instance)
Sleep
Create participant
Create publisher
Get type_name
Create topic
Create DataWriter
Narrow DataWriter
Create Instance
Delete data
Common Example Structure (II)
Modify data
(instance)
Write the data
(instance)
Sleep
Publisher
Create Participant
Create Subscriber
Get type_name
Create topic Create
Listener Create
DataReader
Delete data
Subscriber
Narrow DataReader
Read sample Print the sample
On_data_available
Common Example Structure (III)
Create Participant
Create Subscriber
Get type_name
Create topic Create
Listener Create
DataReader
Delete data
Subscriber
Narrow DataReader
Read sample Print the sample
On_data_available
Common Example Structure (III)
Create Participant
Create Subscriber
Get type_name
Create topic Create
Listener Create
DataReader
Delete data
Subscriber
Narrow DataReader
Read sample Print the sample
On_data_available
Common Example Structure (III)
Create Participant
Create Subscriber
Get type_name
Create topic Create
Listener Create
DataReader
Delete data
Subscriber
Narrow DataReader
Read sample
Print the sample
Common Example Structure (III)
Narrow DataReader
Create Conditions
Create WaitSet
Attach conditions
Create Participant
Create Subscriber
Get type_name
Create topic Create
DataReader Create
Waitset*
Loop: Wait data / printing
Delete Data
Additional Steps WaitSets
Automation process Tasks:
Manage Copyright notice (Check, Add & Delete)
Automated building for different architectures
Windows, Linux or other supported by the tools.
XML validation
Point to Community XSD file (dds tag).
Update DDS version & schema.
Bundle examples
Tools: Bamboo and Perl scripts
Manage Copyright Notice (I) Copies the copyright header to the files under a
folder.
Available for XML and C/C++.
Configurable:
Decide the extensions of the files to process.
Select an option (option_flag): check/copy/delete.
Skip folders or files (e.g.: external examples).
Call syntax: perl ./resources/scripts/copyright.pl <working_directory> <option_flag> <extensions>...
Manage Copyright Notice (II) C Style Copyright Header: Top of the file
Manage Copyright Notice (III) XML Style Copyright Header: after XML definition
Multiplatform Building Call syntax perl ./resources/scripts/compile_[unix|windows].pl <working_directory> <ndds_version> <architecture_c>
Look for IDL files
Call rtiddsgen
Call the compiler
XML Validation Validates all the QoS files with
its corresponding schema version (.xsd).
Call syntax: perl ./resources/scripts/xml_validation.pl <working_directory> <ndds_version>
Validate!
(or error)
xmllint
XML file
URL of the .xsd
file
XML Update Schema location should NOT be a local path, but an
online one:
Three different actions (option_flag): Check whether the <dds> tag has all the 3 attributes.
Add the attributes that the <dds> tag did not have.
Replace the attributes <path_schema> and <ndds_version>.
Call syntax: perl xml_update.pl <working_directory> <option_flag> <ndds_version> <path_schema>
Bundle Examples (I) Bundle the examples in order to upload to a website
Create a .zip & .tar.gz which contains all the examples
Bundle every example separately
Upload all packages & files to a website
Call syntax: perl bundle_examples.pl <working_directory>
<repository_name>
Bundle Examples (II) rticonnextdds-examples
asynchronous_publication
C++
C
Cs
Java
Readme.txt
Batching
builtin_qos_profiles
builtin_topics
coherent_presentation
…
.zip & .tar.gz
.zip & .tar.gz
Bamboo Integration (I) Building steps:
Copyright
XML validation
Compile
Bamboo Integration (II) Shipping steps:
Bamboo Integration Script configuration as Bamboo task
Bamboo Integration (III)
Bamboo Integration (IV)
Content Introduction
Rationale State of Art
Problem definition
Goals
Development Common Example Structure
Examples Repository
Automation Tasks
Results
Conclusions
Results (I) Contribution to the repository
Results (II)
Available soon
Results (III) New examples
Content Introduction
Rationale State of the Art
Problem definition
Goals
Development Common Example Structure
Examples Repository
Automation Task
Results
Conclusions
Conclusions RTI Connext DDS
Easy to create p/s applicatons
Configurable thanks to QoS policies
Big community that needs CI
New Examples
Continuous integration Allow projects to grow easily
Scripts for high scalability
Integrating with different Atlassian’s tools. For instance: JIRA
Future Work Integrate more feature examples
Add the custom content filter example in C#
Evaluate interesting features and concept and create new examples
Update the shipped examples:
Force them to have the same structure
Use a subset of the RTI Community examples repository as shipped examples.
Compile for more platforms
VxWorks, Raspberry Pi, Integrity, AIX, …
TRABAJO FIN DE MÁSTER: AUTOMATED MULTIPLATFORM COMPILATION AND VALIDATION OF A COLLABORATIVE REPOSITORY OF
EXAMPLES USING RTI CONNEXT DDS
Author: Ángel Jesús Martínez Bernal Advisor: María Sara Granados Cabeza Granada, December 2014