Enhancing Real-time CORBAEnhancing Real-time CORBAvia Real-time Java featuresvia Real-time Java features
International Conference on Distributed Computer SystemsTokyo, Japan
Friday, March 19, 2004
Arvind S. KrishnaDouglas C. Schmidt Elec & Comp. Eng. Dept
Vanderbilt University
{arvindk, schmidt}@dre.vanderbilt.edu
Raymond Klefstad
Elec & Comp. Eng. Dept
University of California, [email protected]
Talk Outline
− Tech transitions in the DRE domain Tech transitions in the DRE domain Real-time Real-time middleware & Real-time Javamiddleware & Real-time Java
− RT-Java + real-time middleware RT-Java + real-time middleware ZEN projectZEN project− ZEN R&D processZEN R&D process
− Design & ArchitectureDesign & Architecture− Applying Real-time Java features Applying Real-time Java features − Empirical ResultsEmpirical Results
− Concluding remarks and ReferencesConcluding remarks and References
DRE Domain: Characteristics
Types• Wide applicability
• Range from Total ship-board computing systems to Industrial process control systems
• Common Requirement• Right answer delivered too late becomes
wrong answer
Characteristics and Requirements• Distributed Systems require
capabilities to manage connections and message transfer between separate machines
• Real-time Systems require predictable and efficient control over end-to-end system resources
• Embedded Systems have weight, cost, and power constraints that limit their computing and memory resources
Total Ship C&C Center
Total Ship Computing Environments
Increasingly DRE applications are combined to form “systems of systems”
Increasingly DRE applications are combined to form “systems of systems”
Industrial Process Control
Current Real-time Middleware Trends• Current Real-time CORBA ORBs are developed in C and C++
– ACE+TAO from ISIS Vanderbilt– ORBexpress from OIC– eORB from Prism Technologies
• Real-time CORBA has not been universally adopted by DRE application developers
– Complexity of the CORBA C++ mapping
– Steep learning curve caused by feature rich and complex C++ language
• Increasingly hard to find “good” C++ application developers and retain them
Java Programming Language has emerged having less complexity than C++
– Safety– Simplicity– Productivity
However, Java is not suitable for developing DRE applications:
The scheduling of Java threads is purposely under-specified (so to allow easy implementation of JVM on as many platform as possible)
The GC can preempt for unbounded amount of time Java Threads
Java provides coarse-grained control over memory allocation, and it does not provide access to raw memory
Java does not provide high resolution time, nor access to signals, e.g. POSIX Signals
The Real-Time Specification for Java (RTSJ) extends Java in the following areas:
• New memory management models that can be used in lieu of garbage collection
• Stronger semantics on thread and their scheduling
• Access to physical memory• Asynchronous Event handling mechanism• Timers and Higher time resolution• Priority pre-emptive scheduler
RTSJ Thread & Memory Models
RTSJ Thread ModelReal-time Threads – priority and scheduling characteristics specified
NoHeapRealtimeThreads• Do not “touch” the heap• Use of NHRT threads can
have exec eligibility higher than that of GC
RTSJ Thread ModelReal-time Threads – priority and scheduling characteristics specified
NoHeapRealtimeThreads• Do not “touch” the heap• Use of NHRT threads can
have exec eligibility higher than that of GC
Scoped MemoryRegion based memory based tied to number of active threads in that region
PropertiesAny thread may create a scoped region – using new operatorHowever, only a real-time thread may allocated from that region
Scoped MemoryRegion based memory based tied to number of active threads in that region
PropertiesAny thread may create a scoped region – using new operatorHowever, only a real-time thread may allocated from that region
Immortal Memory• Same lifetime as the JVM• Objects allocated never garbage collected
Immortal Memory• Same lifetime as the JVM• Objects allocated never garbage collected
Physical Memory• Allows access to specific locations based on addresses.
Physical Memory• Allows access to specific locations based on addresses.
Scoped Memory in Action
Heap
time
new A(…)
(new RealtimeThread(…)).start()
Obj A created in Heap region
Scoped Memory in Action
Heap
ma1 = new LTMemory(…)
ma1
<LTMemory>
time(new RealtimeThread(…)).start()
Ma1 is an “inner scope”can hold references only to “outer regions”
Note: ma1 is reference is in heap,
Reference count = 0
Scoped MemoryProperties• Reference counted; no of active threads in region• When reference count of a region drops to zero
– All Objects within that region are considered unreachable
– Finalizers of all objects run; Assignment Rules
• obj in region ma can hold ref to obj in region mb if
lifetime (ma) <= lifetime (heap)
ma1.enter(logic1)
logic1
Logic instance of Runnable ma1 Current Allocation Context
Reference count = 1
Talk Outline
Tech transitions in the DRE domain Tech transitions in the DRE domain Real-time Real-time middleware & Real-time Javamiddleware & Real-time Java
− RT-Java + real-time middleware RT-Java + real-time middleware ZEN projectZEN project− Design & ArchitectureDesign & Architecture− Applying Real-time Java features Applying Real-time Java features − Empirical ResultsEmpirical Results
− Concluding remarks and ReferencesConcluding remarks and References
Motivation for ZEN Real-time ORB
Integrate best aspects of several key technologies
• Java: Simple, less error-prone, large user-base
• Real-time Java: Real-time support
• CORBA: Standards-based distributed applications
• Real-time CORBA: CORBA with Real-time QoS capabilities
ZEN project goals
• Make development of distributed, real-time, & embedded (DRE) systems easier, faster, & more portable
• Provide open-source Real-time CORBA ORB written in Real-time Java to enhance international middleware R&D efforts
Integrate best aspects of several key technologies
• Java: Simple, less error-prone, large user-base
• Real-time Java: Real-time support
• CORBA: Standards-based distributed applications
• Real-time CORBA: CORBA with Real-time QoS capabilities
ZEN project goals
• Make development of distributed, real-time, & embedded (DRE) systems easier, faster, & more portable
• Provide open-source Real-time CORBA ORB written in Real-time Java to enhance international middleware R&D efforts
Phase I – Applying Opt Strategies
• Foot-print Reduction OptimizationFoot-print Reduction Optimization• Micro ORB Architecture Micro ORB Architecture Virtual Virtual Component Pattern Component Pattern
• Micro POA Architecture Micro POA Architecture Pluggable Pluggable componentscomponents
• Request Demux/Dispatch Request Demux/Dispatch OptimizationsOptimizations• Connection Management Connection Management Acceptor-Connector pattern, Reactor Acceptor-Connector pattern, Reactor (java’s nio package)(java’s nio package)
• Buffer Management StrategiesBuffer Management Strategies• Request Demultiplexing Request Demultiplexing Active Active
Demultiplexing & Perfect HashingDemultiplexing & Perfect Hashing
• POA OptimizationsPOA Optimizations • Object Key Processing Strategies Object Key Processing Strategies Asynchronous completion token Asynchronous completion token patternpattern
• Servant lookup Servant lookup Reverse lookup Reverse lookup mapmap
• Concurrency Strategies Concurrency Strategies Half-Half-Sync/Half-AsyncSync/Half-Async
• Foot-print Reduction OptimizationFoot-print Reduction Optimization• Micro ORB Architecture Micro ORB Architecture Virtual Virtual Component Pattern Component Pattern
• Micro POA Architecture Micro POA Architecture Pluggable Pluggable componentscomponents
• Request Demux/Dispatch Request Demux/Dispatch OptimizationsOptimizations• Connection Management Connection Management Acceptor-Connector pattern, Reactor Acceptor-Connector pattern, Reactor (java’s nio package)(java’s nio package)
• Buffer Management StrategiesBuffer Management Strategies• Request Demultiplexing Request Demultiplexing Active Active
Demultiplexing & Perfect HashingDemultiplexing & Perfect Hashing
• POA OptimizationsPOA Optimizations • Object Key Processing Strategies Object Key Processing Strategies Asynchronous completion token Asynchronous completion token patternpattern
• Servant lookup Servant lookup Reverse lookup Reverse lookup mapmap
• Concurrency Strategies Concurrency Strategies Half-Half-Sync/Half-AsyncSync/Half-Async
Object Adapter
ORB CORE
RT-CORBAThread Pool
Phase III Phase III Build a Real-Time CORBA Build a Real-Time CORBA ORB that runs atop a mature RTSJ ORB that runs atop a mature RTSJ LayerLayer
Phase III Phase III Build a Real-Time CORBA Build a Real-Time CORBA ORB that runs atop a mature RTSJ ORB that runs atop a mature RTSJ LayerLayer
Phase II – Applying RTSJPhase I Optimization patterns and principles
• ORB-Core Optimizations• Micro ORB Architecture Virtual Component
Pattern • Connection Management Acceptor-Connector
pattern, Reactor (java’s nio package)• Collocation and Buffer Management Strategies
• POA Optimizations • Request Demultiplexing Active Demultiplexing &
Perfect Hashing• Object Key Processing Strategies Asynchronous
completion token pattern• Servant lookup Reverse lookup map• Concurrency Strategies Half-Sync/Half-Async
Phase II Enhance Predictability by applying RTSJ features• Associate Scoped Memory with Key ORB
Components– I/O Layer : Acceptor-Connector, Transports– ORB Layer: CDR Streams, Message Parsers– POA Layer: Thread-Pools and Upcall Objects
• Using NoHeapRealtimeThreads – Ultimately use NHRT Threads for
request/response processing– Reduce priority inversions from Garbage
Collector
Phase II Enhance Predictability by applying RTSJ features• Associate Scoped Memory with Key ORB
Components– I/O Layer : Acceptor-Connector, Transports– ORB Layer: CDR Streams, Message Parsers– POA Layer: Thread-Pools and Upcall Objects
• Using NoHeapRealtimeThreads – Ultimately use NHRT Threads for
request/response processing– Reduce priority inversions from Garbage
Collector
Talk Outline
– Tech transitions in the DRE domain Tech transitions in the DRE domain Real-time Real-time middleware & Real-time Javamiddleware & Real-time Java
− RT-Java + real-time middleware RT-Java + real-time middleware ZEN projectZEN project− Design & ArchitectureDesign & Architecture− Applying Real-time Java features Applying Real-time Java features − Empirical ResultsEmpirical Results− Open challengesOpen challenges
− Concluding remarks and ReferencesConcluding remarks and References
Applying RTSJ features – Motivation
Design of ZEN for Real-Time CORBA• Apply scoped memory along critical request
processing path• Provide Policies at the POA level for RTSJ
aware users• Allows NHRT threads used for request
processing• Proper use of NHRT threads would minimize GC
execution during request processing
Design of ZEN for Real-Time CORBA• Apply scoped memory along critical request
processing path• Provide Policies at the POA level for RTSJ
aware users• Allows NHRT threads used for request
processing• Proper use of NHRT threads would minimize GC
execution during request processing
Original design of ZEN• All components allocated in heap
• Request processing thread may be preempted by GC (demand garbage collection)
Original design of ZEN• All components allocated in heap
• Request processing thread may be preempted by GC (demand garbage collection)
Goals • Compliance with CORBA specification• Interoperability with classic CORBA• Reduce overhead for applications not using
real-time features• End-user transparent
Goals • Compliance with CORBA specification• Interoperability with classic CORBA• Reduce overhead for applications not using
real-time features• End-user transparent
Analyzing Request Processing Steps
Server Side Connection Acceptance4. An acceptor accepts the new incoming connection.
6. A new connection handler T1 is created to service requests
7. The Transport's event loop waits for data events from the client
Server Side Request Processing Steps11. The request header on connection is read to
determine the size of the request.
12. A buffer of the corresponding size is obtained from the buffer manager to hold request and read data.
13. The request is the demultiplexed to obtain the target POA, servant, and skeleton servicing the request. The upcall is dispatched to the servant after demarshaling the request.
14. The reply is marshaled using the corresponding GIOP message writer; Transport sends reply to the client.
Server Side Connection Acceptance4. An acceptor accepts the new incoming connection.
6. A new connection handler T1 is created to service requests
7. The Transport's event loop waits for data events from the client
Server Side Request Processing Steps11. The request header on connection is read to
determine the size of the request.
12. A buffer of the corresponding size is obtained from the buffer manager to hold request and read data.
13. The request is the demultiplexed to obtain the target POA, servant, and skeleton servicing the request. The upcall is dispatched to the servant after demarshaling the request.
14. The reply is marshaled using the corresponding GIOP message writer; Transport sends reply to the client.
Server ORB
13
Object Adapter
Acceptor 4
5
12
Buffer Manager
Transport T1
6, 7,14
ConnectionCache
C1
C2
C3
11, 14
GIOPMessageParsers
1.0 1.01.1
Repetitive & Ephemeral
Carried out for each client request Typically objects live for one cycle
Repetitive & Ephemeral
Carried out for each client request Typically objects live for one cycle
Independent Steps for two different
clients do not share context
Independent Steps for two different
clients do not share context
Thread Bound
Steps executed by I/O threads Thread-Pool threads
Thread Bound
Steps executed by I/O threads Thread-Pool threads
Application of Scoped Memory – ZEN
Applying Scoped Memory1.Break Steps into three broad
regions based on request processing steps
I/O scope I/O scope read request read requestORB scope ORB scope process request process requestPOA scope POA scope perform upcall send perform upcall send
replyreply2.Recursively enter each space from
I/O POA scopes3.Implicitly exit regions from POA
I/O scope
Applying Scoped Memory1.Break Steps into three broad
regions based on request processing steps
I/O scope I/O scope read request read requestORB scope ORB scope process request process requestPOA scope POA scope perform upcall send perform upcall send
replyreply2.Recursively enter each space from
I/O POA scopes3.Implicitly exit regions from POA
I/O scope
ORB SPACE
POA SPACE
POASPACE
ORBSPACE
I/OSPACE
Independent
Ephemeral Create all objects in a scoped region
• Two requests can be mapped to two separate scope regions
• Temporary objects may be cleared after request processing
Threadbound •Encapsulate steps as “logic” class associate this logic class with real-time threads
•Threads enter the scoped region; processes request; exits region enabling objects to be finalized
Scope Memory in ZEN: Action
time
I/O Scope
ORB Scope
POA Scope
NETWORK
1: (new RealtimeThread(default Scope))
Associate a start scope with real-time
thread
2: (I/O Scope).enter()
I/O Scope is now current active region
I/O thread waits for data
3: waitForData()
The three scopes are created during ORB initialization time
Following Slides are adapted from Angelo Corsaro
I/O Stage Processing
time
I/O Scope
NETWORK1:Data arrival
2:Read Data
new GIOPMessage ()3:Create new Message
The message is created within the scoped region
I/O Scope• Participants The participants for this phase include,
acceptors, and transports
• RTSJ application –Each of these components are thread-bound components
and are designed based on inner logic class – Corresponds to the logic run by the thread
–Instead of creating the entire component in scoped memory, we create the inner logic class in a scoped memory region, mio
–This logic class is associated with the thread at creation time
ORB Stage processing
time
I/O Scope
ORB Scope
NETWORK
new GIOPMessage ()
The thread enters ORB scope to parse request
parseAndProcessRequest()
I/O Scope
ORBScope
Scope StackNested inner scope: all refs from ORB -> I/O are valid
ORB Scope• Participants Message parsers,
CDR Streams
• RTSJ application –The appropriate message parser associated to parse request–The message parser and buffer created in a nested memory region, morb.
–Using RTSJ memory rules, references from the ORB to the I/O space are valid
POA Stage processing
time
I/O Scope
ORB Scope
NETWORK
new GIOPMessage ()parseAndProcessRequest()
POA Scope
Enter POA scope to process request and send response
performUpCall()
Up-call related objects created in this scope
I/O Scope
Scope Stack
POAScope
ORBScope
Steps•Demux request to get target POA, servant and skeleton
•Perform upcall on the servant marshal reply back to client
RTSJ Application•Message parser – parses the request to find target servant and skeleton
•Set up context for the upcall
•Upcall Object – holds info necessary to perform upcall
•Output buffer – holds response
Talk Outline
– Tech transitions in the DRE domain Tech transitions in the DRE domain Real-time Real-time middleware & Real-time Javamiddleware & Real-time Java
− RT-Java + real-time middleware RT-Java + real-time middleware ZEN project ZEN project− Design & ArchitectureDesign & Architecture− Applying Real-time Java featuresApplying Real-time Java features − Empirical ResultsEmpirical Results− Open challengesOpen challenges
− Concluding remarks and ReferencesConcluding remarks and References
Predictability EnhancementOverview• POA Demultiplexing experiment conducted to measure improvement in predictability
Result Synopsis•Average Measures:
• Scoped Memory does have some overhead ~ 3 s
•Dispersion Measures:• Considerable improvement in
predictability • Dispersion improves by a ~
factor of 4
• Worst-Case Measures:• Scoped memory bounds
worst case• Heap shows marked
variability
Associating scoped memory – Does not compromise performance– Significantly enhances predictability– Bounds worst case latency
Associating scoped memory – Does not compromise performance– Significantly enhances predictability– Bounds worst case latency
Enter Exit Analysis
Overview• Quantify overhead incurred by using Scope Memory:
• enter () – entering scope region• exit () – leave the scope region
Result Synopsis•Average Measures:
• Constant enter () time across all message sizes.
• exit () time increases with message size
•Dispersion Measures:• exit () methods incur
considerable variability when compared to enter ()
•Worst-Case Measures:• Similar behavior to both enter
and exit () time
On exit finalizers of objects run, hence larger messages have higher average latency– enter () time uses constant time O(1) algorithm to validate illegal entry
On exit finalizers of objects run, hence larger messages have higher average latency– enter () time uses constant time O(1) algorithm to validate illegal entry
Roundtrip Latency Analysis
Overview• Influence of Scoped memory in the
Roundtrip latency measures
Result Synopsis• Average Measures:
• For smaller clients, scoped memory incurs greater overhead
• As requests increase, Scoped memory outperforms heap
• Dispersion Measures:• Considerable improvement in
predictability • Dispersion improves as much as
50%
• Worst-Case Measures:• Scoped memory bounds worst
case• Though mean values are greater
99% and Worst case measures are bounded
– As number of requests increase, GC activity increases for Heap Memory.
– Scope memory kicks in to reduce GC activity thereby improving processing time
– As number of requests increase, GC activity increases for Heap Memory.
– Scope memory kicks in to reduce GC activity thereby improving processing time
Concluding Remarks & Future Work
Future Real-Time CORBA Research•Policies at the POA level for RTSJ aware
users•Use NHRT threads for request/response
processing•Threading Models for RTSJ•Modeling RTSJ exceptions e.g. ScopedCycleException
•Complete implementation of Real-time CORBA specification
Future Real-Time CORBA Research•Policies at the POA level for RTSJ aware
users•Use NHRT threads for request/response
processing•Threading Models for RTSJ•Modeling RTSJ exceptions e.g. ScopedCycleException
•Complete implementation of Real-time CORBA specification
Concluding Remarks•We presented R&D efforts on
integration of RTSJ and RT-CORBA •Our efforts focus towards effective use
of RTSJ and Real-time CORBA to improve QoS for Java based real-time systems
Concluding Remarks•We presented R&D efforts on
integration of RTSJ and RT-CORBA •Our efforts focus towards effective use
of RTSJ and Real-time CORBA to improve QoS for Java based real-time systems
Downloading ZEN• www.zen.uci.edu
Downloading ZEN• www.zen.uci.edu
References•ZEN open-source download & web page:
•http://www.zen.uci.edu
•Real-time Java (JSR-1):•http://java.sun.com/aboutJava/communityprocess/jsr/
jsr_001_real_time.html
•Dynamic scheduling RFP:•http://www.omg.org/techprocess/meetings/schedule/ Dynamic_Scheduling_RFP.html
•Distributed Real-time Java (JSR-50):•http://java.sun.com/aboutJava/communityprocess/jsr/ jsr_050_drt.html
•AspectJ web page:•http://www.aspectJ.org
•JRate •http://tao.doc.wustl.edu/~corsaro/jRate/