Presented at Vanderbilt University, Nashville, TN March 19, 2002 Model Driven Synthesis and End-to- end Provisioning of QoS-enabled Middleware Aniruddha

Presented at

Vanderbilt University, Nashville, TN

March 19, 2002

Model Driven Synthesis and End-to-end Provisioning of QoS-enabled

Middleware Aniruddha Gokhale

ISIS, Vanderbilt [email protected]/

~gokhale

2

Aniruddha Gokhale MIC-Middleware

Research SynopsisModel, synthesize, and provision middleware technologies at multiple layers for distributed systems that require1.Simultaneous control of multiple QoS properties end-to-end &2.Secure, controlled access to resources from multiple providers

Hardware

Middleware

OS & Protocols

Applications

Research Challenge Research Approach Impact•Domain-specific middleware services shielding lower-level middleware

•Model description•Code generation mapping to lower level middleware

•Model-driven approach to resource management

•Standardized middleware service for fault tolerance

• Influence standards body from experience with prototype

•Fault Tolerant middleware Standard

•High-Performance, Real-time Distribution Middleware

• Iterative process involving benchmarking, quantifying/profiling, and optimizing

•QoS Enabled Middleware

•Network Middleware •DiffServ, MPLS •Standards-based

•Robust Middleware •Based on patterns •Dependable, Extensible

3


Motivating Technology Forces

WIDE AREANETWORK

SA TE L L ITE ST R A C K IN G

ST A T IO NP E E R S

STA TUS INF O

C O M M AN D S B U L K D AT A

TR A N SF E R

LOCAL AREA NETWORK

GROUNDSTATION

PEERS

GATEWAY

• Distributed systems are becoming more complex & mission-critical

• Increasing demand for COTS-based multi-dimensional quality of service (QoS) and resource support

• simultaneous QoS requirements for efficiency, predictability, scalability, security, & dependability

• Simultaneous resource requirements such as bandwidth, storage, compute power

IOM

IOM

IOM

IOM

IOM

IOM

IOM

IOM

IOM

IOM

IOM

IOM

IOM

IOM

IOM

IOM

IOM

IOM

BSE

BSE

BSE

BSE

BSE

BSE

BSEBSE

BSE

Total Ship C&C Center

Total Ship Computing Environments • Hardware is becoming faster & cheaper

• Ubiquitous & affordable wireless/wireline broadband internet connectivity

4


MIDDLEWARE

Solution: DOC Middleware

InfrastructureMiddleware

Platform independent: e.g., ACE, Java VMs, RogueWave, Microsoft CLR

Platform independent: e.g., ACE, Java VMs, RogueWave, Microsoft CLR

Operating Systems & Protocols

Realtime OS e.g., VxWorks, PSOSGeneral OS e.g., Solaris, Win2K, LinuxRealtime OS e.g., VxWorks, PSOSGeneral OS e.g., Solaris, Win2K, Linux

Hardware/Networks Pentium, DiffServ/MPLS in COTS N/WsPentium, DiffServ/MPLS in COTS N/Ws

Domain-SpecificMiddleware Services

Network mgmt, SIP, Location-based services, web hosting services, ERP

Network mgmt, SIP, Location-based services, web hosting services, ERP

Applications Collaboration, B2B, Mission training Collaboration, B2B, Mission training

DistributionMiddleware

RT/FT CORBA, Java RMI, COM+RT/FT CORBA, Java RMI, COM+

Common Middleware Services

Fault tolerance, Notification, Naming, Event, Logging, Transaction, etcFault tolerance, Notification, Naming, Event, Logging, Transaction, etc


Middleware Example: The ACE ORB (TAO)

www.isis.vanderbilt.edu/~gokhale/TAO.html

• TAO is an open-source version of Real-time CORBA

• TAO Synopsis • >> 1,000,000 SLOC

• 100+ person years of effort

• Pioneered R&D on middle-ware design & optimizations

• TAO is basis for many middleware R&D efforts

• Example of good synergy between researchers & practitioners

6


Part IDistribution Middleware Contributions

Work done while at

Washington University

7


End-to-end QoS Requirements Challenges

Design Challenges Reducing presentation

layer overhead Reducing demultiplexing

and dispatching overhead

Maintaining small footprint

Specifying QoS requirements

Assuring predictability, dependability, and scalability

8


Problem: Reducing Marshaling Costs• Marshaling engine

Based on SunSoft implementation

Uses TypeCode interpreter to encode/decode data types

• Design Challenges Small memory footprint Predictable, efficient

performance Minimize interpreter

overhead Must be IIOP-compliant

• Benchmarking Results ACM Sigcomm96, IEEE

Globecom96, ICDCS97

9


Observed Throughput Before Optimizations• ATM testbed 155Mbps• 64 Mbytes of

“oneway” data using IDL sequences

• Different data types e.g., short, long, structs

• Baseline comparison with TTCP

struct BinStruct {

short s;

long l;

char c;

octet o;

float f;

double d; };

10


Analysis of Initial Performance ResultsProblem• Small “mistakes” are

costly over high-speed networks

• Optimizing complex software is hard

Solution• White-box metrics to

pinpoint sources of overhead e.g., Quantify

• Apply optimization principles iteratively

• Validate via white-box and black-box metricsWhite-box analysis for structs

11


Design patterns capture the static & dynamic roles & relationships in solutions that occur repeatedly

Architectural patterns express a fundamental structural organization for software systems that provide a set of predefined subsystems, specify their relationships, & include the rules and guidelines for organizing the relationships between them

Optimization principle patterns document rules for avoiding common design & implementation mistakes that degrade performance

Patterns formalize expert knowledge to help generate software architectures by capturing recurring structures & dynamics and resolving common design forces

Patterns Driven Optimizations

www.posa.uci.edu/

12


Optimization Principle Patterns

• Related work G. Varghese – Sigcomm 96 Clark:89 – TCP header

prediction Peterson:94 (PathFinder),

Engler:96 (DPF), Mahesh:95 Clark:90, Abott:93 – ILP Peterson:96 – outlining O’Malley:94 – USC stub

compiler Lepreau – Flick IDL compiler Hoschka:97 – hybrid stubs

Principle Pattern

Optimize for the common case

Eliminate gratuitous waste

Precompute values, when possible

Use redundant state to speed up expensive operations

Pass hints across layers

Optimize for cache by outlining

13


Throughput Improvements After Optimizations

Best paper award at HICSS-97 Software Trackwww.isis.vanderbilt.edu/~gokhale/PDF/HICSS-97.pdf

BEFORE AFTER

14


Problem: Footprint vs. Performance Tradeoff

• Related work O’Malley:94 –

USC stub compiler Lepreau – Flick

IDL compiler Hoschka:97 –

hybrid stubs

• Design Challenges Interpreted marshaling => smaller footprint but lower

performance Compiled marshaling => larger footprint but higher

performance Predictable, efficient performance, IIOP compliance

15


Compiled vs. Interpreted MarshalingPERFORMANCE FOOTPRINT

www.isis.vanderbilt.edu/~gokhale/PDF/JSAC-99.pdfwww.isisvanderbilt.edu/~gokhale/PDF/Infocom-99.pdf

16


Problem: Reducing Demultiplexing Latency

• Design Challenges• Minimize

demultiplexing layers• Provide predictable

and scalable demultiplexing

• Avoid priority inversions

• Remain CORBA-compliant

17


Solution: De-layered Active Demultiplexing and Perfect Hashing

• Solution Approach• Pre-negotiate

demultiplexing keys• Tunnel the demux

keys with object key• Use perfect hash

functions for operation demuxing since operation names are known apriori

www.isis.vanderbilt.edu/~gokhale/PDF/Globecom-97.pdfwww.isis.vanderbilt.edu/~gokhale/PDF/COOTS-99.pdf

18


Optimized Demultiplexing Results

RANDOM WORST-CASE

www.isis.vanderbilt.edu/~gokhale/PDF/ieee_tc_97.pdf


Summary of CORBA Middleware Optimizations

RTAS98, Journal of RT Systems, IEEE Concurrency, Comm

20


Part IIMiddleware Services Contributions

Work done while at

Bell Laboratories


Fault Tolerant CORBA Contributions Influenced the Fault

tolerant CORBA standard by incorporating principles and patterns from a prototype called DOORS

Patterns-based optimizations in DOORS provides high performance & predictability to DRE systems

DOORS software available at www.bell-labs.com/topic/swdist

www.isis.vanderbilt.edu/~gokhale/PDF/DOA-2000.pdfwww.isis.vanderbilt.edu/~gokhale/PDF/HIPC-2000.pdf

22


Part IIIDomain-specific Middleware Services

Ongoing and future

work at ISIS

Related work including Network Call Centers and Web Services done while at Bell Laboratories

23


Key open challenge

End-to-end, multiple QoS and resource guarantees

Modalitiese.g., MRI, CT,

CR, Ultrasound, etc.

Problem: Qos for Collaborative Grid Applications

24


Solution: QoS-enabled Grid MiddlewareGrid Applications• Multiple QoS properties• Secure, controlled

access to resources from multiple service providers => need SLAs

• E.g., collaborative scientific applications

Grid Service Provider GSP• Shields applications from

multiple service providers• Manages and provisions

resources from multiple resource providers

• Provides web-enabled service creation and provisioning environment

NSF NMI proposal, March 2002 and WWW-2002 paper

25


Resource Reservation in GSP

Challenges • Bandwidth as a resource• Mapping application QoS requirements into DiffServ classes and MPLS

labels• GSP middleware for network bandwidth reservation at Edge Routers

Current Related Collaboration• With researchers at Bell Labs on QoS monitoring in all-IP MPLS networks

26


Part IVModel Driven Synthesis and End-to-end

Provisioning

Ongoing and future

work at ISIS

Related work on model driven network management done while at Bell Laboratories

27


Problem: Using QoS-enabled Middleware• Configuring middleware QoS

parameters is hard• Needs expert understanding of

middleware layers and their APIs => dealing with accidental complexities

• Steep learning curve for application developers

• Applications get tightly coupled with the underlying middleware technology

• Deal with competing technologies e.g., CORBA CCM, EJB/RMI, COM+, XML/SOAP

• Deal with disruptive technologies

Common Middleware Services

DistributionMiddleware

InfrastructureMiddleware

Operating Systems & Protocols

Hardware

Domain-SpecificMiddleware Services

Application

28


• Meta-programming techniques, e.g., UML/XML

• Model-integrated computing e.g., GME

• Middleware-centric pattern languages

• Platform independence, interoperability

• Sophisticated compiler techniques & code generation tools

Open R&D question: How can we apply MDA to distributed, real-time systems most effectively

www.omg.org/mda

• Consequence of codifying years of R&D efforts in:

Candidate Solution: Model Driven Architecture


Model & Component Integrated ACE ORB - MIAO,CIAO

1. Configuring and deploying application services end-to-end

2. Composing components into application server components

3. Configuring application component containers

4. Synthesizing application component implementations

5. Synthesizing middleware-specific configurations

6. Synthesizing middleware implementations

www.isis.vanderbilt.edu/~gokhale/PDF/CACM02.pdf

• Based on Model Integrated Computing and CORBA Component Model

• Uses OMG MDA standards

30


•Researchers & developers of distributed systems face common challenges, e.g.:

•Carefully applying these techniques can yield efficient, scalable, predictable, dependable, & flexible middleware & applications

•Connection management, service initialization, error handling, flow control, event demuxing, distribution, concurrency control, fault tolerance, synchronization, scheduling, & persistence

•Model Driven Architecture intends to resolve the technological and economic forces, however …

Concluding Remarks

•The application of patterns, frameworks, & components can help to resolve these challenges

Key challenge• Patterns bridging the gap between models and middleware

• Model-driven multidimensional QoS and resource management

Hardware

Middleware

OS & Protocols

Applications

31


EXTRA SLIDES

32


Overview of Fault Tolerant CORBAOverview•Provides a standard set of CORBA interfaces, policies, & services

•Entity Redundancy of objects is used for fault tolerance via•Replication•Fault detection & •Recovery from failure

Features•Inter-Operable Group References (IOGR)

•Replication Manager•Fault Detector & Notifier•Message Logging for recovery•Fault tolerance Domains

33


ORB Core Optimizations

Optimization Opportunities to Improve Fault Tolerant CORBA Performance

CORBA Service Optimizations

•Efficient IOGR parsing & connection establishment

•Reliable handling & ordering of GIOP messages

•Predictable behavior during transparent connection establishment & retransmission

•Tracking requests with respect to the server object group

•Support for dynamic system configuration•Bounded recovery time•Minimize overhead of FT CORBA components

34


Effect of Polling Interval on Total Recovery Time

Analysis• Average failure detection time is

half the polling interval• Replica Group Management time

is constant

Key Challenge• Minimize replica group

management time• Setting the right polling interval

Recovery Time = Failure detection time + Replica Group Management TimeRecovery Time = Failure detection time + Replica Group Management Time

35


Problem: Missed polls

Fault Detector

PollingThread

Replica

Replica

Replica

Replica

FaultNotifier

HANGS

HANGS

Naïve implementation of Polling Thread

while (true) {

for each obj to be monitored {

poll the object;

if (no response) {

report fault to notifier

}

}

sleep (polling_interval);

} Allocate new thread per monitored object?

36


Predictable & Scalable Fault MonitoringContext• Periodic polling of several objects & fault notification done in the same polling thread can block the thread

Forces•Must guarantee polling of all registered objects

•Must minimize concurrency overhead

•Must be scalable

Solution •Leader-Followers and ACT architectural pattern

Solution •Leader-Followers and ACT architectural pattern

Problem•Blocking can cause missed polls

AMI

37


Leader/Followers Pattern Dynamics: Concrete

Event Handler

join()

handle_event()

: ThreadPool

: HandleSet

join()

thread 2 sleepsuntil it becomesthe leader

event

thread 1 sleepsuntil it becomesthe leader

deactivate_handle()

join()

Thread 1 Thread 2

handle_events() reactivate_

handle()

handle_event()

event

thread 2waits for anew event,thread 1processescurrentevent

deactivate_handle()

handle_events()

new_leader()

1.Leader thread demuxing

2.Follower thread promotion

3.Event handler demuxing & event processing

4.Rejoining the thread pool

promote_

38


Asynchronous Completion Token PatternDynamic Interactions

handle_event()

•Together with each async operation that a client initiator invokes on a service, transmit information that identifies how the initiator should process the service’s response

•Return this information to the initiator when the operation finishes, so that it can be used to demux the response efficiently, allowing the initiator to process it accordingly

39


Context•RM needs to handle bursts of fault reports

•Handle faults based on importance

Forces•Predictable amount of time for failure recovery irrespective of burst size

•Prioritize fault handling

Problem•Reduced responsiveness

•Wasted resources during normal conditions

•Lack of handling faults based on importance

Solution•Apply the Thread Pool Active Object design pattern

Solution•Apply the Thread Pool Active Object design pattern

Challenge: Scalable, Prioritized Fault Recovery

40


• A fault notifier pushes the fault event to the RM proxy

• The proxy returns a future to the client, & creates a method request, which it passes to the scheduler

• The scheduler enqueues the method request into the activation list (not shown here)

• When the method request becomes runnable, the scheduler dequeues it from the activation list (not shown here) & executes it in a different thread than the client

• The method request executes the method on the servant & writes results, if any, to the future

• Clients obtain the method’s results via the future

Active Object Pattern Dynamics

: Future

method

enqueue

: RMProxy

: Scheduler : RM

: MethodRequest

dispatch call method

read

write

: FaultNotifier

Clients can obtain result from futures via blocking, polling, or callbacksClients can obtain result from futures via blocking, polling, or callbacks

41


GSP User Interface

• Web services user interface• Session-initiation protocol for

creating/joining/leaving collaborative Grid sessions

42


Challenge: Resolving Economic Forces

• Reduce/amortize costs associated with existing product lifecycles

• Rapid Time-to-market of high quality products

• Platform independent, highly interoperable design

• Minimal effort in incorporation of upcoming/new enabling technologies

• Less spending by customers => Shrinking revenues and profit margins

• Maintaining competitive advantage and higher returns on investment

• Plethora of competing technologies e.g., CORBA CCM, EJB/RMI, COM+, XML/SOAP

• Dealing with disruptive technologies or unprecendented hype

Context Solution

Documents

Presented at Vanderbilt University, Nashville, TN March 19, 2002 Model Driven Synthesis and End-to- end Provisioning of QoS-enabled Middleware Aniruddha