Pattern-Oriented Software Architecture · Pattern-Oriented Software Architecture Applying Concurrent & Networked Objects to Develop & Use Distributed Object Computing Middleware Dr

Pattern-Oriented Software ArchitectureApplying Concurrent & Networked Objects

to Develop & UseDistributed Object Computing Middleware

Pattern-Oriented Software ArchitecturePattern-Oriented Software ArchitectureApplying Concurrent & Networked Objects

to Develop & UseDistributed Object Computing Middleware

Dr. Douglas C. [email protected]

http://www.posa.uci.edu/

Electrical & Computing Engineering DepartmentThe Henry Samueli School of Engineering

University of California, Irvine

Montag, 19. April 2004

2

Distributed Architecture

Middleware Patterns Tutorial Outline

Describe OO techniques & languagefeatures to enhance software quality

Stand-aloneArchitecture

Illustrate how/why it’s hard to build robust, efficient,& extensible concurrent & networked applications• e.g., we must address many complex topics that are lessproblematic for non-concurrent, stand-alone applications

OO techniques & language features include:•Patterns (25+), which embody reusable softwarearchitectures & designs

•Frameworks & components, which embody reusablesoftware implementations

•OO language features, e.g., classes, inheritance &dynamic binding, parameterized types & exceptions

Tutorial Organization1. Background & motivation2. Concurrent & network

challenges & solutionapproaches

3. Multiple case studies4. Wrap-up and Q&A

3

CPUs and networks haveincreased by 3-7 orders ofmagnitude in the past decade2,400 bits/sec to

1 Gigabits/sec

10 Megahertz to1 Gigahertz

These advances stemlargely from standardizinghardware & software APIsand protocols, e.g.:

The Road Ahead

• TCP/IP, ATM• POSIX & JVMs• Middleware & components

• Intel x86 & Power PC chipsets

• Ada, C, C++, RT Java

Extrapolating this trend to2010 yields

• ~100 Gigahertz desktops• ~100 Gigabits/sec LANs• ~100 Megabits/sec wireless• ~10 Terabits/sec Internetbackbone

In general, software has notimproved as rapidly or aseffectively as hardware

In general, software has notimproved as rapidly or aseffectively as hardware

Increasing software productivityand QoS depends heavily on COTSIncreasing software productivityand QoS depends heavily on COTS

4

Addressing the COTS “Crisis”

However, this trend presents many vexingR&D challenges for mission-criticalsystems, e.g.,• Inflexibility and lack of QoS• Security & global competition

Distributed systems must increasinglyreuse commercial-off-the-shelf (COTS)hardware & software• i.e., COTS is essential to R&D success

Why we should care:

•Despite IT commoditization, progress inCOTS hardware & software is often notapplicable for mission-criticaldistributed systems

•Recent advances in COTS softwaretechnology can help to fundamentallyreshape distributed system R&D

5

R&D Challenges & Opportunities

High-performance, real-time,fault-tolerant, & secure systems

IOM

IOM

IOM

IOM

IOM

IOM

IOM

IOM

IOM

IOM

IOM

IOM

IOM

IOM

IOM

IOM

IOM

IOM

BSE

BSE

BSE

BSE

BSE

BSE

BSEBSE

BSE

Power-aware ad hoc,mobile, distributed, &embedded systems

Middleware,Frameworks, &Components

Patterns &PatternLanguages

Standards& Open-source

Challenges Opportunities

Autonomous systems

6

The Evolution of COTS

There are multiple COTSlayers & research/

business opportunities

Historically, mission-critical apps werebuilt directly atop hardware

The domain-specific services layer iswhere system integrators can provide themost value & derive the most benefits

The domain-specific services layer iswhere system integrators can provide themost value & derive the most benefits

Standards-based COTS middleware helps:•Manage end-to-end resources•Leverage HW/SW technology advances•Evolve to new environments & requirements

& OS•Tedious, error-prone, & costly over lifecycles

Prior R&D efforts have address some, but byno means all, of these issues

• Layered QoS specification& enforcement

• Separating policies &mechanisms across layers

• Time/space optimizationsfor middleware & apps

• Multi-level globalresource mgmt. &optimization

• High confidence• Stable & robustadaptive systems

Key R&D challenges include:

7

•More emphasis on integration ratherthan programming

•Increased technology convergence &standardization

•Mass market economies of scale fortechnology & personnel

•More disruptive technologies & globalcompetition

•Lower priced--but often lower quality--hardware & software components

•The decline of internally funded R&D•Potential for complexity cap in next-generation complex systems

Consequences of COTS& IT Commoditization

Not all trends bode well forlong-term competitivenessof traditional R&D leaders

Ultimately, competitiveness will dependon success of long-term R&D efforts oncomplex distributed & embedded systems

8

Why middleware-centric reuse works1.Hardware advances

•e.g., faster CPUs & networks2.Software/system architecture

advances•e.g., inter-layer optimizations &meta-programming mechanisms

3.Economic necessity•e.g., global competition forcustomers & engineers

Why We are Succeeding NowRecent synergistic advances in fundamentals:

Revolutionary changes in softwareprocess: Open-source, refactoring,extreme programming (XP), advancedV&V techniques

Patterns & Pattern Languages:Generate software architecturesby capturing recurring structures& dynamics & by resolvingdesign forces

Standards-based QoS-enabledMiddleware: Pluggable service &micro-protocol components &reusable “semi-complete”application frameworks

9

Example:

Applying COTS in Real-time Avionics

Key System Characteristics•Deterministic & statistical deadlines

•~20 Hz•Low latency & jitter

•~250 usecs•Periodic & aperiodic processing•Complex dependencies•Continuous platform upgrades

•Test flown at China Lake NAWS by BoeingOSAT II ‘98, funded by OS-JTF• www.cs.wustl.edu/~schmidt/TAO-boeing.html

•Also used on SOFIA project by Raytheon• sofia.arc.nasa.gov

•First use of RT CORBA in mission computing•Drove Real-time CORBA standardization

•Test flown at China Lake NAWS by BoeingOSAT II ‘98, funded by OS-JTF• www.cs.wustl.edu/~schmidt/TAO-boeing.html

•Also used on SOFIA project by Raytheon• sofia.arc.nasa.gov

•First use of RT CORBA in mission computing•Drove Real-time CORBA standardization

Key Results

Goals•Apply COTS & open systems tomission-critical real-time avionics

10

Time-critical targets require immediate response because:•They pose a clear and present danger to friendly forces &•Are highly lucrative, fleeting targets of opportunity

Example:

Applying COTS to Time-Critical Targets

Adapted from “The Future of AWACS”,by LtCol Joe Chapa

Joint ForcesGlobal Info Grid

Joint ForcesGlobal Info Grid Challenge

is to make this possible!

Challenges arealso relevant toTBMD & NMD

Goals• Detect, identify,track, & destroytime-criticaltargets

• Real-time mission-criticalsensor-to-shooter needs

• Highly dynamic QoSrequirements & environmentalconditions

• Multi-service & assetcoordination

Key SystemCharacteristics

Key Solution Characteristics•Efficient & scalable•Affordable & flexible•COTS-based

•Efficient & scalable•Affordable & flexible•COTS-based

•Adaptive & reflective•High confidence •Safety critical

•Adaptive & reflective•High confidence •Safety critical

Example:

Applying COTS to Large-scale Routers

IOM

IOM

IOM

IOM

IOM

IOM

IOM

IOM

IOM

IOM

IOM

IOM

IOM

IOM

IOM

IOM

IOM

IOM

BSE

BSE

BSE

BSE

BSE

BSE

BSEBSE

BSE Goal•Switch ATM cells +IP packets at terabitrates

Key Software Solution Characteristics

•High confidence & scalable computing architecture•Networked embedded processors•Distribution middleware•FT & load sharing•Distributed & layered resource management

•Affordable, flexible, & COTS

•High confidence & scalable computing architecture•Networked embedded processors•Distribution middleware•FT & load sharing•Distributed & layered resource management

•Affordable, flexible, & COTS

Key SystemCharacteristics•Very high-speed WDMlinks

•102/103 line cards•Stringent requirementsfor availability

•Multi-layer loadbalancing, e.g.:•Layer 3+4•Layer 5

www.arl.wustl.edu

12

Example:

Applying COTS to Hot Rolling MillsGoals•Control the processing of moltensteel moving through a hot rollingmill in real-time

System Characteristics•Hard real-time process automationrequirements• i.e., 250 ms real-time cycles

•System acquires valuesrepresenting plant’s current state,tracks material flow, calculates newsettings for the rolls & devices, &submits new settings back to plant


•Affordable, flexible, & COTS•Product-line architecture•Design guided by patterns & frameworks


•Windows NT/2000•Real-time CORBA (ACE+TAO)

www.siroll.de

13

Example:

Applying COTS to Real-time Image Processing

Goals•Examine glass bottlesfor defects in real-time

SystemCharacteristics•Process 20 bottlesper sec• i.e., ~50 msec perbottle

•Networkedconfiguration

•~10 camerasKey Software Solution Characteristics

•Affordable, flexible, & COTS•Embedded Linux (Lem)•Compact PCI bus + Celeron processors

•Affordable, flexible, & COTS•Embedded Linux (Lem)•Compact PCI bus + Celeron processors

•Remote booted by DHCP/TFTP•Real-time CORBA (ACE+TAO)

www.krones.com

14

Concurrency & SynchronizationMotivations•Leveragehardware/softwareadvances

•Simplify programstructure

• Increaseperformance

• Improve response-time

Key Opportunities & Challenges inConcurrent & Networked Applications

Networking & DistributionMotivations• Collaboration• Performance• Reliability & availability• Scalability & portability• Extensibility• Cost effectiveness

Inherent Complexities•Latency•Reliability•Load balancing•Scheduling•Causal ordering•Synchronization•Deadlocks

Accidental Complexities•Low-level APIs•Poor debugging tools•Algorithmicdecomposition

•Continuous re-invention& re-discover of coreconcepts & components

15

•Present solutions to common softwareproblems arising within a certain context

Overview of Patterns & Pattern Languages

• Define a vocabulary fortalking about softwaredevelopment problems

• Provide a process for theorderly resolution of theseproblems

• Help to generate & reusesoftware architectures

Patterns

Pattern Languages

•Help resolve key design forces

•Flexibility

•Extensibility

•Dependability

•Predictability

•Scalability

•Efficiency

The Proxy Pattern

•Capture recurring structures & dynamicsamong software participants to facilitatereuse of successful designs

•Generally codify expert knowledge ofdesign constraints & “best practices”

www.posa.uci.edu

16

Software Design Abstractions forConcurrent & Networked Applications

Problem•Distributed application functionalityis subject to change since it isoften reused in unforeseencontexts, e.g.,•Accessed from different clients•Run on different platforms•Configured into different run-timecontexts

A component is anencapsulation unit withone or more interfacesthat provide clients withaccess to its services

A framework is an integratedcollection of classes thatcollaborate to produce areusable architecture for afamily of related applications

A class is a unit ofabstraction &implementation inan OOprogramminglanguage

MIDDLEWARE

Solution•Don‘t structure distributed applications as amonoliths, but instead decompose them intoclasses, frameworks, & components

Solution•Don‘t structure distributed applications as amonoliths, but instead decompose them intoclasses, frameworks, & components

17

A Comparison of Class Libraries,Frameworks, & Components

ClassLibraries

Frameworks

Macro-levelMeso-levelMicro-level

Borrow caller’sthread

Inversion ofcontrol

Borrowcaller’s thread

Domain-specific orDomain-independent

Domain-specific

Domain-independent

Stand-alonecomposition

entities

“Semi-complete”

applications

Stand-alonelanguageentities

Components

Class Library Architecture

ADTs

Strings

LocksIPC

MathLOCAL INVOCATIONSAPPLICATION-

SPECIFICFUNCTIONALITY

EVENTLOOP

GLUECODE

Files

GUI

Logging

Component ArchitectureLOCAL/REMOTE INVOCATIONSAPPLICATION-

SPECIFICFUNCTIONALITY

EVENTLOOP

GLUECODE

Naming

Trading

Events

Locking

Framework Architecture

ADTs

Locks

INVOKES

Strings

Files

18

Overview of the ACE FrameworkFeatures•Open-source•200,000+ linesof C++

•30+ person-years of effort

•Ported toWin32, UNIX, &RTOSs

• e.g., VxWorks,pSoS, LynxOS,Chorus, QNX

•Large open-source user community•www.cs.wustl.edu/~schmidt/ACE-users.html

•Commercial support by Riverace• www.riverace.com/

www.cs.wustl.edu/~schmidt/ACE.html

19

Event HandlingService Access & Control

Concurrency Synchronization

Key Capabilities Provided by ACE

20

Observation•Failure rarely resultsfrom unknown scientificprinciples, but fromfailing to apply provenengineering practices &patterns

Benefits of POSA2 Patterns•Preserve crucial designinformation used byapplications & underlyingframeworks/components

•Facilitate design reuse•Guide design choices forapplication developers

URL for POSA Bookswww.posa.uci.eduURL for POSA Bookswww.posa.uci.edu

The POSA2 Pattern Language

21

POSA2 Pattern AbstractsService Access & Configuration Patterns

The Wrapper Facade design patternencapsulates the functions and data provided byexisting non-object-oriented APIs within moreconcise, robust, portable, maintainable, andcohesive object-oriented class interfaces.

The Component Configurator design patternallows an application to link and unlink itscomponent implementations at run-time withouthaving to modify, recompile, or statically relinkthe application. Component Configurator furthersupports the reconfiguration of components intodifferent application processes without having toshut down and re-start running processes.

The Interceptor architectural pattern allowsservices to be added transparently to aframework and triggered automatically whencertain events occur.

The Extension Interface design pattern allowsmultiple interfaces to be exported by acomponent, to prevent bloating of interfaces andbreaking of client code when developers extendor modify the functionality of the component.

Event Handling Patterns

The Reactor architectural pattern allows event-driven applications to demultiplex and dispatchservice requests that are delivered to anapplication from one or more clients.

The Proactor architectural pattern allowsevent-driven applications to efficientlydemultiplex and dispatch service requeststriggered by the completion of asynchronousoperations, to achieve the performancebenefits of concurrency without incurringcertain of its liabilities.

The Asynchronous Completion Token designpattern allows an application to demultiplexand process efficiently the responses ofasynchronous operations it invokes onservices.

The Acceptor-Connector design patterndecouples the connection and initialization ofcooperating peer services in a networkedsystem from the processing performed by thepeer services after they are connected andinitialized.

22

POSA2 Pattern Abstracts (cont’d)Synchronization Patterns

The Scoped Locking C++ idiomensures that a lock is acquired whencontrol enters a scope and releasedautomatically when control leaves thescope, regardless of the return pathfrom the scope.

The Strategized Locking design patternparameterizes synchronizationmechanisms that protect a component’scritical sections from concurrentaccess.

The Thread-Safe Interface designpattern minimizes locking overhead andensures that intra-component methodcalls do not incur ‘self-deadlock’ bytrying to reacquire a lock that is held bythe component already.

The Double-Checked LockingOptimization design pattern reducescontention and synchronizationoverhead whenever critical sections ofcode must acquire locks in a thread-safe manner just once during programexecution.

Concurrency Patterns

The Active Object design pattern decouples methodexecution from method invocation to enhance concurrencyand simplify synchronized access to objects that reside intheir own threads of control.

The Monitor Object design pattern synchronizes concurrentmethod execution to ensure that only one method at a timeruns within an object. It also allows an object’s methods tocooperatively schedule their execution sequences.

The Half-Sync/Half-Async architectural pattern decouplesasynchronous and synchronous service processing inconcurrent systems, to simplify programming withoutunduly reducing performance. The pattern introduces twointercommunicating layers, one for asynchronous and onefor synchronous service processing.

The Leader/Followers architectural pattern provides anefficient concurrency model where multiple threads taketurns sharing a set of event sources in order to detect,demultiplex, dispatch, and process service requests thatoccur on the event sources.

The Thread-Specific Storage design pattern allows multiplethreads to use one ‘logically global’ access point to retrievean object that is local to a thread, without incurring lockingoverhead on each object access.

23

Example of Applying Patterns & Frameworks:

Real-time CORBA & The ACE ORB (TAO)TAO Features•Open-source•500+ classes &500,000+ lines of C++

•ACE/patterns-based•30+ person-years ofeffort

•Ported to UNIX,Win32, MVS, & manyRT & embedded OSs

• e.g., VxWorks, LynxOS,Chorus, QNX

www.cs.wustl.edu/~schmidt/TAO.html

Protocol Properties Explicit Binding

Thread Pools

Scheduling Service

Standard Synchronizers

Portable Priorities

•Large open-source user community•www.cs.wustl.edu/~schmidt/TAO-users.html

•Commercial support by OCI• www.theaceorb.com/

End-to-end Priority Propagation

24

Tutorial Example 1:

Electronic Medical Imaging Systems

Modalitiese.g., MRI, CT, CR,

Ultrasound, etc.

www.syngo.com

Goal•Route, manage, & manipulateelectronic medical images robustly,efficiently, & securely thoughout adistributed environment

System Characteristics•Large volume of “blob” data

•e.g.,10 to 40 Mps•“Lossy” compression isn’t viabledue to liability concerns

•Diverse QoS requirements, e.g.,• Synchronous & asynchronouscommunication

• Streaming communication• Prioritization of requests & streams• Distributed resource management




•General-purpose & embeddedOS platforms

•Middleware technology agnostic

25

8. New

Modalitiese.g., MRI, CT, CR, Ultrasound, etc.

9. Return Ref

7.New

1. Deploy

3. BindFactory

Image Xfer Interface


ConfigurationDatabase

ConfigurationConfiguration

ContainerContainer

Image Acquisition Scenario

Image XferComponent

Servant

Image XferComponent

Servant

2. Enter info

6. Intercept & delegate

Xfer ProxyXfer Proxy

Factory ProxyFactory Proxy

10. Invoke get_image() call

11. Query Config.

12. Check Authorization

5. Find Image

Diagnostic & Clinical Workstations

13. Activate

Factory/FinderFactory/FinderSecurity ServiceSecurity Service

NamingServiceNamingService

4. Find Factory

RadiologyClient

RadiologyClient

14. Delegate

Key Tasks1.Image

routing2.Image

delivery

ImageDatabase

26



ConfigurationDatabase ConfigurationConfiguration

ContainerContainer

Image XferComponent

Servant

Image XferComponent

Servant

Xfer ProxyXfer Proxy

Factory ProxyFactory Proxy

ImageDatabase

Factory/FinderFactory/FinderSecurity ServiceSecurity Service


RadiologyClient

RadiologyClient

Applying Patterns to Resolve KeyDesign Challenges

Patterns help resolve the following common design challenges:•Decoupling suppliers & consumers•Providing mechanisms to find &create remote components

•Locating & creating componentseffectively

•Extending components transparently•Minimizing resource utilization•Enhancing server (re)configurability

•Decoupling suppliers & consumers•Providing mechanisms to find &create remote components

•Locating & creating componentseffectively

•Extending components transparently•Minimizing resource utilization•Enhancing server (re)configurability

•Separating concerns between tiers•Improving type-safety &performance

•Enabling client extensibility•Ensuring platform-neutral &network-transparent OO comm.

•Supporting async comm.•Supporting OO async comm.

•Separating concerns between tiers•Improving type-safety &performance

•Enabling client extensibility•Ensuring platform-neutral &network-transparent OO comm.

•Supporting async comm.•Supporting OO async comm.

Proxy

Broker

Active Object

Publisher/Subscriber

ComponentConfigurator

ExtensionInterface

Factory/Finder

AsyncForwarder/Receiver

Layers

Interceptor

Activator

27

Solution•Apply the Layers architecturalpattern to create a multi-tierarchitecture that separatesconcerns between groups ofsubtasks occurring at distinctlayers in the distributed system

Solution•Apply the Layers architecturalpattern to create a multi-tierarchitecture that separatesconcerns between groups ofsubtasks occurring at distinctlayers in the distributed system

Separating Concerns Between TiersContext• Distributed systems are nowcommon due to the advent of• The global Internet• Ubiquitous mobile & embeddeddevices

Problem• One reason it’s hard to build COTS-based distributed systems is becausea large number of capabilities mustbe provided to meet end-to-endapplication requirements

•Services in the middle-tier participatein various types of tasks, e.g.,•Workflow of integrated “business”processes

•Connect to databases & otherbackend systems for data storage& access

Database Tier• e.g., persistentdata

DBServer

DBServer

Middle Tier• e.g., commonbusiness logic

comp

comp

Application

Server

Presentation Tier• e.g., thin clients

Client Client

28

Applying the Layers Pattern toImage Acquisition

Image servers are middle tier components that:•Provide server-side functionality

•e.g., they are responsible for scalable concurrency & networking•Can run in their own address space•Are integrated into containers that hide low-level system details

Image servers are middle tier components that:•Provide server-side functionality

•e.g., they are responsible for scalable concurrency & networking•Can run in their own address space•Are integrated into containers that hide low-level system details

ImageDatabase

ImageDatabase

Database Tier•e.g., persistentimage data

Middle Tier•e.g., imagerouting & imagetransfer logic

comp

comp

Image

Server

Presentation Tier•e.g., radiologyclients

DiagnosticWorkstations

ClinicalWorkstations

Diagnostic & clinicalworkstations arepresentation tier componentsthat:•Typically representsophisticated GUIelements

•Share the same addressspace with their clients• Their clients are containersthat provide all theresources

•Exchange messages withthe middle tier components

29

Pros & Cons of the Layers Pattern

This pattern has four benefits:•Reuse of layers

• If an individual layer embodies a well-defined abstraction & has a well-defined &documented interface, the layer can bereused in multiple contexts

•Support for standardization• Clearly-defined and commonly-acceptedlevels of abstraction enable thedevelopment of standardized tasks &interfaces

•Dependencies are localized• Standardized interfaces between layersusually confine the effect of code changesto the layer that is changed

•Exchangeability• Individual layer implementations can bereplaced by semantically-equivalentimplementations without undue effort

This pattern also has liabilities:•Cascades of changing behavior

• If layer interfaces & semanticsaren’t abstracted properly thenchanges can ripple when behaviorof a layer is modified

•Lower efficiency• A layered architecture can be lessefficient than a monolithicarchitecture

•Unnecessary work• If some services performed by lowerlayers perform excessive orduplicate work not actually requiredby the higher layer, performancecan suffer

•Difficulty of establishing thecorrect granularity of layers• It’s important to avoid too many &too few layers

30

Overview of Distributed Object ComputingCommunication Mechanisms

Solution•DOC middleware provides multiple types of communication mechanisms

•Collocated client/server (i.e., native function call)•Synchronous & asynchronous RPC/IPC•Group communication•Data streaming

Problem•A single communicationmechanism does not fit alluses!

ContextIn multi-tier systems both the tiers & thecomponents within the tiers must beconnected via communication mechanisms

Next, we’ll explorevarious patterns thatapplications can applyto leverage thesecommunicationmechanisms

Next, we’ll explorevarious patterns thatapplications can applyto leverage thesecommunicationmechanisms

31

Improving Type-safety & Performance

Context• The configuration ofcomponents indistributed systems isoften subject to changeas requirements evolve

• A Service implements the object, which is notaccessible directly

• A Proxy represents the Service andensures the correct access to it• Proxy offers same interface as Service

• Clients use the Proxy to access the Service

Proxy

service

Client

Service

service1 1

AbstractService

serviceSolutionApply the Proxy design pattern toprovide an OO surrogate throughwhich clients can access remoteobjects

SolutionApply the Proxy design pattern toprovide an OO surrogate throughwhich clients can access remoteobjects

Problems• Low-level message passing is fraught withaccidental complexity

• Remote components should look like localcomponents from an application perspective

• i.e., clients & servers should be oblivious tocommunication issues

: Service: Proxy: Clientservice

service

pre-processing:marshaling

post-processing:unmarshaling

32

Applying the Proxy Patternto Image Acquisition

Proxy

get_image()

Client

Image Xfer

get_image()1 1

AbstractService

get_image()

Invoke get_image() call

We can apply the Proxy patternto provide a strongly-typedinterface to initiate & coordinatethe downloading of imagesfrom an image database

When proxies are generated automatically by middleware theycan be optimized to be much more efficient than manualmessage passing•e.g., improved memory management, data copying, &compiled marshaling/demarshaling

When proxies are generated automatically by middleware theycan be optimized to be much more efficient than manualmessage passing•e.g., improved memory management, data copying, &compiled marshaling/demarshaling

Image Xfer Service

Image Xfer ServiceXfer ProxyXfer Proxy

ImageDatabase

RadiologyClient

RadiologyClient

33

Pros & Cons of the Proxy Pattern

This pattern provides three benefits:•Decoupling clients from the locationof server components• By putting all location information &addressing functionality into a proxyclients are not affected by migration ofservers or changes in the networkinginfrastructure

•Potential for time & spaceoptimizations• Proxy implementations can be loaded “on-demand” and can also be used to cachevalues to avoid remote calls

• Proxies can also be optimized to improveboth type-safety & performance

•Separation of housekeeping &functionality• A proxy relieves clients from burdens thatdo not inherently belong to the task theclient performs

This pattern has two liabilities:•Potential overkill viasophisticated strategies• If proxies include overlysophisticated functionality they manyintroduce overhead that defeats theirintended purpose

•Less efficiency due toindirection• Proxies introduce an additional layerof indirection that can be excessive ifthe proxy implementation isinefficient

34

Enabling Client Extensibility

Context• Object models define howcomponents import & exportfunctionality•e.g., UML class diagramsspecify well-defined OOinterfaces

Problem• Many object models assign a single interfaceto each component

• This design makes it hard to evolvecomponents without• Breaking existing client interfaces• Bloating client interfaces

Solution• Apply the ExtensionInterface design pattern toallow multiple interfaces tobe exported by acomponent, to preventbloating of interfaces &breaking of client codewhen developers extendor modify componentfunctionality

Solution• Apply the ExtensionInterface design pattern toallow multiple interfaces tobe exported by acomponent, to preventbloating of interfaces &breaking of client codewhen developers extendor modify componentfunctionality

CreateComponent

Factory

*

*

CreateInstance

CreateComponent

Componentnew

*1

QueryInterface

Root

implements

1+

<<extends>>

Ask for a referenceto an interface

Call anoperation on aninterface

Initialize

unititialize

ServerQueryInterface

service_i

ExtensionInterface i

Call_service

Client

35

Extension Interface Pattern Dynamics

: Client

Start_client

: Factory

CreateInstance(Ext.Intf. 1)new

Ref. To Extension1

create

create

service_1

QueryInterface(Extension Interface 2)

Ref. To Extension2

service2

Note how each extension interface canserve as a “factory” to return objectreference to other extension interfaces

Note how each extension interface canserve as a “factory” to return objectreference to other extension interfaces

: Component : Extension 1 : Extension 2

service_2

36

Pros & Cons of theExtension Interface Pattern

This pattern has five benefits:•Separation of concerns

• Interfaces are strictly decoupled fromimplementations

•Exchangeability of components• Component implementations can evolveindependently from clients that accessthem

•Extensibility through interfaces• Clients only access components via theirinterfaces, which reduces coupling torepresentation & implementation details

•Prevention of interface bloating• Interfaces need not contain all possiblemethods, just the ones associated with aparticular capability

•No subclassing required• Delegation—rather than inheritance—isused to customize components

This pattern also has liabilities:•Overhead due to indirection

• Clients must incur theoverhead of several round-tripsto obtain the appropriate objectreference from a servercomponent

•Complexity & cost fordevelopment & deployment• This pattern off-loads theresponsibility for determiningthe appropriate interface fromthe component designer to theclient application

37

Ensuring Platform-neutral & Network-transparent OO Communication

Problem•We need an architecture that:

•Supports remote method invocation•Provides location transparency•Allows the addition, exchange, orremove of services dynamically

•Hides system details from thedeveloper

Solution•Apply theBroker patternto provide OOplatform-neutralcommunicationbetweennetworked client& servercomponents

Solution•Apply theBroker patternto provide OOplatform-neutralcommunicationbetweennetworked client& servercomponents

Context•Using the Proxy pattern isinsufficient since it doesn‘t addresshow•Remote components are located•Connections are established•Messages are exchanged across anetwork

•etc.

calls1

messageexchange

messageexchange

*

marshalunmarhalreceive_resultservice_p

Client Proxy

calls*

*

call_service_pstart_task

Client

1

marshalunmarshalforward_messagetransmit_message

Bridge

marshalunmarshaldispatchreceive_request

Server Proxy

calls*

start_upmain_loopservice_i

Server

1

1

main_loopsrv_registrationsrv_lookuptransmit_message

Broker1

38

Broker Pattern Dynamics

method(proxy) locate_server

server port

receive_request

marshal

unmarshal

dispatchmethod (impl.)

result

marshalreceive_result

unmarshal

result

start_upregister_service

: Broker: Client Proxy : Server Proxy: Client : Server

assigned port

Broker tools provide thegeneration of necessary client& server proxies from higherlevel interface definitions

GeneratorGenerator ProxyProxyCodeCode

InterfaceInterfaceSpecif.Specif.

39

Applying the Broker Patternto Image Acquisition

•Common Object RequestBroker Architecture (CORBA)

•A family of specifications•OMG is the standards body•Over 800 companies

•CORBA defines interfaces•Rather than implementations

•Simplifies development ofdistributed applications byautomating

•Object location•Connection management•Memory management•Parameter (de)marshaling•Event & request demuxing•Error handling•Object/server activation•Concurrency

InterfaceRepository

IDLCompiler

ImplementationRepository

ClientOBJREF

Object(Servant)

in argsoperation()out args +

return

DIIIDL

STUBSORB

INTERFACE

IDLSKEL

DSI

Object Adapter

ORB CORE GIOP/IIOP/ESIOPS

•CORBA shields applications from environmentheterogeneity•e.g., programming languages, operatingsystems, networking protocols, hardware

40

Pros & Cons of the Broker PatternThis pattern has five benefits:•Portability enhancements

• A broker hides OS & network system detailsfrom clients and servers by using indirection &abstraction layers, such as APIs, proxies,adapters, & bridges

• Interoperability with other brokers• Different brokers may interoperate if theyunderstand a common protocol for exchangingmessages

•Reusability of services• When building new applications, brokersenable application functionality to reuseexisting services

•Location transparency• A broker is responsible for locating servers, soclients need not know where servers arelocated

•Changeability & extensibility ofcomponents• If server implementations change withoutaffecting interfaces clients should not beaffected

This pattern also has liabilities:

•Restricted efficiency• Applications using brokers maybe slower than applicationswritten manually

•Lower fault tolerance• Compared with non-distributedsoftware applications,distributed broker systems mayincur lower fault tolerance

•Testing & debugging may beharder• Testing & debugging ofdistributed systems is tediousbecause of all the componentsinvolved

41

Supporting Async Communication

Solution•Apply the Async Forwarder/Receiver designpattern to allow asynchronous communicationbetween clients & servers

Solution•Apply the Async Forwarder/Receiver designpattern to allow asynchronous communicationbetween clients & servers

Context• Some clients wantto send requests,continue their work,& receive theresults at somelater point in time

Problem•Broker implementations based on conventional RPCsemantics often just support blocking operations• i.e., clients must wait until two-way invocations return

•Unfortunately, this design can reduce scalability &complicate certain use-cases

Introduce intermediary queue(s) between clients &servers:• A queue is used to store messages

• A queue can cooperate with other queues toroute messages

• Messages are sent from sender to receiver• A client sends a message, which is queued &then forwarded to a message processor on aserver that receives & executes them

• A Message API is provided for clients & servers tosend/receive messages

Local Queue

storeforwardremove

Message <<route>>

Client Message API<<send>>

Remote Queue

storeforwardremove<<exec>>

MessageProcessor Message API<<recv>>

42

Async Forwarder/Receiver Pattern Dynamics

: Client

: Messagecreate

storeMessage

forwardMessage

: MessageProcessor

createreceive

receive

Message

Message

exec

send Message

: Message API

: LocalQueue

: RemoteQueue

: MessageAPI

Reply

store

forwardReply

recv Reply

send

Reply

recv

Otherprocessing

43

Applying the Async Forwarder/ReceiverPattern to Image Acquisition

We can apply the Async Forwarder/Receiverpattern to•Queue up image request messages remotelywithout blocking the diagnostic/clinicalworkstation clients

•Execute the requests at a later point & return theresults to the client

Local Queue

storeforwardremove

Message <<route>>

RadiologyClient Message API<<send>>

Remote Queue

storeforwardremove<<exec>>

Message API<<recv>>

Image ServerMessage

Processor

Image Xfer Service

Image Xfer Service

ImageDatabase

RadiologyClient

RadiologyClient

This design also enables other, more advanced capabilities, e.g.,•Multi-hop store & forward persistence•QoS-driven routing, where requests can be delivered to the“best” image database

This design also enables other, more advanced capabilities, e.g.,•Multi-hop store & forward persistence•QoS-driven routing, where requests can be delivered to the“best” image database

44

This pattern provides three benefits:•Enhances concurrency bytransparently leveraging availableparallelism• Messages can be executed remotely onservers while clients perform otherprocessing

•Simplifies synchronized access toa shared object that resides in itsown thread of control• Since messages are processed seriallyby a message processor target objectsoften need not be concerned withsynchronization mechanisms

•Message execution order can differfrom message invocation order• This allows reprioritizing of messages toenhance quality of service

This pattern also has someliabilities:

•Message execution order candiffer from message invocationorder• As a result, clients must be careful notto rely on ordering dependencies

•Lack of type-safety• Clients & servers are responsible forformatting & passing messages

•Complicated debugging• As with all distributed systems,debugging & testing is complex

Pros & Cons of the AsyncForwarder/Receiver Pattern

45

Supporting OO Async Communication

Solution•Apply the Active Object design pattern to decouple method invocation frommethod execution using an object-oriented programming model

Solution•Apply the Active Object design pattern to decouple method invocation frommethod execution using an object-oriented programming model

• A proxy provides an interface that allowsclients to access methods of an object

• A concrete method request is created forevery method invoked on the proxy

• A scheduler receives the method requests& dispatches them on the servant whenthey become runnable

• An activation list maintains pendingmethod requests

• A servant implements the methods• A future allows clients to access theresults of a method call on the proxy

Context• Some clients want to invokeremote operations, continuetheir work, & retrieve theresults at a later point in time

Problem•Using the explicit message-passing API ofthe Async Forwarder/Receiver pattern canreduce type-safety & performance•Similar to motivation for Proxy pattern...

Future

Scheduler

enqueue dispatch

MethodRequest

guardcall

*

Proxy

method_1method_n

ActivationList

enqueuedequeue

Servant

method_1method_n

creates creates maintains

ConcreteMethodRequest1

ConcreteMethodRequest2

46

• A client invokes a method on theproxy

• The proxy returns a future to theclient, & creates a methodrequest, which it passes to thescheduler

• The scheduler enqueues themethod request into theactivation list (not shown here)

• When the method requestbecomes runnable, the schedulerdequeues it from the activationlist (not shown here) & executesit in a different thread than theclient

• The method request executes themethod on the servant & writesresults, if any, to the future

• Clients obtain the method’sresults via the future

Active Object Pattern Dynamics

: Future

method

enqueue

: Proxy : Scheduler : Servant

: MethodRequest

dispatch call method

read

write

: Client

Clients can obtain result from futuresvia blocking, polling, or callbacksClients can obtain result from futuresvia blocking, polling, or callbacks

47

Applying the Active Object Patternto Image Acquisition

•OO developers generally prefermethod-oriented request/responsesemantics to message-orientedsemantics

•The Active Object pattern supportsthis preference via strongly-typedasync method APIs:•Several types of parameters canbe passed:•Requests contain in/inoutarguments

•Results carry out/inoutarguments & results

•Callback object or poller object canbe used to retrieve results

Future

Scheduler

enqueue dispatch

MethodRequestguardcall

*

Proxy

method_1method_n

ActivationList

enqueuedequeue

Servantmethod_1method_n

creates creates maintains

get_image() put_image()

Image Xfer Service

Image Xfer Service

ImageDatabase

RadiologyClient

RadiologyClient

get_image()

future results

48

This pattern provides four benefits:•Enhanced type-safety

• Compared with async message passing•Enhances concurrency & simplifiessynchronized complexity• Concurrency is enhanced by allowing client threads& asynchronous method executions to runsimultaneously

• Synchronization complexity is simplified by using ascheduler that evaluates synchronizationconstraints to guarantee serialized access toservants

•Transparent leveraging of availableparallelism• Multiple active object methods can execute inparallel if supported by the OS/hardware

•Method execution order can differ frommethod invocation order• Methods invoked asynchronous are executedaccording to the synchronization constraintsdefined by their guards & by scheduling policies

This pattern also has someliabilities:

• Performance overhead• Depending on how an activeobject’s scheduler isimplemented, contextswitching, synchronization, &data movement overhead mayoccur when scheduling &executing active objectinvocations

• Complicated debugging• It is hard to debug programsthat use the Active Objectpattern due to the concurrency& non-determinism of thevarious active objectschedulers & the underlyingOS thread scheduler

Pros & Cons of the Active Object Pattern

49

Decoupling Suppliers & Consumers

Decouple suppliers (publishers) &consumers (subscribers) of events:• An Event Channel stores/forwards events• Publishers create events & store them in aqueue maintained by the Event Channel

• Consumers register with event queues,from which they retrieve events

• Events are used to transmit state changeinfo from publishers to consumers

• For event transmission push-models &pull-models are possible

• Filters can filter events for consumersEvent

*

Subscriber

consume

creates receives

Event Channel

attachPublisher detachPublisherattachSubscriberdetachSubscriber

Filter

filter

Publisher

produce

Problem•Having each client call a specific serveris inefficient & non-scalable•A polling strategy leads toperformance bottlenecks

•Work lists could be spread acrossdifferent servers

•More than one client may beinterested in work list content

Context• In large-scale electronic medicalimaging systems, radiologists mayshare “work lists” of patient imagesto balance workloads effectively

Solution•Apply the Publisher/Subscriberpattern to decouple imagesuppliers from image consumers

Solution•Apply the Publisher/Subscriberpattern to decouple imagesuppliers from image consumers

50

Publisher/Subscriber Pattern Dynamics

•The Publisher/Subscriberpattern helps keep thestate of cooperatingcomponents synchronized

•To achieve this it enablesone-way propagation ofchanges: one publishernotifies any number ofsubscribers aboutchanges to its state

attachSubscriber

produce

pushEventevent

eventpushEvent

consume

detachSubscriber

: Event

: Subscriber: Event Channel: Publisher

Key design considerations for the Publisher/Subscriber pattern include:•Push vs. pull interaction models•Control vs. data event notification models•Multicast vs. unicast communication models•Persistence vs. transient queueing models

Key design considerations for the Publisher/Subscriber pattern include:•Push vs. pull interaction models•Control vs. data event notification models•Multicast vs. unicast communication models•Persistence vs. transient queueing models

51

Applying the Publisher/SubscriberPattern to Image Acquisition

Event

*

Radiologist

consume

creates receives

Event Channel

attachPublisher detachPublisherattachSubscriberdetachSubscriber

Filter

filter

Modality

produce

ImageDatabase

RadiologyClient

RadiologyClient

RadiologyClient

RadiologyClient

RadiologyClient

RadiologyClient

RadiologyClient

RadiologyClient

RadiologyClient

RadiologyClient

EventChannelEvent

Channel

•Radiologists can subscribe toan event channel in order toreceive notifications producedwhen modalities publish eventsindicating the arrival of a newimage

•This design enables a group ofdistributed radiologists tocollaborate effectively in anetworked environment

52

Pros & Cons of thePublisher/Subscriber Pattern

This pattern has two benefits:•Decouples consumers &producers of events• All an event channel knows is that ithas a list of consumers, eachconforming to the simple interface ofthe Subscriber class

• Thus, the coupling between thepublishers and subscribers is abstract& minimal

•n:m communication models aresupported• Unlike an ordinary request/responseinteraction, the notification that apublisher sends needn’t designate itsreceiver, which enables a broaderrange of communication topologies,including multicast & broadcast

There is also a liability:•Must be careful with potentialupdate cascades• Since subscribers have noknowledge of each other’s presence,applications can be blind to theultimate cost of publishing eventsthrough an event channel

• Thus, a seemingly innocuousoperation on the subject may causea cascade of updates to observers &their dependent objects

53

Locating & Creating Components Effectively

• An Abstract Home declares an interface foroperations that find and/or create abstractinstances of components

• Concrete Homes implements the abstractHome interface to find specific instances and/orcreate new ones

• Abstract Comp declares interface for a specifictype of component class

• Concrete Comp define instances• A Primary Key is associated with a component

Context• Our electronic medicalimaging system containsmany componentsdistributed in a network

Solution•Apply the Factory/Finder pattern to separate the management of componentlifecycles from their use by client applications

Solution•Apply the Factory/Finder pattern to separate the management of componentlifecycles from their use by client applications

AbstractComp

operation

ConcreteComp

operation

Primary Key

ConcreteHome

findcreate

AbstractHome

findcreate

Problem•How to create new components and/or findexisting ones• Simple solutions appropriate for stand-aloneapplications don’t scale

• “Obvious” solutions for distribution also don’t scale

54

Factory/Finder Pattern Dynamics

• Homes enable the creation &location of components, butwe still need a global namingservice to locate the homes

• Homes enable the creation &location of components, butwe still need a global namingservice to locate the homes

find (“ImageXYZ”); Primary Key

Component

operation

lookup

: Client : Home : Component

: Primary Keycreate

Node

Binding Directory

*

resolvelistNodesnavigatenewBindingnewSubDirremove

getName

getObject

Client

•The Factory/Finder patternis supported by distributedcomponent models•e.g., EJB, COM+, & theCCM

55

Applying the Factory/Finder Patternto Image Acquisition

AbstractComp

operation

ImageXferComp

operation

Primary Key

ImageXferHome

findcreate

AbstractHome

findcreate

1. Deploy

2. BindFactory



4. Intercept & delegate

Factory ProxyFactory Proxy6. Find Image

Factory/FinderFactory/Finder


3. Find Factory

ImageDatabase

RadiologyClient

RadiologyClient

5.New

ConfigurationConfiguration

ContainerContainer

•We can apply the Factory/Finderpattern to create/locate imagetransfer components for imagesneeded by radiologists

• If a suitable component alreadyexists the component home willuse it, otherwise, it will create anew component

56

Pros & Cons of the Factory/Finder Pattern

This pattern has three benefits:•Separation of concerns

• Finding/creating individualcomponents is decoupled fromlocating the factories that find/createthese components

• Improved scalability• e.g., general-purpose directorymechanisms need not manage thecreation & location of large amounts offiner-grained components whoselifetimes may be short

•Customized capabilities• The location/creation mechanism canbe specialized to support keycapabilities that are unique for varioustypes of components

This pattern also has some liabilities:•Overhead due to indirection

• Clients must incur the overhead ofseveral round-trips to obtain theappropriate object reference

•Complexity & cost fordevelopment & deployment• There are more steps involved inobtaining object references, which cancomplicate client programming

57

Context•Component developers maynot know a priori where theircomponents will execute

•Thus, containers areintroduced to:• Shield clients & componentsfrom the details of theunderlying middleware,services, network & OS

• Manage the lifecycle ofcomponents & notifycomponents about lifecycleevents• e.g., activation, passivation, &

transaction progress• Provide components uniformaccess to infrastructureservices• e.g., transactions, security, &

persistence• Register & deploy components

DeclarativeProgramming

ServerComponent

TransactionSecurityResources...

ServerComponent

TransactionSecurityResources...

...

ImperativeProgramming

Container

Client Client

Extending Components Transparently

58

Extending Components Transparently (cont‘d)

Problem• Components should be able to specifydeclaratively in configuration files whichexecution environment they require

• Containers then should provide the rightexecution environment•e.g., by creating a new transaction ornew servant when required

• Framework represents the concreteframework to which we attach interceptors

• Concrete Interceptors implement theevent handler for the system-specificevents they have subscribed for

• Context contains information about theevent & allows modification of systembehavior after interceptor completion

• The Dispatcher allows applications toregister & remove interceptors with theframework & to delegate events tointerceptors

Context

ConcreteInterceptor

handle_event*

callbackcallback

provideprovide

Dispatcher

attachattach

Framework

attach_interceptormanage_interceptorsAbstractInterceptor

handle_event

Solution•Apply the Interceptorarchitectural pattern to attachinterceptors to a frameworkthat can handle particularevents by invoking associatedinterceptors automatically

Solution•Apply the Interceptorarchitectural pattern to attachinterceptors to a frameworkthat can handle particularevents by invoking associatedinterceptors automatically

59

Interceptor Pattern Dynamics• Interceptor are a “meta-programming mechanism”

•Other meta-programmingmechanisms include•Smart proxies•Pluggable protocols•Gateways/bridges• Interface repositories & DII

•These mechanisms providebuilding-blocks to handlevariation translucently &reflectively

•More information on meta-programming mechanismscan be found atwww.cs.wustl.edu/~schmidt/PDF/IEEE.pdf

• Interception can also enable performanceenhancement strategies• e.g., just-in-time activation, objectpooling, & caching

: Application: Framework

: Interceptorcreate

run_event_loop

attachinterceptor

Place interceptor ininternal interceptormap

event

Look forregisteredinterceptors

: Contextcreate

context

handle_event

: Dispatcher

delegate

60

Context

Container

handle_event*

callbackcallback

provideprovide

Dispatcher

attachattach

Image ServerFramework

attach_interceptormanage_interceptorsAbstractInterceptor

handle_event

Applying the Interceptor Patternto Image Acquisition

• A container provides genericinterfaces to a component that itcan use to access containerfunctionality•e.g., transaction control,persistence, security,etc.

• A container intercepts all incomingrequests from clients•It reads the component’srequirements from a XMLconfiguration file & does somepre-processing before actuallydelegating the request to thecomponent

• A component provides eventinterfaces the container invokesautomatically when particularevents occur•e.g., activation or passivation

Container

Component

XMLconfig

61

Pros & Cons of the Interceptor PatternThis pattern has five benefits:•Extensibility & flexibility

• Interceptors allow an application to evolvewithout breaking existing APIs &components

•Separation of concerns• Interceptors decouple the “functional”path from the “meta” path

•Support for monitoring & control offrameworks• e.g., generic logging mechanisms can beused to unobtrusively track applicationbehavior

•Layer symmetry• Interceptors can perform transformationson a client-side whose inverse areperformed on the server-side

•Reusability• Interceptors can be reused for variousgeneral-purpose behaviors

This pattern also has liabilities:•Complex design issues

• Determining interceptor APIs &semantics is non-trivial

•Malicious or erroneousinterceptors• Mis-behaving interceptors canwreak havoc on applicationstability

•Potential interceptioncascades• Interceptors can result in infiniterecursion

62

Minimizing Resource Utilization

Problem• It may not feasible to have all imageserver implementations running allthe time since this ties up end-system resources unnecessarily

Solution• Apply the Activator pattern to spawn servers on-demand inorder to minimize end-system resource utilization

Solution• Apply the Activator pattern to spawn servers on-demand inorder to minimize end-system resource utilization

Context• Image servers are simply one ofmany services running throughoutan distributed electronic medicalimage system

• When incoming requests arrive, theActivator looks up whether a target object isalready active & if the object is not running itactivates the implementation

• The Activation Table stores associationsbetween services & their physical location

• The Client uses the Activator to get serviceaccess

• A Service implements a specific type offunctionality that it provides to clients

Activator

(de)activategetServiceregisterunregisteronShutdown

Activation Table

insertEntryremoveEntrylookupEntrychangeEntry

Service

serviceshutdown

getServicegetService

Client

useService

63

Activator Pattern Dynamics

: Activator : Client

getService

: Implementationactivate

lookupEntry

[not active]

changeEntryobject

service

result

port

onShutdown

changeEntry

: Activ. Table

• A container can be used to activate & passivate a component•A component can be activated/passivated by itself, the container,after each method call, after each transaction, etc.

• A container can be used to activate & passivate a component•A component can be activated/passivated by itself, the container,after each method call, after each transaction, etc.

64

ImR (ringil:5000)Client

ringil:4500plane.exeairplane_poa

server.exepoa_name ringil:5500iiop://ringil:5000/poa_name/object_name

Applying the Activator Patternto Image Acquisition

Activator

(de)activategetServiceregisterunregisteronShutdown

Activation Table

insertEntryremoveEntrylookupEntrychangeEntry

getServicegetService

Client

useService

ImageXferService

serviceshutdown

Server (ringil:5500)

1. some_request

4. LOCATION_FORWARD

2. ping3. is_running

6. some_response

5. some_request

2.1 start

•The Activator pattern is available invarious COTS technologies:•UNIX Inetd “super server”•CORBA Implementation Repository

•We can use the Activator patternto launch image transfer serverson-demand

iiop://ringil:5500/poa_name/object_name

65

Pros & Cons of the Activator PatternThis pattern has three benefits:•Uniformity• By imposing a uniform activationinterface to spawn & control servers

•Modularity, testability, & reusability• Application modularity & reusability isimproved by decoupling serverimplementations from the manner inwhich the servers are activated

•More effective resource utilization• Servers can be spawned “on-demand,”thereby minimizing resource utilizationuntil clients actually require them

This pattern also has liabilities:•Lack of determinism & orderingdependencies• This pattern makes it hard todetermine or analyze the behaviorof an application until itscomponents are activated at run-time

•Reduced security or reliability• An application that uses theActivator pattern may be lesssecure or reliable than anequivalent statically-configuredapplication

• Increased run-time overhead &infrastructure complexity• By adding levels of abstraction &indirection when activating &executing components

66

Enhancing Server (Re)Configurability

This pattern allows an application to link & unlink itscomponent implementations at run-time so new &enhanced services can therefore be added withouthaving to modify, recompile, statically relink, or shutdown & restart a running application

•Certain factors are static, such asthe number of available CPUs &operating system support forasynchronous I/O

•Other factors are dynamic, suchas system workload

ContextThe implementation of certainimage server components dependson a variety of factors:

ProblemPrematurely committing to a particularimage server component configurationis inflexible and inefficient:•No single image server configuration isoptimal for all use cases

•Certain design decisions cannot bemade efficiently until run-time

Solution•Apply the Component Configurator designpattern to enhance server configurability

Solution•Apply the Component Configurator designpattern to enhance server configurability

<<contains>>

components*


ComponentRepository

ConcreteComponent A

ConcreteComponent B

Component init() fini() suspend() resume() info()

67

Component Configurator Pattern Dynamics

: ComponentConfigurator

init()

: ConcreteComponent A

: ConcreteComponent B

: ComponentRepository

insert()

insert()

init()

Concrete

run_component()

run_component()

fini()

remove()

remove()

fini()

Comp. A

ConcreteComp. B

ConcreteComp. A

ConcreteComp. B

1.Componentinitialization

2.Componentprocessing

3.Componenttermination

68

Applying the Component ConfiguratorPattern to Image Acquisition

<<contains>>

components*


ComponentRepository

LRUFile Cache

LFUFile Cache

Component init() fini() suspend() resume() info()

•For example, an image server can applythe Component Configurator pattern toconfigure various Cached VirtualFilesystem strategies•e.g., least-recently used (LRU) orleast-frequently used (LFU)

Image servers can use theComponent Configurator pattern todynamically optimize, control, &reconfigure the behavior of itscomponents at installation-time orduring run-time

Concrete components can bepackaged into a suitable unit ofconfiguration, such as adynamically linked library (DLL)

Concrete components can bepackaged into a suitable unit ofconfiguration, such as adynamically linked library (DLL)

Only the componentsthat are currently in useneed to be configuredinto an image server

Only the componentsthat are currently in useneed to be configuredinto an image server

69

Reconfiguring an Image ServerImage serverscan also bereconfigureddynamically tosupport newcomponents &new componentimplementations

IDLE

RUNNING

SUSPENDED

CONFIGUREinit()

RECONFIGUREinit()

fini()

fini()

resume()

suspend()

EXECUTErun_component()

SUSPEND

RESUME

TERMINATE

TERMINATE

Reconfiguration State Chart

LRU FileCache

ImageServer

# Configure an image server.dynamic File_Cache Component *

img_server.dll:make_File_Cache() "-t LRU"

INITIAL CONFIGURATION

AFTER RECONFIGURATION

LFU FileCache

ImageServer

# Reconfigure an image server.Remove File_Cachedynamic File_Cache Component *

img_server.dll:make_File_Cache() "-t LFU"

70

Pros and Cons of theComponent Configurator Pattern

This pattern offers four benefits:•Uniformity

• By imposing a uniform configuration &control interface to manage components

•Centralized administration• By grouping one or more components intoa single administrative unit that simplifiesdevelopment by centralizing commoncomponent initialization & terminationactivities

•Modularity, testability, & reusability• Application modularity & reusability isimproved by decoupling componentimplementations from the manner in whichthe components are configured intoprocesses

•Configuration dynamism & control• By enabling a component to bedynamically reconfigured withoutmodifying, recompiling, statically relinkingexisting code & without restarting thecomponent or other active componentswith which it is collocated

This pattern also incurs liabilities:•Lack of determinism & orderingdependencies• This pattern makes it hard todetermine or analyze the behavior ofan application until its components areconfigured at run-time

•Reduced security or reliability• An application that uses theComponent Configurator pattern maybe less secure or reliable than anequivalent statically-configuredapplication

• Increased run-time overhead &infrastructure complexity• By adding levels of abstraction &indirection when executingcomponents

•Overly narrow common interfaces• The initialization or termination of acomponent may be too complicated ortoo tightly coupled with its context tobe performed in a uniform manner

71

Tutorial Example 2:

High-performance Content Delivery Servers

•Support many content delivery serverdesign alternatives seamlessly• e.g., different concurrency & event models

•Design is guided by patterns to leveragetime-proven solutions

•Support many content delivery serverdesign alternatives seamlessly• e.g., different concurrency & event models

•Design is guided by patterns to leveragetime-proven solutions

Key Solution Characteristics• Implementation is based on ACEframework components to reduceeffort & amortize prior effort

•Open-source to control costs & toleverage technology advances

• Implementation is based on ACEframework components to reduceeffort & amortize prior effort

•Open-source to control costs & toleverage technology advances

Key SystemCharacteristics•Robust implementation

• e.g., stop malicious clients•Extensible to other protocols

• e.g., HTTP 1.1, IIOP, DICOM•Leverage advanced multi-processor hardware &software

Goal•Download content scalably& efficiently•e.g., images & othermulti-media content types

Graphics

Adapter

GUI

Event Dispatcher

Transfer Protocol

e.g. , HTTP 1.0

Requester

File

Cache

Protocol

Handlers

HTML

Parser

HTTP ClientHTTP Server

GET /index.html HTTP/1.0

<H1>POSA page</H1>...

www.posa.uci.edu

TCP/IP Network

OS Kernel

& Protocols

OS Kernel

& Protocols

72

JAWS Content Server FrameworkKey Sources of Variation•Concurrency models

• e.g.,thread pool vs. thread-perrequest

•Event demultiplexing models• e.g.,sync vs. async

•File caching models• e.g.,LRU vs. LFU

•Content delivery protocols• e.g.,HTTP 1.0+1.1, HTTP-NG,IIOP, DICOM

Event Dispatcher• Accepts client connection

request events, receivesHTTP GET requests, &coordinates JAWS’s eventdemultiplexing strategywith its concurrencystrategy.

• As events are processedthey are dispatched to theappropriate ProtocolHandler.

Protocol Handler• Performs parsing & protocol

processing of HTTP requestevents.

• JAWS Protocol Handler designallows multiple Web protocols, suchas HTTP/1.0, HTTP/1.1, & HTTP-NG, to be incorporated into a Webserver.

• To add a new protocol, developersjust write a new Protocol Handlercomponent & configure it into theJAWS framework.

Cached Virtual Filesystem• Improves Web server

performance by reducing theoverhead of file system accesseswhen processing HTTP GETrequests.

• Various caching strategies, such asleast-recently used (LRU) or least-frequently used (LFU), can beselected according to the actual oranticipated workload & configuredstatically or dynamically.

73

Applying Patterns to Resolve KeyJAWS Design Challenges

Patterns help resolve the following common design challenges:•Efficiently demuxing asynchronousoperations & completions

•Transparently parameterizingsynchronization into components

•Ensuring locks are releasedproperly

•Minimizing unnecessary locking•Synchronizing singletons correctly•Logging access statistics efficiently

•Efficiently demuxing asynchronousoperations & completions

•Transparently parameterizingsynchronization into components

•Ensuring locks are releasedproperly

•Minimizing unnecessary locking•Synchronizing singletons correctly•Logging access statistics efficiently

•Encapsulating low-level OS APIs•Decoupling event demultiplexing &connection management fromprotocol processing

•Scaling up performance via threading•Implementing a synchronized requestqueue

•Minimizing server threading overhead•Using asynchronous I/O effectively

•Encapsulating low-level OS APIs•Decoupling event demultiplexing &connection management fromprotocol processing

•Scaling up performance via threading•Implementing a synchronized requestqueue

•Minimizing server threading overhead•Using asynchronous I/O effectively

Double-checkedLocking

Optimization

Thread-specific Storage

74

Encapsulating Low-level OS APIsProblem• The diversity of hardware & operatingsystems makes it hard to build portable& robust Web server software byprogramming directly to low-leveloperating system APIs, which aretedious, error-prone, & non-portable

The Wrapper Facade patternencapsulates data & functionsprovided by existing non-OO APIswithin more concise, robust,portable, maintainable, & cohesiveOO class interfaces

: Application

method()

: WrapperFacade

: APIFunctionA

functionA()

: APIFunctionB

functionB()

Applicationcalls methods

callsAPI FunctionA()

callsAPI FunctionB()

calls API FunctionC()

void methodN(){functionA();

}

void method1(){functionA();

}functionB();

Wrapper Facade

data

method1()…methodN()Solution

• Apply the Wrapper Facade designpattern to avoid accessing low-leveloperating system APIs directly

Solution• Apply the Wrapper Facade designpattern to avoid accessing low-leveloperating system APIs directly

Context• A Web server must manage a variety ofOS services, including processes,threads, Socket connections, virtualmemory, & files. Most operating systemsprovide low-level APIs written in C toaccess these services

75

Other ACE wrapper facades used inJAWS encapsulate Sockets, process &thread management, memory-mappedfiles, explicit dynamic linking, & timeoperations

Other ACE wrapper facades used inJAWS encapsulate Sockets, process &thread management, memory-mappedfiles, explicit dynamic linking, & timeoperations

Applying the Wrapper Façade Pattern in JAWS

JAWS uses the wrapper facades defined by ACE to ensure its frameworkcomponents can run on many operating systems, including Windows, UNIX,& many real-time operating systems

For example, JAWS usesthe Thread_Mutexwrapper facade in ACEto provide a portableinterface to operatingsystem mutual exclusionmechanisms

Thread_Mutex

mutex

acquire()tryacquire()release()

void acquire(){

calls methods

callsmutex_lock()

callsmutex_trylock()

calls mutex_unlock()

void release(){mutex_unlock(mutex);

}mutex_lock(mutex);

}

JAWS

The Thread_Mutex wrapper in the diagramis implemented using the Solaris thread APIThe Thread_Mutex wrapper in the diagramis implemented using the Solaris thread API

www.cs.wustl.edu/~schmidt/ACE/

The ACE Thread_Mutex wrapper facade isalso available for other threading APIs, e.g.,pSoS, VxWorks, Win32 threads or POSIXPthreads

The ACE Thread_Mutex wrapper facade isalso available for other threading APIs, e.g.,pSoS, VxWorks, Win32 threads or POSIXPthreads

76

Pros and Cons of the Wrapper Façade PatternThis pattern provides three benefits:•Concise, cohesive, & robust higher-level object-oriented programminginterfaces• These interfaces reduce the tedium &increase the type-safety of developingapplications, which descreases certaintypes of programming errors

•Portability & maintainability• Wrapper facades can shield applicationdevelopers from non-portable aspects oflower-level APIs

•Modularity, reusability &configurability• This pattern creates cohesive & reusableclass components that can be ‘plugged’into other components in a wholesalefashion, using object-oriented languagefeatures like inheritance & parameterizedtypes

This pattern can incur liabilities:•Loss of functionality

• Whenever an abstraction is layeredon top of an existing abstraction it ispossible to lose functionality

•Performance degradation• This pattern can degradeperformance if several forwardingfunction calls are made per method

•Programming language &compiler limitations• It may be hard to define wrapperfacades for certain languages dueto a lack of language support orlimitations with compilers

77

Problem•Developers often coupleevent-demuxing &connection code withprotocol-handling code

Decoupling Event Demuxing & ConnectionManagement from Protocol Processing

Context

•Web servers can be accessedsimultaneously by multipleclients

Client

Client

Client

HTTP GETrequest

Connectrequest

HTTP GETrequest

Web Server

Socket Handles

•They must demux & processmultiple types of indicationevents arriving from clientsconcurrently

SolutionApply the Reactor architectural pattern & the Acceptor-Connector designpattern to separate the generic event-demultiplexing & connection-management code from the web server’s protocol code

SolutionApply the Reactor architectural pattern & the Acceptor-Connector designpattern to separate the generic event-demultiplexing & connection-management code from the web server’s protocol code

Event Dispatcher

Sockets

select()

•A common way to demux eventsin a server is to use select()

•This code cannot thenbe reused directly byother protocols or byother middleware &applications

•Thus, changes to event-demuxing & connection codeaffects the server protocolcode directly & may yieldsubtle bugs• e.g., porting it to use TLI orWaitForMultipleObjects()

78

The Reactor PatternThe Reactor architecturalpattern allows event-drivenapplications to demultiplex& dispatch service requeststhat are delivered to anapplication from one ormore clients.

Handleowns

dispatches*

notifies**

handle set

Reactor

handle_events()register_handler()remove_handler()

Event Handler

handle_event ()get_handle()

Concrete EventHandler A


Concrete EventHandler B


SynchronousEvent Demuxer

select ()

<<uses>>

: Main Program : ConcreteEvent Handler

: Reactor : Synchronous Event

Demultiplexer

register_handler()

get_handle()

handle_events() select()

handle_event()

Handle

Handles

Handles

Con. EventHandler Events

service()

event

Observations•Note inversionof control

•Also note howlong-runningevent handlerscan degrade theQoS sincecallbacks stealthe reactor’sthread!

1. Initializephase

2. Eventhandlingphase

79

The Acceptor-Connector PatternThe Acceptor-Connector design pattern decouples the connection &initialization of cooperating peer services in a networked system from theprocessing performed by the peer services after being connected & initialized.

<<activate>>

owns

*

uses uses

<<creates>>owns

uses

owns

<<activate>>

* * *

***

uses

notifies

notifies notifies

Connector

Connector()connect()complete()handle_event ()

Concrete ServiceHandler B

Concrete ServiceHandler A

ConcreteAcceptor

ConcreteConnector

Acceptor

Acceptor()Accept()handle_event ()

peer_acceptor_

ServiceHandler

open()handle_event ()set_handle()

peer_stream_

Dispatcher

select()handle_events()register_handler()remove_handler()

TransportHandle

TransportHandle

TransportHandle

80

Acceptor Dynamics

ServiceHandler Events

: Application : Acceptor : Dispatcher

register_handler()

handle_events()

accept()

open()

register_handler()

handle_event()

service()

: ServiceHandler

open()

ACCEPT_EVENTHandle1Acceptor

: Handle2

Handle2

Handle2

1.Passive-modeendpointinitialize phase

2.Servicehandlerinitialize phase

3.Serviceprocessingphase

•The Acceptor ensures that passive-mode transport endpoints aren’t usedto read/write data accidentally•And vice versa for data transportendpoints…

•There is typically one Acceptorfactory per-service/per-port•Additional demuxing can be doneat higher layers, a la CORBA

81

Synchronous Connector Dynamics

Handle

Addr

: Application : Connector : Dispatcher: ServiceHandler

handle_events()

connect()

open()

register_handler()

handle_event()

service()

ServiceHandler

EventsServiceHandler Handle

get_handle()

Motivation for Synchrony

1.Syncconnectioninitiation phase

2.Servicehandlerinitialize phase


• If the services must beinitialized in a fixedorder & the client can’tperform useful workuntil all connectionsare established

•If connection latency isnegligible•e.g., connecting witha server on thesame host via a‘loopback’ device

• If multiple threads ofcontrol are available & itis efficient to use athread-per-connectionto connect each servicehandler synchronously

82

Asynchronous Connector Dynamics

Addr

: Application : Connector : Dispatcher: ServiceHandler

handle_events()

complete()

connect()

open()

register_handler()

handle_event()

service()

ServiceHandler

ConnectorCONNECT

EVENT

Events

register_handler()ServiceHandler

HandleHandle

Handle

get_handle()

Motivation for Asynchrony

1.Asyncconnectioninitiationphase

2.Servicehandlerinitializephase


• If client is initializing manypeers that can be connectedin an arbitrary order

• If client is establishingconnections over highlatency links

• If client is asingle-threadedapplications

83

Applying the Reactor and Acceptor-Connector Patterns in JAWS



Handleowns

dispatches*

notifies**

handle set

Reactor

handle_events()register_handler()remove_handler()

Event Handler


HTTPAcceptor

HTTPHandlerSynchronous

Event Demuxer

select ()

<<uses>>

The Reactor architecturalpattern decouples:1.JAWS generic

synchronous eventdemultiplexing &dispatching logic from

2.The HTTP protocolprocessing it performsin response to events

1.The connection & initialization of peer client & server HTTP servicesfrom

2.The processing activities performed by these peer services oncethey are connected & initialized

The Acceptor-Connector design pattern can use a Reactor as itsDispatcher in order to help decouple:

84

Reactive Connection Management& Data Transfer in JAWS

85

Pros and Cons of the Reactor PatternThis pattern offers four benefits:•Separation of concerns

• This pattern decouples application-independent demuxing & dispatchingmechanisms from application-specific hookmethod functionality

•Modularity, reusability, &configurability• This pattern separates event-drivenapplication functionality into severalcomponents, which enables the configurationof event handler components that are looselyintegrated via a reactor

•Portability• By decoupling the reactor’s interface fromthe lower-level OS synchronous eventdemuxing functions used in itsimplementation, the Reactor patternimproves portability

•Coarse-grained concurrency control• This pattern serializes the invocation of eventhandlers at the level of event demuxing &dispatching within an application process orthread

This pattern can incur liabilities:•Restricted applicability

• This pattern can be appliedefficiently only if the OS supportssynchronous event demuxing onhandle sets

•Non-pre-emptive• In a single-threaded application,concrete event handlers thatborrow the thread of their reactorcan run to completion & prevent thereactor from dispatching otherevent handlers

•Complexity of debugging &testing• It is hard to debug applicationsstructured using this pattern due toits inverted flow of control, whichoscillates between the frameworkinfrastructure & the method call-backs on application-specific eventhandlers

86

Pros and Cons of the Acceptor-Connector Pattern

This pattern provides three benefits:•Reusability, portability, & extensibility

• This pattern decouples mechanisms forconnecting & initializing service handlers fromthe service processing performed after servicehandlers are connected & initialized

•Robustness• This pattern strongly decouples the servicehandler from the acceptor, which ensures that apassive-mode transport endpoint can’t be usedto read or write data accidentally

•Efficiency• This pattern can establish connections activelywith many hosts asynchronously & efficientlyover long-latency wide area networks

• Asynchrony is important in this situationbecause a large networked system may havehundreds or thousands of host that must beconnected

This pattern also has liabilities:•Additional indirection

• The Acceptor-Connector patterncan incur additional indirectioncompared to using the underlyingnetwork programming interfacesdirectly

•Additional complexity• The Acceptor-Connector patternmay add unnecessary complexityfor simple client applications thatconnect with only one server &perform one service using asingle network programminginterface

87

Scaling Up Performance via Threading

Solution•Apply the Half-Sync/Half-Asyncarchitectural pattern to scale upserver performance by processingdifferent HTTP requestsconcurrently in multiple threads

Solution•Apply the Half-Sync/Half-Asyncarchitectural pattern to scale upserver performance by processingdifferent HTTP requestsconcurrently in multiple threads

Context•HTTP runs over TCP, which usesflow control to ensure that sendersdo not produce data more rapidlythan slow receivers or congestednetworks can buffer and process

•Since achieving efficient end-to-endquality of service (QoS) is importantto handle heavy Web traffic loads, aWeb server must scale upefficiently as its number of clientsincreases

Problem•Processing all HTTP GET requestsreactively within a single-threadedprocess does not scale up, becauseeach server CPU time-slice spendsmuch of its time blocked waiting for I/Ooperations to complete

•Similarly, to improve QoS for all itsconnected clients, an entire Web serverprocess must not block while waiting forconnection flow control to abate so itcan finish sending a file to a client

This solution yields two benefits:1. Threads can be mapped to separate CPUs

to scale up server performance via multi-processing

2. Each thread blocks independently, whichprevents a flow-controlled connection fromdegrading the QoS other clients receive

88

The Half-Sync/Half-Async PatternSyncServiceLayer

AsyncService Layer

QueueingLayer

<<read/write>><<read/write>>

<<read/write>>

<<dequeue/enqueue>> <<interrupt>>

Sync Service 1 Sync Service 2 Sync Service 3

ExternalEvent Source

Queue

Async Service

The Half-Sync/Half-Asyncarchitectural patterndecouples async & syncservice processing inconcurrent systems, tosimplify programmingwithout unduly reducingperformance

• This pattern defines two serviceprocessing layers—one async & onesync—along with a queueing layerthat allows services to exchangemessages between the two layers

• The pattern allows sync services,such as HTTP protocol processing,to run concurrently, relative both toeach other & to async services, suchas event demultiplexing

: External EventSource

: Async Service : Queue

notification

read()

enqueue()

message

: Sync Service

work()

message

read()

message

work()

notification

89

Applying the Half-Sync/Half-AsyncPattern in JAWS

<<get>><<get>>

<<get>>

<<put>>

<<ready to read>>

SynchronousService Layer

AsynchronousService Layer

QueueingLayer

Worker Thread 1 Worker Thread 3

ReactorSocket

Event Sources

Request Queue

HTTP AcceptorHTTP Handlers,

Worker Thread 2

•JAWS uses the Half-Sync/Half-Asyncpattern to processHTTP GET requestssynchronously frommultiple clients, butconcurrently inseparate threads

•JAWS uses the Half-Sync/Half-Asyncpattern to processHTTP GET requestssynchronously frommultiple clients, butconcurrently inseparate threads

•The worker threadthat removes therequestsynchronouslyperforms HTTPprotocol processing &then transfers the fileback to the client

•The worker threadthat removes therequestsynchronouslyperforms HTTPprotocol processing &then transfers the fileback to the client

• If flow control occurson its client connectionthis thread can blockwithout degrading theQoS experienced byclients serviced byother worker threads inthe pool

• If flow control occurson its client connectionthis thread can blockwithout degrading theQoS experienced byclients serviced byother worker threads inthe pool

90

Pros & Cons of theHalf-Sync/Half-Async Pattern

This pattern has three benefits:•Simplification & performance

• The programming of higher-levelsynchronous processing services aresimplified without degrading theperformance of lower-level systemservices

•Separation of concerns• Synchronization policies in eachlayer are decoupled so that eachlayer need not use the sameconcurrency control strategies

•Centralization of inter-layercommunication• Inter-layer communication iscentralized at a single access point,because all interaction is mediatedby the queueing layer

This pattern also incurs liabilities:•A boundary-crossing penalty maybe incurred• This overhead arises from contextswitching, synchronization, & datacopying overhead when data istransferred between the sync & asyncservice layers via the queueing layer

•Higher-level application servicesmay not benefit from the efficiencyof async I/O• Depending on the design of operatingsystem or application frameworkinterfaces, it may not be possible forhigher-level services to use low-levelasync I/O devices effectively

•Complexity of debugging & testing• Applications written with this pattern canbe hard to debug due its concurrentexecution

91

Context•The Half-Sync/Half-Asyncpattern contains a queue

•The JAWS Reactor thread is a‘producer’ that inserts HTTPGET requests into the queue

•Worker pool threads are‘consumers’ that remove &process queued requests

<<get>><<get>>

<<get>>

<<put>>

Worker Thread 1

Worker Thread 3

Reactor

Request Queue


Worker Thread 2

Implementing a Synchronized Request Queue

Problem•A naive implementation of a request queue will incur raceconditions or ‘busy waiting’ when multiple threads insert & removerequests•e.g., multiple concurrent producer & consumer threads cancorrupt the queue’s internal state if it is not synchronized properly

•Similarly, these threads will ‘busy wait’ when the queue is emptyor full, which wastes CPU cycles unnecessarily

92

The Monitor Object Pattern

•This pattern synchronizesconcurrent method executionto ensure that only onemethod at a time runs withinan object

• It also allows an object’smethods to cooperativelyschedule their executionsequences

2..*

usesuses *

Monitor Object

sync_method1()sync_methodN()

Monitor Lock

acquire()release()

Client

Monitor Condition

wait()notify()notify_all()

Solution•Apply the Monitor Object design pattern to synchronize the queue efficiently& conveniently

Solution•Apply the Monitor Object design pattern to synchronize the queue efficiently& conveniently

• It’s instructive to compare Monitor Object pattern solutions with Active Objectpattern solutions•The key tradeoff is efficiency vs. flexibility

93

Monitor Object Pattern Dynamics: Monitor

Object: Monitor

Lock: MonitorCondition

sync_method1()

wait()

dowork()

: ClientThread1

: ClientThread2

acquire()

dowork()

acquire()sync_method2()

release()

notify()

dowork()

release()

the OS thread schedulerautomatically suspendsthe client thread

the OS threadschedulerautomatically resumes the clientthread and thesynchronizedmethod

the OS thread scheduleratomically reacquiresthe monitor lock

the OS thread scheduleratomically releasesthe monitor lock

1. Synchronizedmethodinvocation &serialization

2. Synchronizedmethod threadsuspension

3. Monitorconditionnotification

4. Synchronizedmethod threadresumption

94

Applying the Monitor Object Pattern in JAWS

The JAWS synchronizedrequest queueimplements the queue’snot-empty and not-fullmonitor conditions via apair of ACE wrapperfacades for POSIX-stylecondition variables

usesuses 2

Request Queue

put()get()

Thread_Mutex

acquire()release()

HTTPHandler

Thread Condition

wait()notify()notify_all()

WorkerThread

<<put>> <<get>>

•When a worker thread attempts to dequeue an HTTP GET requestfrom an empty queue, the request queue’s get() methodatomically releases the monitor lock & the worker thread suspendsitself on the not-empty monitor condition

•The thread remains suspended until the queue is no longer empty,which happens when an HTTP_Handler running in the Reactorthread inserts a request into the queue

•When a worker thread attempts to dequeue an HTTP GET requestfrom an empty queue, the request queue’s get() methodatomically releases the monitor lock & the worker thread suspendsitself on the not-empty monitor condition

•The thread remains suspended until the queue is no longer empty,which happens when an HTTP_Handler running in the Reactorthread inserts a request into the queue

95

Pros & Cons of the Monitor Object Pattern

This pattern provides two benefits:•Simplification of concurrencycontrol• The Monitor Object pattern presentsa concise programming model forsharing an object amongcooperating threads where objectsynchronization corresponds tomethod invocations

•Simplification of schedulingmethod execution• Synchronized methods use theirmonitor conditions to determine thecircumstances under which theyshould suspend or resume theirexecution & that of collaboratingmonitor objects

This pattern can also incur liabilities:•The use of a single monitor lock canlimit scalability due to increasedcontention when multiple threadsserialize on a monitor object

•Complicated extensibilitysemantics• These result from the coupling betweena monitor object’s functionality & itssynchronization mechanisms

•It is also hard to inherit from a monitorobject transparently, due to theinheritance anomaly problem

•Nested monitor lockout• This problem is similar to the precedingliability & can occur when a monitorobject is nested within another monitorobject

96

accept()

Minimizing Server Threading Overhead

•When a connection request arrives, theoperating system’s transport layer creates a newconnected transport endpoint, encapsulates thisnew endpoint with a data-mode socket handle &passes the handle as the return value fromaccept()

Context•Socket implementations in certain multi-threadedoperating systems provide a concurrent accept()optimization to accept client connection requests &improve the performance of Web servers thatimplement the HTTP 1.0 protocol as follows:

passive-modesocket handle

accept()

accept()

accept()

•The OS allows a pool of threads in a Web serverto call accept() on the same passive-modesocket handle

•The OS then schedules one of the threads inthe pool to receive this data-mode handle,which it uses to communicate with itsconnected client

accept()

accept()

97

Drawbacks with the Half-Sync/Half-Async Architecture

Solution•Apply the Leader/Followersarchitectural pattern tominimize server threadingoverhead

Solution•Apply the Leader/Followersarchitectural pattern tominimize server threadingoverhead

Problem•Although Half-Sync/Half-Asyncthreading model is morescalable than the purely reactivemodel, it is not necessarily themost efficient design

•CPU cache updates

<<get>><<get>>

<<get>>

<<put>>

Worker Thread 1

Worker Thread 3

Reactor

Request Queue


Worker Thread 2

•e.g., passing a requestbetween the Reactor thread& a worker thread incurs:

•This overhead makes JAWS’ latencyunnecessarily high, particularly onoperating systems that support theconcurrent accept() optimization

•Dynamic memory (de)allocation,

•A context switch, &•Synchronization operations,

98

The Leader/Followers Pattern

This pattern eliminates the need for—&the overhead of—a separate Reactorthread & synchronized request queueused in the Half-Sync/Half-Async pattern

This pattern eliminates the need for—&the overhead of—a separate Reactorthread & synchronized request queueused in the Half-Sync/Half-Async pattern

The Leader/Followers architecturalpattern provides an efficientconcurrency model where multiplethreads take turns sharing eventsources to detect, demux, dispatch, &process service requests that occur onthe event sources

TCP Sockets +select()/poll()

UDP Sockets +select()/poll()

IterativeHandle Sets

TCP Sockets +WaitForMultpleObjects()

UDP Sockets +WaitForMultipleObjects(

)

ConcurrentHandle Sets

Iterative HandlesConcurrent HandlesHandles

Handle Sets

Handleuses

demultiplexes

*

*

Handle Set

handle_events()deactivate_handle()reactivate_handle()select()

Event Handler


Concrete EventHandler B


Concrete EventHandler A


Thread Pool

join()promote_new_leader()

synchronizer

99

Leader/Followers Pattern Dynamics: Concrete

Event Handler

join()

handle_event()

: ThreadPool

: HandleSet

join()

thread 2 sleepsuntil it becomesthe leader

event

thread 1 sleepsuntil it becomesthe leader

deactivate_handle()

join()

Thread 1 Thread 2

handle_events() reactivate_

handle()

handle_event()

event

thread 2waits for anew event,thread 1processescurrentevent

deactivate_handle()

handle_events()

new_leader()

1.Leaderthreaddemuxing

2.Followerthreadpromotion

3.Eventhandlerdemuxing &eventprocessing

4.Rejoining thethread pool

promote_

100

Applying the Leader/FollowersPattern in JAWS

Handleuses

demultiplexes

*

*

Reactor

handle_events()deacitivate_handle()reactivate_handle()select()

Event Handler


HTTPAcceptor


HTTPHandler


Thread Pool

join()promote_new_leader()

synchronizer

Two options:1.If platform supports accept()

optimization then the Leader/Followerspattern can be implemented by the OS

2.Otherwise, this pattern can beimplemented as a reusable framework •The Half-Sync/Half-Async

design can reorder &prioritize client requestsmore flexibly, because ithas a synchronized requestqueue implemented usingthe Monitor Object pattern

• It may be more scalable,because it queues requestsin Web server virtualmemory, rather than the OSkernel

Although Leader/Followers threadpool design is highly efficient theHalf-Sync/Half-Async design may bemore appropriate for certain types ofservers, e.g.:

101

Pros and Cons of theLeader/Followers Pattern

This pattern provides two benefits:•Performance enhancements

• This can improve performance as follows:• It enhances CPU cache affinity andeliminates the need for dynamic memoryallocation & data buffer sharing betweenthreads

• It minimizes locking overhead by notexchanging data between threads, therebyreducing thread synchronization

• It can minimize priority inversion becauseno extra queueing is introduced in theserver

• It doesn’t require a context switch tohandle each event, reducing dispatchinglatency

•Programming simplicity• The Leader/Follower pattern simplifies theprogramming of concurrency models wheremultiple threads can receive requests,process responses, & demultiplexconnections using a shared handle set

This pattern also incur liabilities:

• Implementation complexity• The advanced variants of theLeader/ Followers pattern arehard to implement

•Lack of flexibility• In the Leader/ Followersmodel it is hard to discard orreorder events because thereis no explicit queue

•Network I/O bottlenecks• The Leader/Followers patternserializes processing byallowing only a single threadat a time to wait on the handleset, which could become abottleneck because only onethread at a time candemultiplex I/O events

102

Using Asynchronous I/O EffectivelyContext•Synchronous multi-threading may not be themost scalable way to implement a Web serveron OS platforms that support async I/O moreefficiently than synchronous multi-threading

passive-modesocket handle

AcceptEx()AcceptEx()AcceptEx()

I/O CompletionPort

GetQueuedCompletionStatus()



•When these async operations complete, WinNT1.Delivers the associated completion events

containing their results to the Web server2.Processes these events & performs the appropriate

actions before returning to its event loop

•For example, highly-efficient Web servers canbe implemented on Windows NT by invokingasync Win32 operations that perform thefollowing activities:•Processing indication events, such as TCPCONNECT and HTTP GET requests, viaAcceptEx() & ReadFile(), respectively

•Transmitting requested files to clientsasynchronously via WriteFile() orTransmitFile()

103

The Proactor PatternProblem•Developing software that achievesthe potential efficiency & scalabilityof async I/O is hard due to theseparation in time & space of asyncoperation invocations & theirsubsequent completion events

Solution•Apply the Proactor architectural patternto make efficient use of async I/O

Solution•Apply the Proactor architectural patternto make efficient use of async I/O

Handle

<<executes>>

*

<<uses>>is associated with

<<enqueues>>

<<dequeues>>

<<uses>> <<uses>>Initiator

<<demultiplexes & dispatches>>

<<invokes>>

Event QueueCompletion

AsynchronousOperation Processor

execute_async_op()

AsynchronousOperation

async_op()

AsynchronousEvent Demuxer

get_completion_event()

Proactor

handle_events()

CompletionHandler

handle_event()

ConcreteCompletion

Handler

This pattern allows event-drivenapplications to efficiently demultiplex &dispatch service requests triggered by thecompletion of async operations, therebyachieving the performance benefits of

concurrencywithout incurringits many liabilities

104

Dynamics in the Proactor Pattern

Result

CompletionHandler

Completion

: AsynchronousOperation

: Proactor CompletionHandler

exec_async_

handle_

Result

service()

: AsynchronousOperationProcessor

: Initiator

async_operation()

Result

handle_events()

event

event

Ev. Queue

operation ()

: Completion

Event Queue

Result

event()

1. Initiateoperation

2. Processoperation

3. Run eventloop

4. Generate& queuecompletionevent

5. Dequeuecompletionevent &performcompletionprocessing Note similarities & differences with the Reactor pattern, e.g.:

•Both process events via callbacks•However, it’s generally easier to multi-thread a proactor

Note similarities & differences with the Reactor pattern, e.g.:•Both process events via callbacks•However, it’s generally easier to multi-thread a proactor

105

Applying the Proactor Pattern in JAWSThe Proactor patternstructures the JAWSconcurrent server toreceive & processrequests from multipleclients asynchronously

Handle

<<executes>>

*

<<uses>>is associated with

<<enqueues>>

<<dequeues>>

<<uses>> <<uses>>Web Server

<<demultiplexes & dispatches>>

<<invokes>>

I/O Completion Port

Windows NTOperating System

execute_async_op()

AsynchronousOperation

AcceptEx()ReadFile()WriteFile()

AsynchronousEvent Demuxer


Proactor

handle_events()

CompletionHandler

handle_event()

HTTPAcceptor

HTTPHandler

JAWS HTTP components are split into two parts:1. Operations that execute asynchronously

•e.g., to accept connections & receive client HTTP GETrequests

2. The corresponding completion handlers that process theasync operation results•e.g., to transmit a file back to a client after an asyncconnection operation completes

106

Proactive Connection Management& Data Transfer in JAWS

107

Pros and Cons of the Proactor PatternThis pattern offers five benefits:•Separation of concerns

• Decouples application-independent asyncmechanisms from application-specificfunctionality

•Portability• Improves application portability by allowing itsinterfaces to be reused independently of the OSevent demuxing calls

•Decoupling of threading fromconcurrency• The async operation processor executes long-duration operations on behalf of initiators soapplications can spawn fewer threads

•Performance• Avoids context switching costs by activatingonly those logical threads of control that haveevents to process

•Simplification of applicationsynchronization• If concrete completion handlers spawn nothreads, application logic can be written withlittle or no concern for synchronization issues

This pattern incurs some liabilities:•Restricted applicability

• This pattern can be applied mostefficiently if the OS supportsasynchronous operationsnatively

•Complexity of programming,debugging, & testing• It is hard to program applications& higher-level system servicesusing asynchrony mechanisms,due to the separation in time &space between operationinvocation and completion

•Scheduling, controlling, &canceling asynchronouslyrunning operations• Initiators may be unable tocontrol the scheduling order inwhich asynchronous operationsare executed by anasynchronous operationprocessor

108

Efficiently Demuxing AsynchronousOperations & Completions

Context• In a proactive Webserver async I/Ooperations will yieldI/O completion eventresponses that mustbe processedefficiently

Problem•As little overhead as possible should be incurred todetermine how the completion handler will demux &process completion events after async operationsfinish executing

•When a response arrives, the application shouldspend as little time as possible demultiplexing thecompletion event to the handler that will process theasync operation’s response

•Together with each async operationthat a client initiator invokes on aservice, transmit information thatidentifies how the initiator shouldprocess the service’s response

Solution•Apply the Asynchronous Completion Token design pattern to demux &process the responses of asynchronous operations efficiently

Solution•Apply the Asynchronous Completion Token design pattern to demux &process the responses of asynchronous operations efficiently

•Return this information to the initiatorwhen the operation finishes, so that itcan be used to demux the responseefficiently, allowing the initiator toprocess it accordingly

109

The Asynchronous Completion Token Pattern

Structure and Participants

Dynamic Interactions

handle_event()

110

Applying the AsynchronousCompletion Token Pattern in JAWS

DetailedInteractions

(HTTP_Acceptoris both initiator &

completion handler)

111

Pros and Cons of the AsynchronousCompletion Token Pattern

This pattern has some liabilities:•Memory leaks

• Memory leaks can result if initiators useACTs as pointers to dynamicallyallocated memory & services fail toreturn the ACTs, for example if theservice crashes

•Authentication• When an ACT is returned to an initiatoron completion of an asynchronousevent, the initiator may need toauthenticate the ACT before using it

•Application re-mapping• If ACTs are used as direct pointers tomemory, errors can occur if part of theapplication is re-mapped in virtualmemory

This pattern has four benefits:•Simplified initiator data structures

• Initiators need not maintain complexdata structures to associate serviceresponses with completion handlers

•Efficient state acquisition• ACTs are time efficient because theyneed not require complex parsing ofdata returned with the service response

•Space efficiency• ACTs can consume minimal data spaceyet can still provide applications withsufficient information to associate largeamounts of state to processasynchronous operation completionactions

•Flexibility• User-defined ACTs are not forced toinherit from an interface to use theservice’s ACTs

112

Transparently ParameterizingSynchronization into Components

Context•The various concurrencypatterns described earlier impactcomponent synchronizationstrategies in various ways

•e.g.,ranging from no locks toreaders/writer locks

•In general, components must runefficiently in a variety ofconcurrency models

Problem•It should be possible to customize JAWScomponent synchronization mechanismsaccording to the requirements of particularapplication use cases & configurations

•Hard-coding synchronization strategiesinto component implementations isinflexible

•Maintaining multiple versions ofcomponents manually is not scalable

Solution•Apply the Strategized Locking design pattern to parameterize JAWScomponent synchronization strategies by making them ‘pluggable’ types

Solution•Apply the Strategized Locking design pattern to parameterize JAWScomponent synchronization strategies by making them ‘pluggable’ types

•Each type objectifies aparticular synchronizationstrategy, such as a mutex,readers/writer lock,semaphore, or ‘null’ lock

•Instances of these pluggable types can bedefined as objects contained within acomponent, which then uses these objects tosynchronize its method implementationsefficiently

113

Applying Polymorphic StrategizedLocking in JAWS

class File_Cache {public: // Constructor. File_Cache (Lock &l): lock_ (&l) { }

// A method. const void *lookup (const string &path) const { lock_->acquire(); // Implement the <lookup> method. lock_->release (); }

// ...private: // The polymorphic strategized locking object. mutable Lock *lock_; // Other data members and methods go here...};

Polymorphic Strategized

Locking

class Lock {public: // Acquire and release the lock. virtual void acquire () = 0; virtual void release () = 0;

// ...};class Thread_Mutex : public Lock { // ...};

114

Applying ParameterizedStrategized Locking in JAWS

template <class LOCK>class File_Cache {public: // A method. const void *lookup (const string &path) const { lock_.acquire (); // Implement the <lookup> method. lock_.release (); }

// ...private: // The polymorphic strategized locking object. mutable LOCK lock_; // Other data members and methods go here...};

ParameterizedStrategized

Locking

•Single-threaded file cache.typedef File_Cache<Null_Mutex>

Content_Cache;•Multi-threaded file cache using a thread mutex.typedef File_Cache<Thread_Mutex>

Content_Cache;•Multi-threaded file cache using a readers/writerlock.typedef File_Cache<RW_Lock>

Content_Cache;

Note that the variouslocks need not inheritfrom a common baseclass or use virtual

methods!

Note that the variouslocks need not inheritfrom a common baseclass or use virtual

methods!

115

Pros and Cons of theStrategized Locking Pattern

This pattern provides three benefits:•Enhanced flexibility & customization

• It is straightforward to configure &customize a component for certainconcurrency models because thesynchronization aspects of components arestrategized

•Decreased maintenance effort forcomponents• It is straightforward to add enhancements &bug fixes to a component because there isonly one implementation, rather than aseparate implementation for eachconcurrency model

• Improved reuse• Components implemented using this patternare more reusable, because their lockingstrategies can be configured orthogonally totheir behavior

This pattern also incurs liabilities:•Obtrusive locking

• If templates are used toparameterize locking aspects thiswill expose the locking strategies toapplication code

•Over-engineering• Externalizing a locking mechanismby placing it in a component’sinterface may actually provide toomuch flexibility in certain situations•e.g., inexperienced developersmay try to parameterize acomponent with the wrong typeof lock, resulting in impropercompile- or run-time behavior

116

Ensuring Locks are Released ProperlyContext•Concurrentapplications,such as JAWS,contain sharedresources thatare manipulatedby multiplethreadsconcurrently

Problem•Code that shouldn’t execute concurrently must beprotected by some type of lock that is acquired & releasedwhen control enters & leaves a critical section, respectively

•If programmers must acquire & release locks explicitly, it ishard to ensure that the locks are released in all pathsthrough the code

Solution•In C++, apply the Scoped Lockingidiom to define a guard class whoseconstructor automatically acquires alock when control enters a scope &whose destructor automaticallyreleases the lock when control leavesthe scope

Solution•In C++, apply the Scoped Lockingidiom to define a guard class whoseconstructor automatically acquires alock when control enters a scope &whose destructor automaticallyreleases the lock when control leavesthe scope

// A method.const void *lookup (const string &path) const { lock_.acquire (); // The <lookup> method // implementation may return // prematurely… lock_.release ();}

•e.g., in C++ control can leave a scope due to a return,break, continue, or goto statement, as well as from anunhandled exception being propagated out of the scope

117

Applying the Scoped LockingIdiom in JAWS

template <class LOCK>class Guard {public: // Store a pointer to the lock and acquire the lock. Guard (LOCK &lock) : lock_ (&lock) { lock_->acquire (); }

// Release the lock when the guard goes out of scope, ~Guard () { lock_->release (); }private: // Pointer to the lock we’re managing. LOCK *lock_;};

Generic Guard Wrapper Facade

template <class LOCK>class File_Cache {public: // A method. const void *lookup (const string &path) const { // Use Scoped Locking idiom to acquire // & release the <lock_>automatically. Guard<LOCK> guard (*lock); // Implement the <lookup> method. // lock_ released automatically… }

Applying the GuardInstances of the guardclass can be allocatedon the run-time stack toacquire & release locksin method or blockscopes that definecritical sections

Instances of the guardclass can be allocatedon the run-time stack toacquire & release locksin method or blockscopes that definecritical sections

118

Pros and Cons of theScoped Locking Idiom

This idiom has one benefit:• Increased robustness

• This idiom increases therobustness of concurrentapplications by eliminatingcommon programming errorsrelated to synchronization &multi-threading

• By applying the ScopedLocking idiom, locks areacquired & releasedautomatically when controlenters and leaves criticalsections defined by C++method & block scopes

This idiom also has liabilities:•Potential for deadlock when usedrecursively• If a method that uses the Scoped Locking idiomcalls itself recursively, ‘self-deadlock’ will occur ifthe lock is not a ‘recursive’ mutex

•Limitations with language-specificsemantics• The Scoped Locking idiom is based on a C++language feature & therefore will not be integratedwith operating system-specific system calls• Thus, locks may not be released automaticallywhen threads or processes abort or exit inside aguarded critical section

• Likewise, they will not be released properly ifthe standard C longjmp() function is calledbecause this function does not call thedestructors of C++ objects as the run-time stackunwinds

119

Minimizing Unnecessary Locking

Context•Components in multi-threadedapplications that contain intra-component method calls

•Components that have applied theStrategized Locking pattern

Problem•Thread-safe components should bedesigned to avoid unnecessarylocking

•Thread-safe components should bedesigned to avoid “self-deadlock”

Solution•Apply the Thread-safe Interface design pattern to minimize locking overhead& ensure that intra-component method calls do not incur ‘self-deadlock’ bytrying to reacquire a lock that is held by the component already

Solution•Apply the Thread-safe Interface design pattern to minimize locking overhead& ensure that intra-component method calls do not incur ‘self-deadlock’ bytrying to reacquire a lock that is held by the component already

• Interface methods check•All interface methods, such as C++public methods, should onlyacquire/release component lock(s),thereby performing synchronizationchecks at the ‘border’ of the component.

This pattern structures all components that process intra-component methodinvocations according two design conventions:

• Implementation methods trust• Implementation methods, such asC++ private and protectedmethods, should only performwork when called by interfacemethods.

120

Motivating the Need for theThread-safe Interface Pattern

template <class LOCK>class File_Cache {public: // Return a pointer to the memory-mapped file associated with // <path> name, adding it to the cache if it doesn’t exist. const void *lookup (const string &path) const { // Use the Scoped Locking idiom to acquire // & release the <lock_> automatically. Guard<LOCK> guard (lock_); const void *file_pointer = check_cache (path); if (file_pointer == 0) { // Insert the <path> name into the cache. // Note the intra-class <insert> call. insert (path); file_pointer = check_cache (path); } return file_pointer; } // Add <path> name to the cache. void insert (const string &path) { // Use the Scoped Locking idiom to acquire // and release the <lock_> automatically. Guard<LOCK> guard (lock_); // ... insert <path> into the cache... }private: mutable LOCK lock_; const void *check_cache (const string &) const; // ... other private methods and data omitted...};

Since File_Cacheis a template wedon’t know the

type of lock usedto parameterize it.

121

Applying the Thread-safe InterfacePattern in JAWS

template <class LOCK>class File_Cache {public: // Return a pointer to the memory-mapped file associated with // <path> name, adding it to the cache if it doesn’t exist. const void *lookup (const string &path) const { // Use the Scoped Locking idiom to acquire // and release the <lock_> automatically. Guard<LOCK> guard (lock_); return lookup_i (path); }private: mutable LOCK lock_; // The strategized locking object.

// This implementation method doesn’t acquire or release // <lock_> and does its work without calling interface methods. const void *lookup_i (const string &path) const { const void *file_pointer = check_cache_i (path); if (file_pointer == 0) {

// If <path> name isn’t in the cache then// insert it and look it up again.insert_i (path);file_pointer = check_cache_i (path);// The calls to implementation methods <insert_i> and// <check_cache_i> assume that the lock is held & do work.

}return file_pointer;

Note fewer constraintson the type of LOCK…

122

Pros and Cons of the Thread-safeInterface Pattern

This pattern has some liabilities:•Additional indirection and extra methods

• Each interface method requires at least oneimplementation method, which increases thefootprint of the component & may also add anextra level of method-call indirection for eachinvocation

•Potential for misuse• OO languages, such as C++ and Java, supportclass-level rather than object-level accesscontrol

• As a result, an object can bypass the publicinterface to call a private method on anotherobject of the same class, thus bypassing thatobject’s lock

•Potential overhead• This pattern prevents multiple components fromsharing the same lock & prevents locking at afiner granularity than the component, which canincrease lock contention

This pattern has three benefits:• Increased robustness

• This pattern ensures that self-deadlock does not occur due tointra-component method calls

•Enhanced performance• This pattern ensures that locksare not acquired or releasedunnecessarily

•Simplification of software• Separating the locking andfunctionality concerns can helpto simplify both aspects

123

Synchronizing Singletons Correctly

Context•JAWS uses various singletons to implement components where only oneinstance is required•e.g., the ACE Reactor, the request queue, etc.

Problem•Singletons can be problematic in multi-threaded programs

class Singleton {public: static Singleton *instance () { if (instance_ == 0) { // Enter critical section. instance_ = new Singleton; // Leave critical section. } return instance_; } void method_1 (); // Other methods omitted.private: static Singleton *instance_; // Initialized to 0 by linker.};

Either too little locking…class Singleton {public: static Singleton *instance () { Guard<Thread_Mutex> g (lock_); if (instance_ == 0) { // Enter critical section. instance_ = new Singleton; // Leave critical section. } return instance_; }private: static Singleton *instance_; // Initialized to 0 by linker. static Thread_Mutex lock_;};

… or too much

124

The Double-checked LockingOptimization Pattern

Solution•Apply the Double-Checked Locking Optimization design pattern to reducecontention & synchronization overhead whenever critical sections of codemust acquire locks in a thread-safe manner just once during programexecution

Solution•Apply the Double-Checked Locking Optimization design pattern to reducecontention & synchronization overhead whenever critical sections of codemust acquire locks in a thread-safe manner just once during programexecution

// Perform first-check to// evaluate ‘hint’.if (first_time_in is FALSE){ acquire the mutex // Perform double-check to // avoid race condition. if (first_time_in is FALSE) { execute the critical section set first_time_in to TRUE } release the mutex}

class Singleton {public: static Singleton *instance () { // First check if (instance_ == 0) { Guard<Thread_Mutex> g(lock_); // Double check. if (instance_ == 0) instance_ = new Singleton; } return instance_; }private: static Singleton *instance_; static Thread_Mutex lock_;};

125

Applying the Double-Checked LockingOptimization Pattern in ACE

template <class TYPE>class ACE_Singleton {public: static TYPE *instance () { // First check if (instance_ == 0) { // Scoped Locking acquires and release lock. Guard<Thread_Mutex> guard (lock_); // Double check instance_. if (instance_ == 0) instance_ = new TYPE; } return instance_; }private: static TYPE *instance_; static Thread_Mutex lock_;};

ACE defines a“singleton adapter”template to automatethe double-checkedlocking optimization

typedef ACE_Singleton<Request_Queue>Request_Queue_Singleton;

Thus, creating a “thread-safe” singleton is easy

126

Pros and Cons of the Double-CheckedLocking Optimization Pattern

This pattern has two benefits:•Minimized locking overhead

• By performing two first-time-inflag checks, this patternminimizes overhead for thecommon case

• After the flag is set the firstcheck ensures that subsequentaccesses require no furtherlocking

•Prevents race conditions• The second check of the first-time-in flag ensures that thecritical section is executed justonce

This pattern has some liabilities:•Non-atomic pointer or integralassignment semantics• If an instance_ pointer is used as the flag ina singleton implementation, all bits of thesingleton instance_ pointer must be read &written atomically in a single operation

• If the write to memory after the call to new isnot atomic, other threads may try to read aninvalid pointer

•Multi-processor cache coherency• Certain multi-processor platforms, such as theCOMPAQ Alpha & Intel Itanium, performaggressive memory caching optimizations inwhich read & write operations can execute ‘outof order’ across multiple CPU caches, suchthat the CPU cache lines will not be flushedproperly if shared data is accessed withoutlocks held

127

Logging Access Statistics Efficiently

Context•Web servers often needto log certain information•e.g., count number oftimes web pages areaccessed

Problem•Having a central logging object in a multi-threaded server process can become abottleneck•e.g., due to synchronization required toserialize access by multiple threads

Solution•Apply the Thread-Specific Storagepattern to allow multiple threads touse one ‘logically global’ accesspoint to retrieve an object that islocal to a thread, without incurringlocking overhead on each objectaccess

Solution•Apply the Thread-Specific Storagepattern to allow multiple threads touse one ‘logically global’ accesspoint to retrieve an object that islocal to a thread, without incurringlocking overhead on each objectaccess

Application <<uses>>callsThread

n m

n x mmaintains

m Thread-SpecificObject Setget(key)set(key, object)

Thread-SpecificObject Proxy

keymethod1()…methodN()

Thread-SpecificObjectmethod1()…methodN()

Key Factory

create_key()

128

key 1

key n

thread m

Thread-SpecificObject

Thread-SpecificObject Proxy

Thread-SpecificObject Set

accesses

manages

The application thread identifier, thread-specificobject set, & proxy cooperate to obtain thecorrect thread-specific object

[k,t]

thread 1

Thread-Specific Storage Pattern Dynamics

: Thread-SpecificObject Proxy

method()

: ApplicationThread

: Thread-SpecificObject Set

: Thread-SpecificObject

key

set()

create_key()

: KeyFactory

keyTSObject

129

class Error_Logger {public: int last_error (); void log (const char *format, ...);};

Applying the Thread-SpecificStorage Pattern to JAWS

ApplicationThread

<<uses>>calls

n m

n x mmaintains

m

Key Factory

create_key()

Thread-SpecificObject Setget(key)set(key, object)

ACE_TSS

key

operator->()

Error_Loggerlast_error()log()…

template <class TYPE>Class ACE_TSS {public: TYPE *operator->() const { TYPE *tss_data = 0; if (!once_) { Guard <Thread_Mutex> guard (keylock_); if (!once_) { ACE_OS::thr_keycreate (&key_, &cleanup_hook); once_ = true; } } ACE_OS::thr_getspecific (key, (void **) &tss_data); if (tss_data == 0) { tss_data = new TYPE; ACE_OS::thr_setspecific (key, (void *) tss_data); } return tss_data; }private: mutable pthread_key_t key_; mutable bool once_; mutable Thread_Mutex keylock_; static void cleanup_hook (void *ptr);};

ACE_TSS <Error_Logger> my_logger;// ...if (recv (……) == -1 && my_logger->last_error () != EWOULDBLOCK) my_logger->log (“recv failed, errno = %d”, my_logger->last_error ());};

130

Pros & Cons of the Thread-SpecificStorage Pattern

This pattern has four benefits:•Efficiency

• It’s possible to implement this patternso that no locking is needed toaccess thread-specific data

•Ease of use• When encapsulated with wrapperfacades, thread-specific storage iseasy for application developers touse

•Reusability• By combining this pattern with theWrapper Façade pattern it’s possibleto shield developers from non-portable OS platform characteristics

•Portability• It’s possible to implement portablethread-specific storage mechanismson most multi-threaded operatingsystems

This pattern also has liabilities:• It encourages use of thread-specific global objects• Many applications do not requiremultiple threads to access thread-specific data via a common access point

• In this case, data should be stored sothat only the thread owning the data canaccess it

• It obscures the structure of thesystem• The use of thread-specific storagepotentially makes an application harderto understand, by obscuring therelationships between its components

• It restricts implementation options• Not all languages supportparameterized types or smart pointers,which are useful for simplifying theaccess to thread-specific data

131

UML models ofa softwarearchitecturecan illustratehow a systemis designed, butnot why thesystem isdesigned in aparticular way

Tutorial Example 3:Applying Patterns to Real-time CORBA

Patterns are used throughout The ACE ORB (TAO) Real-time CORBA implementation to codify expert knowledge &to generate the ORB’s software architecture by capturingrecurring structures & dynamics & resolving commondesign forces

Patterns are used throughout The ACE ORB (TAO) Real-time CORBA implementation to codify expert knowledge &to generate the ORB’s software architecture by capturingrecurring structures & dynamics & resolving commondesign forces

http://www.posa.uci.edu

132

The Evolution of TAOTAO ORB• Compliant with CORBA2.4 & some CORBA 3.0• AMI• INS• Portable Interceptors

• Pattern-centric design• Key capabilities

• QoS-enabled• Configurable• Pluggable protocols

• IIOP• UIOP• Shared memory• SSL• VME

• Open-source• Commercially supported

• www.theaceorb.com• Available now

• ZEN (RT Java/RT CORBA)

• www.zen.uci.edu

133

The Evolution of TAO

A/V STREAMING

DYNAMIC/STATICSCHEDULING

A/V Streaming Service• QoS mapping• QoS monitoring• QoS adaptationACE QoS API (AQoSA)• GQoS + RAPI• Integration with A/VStreaming ETA Winter2001

Static Scheduling• Rate monotonic analysisDynamic Scheduling• Earliest deadline first• Minimum laxity first• Maximal urgency firstHybrid Dynamic/Static• Demo in WSOA• ETA Winter 2001

134

The Evolution of TAODYNAMIC/STATIC

SCHEDULINGFT-CORBA

& LOAD BALANCING

FT-CORBA• Entity redundancy• Multiple models

• Cold passive• Warm passive• Active

• IOGR• ETA Winter 2001

Load Balancing• Static & dynamic• LOCATION_FORWARDING

SSL Support• Integrity• Confidentiality• Authentication (limited)Security Service• Authentication• Access control• Non-repudiation• Audit• ETA Winter 2001

A/V STREAMINGSECURITY

135

The Evolution of TAONOTIFICATIONS

A/V STREAMINGSECURITY

TRANSACTIONS

DYNAMIC/STATICSCHEDULING

FT-CORBA & LOAD

BALANCING

CORBA ComponentModel (CCM)• Extension Interfaces• Component navigation• Standardized life-cycles

• Dynamic configuration• QoS-enabledcontainers

• Reflective collocation• ETA Winter 2001

Notification Service• Structured events• Event filtering• QoS properties

• Priority• Expiry times• Order policy

• Compatible w/EventsObject TransactionService• Encapsulates RDBMs• ETA Winter 2001

136

Concluding Remarks

R&D

UserNeeds

StandardCOTS R&D

•Researchers & developers of distributedapplications face common challenges

R&D Synergies

•Patterns, frameworks, &components help to resolvethese challenges

•These techniques can yield efficient,scalable, predictable, & flexiblemiddleware & applications

•e.g., connection management,service initialization, error handling,flow & congestion control, eventdemuxing, distribution, concurrencycontrol, fault tolerancesynchronization, scheduling, &persistence

“Secrets” to R&D success:•Embrace & lead COTS standards•Leverage open-source•Be entrepreneurial & use the Web

•Solve “real” problems•See ideas thru to completion•Leave an enduring legacy

Documents

Pattern-Oriented Software Architecture · Pattern-Oriented Software Architecture Applying Concurrent & Networked Objects to Develop & Use Distributed Object Computing Middleware Dr