The Design and Performance of a CORBAAudio/Video Streaming Service

Sumedh Mungee, Nagarajan Surendran Douglas C. SchmidtYamuna Krishnamurthy

fsumedh,naga,yamunag@cs.wustl.edu fschmidtg@uci.eduDepartment of Computer Science, Washington University Electrical & Computer Engineering Dept.

St. Louis, MO 63130 University of California, Irvine, CA 92697�

This paper appeared a chapter in the bookDesign and Man-agement of Multimedia Information Systems: Opportunitiesand Challenges, edited by Mahbubur Syed and published byIdea Group Publishing, Hershey, USA, in 2001.

1 Introduction

Motivation: Advances in network bandwidth and CPU pro-cessing power have enabled the emergence of multimediaapplications, such as tele-conferencing or streaming video,that exhibit significantly more diverse and stringent quality-of-service (QoS) requirements than traditional data-orientedapplications, such as file transfer or email. For instance,popular Internet-based streaming mechanisms, such as Re-alvideo [RealNetworks, 1998] and Vxtreme [Vxtreme, 1998],allow suppliers to transmit continuous streams of audio andvideo packets to consumers. Likewise, non-continuous mediaapplications, such as medical imaging servers [Hu et al., 1998]and network management agents [Schmidt and Suda, 1994],employ streaming to transfer bulk data efficiently from sup-pliers to consumers.

However, many distributed multimedia applications rely oncustom and/or proprietary low-level stream establishment andsignaling mechanisms to manage and control the presenta-tion of multimedia content. These types of applications runthe risk of becoming obsolete as new protocols and servicesare developed [Huard and Lazar, 1998]. Fortunately, there is ageneral trend to move from programming custom applicationsmanually to integrating applications using reusable compo-nents based on open distributed object computing (DOC) mid-dleware, such as CORBA [Object Management Group, 1999],DCOM [Box, 1997], and Java RMI [Wollrath et al., 1996].

Although DOC middleware is well-suited to handle re-quest/response interactions among client/server applications,the stringent QoS requirements of multimedia applications

�This work was supported in part by AFOSR grant F49620-00-1-0330,Boeing, NSF grant NCR-9628218, DARPA contract 9701516, and Sprint.

have historically precluded DOC middleware from being usedas their data transfer mechanism [Pyarali et al., 1996].For instance, inefficient CORBA Internet Inter-ORBProtocol (IIOP) [Gokhale and Schmidt, 1999] implemen-tations perform excessive data-copying and memoryallocation per-request, which increases packet latency[Gokhale and Schmidt, 1998]. Likewise, inefficient marshal-ing/demarshaling in DOC middleware decreases streamingdata throughput [Gokhale and Schmidt, 1996].

As the performance of DOC middleware steadily improves,however, the stream establishment and control components ofdistributed multimedia applications can benefit greatly fromthe portability and flexibility provided by DOC middleware.Therefore, to facilitate the development of standards-baseddistributed multimedia applications, the Object ManagementGroup (OMG) has defined the CORBA Audio/Video (A/V)Streaming Service specification [OMG, 1997a], which definescommon interfaces and semantics necessary to control andmanage A/V streams.

The CORBA A/V Streaming Service specification definesan architecture for implementing open distributed multime-dia streaming applications. This architecture integrates (1)well-defined modules, interfaces, and semantics for stream es-tablishment and control with (2) efficient data transfer pro-tocols for multimedia data transmission. In addition todefining standard stream establishment and control mecha-nisms, the CORBA A/V Streaming Service specification al-lows distributed multimedia applications to leverage the inher-ent portability and flexibility benefits provided by standards-based DOC middleware.

Our prior research on CORBA middleware hasexplored the efficiency, predictability, and scalabil-ity aspects of ORB endsystem design, includingstatic [Schmidt et al., 1998a] and dynamic [Gill et al., 2001]scheduling, I/O subsystem [Kuhns et al., 1999] and plug-gable ORB transport protocol [O’Ryan et al., 2000]integration, synchronous [Schmidt et al., 2001] and asyn-

1

NETWORK

OS KERNEL

HIGH-SPEED

NETWORK INTERFACE

REAL-TIME I/OSUBSYSTEM

OS KERNEL

HIGH-SPEED

NETWORK INTERFACE

REAL-TIME I/OSUBSYSTEM

ACE COMPONENTS

REAL-TIME ORB COREIOP IOP

AV Service

SUPPLIER CONSUMER

ACE COMPONENTS ACE COMPONENTS

STREAM

ADAPTER

CONTROL AND

MANAGEMENT OBJECTS

STREAM

INTERFACE

AND CONTROL

OBJECTSPluggable

Protocol

STREAM

ADAPTER

STREAM

INTERFACE

AND CONTROL

OBJECTSPluggable

Protocol

AV Service

Figure 1: Layering of TAO’s A/V Streaming Service Atop theTAO ORB Endsystem

chronous [Arulanthu et al., 2000] ORB Core architectures,event processing [Harrison et al., 1997], optimization prin-ciple patterns for ORB performance [Pyarali et al., 1999],and the performance of various commercial and researchORBs [Gokhale and Schmidt, 1996, Schmidt et al., 1998b]over high-speed ATM networks. This chapter focuses onanother important topic in ORB endsystem research:thedesign and performance of the CORBA A/V Streaming Servicespecification.

The vehicle for our research on the CORBA A/V Stream-ing Service is TAO [Schmidt et al., 1998a]. TAO is a high-performance, real-time Object Request Broker (ORB) endsys-tem targeted for applications with deterministic and statisticalQoS requirements, as well as best effort requirements. TheTAO ORB endsystem contains the network interface, OS I/Osubsystem, communication protocol, and CORBA-compliantmiddleware components and services shown in Figure 1.

Figure 1 also illustrates how TAO’s A/V Streaming Ser-vice is built over the TAO ORB subsystem. TAO’sreal-time I/O (RIO) [Kuhns et al., 2000] subsystem runs inthe OS kernel and sends/receives requests to/from clientsacross high-speed, QoS-enabled networks, such as ATMor IP Integrated [Internet Engineering Task Force, 2000b] andDifferentiated [Internet Engineering Task Force, 2000a] Ser-vices. TAO’s ORB components, such as its ORB Core, Ob-

ject Adapter, stubs/skeletons, and servants, run in user-spaceand handle connection management, data transfer, endpointand request demultiplexing, concurrency, (de)marshaling, andapplication operation processing. TAO’s A/V Streaming Ser-vice is implemented atop its user-space ORB components. Atthe heart of TAO’s A/V Streaming Service is itspluggable A/Vprotocol framework. This framework provides the “glue” thatintegrates TAO’s A/V Streaming Service with the underlyingI/O subsystem protocols and network interfaces.

Chapter organization: The remainder of this chapter is or-ganized as follows: Section 0.2 illustrates how we appliedpatterns to develop and optimize the CORBA A/V StreamingService to support the standard OMG interfaces; Section 0.3describes two case studies that illustrate how to develop dis-tributed multimedia applications using TAO’s A/V Stream-ing Service and its pluggable A/V protocol framework; Sec-tion 0.4 presents the results of empirical benchmarks we con-ducted to illustrate the performance of TAO’s A/V Stream-ing Service; Section 0.5 presents concluding results. Forcompleteness, Appendix .1 outlines the intents of all the pat-terns applied in TAO’s A/V Streaming Service; Appendix .2summarizes the CORBA reference model and Appendix .3illustrates the various point-to-point and point-to-multipointstream and flow endpoint bindings implemented in TAO’s A/VStreaming Service.

2 The Design of TAO’s Audio/VideoStreaming Service

This section first presents an overview of the key archi-tectural components in the CORBA A/V Streaming Ser-vice. We then summarize the key design challenges facedwhen developing TAO’s CORBA A/V Streaming Serviceand outline how we applied patterns [Gamma et al., 1995,Buschmann et al., 1996, Schmidt et al., 2000] to resolve thesechallenges. Finally, we describe the design and performanceof the pluggable A/V protocol framework integrated intoTAO’s A/V Streaming Service.

2.1 Overview of the CORBA Audio/VideoStreaming Service Specification

The CORBA Audio/Video (A/V) Streaming Service specifi-cation [OMG, 1997a] defines an architectural model and stan-dard OMG IDL interfaces that can be used to build interopera-ble distributed multimedia streaming applications. Below, weoutline the architectural components and goals of the CORBAA/V Streaming Service specification.

2

2.1.1 Synopsis of Components in the CORBA A/VStreaming Service

The CORBA A/V Streaming Service specification definesflows as a continuous transfer of media between two multi-media devices. Each of these flows is terminated by aflowendpoint. A set of flows, such as audio flow, video flow anddata flow, constitute astream, which is terminated by astreamendpoint. A stream endpoint can have multiple flow endpoints.

Figure 2 shows amultimedia stream, which is represented asa flow between twoflow endpoints. One flow endpoint acts as

End-pointFlow data

AdaptorStream

(Source)

Flow data

Adaptor

ObjectsAnd Management

ObjectControlInterface

ORB CORE

POA

End-point

Stream

ControlInterface

Object

Stream

Stream

POA

Control

(Sink)

MULTIMEDIASTREAM

Figure 2: CORBA A/V Streaming Service Architecture

a source of the data and the other flow endpoint acts as a sink.Note that the control and signaling operations pass through theGIOP/IIOP-path of the ORB, demarcated by the dashed box.In contrast, the data stream usesout-of-bandstream(s), whichcan be implemented using communication protocols that aremore suitable for multimedia streaming than IIOP. Maintain-ing this separation of concerns is crucial to meeting end-to-endQoS requirements.

Each stream endpoint consists of three logical entities: (1) astream interface control objectthat exports an IDL interface,(2) adata source or sink, and (3) astream adaptorthat is re-sponsible for sending and receiving frames over a network.Control and Management objectsare responsible for the es-tablishment and control of streams. The CORBA A/V Stream-ing Service specification defines the interfaces and interactionsof the Stream Interface Control Objectsand the Control andManagement objects. Section 0.2.3 describes the various com-ponents in Figure 2 in detail.

2.1.2 Synopsis of Goals for the CORBA A/V StreamingService

The goals of the CORBA A/V Streaming Service include thefollowing:

Standardized stream establishment and control protocols:Using these protocols, consumers and suppliers can be devel-oped independently, while still being able to establish streamswith one another.

Support for multiple data transfer protocols: TheCORBA A/V Streaming Service architecture separates itsstream establishment and control protocols from its data trans-fer protocols, such as TCP, UDP, RTP, or ATM, thereby allow-ing applications to select the most suitable data transfer proto-cols for a particular network environment or set of applicationrequirements.

Provide interoperability of flows: A flow specificationispassed between two stream endpoints to convey per-flow in-formation, such as format, network host name and address,and flow protocol, required to bind or communication betweentwo multimedia devices.

Support many types of sources and sinks: Commonstream sources include video-on-demand servers, video cam-eras attached to a network, or stock quote servers. Com-mon sinks include video-on-demand clients, display devicesattached to a network, or stock quote clients.

2.2 Overview of Design Challenges and Resolu-tions

Below, we present an overview of the key challenges facedwhen we developed TAO’s CORBA A/V Streaming Serviceand outline how we applied patterns [Gamma et al., 1995,Schmidt et al., 2000] to resolve these challenges. Sec-tions 0.2.3 and 0.2.4 then examine these design and optimiza-tion pattern techniques in more depth. Appendix .1 outlinesthe intents of all the patterns applied in TAO’s A/V StreamingService.

Flexibility in stream endpoint creation strategies: TheCORBA A/V Streaming Service specification defines the in-terfaces and roles of stream components. Many performance-sensitive multimedia applications require fine-grained controlover the strategies governing the creation of their stream com-ponents. For instance, our past studies of Web server per-formance [Hu et al., 1997, Hu et al., 1998] motivate the needto supportadaptiveconcurrency strategies to develop efficientand scalable streaming applications.

In the context of our A/V Streaming Service, we deter-mined that the supplier-side of our MPEG case-study appli-cation (described in Section 0.3.1) required a process-basedconcurrency strategy to maximize stream throughput by al-lowing parallel processing of separate streams. Other typesof applications required different implementations, however.For example, the consumer-side of our MPEG application (de-scribed in Section 0.3.1) benefited from the creation of reactive

3

[Schmidt, 1995] consumers that contain all related endpointswithin a single process.

To achieve a high degree of flexibility, therefore, TAO’s A/VStreaming Service design decouples thebehaviorof streamcomponents from the strategies governing theircreation. Weachieved this decoupling via theFactory MethodandAbstractFactory patterns [Gamma et al., 1995], as described in Sec-tion 0.2.3.

Flexibility in data transfer protocol: A CORBA A/VStreaming Service implementation may need to selectfrom a variety of transfer protocols. For instance,an Internet-based streaming application, such as Re-alvideo [RealNetworks, 1998], may use the UDP protocol,whereas a local intranet video-conferencing tool [et al., 1996]might prefer the QoS features offered by native high-speedATM protocols. Likewise, RTP [Schulzrinne et al., 1994] isgaining acceptance as a transfer protocol for streaming audioand video data over the Internet. Thus, it is essential that aA/V Streaming Service support a range of data transfer proto-colsdynamically.

The CORBA A/V Streaming Service defines a simple spe-cialized protocolSimple Flow Protocol(SFP), which makesno assumptions about the communication protocols used fordata streaming and provides an architecture independent flowcontent transfer. Consequently, the stream establishment com-ponents in TAO’s A/V Streaming Service provide flexiblemechanisms that allow applications to define and use multi-ple network programming APIs, such as sockets and TLI, andmultiple communication protocols, such as TCP, UDP, RTP, orATM.

Therefore, another design challenge we faced was to definestream establishment components that can work with a varietyof data transfer protocols. To resolve this challenge, we ap-plied theStrategypattern [Gamma et al., 1995], as explainedin Section 0.2.3.

Providing a uniform API for different flow protocols:The CORBA A/V Streaming Service specification definesthe flow specification syntax that can be used for con-nection establishment. It defines the protocol names andsyntax for specifying the flow and data transfer proto-col information, but it does not define any interfacesfor protocol implementations. We resolved this omis-sion with ourpluggable A/V protocol framework(describedin Section 0.2.4) using design patterns, described in Ap-pendix .1, such asLayer[Buschmann et al., 1996] ,Acceptor-Connector[Schmidt et al., 2000],Facadeand Abstract Fac-tory [Gamma et al., 1995]. Moreover, TAO’s A/V StreamingService defines a uniform API for the different flow protocols,such as RTP and SFP, that can handle variations using the pol-icy interface described in Section 0.2.4.

Flexibility in stream control interfaces: A/V streamingmiddleware should provide flexible mechanisms that allow de-velopers to define and use different operations for differentstreams. For instance, a video application typically supportsa variety ofoperations, such asplay , stop , andrewind .Conversely, a stream in a stock quote application may supportother operations, such asstart andstop . Since the opera-tions provided by the stream are application-defined, it is use-ful for the control logic component in streaming middlewareto be flexible and adaptive.

Therefore, another design challenge facing designers ofCORBA A/V Streaming Services is to allow applications theflexibility to define their own stream control interfaces and ac-cess these interfaces in an extensible, type-safe manner. InTAO’s A/V Streaming Service implementation, we used theExtension Interface[Schmidt et al., 2000] pattern to resolvethis challenge.

Flexibility in managing states of stream supplier and con-sumers: The data transfer component of a streaming appli-cation often must change behavior depending on the currentstateof the system. For instance, invoking theplay opera-tion on the stream control interface of a video supplier maycause it to enter aPLAYING state. Likewise, sending it thestop operation may cause it to transition to theSTOPPED

state. More complex state machines can result due to addi-tional operations, such asrewind andfast forward op-erations.

Thus, an important design challenge for developers is de-signing flexible applications whose states can be extended.Moreover, in each state, the behavior of supplier/consumerapplications, and the A/V Streaming Service itself, must bewell-defined. To address this issue we applied theState Pat-tern [Gamma et al., 1995], as described in Section 0.3.1. TheState pattern is described in Appendix .1.

Providing a uniform interface for full and light profiles:To allow developers and applications to control and manageflows and streams, the CORBA A/V Streaming Service speci-fication exposes certain of their IDL interfaces. There are twolevels of exposure defined by the CORBA A/V Service: (1)the light profile, where only the stream and stream endpointinterfaces are exposed and the flow interfaces are not exposedand (2) thefull profile, where flow interfaces are also exposed.This two-level design provides more flexibility and granularityof control to applications and developers since flow interfacesare CORBA interfaces and are not locality constrained.

Therefore, the design challenge was to define a uniforminterface for both the light and full profiles to make useof TAO’s pluggable A/V protocol framework. We resolvedthis challenge by deriving the full and light profile end-points from a base interface and by generating the flowspecification using theForward FlowSpec Entry and

4

Reverse FlowSpec Entry classes, as mentioned in sec-tion 0.2.3.

Providing multipoint-to-multipoint bindings: Differentmultimedia applications require different stream endpointbindings. For example, video-on-demand applications re-quire point-to-point bindings between consumer and sup-plier endpoints whereas video-conferencing applications re-quire a multipoint-to-multipoint bindings. The CORBA A/Vspecification defines a point-to-multipoint binding, but not amultipoint-to-multipoint binding, which is left as a responsi-bility of implementors.

Thus, we faced the design challenge of providingmultipoint-to-multipoint bindings for applications that usemulticast protocols provided by the underlying network. Wehave provided a solution based on IP multicast and used toAdapter pattern [Gamma et al., 1995] to adapt it to ATM’smulticast model. The Adapter pattern is used to allow mul-tiple components to work together, even if they were not orig-inally designed to work together. This adaptation was doneby having TAO’s A/V Streaming Service set source ids for theflow producers so that the flow consumers can distinguish thesources. We added support in both SFP and RTP to allow themto be adapted for such bindings. Our implementation of Vic,described in Section 0.3.2, uses TAO’s A/V Streaming Servicemultipoint-to-multipoint binding and its RTP adapter.

2.3 CORBA A/V Streaming Service Compo-nents

The CORBA A/V Streaming Service specification defines aset of standard IDL interfaces that can be implemented to pro-vide a reusable framework for distributed multimedia stream-ing applications. Figure 3 illustrates the key components ofthe CORBA A/V Streaming Service. This subsection de-

MMDevice

MediaCtrl

ControllerStreamEndPoint

Stream

VDev VDevVDevMMDevice

Stream

MediaCtrl

StreamController

EndPoint

One per deviceOne per streamOne per deviceOne per stream

VDev

EndPointStream

MMDevice

MultimediaStreamMultimediaStream

Supplier

Consumer

Figure 3: A/V Streaming Service Components

scribes the design of TAO’s A/V Streaming Service compo-nents shown in Figure 3. The corresponding IDL interface

name for each role is provided in brackets. In addition, we il-lustrate how TAO provides solutions to the design challengesoutlined in Section 0.2.2.

2.3.1 Multimedia Device Factory (MMDevice)

An MMDevice abstracts the behavior of a multimedia device.The actual device can bephysical, such as a video microphoneor speaker, or belogical, such as a program that reads videoclips from a file or a database that contains information aboutstock prices. There is typically oneMMDevice per physicalor logical device.

For instance, a particular device might supportMPEG-1 [ISO, 1993] compression or ULAW audio[SUN Microsystems, 1992]. Such parameters are termed“properties” of theMMDevice. Properties can be asso-ciated with the MMDevice using the CORBA PropertyService [OMG, 1996], as shown in Figure 4.

MMDeviceMMDevicePropertiesProperties

Name (String)Name (String) Value (CORBA "Any" type)Value (CORBA "Any" type)

PropertySetPropertySet"Video_Format" "MPEG", "JPEG", "AVI"

"Movies" "Gandhi", "Star wars" ....

"Connections" 4

define_property ();

get_property_value ()

delete_property ();

Figure 4: Multimedia Device Factory

An MMDevice is also an endpoint factory that creates newendpoints for new stream connections. Each endpoint con-sists of a pair of objects: (1) a virtual device (VDev), whichencapsulates the device-specific parameters of the connectionand (2) theStreamEndpoint , which encapsulates the datatransfer-specific parameters of the connection. The roles ofVDev andStreamEndpoint are described in Section 0.2.3and Section 0.2.3, respectively.

The MMDevice component also encapsulates the imple-mentation ofstrategiesthat govern the creation of theVDevandStreamEndpoint objects. For instance, the implemen-tation of MMDevice in TAO’s A/V Streaming Service pro-vides the following two concurrency strategies:

Process-based strategy: The process-based concurrencystrategy creates new virtual devices and stream endpoints in anew process, as shown in Figure 5. This strategy is useful forapplications that create a separate process to handle each newendpoint. For instance, the supplier in our MPEG player appli-cation described in Section 0.3.1 creates separate processes tostream the audio and video data to the consumer concurrently.

5

Connection

requested

creates

VDev MediaCtrl

StreamEndpointChildProcess

Server Process

MMDevice

Figure 5:MMDevice Process-based Concurrency Strategy

Reactive strategy: In this strategy, endpoint objects for eachnew stream are created in the same process as the factory, asshown in Figure 6. Thus, a single process handles all the

VDev MediaCtrl

StreamEndpoint

Connection

requested

creates

ServerProcess

MMDevice

Figure 6:MMDevice Reactive Concurrency Strategy

simultaneous connectionsreactively [Schmidt, 1995]. Thisstrategy is useful for applications that dedicate one process tocontrol multiple streams. For instance, to minimize synchro-nization overhead, the consumer of the MPEG A/V player ap-plication described in Section 0.3.1 uses this strategy to createthe audio and video endpoints in the same process.

In TAO’s A/V Streaming Service, theMMDevice uses theAbstract Factorypattern [Gamma et al., 1995] to decouple (1)the creation strategy of the stream endpoint and virtual devicefrom (2) the concrete classes that define it. Thus, applica-tions that use theMMDevice can subclass both the strategiesdescribed above, as well as theStreamEndpoint and theVDev that are created.

The Abstract Factory pattern allows applications to cus-tomize the concurrency strategies to suit their needs. For in-stance, by default, the reactive strategy creates new streamendpoints using dynamic allocation,e.g., via the new op-erator in C++. Applications can override this behaviorvia subclassing so they can allocate stream endpoints us-ing other allocation techniques, such as thread-specific stor-age [Schmidt et al., 2000] or special framebuffers.

2.3.2 Virtual Device (VDev)

The virtual device (VDev) component is created by theMMDevice factory in response to a request for a new streamconnection. There is oneVDev per stream. TheVDev is usedby an application to define its response toconfigure re-quests. For instance, if a consumer of a stream wants to use

the MPEG video format, it can invoke theconfigure oper-ation on the supplierVDev, as shown in Figure 7.

Video_VDevVideo_VDevconfigure ();configure ();

VDevVDev

configureconfigure

configure (string name, Any value)

{

if (name == "video_format")

switch (value)

case "MPEG": use_mpeg ();

default: return Exception;

......

}

configure () = 0;configure () = 0;

("video_format",("video_format","MPEG");"MPEG");

Figure 7: Virtual Device

Stream establishmentis a mechanism defined by theCORBA A/V Streaming Service specification to permit thenegotiation of QoS parameters viaproperties. Properties arename-valuepairs, i.e., they have astring name and a cor-responding value. The properties used by the A/V Stream-ing Service are implemented using the CORBA Property Ser-vice [OMG, 1996].

The CORBA A/V Streaming Service specification spec-ifies the names of the common properties used by theVDev objects. For instance, the propertycurrformatis a string that contains the current encoding formate.g.,“MPEG.” During stream establishment, eachVDev can usethe get property value operation on its peerVDev toensure that the peer uses the same encoding format.

When a new pair ofVDev objects are created, eachVDevuses theconfigure operation on its peer to set the streamconfiguration parameters. If the negotiation fails, the streamcan be torn down and its resources released immediately.

Section 0.2.3 describes the CORBA A/V Streaming Servicestream establishment protocol in detail.

2.3.3 Media Controller (MediaCtrl )

The Media Controller (MediaCtrl ) is an IDL interface thatdefines operations for controlling a stream. AMediaCtrlinterface isnotdefined by the CORBA A/V Streaming Servicespecification. Instead, it is defined by multimedia applicationdevelopers to support operations for a specific stream, such asthe following IDL interface for a video service:

interface video_media_control{

void select_video (string name_of_movie);void play ();void rewind (short num_frames);void pause ();void stop ();

};

6

The CORBA A/V Streaming Service provides develop-ers with the flexibility to associate an application-definedMediaCtrl interface with a stream. Thus, the A/V Stream-ing Service can be used with an infinitely extensible variety ofstreams, such as audio and video, as well as non-multimediastreams, such as a stream of stock quotes.

The VDev object represented device-specific parameters,such as compression format or frame rate. Likewise, theMediaCtrl interface is device-specific since different de-vices support different control interfaces. Therefore, theMediaCtrl is associated with theVDev object using theProperty Service [OMG, 1996].

There is typically oneMediaCtrl per stream. In somecases, however, application developers may choose to controlmultiple streams using the sameMediaCtrl . For instance,the video and audio streams for a movie might have a commonMediaCtrl to enable a single CORBA operation, such asplay , to start both audio and video playback simultaneously.

2.3.4 Stream Controller (StreamCtrl )

The Stream Controller (StreamCtrl ) interface abstracts acontinuous media transfer between virtual devices (VDevs). Itsupports operations to bind twoMMDevice objects togetherusing a stream. Thus, theStreamCtrl component binds thesupplier and consumer of a stream,e.g., a video-camera and adisplay. It is the key participant in theStream Establishmentprotocol described in Section 0.2.3.

In general, aStreamCtrl object is instantiated by an ap-plication developer. There is oneStreamCtrl per stream,i.e., per consumer/supplier pair.

2.3.5 Stream Endpoint (StreamEndpoint )

TheStreamEndpoint object is created by anMMDevicein response to a request for a new stream. There is oneStreamEndpoint per stream. AStreamEndpoint en-capsulates the data transfer-specific parameters of a stream.For instance, a stream that uses UDP as its data transfer proto-col will identify its StreamEndpoint via a host name andport number.

In TAO’s A/V Streaming Service, theStreamEndpointimplementation uses patterns, such asDouble DispatchingandTemplate Method[Gamma et al., 1995], described in Ap-pendix .1, to allow applications to define and exchange datatransfer-level parameters flexibly. This interaction is shown inFigure 8 and occurs as follows:

Step 1: An A/V streaming application can inherit fromthe StreamEndpoint class and override the operationhandle connection requested in the new subclassTCP StreamEndpoint .

ConnectionConnectionRequestedRequested

returnreturn

"TCP=tango:8455""TCP=tango:8455"

connection_requested (..){ handle_connection_requested (); return flowspec;}

handle_connection_requested (..){ create transport endpoint; return "TCP=tango:8455";} TCP_StreamEndpointTCP_StreamEndpoint

StreamEndpointStreamEndpoint

Figure 8: Interaction BetweenStreamEndpoint and aMultimedia Application

Step 2: When binding twoMMDevices, theStreamCtrlinvokesconnect on oneStreamEndpoint with the peerTCP StreamEndpoint as a parameter.

Step 3: The StreamEndpoint then requests theTCP StreamEndpoint to establish the connection for thisstream using the network addresses it is listening on.

Step 4: The virtual handle connection requestedoperation of theTCP StreamEndpoint is invoked andconnects with the listening network address on the peer side.

Thus, by applying patterns, theStreamEndpoint designallows each application to configure its own data transfer pro-tocol, while reusing the generic stream establishment controllogic in TAO’s A/V Streaming Service.

2.3.6 Interaction Between Components in the CORBAAudio/Video Streaming Service Model

The preceding discussion in Section 0.2.3 described the struc-ture of components that constitute the CORBA A/V StreamingService. Below, we describe how these componentsinteracttoprovide two key A/V Streaming Service features:stream es-tablishmentandflexible stream control.

Stream establishment: Stream establishment is the processof binding two peers who need to communicate via astream.The CORBA A/V Streaming Service specification defines astandard protocol for establishing a binding between streams.Several A/V Streaming Service components are involved instream establishment. A key motivation for providing an elab-orate stream establishment protocol is to allow components tobe configured independently. This allows the stream establish-ment protocol to remain standard, and yet provide sufficienthooks for multimedia application developers to customize thisprocess for a specific set of requirements. For instance, anMMDevice can be configured to use one of several concur-rency strategies to create stream endpoints. Thus, at each stageof the stream establishment process, individual componentscan be configured to implement desired policies.

7

The CORBA A/V Streaming Service specification identifiestwo peers in stream establishment, which are known as the “A”party and the “B” party. These terms define complimentaryrelationships,i.e., a stream always has anA party at one endand aB party at the other. TheA party may be thesink, i.e.,the consumer, of a video stream, whereas theB party may bethesource, i.e., the supplier, of a video stream and vice versa.

Note that the CORBA A/V Streaming Service specificationdefines twodistinct IDL interfaces for theA andB party end-points. Hence, for a given stream, there will be two distincttypes for the supplier and the consumer. Thus, the CORBAA/V Streaming Service specification ensures that the compli-mentary relationship between suppliers and consumers is type-safe. An exception will be raised if a supplier tries to establisha stream with another supplier accidentally.

Stream establishment in TAO’s A/V Streaming Serviceoccurs in several steps, as illustrated in Figure 9. This

1) bind_devs (aMMDev,bMMDev);

aMMDevbMMDev

B_EndPointA_EndPoint

aVDev bVDev

aStreamCtrl

2.1) create_A

2.2) A_Endpoint, A_Vdev 2.4) B_EndPoint, B_VDev

2.3) create_B

4) connect

5) request_connection

3) configure

Figure 9: Stream Establishment Protocol in the A/V StreamingService

figure shows a stream controller (aStreamCtrl ) bindingthe A party together with theB party of a stream. Thestream controller need not be collocated with either end of astream. To simplify the example, however, we assume thatthe controller is collocated with theA party, and is called theaStreamCtrl . Each step shown in Figure 9 is explainedbelow:

1. The aStreamCtrl binds two Multimedia Device(MMDevice) objects together: Application developers in-voke thebind devs operation onaStreamCtrl . Theyprovide the controller with the object references of twoMMDevice objects. These objects are factories that createthe twoStreamEndpoint s of the new stream.

2. Stream Endpoint creation: In this step,aStreamCtrl requests the MMDevice objects,i.e., aMMDevice and bMMDevice , to create theStreamEndpoint s and VDev objects. TheaStreamCtrl invokes create A and create B op-erations on the twoMMDevice objects. These operations

request them to createA Endpoint and B Endpointendpoints, respectively.

3. VDev configuration: After the two peerVDev objectshave been created, they can use theconfigure operation toexchange device-level configuration parameters. For instance,these parameters can be used to designate the video format andcompression technique used for subsequent stream transfers.

4. Stream setup: In this step,aStreamCtrl invokesthe connect operation on theA Endpoint . This opera-tion instructs theA Endpoint to initiate a connection withits peer. TheA Endpoint initializes its data transfer end-points in response to this operation. In TAO’s A/V Stream-ing Service, applications can customize this behavior usingthe Double Dispatch[Gamma et al., 1995] pattern describedin Section 0.2.3.

5. Stream Establishment: In this step, theA Endpointinvokes therequest connection operation on its peerendpoint. TheA Endpoint passes its network endpointparameters,e.g., hostname and port number, as parame-ters to this operation. When theB Endpoint receives therequest connection operation, it initializes its end ofthe data transfer connection. It subsequently connects to thedata transfer endpoint passed to it by theA Endpoint .

After completing these five stream establishment protocolsteps, a data transfer-level stream is established between thetwo endpoints of the stream. Section 0.2.3 describes how theMedia Controller(MediaCtrl ) can control an establishedstream,e.g., by starting or stopping the stream.

Stream control: EachMMDevice endpoint factory can beconfigured with an application-definedMediaCtrl inter-face, as described in Section 0.2.3. Each stream has oneMediaCtrl and everyMediaCtrl controls one stream.Thus, if a particular movie has two streams, one for audioand the other for video, it will have twoMediaCtrl s. TheMediaCtrl is anExtension Interface described in Ap-pendix .1.

After a stream has been established by the stream con-troller, applications can obtain object references to theirMediaCtrl s from theirVDev. These object references con-trol the flow of data through the stream. For instance, avideo stream might support certain operations, such asplay ,rewind , andstop , and be used as shown below:

// The Audio/Video Streaming Service invokes this// application-defined operation to give the// application a reference to the media controller// for the stream.Video_Client_VDev::set_media_ctrl

(CORBA::Object_ptr media_ctrl,CORBA::Environment &env)

{// "Narrow" the CORBA::Object pointer into

8

// a media controller for the video stream.this->video_control_ =

Video_Control::_narrow (media_ctrl);}

The video control interface can be used to control the stream,as follows:

// Select the video to watch.this->video_control_->select_video ("gandhi");

// Start playing the video stream.this->video_control_->play ();

// Pause the video.this->video_control_->stop ();

// Rewind the video 100 frames.this->video_control_->rewind (100);

Flow specification: When binding two multimedia de-vices, a flow specification is passed between the twoStreamEndpoints to convey per-flow information. A flowspecification represents key aspects of a flow, such as its name,format, flow protocol being used, and the network name andaddress. A flow specification string is analogous to aninter-operable object reference(IOR) in the CORBA object model.The syntax for the interoperable flow specifications is shownin Figure 10. Standardizing the flow specifications ensures that

{"video\out\MIME:video/MPEG",video\TCP=ace.cs;5678"}

ST

RE

AM

EN

DP

OIN

T A

ST

RE

AM

EN

DP

OIN

T B

{"audio\out\MIME:audio/MPEG\SFP:1.0:credit=10\UDP=ace.cs:900","audio\UDP=danzon.cs:9000"}

Figure 10: Flow Specification

two differentStreamEndpoints from two different imple-mentations can interoperate. There are two different flow spec-ifications, depending on the direction in which the flowspec istraveling. If it is from theAparty’sStreamEndpoint to theB party’sStreamEndpointthen it is a “forward flowspec;” theopposite direction is the “reverse flowspec.”

TAO’s CORBA A/V Streaming Service implementa-tion defines two classes,Forward FlowSpec Entry andReverse FlowSpec Entry , that allow multimedia appli-cations to construct theflow specification stringfrom theircomponents without worrying about the syntactic details. Forexample, the entry takes the address as both anINET Addrand astring and provides convenient parsing utilities forstrings.

2.4 The Design of a Pluggable A/V ProtocolFramework for TAO’s A/V Streaming Ser-vice

At the heart of TAO’s A/V Streaming Service is itspluggableA/V protocol framework, which defines a common interfacefor various flow protocols, such as TCP, UDP, RTP, or ATM.This framework provides the “glue” that integrates its ORBcomponents with the underlying I/O subsystem protocols andnetwork interfaces. In this section, we describe the design ofthe pluggable A/V protocol framework provided in TAO’s A/VStreaming Service and describe how we resolved key designchallenges that arose when developing this framework.

2.4.1 Overview of TAO’s Pluggable A/V Protocol Frame-work

The pluggable A/V protocol framework in TAO’s A/V Stream-ing Service consists of the components shown in Figure 11.Each of these components is described below.

RTP RTCP SFP NULL

UDPUDPMCAST

ATMAAL5

TCP

FLOW PROTOCOLFACTORY

FLOW TRANSPORTFACTORY

FLOW PROTOCOL COMPONENT

FLOW TRANSPORT COMPONENT

Figure 11: Pluggable A/V Protocol Components in TAO’s A/VStreaming Service

AV Core: This singleton [Gamma et al., 1995] componentis a container for flow and data transfer protocol factories.An application using TAO’s A/V implementation must ini-tialize this singleton before using any of its A/V classes,such asStreamCtrl and MMDevice. During initializa-tion, the AV Core class loads all the flow protocol fac-tories, control protocol factories, and data transfer facto-ries dynamically using theService Configurator pat-tern [Schmidt et al., 2000] and creates default instances foreach known protocol.

Data Transfer components: The components illustrated inFigure 12 and described below are required for each data trans-fer protocol:

9

ACE_SERVICE_OBJECT

TRANSPORTFACTORY

TRANSPORTACCEPTOR

TRANSPORTCONNECTOR

TRANSPORTHANDLER

TRANSPORT

TAO_AV_FLOWHANDLER

< < INSTANTIATES > >

< < INSTANTIATES > >

match_protocol(protocol_name)

Figure 12: TAO’s A/V Streaming Service Pluggable DataTransfer Components

� Acceptor and Connector: These classes are im-plementations of the Acceptor-Connector pat-tern [Schmidt et al., 2000], which are used to acceptconnections passively and establish connections actively,respectively.

� Transport Factory: This class is an abstract fac-tory [Gamma et al., 1995] that provides interfaces to createAcceptor s andConnector s in accordance to the appro-priate type of data transfer protocol.

� Flow Handler: All data transfer handlers derive fromthe Flow Handler class, whose methods can start, stop,and provide flow-specific functionality for timeout upcalls totheCallback objects, which are described in the followingparagraph.

Callback interface: TAO’s A/V Streaming Service usesthis callback interface to deliver frames and to notifyFlowEndPoints of start and stop events. Multimedia ap-plication developers subclass this interface for each flow end-point, i.e., there are producer and consumer callbacks. TAO’sA/V Streaming Service dispatches timeout events automati-cally so that applications need not write event handling mech-anisms. For example, all flow producers are automatically reg-istered for a timer events with a Reactor. The value for thetimeout is obtained through theget timeout hook methodon the Callback interface. This hook method is calledwhenever a timeout occurs since multimedia applications typ-ically have adaptive timeout values.

Flow protocol components: Flow protocols carry in-bandinformation for each flow that a receiver can use to reproducethe source stream. The following components are required for

each flow protocol supported by TAO’s A/V Streaming Ser-vice:

� Flow Protocol Factory: This class is an abstract fac-tory that creates flow protocol objects.

� Protocol Object: This class defines flow protocol func-tionality. Applications use this class to send frames and theProtocol Object uses application-specifiedCallbackobjects to deliver frames.

Figure 13 illustrates the relationships among the flow proto-col components in TAO’s pluggable A/V protocol framework.

FLOW PROTOCOLFACTORY

PROTOCOL OBJECT

CALLBACK

FLOW HANDLER TRANSPORT

ACE SERVICE OBJECT

< < INSTANTIATES > >

Control Factory

Figure 13: TAO’s A/V Streaming Service Pluggable A/V Pro-tocol Components

AV Connector and AV Acceptor Registry: As mentionedabove, different data transfer protocols require the creation ofcorresponding data transfer factories, acceptors, and connec-tors. TheAV Core class creates theAV Connector andAV Acceptor registry classes to provide a facade that main-tains and accesses the abstract flow and data transfer factoriesboth forlight andfull profile objects. This design gives users asingle interface that hides the complexity of creating and ma-nipulating different data transfer factories.

2.4.2 Applying Patterns to Resolve Design Challenges forPluggable A/V Protocols Frameworks

Below, we outline the key design challenges faced whendeveloping TAO’s pluggable A/V protocol framework anddiscuss how we resolved these challenges by applying var-ious patterns [Gamma et al., 1995, Buschmann et al., 1996,Schmidt et al., 2000].

Adding new data transfer protocols transparently:

� Context: Different multimedia applications often havedifferent QoS requirements. For example, a video applica-tion over an intranet may want to take advantage of native

10

ATM protocols to reserve bandwidth. An audio applicationin a video-conferencing application may want to use a reli-able data transfer protocol, such as TCP, since loss of audiois more visible to users than video and the bit-rate of audioflows are low (�8 kbps using GSM compression). In con-trast, a video application might not want the overhead of re-transmission and slow-start congestion protocol incurred by aTCP [Stevens, 1993]. Thus, it may want to use the facilities ofan unreliable data transfer protocol, such as UDP, since losinga small number of frames may not affect perceived QoS.

� Problem: It should be possible to add new data transferprotocols to TAO’s pluggable A/V protocol framework with-out modifying the rest of TAO’s A/V Streaming Service. Thus,the framework must be open for extensions but closed to mod-ifications,i.e., the Open/Closed principle [Meyer, 1989]. Ide-ally, creating a new protocol and configuring it into TAO’spluggable A/V protocol framework should be all that is re-quired.

� Solution: Use a registry to maintain a collectionof abstract factoriesbased on the Abstract Factory pat-tern [Gamma et al., 1995]. In this pattern, a single classdefines an interface for creating families of related objects,without specifying their concrete types. Subclasses of ab-stract factories are responsible for creating concrete classesthat collaborate amongst themselves. In the context of plug-gable A/V protocols, each abstract factory can create concreteConnector andAcceptor classes for a particular proto-col.

� Applying this solution in TAO’s A/V Stream-ing Service: In TAO’s A/V Streaming Service, theConnector Registry plays the role of the protocol reg-istry. This registry is created by theAV Core class. Fig-ure 14 depicts theConnector Registry and its re-lation to the abstract factories. These factories are ac-

TCP ATMAAL5

UDP

AB

ST

RA

CT

FA

CT

OR

Y

AB

ST

RA

CT

FA

CT

OR

Y

AB

ST

RA

CT

FA

CT

OR

Y

CONNECTOR REGISTRY / ACCEPTOR REGISTRY

Figure 14: Connector Registry

cessed via a facade defined according to theFacadepat-tern [Gamma et al., 1995]. This design hides the complexityof manipulating multiple factories behind a simpler interface.

The Connector Registry described above plays the fa-cade role.

Adding new A/V protocols transparently:

� Context: Multimedia flows often require a flow proto-col since most multimedia flows need to carry in-band infor-mation for the receiver to reproduce the source stream. Forexample, every frame may need a timestamp so that the re-ceiver can play the frame at the right time. Moreover, sequencenumbers will be needed if a connectionless protocol, such asUDP, is used so that the applications can do resequencing. Inaddition, multicast flows may require information, such as asource identification number, to demultiplex flows from dif-ferent sources.

SFP is a simple flow protocol defined by the CORBAA/V Streaming Service specification to transport in-band data. Likewise, the Real-time Transport Protocol(RTP) [Schulzrinne et al., 1994] defines facilities to transportin-band data. RTP is Internet-centric, however, and cannotcarry CORBA IDL-typed flows directly. For example, RTPspecifies that all header fields should be in network-byteorder, whereas the SFP uses CORBA’s CDR encoding andcarries the byte-order in each header.

� Problem: Flow protocols should be able to run over dif-ferent data transfer protocols. This configuration of a flow pro-tocol over different data transfer protocol should be done eas-ily and transparently to the application developers and users.

� Solution: To solve the problem of a flow protocolrunning over different data transfer protocols, we appliedtheLayerspattern [Buschmann et al., 1996] described in Ap-pendix .1. We have structured the flow protocols and datatransfer protocols as two different layers. The flow protocollayer creates the frames with the in-band flow information.The data transfer layer performs the connection establishmentand sends the frames that are sent down from the flow proto-col layer onto the network. The layered approach makes theflow and data transfer protocols independent of each other andhence it is easy to tie different flow protocols with differentdata transfer protocols transparently.

� Applying this solution in TAO’s A/V Streaming Ser-vice: TAO’s A/V Streaming Service provides a uniformdata transfer layer for a variety of flow protocols, includ-ing UDP unicast, UDP multicast, and TCP. TAO’s A/VStreaming Service provides a flow protocol layer using aProtocol Object interface. Likewise, itsAV Core classmaintains a registry of A/V protocol factories.

Adding new protocols dynamically:

11

� Context: When developing new pluggable A/V proto-cols, it is inconvenient to recompile TAO’s A/V StreamingService and applications just to validate a new protocol imple-mentation. Moreover, it is often useful to experiment with dif-ferent protocols to compare their performance, footprint size,and QoS guarantees systematically. Moreover, in telecom sys-tems with 24�7 availability requirements, it is important toconfigure protocols dynamically, even while the system is run-ning. This level of flexibility helps simplify upgrades and pro-tocol enhancements.

� Problem: The user would like to populate the registrydynamicallywith a set of factories during run-time and avoidthe inconvenience of recompiling the AV Service and the ap-plications when different protocols are plugged in. The solu-tion explains how we can achieve this.

� Solution: We can solve the above stated problem usingtheService Configuratorpattern [Schmidt et al., 2000], whichdecouples the implementation of a component from the pointin time when it is configured into the application. By using thispattern, a pluggable A/V protocol framework can dynamicallyload the set of entries in a registry. For instance, a registry cansimply parse a configuration script and dynamically link theservices listed in it.

� Applying this solution in TAO’s A/V Streaming Ser-vice: TheAV Core class maintains all parameters specifiedin a configuration script. Adding a new parameter to repre-sent the list of protocols is straightforward,i.e., the defaultregistry simply examines this list and links the services intothe address-space of the application, using the ACE ServiceConfigurator implementation [Schmidt and Suda, 1994]. ACEprovides a rich set of reusable and efficient components forhigh-performance, real-time communication, and forms theportability layer of TAO’s A/V Streaming Service. Figure 15depicts the connector registry and its relation to the ACE Ser-vice Configurator framework, which is an implementation ofthe Component Configurator pattern [Schmidt et al., 2000].

Control protocols:

� Context: RTP has a control protocol – RTCP – asso-ciated with it. Every RTP participant must transmit RTCPframes that provide control information, such as the name ofthe participant and the tool being used. Moreover, RTCP sendsreception reports for each of its sources.

� Problem: Certain flow protocols, such as SFP, use A/Vinterfaces to exchange control interfaces. The use of RTP fora flow necessitates it to transmit RTCP information. RTCP ex-tracts this control information from RTP packets. Therefore,TAO’s A/V Streaming Service must provide an extensible in-terface for these control protocols, as well as provide a meansfor interacting between the data and control protocols.

ACCEPTOR / CONNECTOR REGISTRY

TRANSPORT FLOW PROTOCOL

RTCPTCP SFP RTPUDPATMAAL5

TC

P F

AC

TO

RY

SF

P F

LO

WF

AC

TO

RY

RT

P F

LO

WF

AC

TO

RY

UD

P F

AC

TO

RY

RT

CP

FL

OW

FA

CT

OR

Y

AA

L5

FA

CT

OR

Y

< < I N S T A N T I A T E S > >

SERVICE CONFIGURATOR

Figure 15: Acceptor-Connector Registry and Service Config-urator

� Solution: The solution is to make the control proto-col information part of the flow protocol. For example, RTPknows that RTCP is its control protocol. Therefore, to reusepluggability features, it may be necessary to make the controlprotocol use the same interfaces as its data components.

� Applying this solution in TAO’s A/V Streaming Ser-vice: During stream establishment,Registry objects willfirst check the flow factory for the configured flow protocol.After thelisten orconnect operation has been performedfor a particular data flow, theRegistry will check if the flowfactory has a control factory. If so, it will perform the sameprocessing for the control factory, except the network endpointport will be one value higher than the data endpoint. Since theCORBA A/V Streaming Service specification does not definea portable way to specify control flow endpoint information,we followed this approach as a temporary solution until theOMG comes up with a portable solution.

The RTCP implementation in TAO’s A/V Streaming Ser-vice uses the same interfaces that RTP does, includingthe Flow Protocol Factory and Protocol Objectclasses. Thus, RTP will call thehandle control inputmethod on the RTCPProtocol Object when a RTP frameis received. This method enables the RTCP object to extractthe necessary control information, such as the sequence num-ber of the last frame.

Interface for variations in flow protocols:

� Context: Above, we explained how TAO’s pluggableA/V protocol framework factors out different flow protocolsand provides a uniform flow protocol interface. In certaincases, however, there are inherent variations in such protocols.

12

For example, RTP must transmit the payload type,i.e., the for-mat of the flow in each frame, whereas SFP uses the controland management interface in TAO’s A/V Streaming Service toset and get the format values for a flow.

Similarly, the RTP control protocol, RTCP, periodicallytransmits participant information, such as the senders nameand email address, whereas SFP does not transmit such infor-mation. Such information does not changed with every frame,however. For example, the name and email address of a partic-ipant in a conference will not change for a session. In addition,the properties of the transfer may need to be controlled by ap-plications. For instance, a conferencing application may notwant to have multicast loopback.

� Problem: An A/V Streaming Service should allow end-users to set protocol-specific variations, while still providinga single interface for different flow protocols. Moreover, thisinterface should be open to changes with the addition of newflow protocol and data transfer protocols.

� Solution: The solution to the above problem is to applytheCORBA Policyframework defined in the CORBA specifi-cation [Object Management Group, 1999]. The CORBA Pol-icy framework allows the protocol component developer to de-fine policy objects that control the behavior of the protocolcomponent. The policy object is derived from the CORBAPolicy interface [Object Management Group, 1999] whichstores the Policy Type [Object Management Group, 1999] andthe associated values.

� Applying this solution in TAO’s A/V Streaming Ser-vice: By defining a policy framework, which is extensibleand follows the CORBA Policy model, the users will haveshorter learning curve to the API and be able to add newflow protocols flexibly. We have defined different policy typesused by different flow protocols that can be accessed by thespecific transport and flow protocol components during framecreation and dispatching. For example we have defined theTAO AV PAYLOAD TYPE POLICY which allows the RTPprotocol to specify the payload type.

3 Case Study: Developing Multime-dia Applications using TAO’s A/VStreaming Service

To evaluate the capabilities of the CORBA-based A/V Stream-ing Service, we have developed several multimedia applica-tions that use the components and interfaces described in Sec-tion 0.2. Thus, this section describes the design of two dis-tributed multimedia applications that use TAO’s A/V Stream-ing Service and pluggable A/V protocol framework to estab-lish and control MPEG and interactive audio/video streams.

3.1 Case Study 1: an MPEG A/V StreamingApplication

This application is an enhanced version of a non-CORBAMPEG player developed at the Oregon Graduate Insti-tute [Chen et al., 1995]. Our application plays movies usingtheMPEG-1video format [ISO, 1993] and the SunULAWau-dio format [SUN Microsystems, 1992]. Figure 16 shows thearchitecture of our A/V streaming application.

Naming ServiceNaming Service

`

MMOVIESOVIES

MPEG

MOV

JPEG

MMDeviceMMDevice

ControlControl(Media(Media

Controller)Controller)

Media(UDP)

SupplierSupplierConsumerConsumer

RegisterRegisterResolveResolve

Figure 16: Architecture of the MPEG A/V Streaming Appli-cation

The MPEG player application uses a supplier/consumer de-sign implemented using TAO. The consumer locates the sup-plier using the CORBA Naming Service [OMG, 1997b] or theTrading Service [OMG, 1997b] to find suppliers that matchthe consumer’s requirements. For instance, a consumer mightwant to locate a supplier that has a particular movie or a sup-plier with the least number of consumers currently connectedto it.

Once a consumer obtains the supplier’sMMDevice objectreference it requests the supplier to establish two streams,i.e.,a video stream and an audio stream, for a particular movie.These streams are established using the protocol described inSection 0.2.3. The consumer then uses theMediaCtrl tocontrol the stream, as described in Section 0.2.3.

The supplier is responsible for sending A/V packets viaUDP to the consumer. For each consumer, the supplier trans-mits two streams, one for the MPEG video packets and one forthe Sun ULAW audio packets. The consumer decodes thesestreams and plays these packets in a viewer, as shown in Fig-ure 17.

This section describes the various components of the con-sumer and supplier. The following table illustrates the numberof lines of C++ source required to develop this system and ap-plication.

Component Lines of codeTAO CORBA ORB 61,524TAO Audio/Video (A/V) Streaming Service 3,208TAO MPEG video application 47,782

13

Figure 17: A TAO-enabled Audio/Video player

Using the ORB and the A/V Streaming Service greatly re-duced the amount of software that otherwise would have beenwritten manually.

3.1.1 Supplier Architecture

The supplier in the A/V streaming application is responsiblefor streamingMPEG-1video frames andULAWaudio samplesto the consumer. The files can be stored in a filesystem acces-sible to the supplier process. Alternately, the video frames andthe audio packets can be sent by live source, such as a videocamera. Our experience with the supplier indicates that it cansupport�10 concurrent consumers simultaneously on a dual-CPU 187 Mhz Sun Ultrasparc-II with 256 MB of RAM over a155 mbps ATM network.

The role of the supplier is to read audio and video framesfrom a file, encode them, and transmit them to the consumeracross the network. Figure 18 depicts the key components inthe supplier architecture.

MMOVIESOVIES

CCONTROLONTROL

DDATAATA

CCONTROLONTROL

DDATAATA

VVIDEOIDEO AAUDIOUDIO

CCONNECTION ONNECTION HHANDLERSANDLERS

createscreatesSSERVERERVER

PPROCESSROCESS

Figure 18: TAO Audio/Video Supplier Architecture

The main supplier process contains anMMDevice endpointfactory described in Section 0.2.3. ThisMMDevice createsconnection handlers in response to consumer connections, us-ing process-based concurrency strategy. Each connection trig-gers the creation of one audio process and one video process.These processes respond to multiple events. For instance, thevideo supplier process responds to CORBA operations, suchasplay andrewind , and sends video frames periodically inresponse to timer events.

Each component in the supplier architecture is described be-low:

The Media controller component: This component in thesupplier process is a servant that implements the Media Con-troller interface (MediaCtrl ) described in Section 0.2.3.The Media Controller responds to CORBA operations fromthe consumer. The interface exported by theMediaCtrlcomponent represents the various operations supported by thesupplier, such asplay , rewind , andstop .

At any point in time, the supplier can be in several states,such asPLAYING, REWINDING, or STOPPED. Depending onthe supplier’s state, its behavior may change in response toconsumer operations. For instance, the supplier ignores aconsumer’splay operation when the supplier is already inthe PLAYING state. Conversely, when the supplier is in theSTOPPEDstate, a consumerrewind operation transitions thesupplier to theREWINDING state.

The key design forces that must be resolved while imple-mentingMediaCtrl s for A/V streaming are (1) allowing thesame object to respond differently, based on its current state,(2) providing hooks to add new states, and (3) providing ex-tensible operations to change the current state.

To provide a flexible design that meet these requirements,the control component of our MPEG player application is im-plemented using theStatepattern [Gamma et al., 1995]. Thisimplementation is shown in

Figure 19. TheMediaCtrl has astate object pointer.

Media Controller

play ()rewind ()stop ()

Media State

play () = 0;rewind () = 0;stop () = 0;

Playing State

play ()rewind ()stop ()

Stopped State

play ()rewind ()stop ()

state

state->play ();

Figure 19: State Pattern Implementation of the Media Con-troller

The object being pointed to by the Media Controller’sstatepointer represents the current state. For simplicity, the fig-

14

ure shows thePlaying State and theStopped State ,which are subclasses of theMedia State abstract baseclass. Additional states, such as theRewinding State ,can be added by subclassing fromMedia State .

The diagram lists three operations:play , rewind andstop . When the consumer invokes an operation on theMedia Controller , this class delegates the operation tothe state object. A state object implements the response toeach operation in a particular state. For instance, therewindoperation in thePlaying State contains the response ofthe media controller to therewind operation when it is in thePLAYING state. State transitions can be made by changing theobject being pointed to by thestate pointer of theMediaController .

In response to consumer operations, the currentstate ob-ject instructs the data transfer component discussed in Sec-tion 0.3.1 to modify the stream flow. For instance, whenthe consumer invokes therewind operation on theMediaController while in theSTOPPEDstate, therewind oper-ation in theStopped State object instructs the data com-ponent to play frames in reverse chronological order.

The Data transfer component: The data component is re-sponsible for transferring data to the consumer. Our MPEGsupplier application reads video frames from aMPEG-1fileand audio frames from a SunULAWaudio file. It sends theseframes to the consumer, fragmenting long frames if necessary.The current implementation of the data component uses theUDP protocol to send A/V frames.

A key design challenge related to data transfer is to have theapplication respond to CORBA operations for the stream con-trol objects,e.g, theMediaCtrl , as well as the data transferevents,e.g., video frame timer events. An effective way todo this is to use theReactorpattern [Schmidt et al., 2000], asshown in Figure 20. TheReactor pattern is described inAppendix .1.

ORBORBDescriptorDescriptor

TimerTimer Data (UDP)Data (UDP)

OS EVENT DEMULTIPLEXING INTERFACE

: Periodic: PeriodicVideo frameVideo frametransmittertransmitter

: Reactor: Reactor

: Feedback: FeedbackHandlerHandler

CORBA ORBCORBA ORB

: Media: MediaControllerController

Figure 20: Reactive Architecture of the Video Supplier

The video supplier registers two event handlers with TAO’s

ORB Reactor . One is a signal handler for the video frametimer events. The other is a UDP socket event handler forfeedback events coming from the consumer. The frames sentby the data component correspond to the current state of theMediaCtrl object, as outlined above. Thus, in thePLAYING

state, the data component plays the audio and video frames inchronological order.

Future implementations of the data transfer component inour MPEG player application will support multiple encodingprotocols via the simple flow protocol (SFP) [OMG, 1997a].SFP encoding encapsulates frames of various protocols withinan SFP frame. It provides standard framing and sequencenumbering mechanisms. SFP uses the CORBA CDR encodingmechanism to encode frame headers and uses a simplecredit-basedflow control mechanism described in [OMG, 1997a].

3.1.2 Consumer Architecture

The role of the consumer is to read audio and video frames offthe network, decode them, and play them synchronously. Theaudio and video servers stream the frames separately. A/Vframe synchronization is performed on consumer. Figure 21depicts the key components in the consumer architecture:

VVIDEOIDEO

BBUFFERUFFER

AAUDIOUDIO

BBUFFERUFFER

VVIDEOIDEO

DDECODEECODE

"R"RAWAW""

PPACKETSACKETS

DDECODED ECODED MMPEGPEG

VVIDEO IDEO PPACKETSACKETS

Video ControlVideo Control

Audio ControlAudio Control

CCONTROLONTROL/A/AUDIOUDIO

GUI/VGUI/VIDEOIDEO

CommandsCommands

Figure 21: TAO Audio/Video Consumer Architecture

The original non-CORBA MPEG con-sumer [Chen et al., 1995] used a process-based concurrencyarchitecture. Our CORBA-based consumer maintain thisarchitecture to minimize changes to the code. Separate pro-cesses are used to do the buffering, decoding, and playback,as explained below:

1. Video buffer: The video buffering process is responsi-ble for reading UDP packets from the network and enqueueingthem in shared memory. The Video Decoder process dequeuesthese packets and performs MPEG decoding operations onthem.

2. Audio buffer: Similarly, the audio buffering processis responsible for reading UDP packets of the network andenqueueing them in shared memory. The Control/Audio

15

Playback process dequeues these packets and sends them to/dev/audio .

3. Video decoder: The video decoding process reads theraw packets sent to it by the Video Buffer process and decodesthem according to the MPEG-1 video specification. These de-coded packets are sent to the GUI/Video process, which dis-plays them.

4. GUI/Video process: The GUI/Video process is responsi-ble for the following two tasks:

� GUI: It provides a GUI to the user, where the user canselect operations likeplay , stop , andrewind . These op-erations are sent to the Control/Audio process via a UNIX do-main socket [Stevens, 1998].

� Video: This component is responsible for displayingvideo frames to the user. The decoded video frames are storedin a shared memory queue.

5. Control/Audio playback process: The Control/Audioprocess is responsible for the following tasks:

� Control: This component receives control messagesfrom the GUI process and sends the appropriate CORBA op-eration to theMediaCtrl servant in the supplier process.

� Audio playback: The audio playback component is re-sponsible for dequeueing audio packets from the Audio Bufferprocess and playing them back using the multimedia soundhardware. Decoding is unnecessary because the supplier usesthe ULAW format. Therefore, the data received can be directlywritten to the sound port, which is/dev/audio on Solaris.

3.2 Case Study 2: The Vic Video-ConferencingApplication

Vic [McCanne and Jacobson, 1995] is a video-conferencingapplication developed at the University of California, Berke-ley. We have adapted Vic to use TAO’s A/V Streaming Ser-vice components and its pluggable A/V protocol frameworkdescribed in Section 0.2. The Vic implementation in TAO usesRTP/RTCP as its flow and data transfer protocols.

3.2.1 Architecture of VicVic provides a video-conferencing application. Audio con-ferencing is done with another tool, Vat [NRG, LBNL, 1995].The Vic family of tools synchronize media streams using aconference bus mechanism, which is the “localhost” synchro-nization mechanisms used via loopback sockets.

The Architecture of Vic is driven largely by theTclObject interface [McCanne and Jacobson, 1995].TclObject provides operations so that operations on theobject can be invoked from a Tcl script. By using Tcl,

Vic allows rapid prototyping and reconfiguration of itsencode/decode paths.

One design challenge we faced while adapting Vic to useTAO’s A/V Streaming Service was to integrate both the GUIand ORB event loops. This was solved using the Reactorpattern [Schmidt et al., 2000]. In particular, we developed aReactor wrapper facade [Schmidt et al., 2000] that unified theGUI and ORB into a single event loop.

3.2.2 Implementing Vic using TAO’s A/V Streaming Ser-vice

Below, we discuss the steps we followed to adapt Vic to useTAO’s A/V Streaming Service.

1. Structuring of conferencing protocols: In this step, wedecomposed the flow, control and data transfer protocols us-ing TAO’s pluggable A/V protocol framework. The originalVic application was highly coupled with RTP. For instance,its encoders and decoders were aware of the RTP headers.We decoupled the dependencies of the encoders/decoders fromRTP-specific details by using theframe info structure andusing TAO’s A/V Streaming ServiceProtocol Object in-terface. The modified Vic still preserves the application-levelframing (ALF) [Clark and Tennenhouse, 1990] model embod-ied in RTP. Moreover, Vic’s RTCP functionality was ab-stracted into the TAO’s pluggable A/V protocol framework, sothe framework automatically defines a RTCP flow for a RTPflow. The modified Vic is independent from the network spe-cific details of opening connections and I/O handling sinceit uses the pluggable A/V protocol framework provided byTAO’s A/V Streaming Service.

Vic uses the multipoint-to-multipoint binding provided byTAO’s A/V Streaming Service, which is described in Ap-pendix .3. Thus, our first step when integrating into TAOwas to determine the proper abstraction for the conferencedevice. A video-conferencing application like Vic serves asboth a source and sink; thus, we needed a source and sinkMMDevice. Moreover, to be extensible for future integrationwith Vat and other multimedia tools, Vic uses flow interfaces,i.e., video is considered as a flow within the conference stream.Since Vat runs in a separate address space, its flow interfacesmust be exposed using TAO’s full profile flow interfaces,i.e.,FDev, FlowProducer , andFlowConsumer .

2. Define callback objects: In this step, we de-fined Callback objects for all the source and sinkFlowEndPoint s. TheSource Callback uses the timerfunctionality to schedule timer events to send the frames. Fig-ure 22 illustrates the sequence of events that trigger the send-ing of frames. When the input becomes ready on the videocard, the grabber reads it and gives it to thetransmitter .

16

RENDERER

DECODERS

H.261 JPEG CELLB

RTPCALLBACK

RTCPCALLBACK

RTPOBJECT

RTPTRANSPORTHANDLER

RTCPTRANSPORT

HANDLER

RTCPOBJECT

RTCPOBJECT

TRANSPORT

ENCODERS

RTPOBJECT

TRANSPORT

CAPTURE

H.261 JPEG CELLB

Bitrate Buffer

Tim

eo

ut

Video

Input Input1

2

3

Timeout

Frame Ready

NETWORK

NETWORK

REACTOR

Figure 22: Architecture of Vic using TAO’s A/V StreamingService

The transmitter then uses theSource Callback ob-ject to schedule a timer to send the frames at the requested bitrate using a bitrate buffer.

On the sink-side, when a packet arrives on the network thereceive frame upcall is done on theSink Callbackobject which using theframe info structure gives it to theright Source object , which then passes it to the right de-coder. To implement RTCP functionality, Vic implements aRTCPCallback to provide Vic-specific source objects.

3. Select a centralized or distributed conference config-uration: In this step, we have ensured that Vic can func-tion both as a participant in a centralized conference, as wellas a loosely-coupled distributed conference. This flexibil-ity is achieved by checking for aStreamCtrl object inthe Naming Service and creating newStreamCtrl if oneis not found in the Naming Service. Thus, by running aStreamCtrl control process that registers itself with theNaming Service, all Vic participants will become part of acentralized conference, which can be controlled from the con-trol process. Conversely, when no such process is run, Vicreverts to the loosely controlled model by creating its ownStreamctrl and transmitting on the multicast address.

4 Performance Results

This section describes the design and results of three perfor-mance experiments we conducted using TAO’s A/V StreamingService.

4.1 CORBA/ATM Testbed

The experiments in this section were conducted using aFORE systems ASX-1000 ATM switch connected to twodual-processor UltraSPARC-2s running Solaris 2.5.1. TheASX-1000 is a 96 Port, OC12 622 Mbs/port switch. EachUltraSPARC-2 contains a 300 MHz Super SPARC CPUs witha 1 Megabyte cache per-CPU. The Solaris 2.5.1 TCP/IP proto-col stack is implemented using the STREAMS communicationframework [Ritchie, 1984].

Each UltraSPARC-2 has 256 Mbytes of RAM and an ENI-155s-MF ATM adaptor card, which supports 155 Megabitsper-sec (Mbps) SONET multimode fiber. The MaximumTransmission Unit (MTU) on the ENI ATM adaptor is 9,180bytes. Each ENI card has 512 Kbytes of on-board memory.A maximum of 32 Kbytes is allotted per ATM virtual circuitconnection for receiving and transmitting frames (for a total of64 Kb). This allows up to eight switched virtual connectionsper card. The CORBA/ATM hardware platform is shown inFigure 23.

FORE SYSTEMSFORE SYSTEMS

ASX ASX 200200BXBX

ATM SWITCHATM SWITCH

(16(16 PORT PORT,, OC3OC3155155MBPSMBPS//PORTPORT,,

9,1809,180 MTU MTU))ULTRAULTRASPARCSPARC 22((FORE ATMFORE ATM

ADAPTORSADAPTORS

AND ETHERNETAND ETHERNET))

Figure 23: Hardware for the CORBA/ATM Testbed

4.2 CPU Usage of the MPEG decoder

The aim of this experiment is to determine the CPU overheadassociated with decoding and playing MPEG-1 frames in soft-

17

ware. To measure this, we used the MPEG/ULAW A/V playerapplication described in Section 0.3.

We used the application to view two movies, one of size128x96 pixels and the other of size 352x240 pixels. We mea-sured the percentage CPU usage for differentframe rates. Theframe rate is the number of video frames displayed by theviewer per second.

The results are shown in Figure 24. These results indicate

0

10

20

30

40

50

60

70

80

90

100

6 9 12 15 18 24 30Frames per second

Per

cent

age

CP

U u

sed

128x96 frame size

352x240 frame size

Figure 24: CPU Usage of the MPEG Decoder

that for large frame sizes (352x240), MPEG decoding in soft-ware becomes expensive, and the CPU usage becomes 100%while playing 12 frames per second, or higher. However, forsmaller frame sizes (128x96), MPEG decoding in softwaredoes not cause heavy CPU utilization. At 30 frames per sec-ond, CPU utilization is�38%.

4.3 A/V Stream Throughput

The aim of this experiment is to illustrate that TAO’s A/VStreaming Service does not introduce appreciable overhead intransporting data. To demonstrate this, we wrote a TCP-baseddata streaming component and integrated it with TAO’s A/Vservice. The producer in this application establishes a streamwith the consumer, using the stream establishment mechanismdiscussed in Section 0.2.3. Once the stream is established, itstreams data via TCP to the consumer.

We measured the throughput,i.e., the number of bytes persecond sent by the supplier to the consumer, obtained by thisstreaming application. We then compared this throughput withthe following two configurations:

� TCP transfer– i.e., by a pair of application processes thatdo not use the CORBA A/V Streaming Service stream es-tablishment protocol. In this case, Sockets and TCP werethe network programming API and data transfer proto-col, respectively. This is the “ideal” case since there is noadditional ORB-related or presentation layer overhead.

� ORB transfer – i.e., the throughput obtained by astream that used anoctet streampassed through theTAO [Schmidt et al., 1998a] CORBA ORB. In this case,the IIOP data path was the data transfer mechanism.

We measured the throughput obtained by varying the buffersize of the sender,i.e., the number of bytes written by the sup-plier in onewrite system call. In each stream, the suppliersent 64 megabytes of data to the consumer.

The results shown in Figure 25 indicate that, as expected,the A/V Streaming Service does not introduce any appre-ciable overhead to streaming the data. In the case of us-

0

20

40

60

80

100

120

140

1 2 4 8 16 32 64 128

Sender buffer size in Kbytes

Thr

ough

put i

n M

egab

its/s

ec

TCP "ideal" caseA/V Stream (via TCP)Octet Stream (via ORB)

Figure 25: Throughput Results

18

ing IIOP as the data transfer layer, the benchmark incurs ad-ditional performance overhead. This overhead arises fromthe dynamic memory allocation, data-copying, and mar-shaling/demarshaling performed by the ORB’s IIOP proto-col engine [Gokhale and Schmidt, 1996]. In general, how-ever, a well-designed ORB can achieve performance equiv-alent to sockets for higher buffer sizes due to various opti-mizations, such as eliding (de)marshaling overhead for octetdata [Gokhale and Schmidt, 1999]

The largest disparity occurred for smaller buffer sizes,where the performance of the ORB was approximately halfthat of the TCP and A/V streaming implementations. Asthe buffer size increases, however, the ORB performance im-proves considerably and attains nearly the same throughput asTCP and A/V streaming. Clearly, there is a fixed amount ofoverhead in the ORB that is amortized and minimized as thesize of the data payload increases.

4.4 Stream Establishment Latency

This experiment measures the time required to establish astream using TAO’s implementation of the CORBA A/Vstream establishment protocol described in Section 0.2.3. Wemeasured the stream establishment latency for the two concur-rency strategies, process-based strategy and reactive strategy,described in Section 0.2.3.

The timer starts when the consumer gets the object refer-ence for the supplier’sMMDevice servant from the NamingService. The timer stops when the stream has been established,i.e., when a TCP connection has been established between theconsumer and the supplier.

We measured the stream establishment time as the num-ber of concurrent consumers establishs connections with thesupplier increased from 1 to 10. The results are shown inFigure 26. When the supplier’sMMDevice is configured touse the process-based concurrency strategy (described in Sec-tion 0.2.3), the time taken to establish the stream is higher,due to the overhead of process creation. For instance, when 10concurrent consumers establish a stream with the producer si-multaneously, the average latency observed is about 2.25 sec-onds with the process-based concurrency strategy. With thereactive concurrency strategy, the latency is only about 0.4 sec-onds.

The process-based strategy is well-suited for supplier de-vices that have multiple streams,e.g., a video camera thatbroadcasts a live feed to many clients. In contrast, the reac-tive concurrency strategy is well-suited for consumer devicesthat have few streams,e.g., a display device that has only oneor two streams.

0

0.5

1

1.5

2

2.5

1 2 3 4 5 6 7 8 9 10

Number of concurrent bind operations

Str

eam

est

ablis

hmen

t tim

e in

sec

onds

Process-based concurrency strategy

Reactive concurrency strategy

Figure 26: Stream Establishment Latency Results

5 Concluding Remarks

The demand for high quality multimedia streaming is grow-ing, both over the Internet and for intranets. Distributed objectcomputing is also maturing at a rapid rate due to middlewaretechnologies like CORBA. The flexibility and adaptability of-fered by CORBA makes it very attractive for use in stream-ing technologies, as long as the requirements of performance-sensitive multimedia applications can be met.This chapter il-lustrates an approach to building standards-based, flexible,adaptive, multimedia streaming applications using CORBA.

Furthermore, there is a lot of activity in the codec commu-nity in designing new formats for audio and video transmis-sion. Active research is also being done in designing new flowand data transfer protocols for multimedia. In such situations,a flexible framework which makes use of the A/V interfacesand also abstracts the network/protocol details is needed toadapt to the new developments. In this chapter we have pre-sented a pluggable A/V protocol framework which providesthe capability to rapidly adapt to new flow and data transferprotocols.

With growing demand for real-time multimedia streamingand conferencing with increase in network bandwidth and the

19

spread of the Internet, TAO provides the first freely-available,open-source implementation of the CORBA Audio/VideoStreaming Service specificationi.e., flow interfaces, point-to-multipoint binding and multipoint-to-multipoint binding forconferencing applications. Our experience with TAO’s A/Vimplementation indicates that the standard CORBA specifica-tion defines a flexible and efficient model for developing flexi-ble and high-performance multimedia streaming applications.

While designing and implementing the CORBA A/VStreaming Service, we learned a number of lessons:

1: We found that CORBA simplifies a number of commonnetwork programming tasks, such as parsing untyped data andperforming byte-order conversions.

2: We found that using CORBA to define the operations sup-ported by a supplier in an IDL interface made it much easierto express the capabilities of the application, as described inSection 0.2.3.

3: Our measurements presented in Section 0.4 revealed thatwhile CORBA provides solutions to many recurring problemsin network programming, using CORBA for data transfer inbandwidth-intensive applications is not as efficient as usinglower-level protocols like TCP, UDP, or ATM directly. Thus,an important benefit of the TAO A/V Streaming Service is toprovide applications the advantages of using CORBA IIOP intheir stream establishment and control modules, while allow-ing the use of more efficient data transfer protocols for multi-media streaming.

4: Enhancing an existing A/V streaming application to useCORBA was a key design challenge. By applying patterns,such as theState, Strategy, [Gamma et al., 1995] andReac-tor [Schmidt et al., 2000], we found it was much easier to ad-dress these design issues. Thus, the use of patterns helped usrework the architecture of an existing MPEG A/V player andmake it more amenable to distributed object computing mid-dleware, such as CORBA.

5: Building the CORBA A/V Streaming Service also helpedus improve TAO, the CORBA ORB used to implement theservice. An important feature added to TAO was support fornested upcalls. This feature allows a CORBA-enabled appli-cation to respond to incoming CORBA operations, while it ismaking a CORBA operation on a remote object. During thedevelopment of the A/V Streaming Service, we also appliedmany optimizations to TAO and its IDL compiler, particularlyfor sequences ofoctet s and theCORBA::Any type.

All the C++ source code, documentation, and bench-marks for TAO and its A/V Streaming Service is available atwww.cs.wustl.edu/ �schmidt/TAO.html .

References[Arulanthu et al., 2000] Arulanthu, A. B., O’Ryan, C., Schmidt, D. C.,

Kircher, M., and Parsons, J. (2000). The Design and Performance of aScalable ORB Architecture for CORBA Asynchronous Messaging. InProceedings of the Middleware 2000 Conference. ACM/IFIP.

[Box, 1997] Box, D. (1997).Essential COM. Addison-Wesley, Reading,Massachusetts.

[Buschmann et al., 1996] Buschmann, F., Meunier, R., Rohnert, H.,Sommerlad, P., and Stal, M. (1996).Pattern-Oriented SoftwareArchitecture – A System of Patterns. Wiley and Sons, New York.

[Chen et al., 1995] Chen, S., Pu, C., Staehli, R., Cowan, C., and Walpole, J.(1995). A Distributed Real-Time MPEG Video Audio Player. InFifthInternational Workshop on Network and Operating System Support ofDigital Audio and Video.

[Clark and Tennenhouse, 1990] Clark, D. D. and Tennenhouse, D. L.(1990). Architectural Considerations for a New Generation of Protocols.In Proceedings of the Symposium on Communications Architectures andProtocols (SIGCOMM), pages 200–208, Philadelphia, PA. ACM.

[Deering and Cheriton, 1990] Deering, S. E. and Cheriton, D. R. (May1990). Multicast routing in datagram internetworks and extended LANs.ACM Transactions on Computer Systems, 8(2):85–110.

[Eide et al., 1997] Eide, E., Frei, K., Ford, B., Lepreau, J., and Lindstrom,G. (1997). Flick: A Flexible, Optimizing IDL Compiler. InProceedingsof ACM SIGPLAN ’97 Conference on Programming Language Designand Implementation (PLDI), Las Vegas, NV. ACM.

[et al., 1996] et al., D. D. (1996). Vaudeville: A High Performance, VoiceActivated Teleconferencing Application. Department of ComputerScience, Technical Report WUCS-96-18, Washington University, St.Louis.

[Fan et al., 1998] Fan, L., Cao, P., Almeida, J., and Broder, A. (1998).Summary Cache: A Scalable Wide-Area Web Cache Sharing Protocol. InSIGCOMM 98, pages 254–265. SIGS.

[Gamma et al., 1995] Gamma, E., Helm, R., Johnson, R., and Vlissides, J.(1995).Design Patterns: Elements of Reusable Object-OrientedSoftware. Addison-Wesley, Reading, Massachusetts.

[Gill et al., 2001] Gill, C. D., Levine, D. L., and Schmidt, D. C. (2001). TheDesign and Performance of a Real-Time CORBA Scheduling Service.Real-Time Systems, The International Journal of Time-CriticalComputing Systems, special issue on Real-Time Middleware, 20(2).

[Gokhale and Schmidt, 1996] Gokhale, A. and Schmidt, D. C. (1996).Measuring the Performance of Communication Middleware onHigh-Speed Networks. InProceedings of SIGCOMM ’96, pages306–317, Stanford, CA. ACM.

[Gokhale and Schmidt, 1998] Gokhale, A. and Schmidt, D. C. (1998).Measuring and Optimizing CORBA Latency and Scalability OverHigh-speed Networks.Transactions on Computing, 47(4).

[Gokhale and Schmidt, 1999] Gokhale, A. and Schmidt, D. C. (1999).Optimizing a CORBA IIOP Protocol Engine for Minimal FootprintMultimedia Systems.Journal on Selected Areas in Communicationsspecial issue on Service Enabling Platforms for Networked MultimediaSystems, 17(9).

[Harrison et al., 1997] Harrison, T. H., Levine, D. L., and Schmidt, D. C.(1997). The Design and Performance of a Real-time CORBA EventService. InProceedings of OOPSLA ’97, pages 184–199, Atlanta, GA.ACM.

[Henning and Vinoski, 1999] Henning, M. and Vinoski, S. (1999).Advanced CORBA Programming With C++. Addison-Wesley, Reading,Massachusetts.

[Hu et al., 1998] Hu, J., Mungee, S., and Schmidt, D. C. (1998). Principlesfor Developing and Measuring High-performance Web Servers overATM. In Proceedings of INFOCOM ’98.

20

[Hu et al., 1997] Hu, J., Pyarali, I., and Schmidt, D. C. (1997). Measuringthe Impact of Event Dispatching and Concurrency Models on Web ServerPerformance Over High-speed Networks. InProceedings of the2nd

Global Internet Conference. IEEE.

[Huard and Lazar, 1998] Huard, J.-F. and Lazar, A. (1998). AProgrammable Transport Architecture with QOS Guarantees.IEEECommunications Magazine, 36(10):54–62.

[Internet Engineering Task Force, 2000a] Internet Engineering Task Force(2000a). Differentiated Services Working Group (diffserv) Charter.www.ietf.org/html.charters/diffserv-charter.html.

[Internet Engineering Task Force, 2000b] Internet Engineering Task Force(2000b). Integrated Services Working Group (intserv) Charter.www.ietf.org/html.charters/intserv-charter.html.

[ISO, 1993] ISO (1993).Coding Of Moving Pictures And Audio For DigitalStorage Media At Up To About 1.5 Mbit/s. International Organisation forStandardisation.

[Kuhns et al., 1999] Kuhns, F., Schmidt, D. C., and Levine, D. L. (1999).The Design and Performance of a Real-time I/O Subsystem. InProceedings of the5th IEEE Real-Time Technology and ApplicationsSymposium, pages 154–163, Vancouver, British Columbia, Canada. IEEE.

[Kuhns et al., 2000] Kuhns, F., Schmidt, D. C., O’Ryan, C., and Levine, D.(2000). Supporting High-performance I/O in QoS-enabled ORBMiddleware.Cluster Computing: the Journal on Networks, Software, andApplications, 3(3).

[McCanne and Jacobson, 1995] McCanne, S. and Jacobson, V. (1995). Vic:A Flexible Framework for Packet Video. InACM Multimedia 95, pages511–522, New York. ACM Press.

[Meyer, 1989] Meyer, B. (1989).Object Oriented Software Construction.Prentice Hall, Englewood Cliffs, NJ.

[Mungee et al., 1999] Mungee, S., Surendran, N., and Schmidt, D. C.(1999). The Design and Performance of a CORBA Audio/VideoStreaming Service. InProceedings of the Hawaiian InternationalConference on System Sciences.

[NRG, LBNL, 1995] NRG, LBNL (1995). LBNL Audio Conferencing Tool(vat). ftp://ftp.ee.lbl.gov/conferencing/vat/.

[Object Management Group, 1999] Object Management Group (1999).TheCommon Object Request Broker: Architecture and Specification. ObjectManagement Group, 2.3 edition.

[Object Management Group, 2000] Object Management Group (2000).TheCommon Object Request Broker: Architecture and Specification. ObjectManagement Group, 2.4 edition.

[Object Management Group, 2001] Object Management Group (2001).TheCommon Object Request Broker: Architecture and Specification. ObjectManagement Group, 2.6 edition.

[OMG, 1996] OMG (1996).Property Service Specification. ObjectManagement Group, 1.0 edition.

[OMG, 1997a] OMG (1997a).Control and Management of A/V Streamsspecification. Object Management Group, OMG Documenttelecom/97-05-07 edition.

[OMG, 1997b] OMG (1997b).CORBAServices: Common Object ServicesSpecification, Revised Edition. Object Management Group, 97-12-02edition.

[O’Ryan et al., 2000] O’Ryan, C., Kuhns, F., Schmidt, D. C., Othman, O.,and Parsons, J. (2000). The Design and Performance of a PluggableProtocols Framework for Real-time Distributed Object ComputingMiddleware. InProceedings of the Middleware 2000 Conference.ACM/IFIP.

[Pyarali et al., 1996] Pyarali, I., Harrison, T. H., and Schmidt, D. C. (1996).Design and Performance of an Object-Oriented Framework forHigh-Performance Electronic Medical Imaging.USENIX ComputingSystems, 9(4).

[Pyarali et al., 1999] Pyarali, I., O’Ryan, C., Schmidt, D. C., Wang, N.,Kachroo, V., and Gokhale, A. (1999). Applying Optimization Patterns tothe Design of Real-time ORBs. InProceedings of the5th Conference onObject-Oriented Technologies and Systems, San Diego, CA. USENIX.

[RealNetworks, 1998] RealNetworks (1998). Realvideo player.www.real.com.

[Ritchie, 1984] Ritchie, D. (1984). A Stream Input–Output System.AT&TBell Labs Technical Journal, 63(8):311–324.

[Schmidt, 1995] Schmidt, D. C. (1995). Reactor: An Object BehavioralPattern for Concurrent Event Demultiplexing and Event HandlerDispatching. In Coplien, J. O. and Schmidt, D. C., editors,PatternLanguages of Program Design, pages 529–545. Addison-Wesley,Reading, Massachusetts.

[Schmidt et al., 1998a] Schmidt, D. C., Levine, D. L., and Mungee, S.(1998a). The Design and Performance of Real-Time Object RequestBrokers.Computer Communications, 21(4):294–324.

[Schmidt et al., 1998b] Schmidt, D. C., Mungee, S., Flores-Gaitan, S., andGokhale, A. (1998b). Alleviating Priority Inversion and Non-determinismin Real-time CORBA ORB Core Architectures. InProceedings of the4th

IEEE Real-Time Technology and Applications Symposium, Denver, CO.IEEE.

[Schmidt et al., 2001] Schmidt, D. C., Mungee, S., Flores-Gaitan, S., andGokhale, A. (2001). Software Architectures for Reducing PriorityInversion and Non-determinism in Real-time Object Request Brokers.Journal of Real-time Systems, special issue on Real-time Computing inthe Age of the Web and the Internet, 21(2).

[Schmidt et al., 2000] Schmidt, D. C., Stal, M., Rohnert, H., andBuschmann, F. (2000).Pattern-Oriented Software Architecture: Patternsfor Concurrent and Networked Objects, Volume 2. Wiley & Sons, NewYork.

[Schmidt and Suda, 1994] Schmidt, D. C. and Suda, T. (1994). AnObject-Oriented Framework for Dynamically Configuring ExtensibleDistributed Communication Systems.IEE/BCS Distributed SystemsEngineering Journal (Special Issue on Configurable Distributed Systems),2:280–293.

[Schulzrinne et al., 1994] Schulzrinne, H., Casner, S., Frederick, R., andJacobson, V. (1994). RTP: A Transport Protocol for Real-TimeApplications. Internet-Draft.

[Stevens, 1993] Stevens, W. R. (1993).TCP/IP Illustrated, Volume 1.Addison-Wesley, Reading, Massachusetts.

[Stevens, 1998] Stevens, W. R. (1998).UNIX Network Programming,Volume 1: Networking APIs: Sockets and XTI, Second Edition.Prentice-Hall, Englewood Cliffs, NJ.

[SUN Microsystems, 1992] SUN Microsystems, I. (1992).Sun Audio FileFormat. Sun Microsystems, Inc.

[Vxtreme, 1998] Vxtreme (1998). Vxtreme player.www.microsoft.com/netshow/vxtreme/.

[Wollrath et al., 1996] Wollrath, A., Riggs, R., and Waldo, J. (1996). ADistributed Object Model for the Java System.USENIX ComputingSystems, 9(4).

A Design Patterns used in the TAOA/V Streaming Service

This section outlines the intents of all the patterns used inTAO’s A/V Streaming Service and its pluggable A/V proto-col framework. The references explore each pattern in greaterdepth.

21

Abstract Factory pattern [Gamma et al., 1995]: This pat-tern provides an interface for creating families of related ordependent objects without specifying their concrete classes.

Acceptor-Connector pattern [Schmidt et al., 2000]: Thispattern decouples the connection and initialization of cooper-ating peer services in a distributed system from the processingperformed by these peer services once they are connected andinitialized.

Adapter [Gamma et al., 1995]: This pattern allows twoclasses to collaborate that were not designed originally to worktogether.

Component Configurator [Schmidt et al., 2000]: Thispattern decouples the implementation of services from thetime when they are configured.

Double Dispatching [Gamma et al., 1995]: In this pattern,when a call is dispatched to a method on a target object froma parent object, the target object in turn makes method callson the parent object to access certain attributes in the parentobject.

Extension Interface [Schmidt et al., 2000]: This patternprevents bloating of interfaces and breaking of client codewhen developers add or modify functionality to existing com-ponents. Multiple extensions can be attached to the same com-ponent, each defining a contract between the component andits clients.

Facade pattern [Gamma et al., 1995] : This pattern pro-vides a unified higher-level interface to a set of interfaces in asubsystem that makes the subsystem easier to use.

Factory Method pattern [Gamma et al., 1995]: This de-fines an interface for creating objects, but lets subclasses de-cide which class to instantiate.

Leader/Follower pattern [Schmidt et al., 2000]: This pat-tern provides a concurrency model where multiple threads ef-ficiently demultiplex events received on I/O handles shared bythe threads and dispatch event handlers that process the events.

Layer pattern [Buschmann et al., 1996]: This patternhelps to structure applications that can be decomposed intogroups of subtasks in which each group of subtasks is at a par-ticular level of abstraction.

Reactor pattern [Schmidt et al., 2000]: This pattern de-multiplexes and dispatches requests that are delivered concur-rently to an application by one or more clients.

State pattern [Gamma et al., 1995]: This pattern allows anobject to alter its behavior when its internal state changes. Theobject will appear to change its class.

Strategy pattern [Gamma et al., 1995]: This pattern de-fines and encapsulates a family of algorithms and makes theminterchangeable.

Template Method [Gamma et al., 1995]: This pattern de-fines the skeleton of an algorithm in an operation, deferringcertain steps to subclasses.

B Overview of the CORBA ReferenceModel

CORBA Object Request Brokers (ORBs) allow clientsto invoke operations on distributed objects without con-cern for object location, programming language, OS plat-form, communication protocols and interconnects, andhardware [Henning and Vinoski, 1999]. Figure 27 il-lustrates the key components in the CORBA referencemodel [Object Management Group, 2001] that collaborate toprovide this degree of portability, interoperability, and trans-parency.1 Each component in the CORBA reference model is

ORB CORE

OBJECT

ADAPTER

GIOP/IIOP

IDLSTUBS

operation()in argsin args

out args + return valueout args + return value

CLIENTCLIENTOBJECTOBJECT((SERVANTSERVANT))

OBJOBJ

REFREF

STANDARD INTERFACESTANDARD INTERFACE STANDARD LANGUAGE MAPPINGSTANDARD LANGUAGE MAPPING

ORB-ORB-SPECIFIC INTERFACESPECIFIC INTERFACE STANDARD PROTOCOLSTANDARD PROTOCOL

IDLIDLSKELETONSKELETON

IDLCOMPILER

IDLCOMPILER

Figure 27: Key components in the CORBA 2.x referencemodel

outlined below:

Client: A client is a role that obtains references to objectsand invokes operations on them to perform application tasks.A client has no knowledge of the implementation of the ob-ject but does know its logical structure according to its inter-face. It also doesn’t know of the object’s location - objectscan be remote or collocated relative to the client. Ideally, aclient can access a remote object just like a local object,i.e.,object !operation(args) . Figure 27 shows how theunderlying ORB components described below transmit remoteoperation requests transparently from client to object.

Object: In CORBA, an object is an instance of an OMGInterface Definition Language (IDL) interface. Each objectis identified by anobject reference, which associates one ormore paths through which a client can access an object on a

1This overview only focuses on the CORBA components relevantto this paper. For a complete synopsis of CORBA’s componentssee [Object Management Group, 2000].

22

server. Anobject ID associates an object with its implemen-tation, called a servant, and is unique within the scope of anObject Adapter. Over its lifetime, an object has one or moreservants associated with it that implement its interface.

Servant: This component implements the operations de-fined by an OMG IDL interface. In object-oriented (OO) lan-guages, such as C++ and Java, servants are implemented us-ing one or more class instances. In non-OO languages, suchas C, servants are typically implemented using functions andstruct s. A client never interacts with servants directly, butalways through objects identified by object references.

ORB Core: When a client invokes an operation on an ob-ject, the ORB Core is responsible for delivering the requestto the object and returning a response, if any, to the client.An ORB Core is implemented as a run-time library linkedinto client and server applications. For objects executing re-motely, a CORBA-compliant ORB Core communicates via aversion of the General Inter-ORB Protocol (GIOP), such asthe Internet Inter-ORB Protocol (IIOP) that runs atop the TCPtransport protocol. In addition, custom Environment-SpecificInter-ORB protocols (ESIOPs) can also be defined.

OMG IDL Stubs and Skeletons: IDL stubs and skele-tons serve as a “glue” between the client and servants, re-spectively, and the ORB. Stubs implement theProxy pat-tern [Gamma et al., 1995] and marshal application parame-ters into a common message-level representation. Conversely,skeletons implement theAdapterpattern [Gamma et al., 1995]and demarshal the message-level representation back intotyped parameters that are meaningful to an application.

IDL Compiler: An IDL compiler transforms OMG IDLdefinitions into stubs and skeletons that are generated automat-ically in an application programming language, such as C++or Java. In addition to providing programming language trans-parency, IDL compilers eliminate common sources of networkprogramming errors and provide opportunities for automatedcompiler optimizations [Eide et al., 1997].

Object Adapter: An Object Adapter is a composite compo-nent that associates servants with objects, creates object refer-ences, demultiplexes incoming requests to servants, and col-laborates with the IDL skeleton to dispatch the appropriateoperation upcall on a servant. Object Adapters enable ORBsto support various types of servants that possess similar re-quirements. This design results in a smaller and simpler ORBthat can support a wide range of object granularities, lifetimes,policies, implementation styles, and other properties. Eventhough different types of Object Adapters may be used by anORB, the only Object Adapter defined in the CORBA specifi-cation is the Portable Object Adapter (POA).

C Supporting Multiple Endpoint Bind-ing Semantics in TAO’s A/V Stream-ing Service

The CORBA A/V Streaming Service can construct differ-ent topologies for establishing streams between stream end-points. For instance, one-to-one, one-to-many, many-to-one,and many-to-many sources and sinks may need to be con-figured in the same stream binding. The need for certainstream endpoint bindings is dictated by the multimedia ap-plications. For example, a video-on-demand application mayrequire a point-to-point binding when sources and sinks arepre-selected. However, a video-conferencing application mayrequire a multipoint-to-multipoint binding to receive from andtransmit to various sources and sinks simultaneously.

This section illustrates the various stream and flow endpointbindings that have been implemented in TAO’s A/V Stream-ing Service and shows how stream endpoints are created andthe connections are established. In TAO’s A/V Streaming Ser-vice, we have implemented the standard point-to-point andpoint-to-multipoint bindings of the stream endpoints. In ad-dition, we have used these configurations as building blocksfor multipoint-to-multipoint bindings.

C.1 Point-to-Point Binding

Below, we describe the sequence of steps during a point-to-point stream establishment, as defined by the CORBA A/Vspecification and implemented in TAO’s A/V Streaming Ser-vice. In our example, we consider the stream establishmentin a video-on-demand (VoD) application that is similar tothe MPEG player application described in Section 0.3.1. Asshown in Figure 28, the VoD server and VoD client devicewith two audio and video flows. The audio flow is carried

WUGS HIGH- SPEEDNETWORK

TAO QOS-ENABLED ORB

RIO SUBSYSTEM

SUPPLIERCONSUMER

TAO QOS-ENABLED ORB

RIO SUBSYSTEM

Figure 28: Video-on-Demand Consumer and Supplier

over TCP and video over UDP. The client must first locate theServerMMDevice reference and then pass itsMMDevice asthe A party and the ServerMMDevice as theB party to theStreamCtrl .

23

Endpoint creation: At this point, the VDev andStreamEndpoint are created for this stream from theMMDevices. The client and server applications can chooseeitherProcess Strategy , where the endpoints are createdin a separate process, or aReactive Strategy , where theendpoints are created in the same process. The pluggable A/Vprotocol framework in TAO’s A/V Streaming Service pro-vides flexibleConcurrency Strategies[Mungee et al., 1999]to create the endpoints, as described in Section 0.2.3.

Creation of flowendpoints: To create a full profile, anMMDevice can act as a container forFDevs . In thiscase, theMMDevice will create a FlowProducer orFlowConsumer from theFDev, depending on the directionof the flow specified in the flow specification parameter. Theflow direction is always with respect to theA side. Thus, thedirection “out” means that the flow originates from theA sideto theB side, whereas “in” means that the flow originates fromtheB side to theA side.

In the above case, the server is streaming the data to theclient. Therefore, the direction of the flow for both au-dio and video will be “in” and theMMDevice will cre-ate a Flowproducer from the audio and videoFDevson the server and aFlowConsumer from the audio andvideo FDevs on the client. TheseFlowProducers andFlowConsumers are then added to theStreamEndpointusing theadd fep call.

The advantage of using the flow interfaces is that theFDevscan be shared across different applications. In our VoD serverexample, the audio and video processes could be running astwo different processes and contain only the flow objects and acontrol process could add theFDevs from these two processesto the stream. Both flows can now be controlled through thesameStreamCtrl interface. This configuration is a muchmore scalable and extensible approach than the implementa-tion of a MPEG player described in Section 0.3.1, where theaudio and video were treated as two separate streams.

VDev configuration: The StreamCtrl then callsset peer on each of theVDevs with the otherVDevs.For light profiles, multimedia application developers areresponsible for implementing theset peer call to check ifall flows are compatible. For full profiles, theVDev interfaceis not used because theFlowEndPoint contain theseconfiguration operations.

Stream setup: During this step the actual connections forthe different flows are established. For light profiles, theflows do not have any interfaces and the flow specificationshould contain the transfer information for each flow. Forexample, the following flow specs are typically passed to thebind devs call from the VoD client:

“audioninnMIME:audio/mpegnnTCP=ace.cs.wustl.edu;10000”

and“videoninnMIME:video/mpegnnUDP=ace.cs.wustl.edu;8080”

In these flow specs, the client is offering to listen for a TCPconnection and the server will connect to the client. Thisconfiguration might be useful if the server is behind a fire-wall. The StreamCtrl calls connect on one of theStreamEndpoints passing the otherStreamEndpoint ,QoS, and the flow spec.

Stream QoS negotiation: The StreamEndpoint willfirst check if the the otherStreamEndpoint has a nego-tiator property defined. If it does,StreamEndpoint callsnegotiate on the negotiator and the client and server cannegotiate the QoS. TAO’s A/V Streaming Service provides adefault implementation that can be overridden by the appli-cations. TheStreamEndpoint then queries the “Avail-ableProtocols” property on the otherStreamEndpoint . Ifthere is no common protocol the Stream setup will fail and theexceptionStreamOpDenied will be thrown.

Light profile connection establishment: The A partyStreamEndpoint will then try to setup the stream for allits flows. For light profiles, the following steps are done foreach flow:

1: TheStreamEndpoint will extract the flow protocoland data transfer protocol information from the flow spec entryfor this flow. If a network address is not specified then a defaultstream endpoint is picked.

2: The StreamEndpoint then does the following ac-tions.

3: It goes through the list of flow protocol factories in theAV Core instance to find if there is any matching flow proto-col. If no flow protocol is specified, it passes the protocol asthe flow protocol string. TAO’s A/V Streaming Service pro-vides “no-op” implementations for all data transfer protocolsso that the layering of the architecture is preserved and a uni-form API is presented to the application. These no-op flowprotocols do not process the frames – they simply pass themto the underlying data transfer protocol.

4: If a flow protocol factory matches the specifiedflow protocol/transfer protocol, theStreamEndpoint thenchecks for the data transfer protocol factory that matches theprotocol specified for this flow.

5: After finding a matching data transfer protocol factory,it creates a one-shot acceptor for this flow passing theFlow-ProtocolFactoryto the acceptor.

6: If the flow protocol factory has an associated controlprotocol factory, theStreamEndpoint then tries to matchthe data transfer factory for that, as well.

24

1 . open(flowspec_entry)

2. create_acceptor()3. open(flowspecentry,flow_factory)

ACCEPTOR REGISTRY

TRANSPORT FACTORY ACCEPTOR

4. on accept

make_protocol_object(entry,endpoint,handler,transport)

FLOW FACTORY

5. get_callback 6.callback

TAO_Base_Endpoint

Figure 29: Acceptor Registry

Figure 29 illustrates the sequence of steps outlined above.In each step, theStreamEndpoint uses base interfaces,such asProtocol Factory , Transport Factory , andAV Acceptor . Therefore, it can be extended easily to sup-port new flow and data transfer protocols. In addition, the ad-dress information is opaque to theStreamEndpoint and ispassed down to theAcceptor that knows how to interpretit. Moreover, since the flow and data transfer protocols can belinked dynamically via the ACE Service Configurator mecha-nisms, applications can take advantage of these protocols bysimply changing the name of the protocols in the flow spec.

After completing the preceding steps, theStreamEndpoint then calls therequest connectionoperation on theB StreamEndpoint with the flowspec.The StreamEndpoint B performs the following steps foreach of the flow:

1: It extracts the flow and data transfer protocol informa-tion from the flow spec entry for this flow. If a network addressis not specified then a default stream endpoint is picked.

2: TheStreamEndpoint then performs the followingactions.

3: Finds a flow protocol factory matching the flow proto-col specified for this flow and in the absence of a flow protocoltries to match a null flow protocol for the specified data trans-fer protocol.

4: Finds a matching data transfer protocol factory and cre-ates a connector for it. Then it callsconnect on the connec-tor, passing it the flow protocol factory.

5: Upon establishing a data transfer connection, the con-nector creates a protocol object for this flow.

6: The flow protocol factory typically creates theapplication-level callback object and sets the protocol objecton theBase EndPoint interface passed to it.

7: If an address was not specified for this flow thenthe StreamEndpoint does the similar steps for listen-ing for those flows and extracts the network endpointsand inserts it into the flowspec to be sent back to theAStreamEndpoint .

The A StreamEndpoint after receiving the reverseflowspec does theconnect for all the flows for whichBStreamEndpoint is listening and also sets the peer addressfor connectionless protocols, such as UDP.

Full profile connection establishment: In the full profile,the flow specification does not contain the data transfer infor-mation for each flow since the flows are represented by flowinterfaces and they need not be collocated in the same process.A StreamCtrl can be used to control different flows, eachof which could reside on a different machine. In this case,eachFlowEndPoint will need to know the network addressinformation. In the full profile stream setup,bind is called ontheStreamCtrl , passing the twoStreamEndpoints .

Figure 30 illustrates the sequence of steps performed for afull profile point-to-point stream setup.

1. g

et_f

ep

FlowEnd

Point

3 . is

_fe

p_com

patib

le

STREAMCTRL

A_STREAMENDPOINT

B_STREAMENDPOINT

FLOW CONNECTION

FLOWENDPOINT A

FLOWENDPOINT B

4.2 open(flow_spec_entry)

ACCEPTOR REGISTRY CONNECTOR REGISTRY

4.4 connect(flow_spec_entry)

3.1 get_property(protocols)

3.2 get_property(format)

1 .

.

. n

1 . . . n

4.1

go_t

o_lis

ten 4.3 connect_to_peer

2. g

et_

fep

Flo

wE

ndP

oin

t

4 . c

onnect(A

,B)

bind(A_StreamEndpoint,B_StreamEndPoint)

Figure 30: Full Profile Point to Point Stream Establishment

1: Flow endpoint matching: TheStreamCtrl obtainsthe flow names in eachStreamEndpoint by queryingthe “flows” property. For each flow name, it then obtainsthe FlowEndPoint using the get fep method on theStreamEndpoint . If the flowspec is empty all the flowsare considered. Otherwise, only the specified flows are con-sidered for stream establishment.

25

It then goes through the list ofFlowEndPoints try-ing to find a match between theFlowEndPoints on theA and B side. Two FlowEndPoints are said to matchif is fep compatible returns true. This call checksto make sure that the format and the protocols of the twoFlowEndPoints match. Applications can override thisbehavior to do more complex checks, such as checking forsemantic nuances of device parameters. For example, theFlowEndPoint may want only a French audio stream,whereas the otherFlowEndPoint may support only En-glish. These requested semantics can be checked by query-ing the property “devParams” and by checking the value for“language.”

The StreamEndpoint then tries to obtain aFlowConnection from the StreamCtrl . The ap-plication developer can set theFlowConnection objectfor each flow using theStreamCtrl . All operations on astream are applied to the containedFlowConnectionsand by setting specializedFlowConnections the usercan customize the behavior of the stream operations. Ifthe stream does not have aFlowConnection then adefault FlowConnection is created and set for thatflow. The StreamEndpoint then calls connect onthe FlowConnection with the producer and consumerendpoints with the flow QoS.

2: Flow configuration: TheFlowConnection callsset peer on each of theFlowEndPoints during theconnect operation and this will let theFlowEndPoints tocheck and set the peerFlowEndpoint’s configuration. Forexample, a video consumer can check theColourModel ,ColourDepth , and VideoResolution and allocate awindow for the specified resolution and also other display re-sources,i.e., colormap, etc. In the case of audio, the quantiza-tion property can be used by the consumer to allocate appro-priate decoder resources.

3: Flow connection establishment: In this step, theFlowConnection calls go to listen on one of theFlowEndPoints with the is mcast parameter set tofalse and also passes the flow protocol that was set onthe FlowConnection using theuse flow protocoloperation. TheFlowEndPoint can raise an exceptionfailedToListen in which case theFlowConnectioncallsgo to listen on the otherFlowEndPoint .

In TAO’s implementation thego to listen does thesequence of operations shown in Figure 29 to accept onthe selected flow protocol and data transfer protocol andalso if needed the control protocol for the flow. Since theFlowEndPoint also derives fromBase EndPoint theCallback andProtocol Objects will be set on the end-point. In the case of theFlowProducer the gettimeoutwill be called on theCallback object to register for timeout

events.The FlowConnection then callsconnect to peer

on the otherFlowEndPoint with the address returned bythe listeningFlowEndPoint and also the flowname. In thecase of connectionless protocols, such as UDP, the listeningFlowEndPoint may need to know the reverse channel tosend the data in which case it can call theget rev channeloperation to get it.

When FlowEndPoint calls connect to peer , se-quence of steps shown in Figure 31 will occur to connect tothe listening endpoint. With the above sequence of steps a

1 . connect

2 . create_connector() /get_connector()

3 . open(flowspec,flow_factory)

CONNECTOR REGISTRY

TRANSPORT FACTORY CONNECTOR

4 . make_protocol_object

FLOW FACTORY

5 . get_callback

Endpoint

Figure 31: Connector Registry

stream will be established in a point-to-point binding betweentwo multimedia devices.

C.2 Point-to-Multipoint Binding

TAO’s point-to-multipoint binding support is essential to han-dle broadcast/multicast streaming servers. With new tech-nologies, such as Web Caching [Fan et al., 1998], multicastupdates of web pages and streaming media files is becom-ing common place. In addition, it has become common onwebsites to broadcast live events using technologies like Re-alPlayer. For example, during the World Cup Cricket 99,millions of people listened to the live commentaries of thematches from the BBC website.

In such cases, it would be ideal for the servers to use mul-ticast technologies like IP multicast to reduce server connec-tions and load. TAO’s point-to-multipoint binding essentiallyprovides such an interface for a source to multicast the flowsto multiple sinks as shown in figure 32. TAO’s A/V StreamingService implementation provides a binding based on IP multi-cast [Deering and Cheriton, 1990]. In this section we explainthe sequence of steps that lead to a point-to-multipoint streamestablishment both in the light and full profiles.

26

LIVESTREAM ENDPOINT

VIDEO FLOWPRODUCER

ENGLISHFLOWPRODUCER

SPANISHFLOWPRODUCER

STREAMCTRL

VIDEO FLOW ENGLISH FLOW SPANISH FLOW

VIDEO

FLOW

CONSUMER

ENG

FLOW

CONSUMER

En

glish

Vid

eo

Sp

an

ish

ENG

FLOW

CONSUMER

VIDEO

FLOW

CONSUMER

ENG

FLOW

CONSUMER

TV DEVICETV DEVICE

Vid

eoEng

lish

En

glish

Spanish

Video

TV ENDPOINTTV ENDPOINTRADIO ENDPOINT

Figure 32: Point-to-Multipoint Binding

Adding a multipoint source: A multipoint sourceMMDevice must be added before any sinks can be added tothe Stream. For example, the multicast server could add itselfto theStreamCtrl and expose theStreamCtrl interfacethrough a standard CORBA object location service, such asNaming or Trading. If theB party MMDevice parameter tobind devs is nil, the source is assumed to be a multicastsource. As with a point-to-point stream, the endpoints forthe source device are created,i.e., the StreamEndpointand VDev for light profiles, and theStreamEndpointcontaining FlowProducers for the full profile. Un-like the point-to-point stream, however, there can only beFlowProducers in the MMDevice. Figure 33 shows thecreation of endpoints in point-to-multipoint bindings.

Multicast configuration interface: In the case of a mul-tipoint binding there can be numerous sinks. Therefore,the CORBA A/V Streaming Service specification providesan MCastConfigIf interface, which is used instead ofusing point-to-pointVDev configurations. Upon additionof a multipoint source, theStreamCtrl creates a newMCastConfigIf interface and sets it as the multicast peerof the sourceVDev. This design allows the stream binding touse multicasting technologies to distribute the stream config-uration information instead of using numerous point-to-pointconfigurations.

The MCastConfigIf interface provides operations toset the initial configuration of the stream example viathe set initial configuration operation. This op-eration can be called by the sourceVDev during the

MPOINT SOURCE

FDEVA - ENDPOINT

B_VDEV

FDEV

MPOINT SINK

B - ENDPOINT

2.1

Cre

ate

- A

A_E

ndpo

int,A

_VDev 3.1 C

reate - B

2.2

Cre

ate

Pro

duce

r2.3

Flo

w P

roduce

r

2.4

Add_F

ep

3.2

Cre

ate

Consum

er 3

.3 F

low

Consum

er 3

.4 A

dd_F

ep

Light Profile

Full ProfileSTREAM CTRL

A_VDEV

MCAST CONFIG IF

Set M

cast P

eer A

dd P

eer

A . bind_devs ( Mpoint_source,Nil)

B . bind_devs ( Nil,Mpoint_sink)

B_E

ndpoint,B_V

Dev

1 . . . n

1 . . . n

Figure 33: Creation of Endpoints in the Point-to-MultipointBinding

set MCast peer call. This information is conveyed tothe multicast sinkVDev during theset peer call on theMCastConfigIf when a multicast sink is added. TheMCastConfigIf performs the configuration operation us-ing point-to-point invocation on all sinkVDevs.

Adding multicast sinks: When a sink wants to join a streamas a multicast sink, it can callbind devs with a nil Aparty MMDevice. This call will create the endpoints forthe multicast sink,i.e. the StreamEndpoint and theVDev. For full profiles, theStreamEndpoint will containFlowConsumers . For light profiles, theVDev is added totheMCastConfigIf .

Multicast connection establishment: The StreamCtrlthen callsconnect leaf on the multicast source endpointfor the multicast sink endpoint. In TAO, theconnect leafoperation will throw thenotSupported exception. TheStreamCtrl will then try the IP multicast model usingthemulticonnect call on the sourceStreamEndpoint .The following steps occur whenmulticonnect is called ona StreamEndpoint A for each flow in the full profile:

1: TheStreamEndpoint makes sure that the endpointis indeed aFlowProducer .

2: It then checks to see if aFlowConnection interfaceexists for this flow in theStreamCtrl , which is obtainedthrough theRelated StreamCtrl property.

27

3: In the absence of aFlowConnection , theStreamEndpoint A will create aFlowConnection andset the multicast address to be used for this flow on theFlowConnection . An application configure this ad-dress by passing it to theStreamEndpoint during itsinitialization. The A/V specification does not define howmulticast addresses are allocated to flows. Thus, TAO’sStreamEndpoint uses a base multicast address and assignsdifferent ports for the flows and sets theFlowConnectionon theStreamCtrl . We ultimately plan to strategize this al-location so applications can decide on the multicast addressesto use for each flow.

4: The StreamEndpoint then callsadd produceronFlowConnection .

5: The call to add producer will result in aconnect mcast on theFlowProducer , passing the mul-ticast address with which to connect. TheFlowProducerthen returns the address to which it will multicast the flow. Ifthe return address is complete with network address, then IPMulticast is used. In contrast, if the return address specifiesonly the protocol name an ATM-style multicast is used.

6: In addition, the FlowConnection creates aMCastConfigIf if it has not been created and sets it asthe multicast peer on theFlowProducer . Since the sameMCastConfigIf is used for bothFlowEndPoint andVDev, the parameters toMCastConfigIf are passed asCORBA objects. It is the responsibility ofMCastConfigIfto check whether the peer is aVDev or aFlowEndpoint .

7: The connect mcast does the actual connection tothe multicast address and results in the sequence of steps formulticast accept using the pluggable A/V protocols.

Figure 34 illustrates these steps graphically. The steps de-scribed above occur for each multipoint sink that is added tothe stream. TAO’s pluggable A/V protocol framework is con-figured with both full profile and light profile objects. It isalso configured in the point-to-point and point-to-multipointbindings. Thus, the control and management implementationobjects can be closed for modifications, yet new flow and datatransfer protocols can be added flexibly to the framework with-out modification to these interface implementations. A similarset of steps happens whenmulticonnect is called on theStreamEndpoint B.

C.3 Multipoint-to-Multipoint Binding

The multipoint-to-multipoint binding is important for appli-cations, such as video-conferencing, where there are multiplesource and sink participants. The CORBA A/V Streaming Ser-vice specification does not mandate any particular protocol formultipoint-to-multipoint binding, leaving it to implementors

STREAMCTRL

A_ENDPOINT B_ENDPOINT

FLOWCONNECTION

FLOW PRODUCER

ACCEPTORREGISTRY

MCASTCONFIGIF

FLOW CONSUMER

CONNECTORREGISTRY

1. M

ulti

connect

(flo

ws)

1.1 add_producer

1.2

Connect

Mca

st

1.3

Addre

ss

open(flow_spec_entry)

1.4

Set_

Mca

st_peer

1.1

Open(f

low

_spec_entr

y)

1.2

Flo

w_spec_entr

y

1.5,

1.3

Flo

w_s

pec

2.1 Multiconnect

2.2

add_

cons

umer

2.5 set_peer

2.2

open(flo

w_spec_entry

)

2.4

connect (f

low

_spec_entr

y)

2.3

conn

ect_

to_p

eer

Figure 34: Connection Establishment in the Point-to-Multipoint Binding

to provide this feature. In TAO, we provide a multipoint-to-multipoint binding by extending the point-to-multipoint bind-ing based on IP multicast. We assume a Leader/Followerpattern [Schmidt et al., 2000] for the sources, where the firstsource that is added to the stream will become theleaderforthe multipoint-to-multipoint binding, and every other sourcebecome afollower. This design implies that all stream proper-ties, such as format and codec, will be selected by the leader.

28