Customer-managed end-to-end lightpath provisioning

Copyright © 2005 Crown in the right of Canada. Published by John Wiley & Sons, Ltd.

Customer-managed end-to-end lightpathprovisioning

By Jing Wu*,†, Michel Savoie, Scott Campbell, Hanxi Zhang, Gregor v. Bochmann and Bill St. Arnaud

Customer-owned and managed optical networks bring new cost-savingbenefits. Two types of such networks are becoming widely used: metrodark fiber networks and long-haul leased wavelength networks.Customers may invoke a special QoS mechanism where end-to-end (E2E)lightpaths are dynamically established across multiple independentlymanaged customer domains. The cost of bandwidth is substantiallyreduced since it largely becomes a capital cost rather than an ongoingservice charge. Customers can optimize the overall resource consumptionby utilizing resources from different suppliers. Remote peering and transitreduce the Internet connectivity cost. Bandwidth and quality of serviceare guaranteed because customers directly peer with each other usingtransport networks. An architecture for a customer-managed E2Elightpath provisioning system is presented. Integration with Gridapplications is discussed and a prototype demonstration is described.Copyright © 2005 Crown in the right of Canada. Published by John Wiley& Sons, Ltd.

Jing Wu is a Research Scientist at the Communications Research Centre Canada. His research interests include control and management ofoptical networks, networking protocols and algorithms, network performance evaluation and optimization. He received his PhD degree inSystems Engineering from Xian Jiao Tong University, China.

Michel Savoie is the Research Program Manager for the Broadband Applications and Optical Networks group at the Communications ResearchCentre Canada. He is the Project Manager for ‘User-Controlled End-to-End Lightpath Provisioning over CA*net 4’. He holds a BSc and MScin Electrical Engineering from the University of New Brunswick, Canada.

Scott Campbell is a Network Researcher at the Communications Research Centre Canada. He designs and develops agent-based network man-agement and control software for all optical networks. He has a bachelor degree in Computer Science from Dalhousie University, Canada.

Hanxi Zhang is a Research Engineer at the Communications Research Centre Canada. His research interests include distributed computing,web services, and protection/restoration for optical WDM networks. He obtained his MASc degree from the University of Ottawa, Canada.

Gregor v. Bochmann is a Professor at the University of Ottawa. Previously, he was a Professor at the University of Montreal for 25 years. Heis a fellow of the IEEE and ACM, and a member of the Royal Society of Canada. He has worked in the areas of software engineering, distrib-uted systems and optical networks.

Bill St. Arnaud is the Senior Director of Advanced Networks for CANARIE Inc., Canada’s advanced Internet development organization. Hehas been responsible for the coordination and implementation of Canada’s next-generation optical Internet initiative called CA*net 4. He is agraduate of Carleton University, Canada.

*Correspondence to: Jing Wu, Communications Research Centre Canada, 3701 Carling Avenue, PO Box 11490, stn H, Ottawa, Ontario, CanadaK2H 8S2†E-mail: [email protected]

INTERNATIONAL JOURNAL OF NETWORK MANAGEMENTInt. J. Network Mgmt 2005; 15: 349–362Published online in Wiley InterScience (www.interscience.wiley.com). DOI: 10.1002/nem.581

Introduction

There are basically two types of customer-owned and managed optical networks:metro dark fiber networks and long-haul

wavelength networks. Schools, hospitals and gov-ernment departments are acquiring their owndark fibers in metropolitan areas. They participatein so-called ‘condominium’ dark fiber networks tobetter manage their connectivity and bandwidth.They light up the fibers with their own equipmentand interconnect their fibers to either like-mindedinstitutions, commercial service providers or Inter-net Exchanges as they so choose. In the long-haularea, many providers are selling or leasing point-to-point wavelength channels. Some providers areoffering ‘condominium’ wavelength solutions,where a number of customers share the capitalcosts of deploying long-haul optical networks. Inreturn, each customer in the condominium con-sortium owns a set of wavelength channels. Thesepurchased or leased wavelength channels can gen-erally be treated as an asset rather than a telecom-munication service. The institutions virtuallyextend their dark fiber networks many thousandsof kilometers without having to purchase andmaintain their own optical repeaters and associ-ated equipment.

The advent of object-oriented and Web Serviceprotocols such as JXTA, J2EE, CORBA, SOAP, XMLand Jini indicates that optical network design maybe on the cusp of a new approach in network man-agement and in the control and management ofwavelength channels. Rather than making signal-ing requests to a carrier to establish wavelengthcircuits, customers in the future may purchasewavelength channels and cross-connects fromtrading exchanges or from each other on a peer-to-peer basis, constructing their own optical net-works and managing them directly through theseobject-oriented protocols.

In this paper we first discuss the benefits andtechnical challenges presented by customer-owned and managed optical networks. In the thirdsection we present an architectural framework,communication protocols and software engineer-ing technologies that are useful for building man-agement tools for this context. In the fourth sectionwe describe the system and software architectureof a management system developed for customer-managed lightpath provisioning. The integration

with Grid applications is discussed. A prototypedemonstration made at the Eighth Global GridForum is described in the fifth section to illustratethe deployment and service interactions. In thesixth section existing technologies for the controland management of multi-domain or inter-domain optical networks are summarized. Theseventh section concludes the paper.

Benefits and Challenges of Customer-Owned

Optical Networks—Benefits—

In customer-owned optical networks, the cost ofbandwidth is substantially reduced, as it nowlargely becomes a capital cost rather than anongoing monthly service charge. For example, a10-year leased fiber across Chicago costs what acarrier charges for a gigabit Ethernet service permonth covering the same distance. As relativelyinexpensive gigabit Ethernet and coarse wave-length division multiplexing technology catch on,exponential penetration in the market will likelyoccur. To expand the capacity of an existing darkfiber network, customers can add new wavelengthchannels. Since some all-optical networks aretransparent to bit rate and data format, anotheroption is to increase the bit rate of existing light-paths. The latter option also applies to wavelengthnetworks. Compared to renegotiating a new con-tract with service providers, customer-ownedoptical networks have significant cost savingswhen the demand for bandwidth increases dra-matically over time.

Compared to renegotiating a new contractwith service providers, customer-owned

optical networks have significant costsavings when the demand for bandwidthincreases dramatically over time.

The customer is able to optimize the overallresource consumption. The customer purchasesdark fibers and/or wavelength channels from anumber of independent suppliers, as well as par-

350 J. WU ET AL.

Copyright © 2005 Crown in the right of Canada. Published by John Wiley & Sons, Ltd. Int. J. Network Mgmt 2005; 15: 349–362

ticipating in condominium wavelength networksbuilt for some portions of their network. There-fore, the customer has more flexibility in networkplanning and deployment and is able to negotiatethe best deal from different suppliers. The cus-tomer may fine-tune the usage of each resourcefrom each independent supplier.

Customer-managed networks reduce Internetcosts via remote peering and transit. Customersmay directly peer with each other and, moreimportantly, set up bandwidth guaranteed con-nections to no-cost peering exchanges. The cus-tomers manage their peering relationshipswithout having to contact a central managementbody or pay expensive Internet transit fees. Thereis a large incentive for many smaller ISPs andlarger enterprise networks of roughly equal size todirectly interconnect to each other on a settlement-free basis. Many commercial Internet serviceproviders interconnect to each other at InternetExchange points without charging each othermoney for the exchange of traffic. As long as trafficlevels remain reasonably balanced, this is a verypractical cost saving technique on Internet transitfees. However, when a smaller ISP or an institu-tion connects to a larger ISP, there is generally animbalance of traffic and the larger ISP will chargeboth peering and transit fees. Purchasing directconnections to other ISPs who will undertake set-tlement-free peering can result in significant costsavings over what would normally be paid intransit fees to a large upstream ISP.

Since the customer directly owns andmanages an optical network, the

bandwidth and quality of service areguaranteed.

Since the customer directly owns and managesan optical network, the bandwidth and quality ofservice are guaranteed. The complexity of servicemanagement at the IP layer is removed. A newopportunity for potential cost savings is intro-duced by eliminating expensive high-end routersin the core and replacing them with opticalswitches. However, there is a cost in terms ofnetwork efficiency as the functionality of IP packetmultiplexing is lost. The trade-off needs to be

explored regarding the bandwidth efficiency andthe cost of wavelength channels versus the cost ofrouters.

It may be possible to convey the advantages ofconnectionless networks like the Internet to con-nection-oriented networks. Customer-empowerednetworks echo many of the advantages of connec-tionless networks. Now, circuits can be created andset up on demand by the customer without sig-naling a carrier or central management authority.Connection-oriented services can be deployed ona peer-to-peer basis. Connectionless packets can betransferred between computers with guaranteedthroughput. Future networks may very well behybrid facilities where attached devices will beable to transmit packets through a default connec-tionless facility and at the same time be able to setup end-to-end circuits on a peer-to-peer basis forhigh-bandwidth applications. Circuits and packetswill be treated equally and managed in a seamlessand transparent manner by the application ornetwork-connected device.

Customer-managed end-to-end lightpath provi-sioning is a traffic engineering mechanism forinter-domain applications. The primary applica-tion for this technology is Lambda Grids tosupport high-end scientific research in high-energy physics, astronomy, bio-informatics, etc.Customer-managed lightpath provisioning allowsthese applications to invoke a special QoS mecha-nism where a separate optical Border GatewayProtocol (BGP) point-to-point connection is set up,operating in parallel with the normal hop-by-hopBGP route. Grid applications’ data traffic is thenautomatically rerouted over the point-to-pointoptical BGP path rather than the normally con-gested routed path (Figure 1).

—Technical Challenges—

The first technical challenge is the managementof networks with resources from different sources.Only the customer has total visibility of its ownnetwork and no provider can see all the networkelements. The traditional centrally managed hier-archical networking technologies, e.g. GeneralizedMulti-Protocol Label Switching (GMPLS)1 andAutomatic Switched Optical/Transport Network(ASON/ASTN), assume that the provider has totalvisibility of all network elements and a common

CUSTOMER-MANAGED END-TO-END 351


management system is used for all optical equip-ment. The customer-managed networks resemblesome features of virtual private networks (VPNs).2

A VPN is provisioned over public or third-partynetwork infrastructures to provide dedicated con-nectivity to a closed group of customers. However,the VPN technology allows provisioning of customer networks within a single provider’sdomain. Clearly this type of architecture is notpractical with customer-managed networks,where resources are provided by multiple suppli-ers. For protection and restoration, the customer,rather than any provider, is in a better position todecide the optimal solution. How to coordinateprotection and restoration involving multipleproviders is an open issue.

The second challenge is the collaboration amongmultiple independent customers without coordi-nation through centralized management. Cus-tomer-managed networks adopt the peer-to-peerarchitecture, in which customers peer with eachother. Each customer domain not only receivestransport services from other customer domainsbut also contributes new transport services.During the establishment of an end-to-end con-nection, each segment of the connection betweendomains is set up on a peer-to-peer basis. Centralguiding intelligence and arbitration of conflictsmay be necessary, but day-to-day managementand per connection control should be decentral-ized. An end-to-end connection from one customerto another involves at least two different customer

domains, and if transit is required one or moreintermediate transit domains may participate.Therefore the collaboration among multiple inde-pendent customers is critical for end-to-end con-nection provisioning. How to search and takecontrol of resources in collaborative domains hasto be addressed. Policy enforcement, authorizationand authentication have to be applied. The orga-nization of customer federations is a new issue.

The third challenge is the dynamic partitioningof a provider’s resource to customers. VPN tech-nology allows partitioning of a provider’s resourceto customers. However, VPNs are not as dynamicas some emerging applications require, e.g. Gridcomputing. Some customers prefer significantcontrol and management capabilities in theprovider’s domain. They want a fine-grainedresource allocation, which enables further opti-mization of the overall resource consumption.Deploying and upgrading customers’ services isdifficult and time-consuming in current networksowing to the closed, integrated architecture ofnetwork nodes. How to manage a provider’snetwork element in a condominium fashion ischallenging.

How to manage a provider’s networkelement in a condominium fashion is

challenging.

352 J. WU ET AL.


Normal BGP pathx.x.x.1 y.y.y.1

OBGP pathOnly y.y.y.1advertised tox.x.x.1 viaOBGP path

Only x.x.x.1advertised toy.y.y.1 viaOBGP path

Optical “ Peermaker ”

Figure 1. An application or end user controls peering of BGP optical paths for transfer of ‘elephants’(i.e., vast amounts of data)

Architectural Principles forCustomer-Managed Networks

—Network Architecture andManagement Domains—

We assume a general mesh type of networkarchitecture where the nodes consist of opticalcross-connects and the edges consist of opticalfibers over which multiplexing is providedthrough wavelength division multiplexing. Theoptical lightpaths are mostly used as links betweenpacket switches and IP routers that are connectedat the endpoints of these lightpaths.

In analogy to the architecture of the Internet, weassume that the optical network is partitioned intoa number of administrative domains. In customer-managed optical networks, each domain repre-sents an individual customer or a group ofcustomers. Within each domain, we assume thatcomplete management information would beavailable, which could be consulted when a newlightpath is to be established. However, betweendifferent domains, only partial information wouldbe exchanged.

Customer domains may choose to interconnectto each other using condominium dark fibers or

leased wavelength channels, or peer with eachother at condominium peering points. Figure 2shows these two scenarios. The peering points areusually non-blocking optical cross-connects.Rather than having one central organizationmanage these peering points, their management ispartitioned amongst different customers.

Each cross-connect on the peering points can bemanaged independently by the individual cus-tomer. Interconnections between independent cus-tomers then must be done on a bilateral peeringbasis. The example in Figure 3 illustrates the prin-ciples of peering points. The peering point is par-titioned into four separate management domainsrepresenting four customers. Associated with eachmanagement domain are the essential functions:grooming, switching and control plane services.Instead of having a single management interfacefor all these functions as in a traditional opticalnetwork, agents or objects are associated with eachrespective function for every customer on thepeering point.

—Advertising Network Resources—

To establish an end-to-end lightpath across mul-tiple customer domains, it is sometimes necessary



CustomerDomain A Customer

Domain B

CustomerDomain C

Condominium Dark Fiber

Internal Links

CondominiumPeering Point

Figure 2. Customer domains may interconnect to each other using condominium dark fibers or leasedwavelength channels, or peer with each other at condominium peering points

to concatenate lightspans that belong to differentcustomers. A lightspan is a single physical link ora logical link composed of several concatenatedphysical links. The following two cases may beconsidered:

• Peering. Two lightspans from node A to B andfrom B to C are to be interconnected at thepeering point B to create a new lightspan fromA to C; the lightspans from A to B and from Bto C may belong to different customers.

• Leasing. To establish a lightspan from A to C,a given customer P1 may own a free lightspanfrom A to B, and needs another free lightspanfrom B to C to be interconnected with theformer; another customer P2 may own such alightspan and be willing to sublease it to P1.

To facilitate the search for free lightspans thatbelong to other customers, free resources should

be publicly advertised. The concept of servicedirectories has been proposed in distributed com-puting and realized as the Common ObjectRequest Broker Architecture (CORBA), JiniLookup Services and JavaSpaces,3,4 or Web Ser-vices directories.5 Each resource or service that isto be made available to other customers must beregistered in the service directory. Potential customers of these resources or services mayquery the directory to find a resource that meets their requirements. The object-oriented paradigm is used to support meaningful queries.Each registered object instance belongs to a class that defines the properties of the objectinstances. In addition, the class defines a certainnumber of attributes, and each instance is charac-terized by the values of its attributes. A customer’ssearch for a service will therefore indicate the classof service desired and possibly some attribute

354 J. WU ET AL.


X

X

X

X

X

X

X

X

X

X

X

X

Customer A Customer A

Customer B Customer D

Customer C Customer C

Grooming Objects Customer A's Partition

Customer C's PartitionCustomer B's Partition

Customer D's PartitionGrooming Objects

Switch Control Objects

OSPF

OBGP

OBGP

OBGP

OSPF

OSPF

OBGP

OBGP

OBGP

OSPF

Optional Signalling and Control Plane Objects

Object Associations

Figure 3. A peering point with customer control of ports and cross-connects

values. For instance, customer P1, in the exampleabove, will search the directory for an object ofclass lightspan with the attributes source = B anddestination = C.

It is also conceivable that a given customer, e.g.P1, has leased a lightspan from customer P2 tobuild a longer lightspan, e.g. from A to C. Now P1may subdivide the bandwidth of the lightspan andcreate multiple low-bandwidth lightspans from Ato C. P1 may use a few of these lightspans and mayreadvertise the others as available in one of theservice directories. The advertised leases could beassociated with prices. In this way, the leasingmechanism may be used as a basis for establishinga broker market for optical networking.

—Distributed ResourceManagement—

Even if we have access to all the informationabout available lightspans and the possibility ofleases, the actual resource reservations necessaryfor the establishment of a new lightpath will gen-erally involve several customers. Therefore,several databases will be maintained for the stateinformation about the different resources. To avoidinconsistencies due to concurrent access to theseresources by several customers, it is important toforesee appropriate mechanisms for mutual exclu-sion of access. Since persistent storage is requiredin the presence of occasional crashes of the com-puters that maintain the databases, the transactionconcept developed for centralized and distributeddatabases appears to be useful here. A transactionis a sequence of actions, such as reading the statusof resources and requesting resource reservations,that are all executed as specified (i.e., the transac-tion commits) or not executed at all (i.e., the trans-action aborts). This applies even in the case whereone of the computers managing certain resourcescrashes during the operations (in this case thetransaction aborts).

It is interesting to note that JavaSpaces8 repre-sents a service that provides persistent storage ofobject instances, retrieval of object instances andtransaction management involving actions takingplace in different JavaSpaces possibly residing ondifferent computers. This technology is thereforean interesting platform for implementing distrib-uted resource management tools.

—Using Jini and JavaSpaces in theImplementation of a Management

Tool—

Jini and JavaSpaces provide a number of advan-tages for developing distributed applications. Jiniruns on top of Java and uses the Remote MethodInvocation (RMI) to access remote services. Jinialso provides a set of application programminginterfaces that hide the underlying complexity ofdistributed computing from the customer.9 TheJini Lookup Service (JLS), a distributed service reg-istry, allows customers to find services withouthaving to know where they are located. The wholenetwork provides a lookup service. Multiplelookup services may be introduced for redun-dancy and fault tolerance. In the start-up process,customers obtain the lookup service either by pre-configuration or through a discovery protocol thatuses multicast throughout the network. Throughthe federation of JLSs, a customer can find anyservice in any domain. Services that are registeredin the JLS are persistent and will be maintained aslong as the service is alive. Jini also provides mech-anisms for distributed events, distributed leasingand transactions.

JavaSpaces provides a distributed data store forJava objects. Objects stored in JavaSpaces areloosely coupled; anyone can take an object from aspace without knowing (or caring) the detailsabout the person who put it there. Operations onJavaSpaces are transactionally secure if the trans-action mechanism is used. All the service calls ina transaction are committed, or none of them iscommitted. Transactions are supported for a singleoperation on a single space as well as multipleoperations over many spaces. Like the JLS, Java-Spaces are also persistent; an object will remain ina space until it is explicitly removed. It alsoincludes the search facilities of Jini and its mecha-nisms for distributed leases.

As mentioned below, our prototype system pro-vides a management interface that conforms to theWeb Services standard, which uses the XML(Extended Markup Language) coding scheme forclient–server communication. Since the Web Ser-vices registries provide a service quite similar tothe Jini Lookup Service, we were contemplatingthe possibility of using the Web Services standardalso within our distributed prototype system;however, we decided to use the Jini technology



internally. The JLS is more powerful and maturethan the XML-based service registries. Since Jinipasses Java objects via RMI, there is no need tohave XML schema definitions for all remoteservice calls. Although using Jini/JavaSpaceslimits us to the Java programming language (whileXML is language independent), the internal Jiniservice calls are transparent to the customer and toother applications.

Design of a Management ToolBased on Jini and JavaSpaces

—System Architecture—

The architecture of a lightpath managementsystem,6 which we call a User-Controlled Light-path Provisioning (UCLP) system, is shown inFigure 4. Our UCLP system is primarily designed

to support Grid applications defined in the OpenGrid Service Infrastructure (OGSI). The UCLPsystem enables Grid applications to dynamicallyset up end-to-end lightpaths across multiple inde-pendently managed customer domains to transfervast amount of data. The figure shows the genericarchitecture and does not show the replication ofthe system components in the different parts of thenetwork. Typically, one instance of each compo-nent shown would exist in each federation (i.e., a collaborative group of customer domains).However, they may also be shared. The JLS, Java-Spaces, Switch Communication Service (whichinterfaces to a single switch or a cloud of tightlycoupled switches) and the Grid Service AccessPoint (SAP) may run on different computers. TheJini SAP and the LPO Service are downloaded tothe process using them, in this case the Grid SAP.

This architecture uses the concept of servicedirectories at two levels. First, internally, the JLS is

356 J. WU ET AL.


JiniLookupService

LightspanObjectService

JiniServices Jini Service Access Point

Switch Communication Service

Resource Object Management

Switch Specific Functions

TL1 O-UNI

LightspanObjects

ResourceObjects

JavaSpaces

User(OGSI Grid Client)

XML/SOAP

User Functions

Grid Service Access Point

Admin Functions

or

Figure 4. Overview of UCLP architecture

used to find the different instances of switches andJavaSpaces in the network. Second, a Grid SAPadvertises its service instantiation through a WebServices Inspection Language (WSIL) pointer or anentry in the Universal Description, Discovery andIntegration (UDDI) database. The client commu-nicates the customer requests to the Grid SAPusing the Simple Object Access Protocol (SOAP)adopted for Web Services. These requests are con-verted into Java procedure calls within the GridSAP which then performs these calls on its localJini SAP. Then, the Jini SAP executes these com-mands with the help of the other componentswithin the system.

In a typical system configuration, each federa-tion has its own set of services supported by the

UCLP system (for short, we call them UCLP ser-vices), including its own JavaSpaces and JLS. Eventhough many services are accessible across feder-ations, they are maintained independently of thosein other federations. It should be noted thatalthough resources are shared among differentdomains, it is still important to maintain theadministrative boundaries between each domainto avoid confusion about the ownership of assetsand administrative privileges.

—Lightpath Management Services—

Figure 5 shows the system architecture in moredetail and indicates the main service methods pro-



UCLPServices

Architecture

The Switch Implementation classimplements the CS Interface to proxybetween different network devices

LPO DB

Switch StateDB

JavaSpace

CS Interface

Stateless services that areexecuted at the client

LPOService

Notification From Switch

Delete LPO

Find JavaSpace

Add LPODelete E2E Connection

Concatenate LPOPartition LPO

Create E2E Connection

Query JavaSpace

Private methods within theservice

SwitchCommunication

Service

Create ROQuery Switch

Undo XCMake XC

TL1O-UNI,GMPLS

RO Manager


AdminFunctions

Display ResourcesCreate New LPO

Delete LPOModify LPO

UserFunctions

ConnectionRequestConnectionDeleteConnectionStatus

Notification

To ASTo OXC

Display ResourcesCreate Fund LPODelete Fund LPO

Modify LPO

ConnectionRequestConnectionDeleteConnectionStatus

NotificationFind Switch Path

Find LPOs

Jini SAP

LPO Service

JavaSpace Proxy

CS Interface

Jini Lookup Service

Jini SAP

Federated JLS1

Federated JLSn

Figure 5. Detailed UCLP services architecture

vided by the component interface. The manage-ment services provided by our system are classi-fied into two groups: those only available toadministrative customers and those available to allcustomers. The latter include in particular ‘Con-nectionRequest’, by which a customer can requestthe establishment of an end-to-end connectionfrom a given entry port of a given switch to a givenexit port on another given switch, possibly belong-ing to a different federation. One of the functionsreserved to administrative customers is the addi-tion of new physical links to the available opticalnetwork.

In our design, Light Path Objects (LPOs) areobjects stored in JavaSpaces. An LPO is an abstrac-tion of a lightspan. It is associated with a set ofattributes and methods that enable possiblepeering to other LPOs at a switch to create an end-to-end connection or a longer lightspan. Sup-ported customer operations on LPOs include:concatenating two LPOs, partitioning one LPOinto many LPOs sharing common start and end-points but with smaller bandwidth allocations,and reserving/using/releasing LPOs. The admin-istrative operations include adding new LPOs anddeleting LPOs corresponding to changes in thephysical layer and the allocation of new resources.

For the execution of the ConnectionRequest, JiniSAP uses the internal methods FindSwitchPathand FindLPOs. The latter searches through thepertinent JavaSpaces to look for LPOs with attrib-utes suitable for the end-to-end connection to bebuilt. It also uses the functions provided by themethods of the LPO Service.

—Integration with Grid Applications—

To make UCLP services easy to utilize by con-ventional network applications, the interface toUCLP services is IP oriented. For example, the‘create E2E connection’ operation takes two publicIP addresses of source and destination hosts asinputs and returns information about the newlycreated connection, including the IP addresses ofthe UCLP endpoint network interface cards of thetwo hosts. Therefore, the integration becomes asimple IP replacement. Taking GridFTP as anexample, a UCLP-enabled GridFTP client is actu-ally a simple wrapper of the original GridFTPclient. The UCLP service is called first to create anE2E connection. Then the original GridFTP appli-cation is performed for clients with the UCLP end-point IP addresses to transfer data through theestablished UCLP connection.

However, this simple solution won’t work if thetransfer is initiated by a third party (Figure 6). Inthis case, the UCLP endpoint IP addresses of the source and destination host are unknown tothe client that initiates the transfer. Although theFTP/GridFTP protocol is general enough toachieve this kind of transfer, the existing GridFTPclient doesn’t support third-party initiated filetransfers yet. It assumes that FTP control messagesand data flows always go through the samenetwork interface. Therefore a more sophisticatedGridFTP client is needed to realize a third-partyinitiated file transfer over a UCLP connection.

We use the Reliable File Transfer (RFT) serviceas an alternative solution. The RFT service is an

358 J. WU ET AL.


192.168.0.1labpc1-uclp.canet4.net

192.168.0.2labpc2-uclp.canet4.net

205.189.33.171labpc1.canet4.net

205.189.33.172labpc2.canet4.net

GridFTP/RFT

Server 1 Server 2

RFT Client

UCLP

Internet

GridFTP/RFT

Figure 6. A third-party initiated file transfer through UCLP

OGSI-compliant Grid service that provides inter-faces for controlling and monitoring third-partyfile transfers between GridFTP servers. Using theRFT service, clients can connect to the RFT serviceusing one IP address and ask it to perform trans-fers between other IP addresses. Therefore, third-party transfers can be realized easily by a simplewrapper of the RFT client. The only thing is tomake sure UCLP end points are reachable by theRFT service. This can be easily done by starting theRFT services on the hosts with a UCLP end point.

Prototype Demonstration at theEighth Global Grid Forum

The prototype of our UCLP system was demon-strated at the Eighth Global Grid Forum (GGF8)held in Seattle on June 24–27, 2003. An RFT appli-cation was used to initiate the setup of an E2Elightpath (in the demonstration, a SONET STS-3connection was established to simulate a light-path). An RFT client used the established E2Elightpath to perform a server-to-server file trans-fer. The Grid FTP client ran at the site of the GGF8,while the UCLP system ran remotely in our labs inOttawa, Canada. The UCLP deployment architec-ture for the demonstration is shown in Figure 7.

The purpose of this demonstration was mainlyto show the system architecture and the interac-tions among service modules. Therefore, only onecustomer domain was used in the demonstration.An E2E lightpath was successfully set up dynam-ically on the demand of the Grid client. Thedemonstration proved that the design of ourUCLP system is technically feasible. Service archi-tecture and interactions are shown in Figure 8. Themajor steps of interactions are listed as follows:

1. Grid Service receives the request andinvokes the Jini SAP.

2. User makes ‘connectionRequest’ call via theGrid client (user enters IP address of sourceand destination machines that are to be connected).

3. Jini SAP queries the JLS to get the proxy tothe JavaSpaces the resource objects andLPOs reside in.

4. Jini SAP uses the source and destination IPaddresses entered by the user to search theJavaSpaces for the resource objects for thatconnection.

5. Once the specific switch information isretrieved, it can be used to find the LPO.

6. Jini SAP then uses the source switch ID fromthe LPO to download the proxy for the SCScontrolling Switch 1 (ONS 1).

7. Jini SAP does the same for Switch 2 (ONS 2).8. Jini SAP sends a ‘Cross-connect’ command

to SCS 1.9. SCS 1 initiates a TL1 session with the switch

to make cross-connection at Switch 1.10. Jini SAP sends a ‘Cross-connect’ command

to SCS 2.11. SCS 2 initiates a TL1 session with the switch

to make cross-connection at Switch 2.12. The two servers are now connected via

LPO 1.

Related Work—Status of Control Techniques forMulti-Domain Optical Networks—

The E2E lightpath provisioning in the customer-managed environment involves multiple indepen-dent customers without coordination throughcentralized control or management. This feature



PC 1Grid Client

LPO 1CANARIEONS Lab

Server A Server B

CRCBADLAB TM

GGF8

STS3CONS 1 ONS 2

PC 2Grid SAP

PC 3Jini Lookup Service

JavaSpacesJini SAP

SCS_ONS1SCS_ONS2

Figure 7. UCLP deployment architecture for thedemonstration at the Eighth Global Grid Forum

has some similarities to multi-domain control ormanagement. In this section, we will summarizethe status and existing techniques of multi-domainor inter-domain control and management foroptical networks.

The motivations for dividing a network intodomains are identified as:7 (1) to define adminis-trative boundaries between network operators; (2)to allow for scalability of routing and/or signal-ing; (3) to enable isolation of partitions for securityor reliability; (4) to accommodate technology dif-ferences between systems, for example, by meansof partitioning a single carrier’s network into sep-arate single-vendor sub-networks.

Standards organizations are active to define therequirements, framework and standards of theinter-domain control plane for optical networks.Three classes of functionality have been identi-fied:8 discovery, signaling and routing. The dis-covery process obtains information about theconnectivity to and capabilities of neighbornetwork elements and sub-networks. The signal-ing or connection control is to set up, tear down,modify and maintain connections across net-works. The routing is to exchange reachability,topology and resource status information.

So far, the control architecture and protocols forcustomer-controlled optical networks have not

360 J. WU ET AL.


Jini Lookup Service

Grid Client(initiator)

LPO 1CANARIEONS Lab

1

2

8

6 5

3

12

Server A Server B


User Functions

4

7

10

9 11

ONS 1 ONS 2

Lightspan Objects:

LPO 1 from ONS 1 toONS 2

Resource Objects:

Source resource objectfrom Server A to ONS 1

Destination resourceobject from ONS 2 to

Server B

JavaSpaces

JavaSpaces Proxy

Jini SAP

SCS Interface 2

SCS Interface 1

SCS 1

TL1

Dis-crossconnetCross-connect

SCS 2

TL1

Dis-crossconnetCross-connect

UCLP System at CRC BADLAB TM

Figure 8. Service architecture and interactions for the demonstration at the Eighth Global Grid Forum

been well defined. This situation will remain for aperiod of time before the control architecture andprotocols for inter-domain optical networking arewell developed. The market demands play a keyrole in the process of the standardization of thecontrol architecture and protocols for customer-controlled optical networks. In the early stages, theinteroperability and performance of control proto-cols may not be the primary concern for customer-controlled or customer-managed optical networks.

—Existing ManagementTechniques for Customer-

Managed Optical Networks—The ability of active networks to program the

network enables new protocols and innovativecost-effective technologies to be easily deployed atintermediate nodes. An important application ofactive networks is the distribution of networkmanagement functionality,9 e.g. network configu-ration management. The configuration manage-ment performs several tasks such as discoveringnew devices, maintaining topology information,partitioning resources, controlling (e.g. installing,updating, reconfiguring) the software of thenetwork elements remotely, setting up VPNs, etc.A programmable controller in a resource agent is the key building block that allows customers to customize network control and provides the resource agent with the intelligence forautonomous operations. Each customer networkmanagement system has its own programmablecontrollers in the resource agents. The program-mable controller can download software packageswith different functions. An example is the signal-ing and routing control software required for eachlogical subset of a switch.

Although a virtual active network was proposedto transform a multi-domain environment into asingle domain view for customers,10 the manage-ment of VPNs and active networks mainly con-centrates on the partitioning of carriers’ networkresources and the autonomous operation ofresources partitioned for individual customerswithout or with minimal interaction with the car-riers. These management solutions meet therequirements for the operation of network switch-ing elements in a condominium fashion, whichwere defined by the IETF in Anderson and

Buerkle.11 However, the collaboration amongautonomously operated networks is notaddressed. In a customer-managed opticalnetwork, to establish an E2E lightpath, multipleindependent customer domains participate collab-oratively. This motivates us to use the Jini tech-nology to design a management system forcustomer-managed E2E lightpath provisioning.The federation concept and associated functions inthe Jini technology facilitate our system design.

ConclusionsWith the availability of dark fibers in some

metropolitan areas and leased or condominiumlong-haul point-to-point wavelength networks,customers have the opportunity to construct and manage their own networks. Thus, the E2Elightpath provisioning involves multiple inde-pendently managed customer domains. The distributed resource management, collaborationamong independent domains and partitioning ofprovider’s resources to customers are major issuesfor this new network architecture.

A management system for such applicationsmay be built based on the concept of service direc-tories. Jini and JavaSpaces provide a number ofadvantages for developing such applications. Wepresented a design of a management system forcustomers to provision E2E lightpaths across mul-tiple independent customer domains. This tool canbe used as a traffic engineering mechanism forinter-domain applications, where separate point-to-point lightpaths are set up operating in parallelwith the normal hop-by-hop routes. Interfaces toGrid applications are designed so that this tool canbe easily integrated into Grid applications. Theprototype of our management system was suc-cessfully demonstrated at the Eighth Global GridForum to dynamically establish an E2E lightpathon the demand of a Grid client to do a Grid FTPserver-to-server file transfer.

The architecture for customer-managed opticalnetworks is the foundation for the future brokertrading market of optical networking, where thedynamic configuration of network resources andthe management of partnership and leasing areessential. It also finds significant opportunities innon-profit research and educational networks,where reduced operational cost is observed.



AcknowledgementsWe thank Jun Chen, Wei Zhang, Ling Zou andMathieu Lemay for their contributions to theimplementation of the prototype system for cus-tomer-managed end-to-end lightpath provision-ing. We thank Sergi Figuerola, Eduard Grasa,Joaquim Recio and Albert López from the Techni-cal University of Catalonia, Barcelona, Spain, fortheir participation in the discussion. This researchis partially supported by CANARIE, Inc. under adirected research program.

References1. Zhang Z, Fu J, Guo D, Zhang L. Lightpath routing

for intelligent optical networks. IEEE Network 2001;15(4): 28–35.

2. Isaacs R, Leslie I. Support for resource-assured anddynamic virtual private networks. IEEE Journal onSelected Areas in Communications 2001; 19(3): 460–472.

3. Freeman E, Hupfer S, Arnold K. JavaSpaces Princi-ples, Patterns, and Practice. Addison-Wesley:Reading, MA, 1999.

4. Li S et al. Professional Jini. Wrox Press Inc., 2000.5. Curbera F et al. Unraveling the Web Services Web:

an introduction to SOAP, WSDL, and UDDI. IEEEInternet Computing 2002; 6(2): 86–93.

6. Wu J, Campbell S, Savoie M, Zhang H, BochmannG, St Arnaud B. User-managed end-to-end lightpathprovisioning over CA*net 4. In Proceedings ofNational Fiber Optic Engineers Conference (NFOEC2003), Orlando, FL, 7–11 September 2003.

7. Bernstein GM, Sharma V, Ong L. Interdomainoptical routing. Journal of Optical Networking 2002;1(2): 80–92.

8. Strand J et al. NNI requirements and frameworkresource document. Optical Internetworking Forum(OIF) technical document OIF2001.535, 2002.

9. Boutaba R, Polyrakis A. Projecting advanced enter-prise network and service management to activenetworks. IEEE Network 2002, 16(1): 28–33.

10. Brunner M. A service management toolkit for activenetworks. In Proceedings of IEEE/IFIP Network Oper-ations and Management Symposium (NOMS), Honolulu, HI, April 2000; 265–278.

11. Anderson T, Buerkle J. Requirements for thedynamic partitioning of switching elements. IETFRFC 3532, May 2003. �

362 J. WU ET AL.


If you wish to order reprints for this or anyother articles in the International Journal ofNetwork Management, please see the SpecialReprint Instructions inside the front cover.

Documents

Customer-managed end-to-end lightpath provisioning