A Survey of Programmable Networks

Andrew T. Campbell1, Herman G. De Meer2, Michael E. Kounavis1,Kazuho Miki3, John B. Vicente4, and Daniel Villela1

1Center for Telecommunications Research,Columbia University

2 University of Hamburg, Germany3 Hitachi Limited

4 Intel Corporation

genesis@comet.columbia.edu

AbstractIn this paper we present a programmable networking model that provides a commonframework for understanding the state-of-the-art in programmable networks. A numberof projects are reviewed and discussed against a set of programmable networkcharacteristics. We believe that a number of important innovations are creating aparadigm shift in networking leading to higher levels of network programmability. Theseinnovations include the separation between transmission hardware and control software,availability of open programmable network interfaces, accelerated virtualization ofnetworking infrastructure, rapid creation and deployment of new network services andenvironments for resource partitioning and coexistence of multiple distinct networkarchitectures. We present a simple qualitative comparison of the surveyed work andmake a number of observations about the direction of the field.

1. INTRODUCTIONThe ability to rapidly create, deploy and managenovel services in response to user demands is a keyfactor driving the programmable networking researchcommunity. Results from this field of research arelikely to have a broad impact on customers, serviceproviders and equipment vendors across a range oftelecommunication sectors, including broadband,mobile and IP networking. Competition betweenexisting and future Internet Service Providers (ISPs)could solely hinge on the speed at which one serviceprovider can respond to new market demands overanother. The introduction of new services is achallenging task and calls for major advances inmethodologies and toolkits for service creation andenabling network technologies. A vast amount ofservice-specific computation, processing andswitching must be handled and new networkprogramming environments have to be engineered to

enable future networking infrastructures to be open,extensible and programmable.

Before we can meet this challenge, we need to betterunderstand the limitations of existing networks andthe fundamentals for making networks moreprogrammable. There is growing consensus that thesenetwork fundamentals are strongly associated withthe deployment of new network programmingenvironments, possibly based on “network-wideoperating system support”, that explicitly recognizeservice creation, deployment and management in thenetwork infrastructure. For example a futureprogrammable network operating system could bebased on active network execution environments [42]operating on node operating system [40] or opensignaling network kernels [30] supporting thecoexistence of multiple control architectures [33].Both of these proposals squarely address the sameproblem: how to open the network up and accelerateits programmability in a controlled and secure

manner for the deployment of new architectures,services and protocols.

The separation of communications hardware (i.e.,switching fabrics, routing engines) from controlsoftware is fundamental to making the network moreprogrammable. Such a separation is difficult torealize today. The reason for this is that switches androuters are vertically integrated - akin to mainframesof the 70s. Typically, service providers do not haveaccess to switch/router control environments (e.g.Cisco’s IOS operating system), algorithms (e.g.routing protocols) or states (e.g., routing tables, flowstates). This makes the deployment of new networkservices, which may be many orders of magnitudemore flexible than proprietary control systems,impossible due to the closed nature of network nodes.The question is, how do we go about ‘opening up theboxes’ for deployment of third party control softwareand services?

This paper examines the state of the art inprogrammable networks. In Section 2, we present anddiscuss two schools of thought on programmablenetworks advocated by the Active Networks (AN)[20] and Open Signalling (Opensig) [39]communities. The state-of-the-art in programmablenetworks is rather complex to analyze beyondhistorical differences. Recently, a number ofprogrammable network toolkits have beenimplemented. By reviewing each contribution in turn,we arrive at a common set of features that govern theconstruction of these programmable networks. InSection 3, we present a generalized model andcommon set of characteristics to better understandthe contributions found in the literature. Followingthis, in Section 4, we discuss a number of specificprojects and characterize them in terms of a simpleset of characteristics. In Section 5, we present asimple qualitative comparison of the surveyed workand make a number of observations about thedirection of the field. We believe that a number ofimportant innovations are creating a paradigm shift innetworking leading to higher levels of networkprogrammability. This leads us to the conclusion thatthe ultimate challenge facing the programmablenetworking community is the development ofprogrammable virtual networking environments.

2. METHODOLOGIESThere has been an increasing demand to add newservices to networks or to customize existing networkservices to match new application needs. Recentexamples of this include the introduction of integratedand differentiated services to IP networks offeringenhanced IP QOS. The introduction of new servicesinto existing networks is usually a manual, timeconsuming and costly process. The goal ofprogrammable networking is to simplify thedeployment of new network services, leading tonetworks that explicitly support the process ofservice creation and deployment. There is generalconsensus that programmable network architecturescan be customized, utilizing clearly defined openprogrammable interfaces (i.e., network APIs) and arange of service composition methodologies andtoolkits.

Two schools of thought have emerged on how tomake networks programmable. The first school isspearheaded by the Opensig community, which wasestablished through a series of internationalworkshops. The other school, established byDARPA, constitutes a large number of diverse ANprojects. The Opensig community argues that bymodeling communication hardware using a set ofopen programmable network interfaces, open accessto switches and routers can be provided, therebyenabling third party software providers to enter themarket for telecommunications software. TheOpensig community argues that by “opening up” theswitches in this manner, the development of new anddistinct architectures and services (e.g., virtualnetworking [34]) can be realized. Open signaling asthe name suggests takes a telecommunicationsapproach to the problem of making the networkprogrammable. Here, there is a clear distinctionbetween transport, control and management thatunderpin programmable networks and an emphasison service creation with QOS. Recently, the IEEEProject 1520 [9] on Programmable Interfaces forNetworks is pursuing the Opensig approach in anattempt to standardize programming interfaces forATM switches, IP routers and mobiletelecommunications networks. Physical networkdevices are abstracted as distributed computingobjects (e.g. virtual switches [15], switchlets [33],and virtual base stations [6]) with well-defined open

programmable interfaces. These open interfacesallow service providers to manipulate the states of thenetwork using middleware toolkits (e.g., CORBA) inorder to construct and manage new network services.

The AN community advocates the dynamicdeployment of new services at runtime mainly withinthe confines of existing IP networks. The level ofdynamic runtime support for new services goes farbeyond that proposed by the Opensig community,especially when one considers the dispatch, executionand forwarding of packets based on the notion of“active packets”. In one extreme case of activenetworking, “capsules” [42] comprise executableprograms, consisting of code (for example Java code)and data. In active networks, code mobility representsthe main vehicle for program delivery, control andservice construction. The granularity of control canrange from the packet and flow levels through theinstallation of completely new switchware [3]. Theterm ‘granularity of control’ [12] refers to the scopeof switch/router behavior that can be modified by areceived packet. At one extreme, a single packetcould boot a complete software environment seen byall packets arriving at the node. At the other extreme,a single packet (e.g., a capsule) can modify thebehavior seen only by that packet. Active networksallow the customization of network services at packettransport granularity, rather than through aprogrammable control plane. Active networks offermaximum flexibility in support of service creationbut with the cost of adding more complexity to theprogramming model. The AN approach is, however,an order of magnitude more dynamic than Opensig’squasi-static network programming interfaces.

Both communities share the common goal to gobeyond existing approaches and technologies forconstruction, deployment and management of newservices in telecommunication networks. Bothmovements include a broad spectrum of projects withdiverse architectural approaches. For example, fewAN projects consider every packet to be an activecapsule and similarly few Opensig projects considerprogrammable network interfaces to be static. TheOpensig approach, however, clearly separatesnetwork control from information transport and isprimarily focused on programmable switches thatprovide some level of QOS support. In contrast,projects under the AN umbrella have historically

been focused on IP networks, where the control anddata paths are combined.

3. PROGRAMMABLE NETWORKING MODEL

3.1 Communications and ComputationA programmable network is distinguished from anyother networking environment by the fact that it canbe programmed from a minimal set of APIs fromwhich one can ideally compose an infinite spectrumof higher level services. We present a generalizedmodel for programmable networks as a three-dimensional model illustrated in Figure 1. This modelshows the Internet reference model (viz. application,transport, network, link layers) augmented withtransport1, control and management planes. Thedivision between transport, control and managementallows our model to be generally applicable totelecommunications and Internet technologies. Thenotion of the separation between transport, controland management is evident in architectures. In thecase of Internet there is a single data path but clearlyone can visualize transport (e.g., video packets),control (e.g., RSVP) and management (e.g., SMNP)mechanisms. In the case of telecommunicationnetworks there is typically support in the architecturefor transport, control and management functions.This division is motivated by the different ways thesenetworking functions utilize the underlying hardwareand by the distinct time scales over which theyoperate. In both cases, the planes of our generalizedmodel remain neutral supporting the design space ofdifferent networking technologies.

The programmability of network services is achievedby introducing computation inside the network,beyond the extent of the computation performed inexisting routers and switches. To distinguish thenotion of a “programmable network architecture”from a “network architecture”, we have extended thecommunication model and augmented it with acomputation model, explicitly acknowledging theprogrammability of network architectures. We canview the generalized model for programmablenetworks as comprising conventional communication,encompassing the transport, control and management

1 In this case planes collectively represent cross-layer services

and protocols.

planes, and computation as well, as illustrated inFigure 1. Collectively, the computation andcommunication models make up a programmablenetwork. The computation model providesprogrammable support across the transport, controland management planes, allowing a network architectto program individual layers (viz. application,transport, network and link layers) in these planes.Another view is that programmable support isdelivered to the transport, control and managementplanes through the computation model.

Figure 1: Computation and Communication Models

In Figure 2, an alternative view of the generalizedmodel is shown. The key components of thecomputation model are represented as a distributednetwork programming environment and a set of“node kernels”2. Node kernels are node operatingsystems realizing resource management. Node kernelshave local significance only, that is, they managesingle node resources, potentially shared by multipleprogrammable network architectures. The networkprogramming environment provides middlewaresupport to distributed network programming services.Figure 2 illustrates the separation of switchinghardware from programming and communicationsoftware. Two sets of interfaces are exposed. Thefirst set of interfaces represents the networkprogramming interfaces between networkprogramming environments and programmable

2 We borrow the term node kernel from the work on NodeOS

[40] and broadband kernels [30] by the active networking andOpensig communities, respectively.

network architectures. The lower set of interfacesrepresents the node interfaces between node kernelsand network programming environments. We believethat there needs to be some agreement orstandardization of these interfaces to allow platform-independent network programming. This is likely tohappen through a number of forums, e.g., IEEEProgrammable Interfaces for Networks [9], DARPAActive Networks [20], Multiservice Switching Forum[38], OPENSIG [39] and IETF (e.g., the new workitem on GSMP) or by the emerging newprogrammable network industries [48] [19].

Research on programmable networks is focused onall facets of this model. Different programmingmethodologies, levels of programmability, andcommunication technologies have been investigated.Some projects, especially from the Opensigcommunity have placed more emphasis on APIdefinitions. Others focus on issues related to codemobility or contribute to the application domain.Dynamic “plug-ins” have been investigated for theconstruction or potential extension of new protocolsor applications. In what follows, we provide a moredetailed overview of the components in ourgeneralized model. It is our belief that independentcontributions to the field are beginning to convergeand it is our intention to indicate this convergence byway of survey.

3.2 Node KernelMany node vendors incorporate operating systemsupport into their switches and routers to handlecommunication functions of network nodes, e.g.CISCO routers use the IOS environment and ATMLATM switches use the ATMOS micro-kernel.Typically, these node operating systems support avariety of communications activities, e.g., signaling,control and management processes, inter-processcommunication, forwarding functions, anddownloading of new boot images. Currently, thesenode operating systems are closed to third partyproviders because of their proprietary nature, andthey are limited in their support for evolving networkprogramming environments. While the idea ofintroducing computation power into nodes is not new,there is a greater need for computation elements toabstract node functionality and allow it to be openand programmable. The computation

managem

ent planecontrol plane

transport planelink layer

network layer

transport layer

application layer

comput

ation

model

communi

cation

model

model, introduced in the previous section, enables theprogrammability of the communication model andrequires low-level programmable support forcommunication abstractions (e.g., packets, flows,tunnels, virtual paths), dynamic resource partitioningand security considerations.

We describe this low-level programming environmentthat runs on switch/routers as the node kernel. Thenode kernel represents the lowest level ofprogrammability, providing a small set of nodeinterfaces. These interfaces support the manipulationof the node state (e.g., accessing and controlling thenode resources) and the invocation of communicationservices (e.g. communication abstractions andsecurity). The node kernel is responsible for sharingnode computational (e.g., sharing the CPU) andcommunication resources, (e.g., partitioning thecapacity of a multiplexer), as well supporting coresecurity services. A node kernel may operate on anytype of network node, end-system or device, forexample, IP router, ATM switch, or base station. Itmay also provide access to dedicated hardwareoffering fast packet processing services to networkprogramming environments. A node kernel has localsignificance only, providing the networkprogramming environment with a set of low-levelprogramming interfaces, that are used by networkarchitects to program network architectures in asystematic manner.

3.3 Network Programming EnvironmentNetwork programming environments support theconstruction of networks, enabling the dynamicdeployment of network services and protocols.Network programming environments supportdifferent levels of programmability, programmingmethodologies, networking technologies and

application domains. Network programmingenvironments operate over a set of well-defined nodekernel interfaces offering distributed toolkits for therealization of programmable network architecturesthrough the deployment of distributed servicecomponents. In this sense, one can view network-programming environments as the “middleware glue”between executing network architectures and the nodekernels themselves, as illustrated in Figure 2.Network programming environments provide networkarchitect/designers with the necessary environmentand tools for building distinct network architecturesthat run in a distributed fashion on multiple nodekernels. In this sense network programmingenvironments support the programming of networkarchitectures in the same way that softwaredevelopment kits (SDKs) allow developers to buildnew applications that run on native operatingsystems.

This “middleware glue” can be constructed fromscratch or be built on top of well-defined distributedobject computing environments. For example, thexbind [15] and mobiware [6] toolkits addressprogrammability of broadband and mobile networks,respectively, and are built using COBRA middlewaretechnology. Other approaches use mobile codetechnology and virtual machines to dynamicallyprogram the network. For example, the ActiveNetwork Transport System (ANTS) incorporatescapsule technology [45], leveraging the Java VirtualMachine for new protocol deployment. Bothapproaches result in toolkits that execute on nodekernels offering a high level of programmability forservice creation and deployment of distinct networkarchitectures.

Network programming environments offer a set ofopen interfaces and services to network

Figure 2: Generalized Model for Programmable Networks

programmable network architecture

network programming environment

node kernel node kernel

node HW node HW

nodeinterfaces

networkprogramminginterfaces

communicationmodel

computationmodel

designers/architects to program distinct networkarchitectures. Network programming environmentssupport the construction of network architecturesthrough service composition, service control, andresource and state management. Services offered bynetwork programming environments can range fromsimple Remote Procedure Calling (RPC) betweendistributed network objects to sophisticated dynamicloading of mobile code and fast compilation ofintermediate machine-independent representation.Different types of network programmingenvironments offer different levels ofprogrammability to network architectures. Forexample, mobile code technologies offer the mostradical solution to the development of services inprogrammable networks when compared to RPC-based object middleware. We identify the ‘level ofprogrammability’ as an important characteristic ofprogrammable networks.

3.4 Programmable Network ArchitectureThe goal of network programming environments is toprovide the necessary support to dynamicallyprogram new network architectures. Networkprogramming environments do not offer core networkalgorithms (e.g., routing, signaling) that define anddifferentiate network architecture in the same waythat operating systems do not embed applicationspecific algorithms in the kernel. Rather, a networkprogramming environment offers a set of networkprogramming interfaces for constructing networkarchitectures. Philosophically this is similar toconstructing new applications using softwaredevelopment kits. However in this case theapplication is the network architecture.

We broadly define network architecture as having thefollowing attributes3:

• network services, which the network architecturerealizes as a set of distributed network algorithmsand offers to the end systems;

• network algorithms, which includes transport,signaling/control and management mechanisms;

3 This is of course an over simplification of a complex system.

Our goal here is to be illustrative in support of thegeneralized model and not definitive regarding a definition ofnetwork architecture.

• multiple time scales, which impact and influencethe design of the network algorithms; and

• network state management, which includes thestate that the network algorithms operate on (e.g.,switching, routing, QOS state) to supportconsistent services.

Network programming environments offer creationand deployment tools and mechanisms that allownetwork architects to program and build new networkarchitectures. Programmable network architecturesare realized through the deployment of a set ofnetwork algorithms that take into account networkstate and reflect the time scales over which thesealgorithms operate. Network algorithms arepotentially as diverse as the application base thatexists in the end-systems today. Programmablenetwork architectures may range from simple best-effort forwarding architectures to complex mobileprotocols that respond dynamically to changes inwireless QOS and connectivity. Given this diversity,it is necessary that both network programmingenvironments and node kernels are extensible andprogrammable to support a large variety ofprogrammable network architectures.

4. PROGRAMMABLE NETWORKSFollowing on from the discussion of the generalizedmodel for programmable networks, we now survey anumber of programmable networking projects thathave emerged in the literature. We attempt to identifyessential contributions of the various projects to thefield in terms of a set of characteristics. The survey isnot intended to represent an exhaustive review of thefield4. Rather, we discuss a set of projects that arerepresentative of each programmable networkcharacteristic introduced, focusing on the pertinentand novel features of each project and then, inSection 5, we compare them to the generalized modelintroduced in the preceding section.

4.1 CharacteristicsA number of research groups are actively designingand developing programmable network prototypes.Each group tends to use its own terminology.However, on examination one can observe a common

4 For a survey on active networks see [43].

set of characteristics that govern the construction ofthese programmable networks. We use thesecharacteristics to better understand the field:

• networking technology, which implicitly limitsthe programmability that can be delivered tohigher levels. For example, some technologies aremore QOS programmable (e.g., ATM), scalable(e.g., Internet) or limited in bandwidthavailability (e.g., mobile networks);

• level of programmability, which indicates themethod, granularity and time scale over whichnew services can be introduced into the networkinfrastructure. This in turn is strongly related tolanguage support, programming methodology ormiddleware adopted. For example, distributedobject technology can be based on RPC [46] ormobile code [45] methodologies resulting inquasi-static or dynamically composed networkprogramming interfaces;

• programmable communications abstractions,which indicate the level of virtualization andprogrammability of networking infrastructurerequiring different middleware and potentiallynetwork node support (e.g., switch/router, basestation). For example, programmablecommunications abstractions include virtualswitches [30], switchlets [33], active nodes [40],universal mobile channels [32] and virtual activenetworks [21]; and

• architectural domain, which indicates thetargeted architectural or application domain (e.g.,signaling, management, transport). Thispotentially dictates certain design choices andimpacts the construction of architectures, andservices offered, calling for a wide range ofmiddleware support. Examples include,composing application services [4],programmable QOS control [30] and networkmanagement [41]).

4.2 Networking TechnologyA number of programmable network prototypes havebeen targeted to specific networking technologies.The motivation behind these projects is to make thetargeted networking technology more programmable

in an attempt to overcome particular deficienciesassociated with supporting communication services.

4.2.1 IP networks: Smart PacketsThe University of Kansas has developed smartpackets, a code-based specialized packet conceptimplemented in a programmable IP environment [29].Smart packets represent elements of in-band or out-of-band mobile code based on Java classes. Smartpackets propagate state information in the form ofserialized objects and carry identifiers forauthentication purposes. An active node architecturesupports smart packets by exposing a set of resourceabstractions and primitives made accessible to smartpackets. Active nodes incorporate:

• resource controllers, which provide interfaces tonode resources;

• node managers, which impose static limits onresource usage; and

• state managers, which control the amount ofinformation smart packets may leave behind at anactive node.

The active node supports a feedback-schedulingalgorithm to allow partitioning of CPU cycles amongcompeting tasks and a credit-based flow-controlmechanism to regulate bandwidth usage. Each smartpacket is allocated a single thread of CPU and someamount of node resources. Active nodes also includerouter managers that support both default routingschemes and alternative routing methods carried bysmart packets. The smart packets testbed has beenused to program enhanced HTTP and SMTP servicesthat show some performance benefits overconventional HTTP and SMTP by reducingexcessive ACK/NAK responses in the protocols. Abeacon routing scheme supports the use of multiplerouting algorithms within a common physical IPnetwork based on smart packets.

4.2.2 ATM Networks: xbindATM technology provides connection-orientedcommunications and has been tailored towards QOSprovisioning of multimedia networks. Althoughessential features of QOS provisioning, such asadmission control and resource reservation, areinherently supported by the ATM technology, its

signaling component is unsuitable for practical usagedue to its significant complexity. xbind [15]overcomes these service creation limitations byseparating control algorithms from thetelecommunications hardware. Emphasis is placed onthe development of interfaces to provide open accessto node resources and functions, using virtual switchand virtual link abstractions. The interfaces aredesigned to support the programmability of themanagement and control planes in ATM networks.

The xbind broadband kernel [47], which is based onthe XRM model [15], incorporates three networkmodels abstracting a broadband network, multimedianetwork and service network. The multimedianetwork supports programmable networkmanagement, network control, state management,connection management and media stream control.The xbind testbed incorporates multivendor ATMswitches using open signaling and service creation tosupport a variety of broadband services, transportand signaling systems with QOS guarantees.

4.2.3 Mobile Networks: MobiwareMobiware [6] is a software-intensive openprogrammable mobile architecture extending thexbind model of programmability to packet basedmobile networks for the delivery of adaptive mobileservices over time-varying wireless links. Mobiwareincorporates object-based, CORBA programmabilityfor the control plane but also allows active transportobjects (i.e., code plug-ins) based on Java byte codeto be loaded into the data path. At the transportlayer, an active transport environment injectsalgorithms into base stations providing value-addedservice support at strategic points inside the network.At the network layer, a set of distributed objects thatrun on mobile devices, access points and mobile-capable switches, interact with each other to supportprogrammable handoff control and different styles ofQOS adaptation. The MAC layer has also been madeprogrammable.

The following mobile services have been programmedusing the mobiware toolkit [37]:

• QOS-controlled handoff, which supportsautomatic media scaling and error control basedon an adaptive-QOS API and wireless channelconditions;

• mobile soft-state, which provides mobile deviceswith the capability to respond to time varyingQOS through a periodic reservation andrenegotiation process; and

• flow bundling, which supports fast handoff incellular access networks.

The mobiware testbed supports a variety of scalableaudio and video services to mobile devices in additionto traditional web based data services.

4.3 Level of ProgrammabilityThe level of programmability expresses thegranularity at which new services can be introducedinto the network infrastructure. One can consider aspectrum of possible choices from highly dynamic tomore conservative levels of programmability. At oneend of this spectrum, capsules [42] carry code anddata enabling the uncoordinated deployment ofprotocols. Capsules represent the most dynamicmeans of code and service deployment into thenetwork. At the other end of the spectrum there aremore conservative approaches to networkprogrammability based on quasi-static networkprogramming interfaces using RPCs betweendistributed controllers [46] to deploy new services.Between the two extremes lie a number of othermethodologies combining dynamic plug-ins, activemessaging and RPC. Different approaches have adirect bearing on the speed, flexibility, safety,security and performance at which new services canbe introduced into the infrastructure.

4.3.1 Capsules: ANTSANTS [45], developed at MIT, enables theuncoordinated deployment of multiple communicationprotocols in active networks providing a set of coreservices including support for the transportation ofmobile code, loading of code on demand and cachingtechniques. These core services allow networkarchitects to introduce or extend existing networkprotocols. ANTS provides a network programmingenvironment for building new capsule-basedprogrammable network architectures. Examples ofsuch programmed network services include enhancedmulticast services, mobile IP routing and application-level filtering. The ANTS capsule-driven executionmodel provides a foundation for maximum network

programmability in comparison to other APIapproaches. Capsules serve as atomic units ofnetwork programmability supporting processing andforwarding interfaces. Incorporated features includenode access, capsule manipulation, control operationsand soft-state storage services on IP routers. Activenodes execute capsules and forwarding routines,maintain local state and support code distributionservices for automating the deployment of newservices. The ANTS toolkit also supports capsuleprocessing quanta as a metric for node resourcemanagement.

4.3.2 Active Extensions: SwitchwareSwitchware [3], being developed at University ofPennsylvania, attempts to balance the flexibility of aprogrammable network against the safety andsecurity requirements needed in a sharedinfrastructure such as the Internet. The Switchwaretoolkit allows the network architects to trade-offflexibility, safety, security, performance and usabilitywhen programming secure network architectures. Atthe operating system level, an active IP-routercomponent is responsible for providing a securefoundation that guarantees system integrity. Activeextensions can be dynamically loaded into secureactive routers through a set of security mechanismsthat include encryption, authentication and programverification. The correct behavior of active extensionscan be verified off-line by applying ‘heavyweight’methods, since the deployment of such extensions isdone over slow time scales.

Active extensions provide interfaces for moredynamic network programming using active packets.Active packets can roam and customize the networkin a similar way as capsules do. Active packets arewritten in functional languages (e.g., Caml andPLAN [28]) and carry lightweight programs thatinvoke node-resident service routines supported byactive extensions. There is much less requirement fortesting and verification in the case of active packetsthan for active extensions, given the confidence thatlower level security checks have already been appliedto active extensions. Active packets cannot explicitlyleave state behind at nodes and they can access stateonly through clearly defined interfaces furnished byactive extension software. Switchware appliesheavyweight security checks on active extensions,

which may represent major releases of switch code,and more lightweight security checks on activepackets. This approach allows the network architectto balance security concerns against performancerequirements. The security model of Switchwareconsiders public, authenticated and verified facilities.

4.3.3 Composition Languages: CANEsCapsules, active messages and active extensionspromote the creation of new services through thecomposition of new building blocks or by addingcomponents to existing services. The CANEs projectled by researchers at University of Kentucky andGeorgia Tech. aim to define and apply servicecomposition rules as a general model for networkprogrammability [14]. A composition method is usedto construct composite network services fromcomponents. A composition method is specified as aprogramming language with enhanced languagecapabilities that operates on components to constructprogrammable network services. Attributes of a goodcomposition method include high performance,scalability, security and ease of management.Features of well-structured composition methodscombine:

• control on the sequence in which components areexecuted;

• control on shared data among components;

• binding times, which comprise compositecreation and execution times;

• invocation methods, which are defined as eventsthat cause a composite to be executed; and

• division of functionality among multiplecomponents, which may either reside at an activenode or be carried by packets.

PLAN, ANTS and Netscript [21] (described inSection 4.4.2) are examples of composition methods.LIANE is proposed within the CANEs project as acomposition method that incorporates all theaforementioned features. The key idea of LIANE isthat services are composed from basic underlyingprograms that contain processing slots. Users insertprograms for customization in these slots. TheCANEs definition of service compositionencompasses the Opensig approach to networkprogrammability indicating how different approaches

to programmable networking complement each otherby addressing the same goal from differentperspectives.

4.3.4 Network APIs: xbindThe xbind broadband kernel is based on a bindingarchitecture and a collection of node interfacesreferred to as Binding Interface Base (BIB) [2]. TheBIB provides abstractions to the node state andnetwork resources. Binding algorithms run on top ofthe BIB and bind QOS requirements to networkresources via abstractions. The BIB is designed tosupport service creation through high-levelprogramming languages. The interfaces are staticwhile supporting universal programmability. Thequasi-static nature of the BIB interfaces, allow forcomplete testing and verification of the correctness ofnew functions, on emulation platforms, before anyservice is deployed. The concept of active packets orcapsules containing both programs and user data isnot considered in the xbind approach toprogrammability. Rather, communication isperformed using RPCs between distributed objectsand controllers based on OMG’s CORBA. Theapproach taken by xbind promotes interoperabilitybetween multi-vendor switch market supportingresource sharing and partitioning in a controlledmanner.

4.4 Programmable CommunicationsAbstractionsAbstractions and partitioning of resources areessential concepts in programmable networking.Programmable communications abstractions mayrange from node resources to complete programmablevirtual networks. Other programmablecommunications abstractions include programmablevirtual routers, virtual links and mobile channels.Abstracting the network infrastructure throughvirtualization and making it programmable is a majorcontribution of the field that encompasses a numberof different projects.

4.4.1 Active Node Abstractions: NodeOSMembers of the DARPA active network program[20] are developing an architectural framework foractive networking [11]. A node operating systemcalled NodeOS [40] represents the lowest level of the

framework. NodeOS provides node kernel interfacesat routers utilized by multiple executionenvironments, which support communicationabstractions such as threads, channels and flows.Development of an execution environment is anontrivial task and it is anticipated [12] that the totalnumber of execution environments will not be large.Encapsulation techniques based on an active networkencapsulation protocol (ANEP) [5] support thedeployment of multiple execution environmentswithin a single active node. ANEP defines anencapsulation format allowing packets to be routedthrough multiple execution environments coexistingon the same physical nodes. Portability of executionenvironments across different types of physical nodesis accomplished by the NodeOS, by exposing acommon, standard interface. This interface definesfour programmable node abstractions: threads,memory, channels and flows. Threads, memory andchannels abstract computation, storage, andcommunication capacity used by executionenvironments, whereas flows abstract user data-pathswith security, authentication and admission controlfacilities. An execution environment uses the NodeOSinterface to create threads and associate channelswith flows. The NodeOS supports QOS usingscheduling mechanisms that regulate the access tonode computation and communication resources. Thearchitectural framework for active networking isbeing implemented in the ABONE testbed [1]allowing researchers to prototype new activearchitectures.

4.4.2 Virtual Active Networks: NetscriptThe Netscript project [49] at Columbia Universitytakes a functional language-based approach tocapture network programmability using universallanguage abstractions. Netscript is a strongly typedlanguage that creates universal abstractions forprogramming network node functions. Unlike otheractive network projects that take a language-basedapproach Netscript is being developed to supportVirtual Active Networks as a programmableabstraction. Virtual Active Network [21] abstractionscan be systematically composed, provisioned andmanaged. In addition, Netscript automatesmanagement through language extensions thatgenerate MIBs. Netscript leverages earlier work ondecentralized management and agent technologies

that automatically correlate and analyze the behaviormonitored by active MIB elements. A distinguishingfeature of Netscript is that it seeks to provide auniversal language for active networks in a mannerthat is analogous to postscript. Just as postscriptcaptures the programmability of printer engines,Netscript captures the programmability of networknode functions. Netscript communication abstractionsinclude collections of nodes and virtual links thatconstitute virtual active networks.

4.4.3 Virtual ATM Networks: TempestThe Tempest project at the University of Cambridge[34] has investigated the deployment of multiplecoexisting control architectures in broadband ATMenvironments. Novel technological approachesinclude the usage of software mobile agents tocustomize network control and the consideration ofcontrol architectures dedicated to a single service.Tempest supports two levels of programmability andabstraction. First, switchlets, which are logicalnetwork elements that result from the partition ofATM switch resources, allow the introduction ofalternative control architectures into an operationalnetwork. Second, services can be refined bydynamically loading programs that customizeexisting control architectures. Resources in an ATMnetwork can be divided by using two softwarecomponents: a switch control interface called arieland a resource divider called prospero. Prosperocommunicates with an ariel server on an ATMswitch, partitions the resources and exports aseparate control interface for each switchlet created.A network builder creates, modifies and maintainscontrol architectures.

4.5 Architectural DomainsMost programmable network projects are related tothe introduction of services into networks. However,most projects are targeted to a particulararchitectural domain (e.g., QOS control, signaling,management, transport and applications). In whatfollows we discuss three projects that address theapplication, resource management and networkmanagement domains.

4.5.1 Application-Level: Active ServicesIn contrast to the main body of research in activenetworking, Amir et al. [4] call for the preservationof all routing and forwarding semantics of theInternet architecture by restricting the computationmodel to the application layer. The Active Servicesversion 1 (AS1) programmable service architectureenables clients to download and run service agents atstrategic locations inside the network. Service agentscalled “servents” are restricted from manipulatingrouting tables and forwarding functions that wouldcontravene the IP-layer integrity. The AS1architecture contains a number of architecturalcomponents:

• a service environment, which defines aprogramming model and a set of interfacesavailable to servents;

• a service-location facility, which allows clients to‘rendezvous’ with the AS1 environment byobtaining bootstrapping and configurationmechanisms to instantiate servents5;

• a service management system, which allocatesclusters of resources to servents using admissioncontrol and load balancing of servents underhigh-load conditions;

• a service control system, which provides dynamicclient control of servents once instantiated withinan AS1 architecture;

• a service attachment facility, which providesmechanisms for clients that can not interactdirectly with the AS1 environment through soft-state gateways; and

• a service composition mechanism, which allowsclients to contact multiple service clusters andinterconnect servents running within and acrossclusters.

The AS1 architecture is programmable at theapplication layer supporting a range of application

5 Servents are launched into the network by an active service

control protocol (ASCP), which includes an announce-listenprotocol for servers to manage session state consistency, soft-state to manage expiration due to timeouts and multicastdamping to avoid flooding the environment with excessiveservents.

domains. In [4], the MeGa architecture isprogrammed using AS1 to support an active mediagateway service. In this case, servents providesupport for application-level rate control andtranscoding techniques.

4.5.2 Resource Management: DarwinThe Darwin Project [17] at Carnegie MellonUniversity is developing a middleware environmentfor the next generation IP networks with the goal ofoffering Internet users a platform for value-added andcustomizable services. The Darwin project is focusedtoward customizable resource management thatsupports QOS. Architecturally, the Darwinframework includes Xena, a service broker that mapsuser requirements to a set of local resources, resourcemanagers that communicate with Xena using theBeagle signaling protocol, and hierarchicalscheduling disciplines based on service profiles. TheXena architecture takes the view that the IPforwarding and routing functions should be left intact and only allows restricted use of active packettechnology in the system.

Alongside the IP stack, Darwin introduces a controlplane that builds on similar concepts such as thoseleveraged by broadband kernels [30] and activeservices [4]. The Xena architecture is madeprogrammable and incorporates active technologiesin a restricted fashion. A set of service delegatesprovides support for active packets. Delegates can bedynamically injected into IP routers or servers tosupport application specific processing (e.g.,sophisticated semantic dropping) and value-addedservices (e.g., transcoders). A distinguishing featureof the Darwin architectural approach is thatmechanisms can be customized according to userspecific service needs defined by space, organizationand time constraints. While these architecturalmechanisms are most effective when they work inunison each mechanism can also be combined withtraditional QOS architecture components. Forexample, the Beagle signaling system could beprogrammed to support RSVP signaling for resourcereservation, while the Xena resource brokers andhierarchical schedulers could support traffic control.

4.5.3 Network Management: Smart PacketsThe Smart Packets Project [41] (not to be confusedwith University of Kansas smart packets) at BBNaims to improve the performance of large andcomplex networks by leveraging active networkingtechnology. Smart Packets are used to movemanagement decision making points closer to thenodes being managed, target specific aspects of thenode for management and abstract managementconcepts to language constructs. Management centerscan send programs to managed nodes. Thus themanagement process can be tailored to the specificinterests of the management center reducing theamount of back traffic and data requiringexamination. A smart packet consists of a header andpayload encapsulated using ANEP [5]. Smart packetsmay carry programs to be executed, results fromexecution, informational messages or reports on errorconditions. Smart Packets are written in twoprogramming languages:

• sprocket, which is a high-level C-like, languagewith security threatening constructs, and

• spanner, which is a low-level assembly-likelanguage, that can result in tighter, optimizedcode.

Sprocket programs are compiled into spanner code,which in turn is assembled into a machine-independent binary encoding placed into smartpackets. Meaningful programs perform networkingfunctions and MIB information retrieval.

5. DISCUSSIONWe have introduced a set of characteristics and ageneralized model for programmable networks tohelp understand and differentiate the diverse set ofprogrammable network projects discussed in thispaper. In what follows we provide a brief comparisonof these projects and other work in the field.

5.1 ComparisonIn this section we present a simple qualitativecomparison of the programmable networks surveyedin Section 4. Table 1 presents the comparison withrespect to the characteristics and generalized modelfor programmable networks presented in Section 3and 4, respectively.

5.2 Open Programmable InterfacesThe use of open programmable network interfaces isevident in many programmable network projectsdiscussed in this survey. Open interfaces provide afoundation for service programming and theintroduction of new network architectures.

The xbind broadband kernel supports acomprehensive Binding Interface Base usingCORBA/IDL to abstract network ATM devices, stateand control. A number of other projects focussed onprogramming IP networks (e.g., ANTS, Switchware,CANEs) promote the use of open APIs that abstractnode primitives, enabling network programmabilityand the composition of new services. Many networkprogramming environments shown in Table 1 takefundamentally different approaches to providing openinterfaces for service composition. The programmingmethodology adopted (e.g., distributed objecttechnology based on RPC, mobile code or hybridapproaches) has a significant impact on anarchitecture’s level of programmability; that is, thegranularity, time scales and complexity incurredwhen introducing new APIs and algorithms into thenetwork.

Two counter proposals include the xbind and ANTSAPIs. While the ANTS approach to the deploymentof new APIs in extremely flexible presenting a highlydynamic programming methodology it represents acomplex programming model in comparison to thesimple RPC model. In contrast, the xbind bindinginterfaces and programming paradigm is based on aset of CORBA IDL and RPC mechanisms. Incomparison to capsule-based programmability thexbind approach is rather static in nature and theprogramming model less complex. These approachesrepresent two extremes of network programmability.

One could argue that quasi-static APIs based on RPCis a limited and restrictive approach. A counterargument is that the process of introducing andmanaging APIs is less complex than the capsule-based programming paradigm, representing a moremanageable mechanism for service composition andservice control. Similarly one could argue that activemessage and capsule-based technologies are more‘open’ because of the inherent flexibility of theirnetwork programming models given that capsules can

graft new APIs onto routers at runtime. The xbindapproach lacks this dynamic nature at the cost of asimplified programming environment. Other projectsadopt hybrid approaches. For example the mobiwaretoolkit combines the static APIs with the dynamicintroduction of Java service plug-ins when needed[7]. A clear movement of the field is to open up thenetworks and present APIs for programming newarchitectures, services and protocols. As we discussin the next section the field is arguing that theswitches, routers and base stations should open upultimately calling for open APIs everywhere.

5.3 Virtualization and Resource Partitioning

Many projects use virtualization techniques tosupport the programmability of different types ofcommunication abstractions. The Tempestframework [33] presents a good example of the useof virtualization of the network infrastructure. Low-level physical switch interfaces are abstractedcreating sets of interfaces to switch partitions calledswitchlets. Switchlets allow multiple controlarchitectures to coexist and share the same physicalswitch resources (e.g., capacity, switching tables,name space, etc.). Typically, abstractions found inprogrammable networks are paired with safe resourcepartitioning strategies that enable multiple services,protocols and different programmable networkingarchitectures to coexist. Virtualization of the networkin this manner presents new levels of innovation inprogrammable networks that have not beenconsidered before. All types of network componentscan be virtualized and made programmable fromswitches and links [15] to switchlets [33], activenodes [40], routelets [13] and virtual networks [21],[34], [13].

The NodeOS interface [40] provides a similarabstraction to node resources. The use of openinterfaces allows multiple network programmingenvironments (or execution environments using activenetworking terminology) to coexist within a commonphysical node architecture. In this case, the ANEP [5]protocol provides encapsulation as a mechanism fordelivering packets to distinct execution environments.

Table 1: Comparison of Programmable Networks

Using encapsulation in this manner allows fordifferent overlay execution environments (e.g.,ANTS, Switchware, or Netscript) to execute on thesame router using a single, common node kernel. Thenotion of virtualization is not a new concept,however. Similar motivation in the Internetcommunity has led to the advent of the Mbone. Newdirections in the virtualization of the Internet haveprompted the proposal for X-bone [44], shown inTable 1, which will provide a network programmingenvironment capable of dynamically deployingoverlay networks. As Table 1 illustrates, otherprojects such as Supranet [23] advocate tunnelingand encapsulation techniques for the separation andprivacy among coexisting, collaborativeenvironments.

5.4 Programmable Virtual NetworkingThe dynamic composition and deployment of newservices can be extended to include the compositionof complete network architectures as virtualnetworks. The Netscript project [49] supports thenotion of Virtual Active Networks [21] over IPnetworks. Virtual network engines interconnect setsof virtual nodes and virtual links to form virtualactive networks. The Tempest framework [34]supports the notion of virtual networks using safepartitioning over ATM hardware. Tempest offers twolevels of programmability. First, network controlarchitectures can be introduced over long time scalesthrough a ‘heavyweight’ deployment process.Second, ‘lightweight’ application-specificcustomization of established control architecturestake place over faster time scales. The abstraction ofphysical switch partitions within the Tempestframework has led to the implementation of multiplecoexisting control architectures. The Tempeststrategy aims to address QOS through connection-oriented ATM technology and investigates physicalresource sharing techniques between alternativecontrol architectures. Both Darwin [17] and Netscript[49] projects support the notion of sharing theunderlying physical infrastructure in a customizedway as well. As discussed in the previous section, theNodeOS [40] project also provides facilities forcoexisting execution environments.

5.5 Spawning NetworksIn [13] we describe spawning networks, a new classof programmable networks that automate the

creation, deployment and management of distinctnetwork architectures “on-the-fly”. The term“spawning” finds a parallel with an operating systemspawning a child process, typically operating over thesame hardware. We envision programmable networksas having the capability to spawn not processes butcomplex network architectures [31]. The enablingtechnology behind spawning is the Genesis Kernel[13], a virtual network operating system thatrepresents a next-generation approach to thedevelopment of network programming environments.A key capability of Genesis is its ability to support avirtual network life cycle process for the creation anddeployment of virtual networks through:

• profiling, which captures the “blueprint” of avirtual network architecture in terms of acomprehensive profiling script;

• spawning, which executes the profiling scriptto set-up network topology, and address spaceand bind transport control and managementobjects into the physical infrastructure; and

• management, which supports virtual networkarchitecting and resource management.

Virtual networks, spawned by the Genesis Kerneloperate in isolation with their traffic being carriedsecurely and independently from other networks.Furthermore, “child” networks, created throughspawning by “parent” networks inherit architecturalcomponents from their parent networks, including lifecycle support. Thus a child virtual network can be aparent (i.e., provider) to its own child networks,creating a notion of “nested virtual networks” withina virtual network.

6. CONCLUSIONIn this paper, we have discussed the state-of-the-artin programmable networks. We have presented a setof characteristics and generalized model forprogrammable networks, which has allowed us tobetter understand the relationship between theexisting body of work on programmable networking.The generalized model comprises communication andcomputation models. By “grafting” a computationmodel to the communication model a networkarchitecture can be made programmable. Thegeneralized model includes node kernels to managenetwork node resources, and network programming

environments that provide tools for programmingnetwork architectures.

We believe that a number of important innovationsare creating a paradigm shift in networking leading tohigher levels of network programmability. These are:

• separation of hardware from software;

• availability of open programmable interfaces;

• virtualization of the networking infrastructure;

• rapid creation and deployment of new networkservices; and

• safe resource partitioning and coexistence ofdistinct network architectures over the samephysical networking hardware.

Programmable networks provide a foundation forarchitecting, composing and deploying virtualnetwork architectures through the availability of openprogrammable interfaces, resource partitioning andthe virtualization of the networking infrastructure.We believe that a key challenge is the development ofprogrammable virtual networking environmentsbased on these foundations.

7. ACKNOWLEDGEMENTSThis work is supported in part by the NationalScience Foundation (NSF) under CAREER AwardANI-9876299 and with support from COMETGroup industrial sponsors. In particular, we wouldlike to thank the Intel Corporation, Hitachi Limitedand Nortel Networks for supporting the GenesisProject. John B. Vicente (Intel Corp) would like tothank the Intel Research Council for their supportduring his visit with the Center forTelecommunications Research, Columbia University.Miki Kazuho (Hitachi, Ltd) would like to express histhanks to Hitachi Ltd for their support of his work onProgrammable Networks at Columbia University.Hermann G. De Meer is grateful to DeutscheForschungsgemeinschaft (DFG) for providing hisfellowship and research grant Me 1703/2-1. DanielA. Villela would like to thank the National Councilfor Scientific and Technological Development(CNPq-Brazil) for sponsoring his scholarship atColumbia University (ref. 200168/98-3).

8. REFERENCES[1] ABONE, Active network Backbone,

http://www.csl.sri.com/ancors/abone/[2] Adam, C.M., Lazar, A.A., Lim, K.-S., and

Marconcini, F., “The Binding Interface BaseSpecification Revision 2.0”, OPENSIGWorkshop on Open Signalling for ATM,Internet and Mobile Networks, Cambridge, UK,April 1997.

[3] Alexander, D.S., Arbaugh, W.A., Hicks, M.A.,Kakkar P., Keromytis A., Moore J.T., NettlesS.M., and Smith J.M., “The SwitchWare ActiveNetwork Architecture”, IEEE Network SpecialIssue on Active and Controllable Networks,vol. 12 no. 3, 1998.

[4] Amir E., McCanne S., and Katz R., “An ActiveService Framework and its Application to real-time Multimedia Transcoding”, ProceedingsACM SIGCOMM’ 98, Vancouver, Canada

[5] Alexander D.S., Braden B., Gunter C.A.,Jackson W.A., Keromytis A.D., Milden G.A.,and Wetherall D.A., “Active NetworkEncapsulation Protocol (ANEP)”, ActiveNetworks Group Draft, July 1997

[6] Angin, O., Campbell, A.T., Kounavis, M.E.,and Liao, R.R.-F., “The Mobiware Toolkit:Programmable Support for Adaptive MobileNetworking”, IEEE Personal CommunicationsMagazine, Special Issue on Adaptive MobileSystems, August 1998.

[7] Balachandran, A., Campbell, A.T., andKounavis, M.E, “Active Filters: DeliveringScalable Media to Mobile Devices” , Proc.Seventh International Workshop on Networkand Operating System Support for DigitalAudio and Video, St Louis, May, 1997.

[8] Bershad,B.N., et al., “Extensibility, Safety andPerformance in the SPIN Operating System”,Fifth ACM Symposium on Operating SystemsPrinciples, Copper Mountain, December 1995.

[9] Biswas, J., et al., " The IEEE P1520 StandardsInitiative for Programmable NetworkInterfaces” IEEE Communications Magazine,Special Issue on Programmable Networks,October, 1998.

[10] Braden,B., “Active Signaling Protocols”, ActiveNetworks Workshop, Tucson AZ, March 1998.

[11] Calvert, K. et al, “Architectural Framework forActive Networks”, Active Networks WorkingGroup Draft, July 1998.

[12] Calvert, K. et. al, “Directions in Activenetworks”, IEEE Communications Magazine,Special Issue on Programmable Networks,October 1998.

[13] Campbell A.T., De Meer H.G., Kounavis M.E.,Miki K., Vicente J.B., and Villela D., “TheGenesis Kernel: A Virtual Network OperatingSystem for Spawning Network Architectures”,Second International Conference on OpenArchitectures and Network Programming(OPENARCH), New York, 1999.

[14] "CANEs: Composable Active NetworkElements", http://www.cc.gatech.edu/projects/canes/

[15] Chan, M.-C., Huard, J.-F., Lazar, A.A., andLim, K.-S., “On Realizing a Broadband Kernelfor Multimedia Networks”, 3rd COST 237Workshop on Multimedia Telecommunicationsand Applications, Barcelona, Spain, November25-27, 1996.

[16] Chen and Jackson, Editorial, IEEE NetworkMagazine, Special Issue on Programmable andActive Networks, May 1998

[17] Chandra, P. et al., “Darwin: CustomizableResource Management for Value-addedNetwork Services”, Sixth IEEE InternationalConference on Network Protocols (ICNP'98),Austin, October 1998.

[18] Coulson, G., et al., “The Design of a QOS-Controlled ATM-Based CommunicationsSystem in Chorus”, IEEE Journal of SelectedAreas in Communications, vol.13, no.4, May1995.

[19] Cplane Inc., www.cplane.com[20] DARPA Active Network Program,

http://www.darpa.mil/ito/research/anets/projects.html, 1996.

[21] Da Silva, S., Florissi, D. and Yemini, Y.,“NetScript: A Language-Based Approach toActive Networks”, Technical Report, ComputerScience Dept., Columbia University January27, 1998.

[22] Decasper, D., Parulkar, G., Plattner, B., “AScalable, High Performance Active NetworkNode”, IEEE Network, January 1999.

[23] Delgrossi, L. and Ferrari D., “A VirtualNetwork Service for Integrated-ServicesInternetworks”, 7th International Workshop onNetwork and Operating System Support forDigital Audio and Video, St. Louis, May 1997.

[24] Engler, D.R., Kaashoek, M.F. and O’Toole ,J.,“Exokernel: An Operating System Architecturefor Application-Level Resource Management”,Fifth ACM Symposium on Operating SystemsPrinciples, Copper Mountain, December 1995.

[25] Feldmeier, D.C., at al. “Protocol Boosters”,IEEE Journal on Selected Areas inCommunications, Special Issue on ProtocolArchitectures for the 21st Century, 1998.

[26] Ferguson, P. and Huston, G., “What is aVPN?”, OPENSIG'98 Workshop on OpenSignalling for ATM, Internet and MobileNetworks, Toronto, October 1998.

[27] Hartman, J., et al., “Liquid Software: A NewParadigm for Networked Systems”, TechnicalReport 96-11, Dept. of Computer Science,Univ. of Arizona, 1996.

[28] Hicks, M., et al., “PLAN: A ProgrammingLanguage for Active Networks”, Proc ICFP'98,1998.

[29] Kulkarni, A.B. Minden G.J., Hill, R., Wijata,Y., Gopinath, A., Sheth, S., Wahhab, F., Pindi,H., and Nagarajan, A., “Implementation of aPrototype Active Network”, First InternationalConference on Open Architectures andNetwork Programming (OPENARCH), SanFrancisco, 1998.

[30] Lazar, A.A.,“Programming TelecommunicationNetworks”, IEEE Network, vol.11, no.5,September/October 1997.

[31] Lazar, A.A., and A.T Campbell, “SpawningNetwork Architectures”, Technical Report,Center for Telecommunications Research,Columbia University, 1997.

[32] Liao, R.-F. and Campbell, A.T., “OnProgrammable Universal Mobile Channels in aCellular Internet”, 4th ACM/IEEE InternationalConference on Mobile Computing and

Networking (MOBICOM'98) , Dallas, October,1998

[33] Van der Merwe, J.E., and Leslie, I.M.,“Switchlets and Dynamic Virtual ATMNetworks”, Proc Integrated NetworkManagement V, May 1997.

[34] Van der Merwe, J.E., Rooney, S., Leslie, I.M.and Crosby, S.A., “The Tempest - A PracticalFramework for Network Programmability”,IEEE Network, November 1997.

[35] DARPA Active Network Mail List Archives,1996. http://www.ittc.ukans.edu/Projects/Activenets

[36] Montz, A.B., et al., “Scout: A Communications-Oriented Operating System”, Technical Report94-20, University of Arizona, Dept. ofComputer Science, June 1994.

[37] Mobiware Toolkit v1.0 source code distributionhttp://www.comet.columbia.edu/ mobiware

[38] Multiservice Switching Forum (MSF) ,www.msforum.org

[39] Open Signaling Working Groupcomet.columbia.edu/opensig/

[40] Peterson L., “NodeOS Interface Specification”,Technical Report, Active Networks NodeOSWorking Group, February 2, 1999

[41] Schwartz, B., Jackson, W.A., Strayer W.T.,Zhou, W., Rockwell, R.D., and Partridge, C.,"Smart Packets for Active Networks”, SecondInternational Conference on OpenArchitectures and Network Programming(OPENARCH), New York, 1999.

[42] Tennenhouse, D., and Wetherall, D., “Towardsan Active Network Architecture”, Proceedings,Multimedia Computing and Networking, SanJose, CA, 1996.

[43] Tennenhouse, D., et al., “A Survey of ActiveNetwork Research”, IEEE CommunicationsMagazine, January 1997.

[44] Touch, J. and Hotz, S., "The X-Bone", ThirdGlobal Internet Mini-Conference inconjunction with Globecom '98 Sydney,Australia, November 1998.

[45] Wetherall, D., Guttag, J. and Tennenhouse, D.,“ANTS: A Toolkit for Building andDynamically Deploying Network Protocols”,

Proc. IEEE OPENARCH'98, San Francisco,CA, April 1998.

[46] Vinoski, S.,“CORBA: Integrating DiverseApplications Within Distributed HeterogeneousEnvironments”, IEEE CommunicationsMagazine, Vol. 14, No. 2, February, 1997.

[47] xbind code http://comet.columbia.edu/xbind[48] Xbind Inc., www.xbind.com[49] Yemini, Y., and Da Silva, S, "Towards

Programmable Networks", IFIP/IEEEInternational Workshop on DistributedSystems: Operations and Management,L'Aquila, Italy, October, 1996.