A study of encoding overhead in network management protocols

INTERNATIONAL JOURNAL OF NETWORK MANAGEMENTInt. J. Network Mgmt (2012)Published online in Wiley Online Library (wileyonlinelibrary.com) DOI: 10.1002/nem.1801

A study of encoding overhead in network management protocols

Pedro Gonçalves,1,*† José Luís Oliveira2 and Rui Aguiar2

1ESTGA/IT, Universidade de Aveiro, Campus Universitário de Santiago, Aveiro, Portugal2DETI/IT, Universidade de Aveiro, Campus Universitário de Santiago, Aveiro, Portugal

SUMMARY

Next-Generation Network (NGN) is a critical scenario in terms of network management because of itsnetwork dimension, its number of users and its heterogeneity. Since the introduction of the Simple NetworkManagement Protocol (SNMP) at the beginning of the 1990s, much effort has been devoted to thedevelopment of new network management technologies. Both the Desktop Management Task Force(DMTF) and the Internet Engineering Task Force (IETF) have developed different network and systemmanagement protocols, such as Common Open Policy Service, Web-Based Enterprise Management,Network Configuration and even adapted other protocols, such as Diameter and Web Services. A networkmanagement technology with poor scalability could compromise NGN management and ultimately NGNnetwork behaviour. This paper analyses the network overhead of several management technologiesdeveloped by the DMTF and IETF, and goes on to compare their results with the usage of SNMP.Furthermore, some deployment recommendations are proposed for performance optimization in NGNs.Copyright © 2012 John Wiley & Sons, Ltd.

Received 6 July 2011; Revised 16 December 2011; Accepted 18 December 2011

1. INTRODUCTION

Network management requirements have changed a lot since the introduction of the Simple NetworkManagement Protocol (SNMP) in the early 1990s. There was an enormous increase in network traffic,in the number of users and services as well as a remarkable increase of diversity in network elements.The simplicity of SNMP made it an omnipresent management solution, present from the simplestnetwork printer to network elements. Its weaknesses limited its usage to monitoring actions and madethe majority of vendors develop customized configuration support.During these two decades, great effort has been devoted to the development of new management

technologies, mainly inside two standardization bodies: the Internet Engineering Task Force (IETF)and the Desktop Management Task Force (DMTF).The DMTF standardized a web-based management technology, named Web-Based Enterprise

Management (WBEM), aiming to integrate system and network management. WBEM technologyuses an information model previously standardized by the DMTF and makes wide use of popularweb technologies, such as XML for information encoding and HTTP as a transport protocol. WBEMsolutions have been shown to require considerable computational resources from the entity involved inthe management process and to require huge amounts of signalling. Recently, the DMTF has beenstandardizing and promoting a new management technology based on Web Services (WS). WSdistributed systems allow enormous interoperability and easy development. Although WS technology

*Correspondence to: Pedro Gonçalves, ESTGA/IT, Universidade de Aveiro, Campus Universitário de Santiago, Aveiro,Portugal.†E-mail: [email protected]

Copyright © 2012 John Wiley & Sons, Ltd.

P. GONÇALVES, J. L. OLIVEIRA AND R. AGUIAR

wastes fewer resources than WBEM [1], network management developers have been uncomfortablewith the technological verbosity in the network management processes.The Common Open Policy Service (COPS) protocol resulted from an attempt by the IETF to

overcome some SNMP drawbacks. Although COPS represents a significant conceptual change fromSNMP, industry considered it just one step ahead in a new management technology. The IETFacknowledged that COPS technology could be criticized, and has been standardizing a new networkmanagement technology based on XML, named Network Configuration (NETCONF). TheNETCONF protocol [2] uses Simple Object Access Protocol (SOAP) Remote Procedure Call (RPC)messages and offers several options for transport such as HTTP, Secure Shell (SSH), BlocksExtensible Exchange Protocol (BEEP) and more recently Transport Layer Security (TLS).In recent years, Next-Generation Networks (NGNs) have appeared as a new network architecture in

the standardization process by several standardization bodies like the International TelecommunicationUnion Telecommunication Standardization Sector (ITU-T) [3], 3rd Generation Partnership Project(3GPP) [4], European Telecommunications Standards Institute (ETSI) [5] or the Broadband Forum [6].At the core of these architectures is a component named IP Multimedia Subsystem (IMS) [4]developed by 3GPP with the purpose of delivering multimedia services to IP-based mobile networks.IMS decouples the service from network management and is completely agnostic in terms of accessnetwork technology. The Diameter protocol has wide usage in the IMS network event in management-related tasks: in the user registration process, in the charging process, and since IMS release 7 it has alsobeen used as a policy transport protocol [7].Nevertheless, in terms of network management, the architecture is still under definition. For

instance, the 3GPP consortium did not fully define the NGN management architecture, but just defineda generic management architecture based on the Telecommunications Management Network (TMN)approach [8].The management technology has a strong influence on the performance of managed elements,

especially on network elements. Characteristics such as message encoding efficiency, the memoryrequirements from the management applications and the overhead caused by security mechanisms[9] are details that have major implications on the network scalability and performance. This workfocuses on the comparison of the messaging encoding efficiency of the management technologies.NGNs require a scalable management technology because of their dimensions, the large number of

users and heterogeneity of network equipment. Both Diameter and NETCONF seem strong candidatesfor NGN network management protocols. Diameter-based interfaces are already widely used in thecommunication of the NGN network entities, but for operational tasks. Its usage could be extended formanagement purposes, and thus reusing the interface. On the other hand, NETCONF is a new networkmanagement protocol that has been developed inside IETF, and IMS reuses several IETF technologies.In this paper, we present a comparison of the signalling generated by different management

protocols standardized by IETF (SNMP, COPS, Diameter and NETCONF) and by the DMTF(CIM/XML over HTTP and YIN over SOAP). These results should be useful for selection ofmanagement solutions for environments like IMS-based NGN on the wireless mesh networks. Thispaper is organized as follows. Section 2 describes related work conducted so far, and Section 3 givesa general overview of several management technologies selected. Section 4 analyses the encodingperformed by these technologies. In Section 5 we describe the test scenario and in Section 6 wediscuss the result of the evaluation carried out. Section 7 lists some deployment recommendations,and conclusions are finally provided in Section 8.

2. RELATED WORKS

Much work has been done in the area of performance evaluation, particularly with SNMP. Andreyet al. [10] surveyed the SNMP-related performance studies performed over the last 10 years. Theydiscovered that those studies used different techniques and scenarios and addressed different metrics.In the paper they propose techniques, approaches and metrics to be followed in order to reach abenchmarking framework that would allow quantifying the performance of SNMP-based applicationand reuse of the performance values obtained in future works.

Copyright © 2012 John Wiley & Sons, Ltd. Int. J. Network Mgmt (2012)DOI: 10.1002/nem

ENCODING OVERHEAD IN NETWORK MANAGEMENT PROTOCOLS

Schönwälder et al. carried out an SNMP traffic analysis [11]. They verified that the most usedversions of SNMP are SNMPv1 and SNMPv2. Moreover, they identified the most frequent messagesin real SNMP environments.Moura et al. [12] presented a performance evaluation of the Web Service for management

applications, and they observed a performance gain of the DMTF standard over its OASIS equivalent.Neisse et al. [13] describe the implementation of an SNMP to WS gateway and they evaluatedthe bandwidth consumption of these different technologies. Pras et al. [9] methodically analyseSNMP message encoding and the signalling produced. They compared the signalling volume, thecomputation resources and the time-to-relay of SNMP and WS. Additionally they studied thecompression effect over the signalling volume. Pavlou et al. [14] performed a performance evaluationof SNMP, CORBA and WS technologies. They studied the memory requirements, time-to-reply andsignalling volume. Lima et al. [15] compare the SNMP and WS as notification technologies. Theirpaper analyses network usage and the time-to-reply of the management technologies. Furthermore,it proposed and evaluated an SNMP to WS gateway that responds to the network element traps andforwards the monitoring information to a management server in the form of a WS notification.Yoo et al. [16] carried out a performance evaluation of the NETCONF protocol with the various

transport options at the time. They also proposed methods for improving the performance ofNETCONF-based management solutions. Franco et al. [17] compared the performance of theNETCONF, COPS-PR and SOAP protocols. They found that although NETCONF and SOAPproducemore signalling than COPS-PR, they could use compression techniques in order to compensate.Gonçalves et al. [1] evaluated the performance of several management protocols in a well-

defined configuration management scenario. The study considered signalling volume and memoryrequirements. A general gain in performance of the binary-based technologies over XML-based wasobserved. Gonçalves et al. [18] evaluated the performance effect of replacement of the policy protocolin an NGN Policy and Charging Control (PCC) architecture [7]. It was observed that the new NGNarchitecture release added considerable complexity to the interface by means of adding charging-related information, and a considerable volume of extra signalling caused by the Diameter interface.The performance comparison studies mentioned above present comparisons of two or at most three

technologies, but do not allow a clear choice of themost appropriate technology for network managementdeployment. Moreover, since there is no uniform test scenario, it is not possible to correlate results of thedifferent studies.

3. TECHNOLOGY REVIEW

The current management landscape is populated with a multiplicity of protocols, initially developed asanswers to different requirements. We selected some of the most relevant technologies nowadays, allpotential candidates as management frameworks for NGN, in order to be able to evaluate the trafficcost imposed by management.The SNMP was proposed in 1990 as a simple application layer protocol that implements communi-

cation between a management console station and managed agents. Currently, SNMP implements sixoperations: for information request (Get, GetNext, GetBulk), one operation for information writing(Set), an operation for event notification (Trap) and the InformRequest introduced in SNMPv2. Theprotocol messages are coded in small-size packets and transported by User Datagram Protocol(UDP)—in order to allow a lightweight message transport in overloaded networks. Version 2 of theprotocol was proposed in RFC 1441–1452 with some enhancements to the SNMPv1 data typesand with the GetBulk message. Version 3 (RFC 2271–2275) was proposed in 1998 with someenhancements in terms of security and remote configuration. The SNMP protocol is widely used todayin network management as well as in the area of equipment management, mainly as a monitoring tool.Despite the fact that the status of SNMPv1 and SNMPv2 is historic and only SNMPv3 is full standard,SNMPv3 is not much used in network management [11].COPS was later proposed by the IETF as a query/response protocol for policy information exchange.

COPS is a binary protocol that transports messages between the COPS manager (designated asPolicy Definition Point (PDP)) and its managed entities, (the Policy Enforcement Points (PEPs)), using



Transmission Control Protocol (TCP). Client and server maintain a COPS connection and they identifyall the messages using a unique handle. Two models of the protocol were proposed: the outsourcing—COPS-RSVP [19]—and the provision model—COPS for Policy Provisioning (COPS-PR) [20].In the outsourcing model, external events in the PEP must be handled by the PDP in a pure query/

response manner. The protocol is typically implemented between a router (PEP) and a server (PDP) foradmission control purposes. The router sends COPS requests (REQ messages) to the server when itreceives network access requests from the network clients; the server, after taking a decision aboutthe access request, answers with decision (DEC) messages.In the provision model, COPS implements event notification (REQ) messages from routers to the

server, and configuration (DEC) messages from the server to the router. The server performs initialconfiguration of the PEPs that will operate according to predefined policies. The COPS client sendsREQ messages to the server when it receives a user request or when an internal event is generatedin the client. The server sends unsolicited DEC messages to the client, although the model does notmake any assumption about the correlation between REQ and DEC messages.All COPS messages have an 8-byte-long packet header. A typical COPS REQ message is 24 bytes

long and a DEC message has the size of the decision object plus 32 bytes. The COPS protocol did notinitially gain much attention from the industry, despite bringing a significant conceptual change.Nowadays, COPS is considered a substandard technology and IETF is not encouraging furtherdevelopments of the technology [17].The Diameter protocol was proposed within the Authentication, Authorization and Accounting

(AAA) framework, as the successor to the RADIUS AAA protocol. The Diameter Base Protocol isthe core model and several extensions tailored for specific applications were also proposed, such asthe Diameter Network Access Server Application (NASREQ), the Diameter mobile IPv4 Application(MobileIP) and the Diameter Session Initiation Protocol. Diameter also started to provide severalextensions for different management requirements.More recently, Diameter gained a prominent position since it was widely adopted inside the 3GPP

IMS platform [7], as the communication protocol for several functional entities. Diameter wasproposed as the protocol for AAA communication, and it was also proposed as the messaging solutionfor QoS negotiation issues [7]. In March 2008, the Diameter Policy Processing Application waspublished—an extension of Diameter as a policy management protocol—enlarging the originalAAA target [21] and replacing COPS as a policy management protocol.Diameter architecture is complex and computational resource consuming. Diameter entities

implement a complex state machine that requires large computational capabilities. The protocolsupports a message routing mechanism that allows Diameter entities to route messages from oneDiameter element to another. Furthermore, making use of capability announcements, Diameterelements have to implement a message adaptation mechanism in order to interconnect Diameterelements that cannot communicate directly.On another front, the WBEM initiative was born in 1996 with sponsorship from several companies.

The goal was to unify desktop management with network management and to create a multi-vendorand multi-platform management framework. The developed technology, WBEM, is based on threeconcepts: it represents the management data in Common Information Model (CIM), it encodes themanagement information in eXtensible Markup Language (XML) and it transports the managementinformation over HTTP. CIM is an object-oriented model that allows representation of managementinformation, as well as the relationships between management entities.WBEM solutions include four components:

• the CIM client typically used by the human operator during management tasks;• a CIM Object Manager (CIMOM) that is the main component of the system maintaining thedialogue with the CIM client and the management information;

• a CIM repository;• CIM providers that perform the interface between the CIM server and its specific managedequipment (such as a managed server or a router).

The definition of a new CIM extension also involves development of the correspondent CIMprovider that will implement the functional logic of the defined objects (configuring, monitoring, etc.).



Providers can be classified as instance (for object representation), indication (notifications), association(to define objects’ relations) or method (applied for the invocation of remote methods). The integrationof underlying management technologies in WBEM is implemented through specific providers, andadapters for common protocols are already available [22].WBEM technology is used mainly in the area of desktop management. Several open source and

commercial implementations based on WBEM technology exist. Typically, each company thatsponsors the open source project commercializes its own WBEM-based management product.WS is a very popular distributed systems technology nowadays. It is a XML technology based on

W3 standards like SOAP and the Web Service Definition Language (WSDL) and is supported in manyplatforms and by many vendors. WS commonly communicate through the exchange of SOAPmessages typically transported in HTTP or by the SMTP protocol. They offer significant interoperabilitybecause of the XML description tags used in the object encoding, and they are language and platformindependent. A Web Service is described in a WSDL document, which defines the operations and itsparameters, the service location, and the implemented protocols for each operation.Two Web Service initiatives were promoted for management: Web Services for Management

(WS-Management) [12], specified by the DMTF, and the Management Using Web Services (MUWS),proposed by the Organization for the Advance of Structured Information Standards (OASIS). TheMUWS standard was initially developed by Hewlett Packard and was standardized inside the OASIS.WS-Management is a specification initially developed by a commercial consortium and released inApril 2006 by the DMTF. MUWS offers a richer solution to manage distributed systems. However,due to a simpler set of operations and a more lightweight specification, WS-Management obtains betterperformance results [12].Several WS-based management implementations exist: MUSE [23], which follows the MUWS

architecture, Wiseman [24] and Openwsman [25], tightly coupled to the DMTF WS specification.NETCONF is also an XML-based protocol standardized inside IETF with the goal of performing

management of heterogeneous network equipment. The protocol communication is based on RemoteProcedure Call technology encoded in XML messages, and it follows the traditional client/servermodel where the agent takes the server role. NETCONF messages can be transported over severalprotocols like SSH, BEEP, SOAP or TLS, which has been receiving most attention lately.The protocol supports operations for configuration reading, edition, copy and deletion, and it

supports a lock mechanism for exclusive data access.NETCONF was conceptually divided into four layers, as illustrated in Figure 1. The application

protocols layer is responsible for supporting connection-oriented, reliable and secure communicationbetween the management entities. The RPC layer supplies a simple mechanism for request–reply usingXML messages transported by the lower-layer protocols. It transports the operations layer informationdata encoded in XML, which consists of a basic set of operations invoked by the RPC calls. Finally theupper layer, Content, consists of management information.

Figure 1. NETCONF stack



Netmod WG proposed a data definition language named YANG [26], which follows a C-like syntax.YANG independent Notation (YIN) is a YANG XML representation that allows a bidirectional losslesstranslation more targeted to machine parsing.NETCONF-based management technology has been receiving much attention both from industry

and academics, and several implementations [27,28], SDK [29,30] and APIs [31] have been created.Table 1 summarizes the main details of the technologies studied.

4. OVERHEAD ANALYSIS AND ANALYTICAL APPROACH

Message encoding is one of the most important aspects for the performance of management protocols,since the efficiency of message encoding mostly determines the overhead caused by the protocol ofmanagement. Another key aspect has to do with the number of messages exchanged between themanagement entities for each of the management technologies.

4.1. Message encoding

Signalling length depends greatly on the size of the response messages because the response messagetransports the returned objects exchanged during the experiences, and they increase linearly asmanagement information increases.Pras et al. [9] analysed SNMP signalling extensively and determined that the size of an SNMP

response message is defined by equation (1). The equation represents a message header of 27 bytesand an object encoding header with 5 bytes and with an Object IDentifier (OID) object having a typicalsize of 6 bytes:

LResp ¼ 27 þ NObj� 5 þ LOID þ LDatað Þ (1)

Generally, the size of the response messages (LResp) can be expressed as the sum of the messageheader size (LHeader) with the product of the number of objects (NObj) exchanged by the individualobject size (LObjEncod), as in equation (2):

LResp ¼ LHeader þ NObj� LOnjEncod ¼ 27 þ NOb� 11þ LDatað Þ (2)

In the following paragraphs we propose a set of equations to represent analytically messageencoding overhead for several management protocols.

A. The COPS DEC message has a set of COPS objects included: a common header, a messagehandle and a context object, and it encodes the data objects inside a COPS ClientSI object.Its size can be defined as in equation (3):

LDec ¼ LCommonHead þ LHandle þ LContextDec þ Σ LClientSI (3)

Table 1. Summary of the technology characteristics

Characteristic SNMP WBEM COPS-PR Diameter WS NETCONF

Standardizationbody

IETF DMTF IETF IETF DMTF IETFOASIS

Year 1990 1997 2000 2003 2004 2006Typical usage Network

monitoringServermanagement

Networkconf.

AAAC Servermanagement

NetworkConfiguration

Policyprovisioning

AdmissionControl

Transport UDP andTCP

TCP TCP TCP TCP ssh, SOAP,BEEP, TLS

Data model MIB CIM PIB None CIM YANGEncoding BER XML BER BER XML XMLAgent role Server Server Client Server Server Server



ClientSI object, as defined in equation (4), contains three small-sized fields before it includes theinformation represented: the object length represented in 4 bytes, a 2-byte-long C_Num field and a2-byte-long C_Type field. COPS encoding represents the information in a very compact manner, asillustrated in equation (4):

LClientSI ¼ LObjLength þ LCNum þ LCType þ LData (4)

B. The Diameter response message includes a common header, a session identification AVP, anAuthApplication AVP, a set of AVP objects identifying the origin and the destination entitiesand a list of AVP objects that contains the management objects. The message length can becalculated using the expression in equation (5):

LReq ¼ LCHeader þ LSess þ LAuthApp þ LOrigHost þ LOrgReal þ LDstRelþΣ LDataAVP

(5)

The data AVP format contains an AVP code field, a flags field, a field containing the AVP length andfinally the transported data. Its length can be obtained from the expression in equation (6):

LDataAVP ¼ LAVPCode þ LAVPFlags þ LAVPLength þ LData (6)

C. The WBEM response message includes an HTTP header representing 255 bytes of information,a 360-byte XML header followed by a list of the encoded objects and an 85-byte XML footer.Its size can be calculated from the expression in equation (7):

LResp ¼ LHTTP þ LRespXMLHead þ NObj� LObj þ LRespXMLFooter (7)

WBEM encoding is extremely verbose and it repeats the object key property information as illustratedin Figure 2.The WBEM-encoded object includes a header, some XML tags, the property name, theencoded data and an object footer. Its length can be obtained using the expression in equation (8):

LObj ¼ LObjHead þ LPropXMLTags þ LPropName þ LData þ LObjFoot (8)

An analysis of the WBEM-encoded object represented in Figure 2 allows verification that the objectheader has 396 bytes, property XML tags occupy 60 bytes and the object footer occupies 35 bytes.The equation that describes a WBEM-encoded object can be simplified to equation (9):

LObj ¼ 395 þ 60þ LPropName þ LData þ 35 ¼ 490þ LPropName þ LData (9)

D. The WS response message includes an HTTP header, a SOAP envelope and a list of theencoded objects. Its length can be obtained using the expression in equation (10):

LResp ¼ LHTTP þ LSOAPEnv þ NOb� LObj (10)

WS object encoding is also much more efficient then WBEM. WS makes use of a schema, as can beobserved in Figure 3, in order to describe the object structure, avoiding repetition of the completeobject description.The object encoding includes an object header that defines the validating schemaand a pair of XML tags. Its length can be obtained using the expression in equation (11), andsimplifying we obtain equation (12):

Figure 2. WBEM object encoding

Figure 3. WS object encoding



LObj ¼ LHead þ LPropName þ 2� LPropName þ 4ð Þ þ LData þ 1 (11)

LObj ¼ 156 þ 3� LPropName þ LData (12)

E. The NETCONF response message includes a transport header, a SOAP envelope and the SOAPresponse information, followed by the list of encoded objects. The response message size can becalculated using the expression illustrated in equation (13):

LResp ¼ LHTTP þ LSOAPEnv þ LRPCResp þ NObj� LObj (13)

NETCONF object encoding, illustrated in Figure 4, is more efficient than WS object.Taking intoaccount the <config> tags size, the NETCONF-encoded object length can be calculated from theexpression illustrated in equation (14):

LObj ¼ 2� 11 þ LPropNameð Þ þ LData (14)

4.2. Header and data lengths

The header length was analyzed for each of the technologies and is represented in Table 2. The datashow impressive differences between technologies, as expected. A WBEM message header is 25 timesbigger than an SNMP message header. Moreover, and with a much deeper impact on signalling cost, aCOPS object header is 124 times smaller than a WBEM object header. Between the XML-basedtechnologies there is also a huge difference: the NETCONF object header is 22 times smaller thanthe WBEM object header.Each technology performs data representation inside the message in a different manner. The major

differences observed in Table 3 are the encoding of numbers and addresses.XML-based technologies convert numeric data to text in order to keep the human readability of the

data, while the binary technologies keep the numeric format of the information in the encoded message.XML-based technologies perform the address translation keeping the address information in clear text,while binary technologies keep the information in numeric format, saving a lot of signalling.

Figure 4. NETCONF-encoded object

Table 2. Header sizes

Technology Message header Object header

SNMP 27 11COPS 40 4Diameter 83 8WBEM 695 496WS 581 183NETCONF 288 22

Table 3. Data encoding sizes

Technology Byte Integer Address

SNMP 1 1 8COPS 1 1 8Diameter 1 1 38WBEM 3 5 38WS 3 5 38NETCONF 3 5 38


Signalling size

100

1000

10000

100000

1000000

10000000

1 10 100 1000 10000

Number of objects

Sig

nal

ling

(B

yte)

WBEM

WS

NETCONF

SNMP

Diameter

COPS

Figure 5. Generated signalling comparison, excluding transport component


4.3. Signalling load

Taking into account the message encoding performed by each of the technologies described in theprevious paragraphs, and the data representation size listed in Table 3, it is possible to plot the signallinggenerated during the process of exchanging the configuration objects. Taking into account thefrequency each data type appears in Schönwälder et al. [11] and if we normalize the results based oneach data type frequency we obtain the amount of signalling produced by the applications of eachtechnology, as illustrated in Figure 5. Note that this signalling analysis did not consider transport-levelsignalling (TCP/UDP) of the generated traffic.

5. EXPERIMENTAL EVALUATION

For our comparison analysis we created a test scenario. This test scenario reflects a configurationoperation, where the client application requests the configuration from the server application. Sincethe focus of this study was the complexity of management technologies, and more specifically thesignalling they generate, the management scenario follows a Manager–Agent approach, although sucha scenario does not represent an operational network scenario [11] where one manager is alwaysresponsible for a large number of agents.

5.1. Experimental framework

During tests, the traffic exchanged between the server and the client was captured and the traces wereprocessed to obtain the number of packets, number of bytes, protocol header sizes and message requestsizes. These tests were repeated 20 times for each of the management protocols under evaluation, andthe averages of the test values were calculated.For each technology, an adequate pair of applications was selected—a client and a server:

• We developed a basic client and a Diameter server that use a Diameter API implementing RFC3588 and RFC 5224 specifications.

• We developed a basic client and COPS server, interfaced by a COPS++ API [32]. The APIimplements the RFC 2748 specification.

• We developed a basic SNMP agent based on the Agent++ open source project, and the SNMPmanager was a Net-SNMP client application. The SNMP messages were based on SNMPv2.SNMPv3 is not much used in operational networks [11] and SNMPv1 would not allow the useof SNMP bulk mechanisms.

• We customized an OpenWBEM framework and, to represent the Interface objects, several CIMextensions were developed as well as an instance provider to deal with these CIM instances. TheWBEM client used in the experiments was an OpenWBEM client application named owexecwql.



• The WS scenario was constructed through a customized Openwsman solution.• The NETCONF scenario was constructed through the creation of a basic client and serverapplication, using NETCONF over SOAP interface.

In Table 4 we summarize the most relevant characteristics of the implementations used in the tests,the messages exchanged and their lengths, and the specification they follow.Applications were tested in an isolated fast Ethernet LAN segment in order to avoid external

disturbances. During the start-up, the server application read the management information from arelational database where a predefined number of objects according to the desired information volumewere inserted. Client applications requested configuration from server applications that answer withdatabase information using their communication protocol.

5.2. Messages and data type frequency

The messages exchanged in the tests, and listed in Table 4, follow the scenario described at thebeginning of the last section.The SNMP client requests configuration from the SNMP agent using getBulkRequest messages,

which responds by means of getResponse messages, the most frequent messages in SNMP traffic [11].The COPS client sends a 126-byte REQ message to the COPS server requesting configuration,which responds with a DEC message returning the requested configuration in a typical COPS-PRscenario. The configuration message is acknowledged by the COPS client with a 120-byte RPTmessage reporting the success of the decision installation. The Diameter client requests its configurationby means of a 214-byte RAR message, and the server responds with a RAA message.The WBEM client requests its configuration using an 884-byte EnumerateInstances message to the

server, which responds with an EnumerateInstancesResponse message. The WS client requestsconfiguration from the WS server with a 935-byte Enumerate request message, which responds withan EnumerateResponse message. The NETCONF client requests configuration from the server witha 409-byte rpc message, which responds with an rpc-reply message.The data elements exchanged in the experiments followed the data type frequency observed by

Schönwälder et al. [11] in the SNMP traces. The data type frequencies observed in this study weregrouped by size and the cumulative sum was calculated. During the experiments, the traffic captureswere collected for each of the data types. Later, the data type frequencies were taken into account inorder to achieve the effect each data type has on overall technology performance.

6. RESULT ANALYSIS

The results of the experiments were processed and we plotted the signalling volume values, thenumber of packets and the efficiency of each technology.

6.1. Signalling results

The tests show that the signalling created by the management applications during the experimentsincreases linearly with the increase in the number of objects exchanged.Figure 6 compares the values obtained in the tests and the values obtained from the equations in Section

4. Analysing the figure, we can see that the theoretical values are in line with the test values. The exceptionis the SNMP-generated signalling, where we obtained about 30% more signalling load than expected.After a detailed trace analysis we identified inefficient usage of the UDP packet payload by the SNMPagent application. By not using the complete transport payload, the SNMP application needs to send morepackets, increasing the overhead caused by the additional getBulkRequest and getResponse headers.The number of IP packets exchanged between management entities of each technology (Figure 7),

including the TCP session establishment and session close packets for the technologies that use TCP,increases linearly with the increase in the number of objects exchanged. The technology that sendsmost data packets is WBEM, while the one that sends fewest is COPS.


Table

4.Sum

maryof

thetested

implem

entatio

ncharacteristics

Technology

Applicationtype

Interface

Protocolversion

Operatio

ns

Nam

eSize

SNMP

Custom-builtagentandnet-snmp

Based

onAgent++

RFC3410

getBulkR

equest

87get-Response

*

COPS-PR

Custom-builtserver

andclient

Based

onCOPS++[32]

RFC2748

REQ

126

DEC

*RPT

120

Diameter

Basic

applications

Based

onDiameter

API

RFC3588

RAR

214

RFC5224

RAA

*

WBEM

Customized

versionof

OpenW

BEM

server

andow

execwql

OpenW

BEM

CIM

Operatio

nsover

HTTP—

v1.3.0

EnumerateInstances

884 *

WSMAN

Customized

versionof

Openw

sman

server

andwsm

anas

client

Openw

sman

Web

ServicesforManagem

ent

(WS-M

anagem

ent)Specificatio

nwsen:Enumerate

935

wsen:Enumerate

Response

*

NETCONF

Custom-builtserver

andclient

NETCONFover

SOAP

RFC4743

rpc

409

rpc-reply

*



Signalling exchanged

100

1000

10000

100000

1000000

10000000

1 10 100 1000 10000

Number of objects

Sig

nal

ling

(B

ytes

)

WS - Pra

WS - Teor

NETCONF - Pra

NETCONF - Teor

SNMP - Pra

SNMP - Teor

Figure 6. Comparison of signalling values obtained by theoretical and practical means

Figure 7. Number of IP packets exchanged


SNMP shows mixed behaviour: for a small amount of exchanged objects it sends a small number ofpackets—even lower than other binary technologies. SNMP has the shortest header of the technologiesunder evaluation, and the UDP protocol presents a shorter header than the TCP protocol. Moreover,since UDP does not implement a session, network overhead caused by session control does not occur.However, because of the absence of a session in the underlying protocols, when the number of objectsis high the management entities have to break the object information into separate parts that can fit inthe UDP packet and repeat the request response/messages until the complete set of objects can betransferred. This is the reason why the SNMP protocol cannot scale efficiently, as underlined in Praset al. [9] and Neisse et al. [13].The larger the number of packets exchanged between management entities, the greater the probability

of collisions occurring between management information packets and the remaining network traffic.Furthermore, since SNMP does not implement session control mechanisms, SNMP solutions might showbehaviour degradation with increased volume of information management.The binary protocols signalling volume curves intercept for object numbers between 5 and 20.

Figure 8 illustrates the captured signalling volume evolution of the binary protocols for small numbersof objects. We can see that Diameter signalling exceeds COPS signalling volume for six objects.Although the COPS protocol performs more efficient object encoding, it implements a 120-byteRPT message that is not implemented in any other technology under test.The SNMP protocol keeps the signalling volume smaller than COPS and Diameter up to

the point where it needs a new getBulkRequest/getResponse message pair. On the other hand,TCP-transported protocols make use of the established TCP session, avoiding new request/response messages.


Figure 8. Detailed graph of signalling values for small number of objects exchanged


6.2. Protocol efficiency

The protocol efficiency can be obtained by dividing the amount of information by the total volume ofsignalling exchanged by the management entities. Figure 9 illustrates the efficiency evolution of themanagement technologies under analysis. Generally, the transport efficiency of the managementprotocols increases with the increase of exchanged information until it reaches a maximum. The gainis obtained because of two factors: more efficient use of the transport capacity of lower-layer protocols,and (in the case of management technologies that use TCP transport) the dilution effect of requestmessage sizes in the total signalling volume. The more verbose the technology, the sooner it reachesmaximum efficiency, and with a lower value.For a larger number of objects, WBEM (0.9%) and WS (2.3%) are the least efficient technologies

under analysis. NETCONF (14.2%), despite being an XML-based technology, is more efficient thanSNMP (13.7%). NETCONF efficiently encodes the management data and, contrary to SNMP, usesthe entire payload of the transport packet. Moreover, SNMP protocol uses wide OID objects thatseverely degrade its encoding efficiency and uses additional request/response messages to continuethe information exchange as soon as a response packet payload limit is reached. The most efficienttechnologies were Diameter and COPS, which reached respectively 18.5% and 36.4%.SNMP occupied the best position for a smaller number of objects but, owing to limitations caused by

the UDP transport, the technology reached the efficiency limit sooner than the remaining technologies.

6.3. Performance comparison

Since SNMP continues to be the de facto standard for network management, we compared the individualsignalling of each technology with SNMP. Figure 10 illustrates the results obtained.

Protocol Efficiency

0

5

10

15

20

25

30

35

40

1 10 100 1000 10000

Number of objects

Sig

nal

ling

/ In

form

atio

n r

atio

(%

)

COPS

Diameter

SNMP

NETCONF

WS

WBEM

Figure 9. Protocol efficiency comparison


Signalling compared with SNMP

0.1

1

10

100

1 10 100 1000 10000

Number of Objects

Rat

io w

ith

SN

MP

sig

nal

ling

WS NETCONF WBEM COPS Diameter

Figure 10. Signalling compared with SNMP


Owing to the good performance SNMP obtained for small volumes of data, all the technologies showworse performance for data volumes lower than 20 objects. For large volumes of data, WBEM (16,7%)andWS (6.4%) have poorer performance than SNMP. NETCONF shows almost the same performanceas SNMP, while Diameter (0.7%) and COPS (0.4%) show better performance than SNMP.

7. DEPLOYMENT RECOMMENDATIONS

In order to assure the performance of a management solution, it is important to choose the technologyaccording to the expected data volume. Each technology showed different performance for differentmanagement data volumes.SNMP is the best example of a technology that degrades its performance with an increase of

data volume [33]. There is not much a management solution designer can do to improve SNMPperformance: keep the OID depth under control as much as possible during Management InformationBase (MIB) design and avoid long community strings.COPS technology is extremely efficient with large data volumes: its shortcomings have to do with

the RPT message, which is more expressive for small volumes of data.Diameter presents a considerable overhead caused by the complexity of its header, especially for

small data volumes. It is possible to reduce such overhead using short-destination realm identifiers.With respect to the Diameter object encoding, it is possible to reduce the signalling if an applicationdispatches the complete records in the tabular information. Record-by-record transfer avoids repetitionof the AVP header, contrary to what happens in a field-by-field scenario.NETCONF shows a considerable efficiency gain over the remaining XML-based technologies,

especially over WBEM. Still, some measures can be taken to reduce the amount of signal produced,e.g. using short leaf node names would reduce the object encoding size.3GPP NGN management architecture is not fully defined. Management protocols have not yet

been defined, but two alternatives seem viable: Diameter, because it is omnipresent inside the NGNIMS network; and NETCONF, because it has been pushed by the IETF and many IETF developedtechnologies are reused in the 3GPP NGN network. Thus an interesting detail of the comparisonconcerns the NETCONF/Diameter performance.There is no significant difference in the signalling overhead associated with NETCONF and

Diameter. However, Diameter technology implements a heavy state machine and message routingmechanisms that require considerable computational capabilities from the management entities,making it less suitable for the NGN scenario.WBEM is extremely verbose, as illustrated by the object encoding represented in Figure 2. It is

possible to decrease WBEM response messages using smaller property names, but the effect willnot be expressive because this does not decrease the message header. WBEM solutions, if used in highdata volumes, should include a technology adaptor for a less verbose technology [13,34].



WS technology presents an efficiency gain over the previous DMTF standard, but as an XML-basedtechnology it is still very verbose. It is, however, possible to make a small reduction of the objectencoding if short property names are chosen.According to the results published in a previous study [1] it is possible to obtain considerable gains

in signalling volume, especially in XML-based technologies, by using compression techniques. It is,however, necessary to take into account the increase in response times due to information compressionand decompression processes, which were not dealt with in that study. A balance between signallinggain and response time has to be found according to the volume of information expected.

8. CONCLUSION

In this paper we analyse the information encoding of several management technologies and compareanalytically and through experiments the signalling produced by each of the technologies underanalysis. We compare the overhead caused by management protocols as a function of several volumesof management information.Binary-based protocols showed a lower overhead in signalling volume, although XML-based

technologies have been improving their performance. NETCONF is very efficient, with performance verysimilar to binary technology. The IETF succeeded here in the design of an XML-based managementtechnology that is more efficient than DMTF WS. For high information volumes, NETCONF producesa signalling volume similar to SNMP, while WS produces 6.4 times more signalling than SNMP.The efficiency of the management technologies varies with the volume of management information:

for low information volumes, SNMP is the most efficient technology, while for high information volumesCOPS is the most efficient. Thus choice of a management technology has to take into consideration theexpected volume of information.Several measures can be adopted during the management application design/development in order

to reduce technology drawbacks and to improve performance. For instance, signalling produced byXML-based protocols is greatly affected by the length of property names: the shorter the propertynames, the smaller the signalling produced.NETCONF should be considered as a management information protocol candidate in the NGN

management architecture definition: NETCONF is an IETF technology like the majority of protocolsused inside the IMS NGN architecture, and it does not involve complex mechanisms like the Diameterstate machine nor the Diameter message routing mechanism, while showing reasonable efficiency.

REFERENCES

1. Gonçalves P, Oliveira JL, Aguiar RL. An evaluation of network management protocols. In 11th IFIP/IEEE InternationalSymposium on Integrated Network Management (IM 2009), New York, 2009.

2. Enns R. NETCONF Configuration Protocol. RFC 4741, 2006.3. ITU-T. General overview of NGN, Rec. Y. 2001, 2004.4. 3GPP. IP Multimedia Subsystem (IMS) Stage 2, Release 9, 2011.5. TISPAN. NGN Functional Architecture. ETSI ES 282 001, V2.0.0, 2008.6. Broadband Forum. Available: http://www.broadband-forum.org/ [24 September 2011].7. Kueh V, Wilson M. Evolution of policy control and charging (PCC) architecture for 3GPP evolved system architecture. In

63rd Vehicular Technology Conference (VTC 2006), IEEE 2006; 259–263.8. 3GPP. Telecommunication Management: Architecture, Release 10, 2011.9. Pras A, Drevers T, van de Meent R, Quartel DAC. Comparing the performance of SNMP and Web services based

management. IEEE Transactions on Network and Service Management 2004; 2: 1–11.10. Andrey L, Festor O, Lahmadi A, Pras A, Schönwälder J. Survey of SNMP performance analysis studies. International

Journal of Network Management 2009; 19: 527–548.11. Schönwälder J, Pras A, Harvan M, Schippers J, van de Meent R. SNMP traffic analysis: approaches, tools, and first results.

In 10th IFIP/IEEE International Symposium on Integrated Network Management (IM ’07), 2007; 323–332.12. Moura G, Silvestrin G, Sanchez R, Gaspary L, Granville L. On the performance of Web Services management standards: an

evaluation of MUWS and WS-Management for network management. In 10th IFIP/IEEE International Symposium onIntegrated Network Management (IM 2007), Munich, Germany, 2007; 459–468.


http://www.broadband-forum.org/


13. Neisse R, Vianna R, Granville L, Almeida M, Tarouco L. Implementation and bandwidth consumption evaluation of SNMPto Web services gateways. In Network Operations and Management Symposium (NOMS 2004), Vol. 1, IEEE/IFIP, 2004;715–728.

14. Pavlou G, Flegkas P, Gouveris S, Liotta AALA. On management technologies and the potential of Web services.Communications Magazine, IEEE 2004; 42: 58–66.

15. Lima W, Alves R, Vianna R, Almeida M, Tarouco L, Granville L. Evaluating the performance of SNMP and Web Servicesnotifications. In 10th IEEE/IFIP Network Operations and Management Symposium (NOMS 2006), 2006; 546–556.

16. Yoo S-M, Ju HT, JW Hong. Performance improvement methods for NETCONF-based configuration management. InManagement of Convergence Networks and Services. Springer: Berlin, 2006; 242–252.

17. Franco T, Lima W, Silvestrin G, Pereira RC, Almeida M, Tarouco L, Granville L, Beller A, Jamhour E, Fonseca M.Substituting COPS-PR: an evaluation of NETCONF and SOAP for policy provisioning. In Seventh IEEE InternationalWorkshop on Policies for Distributed Systems and Networks (Policy 2006), 2006; 10–204.

18. Gonçalves P, Azevedo R, Gomes D, Oliveira JL, Aguiar RL. Evaluation of policy based admission control mechanisms inNGN. In International Conference on Telecommunications, Marrakech, Morocco, 2009.

19. Boyle J, Cohen R, Durham D, Rajan R, Herzog S, Sastry A. COPS usage for RSVP. RFC 2749, 2000.20. Chan KH, Durham D, Gai S, Herzog S, McCloghrie K, Reichmeyer F, Seligson J, Yavatkar R, Smith A. COPS usage for

policy provisioning (COPS-PR). RFC 3084, 2001.21. Brenner M. Diameter policy processing application. RFC 5224, 2008.22. Lee S-J, Choi M-J, Yoo S-M, Hong JW, Cho H-N, Ahn C-W, Jung S-I. Design of a WBEM-based management system for

ubiquitous computing servers (2006). Available: http://www.dmtf.org/education/academicalliance/mjchoi_2004.pdf[18 July 2008].

23. Beran P, Habel G, Schikuta E. SODA: a distributed data management framework for the internet of services. In SeventhInternational Conference on Grid and Cooperative Computing (GCC ’08), 2008; 292–300.

24. Sun. Wiseman: A Java implementation of WS-Management. Available: https://wiseman.dev.java.net/ [4 May 2005].25. Openwsman | WS—Management for All. http://www.openwsman.org/ [2 May 2008].26. Bjorklund M. YANG: A Data Modeling Language for the Network Configuration Protocol (NETCONF). RFC 6020, 2008.27. Tumar I, Tran HM, Schönwälder J. NETCONF Interoperability Testing, Vol. 5637. Springer: Berlin, 2009; 83–94.28. Cridlig V, Abdelnur H, State R, Festor O. XBGP-MAN: an XML management architecture for BGP. International Journal

of Network Management 2006; 16: 295–309.29. Tavares P, Gonçalves P, Oliveira JL. An IDE for NETCONF management applications. In 7th Latin American Network

Operations and Management Symposium, Quito, Equador, 2011.30. Yuma—YANG-based UnifiedModular Automation Tools. Available: http://sourceforge.net/projects/yuma/ [21 February 2011].31. Netopeer—Remore configuration system using NETCONF protocol. Available: http://code.google.com/p/netopeer/

[21 February 2011].32. Gomes D., COPS++, http://atnog.av.it.pt/projects/copspp [30 January 2012].33. Sprenkels R, Martin-Flatin J-P. Bulk transfers of MIB data. Simple Times 1999; 7: 1–7.34. Yoon JH, Ju H-T, Hong J. Development of SNMP-XML translator and gateway for XML-based integrated network

management. International Journal of Network Management 2003; 13: 259–276.

AUTHORS’ BIOGRAPHIES

Pedro Gonçalves is an assistant professor at Águeda Technology and Management School (ESTGA) of AveiroUniversity. He received a PhD degree in computer engineering in 2010 from University of Aveiro, and he is aninvestigator of Communications Institute. His main interests include quality of service, management technologiesand autonomic management.

José Luís Oliveira is associate professor at the Dept. of Electronics, Telecommunications and Informatics (DETI)of University of Aveiro. He holds a PhD on distributed systems and network management and his main researchinterests also include computational methods for bioinformatics and biomedical informatics. He has publishedmore than 200 papers on book chapters, journals and international conferences.

Rui Aguiarreceived a Ph.D. degree in electrical engineering in 2001 from the University of Aveiro. He iscurrently an associate professor at the University of Aveiro and an adjunct professor at the INI, Carnegie MellonUniversity. He has more than 300 published papers in those areas. He has served as technical and general chair ofseveral conferences, such as ICNS ’05, ICT ’06, ISCC ’07 and Monami’2011.


http://www.dmtf.org/education/academicalliance/mjchoi_2004.pdf

http://atnog.av.it.pt/projects/copspp

http://code.google.com/p/netopeer/

http://atnog.av.it.pt/projects/copspp

Documents

A study of encoding overhead in network management protocols