Performance Comparison of Resilience Mechanismsfor Stateless Multicast Using BIER

Wolfgang Braun1, Manuel Albert1, Toerless Eckert2, Michael Menth1University of Tubingen, Department of Computer Science, Germany1, Cisco Systems, Inc.2

Abstract—Bit Indexed Explicit Replication (BIER) is a novelmulticast forwarding scheme for IP networks that avoids states inreplicating routers by encoding the multicast information into abit string in the packet header. In addition, the BIER-TE variantencodes the multicast tree in the header and allows for networkprogrammability.

We propose the use of maximally redundant trees (MRTs)for 1+1 protection in BIER that currently lacks this feature.We further discuss three different fast reroute (FRR) protectionschemes for BIER-TE we have proposed in the Internet Engineer-ing Task Force (IETF). They use header modification only (HM),rely on point-to-point tunnels (PPT), or leverage BIER-in-BIERencapsulation (BBE). We compare them regarding protectioncoverage, path lengths, traffic loads, required network capacity,state requirements and overhead in a large number of networks.The results serve the discussions in IETF where BIER and BIER-TE are currently standardized.

Index Terms—Bit Indexed Explicit Replication, Multicast,Resilience, Scalability, Resource Management

I. INTRODUCTION

IP multicast allows efficient data transmission from onesender to many receivers. In general, it is desirable that trafficis optimally forwarded in such a way that the least packetduplication and load in the network occurs. The research of thepast decades improved many areas for multicast, e.g., traffic-engineered multicast deployment. However, network operatorstoday still face several operational problems when applyingcurrent multicast protocols, e.g., Protocol Independent Mul-ticast (PIM), to common use cases such as L3VPN, IPTV,and over-the-top services [1], [2]. The protocols generally relyon explicit tree building mechanisms. The trees have to beinstalled in the routers and increase their overall state. Forsome use cases, too much state is required in the routers sothat they are unable to optimally forward the multicast traffic.E.g., operators often flood packets of some multicast flows toall potential egress nodes regardless of a subscription. Thiswastes bandwidth to save state in routers and operators haveto find the right tradeoff. Multicast state in routers also causesoperational issues if the network is reconfigured or subscribersare added or removed. The control plane requires many routersto participate in the process, compute new trees, and installnew rules which may result in convergence times of multipleminutes in large multicast deployments.

The authors acknowledge the funding by the Deutsche Forschungsgemein-schaft (DFG) under grant ME2727/1-1. The authors alone are responsible forthe content of the paper.

The IETF currently works on Bit Indexed Explicit Repli-cation (BIER) to address the issues mentioned above. WithBIER, multicast traffic can be forwarded without per-multicast-flow state by encoding the egress nodes of a mul-ticast flow in a new BIER header [3]. Ingress nodes addthis BIER header to multicast packet, transit nodes onlyforward them without multicast-flow-specific information, andegress nodes remove the BIER header. Thus, BIER supports asimple multicast overlay which is easy to operate and requiresminimal state in routers. In addition, Traffic Engineering forBIER (BIER-TE) [4] is proposed in the same working group.It encodes the multicast tree in the BIER header and allows fortraffic-engineered multicast trees with minimal state overhead.

In this work, we propose a 1+1 protection mechanism forBIER because it currently lacks such a mechanism. It isbased on existing IETF techniques and is compatible with thecurrent BIER specification. We further suggest three differentfast reroute (FRR) mechanisms for BIER-TE that we recentlyalso proposed for standardization in IETF [5]. We explainthe operation of these mechanisms and constraints for thecomputation of their path layout. We analyze and comparethe proposed resilience mechanisms for BIER and BIER-TE on 220 network topologies with regard to protectioncoverage, path lengths, network loads, capacity requirements,state requirements, and header overhead.

The remainder of this paper is structured as follows. Sec-tion II discusses related work. In Section III we explainBIER and propose a novel 1+1 protection mechanism forBIER. Section IV introduces BIER-TE and propose threedifferent FRR mechanisms for BIER-TE. Section V discussesperformance results. Finally, Section VI concludes the paper.

II. RELATED WORK

There are various approaches to construct node-redundantpairs of multicast trees [6]–[10] that can be used to imple-ment a 1+1 protection scheme for multicast traffic. Theseapproaches differ by the considered objective function suchas cost, bandwidth, computation complexity, required net-work state, and update complexity. We leverage the routingtopologies of Maximally Redundant Trees (MRTs) [10], [11]as node-redundant pairs of multicast trees for a novel BIERprotection mechanism because they impose state and com-putation requirements for routing underlays that scale wellfor large networks. The Parallel Redundancy Protocol (PRP)[12] and High Availability Seamless Redundancy (HSR) [13]

978-3-901882-89-0 @2017 IFIP 230

provide node-redundant multicast trees that suffice hard real-time constraints in industrial Ethernet networks.

Reliability for multicast traffic can be implemented usingacknowledgments (ACKs) and selective repeat transmissionssimilar to TCP. However, the number of ACKs of manyreceivers likely overburden a source. Therefore, the Reli-able Multicast Transport Protocol (RMTP) [14] and otherapproaches [15], [16] use a shared ACK tree structure toovercome this problem.

MPLS is currently deployed in many ISP networks and pro-vides multicast services with point-to-multipoint (P2MP) LSPs[17]. Such P2MP services support FRR by local repair whenRSVP-TE is used. However, these solutions can be unsuitablewhen multicast group memberships change frequently [18].

There are various algorithms and mechanisms to computetraffic-engineered multicast trees based on multiple objectivefunctions. An exhaustive survey of existing methods is givenin [19]. The approaches are classified by objective functions,constraints, etc. The authors also propose a generalized multi-objective optimization that is based on load-balanced multipletrees. Most works focus on the computing algorithm and donot discuss the required router state in detail. Yet, these worksare orthogonal to the BIER-TE approach because BIER-TEdoes not propose any path layout but rather supports encodingof optimized multicast trees in the BIER header.

There are several software-defined networking (SDN) ap-proaches for IP multicast. In [20], the authors implementIP multicast using VXLAN in datacenters and remove theneed for periodic control plane interaction. A highly scal-able IP multicast datacenter method is proposed in [21]which leverages flow aggregation to support large numbersof multicast joins simultaneously. “Dynamic Software-DefinedMulticast” [22] reduces control plane complexity in multicastdeployments of ISP networks and adds traffic engineeringaspects using multiple trees similar to [19]. BIER and mostSDN approaches simplify the control plane while supportingfrequent multicast changes. BIER introduces header overheadbut only requires a minimal amount of state in transit routers.In contrast, most SDN methods leverage existing header fieldsbut require significantly more state in forwarding devices.

III. BIT INDEXED EXPLICIT REPLICATION (BIER)

We first introduce the BIER architecture as defined in [3]and then propose a 1+1 protection scheme for BIER that iscompatible with the current specification.

A. Architecture and Forwarding Procedure

BIER provides a multicast overlay without any states formulticast tree on core routers. A BIER domain consists ofso-called bit forwarding routers (BFRs). Ingress BFRs add aBIER header to incoming multicast packets. The BIER headerindicates all BFRs that should receive a copy of the packet,so-called egress BFRs. To that end, each BFR in the networkis represented by one bit position in the BIER header which isset if the BFR is an egress BFR for the packet. BFRs forwardBIER packets solely based on their BIER header and a Bit

Indexed Forwarding Table (BIFT) whose entries depend on therouting underlay, which may be an Interior Gateway Protocol(IGP) such as OSPF or ISIS.

The forwarding procedure works as follows. A BFR es-sentially forwards BIER packets towards all egress BFRsindicated in the BIER header over the routing underlay. How-ever, it sends at most one copy towards each next-hop (NH)and clears all egress BFRs in the BIER header that are notreachable by itself over the respective NH. The BIFT containsinformation that supports a BFR to perform these operationsefficiently for fast packet processing. When a packet reachesan egress BFR, the BIER header is decapsulated and the packetis forwarded as usual. A salient feature of BIER is that onlyingress BFRs need to know multicast groups including egressBFRs in order to add appropriate BIER headers to packets. Allother BFRs in the BIER domain do not need to know thesegroups for multicast forwarding. This makes BIER scalableand requires only reconfiguration of ingress BFRs if multicastgroup membership of egress BFRs changes. Essentialy, BIERremoves the multicast state of conventional multicast routingprotocols from core routers by encoding the multicast egressnodes into the packet headers.

BFRs may support bit strings (BIER headers) between 64and 4096 bits, the support for a bit string length 256 ismandatory. Amongst other protocol information, it mainlycontains the bits for potential egress BFRs. If the bitstringis too small to accommodate all BFRs, the set of potentialegress BFRs can be broken down into several sets [3]. Theybasically receive traffic over egress-BFR-disjoint but possi-bly overlapping multicast trees which increases the trafficload in the network. Thereby, very large topologies can besupported. BIER supports different subdomains for whichdifferent routing underlays may be used. Thereby, they canbe leveraged to forward multicast packets differently, e.g., fortraffic engineering purposes. If multiple subdomains are in use,the ingress BFR chooses the appropriate one for an incomingpacket and indicates it in the BIER header.

B. Multicast only Fast Reroute (MoFRR for BIER)

BIER does not provide a protection mechanism but ratherrelies on the restoration process of the routing underlay whichmay be slow and cause packet loss. We propose to combineMulticast only Fast Reroute (MoFRR) [23] with BIER toprovide fast protection. MoFRR is based on 1+1 protection:the traffic is duplicated at the source and sent over redundantpaths to the destination. The destination node continuouslymeasures the quality of the streams from both paths, selectsthe one with highest quality for forwarding, and discards theother. In case of multicast, two redundant trees are required.The ingress node duplicates the traffic, sends it over both trees,and egress nodes forward packets from only one of them. Ifone tree fails, the egress node is likely to still receive the trafficfrom the other tree.

The BIER architecture may be upgraded as follows toimplement MoFRR. At least two subdomains with differentrouting underlays are needed to allow for redundant multicast

2017 IFIP/IEEE International Symposium on Integrated Network Management (IM2017) 231

Fig. 1: The blue and red MRT routing topologies forwardtraffic over node-redundant paths.

trees. The ingress BFR copies incoming BIER packets to twosubdomains that provide redundant multicast trees. The egressBFRs read from the two streams and forward only a singlecopy.

A challenge is the provision of two routing underlays suchthat they yield redundant trees. We propose to leverage Max-imally Redundant Trees (MRTs) for that purpose which havebeen standardized by the IETF [24]. This idea has already beenproposed for other multicast mechanisms in [25], but the drafthas been abandoned. MRTs calculate dual routing topologiesin a distributed way. In the absence of failures, a packet canbe delivered of both of them to all destinations. The resultingpaths in the two routing topologies do not necessarily forma pair of trees but the paths are node-redundant in the sensethat the packet reaches any destination over at least one routingtopology in case of any single link failure. Figure 1 illustrateshow a packet is forwarded over two redundant topologies (blueand red) from one source to all other nodes depending on itsdestination address. The straight lines represent a tree alongwhich a packet may be delivered to all intermediate nodes andleaves. The dashed lines show how the packet reaches a singleremaining node over an additional path whose intermediatenodes receive packets over the tree. This example helps tounderstand some evaluation results in Section V. However,traffic is mostly carried over real trees and it was in factdifficult to find this small counterexample.

To protect unicast forwarding, MRTs are applied as follows.In the failure-free case, traffic is forwarded over shortest pathsinstead of one of the two routing topologies. In case of afailure, it is locally rerouted over a working routing topology.This may lead to very long backup paths [26].

We propose to use the MRT routing topologies for multicastforwarding although they do not form a tree. Nevertheless,we denote them as multicast trees when used in that context.For this purpose, multicast traffic is distributed along thetwo routing topologies to all required destinations. In caseof a failure on one topology, the traffic is dropped insteadof being switched to the other. In the context of BIER, thetwo MRT routing topologies must be maintained by routingunderlays in the two different subdomains and ingress BFRscopy incoming multicast packets to both of them. Packets aredelivered only once to egress BFRs over a routing topologybecause BFRs modify the set of egress BFRs on forwarding in

the packet header. This mechanism prevents that egress BFRsaccidentally obtain separate copies over a solid and the dashedpath within a routing topology and ensures that BIER packetsare forwarded only when needed.

Standardized algorithms for calculation of the routingtopologies are available in [11]. They can be computed usinga few spanning tree operations in a very fast manner which isoften desired in ISP networks. However, the path layout is notalways optimal. Therefore, this MRT-based MoFRR solutionis appealing for the deployment of BIER in large networks.The improvement of a node-redundant path layout for routingunderlays with little state is an open research question.

IV. BIER TRAFFIC ENGINEERING (BIER-TE)

In this section, we present the BIER-TE architecture and itsforwarding operations. We suggest a general operation of FRRfor BIER-TE an then propose three different implementationoptions for BIER-TE FRR.

A. The BIER-TE Architecture

The BIER-TE architecture [4] is based on a segment routing[27] approach that is similar to SDN in the sense that thepath layout for each flow can be explicitly configured by acontroller. BIER-TE leverages the BIER header defined in theBIER architecture [3]. There are two main differences betweenBIER and BIER-TE. First, BIER-TE encodes both the linksand the egress nodes of a multicast tree in the BIER headerwhile BIER only encodes the latter. Second, unlike BIER,BIER-TE does not necessarily require an IGP control networkor a routing underlay.

The BIER-TE architecture consists of a BIER controller andBFRs. The controller computes traffic-engineered multicasttrees and instructs ingress BFRs to apply appropriate BIERheaders to incoming multicast traffic. These BIER headerscontains forwarding information and reflects the multicaststructure. Furthermore, the controller installs forwarding rulesin the forwarding tables of the BFRs which are called BitIndexed Forwarding Table (BIFT). Their contents is indepen-dent of existing multicast flows. A link between two BFRs inthe BIER overlay is called an adjacency. An adjacency maybe a physical link directly connected to a BFR neighbor or atunnel provided by the routing underlay (remote adjacency).It is possible to have more than one adjacency between twoBFRs when the destination BFR is reachable through differentinterfaces.

B. BIER-TE Forwarding

We illustrate the main forwarding procedure in Figure 2.Packets are sent from A to C and D. The initial headercontains the links A→C, A→B, B→D and the egress nodes Cand D. The egress node bits are called local_decap bits.The adjacencies and the local_decap bit of a BFR arecalled Bits of Interest (BOI) and are highlighted in blue in theheaders. There are two BOI for A, A→C and A→B. A clearsits BOI from the header and sends the packets to C and B.The packet arriving at C only has C set. The BIER header

2017 IFIP/IEEE International Symposium on Integrated Network Management (IM2017)232

Fig. 2: The BIER header contains adjacencies (first line) andlocal_decap bits (second line). The BOI of the nodes arehighlighted in blue. The BIER header is modified before thepacket is forwarded to the NH.

Fig. 3: With node protection, the PLR reroutes traffic to alldownstream next-next-hops (DS-NNH).

is decapsulated and the packet leaves the BIER overlay. Notethat the BIER header is not empty at C and still containsbits with regard to the other subtree of the multicast tree.This is important for one of the FRR schemes presented inSection IV-C. The packets at subtree at B are processed in thesame way as at A.

C. Fast Reroute for BIER-TE

The reachability of neighboring BFRs over a specific ad-jacency can be controlled by a Bidirectional ForwardingDetection (BFD) component so that a BFR can detect thata neighbor is no longer reachable and locally reroute affectedtraffic, i.e., the BFR acts as point of local repair (PLR).

If a BFR detects that a BIER packet needs to be forwardedover a failed adjacency, the BFR uses information in the BIERheader to consult the BIER-TE Adjancency Fast reroute Table(BTAFT). This yields a backup path including its encodingthat the BFR applies to the packet header. Then, the BFRforwards the packet over another adjacency according to themodified header. The state information in the BTAFT of aBFR depends only on the number of its neighbors but not ontraversing multicast flows.

To protect a link failure, the PLR forwards an affectedpacket to its NH over a backup path that bypasses the failedlink. To protect a node failure, the PLR forwards an affected

packets to all downstream next-next-hops (DS-NNH) overpossibly several backup paths that bypass the failed node. ADS-NNH is a next-next-hop (NNH) of a PLR that receivesthe packet over the failed subtree. This concept is illustratedin Figure 3. A BFR can efficiently determine DS-NNHs usingthe BIER header and the information in the BTAFT. Whenlink and node protection is combined, the PLR forwards anaffected packet to all DS-NNHs and to the NH only if theNH is a destination of the packet. In general, the BFR cannotdifferentiate between link and node failures. Therefore, thecontroller configures the BTAFT of all BFRs such that linkprotection, node protection, or both are supported.

D. Three Implementation Options for BIER-TE FRR

We propose three different implementation options forBIER-TE FRR. For more technical details, in particular forthe HM method, we refer to our specification in [5].

1) Point-to-Point Tunneling (PPT): With PPT, the PLRreroutes BIER packets by tunneling them to appropriate NHsand DS-NNHs over unicast tunnels. They are provided bya routing underlay and bypass the failed links and nodes,respectively. E.g., MPLS [28] may be used as a routingunderlay. The provision of the tunnels possibly complicatesthe operation of the routing underlay and increases its stateinformation. Moreover, each tunnel represents an additionaladjacency and requires a separate bit in the BIER header. Ifa PLR reroutes a BIER packet over several unicast tunnels,some of them may share common links, which unnecessarilyincreases the traffic load on these links compared to the useof point-to-multipoint tunnels.

2) BIER-in-BIER Encapsulation (BBE): With BBE, thePLR identifies the set of NH and DS-NNHs to which a BIERpacket needs to be forwarded. The BTAFT helps to create aBIER-TE header towards these nodes avoiding the failed linkor node, respectively. The BIER packet is encapsulated withthat additional BIER-TE header and sent over the point-to-multipoint structure, which avoids unnecessary traffic increaseon some links. The BIER packet is decapsulated at the egressnodes of this multipath.

3) Header Modification (HM): With HM, the backup pathis encoded in the existing BIER header through application ofan AddBitmask and a ResetBitmask. Due to the forwardingmechanism of BIER-TE, this may cause duplicate packetsfor some multicast leaves. Therefore, some bits have to becleared to avoid such duplicates by applying a ResetBitmask.We explain the occurrence of duplicates by the example shownin Figure 4. There are two multicast trees: (1) A sends to Cand D, (2) B sends to C and D. If the link B→C fails, packetshave to be rerouted by B over A and D towards C. Thus,B→A, A→D and D→C are added to the header. If we do sowithout clearing additional bit, the BIER header for multicasttree (1) still contains at node B the local_decap(D). Asa consequence, the packet will be delivered to C and D.However, the packet is also directly delivered from the sourceA to D. Thus, local_decap(D) should be cleared in theheader of the rerouted packet. This is different for multicast

2017 IFIP/IEEE International Symposium on Integrated Network Management (IM2017) 233

Fig. 4: If A→D fails, clearing the local_decap(D) bit atB prevents a duplicate packet at D for multicast tree (1) butcauses packet loss at D for multicast tree (2).

tree (2) because BIER-TE sends only a single packet overeach interface and must, therefore, encode the backup pathin the BIER header. If the local_decap(D) is set beforetransmission at B towards A, the packet will be delivered toD, otherwise it will not be delivered so that D loses the packetalthough its reachability is not affected by the failure.

Thus, after the backup path is added to the packet headerthrough the AddBitmask, a ResetBitmask must be appliedbefore sending the packet to avoid duplicates which, however,may cause packet loss for other destination. The ResetBitmaskcontains both the local_decap bits of the nodes on thebackup path and their outgoing adjacencies. The latter areneeded to ensure that duplicates are not propagated into othermulticast subtree if the backup path traverses them.

Although the HM method may lose some packets in case offailures, it is of interest because it avoids the overhead of anencapsulation header, it does not extend the BIER header byadding further adjacencies, and does not require support froma routing underlay.

4) Notation: Link and node protection {L,N} can be im-plemented with any of the presented protection methods forBIER-TE. We denote them by {HM, PPT, BBE}{L,N}.

V. RESULTS

In this section we evaluate key performance metrics of thediscussed FRR mechanisms for BIER and BIER-TE. We firstpresent the networks under study and explain our evaluationmethodology. We quantify the effectiveness of the HM protec-tion variants for BIER-TE. For the other protection methodswe compare path lengths, consider resource requirements, anddiscuss state and header overheads.

A. Networks under Study

For our study, we leverage networks from the Topology Zoo[29] which contains research and commercial wide area andInternet service provider networks from mainly North Amer-ica and Europe. We simplify the networks by consecutivelyremoving all vertices that are attached to the network withonly a single edge. We consider a network to have a ringstructure if at least 60% of its nodes have node degree two,otherwise it has a mesh structure. This definition categorizes

our 220 considered networks into 118 ring topologies TR and102 mesh topologies TM .

The networks vary in size and their average numbers ofnodes are 17.7 and 21.2 for TR and TM , respectively, theaverage numbers of unidirectional links are 22.9 and 33.4.Unlike BIER, BIER-TE encodes not only nodes but also linksin the header. To that end, BIER-TE requires in ring topologieson average 40.7 bits in the header and at most 154 bits. Inmesh topologies, BIER-TE requires on average 54.7 bits andat most also 143 bits.

B. Methodology

We analyze the forwarding behavior for all BIER variantsin all 220 considered topologies. A topology is representedas a graph G = (V,E). We investigate three different sets offailure scenario. First, the failure-free scenario (FF). Second,the set of single link failures (SLF). It contains all bidirectionalsingle link failures and, thus, consists of |E| scenarios. Andthird, the set of single node failures (SNF).

The Topology Zoo does not provide traffic models or ma-trices. Therefore, we define a traffic model that is suitable fora systematic evaluation of multicast protection mechanisms.Every node in the network is sender of a multicast tree whichhas all other nodes as receivers, and any node sends the sameunit rate. There is no other traffic in the network.

We apply shortest path routing to construct multicast treesfor BIER-TE, shortest path around links for link protection,and shortest path around nodes for node protection methods.The path layout for MoFRR for BIER leverages MRT rout-ing topologies that are computed according to the lowpointalgorithm in [11].

C. Efficiency of BIER-TE Protection with Header Modification

As outlined in the previous section, BIER-TE FRR withHM may lose traffic in order to avoid duplicate packets. Inthe following, we quantify the average fraction of traffic thatmay get lost for HM in all SLF and SNF, respectively. Also theperfect FRR schemes PPT and BBE lose traffic under someconditions:• The network is not two-connected. If a critical link or

node fails, the network is disected so that senders in onepart of the network cannot reach the senders in the otherpart of the network.

• In case of node failures, traffic from or to a failed(ingress/egress) node is lost.

• If link protection is applied, traffic cannot be protectedin case of a node failure.

Therefore, we consider the traffic loss occurred for PPTN aslower bound on avoidable traffic loss.

Figure 5 shows the cumulative distribution function (CDF)of the percentage of lost traffic over all networks. Every datapoint on a curve corresponds to one network. Note that thecurves consist of many more data points than markers whichonly improve their readability.

In case of SLF, all traffic can be protected by perfect FRRmethods in more than 90% of the networks. The remaining

2017 IFIP/IEEE International Symposium on Integrated Network Management (IM2017)234

Fig. 5: CDF of the percentage of lost multicast traffic in allnetworks for SLF and SNF, respectively.

networks are not two-connected with regard to links so thatthe failure of a specific link disects the network. Anothercurve illustrates that between 7% and 50% of the traffic is lostwithout protection (No FRR). Surprisingly, HM methods causeabout the same amount of lost traffic due to subtree pruning.When HM is configured to protect against node failures, itloses a bit more traffic than without protection while when itis configured to protect against link failures, it yields slightlyless traffic loss than without protection.

In case of SNF, also the perfect FRR methods lose between2% and 50% of the traffic in case of node failures, but atmost 10% in 60% of the networks. This happens for thereasons given above and is unavoidable. Without protection,significantly more traffic is lost. The HM methods lead to evenclearly more traffic loss than without protection. The reasonfor that node failures activate more backup paths than linkfailures so that more traffic is pruned from subtrees.

These results clearly demonstrate that HM is not efficientto protect against failures, yet it can be counterproductive inparticular in case of node failures. Therefore, we exclude theHM methods from further discussions.

Nevertheless, HM can fully protect 20% – 40% of allmulticast flows against SLF in 80% of the networks. Thispotential of HM could be further elaborated in future studies,which can be of interest to protect small multicast trees insome networks with only little technological complexity.

Fig. 6: CDF of maximum path lengths H for shortest pathrouting in the failure-free case and for BIER variants in SLF.

D. Path Lengths

The path layout depends on the applied routing mechanismand impacts path length in failure-free and failure cases. Itis the same for PPT and BBE, but depends on link or nodeprotection. From previous work [26] we know that MRTsmay lead to excessive path length. Therefore, a comparisonof the new resilience mechanisms with regard to path lengthis important.

We observe that most average path lengths are betwee2 and 6 hops in the failure-free case. When we consideronly average path lengths of affected multicast flows for allSLF, we mostly observe an average path stretch between0.5 up to 1 hop for BIER-TE FRR mechanisms that bypassthe traffic on shortest paths around the failure location. Thevalues are rather small as the paths to some destinations arenot extended. Most interesting is the finding that the pathlengths for BIER MoFRR using MRTs are hardly longer thanthose for BIER-TE FRR although MRTs cause significant pathstretch when used for the protection of unicast flows. Thiscan be explained as follows. MRT routing topologies are useddifferently for multicast compared to unicast. Unicast flowsare carried over shortest paths until a failure occurs and arethen switched to a working MRT routing topology. In contrast,multicast flows are transmitted in parallel over both MRTrouting topologies without any switching. During failure-freeoperation, we consider the length of the shortest of both paths,in case of a failure, we consider the length of the working path,which may be shorter than the failed path.

The longest path prolongation in failure cases can besignificant. Figure 6 shows the CDF of the maximum pathlength over all networks in the failure-free scenario and forSLF scenarios. The longest paths are mostly twice as long asin the failure-free scenario. Again, BIER with MoFRR andBIER-TE with PPT lead to about the same maximum pathlengths. In case of node protection, path lengths are slightlylonger than in case of link protection because traffic for NNHsis explicitly bypassed around the NH which may extend thebackup path length.

Thus, MRT-based MoFRR for BIER does not cause exces-sive path length compared to shortest path routing, but backuppaths of any FRR mechanism can lead to significant path

2017 IFIP/IEEE International Symposium on Integrated Network Management (IM2017) 235

Fig. 7: CDF of the average and maximum ratios of lengths ofredundant paths for MoFRR in all networks.

stretch – not on average, but in the worst case.

E. Difference in Path Lengths for MoFRR

With MoFRR, egress routers measure the quality of thetraffic stream received over the two independent paths andchoose the traffic from the one with highest quality. In case ofa failure, they detect the failure by the fact that the signal fromone path is lost and choose the signal from the remaining path.The detection is faster and the switch-over smoother if packetsfrom both paths are received simultaneously by the egress nodeor with only little delay difference. Delay difference may resultfrom different path lengths. We quantify different path lengthsby the ratio R of the longer and shorter length of the tworedundant paths over the two MRT routing topologies.

Figure 7 shows the CDF of the average ratio Ravgand themaximum ratio Rmaxover all considered network topologies.The average ratio is about 2 for most topologies, i.e., mostlyone of the path is twice as long as the other path. Maximumratios are significantly larger. For 60% of the topologies, themaximum ratio is between 2 and 10, and for the other networkswe observe maximum ratios between 10 and 33. As a result,the delay difference of both redundant paths for MoFRR canbe significant for some leaves of some multicast trees. Thismakes failure detection more difficult than for simultaneouslyreceived signals, requires more buffer, and may cause morejitter or packet loss in case of a switch-over.

F. Load and Capacity Analysis

MoFRR for BIER duplicates all traffic at the source. PPTNfor BIER-TE sends traffic over multiple unicast tunnels to DS-NNHs in case of node protection while BBEN for BIER-TEuses multicast for that purpose. This observation calls for ananalysis of traffic loads and required transmission capacities.

We first consider the average load in the network, i.e., wesummarize the rates of all links in the network in particularfailure scenarios and average over all of them. To reportload values from differently large networks in one figure, wenormalize the average network load by the average networkload in the failure-free case for shortest path routing whichrepresents a lower bound. This yields an average network loadrelative to shortest path routing without failures that we callrelative average network load. Figure 8 shows CDFs of therelative average network loads for SLF over all networks for

Fig. 8: CDFs of relative average network loads (for differentnetwork topologies) and relative network capacities.

BIER with MoFRR, PPT for BIER-TE with link and nodeprotection, respectively, and for BBE for BIER-TE with nodeprotection. One figure provides results for mesh topologiesand another for ring topologies. BIER-TE protection methodscause in most mesh topologies an average load increase ofup to 50%, but in most ring topologies a load increase ofeven up to 100% because more traffic is affected by failuresand backup paths are longer in ring topologies than in meshtopologies. PPT with node protection causes more networkload than with link protection because the PLR replicates thetraffic and sends it over separate point-to-point tunnels to theDS-NNHs. This is unlike for BBE with node protection whichleads to hardly more network load than PPT or BBE with linkprotection. With MoFRR and BIER we observe at least twicethe relative average network load as in failure-free scenarios,and in some networks slightly larger values. The latter maybe surprising with the notion of two redundant MRT-basedmulticast trees for MoFRR in mind. But this believe is wrongas MRT routing topologies do not necessarily form trees insome networks (cf. Section III-B).

2017 IFIP/IEEE International Symposium on Integrated Network Management (IM2017)236

Second, we investigate the required network capacity tocover all SLF and SNF in a network. We compute the maxi-mum traffic load per link over all considered failure scenariosand summarize these maximum rates of all links in a network.For easier comparison, we normalize the required networkcapacity by the network load in the failure-free scenario withshortest path routing and obtain a relative required networkcapacity. The third chart of Figure 8 illustrates the CDFof the relative required network capacity in all networks.Surprisingly, BIER with MoFRR requires by far the leastcapacity in most networks. BIER-TE variants require morecapacity because traffic may be routed on very different pathsdepending on the specific failure so that high traffic rates canoccur on many links. This is different with BIER with MoFRR:in case of a failure, traffic is not rerouted but dropped so thatthe required capacity for a network equals its traffic load in thefailure-free scenario. BIER-TE with PPT and node protectionrequires very large amounts of transmission resources, mostly70% more than other BIER-TE variants and 2 – 3 times asmuch as BIER with MoFRR. The reason is that with PPT andnode protection a PLR deviates traffic towards multiple DS-NNHs by unicast instead of multicast. This causes very largerates on backup paths for which capacity must be provided.

G. State and Header Overhead Considerations

BIER and BIER-TE require additional routing tables whosecontents depends only on the topology and the routing in theunderlay but not on supported multicast traffic which is goodfor scalability.

BIER without FRR requires only a single routing plane, e.g.,shortest path routing. In contrast, BIER with MoFRR basedon MRT routing topologies requires two additional routingsplanes whose state information scales with the number ofnodes in the network. MRTs are adopted in the IETF andprovide acceptable scaling in ISP networks [11]. They increasethe routing state of normal IP routing by approximately 200%.Also the state information for BIER routing tables increaseslinearly with the number of nodes. The BIER header requiresone bit for each node in the network. The default size of theBIER header is 256 bits (32 bytes) and was sufficient for allconsidered networks. Larger headers of up to 4096 bits arepossible.

BIER-TE without FRR requires one bit per node in thenetwork and one bit per link, which causes larger headeroverhead than BIER. Moreover, a controller is needed thatconstructs the multicast trees and encodes them in the packetheaders. The FRR methods BBE and PPT have differentscaling properties. BBE requires two BIER headers whichresults into an additional 32 bytes header overhead in our studybecause all networks were small enough to encode their linksand nodes in a 256 bits header. PPT leverages point-to-pointtunnels provided by a routing underlay. These tunnels needto be encoded as forward-adjacencies in the BIER header sothat even more bits are needed. Moreover, the tunnels need tobe provided by the routing underlay, increasing its complexity.To protect against link failures, every unidirectional link in the

network has to be protected so that the number of additionalbits equals the number of unidirectional links. The networksunder study had 56 links on average and at most 176 links. Toprotect against node failures, every node needs to be protectedby additional tunnels. In the networks under study, 156 tunnelswere needed on average and at most 716 tunnels. Thus, forthe largest network, the BIER header requires for PPT 892additional bits (112 bytes) for unicast tunnels. This causesmore overhead than a small 32 bytes encapsulation header forBBE. Moreover, tunneling over the routing underlay may alsoadd some header overhead. Therefore, BBE may be the mostefficient FRR melthod for BIER-TE in terms of complexityand header overhead.

As BIER represents only nodes in the headers while BIER-TE represents both links and nodes, BIER-TE leads to largerheader overhead than BIER which may be relevant for verylarge networks so that the resulting header size may betechnically still feasible but not efficient. When fast protectionis required, the complexity of the routing underlays maybe problematic for BIER MoFRR because it requires threedifferent routing planes whereas BIER-TE with BBE does noteven depend on a routing underlay.

VI. CONCLUSION

In this work we proposed fast reroute (FRR) mechanisms forBIER and BIER-TE and compared their performance on 220network topologies. We suggested to implement MoFRR forBIER leveraging the calculation of MRT topologies to obtainredundant multicast forwarding structures. For BIER-TE FRRwe suggested three different methods: header modification(HM), unicast tunneling (PPT), and BIER-TE-in-BIER-TEtunneling (BBE).

BIER MoFRR implements 1+1 protection and, therefore,causes at least double traffic load during operation compared towithout protection. However, it requires mostly less additionalcapacity than BIER-TE-FRR because it does not reroute trafficin failure cases. Although the orignal MRT method is knownfor possibly long backup paths, path lengths resulting fromBIER with MoFRR do not exhibit significantly more pathstretch than with BIER-TE-FRR. An implementation challengemay be a possibly large difference in length of the tworedundant paths. Below the line, MoFRR leveraging MRTcalculation seems an effective FRR option for BIER.

HM is the simplest method for BIER-TE FRR in termsof technology. However, it is not effective because it losesabout the same amount of traffic as without protection just toavoid duplicate packets. In contrast, PPT and BBE can protectagainst all failures if topologically possible. PPT configuredfor node protection may lead to excessive capacity require-ments, large header overhead, and complicates the routingunderlay. As BBE avoids these drawbacks, requires only anadditional BIER header of moderate size, and does not needsupport from the routing underlay, we recommend BBE aspreferred protection method for BIER-TE.

2017 IFIP/IEEE International Symposium on Integrated Network Management (IM2017) 237

REFERENCES

[1] G. Shepherd, A. Dolganow, and arkadiy.gulko@thomsonreuters.com,“Bit Indexed Explicit Replication (BIER) Problem Statement,”Internet Engineering Task Force, Internet-Draft draft-ietf-bier-problem-statement-00, Apr. 2016, work in Progress. [Online]. Available:https://tools.ietf.org/html/draft-ietf-bier-problem-statement-00

[2] C. Bestler, N. Kumar, R. Asati, M. Chen, X. Xu, A. Dolganow,T. Przygienda, arkadiy.gulko@thomsonreuters.com, D. Robinson, andV. Arya, “BIER Use Cases,” Internet Engineering Task Force, Internet-Draft draft-ietf-bier-use-cases-04, Jan. 2017, work in Progress. [Online].Available: https://tools.ietf.org/html/draft-ietf-bier-use-cases-04

[3] I. Wijnands, E. C. Rosen, S. Aldrin, T. Przygienda, and A. Dolganow,“Multicast using Bit Index Explicit Replication,” Internet EngineeringTask Force, Internet-Draft draft-ietf-bier-architecture-05, Oct. 2016,work in Progress. [Online]. Available: https://tools.ietf.org/html/draft-ietf-bier-architecture-05

[4] T. Eckert, G. Cauchie, W. Braun, and M. Menth, “Traffic Engineeringfor Bit Index Explicit Replication BIER-TE,” Internet EngineeringTask Force, Internet-Draft draft-eckert-bier-te-arch-04, Jul. 2016, workin Progress. [Online]. Available: https://tools.ietf.org/html/draft-eckert-bier-te-arch-04

[5] ——, “Fast ReRoute (FRR) Extensions for BIER-TE,”Internet Engineering Task Force, Internet-Draft draft-eckert-bier-te-frr-00, Jul. 2016, work in Progress. [Online]. Available:https://tools.ietf.org/html/draft-eckert-bier-te-frr-00

[6] W. Zhang, G. Xue, J. Tang, and K. Thulasiraman, “Faster Algorithms forConstruction of Recovery Trees Enhancing QoP and QoS,” IEEE/ACMTrans. Netw., vol. 16, no. 3, pp. 642–655, Jun. 2008.

[7] N. Taft-Plotkin, B. Bellur, and R. Ogier, “Quality-of-Service Routingusing Maximally Disjoint Paths,” in International Workshop on Qualityof Service, 1999, pp. 119–128.

[8] Y. Guo, F. A. Kuipers, and P. V. Mieghem, “Link-disjoint Paths forReliable QoS Routing,” Int. J. Communication Systems, vol. 16, no. 9,pp. 779–798, 2003.

[9] S. Cho, T. Elhourani, and S. Ramasubramanian, “Independent DirectedAcyclic Graphs for Resilient Multipath Routing,” IEEE/ACM Transac-tions on Networking, vol. 20, no. 1, pp. 153 –162, Feb. 2012.

[10] G. Enyedi, G. Retvari, P. Szilagyi, and A. Csaszar, “IP Fast ReRoute:Lightweight Not-Via,” in IFIP-TC6 Networking Conference (Network-ing), Aachen, Germany, May 2009.

[11] G. S. Envedi, A. Csaszar, A. Atlas, C. Bowers, and A. Gopalan,“An Algorithm for Computing IP/LDP Fast Reroute Using MaximallyRedundant Trees (MRT-FRR),” RFC 7811, Jun. 2016.

[12] H. Kirrmann, M. Hansson, and P. Muri, “IEC 62439 PRP: BumplessRecovery for Highly Available, Hard Real-Time Industrial Networks,”in 2007 IEEE Conference on Emerging Technologies and FactoryAutomation (EFTA 2007), Sept 2007, pp. 1396–1399.

[13] H. Heine and O. Kleineberg, “The High-Availability Seamless Redun-dancy Protocol (HSR): Robust Fault-Tolerant Networking and LoopPrevention Through Duplicate Discard,” in IEEE International WorkshopFactory Communication Systems (WFCS), May 2012, pp. 213–222.

[14] S. Paul, K. K. Sabnani, J. C. H. Lin, and S. Bhattacharyya, “ReliableMulticast Transport Protocol (RMTP),” IEEE Journal on Selected Areasin Communications, vol. 15, no. 3, pp. 407–421, Apr 1997.

[15] B. N. Levine, D. B. Lavo, and J. J. Garcia-Luna-Aceves, “The Case forReliable Concurrent Multicasting Using Shared ACK Trees,” in ACMInternational Conference on Multimedia, 1996, pp. 365–376.

[16] Y. Ofek and B. Yener, “Reliable Concurrent Multicast From BurstySources,” in IEEE Infocom, vol. 3, Mar 1996, pp. 1433–1441.

[17] D. Frost, S. Bryant, M. Bocci, and L. Berger, “A Framework for Point-to-Multipoint MPLS in Transport Networks,” RFC 7167, Oct. 2015.

[18] S. Y. (Ed.), “RFC4655: Signaling Requirements for Point-to-MultipointTraffic-Engineered MPLS Label Switched Paths (LSPs),” Apr. 2006.

[19] R. Fabregat, Y. Donoso, B. Baran, F. Solano, and J. L. Marzo, “Multi-objective Optimization Scheme for Multicast Flows: A Survey, a Modeland a MOEA Solution,” in IFIP/ACM Latin American Conference onNetworking, 2005, pp. 73–86.

[20] Y. Nakagawa, K. Hyoudou, and T. Shimizu, “A management methodof ip multicast in overlay networks using openflow,” in Proceedings ofthe First Workshop on Hot Topics in Software Defined Networks, ser.HotSDN ’12. New York, NY, USA: ACM, 2012, pp. 91–96.

[21] X. Li and M. J. Freedman, “Scaling IP Multicast on DatacenterTopologies,” in Proceedings of the Ninth ACM Conference on EmergingNetworking Experiments and Technologies, ser. CoNEXT ’13. NewYork, NY, USA: ACM, 2013, pp. 61–72.

[22] J. Ruckert, J. Blendin, R. Hark, and D. Hausheer, “DYNSDM: Dynamicand Flexible Software-Defined Multicast for ISP Environments,” in In-ternational Conference on Network and Services Management (CNSM),Nov 2015, pp. 117–125.

[23] A. Karan, C. Filsfils, I. Wijnands, and B. Decraene, “Multicast only FastRe-Route,” https://tools.ietf.org/html/rfc7431, Aug. 2015.

[24] G. S. Envedi, A. Atlas, and C. Bowers, “An Architecture for IP/LDP FastReroute Using Maximally Redundant Trees (MRT-FRR),” RFC 7812,Jun. 2016.

[25] A. Atlas, R. Kebler, I. Wijnands, A. Csaszar, and G. Enyedi, “An Ar-chitecture for Multicast Protection Using Maximally Redundant Trees,”http://tools.ietf.org/id/draft-atlas-rtgwg-mrt-mc-arch, Mar. 2012.

[26] M. Menth and W. Braun, “Performance Comparison of Not-Via Ad-dresses and Maximally Redundant Trees (MRTs),” in IFIP/IEEE Inter-national Symposium on Integrated Network Management (IM), Ghent,Belgium, Apr. 2013.

[27] C. Filsfils, S. Previdi, B. Decraene, S. Litkowski, and R. Shakir,“Segment Routing Architecture,” Internet Engineering Task Force,Internet-Draft draft-ietf-spring-segment-routing-10, Nov. 2016, work inProgress. [Online]. Available: https://tools.ietf.org/html/draft-ietf-spring-segment-routing-10

[28] E. C. Rosen, A. Viswanathan, and R. Callon, “RFC3031: MultiprotocolLabel Switching Architecture,” Jan. 2001.

[29] S. Knight, H. X. Nguyen, N. Falkner, R. Bowden, and M. Roughan,“The Internet Topology Zoo,” IEEE Journal on Selected Areas inCommunications, vol. 29, no. 9, pp. 1765 –1775, Oct. 2011.

2017 IFIP/IEEE International Symposium on Integrated Network Management (IM2017)238