Efficient Synthesis of Network Updates

Jedidiah McClurgCU Boulder

jedidiah.mcclurg@colorado.edu

Hossein HojjatCornell Universityhojjat@cornell.edu

Pavol CernyCU Boulder

pavol.cerny@colorado.edu

Nate FosterCornell University

jnfoster@cs.cornell.edu

AbstractSoftware-defined networking (SDN) is revolutionizing the net-working industry, but current SDN programming platforms do notprovide automated mechanisms for updating global configurationson the fly. Implementing updates by hand is challenging for SDNprogrammers because networks are distributed systems with hun-dreds or thousands of interacting nodes. Even if initial and finalconfigurations are correct, naıvely updating individual nodes canlead to incorrect transient behaviors, including loops, black holes,and access control violations. This paper presents an approach forautomatically synthesizing updates that are guaranteed to preservespecified properties. We formalize network updates as a distributedprogramming problem and develop a synthesis algorithm based oncounterexample-guided search and incremental model checking.We describe a prototype implementation, and present results fromexperiments on real-world topologies and properties demonstratingthat our tool scales to updates involving over one-thousand nodes.

Categories and Subject Descriptors D.2.4 [Software En-gineering]: Software/Program Verification—Formal methods;D.2.4 [Software Engineering]: Software/Program Verification—Model checking; F.3.1 [Logics and Meanings of Programs]:Specifying and Verifying and Reasoning about Programs—Logics of programs; F.4.1 [Mathematical Logic and For-mal Languages]: Mathematical Logic—Temporal logic; C.2.3[Computer-communication Networks]: Network Operations—Network Management

Keywords synthesis, verification, model checking, LTL, networkupdates, software-defined networking, SDN

1. IntroductionSoftware-defined networking (SDN) is a new paradigm in whicha logically-centralized controller manages a collection of pro-grammable switches. The controller responds to events such astopology changes, shifts in traffic load, or new connections fromhosts, by pushing forwarding rules to the switches, which processpackets efficiently using specialized hardware. Because the con-troller has global visibility and full control over the entire network,SDN makes it possible to implement a wide variety of networkapplications ranging from basic routing to traffic engineering, data-center virtualization, fine-grained access control, etc. [6]. SDN has

Permission to make digital or hard copies of all or part of this work for personal orclassroom use is granted without fee provided that copies are not made or distributedfor profit or commercial advantage and that copies bear this notice and the full citationon the first page. Copyrights for components of this work owned by others than ACMmust be honored. Abstracting with credit is permitted. To copy otherwise, or republish,to post on servers or to redistribute to lists, requires prior specific permission and/or afee. Request permissions from permissions@acm.org.PLDI’15, June 13–17, 2015, Portland, OR, USA.Copyright c� 2015 ACM 978-1-4503-3468-6/15/06. . . $15.00.http://dx.doi.org/10.1145/2737924.2737980

been used in production enterprise, datacenter, and wide-area net-works, and new deployments are rapidly emerging.

Much of SDN’s power stems from the controller’s ability tochange the global state of the network. Controllers can set up end-to-end forwarding paths, provision bandwidth to optimize utiliza-tion, or distribute access control rules to defend against attacks.However, implementing these global changes in a running networkis not easy. Networks are complex systems with many distributedswitches, but the controller can only modify the configuration ofone switch at a time. Hence, to implement a global change, anSDN programmer must explicitly transition the network througha sequence of intermediate configurations to reach the intended fi-nal configuration. The code needed to implement this transition istedious to write and prone to error—in general, the intermediateconfigurations may exhibit new behaviors that would not arise inthe initial and final configurations.

Problems related to network updates are not unique to SDN.Traditional distributed routing protocols also suffer from anomaliesduring periods of reconvergence, including transient forwardingloops, blackholes, and access control violations. For users, theseanomalies manifest themselves as service outages, degraded per-formance, and broken connections. The research community hasdeveloped techniques for preserving certain invariants during up-dates [9, 32, 36], but none of them fully solves the problem, as theyare limited to specific protocols and properties. For example, con-sensus routing uses distributed snapshots to ensure connectivity, butonly applies to the Border Gateway Protocol (BGP) [16].

It might seem that SDN would exacerbate update-related prob-lems by making networks even more dynamic—in particular, mostcurrent platforms lack mechanisms for implementing updates in agraceful way. However, SDN offers opportunities to develop high-level abstractions for implementing updates automatically whilepreserving key invariants. The authors of B4—the controller man-aging Google’s world-wide inter-datacenter network—describe avision where: “multiple, sequenced manual operations [are] not in-volved [in] virtually any management operation” [14].

Previous work proposed the notion of a consistent update [33],which ensures that every packet is processed either using the initialconfiguration or the final configuration but not a mixture of the two.Consistency is a powerful guarantee preserving all safety proper-ties, but it is expensive. The only general consistent update mech-anism is two-phase update, which tags packets with versions andmaintains rules for the initial/final configurations simultaneously.This leads to problems on switches with limited memory and canalso make update time slower due to the high degree of rule churn.

We propose an alternative. Instead of forcing SDN operatorsto implement updates by hand (as is typically done today), or us-ing powerful but expensive mechanisms like two-phase update, wedevelop an approach for synthesizing correct update programs ef-ficiently and automatically from formal specifications. Given ini-tial and final configurations and a Linear Temporal Logic (LTL)property capturing desired invariants during the update, we either

Permission to make digital or hard copies of all or part of this work for personal orclassroom use is granted without fee provided that copies are not made or distributedfor profit or commercial advantage and that copies bear this notice and the full citationon the first page. Copyrights for components of this work owned by others than ACMmust be honored. Abstracting with credit is permitted. To copy otherwise, or republish,to post on servers or to redistribute to lists, requires prior specific permission and/or afee. Request permissions from Permissions@acm.org.

PLDI’15, June 13–17, 2015, Portland, OR, USACopyright 2015 ACM 978-1-4503-3468-6/15/06...$15.00http://dx.doi.org/10.1145/2737924.2737980

Co

nsi

st

ent *

Complete * Well D

ocu

mented * Easy to

R

euse

*

* Evaluated *

PLD

I *

Artifact * A

EC

314

C1 C2

A1 A2 A3 A4

T1 T2 T3 T4

H1 H2 H3 H4

Figure 1: Example topology.

0 2 4 60

33%

66%

100%

Time (s)(a)

Prob

esR

ecei

ved

T1 T2 T3 T4 A1 A2 A3 A4 C1 C20

1X

2X

(b) Switch ID

Rul

eO

verh

ead

Figure 2: Example naıve (blue/solid-line), two-phase (green/solid-bar), and or-dering (red/dashed) updates: (a) probes received; (b) per-switch rule overhead.

generate an SDN program that implements the initial-to-final tran-sition while ensuring that the property is never violated, or fail if nosuch program exists. Importantly, because the synthesized programis only required to preserve the specified properties, it can leveragestrategies that would be ruled out in other approaches. For example,if the programmer specifies a trivial property, the system can updateswitches in any order. However, if she specifies a more complexproperty (e.g. firewall traversal) then the space of possible updatesis more constrained. In practice, our synthesized programs requireless memory and communication than competing approaches.

Programming updates correctly is challenging due to the con-currency inherent in networks—switches may interleave packet andcontrol message processing arbitrarily. Hence, programmers mustcarefully consider all possible event orderings, inserting synchro-nization primitives as needed. Our algorithm works by searchingthrough the space of possible sequences of individual switch up-dates, learning from counterexamples and employing an incremen-tal model checker to re-use previously computed results. Our modelchecker is incremental in the sense that it exploits the loop-freedomof correct network configurations to enable efficient re-checkingof properties when the model changes. Because the synthesis al-gorithm poses a series of closely-related model checking ques-tions, the incrementality yields enormous performance gains onreal-world update scenarios.

We have implemented the algorithm and heuristics to furtherspeed up synthesis and eliminate spurious synchronization. Wehave interfaced the tool with Frenetic [8], synthesized updates forOpenFlow switches, and used our system to process actual trafficgenerated by end-hosts. We ran experiments on a suite of real-worldtopologies, configurations, and properties—our results demonstratethe effectiveness of synthesis, which scales to over one-thousandswitches, and incremental model checking, which outperforms apopular symbolic model checker used in batch mode, and a state-of-the-art network model checker used in incremental mode.

In summary, the main contributions of this paper are:• We investigate using synthesis to automatically generate net-

work updates (§2).• We develop a simple operational model of SDN and formalize

the network update problem precisely (§3).• We design a counterexample-guided search algorithm that

solves instances of the network update problem, and prove thisalgorithm to be correct (§4).

• We present an incremental LTL model checker for loop-freemodels (§5).

• We describe an OCaml implementation with backends to third-party model checkers and conduct experiments on real-worldnetworks and properties, demonstrating strong performance im-provements (§6). †

Overall, our work takes a challenging network programming prob-lem and automates it, yielding a powerful tool for building dynamicSDN applications that ensures correct, predictable, and efficientnetwork behavior during updates.

2. OverviewTo illustrate key challenges related to network updates, considerthe network in Figure 1. It represents a simplified datacenter topol-ogy [1] with core switches (C1 and C2), aggregation switches (A1to A4), top-of-rack switches (T1 to T4), and hosts (H1 to H4). Ini-tially, we configure switches to forward traffic from H1 to H3 alongthe solid/red path: T1-A1-C1-A3-T3. Later, we wish to shift trafficfrom the red path to the dashed/green path, T1-A1-C2-A3-T3 (per-haps to take C1 down for maintenance). To implement this update,the operator must modify forwarding rules on switches A1 and C2,but note that certain update sequences break connectivity—e.g., up-dating A1 followed by C2 causes packets to be forwarded to C2 be-fore it is ready to handle them. Figure 2(a) demonstrates this with asimple experiment performed using our system. Using the Mininetnetwork simulator and OpenFlow switches, we continuously sentICMP (ping) probes during a “naıve” update (blue/solid line) andthe ordering update synthesized by our tool (red/dashed line). Withthe naıve update, 100% of the probes are lost during an interval,while the ordering update maintains connectivity.

Consistency. Previous work [33] introduced the notion of a con-sistent update and also developed general mechanisms for ensuringconsistency. An update is said to be consistent if every packet isprocessed entirely using the initial configuration or entirely usingthe final configuration, but never a mixture of the two. For exam-ple, updating A1 followed by C2 is not consistent because packetsfrom H1 to H3 might be dropped instead of following the red pathor the green path. One might wonder whether preserving consis-tency during updates is important, as long as the network eventu-ally reaches the intended configuration, since most networks onlyprovide best-effort packet delivery. While it is true that errors canbe masked by protocols such as TCP when packets are lost, thereis growing interest in strong guarantees about network behavior.For example, consider a business using a firewall to protect internalservers, and suppose that they decide to migrate their infrastructureto a virtualized environment like Amazon EC2. To ensure that thisnew deployment is secure, the business would want to maintain thesame isolation properties enforced in their home office. However, abest-effort migration strategy that only eventually reaches the tar-get configuration could step through arbitrary intermediate states,some of which may violate this property.

Two-Phase Updates. Previous work introduced a generalconsistency-preserving technique called two-phase update [33].The idea is to explicitly tag packets upon ingress and use theseversion tags to determine which forwarding rules to use at eachhop. Unfortunately, this has a significant cost. During the transition,switches must maintain forwarding rules for both configurations,effectively doubling the memory requirements needed to completethe update. This is not always practical in networks where theswitches store forwarding rules using ternary content-addressable

† The PLDI 2015 Artifact Evaluation Committee (AEC) found that our tool“met or exceeded expectations.”

315

memories (TCAM), which are expensive and power-hungry. Fig-ure 2(b) shows the results of another simple experiment where wemeasured the total number of rules on each switch: with two-phaseupdates, several switches have twice the number of rules comparedto the synthesized ordering update. Even worse, it takes a non-trivial amount of time to modify forwarding rules—sometimes onthe order of 10ms per rule [15]! Hence, because two-phase updatesmodify a large number of rules, they can increase update latency.These overheads can make two-phase updates a non-starter.

Ordering Updates. Our approach is based on the observation thatconsistent (two-phase) updates are overkill in many settings. Some-times consistency can be achieved by simply choosing a correctorder of switch updates. We call this type of update an orderingupdate. For example, to update from the red path to the green path,we can update C2 followed by A1. Moreover, even when we can-not achieve full consistency, we can often still obtain sufficientlystrong guarantees for a specific application by carefully updatingthe switches in a particular order. To illustrate, suppose that insteadof shifting traffic to the green path, we wish to use the blue (dashed-and-dotted) path: T1-A2-C1-A4-T3. It is impossible to transitionfrom the red path to the blue path by ordering switch updates with-out breaking consistency: we can update A2 and A4 first, as theyare unreachable in the initial configuration, but if we update T1 fol-lowed by C1, then packets can traverse the path T1-A2-C1-A3-T3,while if we update C1 followed by T1, then packets can traverse thepath T1-A1-C1-A4-T3. Neither of these alternatives is allowed in aconsistent update. This failure to find a consistent update hints at asolution: if we only care about preserving connectivity between H1and H3, then either path is actually acceptable. Thus, either updat-ing C1 before T1, or T1 before C1 would work. Hence, if we relaxstrict consistency and instead provide programmers with a way tospecify properties that must be preserved across an update, then or-dering updates will exist in many situations. Recent work [15, 25]has explored ordering updates, but only for specific properties likeloop-freedom, blackhole-freedom, drop-freedom, etc. Rather thanhandling a fixed set of “canned” properties, we use a specificationlanguage that is expressive enough to encode these properties andothers, as well as conjunctions/disjunctions of properties—e.g. en-forcing loop-freedom and service-chaining during an update.

In-flight Packets and Waits. Sometimes an additional synchro-nization primitive is needed to generate correct ordering updates(or correct two-phase updates, for that matter). Suppose we wantto again transition from the red path to blue one, but in addition topreserving connectivity, we want every packet to traverse either A2or A3 (this scenario might arise if those switches are actually mid-dleboxes which scrub malicious packets before forwarding). Nowconsider an update that modifies the configurations on A2, A4, T1,C1, in that order. Between the time that we update T1 and C1, theremight be some packets that are forwarded by T1 before it is up-dated, and are forwarded by C1 after it is updated. These packetswould not traverse A2 or A3, and so indicate a violation of the spec-ification. To fix this, we can simply pause after updating T1 untilany packets it previously forwarded have left the network. We thusneed a command “wait” that pauses the controller for a sufficientperiod of time to ensure that in-flight packets have exited the net-work. Hence, the correct update sequence for this example wouldbe as above, with a “wait” between T1 and C1. Note that two-phase updates also need to wait, once per update, since we mustensure that all in-flight packets have left the network before delet-ing the old version of the rules on switches. Other approaches havetraded off control-plane waiting for stronger consistency, e.g. [24]performs updates in “rounds” that are analogous to “wait” com-mands, and Consensus Routing [16] relies on timers to obtain wait-like functionality. Note that the single-switch update time can be

on the order of seconds [15, 22], whereas typical datacenter transittime (the time for a packet to traverse the network) is much lower,even on the order of microseconds [3]. Hence, waiting for in-flightpackets has a negligible overall effect. In addition, our reachability-based heuristic eliminates most waits in practice.

Summary. This paper presents a sound and complete algorithmand implementation for synthesizing a large class of ordering up-dates efficiently and automatically. The updates we generate ini-tially modify each switch at most once and “wait” between updatesto switches, but a heuristic removes an overwhelming majority ofunnecessary waits in practice. For example, in switching from thered path to the blue path (while preserving connectivity from H1 toH3, and making sure that each packet visits either A3 or A4), ourtool produces the following sequence: update A2, then A4, thenT1, then wait, then update C1. The resulting update can be exe-cuted using the Frenetic SDN platform and used with OpenFlowswitches—e.g., we generated Figure 2 (a-b) using our tool.

3. Preliminaries and Network ModelTo facilitate precise reasoning about networks during updates, wedevelop a formal model in the style of Chemical Abstract Ma-chine [4]. This model captures key network features using a simpleoperational semantics. It is similar to the one used by [11], but isstreamlined to model features most relevant to updates.

3.1 Network ModelBasic structures. Each switch sw , port pt , or host h is identifiedby a natural number. A packet pkt is a record of fields containingheader values such as source and destination address, protocol type,and so on. We write {f1; . . . ; fk} for the type of packets havingfields f

i

and use “dot” notation to project fields from records. Thenotation {r with f = v} denotes functional update of r.f .

Forwarding Tables. A switch configuration is defined in termsof forwarding rules, where each rule has a pattern pat specifiedas a record of optional packet header fields and a port, a list ofactions act that either forward a packet out a given port (fwd pt)or modify a header field (f :=n), and a priority that disambiguatesrules with overlapping patterns. We write {pt?; f1?; . . . ; fk?} forthe type of patterns, where the question mark denotes an optiontype. A set of such rules ruls forms a forwarding table tbl . Thesemantic function [[tbl ]] maps packet-port pairs to multisets of suchpairs, finding the highest-priority rule whose pattern matches thepacket and applying the corresponding actions. If there are multiplematching rules with the same priority, the function is free to pickany of them, and if there are no matching rules, it drops the packet.The forwarding tables collectively define the network’s data plane.

Commands. The control plane modifies the data plane by is-suing commands that update forwarding tables. The command(sw , tbl) replaces the forwarding table on switch sw with tbl (wecall this a switch-granularity update). We model this command asan atomic operation (it can be implemented with OpenFlow bun-dles [31]). Sometimes switch granularity is too coarse to find an up-date sequence, in which case one can update individual rules (rule-granularity). Our tool supports this finer-grained mode of opera-tion, but since it is not conceptually different from switch granular-ity, we frame most of our discussion in terms of switch-granularity.

To synchronize updates involving multiple switches, we includea wait command. In the model, the controller maintains a natural-number counter known as the current epoch ep. Each packet isannotated with the epoch on ingress. The control command incrincrements the epoch so that subsequent incoming packets are an-notated with the next epoch, and flush blocks the controller until allpackets annotated with the previous epoch have exited the network.

316

Switch sw 2 NPort pt 2 NHost h 2 NPriority pri 2 NEpoch ep 2 NField f ::= src | dst | typ | ..

Packet pkt ::= {f1; ..; fk

}Pair pr ::= (pkt , pt)Pattern pat ::= {pt?; f1?; ..; f

k

?}Action act ::= fwd pt | f :=n

Rule rul ::= {pri ; pat ; acts}Table tbl ::= ruls

Location loc ::= h | (sw , pt)Command cmd ::= (sw , tbl) | incr | flushSwitch S ::= {sw ; tbl ; prs}Link L ::= {loc; pkts; loc0}Controller C ::= {cmds; ep}Element E ::= S | L | C

Data PlaneL.loc = h L.loc0 = (sw 0

, pt 0) L.pkts = pkts C.ep = ep

C, L �! C, {L with pkts = pktep ::pkts}IN

L.loc = (sw , pt) L.loc0 = h L.pkts = (pktep ::pkts)

L

(sw,pt,pkt)�������! {L with pkts = pkts}OUT

L.loc

0 = (sw , pt) L.pkts = (pktep ::pkts) S.sw = sw [[S.tbl ]](pkt , pt) = {(pkt1, pt1), .., (pktn, ptn)}

L, S

(sw,pt,pkt)�������! {L with pkts = pkts}, {S with prs = S.prs ] {(pktep1 , pt1), .., (pktepn

, ptn

)}}PROCESS

S.sw = sw S.prs = {(pktep , pt)} ] prs L.loc = (sw , pt)

S, L �! {S with prs = prs}, {L with pkts = L.pkts@[pktep ]}FORWARD

Control Plane and Abstract MachineC.cmds = ((sw , tbl)::cmds) S.sw = sw

C, S �! {C with cmds = cmds}, {S with tbl = tbl}UPDATE

C.cmds = (incr ::cmds)

C �! {C with cmds = cmds; ep = C.ep + 1}INCR

C.cmds = (flush::cmds) ep(S1, .., Sk

, L1, .., Lm

) = C.ep

S1, .., Sk

, L1, .., Lm

, C �! S1, .., Sk

, L1, .., Lm

, {C with cmds = cmds}FLUSH

Es1o�! Es01

Es1 ] Es2o�! Es01 ] Es2

CONGRUENCE

Figure 3: Network model.

We introduce a command wait defined as incr ;flush . The epochsare included in our model solely to enable reasoning. They do notneed to be implemented in a real network—all that is needed is amechanism for blocking the controller to allow a flush of all packetscurrently in the network. For example, given a topology, one couldcompute a conservative delay based on the maximum hop count,and then implement wait by sleeping, rather than synchronizingwith each switch. Note that we implicitly assume failure-freedomand packet-forwarding fairness of switches and links, i.e. there isan upper bound on each element’s packet-processing time.

Elements. The elements E of the network model includeswitches S

i

, links Lj

, and a single controller element C, and anetwork N is a tuple containing these. Each switch S

i

is encodedas a record comprising a unique identifier sw , a table tbl of pri-oritized forwarding rules, and a multiset prs of pairs (pkt , pt) ofbuffered packets and the ports they should be forwarded to respec-tively. Each link L

j

is represented by a record consisting of twolocations loc and loc0 and a list of queued packets pkts , where alocation is either a host or a switch-port pair. Finally, controller Cis represented by a record containing a list of commands cmds andan epoch ep. We assume that commands are totally-ordered. Thecontroller can ensure this by using OpenFlow barrier messages.

Operational semantics. Network behavior is defined by small-step operational rules in Figure 3. These define interactions be-tween subsets of elements, based on OpenFlow semantics [28].States of the model are given by multisets of elements. We write{x} to denote a singleton multiset, and m1 ]m2 for the union ofmultisets m1 and m2. We write [x] for a singleton list, and l1@l2for concatenation of l1 and l2. Each transition N

o

�! N 0 is anno-tated, with o being either an empty annotation, or an observation(sw , pt , pkt) indicating the location and packet being processed.

The first rules describe date-plane behavior. The IN rule admitsarbitrary packets into the network from a host, stamping themwith the current controller epoch. The OUT rule removes a packetbuffered on a link adjacent to a host. PROCESS processes a singlepacket on a switch, finding the highest priority rule with matchingpattern, applying the actions of that rule to generate a multiset ofpackets, and adding those packets to the output buffer. FORWARD

moves a packet from a switch to the adjacent link. The final rulesdescribe control-plane behavior. UPDATE replaces the table on asingle switch. INCR increments the epoch on the controller, andFLUSH blocks the controller until all packets in the network areannotated with at least the current epoch (ep(Es) denotes thesmallest annotation on any packet in Es). Finally, CONGRUENCE,allows any sub-collection of network elements to interact.

3.2 Network Update ProblemIn order to define the network update problem, we need to firstdefine traces of packets flowing through the network.

Packet traces. Given a network N , our operational rules can gen-erate sequences of observations. However, the network can processmany packets concurrently, and we want observations generated bya single packet. We define a successor relation v for observations(Definition 7, Appendix A). Intuitively o

ep

v o0 if the network candirectly produce the packet in o0 by processing o in the epoch ep.

Definition 1 (Single-Packet Trace). Let N be a network. A se-

quence (o1 · · · ol) is a single-packet trace of N if No

01�! . . .

o

0k�!

Nk

such that (o1 · · · ol) is a subsequence of (o01 · · · o0k

) for which• every observation is a successor of the preceding observation

in monotonically increasing epochs, and• if o1 = o0

j

= (sw , pt , pkt), then 9o0i

2 {o01, · · · , o0j�1} such

that the o0i

transition is an IN moving pkt from host to (sw , pt)and none of o0

i

, · · · , o0j�1 is a predecessor of o1, and

• the ol

transition is an OUT terminating at a host.

Intuitively, single-packet traces are end-to-end paths through thenetwork. We write T (N) for the set of single-packet traces gener-ated by N . A trace (o1 · · · ok) is loop-free if o

i

6= oj

for all distincti and j between 1 and k. We consider only loop-free traces, sincea network that forwards packets around a loop is generally con-sidered to be misconfigured. In the worst case, forwarding loopscan cause a packet storm, wasting bandwidth and degrading perfor-mance. Our tool automatically detects/rejects such configurations.

LTL formulas. Many important network properties can be under-stood by reasoning about the traces that packets can take through

317

the network. For example, reachability requires that all packetsstarting at src eventually reach dst. Temporal logics are an ex-pressive and well-studied language for specifying such trace-basedproperties. Hence, we use Linear Temporal Logic (LTL) to describetraces in our network model. Let AP be atomic propositions thattest the value of a switch, port, or packet field: f

i

= n. We callelements of the set 2AP traffic classes. Intuitively, each traffic classT identifies a set of packets that agree on the values of particularheader fields. An LTL formula ' in negation normal form (NNF)is either true, false, atomic proposition p in AP , negated propo-sition ¬p, disjunction '1 _ '2, conjunction '2 ^ '2, next X',until '1U'2, or release '1R'2, where '1 and '2 are LTL for-mulas in NNF. The operators F and G can be defined using otherconnectives. Since (finite) single-packet traces can be viewed as in-finite sequences of packet observations where the final observationrepeats indefinitely, the semantics of the LTL formulas can be de-fined in a standard way over traces. We write t |= ' to indicatethat the single-packet trace t satisfies the formula ' and T |= 'to indicate that t |= ' for each t in T . Given a network N and aformula ', we write N |= ' if T (N) |= '.

Problem Statement. Recall that our network model includescommands for updating a single switch, incrementing the epoch,and waiting until all packets in the preceding epoch have beenflushed from the network. At a high-level, our goal is to identifya sequence of commands to transition the network between config-urations without violating specified invariants. First, we need a bitof notation. Given a network N , we write N [sw tbl ] for theswitch update obtained by updating the forwarding table for switchsw to tbl . We call N static if C.cmds is empty. If static networksN1, Nn

have the same traces T (N1) = T (Nn

), then we say theyare trace-equivalent, N1 ' N

n

.

Definition 2 (Network Update). Let N1 be a static network. Acommand sequence cmds induces a sequence N1, . . . , Nn

of staticnetworks if c1 · · · cn�1 are the update commands in cmds , and foreach c

i

= (sw , tbl), we have Ni

[sw tbl ] ' Ni+1.

We write N1cmds�! N

n

if there exists such a sequence of staticnetworks induced by cmds which ends with N

n

.We call N stable if all packets in N are annotated with the same

epoch. Intuitively, a stable network is one with no in-progress up-date, i.e. any preceding update command was finalized with a wait.Consider the set of unconstrained single-packet traces generated byremoving the requirement that traces start at an ingress (see Defi-nition 8, Appendix A). This includes T (N) as well as traces ofpackets initially present in N . We call this ¯

T (N), and note that fora stable network N , ¯T (N) is equal to T (N).

Definition 3 (Update Correctness). Let N be a stable static net-work and let ' be an LTL formula. The command sequence cmdsis correct with respect to N and ' if ˆN |= � where ˆN is obtainedfrom N by setting C.cmds = cmds .

A network configuration is a static network which contains nopackets. We can now present the problem statement.

Definition 4 (Update Synthesis Problem). Given stable static net-work N , network configuration N 0, and LTL specification ', con-struct a sequence of commands cmds such that (i) N

cmds�! N 00

where N 00' N 0, and (ii) cmds is correct with respect to '.

3.3 Efficiently Checking Network PropertiesTo facilitate efficient checking of network properties via LTL modelcheckers, we show how to model a network as a Kripke structure.

Kripke structures. A Kripke structure is a tuple (Q,Q0, �,�),where Q is a finite set of states, Q0 ✓ Q is a set of initial states,

� ✓ Q ⇥ Q is a transition relation, and � : Q ! 2

AP labelseach state with a set of atomic propositions drawn from a fixed setAP . A Kripke structure is complete if every state has at least onesuccessor. A state q 2 Q is a sink state if for all states q0, �(q, q0)implies that q = q0, and we call a Kripke structure DAG-like ifthe only cycles are self-loops on sink states. In this paper, we willconsider complete and DAG-like Kripke structures. A trace t is aninfinite sequence of states, t0t1 . . . such that 8i � 0 : �(t

i

, ti+1).

Given a trace t, we write ti for the suffix of t starting at the i-thposition—i.e., ti = t

i

ti+1 . . .. Given a set of traces T , we let T i

denote the set {ti | t 2 T }. Given a state q of a Kripke structureK, let traces

K

(q) be the set of traces of K starting from q andsucc

K

(q) be the set of states defined by q0 2 succK

(q) if and onlyif �(q, q0). We will omit the subscript K when it is clear form thecontext. A Kripke structure K = (Q,Q0, �,�) satisfies an LTLformula ' if for all states q0 2 Q0 we have that traces(q0) |= '.

Network Kripke structures. For every static N , we can gener-ate a Kripke structure K(N) containing traces which correspondaccording to an intuitive trace relation . (Definition 9, 10, Ap-pendix A). We currently do not reason about packet modification,so the Kripke structure has disjoint parts corresponding to the traf-fic classes. It is straightforward to enable packet modification, byadding transitions between the parts of the Kripke structure, but weleave this for future work. We now show that the generated Kripkestructure faithfully encodes the network semantics.

Lemma 1 (Network Kripke Structure Soundness). Let N be astatic network and K = K(N) a network Kripke structure. Forevery single-packet trace t in T (N) there exists a trace t0 of Kfrom a start state such that t . t0, and vice versa.

This means that checking LTL over single-packet traces can beperformed via LTL model-checking of Kripke structures.

Checking network configurations. One key challenge arises be-cause the network is a distributed system. Packets can “see” an in-consistent configuration (some switches updated, some not), andreasoning about possible interleavings of commands becomes in-tractable in this context. We can simplify the problem if we ensurethat each packet traverses at most one switch that was updated afterthe packet entered the network.

Definition 5 (Careful Command Sequences). A sequence of com-mands (cmd1 · · · cmd

n

) is careful if every pair of switch updatesis separated by a wait command.

In the rest of this paper, we consider careful command sequences,and develop a sound and complete algorithm that finds them ef-ficiently. Section 4 describes a technique for removing wait com-mands that works well in practice, but we leave optimal wait re-moval for future work. Recall that T (N) denotes the sequence ofall traces that a packet could take through the network, regardless ofwhen the commands in N.cmds are executed. This is a superset ofthe traces induced by each static N

i

in a solution to the network up-date problem. However, if cmds is careful, then each packet onlyencounters a single configuration, allowing the correctness of thesequence to be reduced to the correctness of each N

i

.

Lemma 2 (Careful Correctness). Let N be a stable network withC.cmds careful and let ' be an LTL formula. If cmds is carefuland N

i

|= � for each static network in any sequence induced bycmds , then cmds is correct with respect to '.

In Lemmas 5 and 6 (Appendix A), we show that checking theunique sequence of network configurations induced by cmds isequivalent to the above. Next we will develop a sound and com-plete algorithm that solves the update synthesis problem for carefulsequences by checking configurations.

318

Procedure ORDERUPDATE(Ni

,Nf

,')Input: Initial static network N

i

, final static configuration Nf

, formula '.Output: update sequence L, or error ✏ if no update sequence exists1: W false . Formula encoding wrong configurations.2: V false . Formula encoding visited configurations.3: (ok, L) DFSFORORDER(N

i

,K(Ni

),?,',�0)4: if ok then return L

5: else return ✏ . Failure—no update exists.Procedure DFSFORORDER(N ,K,s,',�)Input: Static network N and Kripke structure K, next switch to update s,

formula ', and labeling �.Output: Boolean ok if a correct update exists; correct update sequence L

6: if N |= V _W then return (false, [ ])

7: if s = ? then (ok, cex,�) modelCheck(K,')8: else9: (N ,K, S) swUpdate(N , s)

10: (ok, cex,�) incrModelCheck(K,', S,�)

11: V V _makeFormula(N )12: if ¬ok then13: W W _makeFormula(cex)14: return (false, [ ])

15: if N = Nf

then return (true, [s])

16: for s

0 2 possibleUpdates(N ) do17: (ok, L) DFSFORORDER(N ,K, s

0,',�)

18: if ok then return (true, (upd s

0) :: wait :: L)

19: return (false, [ ])

Figure 4: ORDERUPDATE Algorithm.

4. Update Synthesis AlgorithmThis section presents a synthesis algorithm that searches throughthe space of possible solutions, using counterexamples to detectwrong configurations and exploiting several optimizations.

4.1 Algorithm DescriptionORDERUPDATE (Figure 4) returns a simple sequence of updates(one in which each switch appears at most once), or fails if no suchsequence exists. Note that we could broaden our simple definition,e.g. k-simple, where each switch appears at most k times, but wehave found the above restriction to work well in practice. The coreprocedure is DFSFORORDER, which manages the search and in-vokes the model checker (we use DFS because we expect commonproperties/configurations to admit many update sequences). It at-tempts to add a switch s to the current update sequence, yielding anew network configuration. We maintain two formulas, V and W ,tracking the set of configurations that have been visited so far, andthe set of configurations excluded by counterexamples.

To check whether all packet traces in this configuration satisfythe LTL property ', we use our (incremental) model checkingalgorithm (discussed in Section 5). First, we call a full check of themodel (line 7). The model checker labels the Kripke structure nodeswith information about what formulas hold for paths starting at thatstate. The labeling (stored in �) is then re-used in the subsequentmodel checking calls for related Kripke structures (line 10). Theparameters passed in the incremental model checking call are:updated Kripke structure K, specification ', set of nodes S inK whose transition function has changed by the update of theswitch s, and correct labeling � of the Kripke structure before theupdate. Note that before the initial model checking, we convertthe network configuration N to a Kripke structure K. The updateof K is performed by a function swUpdate that returns a triple(N 0, S,K0

), where N 0 is the new static network, K0 is the updatedKripke structure obtained as K(N 0

), and S is the set of nodes thathave different outgoing transitions in K0.

If the model checker returns true, then N is safe and the searchproceeds recursively, after adding (upd s0) to the current sequence

of commands. If the model checker returns false, the search back-tracks, using the counterexample-learning approach below.

4.2 OptimizationsWe now present optimizations improving synthesis (pruning withcounterexamples, early search termination), and improving effi-ciency of synthesized updates (wait removal).

A. Counterexamples. Counterexample-based pruning learns net-work configurations that do not satisfy the specification to avoidmaking future model checking calls that are certain to fail. Thefunction makeFormula(cex ) (Line 13) returns a formula repre-senting the set of switches that occurred in the counterexampletrace cex, with flags indicating whether each switch was updated.This allows equivalent future configurations to be eliminated with-out invoking the model checker. Recall the red-green example inSection 2 and suppose that we update A1 and then C2. At the inter-mediate configuration obtained by updating just A1, packets will bedropped at C2, and the specification (H1-H3 connectivity) will notbe satisfied. The formula for the unsafe set of configurations thathave A1 updated and C2 not updated will be added to W . In prac-tice, many counterexamples are small compared to network size,and this greatly prunes the search space.

B. Early search termination. The early search termination op-timization speeds up termination of the search when no (switch-granularity) update sequence is possible. Recall how we use coun-terexamples to prune configurations. With similar reasoning, wecan use counterexamples for pruning possible sequences of up-dates. Consider a counterexample trace which involves three nodesA,B,C, with A updated, B updated, and C not updated. This canbe seen as requiring that C must be updated before A, or C mustbe updated before B. Early search termination involves collectingsuch constraints on possible updates, and terminating if these con-straints taken together form a contradiction. In our tool, this is doneefficiently using an (incremental) SAT solver. If the solver deter-mines that no update sequence is possible, the search terminates.For simplicity, early search termination is not shown in Figure 4.

C. Wait removal. This heuristic eliminates waits that are un-necessary for correctness. Consider an update sequence L =

cmd0cmd1 · · · cmdn

, and consider some switch update cmdk

=

(upd s). In the configuration resulting from executing the sequencecmd0cmd1 · · · cmd

k�1, if the switch s cannot possibly receivea packet which passed through some switch s0 before an updatecmd

j

=(upd s0) where j < k, then we can update s without wait-ing. Thus, we can remove some unnecessary waits if we can main-tain reachability-between-switches information during the update.Wait removal is not shown in Figure 4, but in our tool, it operates asa post-processing pass once an update sequence is found. In prac-tice, this removes a majority of unnecessary waits (see § 6).

4.3 Formal PropertiesThe following two theorems show that our algorithm is sound forcareful updates, and complete if we limit our search to simpleupdate sequences (see Appendix B for proofs).

Theorem 1 (Soundness). Given initial network Ni

, final configu-ration N

f

, and LTL formula ', if ORDERUPDATE returns a com-mand sequence cmds , then N

i

cmds�! N 0 s.t. N 0

' Nf

, and cmdsis correct with respect to ' and N

i

.

Theorem 2 (Completeness). Given initial network Ni

, final con-figuration N

f

, and specification ', if there exists a simple, carefulsequence cmds with N

i

cmds�! N 0 s.t. N 0

' Nf

, then ORDERUP-DATE returns one such sequence.

319

5. Incremental Model CheckingWe now present an incremental algorithm for model checkingKripke structures. This algorithm is central to our synthesis tool,which invokes the model checker on many closely related struc-tures. The algorithm makes use of the fact that the only cycles in theKripke structure are self-loops on sink nodes—something that istrue of structures encoding loop-free network configurations—andre-labels the states of a previously-labeled Kripke structure with the(possibly different) formulas that hold after an update.

5.1 State LabelingWe begin with an algorithm for labeling states of a Kripke struc-ture with sets of formulas, following the approach of [39] (WVS)and [37]. The WVS algorithm translates an LTL formula ' into alocal automaton and an eventuality automaton. The local automa-ton checks consistency between a state and its predecessor, andhandles labeling of all formulas except '1 U '2, which is checkedby the eventuality automaton. The two automata are composed intoa single Buchi automaton whose states correspond to subsets of theset of subformulas of ' and their negations. Hence, we label eachKripke state by a set L of sets of formulas such that if a state q islabeled by L, then for each set of formulas S in L, there exists atrace t starting from q satisfying all the formulas in S.

We now describe state labeling precisely. Let ' be an LTLformula in NNF. The extended closure of ', written ecl('), is theset of all subformulas of ' and their negations:• true 2 ecl(')• ' 2 ecl(')• If 2 ecl('), then ¬ 2 ecl(')

(we identify with ¬¬ , for all ).• If '1 _ '2 2 ecl('), then '1 2 ecl(') and '2 2 ecl(').• If '1 ^ '2 2 ecl('), then '1 2 ecl(') and '2 2 ecl(').• If X '1 2 ecl('), then '1 2 ecl(').• If '1 U '2 2 ecl('), then '1 2 ecl(') and '2 2 ecl(')• If '1 R'2 2 ecl('), then '1 2 ecl(') and '2 2 ecl(').

A subset M ⇢ ecl(') of the extended closure is said to bemaximally consistent if it contains true and is simultaneouslyclosed and consistent under boolean operations:• true 2M• 2M iff ¬ 62M (we identify with ¬¬ , for all )• '1 _ '2 2M iff ('1 2M or '2 2M )• '1 ^ '2 2M iff ('1 2M and '2 2M )

Likewise, the relation follows(M1,M2) captures the notion of suc-cessor induced by LTL’s temporal operators, lifted to maximally-consistent sets. We say follows(M1,M2) holds if and only if all ofthe following hold:• X '1 2M1 iff '1 2M2

• '1 U '2 2M1 iff�'2 2M1 _ ('1 2M1 ^ '1 U '2 2M2)

�

• '1 R'2 2M1 iff�'1 2M1 _ ('2 2M1 ^ '1 R'2 2M2)

�

Given a trace t and a maximally-consistent set M , we write t |= Mif and only if for all 2M , we have t |= .

For the rest of this section, we fix a Kripke structure K =

(Q,Q0, �,�), a state q in Q, an LTL formula ' in NNF, and amaximally-consistent set M ⇢ ecl(').

To compute the label of a state q, there are two cases dependingon whether it is a sink state or a non-sink state. If q is a sink state,the function HoldsSink(q,M) computes a predicate that is true ifand only if, for all 2 M and the unique trace t starting from q,we have t |= . More formally, HoldsSink(q,M) is defined to be(8 2 M : Holds0 (q, )), where Holds0 is defined as in Figure5. The function Holds0 computes a predicate that is true if andonly if holds at q. For example, Holds0 (q,�1 U �2) is definedas Holds0 (q,�2) because the only transition from q is a self-loop.

Holds0 (q, p) = q |= pHolds0 (q,¬p) = q 6|= p

Holds0 (q,�1 ^ �2) = Holds0 (q,�1) ^Holds0 (q,�2)

Holds0 (q,�1 _ �2) = Holds0 (q,�1) _Holds0 (q,�2)

Holds0 (q,X�) = Holds0 (q,�)Holds0 (q,�1 U �2) = Holds0 (q,�2)

Holds0 (q,�1 R �2) = Holds0 (q,�1) _Holds0 (q,�2)

Figure 5: The Holds0 function

For the second case, suppose q is a non-sink state. If we aregiven a labeling for succ

K

(q) (the successors of the node q), wecan extend it to a labeling for q. Let V ✓ Q be a set of vertices. Afunction labGr

K

is a correct labeling of K with respect to ' andV if for every v 2 V , it returns a set L of maximally consistentsets such that (a) M 2 L if and only if M ⇢ ecl('), and (b)there exists a trace t in traces(v) such that t |= M . Suppose thatlabGr

K

is a correct labeling of K with respect to ' and succK

(q).The function Holds

K

(q,M, labGrK

) computes a predicate that istrue if and only if there exists a trace t in traces

K

(q) with t |= M .Formally, Holds

K

(q,M, labGrK

) is defined as (�(q) = (AP \M)) ^ 9q0 2 succ

K

(q),M 02 labGr

K

(q0) : follows(M,M 0).

The following captures the correctness of labeling:

Lemma 3. First, HoldsSink(q,M) , 9t 2 traces(q) : t |= Mfor sink states q. Second, if labGr

K

is a correct labeling withrespect to ' and succ

K

(q), then HoldsK

(q,M, labGrK

) ()

9t 2 tracesK

(q) : t |= M .

Finally, we define labelNodeK

(', q, labGrK

), which com-putes a label L for q such that M 2 L if and only if there existsa trace t 2 traces

K

(q) such that t |= M for all M ⇢ ecl(').We assume that labGr

K

is a correct labeling of K with respect to' and succ(q). For sink states, labelNode

K

(', q, labGrK

) returns{M | M 2 ecl(')^HoldsSink(q,M)}, while for non-sink statesit returns {M | M 2 ecl(') ^ Holds

K

(q,M, labGrK

)}.

5.2 Incremental algorithmTo incrementally model check a modified Kripke structure, wemust re-label its states with the formulas that hold after the update.

Consider two Kripke structures K = (Q,Q0, �,�) and K0=

(Q0, Q00, �

0,�0), such that Q0 = Q0

0. Furthermore, assume thatQ = Q0, and there is a set U ✓ Q such that � and �0 differ only onnodes in U . We call such a triple (K,K0, U) an update of K.

An update (K,K0, U) might add or remove edges connectedto a (small) set of nodes, corresponding to a change in the ruleson a switch. Suppose that labGr

K

is a correct labeling of K withrespect to' and Q. The incremental model checking problem is de-fined as follows: we are given an update (K,K0, U), and labGr

K

,and we want to know whether K0 satisfies '. The naıve approach isto model check K0 without using the labeling labGr

K

. We call thisthe monolithic approach. In contrast, the incremental approach useslabGr

K

(and thus intuitively re-uses the results of model checkingK to efficiently verify K0).

Example. Consider the left side of Figure 6, with H the onlyinitial state. Suppose that the update modifies J , and the �0 relationapplied to J only contains the pair (J,N), and consider labelingthe structure with formulas F a, F b, and F a _ F b. To simplifythe example, we label a node by all those formulas which hold forat least one path starting from the node (note that in the algorithm,a node is labeled by a set of sets of formulas, rather than a set offormulas). We will have that all the nodes are labeled by F a_F b,and in addition the nodes K, I,H,M, J contain label F a, and thenodes L, I,H,N contain F b. Now we want to relabel the structureafter the update (right-hand side). Given that the update changesonly node J , the labeling can only change for J and its ancestors.

320

H

I J

a K b L a M b N

H

I J

a K b L a M b N

Figure 6: Incremental labeling—Initial (left), Final (right)

We therefore start labeling node J , and find that it will now belabeled with F b instead of F a. Labeling proceeds to H , whoselabel does not change (still labeled by all of F a, F b, F a _ F b).The labeling process could then stop, even if H has ancestors.

Re-labeling states. Let ancestorsK

(V ) be the ancestors of Vin K—i.e., a set of vertices s.t. ancestors

K

(V ) ✓ Q and q 2ancestors

K

(V ), if some node v 2 V is reachable from q. To de-fine incremental model checking for ', we need a function accept-ing a property ', set of vertices V , labeling labGr

K

that is correctfor K with respect to ' and Q \ ancestors

K

(V ), and returns acorrect labeling of K with respect to ' and Q. This function is:

relblK

(', labGrK

, V ) =

(labGr

K

if V = ;

relblK

(', labGr 0K

, V 0) otherwise

where labGr 0K

(v) is labelNodeK

(', v, labGrK

) if v 2 V , andit is labGr

K

(v) if v 62 V . The set V 0 is {q | 9v 2 V : v 2succ

K

(q)}.

Theorem 3. Let V ✓ Q be a set of vertices and labGrK

acorrect labeling with respect to ' and Q \ ancestors

K

(V ). Thenrelbl

K

(', labGrK

, V ) is a correct labeling w.r.t. ' and Q.

Given a labeling that is correct with respect to ' and Q, itis easy to check whether ' is true for all the traces starting inthe initial states: the predicate checkInitStates

K

(labGrK

,') isdefined as 8q0 2 Q0,M 2 labGr

K

(q0) : ' 2 M . Next,let Q

f

be the set of all sink states of K. Then ancestorsK

(Qf

)

is the set Q of all states K. Therefore, for any initial labelinglabGr0

K

, relbl(', labGr0K

, Qf

) is a correct labeling with respectto ' and Q. The function modelCheck

K

(') is defined to be equalto checkInitStates

K

(relblK

(', labGr0K

, Qf

),'), where we canset labGr0

K

to be the empty labeling �v.;.We now define our incremental model checking function. Let

(K,K0, U) be an update, and labGrK

a previously-computed cor-rect labeling of K with respect to ' and Q, where Q is the setof states of K. The function incrModelCheck(K,', U, labGr

K

)

is defined as checkInitStatesK

0(relbl

K

0(', labGr

K

, U),'). Thefollowing shows the correctness of our model checking functions(proof of this and the previous theorem are in Appendix C).

Corollary 1. First, modelCheckK

(') = true () K |=

'. Second, for (K,K0, U) and labGrK

as above, we haveincrModelCheck(K,', U, labGr

K

) = true () K |= '.

The runtime complexity of the modelCheckK

function isO(|K| ⇥ 2

|'|). The runtime complexity of the incrModelCheck

function is O(|ancestorsK

(U)|⇥2

|'|), where U is the set of nodes

being updated.

Counterexamples. This incremental algorithm can generatecounterexamples in cases where the formula does not hold. A for-mula ¬' does not hold if an initial state is labeled by L, such thatthere exists a set M 2 L, such that ¬' 2 M . Examining the def-inition of labelNode

K

, we find that in order to add a set M to thelabel L of a node q, there is a set M 0 in the label of one a child q0 ofq that explains why M is in L. The first node of the counterexampletrace starting from q is one such child q0.

6. Implementation and ExperimentsWe have built a prototype tool that implements the algorithms de-scribed in this paper. It consists of 7K lines of OCaml code. Thesystem works by building a Kripke structure (§3) and then repeat-edly interacting with a model checker to synthesize an update. Wecurrently provide four checker backends: Incremental uses incre-mental relabeling to check and recheck formulas, Batch re-labelsthe entire graph on each call, NuSMV queries a state-of-the-artsymbolic model checker in batch mode, and NetPlumber queriesan incremental network model checker [19]. All tools except Net-Plumber provide counterexample traces, so our system learns fromcounterexamples whenever possible (§4).

Experiments. To evaluate performance, we generated configura-tions for a variety of real-world topologies and ran experimentsin which we measured the amount of time needed to synthesizean update (or discover that no order update exists). These experi-ments were designed to answer two key questions: (1) how the per-formance of our Incremental checker compares to state-of-the-arttools (NuSMV and NetPlumber), and (2) whether our synthesizerscales to large topologies. We used the Topology Zoo [21] dataset,which consists of 261 actual wide-area topologies, as well as syn-thetically constructed Small-World [29] and FatTree [1] topologies.We ran the experiments on a 64-bit Ubuntu machine with 20GBRAM and a quad-core Intel i5-4570 CPU (3.2 GHz) and imposeda 10-minute timeout for each run. We ignored runs in which thesolver died due to an out-of-memory error or timeout—these areinfrequent (less than 8% of the 996 runs for Figure 7), and our In-cremental solver only died in instances where other solvers did too.

Configurations and properties. A recent paper [23] surveyeddata-center operators to discover common update scenarios, whichmostly involve taking switches on/off-line and migrating traffic be-tween switches/hosts. We designed experiments around a similarscenario. To create configurations, we connected random pairs ofnodes (s, d) via disjoint initial/final paths W

i

,Wf

, forming a “dia-mond”, and asserted one of the following properties for each pair:• Reachability: traffic from a given source must reach a certain

destination: (port = s)) F (port = d)• Waypointing: traffic must traverse a waypoint w:(port=s))

�(port 6=d)U ((port=w) ^ F (port=d))

�

• Service chaining: traffic must waypoint through several inter-mediate nodes: (port = s)) way(W,d), where

way([ ], d) ⌘ F (port = d)way(w

i

:: W,d) ⌘

�(

Vwk2W

port 6=wk

^ port 6=d)U ((port = w

i

) ^ way(W,d))�.

Incremental vs. NuSMV/Batch. Figure 7 (a-c) compares the per-formance of Incremental and NuSMV backends for the reachabilityproperty. Of the 247 Topology Zoo inputs that completed success-fully, our tool solved all of them faster. The measured speedupswere large, with a geometric mean of 447.23x. For the 24 FatTreeexamples, the mean speedup was 465.03x, and for the 25 Small-World examples, the mean speedup was 4484.73x. We also com-pared the Incremental and Batch solvers on the same inputs. Incre-mental performs better on almost all examples, with mean speedupof 4.26x, 5.27x, 11.74x on the datasets shown in Figure 7(a-c) andmaximum runtimes of 0.36s, 2.80s, and 0.92s respectively. Themaximum runtimes for Batch were 6.71s, 39.75s, and 12.50s.

Incremental vs. NetPlumber. We also measured the performanceof Incremental versus the network property checker NetPlumber(Figure 7(d-f)). Note that NetPlumber uses rule-granularity forupdates, so we enabled this mode in our tool for these experiments.For the three datasets, our checker is faster on all experiments, withmean speedups of (6.41x, 4.90x, 17.19x). NetPlumber does not

321

0 200 400 600 (switches)0

50

100 (a)

Run

time

(s)

Topology Zoo

NuSMVBatchIncremental

0 200 400 600 (switches)0

50

100 (b)FatTree

0 200 400 600 (switches)0

50

100 (c)Small-World

0 5k 10k (rules)0

20

40

60 (d)

Run

time

(s)

NetPlumberIncremental

0 5k 10k (rules)0

20

40

60 (e)

0 5k 10k (rules)0

20

40

60 (f)

Figure 7: Relative performance results: (a-c) Performance of Incremental vs. NuSMV, Batch, NetPlumber solvers on Topology Zoo, FatTree, Small-Worldtopologies (columns); (d-f) Performance of Incremental vs. NetPlumber (rule-granularity).

0 200 400 600 800 1k (switches)0

50

100

150

200

(g)

Run

time

(s)

Service ChainingWaypointingReachability

0 200 400 600 800 1k (switches)0

50

100

150

200

(h)

Run

time

(s)

0 10k 20k 30k 40k 50k (rules)0

500

1k(i)

Run

time

(s)

Figure 8: (g) Scalability of Incremental on Small-World topologies of in-creasing size; (h) Scalability when no correct switch-granularity update ex-ists (i.e. algorithm reports “impossible”), and (i) Scalability of fine-grained(rule-granularity) approach for solving switch-impossible examples in (h).

report counterexamples, putting it at a disadvantage in this end-to-end comparison, so we also measured total Incremental versusNetPlumber runtime on the same set of model-checking questionsposed by Incremental for the Small-World example. Our tool is stillfaster on all instances, with a mean speedup of 2.74x.Scalability. To quantify our tool’s scalability, we constructedSmall World topologies with up to 1500 switches, and ran experi-ments with large diamond updates—the largest has 1015 switchesupdating. The results appear in Figure 8(g). The maximum synthe-sis times for the three properties were 129.04s, 30.11s, and 0.85s,which shows that our tool scales to problems of realistic size.Infeasible Updates. We also considered examples for whichthere is no switch-granular update. Figure 8(h) shows the resultsof experiments where we generated a second diamond atop the firstone, requiring it to route traffic in the opposite direction. Usingswitch-granularity, the inputs are reported as unsolvable in maxi-mum time 153.48s, 33.48s, and 0.69s. Using rule-granularity, theseinputs are solved successfully for up to 1000 switches with maxi-mum times of 776.13s, 512.84s, and 82.00s (see Figure 8(i)).Waits. We also separately measured the time needed to run thewait-removal heuristic for the Figure 8 experiments. For (g), themaximum wait-removal runtime was 0.89s, resulting in 2 neededwaits for each instance. For (i), the maximum wait-removal runtimewas 103.87s, resulting in about 2.6 waits on average (with a maxi-mum of 4). For the largest problems in (g) and (i), this correspondsto removal of 1397/1399 and 55823/55826 waits (about 99.9%).

7. Related WorkThis paper extends preliminary work reported in a workshop pa-per [30]. We present a more precise and realistic network model,and replace expensive calls to an external model checker with callsto a new built-in incremental network model checker. We extendthe DFS search procedure with optimizations and heuristics thatimprove performance dramatically. Finally, we evaluate our tool ona comprehensive set of benchmarks with real-world topologies.

Synthesis of concurrent programs. There is much previous workon synthesis for concurrent programs [12, 35, 38]. In particular,work by Solar-Lezama et al. [35] and Vechev et al. [38] synthesizessequences of instructions. However, traditional synthesis and syn-thesis for networking are quite different. First, traditional synthesisis a game against the environment which (in the concurrent pro-gramming case) provides inputs and schedules threads; in contrast,our synthesis problem involves reachability on the space of config-urations. Second, our space of configurations is very rich, meaningthat checking configurations is itself a model checking problem.

Network updates. There are many protocol- and property-specific algorithms for implementing network updates, e.g. avoid-ing packet/bandwidth loss during planned maintenance to BGP [10,32]. Other work avoids routing loops and blackholes during IGPmigration [36]. Work on network updates in SDN proposed thenotion of consistent updates and several implementation mech-anisms, including two-phase updates [33]. Other work explorespropagating updates incrementally, reducing the space overhead onswitches [17]. As mentioned in Section 2, recent work proposes or-dering updates for specific properties [15], whereas we can handlecombinations and variants of these properties. Furthermore, SWANand zUpdate add support for bandwidth guarantees [13, 23]. Zhouet al. [40] consider customizable trace properties, and propose a dy-namic algorithm to find order updates. This solution can take intoaccount unpredictable delays caused by switch updates. However,it may not always find a solution, even if one exists. In contrast, weobtain a completeness guarantee for our static algorithm. Ludwiget al. [24] consider ordering updates for waypointing properties.

Model checking. Model checking has been used for network ver-ification [2, 18, 20, 26, 27]. The closest to our work is the incre-mental checker NetPlumber [19]. Surface-level differences includethe specification languages (LTL vs. regular expressions), and Net-Plumber’s lack of counterexample output. The main difference isincrementality: Netplumber restricts checking to “probe nodes,”keeping track of “header-space” reachability information for thosenodes, and then performing property queries based on this. In con-trast, we look at the property, keeping track of portions of theproperty holding at each node, which keeps incremental recheck-

322

ing times low. The empirical comparison (Section 6) showed betterperformance of our tool as a back-end for synthesis.

Incremental model checking has been studied previously, with[34] presenting the first incremental model checking algorithm, foralternation-free µ-calculus. We consider LTL properties and spe-cialize our algorithm to exploit the no-forwarding-loops assump-tion. The paper [7] introduced an incremental algorithm, but it isspecific to the type of partial results produced by IC3 [5].

8. ConclusionWe present a practical tool for automatically synthesizing cor-rect network update sequences from formal specifications. We dis-cuss an efficient incremental model checker that performs ordersof magnitude better than state-of-the-art monolithic tools. Exper-iments on real-world topologies demonstrate the effectiveness ofour approach for synthesis. In future work, we plan to explore bothextensions to deal with network failures and bandwidth constraints,and deeper foundations of techniques for network updates.

AcknowledgmentsThe authors would like to thank the PLDI reviewers and AECmembers for their insightful feedback on the paper and arti-fact, as well as Xin Jin, Dexter Kozen, Mark Reitblatt, and Jen-nifer Rexford for helpful comments. Andrew Noyes and ToddWarszawski contributed a number of early ideas through an un-dergraduate research project. This work was supported by theNSF under awards CCF-1421752, CCF-1422046, CCF-1253165,CNS-1413972, CCF-1444781, and CNS-1111698; the ONR underAward N00014-12-1-0757; and gifts from Fujitsu Labs and Intel.

References[1] M. Al-Fares, A. Loukissas, and A. Vahdat. A Scalable, Commodity

Data Center Network Architecture. In SIGCOMM, 2008.[2] E. Al-Shaer and S. Al-Haj. FlowChecker: Configuration Analysis and

Verification of Federated OpenFlow Infrastructures. In SafeConfig,2010.

[3] M. Alizadeh, A. Greenberg, D. Maltz, J. Padhye, P. Patel, B. Prab-hakar, S. Sengupta, and M. Sridharan. Data Center TCP (DCTCP). InSIGCOMM, pages 63–74, 2010.

[4] G. Berry and G. Boudol. The Chemical Abstract Machine. In POPL,pages 81–94, 1990.

[5] A. Bradley. SAT-Based Model Checking without Unrolling. InVMCAI, 2011.

[6] M. Casado, N. Foster, and A. Guha. Abstractions for Software-Defined Networks . CACM, 57(10):86–95, Oct. 2014.

[7] H. Chockler, A. Ivrii, A. Matsliah, S. Moran, and Z. Nevo. IncrementalFormal Verification of Hardware. In FMCAD, pages 135–143, 2011.

[8] N. Foster, R. Harrison, M. Freedman, C. Monsanto, J. Rexford,A. Story, and D. Walker. Frenetic: A Network Programming Lan-guage. In ICFP, pages 279–291, 2011.

[9] P. Francois and O. Bonaventure. Avoiding Transient Loops during theConvergence of Link-state Routing Protocols. IEEE/ACM Transac-tions on Networking, 15(6):1280–1292, 2007.

[10] P. Francois, O. Bonaventure, B. Decraene, and P.-A. Coste. AvoidingDisruptions during Maintenance Operations on BGP Sessions. IEEETransactions on Network and Service Management, 4(3):1–11, 2007.

[11] A. Guha, M. Reitblatt, and N. Foster. Machine-Verified NetworkControllers . In PLDI, June 2013.

[12] P. Hawkins, A. Aiken, K. Fisher, M. Rinard, and M. Sagiv. ConcurrentData Representation Synthesis. In PLDI, pages 417–428, June 2012.

[13] C.-Y. Hong, S. Kandula, R. Mahajan, M. Zhang, V. Gill, M. Nanduri,and R. Wattenhofer. Achieving High Utilization with Software-DrivenWAN. In SIGCOMM, pages 15–26, Aug. 2012.

[14] S. Jain, A. Kumar, S. Mandal, J. Ong, L. Poutievski, A. Singh,S. Venkata, J. Wanderer, J. Zhou, M. Zhu, J. Zolla, U. Holzle, S. Stu-art, and A. Vahdat. B4: Experience with a Globally-deployed SoftwareDefined WAN. In SIGCOMM, 2013.

[15] X. Jin, H. Liu, R. Gandhi, S. Kandula, R. Mahajan, M. Zhang, J. Rex-ford, and R. Wattenhofer. Dynamic Scheduling of Network Updates.In SIGCOMM, pages 539–550, 2014.

[16] J. P. John, E. Katz-Bassett, A. Krishnamurthy, T. Anderson, andA. Venkataramani. Consensus Routing: The Internet as a DistributedSystem. In NSDI, pages 351–364, 2008.

[17] N. P. Katta, J. Rexford, and D. Walker. Incremental Consistent Up-dates. In HotSDN, pages 49–54. ACM, 2013.

[18] P. Kazemian, G. Varghese, and N. McKeown. Header Space Analysis:Static Checking for Networks. In NSDI, 2012.

[19] P. Kazemian, M. Chang, H. Zeng, G. Varghese, N. McKeown, andS. Whyte. Real Time Network Policy Checking Using Header SpaceAnalysis. NSDI, pages 99–112, 2013.

[20] A. Khurshid, W. Zhou, M. Caesar, and P. Godfrey. VeriFlow: VerifyingNetwork-wide Invariants in Real Time. ACM SIGCOMM CCR, 2012.

[21] S. Knight, H. Nguyen, N. Falkner, R. Bowden, and M. Roughan.The Internet Topology Zoo. IEEE Journal on Selected Areas inCommunications, 29(9):1765–1775, Oct. 2011.

[22] A. Lazaris, D. Tahara, X. Huang, L. Li, A. Voellmy, Y. Yang, andM. Yu. Tango: Simplifying SDN Programming with Automatic SwitchBehavior Inference, Abstraction, and Optimization. 2014.

[23] H. H. Liu, X. Wu, M. Zhang, L. Yuan, R. Wattenhofer, and D. Maltz.zUpdate: Updating Data Center Networks with Zero Loss. In SIG-COMM, pages 411–422. ACM, 2013.

[24] A. Ludwig, M. Rost, D. Foucard, and S. Schmid. Good NetworkUpdates for Bad Packets: Waypoint Enforcement Beyond Destination-Based Routing Policies. In HotNets, 2014.

[25] R. Mahajan and R. Wattenhofer. On Consistent Updates in SoftwareDefined Networks. In SIGCOMM, Nov. 2013.

[26] H. Mai, A. Khurshid, R. Agarwal, M. Caesar, P. Godfrey, and S. T.King. Debugging the Data Plane with Anteater. In SIGCOMM, 2011.

[27] R. Majumdar, S. Tetali, and Z. Wang. Kuai: A Model Checker forSoftware-defined Networks. In FMCAD, 2014.

[28] N. McKeown, T. Anderson, H. Balakrishnan, G. Parulkar, L. Peterson,J. Rexford, S. Shenker, and J. Turner. OpenFlow: Enabling Innovationin Campus Networks. ACM SIGCOMM CCR, 2008.

[29] M. E. Newman, S. H. Strogatz, and D. J. Watts. Random Graphs withArbitrary Degree Distributions and their Applications. 2001.

[30] A. Noyes, T. Warszawski, and N. Foster. Toward Synthesis of NetworkUpdates. In SYNT, July 2013.

[31] Open Networking Foundation. OpenFlow 1.4 Specification, 2013.[32] S. Raza, Y. Zhu, and C.-N. Chuah. Graceful Network State Migrations.

IEEE/ACM Transactions on Networking, 19(4):1097–1110, 2011.[33] M. Reitblatt, N. Foster, J. Rexford, C. Schlesinger, and D. Walker.

Abstractions for Network Update. In SIGCOMM, 2012.[34] O. Sokolsky and S. Smolka. Incremental Model Checking in the

Modal Mu-Calculus. In CAV, pages 351–363, 1994.[35] A. Solar-Lezama, C. G. Jones, and R. Bodik. Sketching Concurrent

Data Structures. In PLDI, pages 136–148, 2008.[36] L. Vanbever, S. Vissicchio, C. Pelsser, P. Francois, and O. Bonaven-

ture. Seamless Network-wide IGP Migrations. In SIGCOMM, 2011.[37] M. Y. Vardi and P. Wolper. An Automata-Theoretic Approach to

Automatic Program Verification (Preliminary Report). In LICS, 1986.[38] M. Vechev, E. Yahav, and G. Yorsh. Abstraction-guided Synthesis of

Synchronization. In POPL, pages 327–338, 2010.[39] P. Wolper, M. Y. Vardi, and A. P. Sistla. Reasoning about Infinite

Computation Paths (Extended Abstract). In FOCS, 1983.[40] W. Zhou, D. Jin, J. Croft, M. Caesar, and B. Godfrey. Enforcing

Generalized Consistency Properties in Software-Defined Networks. InNSDI, 2015.

323

A. Network Model Auxiliary DefinitionsWe first define what it means for a table to be active, i.e. thecontroller contains an update that will eventually produce that table.

Definition 6 (Active Forwarding Table). Let N be a network. Theforwarding table tbl is active in the epoch ep for the switch sw if1. ep = 0 and tbl is the initial table of sw in N , or2. ep > 0 and either (a) if there exists a command (sw 0, tbl 0) 2

C.cmds such that sw = sw 0 and the number of wait com-mands preceding (sw , tbl) in C.cmds is ep, then tbl = tbl 0, or(b) if there does not exist such a command, then tbl is the tableactive for the switch sw in epoch ep � 1.

Next we define what it means for an observation o0 to succeed o.

Definition 7 (Successor Observation). Let N be a network and leto = (sw , pt , pkt) and o0 = (sw 0, pt 0, pkt 0) be observations. The

observation o0 is a successor of o in ep, written oep

v o0, if either:• there exists a switch S

i

and link Lj

such that Si

.sw = sw andSi

.tbl is active in ep and Lj

.loc = (sw , ptj

) and Lj

.loc0 =

(sw 0, pt 0) and (ptj

, pkt 0) 2 [[Si

.tbl ]](pt , pkt), or• there exists a switch S

i

, a link Lj

, and a host h such thatSi

.sw = sw and Si

.tbl is active in ep and Lj

.loc = (sw , pt 0)and L

j

.loc0 = h and (pt 0, pkt 0) 2 [[Si

.tbl ]](pt , pkt).

Intuitively oep

v o0 if the packet in o could have directly producedthe packet in o0 in ep by being processed on some switch. The twocases correspond to an internal and egress processing steps.

Definition 8 (Unconstrained Single-Packet Trace). Let N be anetwork. The sequence (o1 · · · ol) is a unconstrained single-packet

trace of N if No

01�! . . .

o

0k�! N

k

such that (o1 · · · ol) is asubsequence of (o01 · · · o0

k

) for which• every observation is a successor of the preceding observation

in monotonically increasing epochs, and

• if o1 = o0j

= (sw , pt , pkt), i.e. No

01�! . . .

o

0j=o1����! N

j

o

0j+1���!

. . .o

0k�! N

k

, then no o0i

2 {o01, · · · , o0j�1} precedes o1, and

• the ol

transition is an OUT terminating at a host.

Unconstrained single-packet traces are not required to begin at ahost. We write ¯

T (N) for the set of unconstrained single-packettraces generated by N , and note that T (N) ✓

¯

T (N).

Definition 9 (Network Kripke Structure). Let N be a static net-work. We define a Kripke structure K(N) = (Q,Q0, �,�) as fol-lows. The set of states Q comprises tuples of the form (sw , pt , T

k

).The set Q0 contains states (sw , pt , T

k

) where sw and pt are adja-cent to an ingress link—i.e., there exists a link L

j

and host h suchthat L

j

.loc = h and Lj

.loc0 = (sw , pt). Transition relation � con-tains all pairs of states (sw , pt , T

k

) and (sw 0, pt 0, T 0k

) where thereexists a switch S and a link L such that S.sw = sw and either:• there exists a link L

j

and packets pkt 2 Tk

and pkt 0 2 T 0k

such that L.loc0 = (sw , pt) and Lj

.loc = (sw , ptj

) andL

j

.loc0 = (sw 0, pt 0) and (pkt 0, ptj

) 2 [[S.tbl ]](pkt , pt).• there exists a link L

j

, a host h , and packets pkt 2 Tk

andpkt 0 2 T 0

k

such that L.loc0 = (sw , pt) and Lj

.loc = (sw , pt 0)and L

j

.loc0 = h and (pkt 0, pt 0) 2 [[S.tbl ]](pkt , pt).•(sw , pt , T

k

) = (sw 0, pt 0, T 0k

) and there exists a packet pkt 2Tk

such that L.loc0 = (sw , pt) and [[S.tbl ]](pkt , pt) = {}.•(sw , pt , T

k

) = (sw 0, pt 0, T 0k

) and there exists a link Lj

andhost h such that L

j

.loc = (sw , pt) and Lj

.loc0 = h .Finally, the labeling function � maps each state (sw , pt , T

k

) toTk

, which captures the set of all possible header values of packetslocated at switch sw and port pt .

The four cases of the � relation correspond to forwarding packets toan internal link, forwarding packets out an egress, dropping packetson a switch, or reaching an egress (inducing a self-loop).

We can relate the observations generated by a network N andthe traces of the Kripke structure generated from it.

Definition 10 (Trace Relation). Let N be a static network andK a Kripke structure. Let . be a relation on observations of Nand states of K defined by (sw , pt , pkt) . (sw , pt , T

k

) if andonly if pkt 2 T

k

. Lift . to a relation on (finite) sequences ofobservations and (infinite) traces by repeating the final observationand requiring . to hold pointwise: o1 · · · ok . t if and only ifoi

. ti

for i from 1 to k and ok

. tj

for all j > k.

Lemma 4 (Traces of a Stable Network). Let N be a stable network.Then for each trace t 2 ¯

T (N), there exists a trace t0 2 T (N) suchthat t is a suffix of t0.

Lemma 5 (Trace-Equivalence). Let N1, Nn

be static networkswhere N1 ! · · · ! N

n

and no transition is an update command.For a single-packet trace t, we have t 2 T (N1) () t 2 T (N

n

).

Lemma 6 (Induced Sequence of Networks). Let N1 be a staticnetwork, and let N 0

1 be the network obtained by emptying allpackets from N1. Let cmds be a sequence of commands, and letc1 · · · cn�1 be the subsequence of update commands. Construct thesequence N 0

1 ! · · · ! N 0n

of empty networks by executing theupdate commands in order. Now, given any sequence N1 ! · · ·!

Nn

induced by cmds , we have Ni

' N 0i

for all i.

In other words, any induced sequence of static networks is point-wise trace-equivalent to the unique sequence of network configura-tions generated by running the update commands in order.

B. Synthesis Algorithm Correctness ProofsLemma 1 (Network Kripke Structure Soundness). Let N be astatic network and K = K(N) a network Kripke structure. Forevery single-packet trace t in T (N) there exists a trace t0 of Kfrom a start state such that t . t0, and vice versa.

Proof. We proceed by induction over k, the length of the (finiteprefix of the) trace. The base case k = 1 is easy to see, sincethe lone observation in t must be on an ingress link, meaning thecorresponding state in K will be an initial state with a self-loop(case 3 of Definition 9), and these are equivalent via Definition 10.

For the inductive step (k > 1), we wish to show both directionsof subtrace relation . to conclude equivalence. First, let t =

o1, · · · , ok+1 be a single-packet trace of length k + 1 in T (N),and we must show that 9t0 2 K(N) such that t . t0. Let tk

be the prefix of t having length k. By our induction hypothesis,there exists t0k = s1, · · · , sk�1, sk, sk, · · · 2 K(N) such thattk . t0k. We have the successor relation o

k

v ok+1, so Definition

7 and 9 tells us that we have a transition sk

! s0 for somes0 2 K. We see that this s0 is exactly what we need to constructt0 = s1, · · · , sk, s

0, s0, · · · which satisfies the relation t . t0.Now, let t0 = s1, · · · , sk, sk+1, sk+1, · · · be a trace in K(N)

for which the finite prefix has length k + 1. We must show that9t 2 T (N) such that t . t0. Let t0k = s1, · · · , sk�1, sk, sk, · · · ,and by our induction hypothesis, and there exists tk = o1, · · · , oksuch that tk . t0k. Consider transition s

k

! sk+1. If s

k

= sk+1,

then t0 = t0k, so we can let t = tk, and conclude that t . t0.Otherwise, if s

k

6= sk+1, then we have one of the first two cases

in Definition 9, which correspond to the cases in Definition 7,allowing us to construct an o

k+1 such that ok

v ok+1. We let

t = o1, · · · , ok, ok+1, and conclude that t . t0.

324

We want to develop a lemma showing that the correctness of carefulcommand sequences can be reduced to the correctness of eachinduced N

i

, so we start with the following auxiliary lemma:

Lemma 7 (Traces of a Careful Network). Let N be a stablenetwork with C.cmds careful, and consider a sequence of staticnetworks induced by C.cmds . For every trace t 2 T (N) thereexists a stable static network N

i

in the sequence s.t. t 2 T (Ni

).

Proof. I. First, we show that at most one update transition can be

involved in the trace. In other words, if No

01�! . . .

o

0k�! N

k

wheret = o1 · · · on is a subsequence of o01 · · · o0

k

, and if f : N ! N is abijection between o

i

indices and o0i

indices, then at most one of thetransitions o0

f(1), · · · , o0f(n) is an UPDATE transition.

Assume to the contrary that there are more than one suchtransitions, and consider two of them, o0

i

, o0j

where i, j 2

{f(1), · · · , f(n)}, assuming without loss of generality that i < j.Now, since the sequence C.cmds is careful, we must have both anINCR and FLUSH transition between o0

i

and o0j

. This means that thesecond update o0

j

cannot happen while the trace’s packet is still inthe network, i.e. j > f(n), and we have reached a contradiction.

II. Now, if there are zero update transitions, we are done, sincethe trace is contained in the first static N . If there is one updatetransition N

k+1 = Nk

[sw tbl ], and this update occurs beforethe packet reaches sw in the trace, then the trace is fully containedin N

k+1. Otherwise, the trace is fully contained in Nk

.

Lemma 2 (Careful Correctness). Let N be a stable network withC.cmds careful and let ' be an LTL formula. If cmds is carefuland N

i

|= � for each static network in any sequence induced bycmds , then cmds is correct with respect to '.

Proof. Consider a trace t 2 T (N). From Lemma 7, we havet 2 T (N

i

) for some Ni

in the induced sequence. Thus t |= ',since our hypothesis tells us that N

i

|= '. Since this is true foran arbitrary trace, we have shown that T (N) |= ', i.e. N |= ',meaning that cmds is correct with respect to '.

Theorem 1 (Soundness). Given initial network Ni

, final configu-ration N

f

, and LTL formula ', if ORDERUPDATE returns a com-mand sequence cmds , then N

i

cmds�! N 0 s.t. N 0

' Nf

, and cmdsis correct with respect to ' and N

i

.

Proof. It is easy to show that if ORDERUPDATE returns cmds , thenN

i

cmds�! N 0 where N 0

' Nf

. Each update in the returned sequencechanges a switch configuration of one switch s to the configurationN

f

(s), and the algorithm terminates when all (and only) switchess such that N

i

(s) 6= Nf

(s) have been updated.Observe that if ORDERUPDATE returns cmds , the sequence can

be made careful by choosing an adequate time delay between eachupdate command, and for all j 2 {0, · · · , n}, N

j

|= '. This isensured by the call to a model checker (Line 7). We use Lemma 2to conclude that cmds is correct with respect to ' and N

i

.

To show that ORDERUPDATE is complete with respect to simpleand careful command sequences, we observe that ORDERUPDATEsearches through all simple and careful sequences.

Theorem 2 (Completeness). Given initial network Ni

, final con-figuration N

f

, and specification ', if there exists a simple, carefulsequence cmds with N

i

cmds�! N 0 s.t. N 0

' Nf

, then ORDERUP-DATE returns one such sequence.

C. Incremental Checking Correctness ProofsLemma 3. First, HoldsSink(q,M) , 9t 2 traces(q) : t |= Mfor sink states q. Second, if labGr

K

is a correct labeling withrespect to ' and succ

K

(q), then HoldsK

(q,M, labGrK

) ()

9t 2 tracesK

(q) : t |= M .

Proof. First, for sink states, observe that there is a unique trace tin traces(q), as q is a sink state. We first prove that t |= ' iffHolds0 (q,'). We prove this by induction on the structure of theLTL formula. Then we observe that there is a unique maximally-consistent set M such that t |= M . This is the set { | t |= ^ 2ecl(')}. We then use the definition of HoldsSink(q,M) for sinkstates to conclude the proof.

Now consider non-sink states: we first prove soundness, i.e., ifHolds

K

(q,M, labGrK

), then there exists t 2 traces(q) such thatt |= M . We have Holds

K

(q,M, labGrK

) iff (�(q) = (AP\M))

and there exists q0 2 succK

(M), and M 02 labGr

K

(q0) such thatfollows(M,M 0

). By assumption of the theorem, we have that ifM 02 labGr

K

(q0), then there exists a trace t0 in traces(q0) suchthat t0 |= M 0. Consider a trace t such that t0 = q and t1 = t0.For each 2 M , we can prove that t |= as follows. The basecase of the proof by induction is implied by the fact that q |=

(AP \M). The inductive cases are proven using the definitions ofmaximally-consistent set and the function follows . We now provecompleteness, i.e., that if there exists a trace t in traces

K

(q) suchthat t |= M , then Holds

K

(q,M, labGrK

) is true. Let t be thetrace qq1q2 . . .. It is easy to see that if M is a maximally-consistentset, and t |= M , then M = { | 2 ecl(') ^ t |= }. Let usconsider the set of formulas S = { | 2 ecl(') ^ t1 |= }.Observe that S is a maximally-consistent set. By assumption of thetheorem, we have that S is in labGr

K

(q1). It is easy to verify thatfollows(M,S).

Theorem 3. Let V ✓ Q be a set of vertices and labGrK

acorrect labeling with respect to ' and Q \ ancestors

K

(V ). Thenrelbl

K

(', labGrK

, V ) is a correct labeling w.r.t. ' and Q.

Proof. We first note that only ancestors of nodes in V are re-labeled—all the other nodes are correctly labeled by assumptionon labGr . We say that a node q is at level k w.r.t. a set of verticesT iff the longest simple path from q to a node in T is k. Let H

k

be the set of nodes at level k from V . We prove by induction on kthat at k-th iteration, we have a correct labeling of K w.r.t. ' and(S \ ancestors

K

(V ))[Hk

, where S is the set of states of K. Wecan prove the inductive claim using Lemma 3.

Corollary 1. First, modelCheckK

(') = true () K |=

'. Second, for (K,K0, U) and labGrK

as above, we haveincrModelCheck(K,', U, labGr

K

) = true () K |= '.

Proof. Using Theorem 3, and the fact that the set ancestorsK

(Sf

)

is the set S of all states K, we obtain that labGrK

=

relblK

(', labGr0K

, Sf

) is a correct labeling of K with respectto ' and S. In particular, for all initial states q0, we have thatfor all M ⇢ ecl('), m 2 labGr

K

(q0) iff there exists a tracet 2 traces

K

(q0) such that t |= M . We now use the definition ofcheckInitStates to show that if checkInitStates returns true, thenthere is no initial state q0 such that there exists M 2 labGr

K

(q0)such that ¬' 2 M . Thus for all initial states q0, for all traces t intraces(t0), we have that t |= '.

The proof for incremental model checking is similar.

325