Distributed Network Service Policy Enforcement

June 17, 2016

Distributed Network Service Policy Enforcement

Jun Li

Contributions from my students, Xiang Wang and Xiaohe Hu

Outline

• Background

• Related Work

• Policy Space Analysis

• Policy Enforcement with PSA

2016/6/17 NSLab, RIIT, Tsinghua Univ. 2

Orthogonal Perspective

• Forwarding vs. Service


Forwarding Service

Task Delivery Security, Measurement, Optimization

Logical Object Packet Flow

Physical Object L2 ~ L3, Header L3 ~ L7, Header + Payload

Basis Topology Resource/Policy

State Stateless Stateful

Manner Local autonomy Global governance

Device Switch/Router Middlebox

Algorithm Routing origination,

Routing lookup Packet classification, Pattern matching,

AppID, Traffic management

Outline

• Background

• Related Work




Data Center Network (DCN)

• Virtualization & Multi-tenancy


Aggregation

Core

Access

Storage Storage Compute Compute Compute Compute

Security

Fabric

Access

Storage Storage Compute Compute Compute Security

Traditional DCN architecture Cloud DCN architecture

Cloud DCN

• Logical View vs. Tenant View


[X. Wang et al., CouldCom 2012]

LiveCloud

Operator view for orchestration Tenant view for provision

Service in Cloud DCN

• Network Service Mechanism

– Share vs. Aggregation

– Software vs. Hardware


Router

Internet

Subnet 1

Subnet 2

AB

Security Policy

CD

E

Edge vSwitch

Security Controller

Network Controller

Security Workload Scheduler

Tenant Subnet1

Tenant Subnet2

Security Network

Host Host Host Host

Fabric Switch

Tenant view Operator view

[X. Wang et al., ICCCN 2014]

Tualatin

Policy in Cloud DCN

• Network Service Policy

– Global vs. Local

– Distributed and Dynamic Interaction


Rule1 Rule3

Rule2 Rule4

Policy Management

Tenant / Operator

NetworkState

Policy Management (I)


Centralized Control Plane

Service Policy Configuration

Topology

Forwarding Policy

* -> 10.2.1.*, ssh DROP

10.1.*.* -> * FWIDS

10.1.1.*->10.2.1.*

𝑝𝑎𝑡ℎ{𝑠𝑤3. 𝑠𝑤2. 𝑠𝑤1}

𝑠𝑤1: Forwarding Rules

Forwarding Device

𝑚𝑏−𝑐𝑙𝑢𝑠𝑡𝑒𝑟1: Service Rules

id=1,in_port=1,ip_dst=10.2.1.*,ssh DROP

id=3,in_port=1,ip_dst=* Signtr

Middlebox

Distributed Data Plane

Measurement Security Application Policy

Admin

Optimization Users and

Applications

Network Controller

Service Controller

id=1,in_port=3,ip_dst=* port2

id=3,in_port=2,ip_dst=10.2.1.* port1

1

2

3 4

5 6

7 1

2

Policy Management (II)


Policy Enforcement

Policy Definition

Policy Composition

Policy Assignment

Policy Dispatch

Policy Lookup

Policy Verification

User/App1

* -> 10.2.1.*, ssh DROP

10.1.*.*-> * FWIDS Service Policy

User/App3 User/App2

Forwarding Policy 10.1.1.*->10.2.1.*

𝑝𝑎𝑡ℎ{𝑠𝑤3. 𝑠𝑤2. 𝑠𝑤1}

Application Plane

Centralized Control Plane

Distributed Data Plane

Policy Language

Outline

• Background

• Related Work




Policy Language

• Policy Definition

– DATALOG-based query

• FML [T. L. Hinrichs et al., WREN’09]

– Logical labels

• PGA [C. Prakash et al., SIGCOMM’15]

• Policy Composition

– High-level language for writing and composing modules

• Frenetic [N. Foster et al., SIGPLAN’11]

• Pyretic [C. Monsanto et al., NSDI’13]


Policy Enforcement (I)

• Policy Assignment

– Distributed policies on switches w/ forwarding change

• DIFANE [M. Yu et al., SIGCOMM’11]

• vCRIB [M. Moshref et al., NSDI’13]

– Distributed policies on switches w/o forwarding change

• Palette [Y. Kanizo et al., INFOCOM’13]

• One-Big-Switch [N. Kang et al., CoNEXT’13]

– Distributed policies on middleboxes w/ forwarding change

• SIMPLE [Z. A. Qazi et al., SIGCOMM’13]

– Distributed policies on middleboxes w/o forwarding change

• MBPE [X. Wang et al., TON’16]


Policy Enforcement (II)

• Policy Dispatch

– Incremental updates

• Update Abstractions [M. Reitblatt et al., SIGCOMM’12]

• CCG [W. Zhou et al., NSDI’15]

– Minimal flow-table

• DevoFlow [A. R. Curtis et al., SIGCOMM’11]

• Policy Verification

– Firewall policy analysis

• FDD [M. G. Gouda et al., ICDCS’04]

– Header space analysis

• HSA [P. Kazemian et al., NSDI’12 & 13]

– Real-time verification

• Veriflow [A. Khurshid et al., NSDI’13]


Outline

• Background

• Related Work


– Motivation

– Design

– Evaluation



Motivation

• What is the model and theoretical foundation of

service policy?

• What is the common policy enforcement

functionalities for assignment, dispatch, and

verification?

• What is the performance requirements for

practical policy enforcement?


Header Space Analysis (HSA)

• A simple abstraction to model all kinds of forwarding

functionalities

– Mathematical modeling

– Header space + Transfer function

– Forwarding policy distribution,

optimization, and conflict detection


01110011…1

L

Header

0xxxx0101xxx

T1(h, p)

R1 R2 R3

T2(h, p)

T3(h, p)

[P. Kazemian et al., NSDI 2012 & 2013]

Problems with HSA

• HSA is inefficient for service policy management

– Space representation in indiscriminate bits

• The number of HSA computing dimensions is 104 for the classic

5-tuple policy.

• Service policies usually contain arbitrary range values.

– Set operations in an overlapping manner

• Header space (xxxx) minus point (1010) is (xxx1) union (xx0x)

union (x1xx) union (0xxx), resulting in more computing tasks and

duplicated sub-spaces while doing set operations.

– Lack of efficient indexing data structures

• Set operations are conducted in a linear manner.


Outline

• Background

• Related Work


– Motivation

– Design

– Evaluation



Policy Space Analysis

• Computational geometry view

– Multi-dimensional space

• Expression

– HyperRect: a D-field rule is viewed as a range-based D-

dimension hyper-rectangle

– PolicySpace: a set of multiple non-overlapping HyperRects

• Boolean and Set Operations

– Supported by both HyperRect and PolicySpace

– Boolean Operations

• is_equal, is_subset, and is_intersected

– Set Operations

• intersect, subtract, and union


[X. Wang et al., TON 2016]

PSA

PSA Implementation • HyperRect Operation

– Different dimension inspecting sequences produce diverse operation results

• PolicySpace Operation

– Most operations implemented as the iteration of the same operations of the

HyperRect

– is_equal implemented as bidirectional is_subset operations

– is_subset implemented as volume comparison of the minuend policy space

and the intersected policy space based on the non-overlapping property


Subtract Union

Outline

• Background

• Related Work


– Motivation

– Design

– Evaluation



PSA Evaluation (I)

• Spatial performance

– Iterated subtraction of

high-priority rules from

low-priority rules to

construct a non-

overlapping ruleset


0.0E+00

1.0E+03

2.0E+03

3.0E+03

4.0E+03

5.0E+03

6.0E+03

7.0E+03

1 11 21 31 41 51 61 71 81 91

基础数据类型实例（个）

规则数目

HSA_ACL1_100 HSA_FW1_100

HSA_IPC1_100 PSA_ACL1_100

PSA_FW1_100 PSA_IPC1_100

Num

ber

of

Wild

card

or

HyperR

ect

Number of Operated Rules

PSA vs. HSA

• Temporal performance

PSA Evaluation (II)


Linear Array R Tree

1.0E+01

1.0E+03

1.0E+05

1.0E+07

处理时间（ms）

规则集

R树索引线性存储

Pro

cess

ing t

ime (

ms)

Ruleset

1.0E+04

1.0E+06

1.0E+08

1.0E+10处理时间（us）

规则集

体积计算空间剪切

Pro

cess

ing t

ime (

ms)

Ruleset

Space Clipping Volume Comparison

— Iterated subtraction of high-priority rules from low-priority rules to construct a non-overlapping ruleset

— is_equal operation between the union of non-overlapping rules and the union of overlapping rules

Outline

• Background

• Related Work



– Topological Analysis of Service Policy

– Policy Assignment

– Policy Verification


Topological Analysis of Service Policy (I)

• Analysis of three classical policies

– Policy topology: Hierarchical addresses

– Topological statistics: Rule distribution


Access Control

0.0

.0.0

/58

.0.0

.0/6

12

.0.0

.0/7

14

.0.0

.0/8

15

.0.0

.0/8

16

.0.0

.0/4

32

.0.0

.0/3

64

.0.0

.0/2

12

8.0

.0.0

/1

0100200300400500600

0.0

.0.0

/58

.0.0

.0/6

12

.0.0

.0/7

14

.0.0

.0/8

15

.0.0

.0/8

16

.0.0

.0/4

32

.0.0

.0/3

64

.0.0

.0/2

12

8.0

.0.0

/1

源地址段

规则数目

目的地址段Destination Source

Num

ber

of

Rule

s

(exact rules)

Topological Analysis of Service Policy (II)


Firewall

IP Chain

10

.0.0

.0/8

16

.1.0

.4/3

21

31

.10

7.3

.43/

32

13

1.1

07.

39.

10/3

21

31

.10

7.3

9.11

/32

17

2.1

6.0

.0/1

21

92

.5.4

.0/2

31

92

.16

8.0

.0/1

61

92

.24

6.1

08.4

/32

19

8.3

2.1

76.

6/3

22

04

.12

3.2

.5/3

22

04

.15

2.1

84.0

/21

20

6.1

59.

213

.125

/32

20

7.1

59.

9.1

18/3

2

0

40

80

120

160

10

.0.0

.0/8

16

.1.0

.4/3

21

31

.10

7.3

.43/

32

13

1.1

07.

39.

10/3

21

31

.10

7.3

9.11

/32

17

2.1

6.0

.0/1

21

92

.5.4

.0/2

31

92

.16

8.0

.0/1

61

92

.24

6.1

08.4

/32

19

8.3

2.1

76.

6/3

22

04

.12

3.2

.5/3

22

04

.15

2.1

84.0

/21

20

6.1

59.

213

.125

/32

20

7.1

59.

9.1

18/3

2

源地址段

规则数目

目的地址段

Num

ber

of

Rule

s

Source Destination

(exact rules)

1.0

.0.0

/81

0.0

.0.0

/81

04

.0.0

.0/8

11

1.0

.0.0

/81

21

.0.0

.0/8

15

0.0

.0.0

/81

72

.0.0

.0/8

19

2.0

.0.0

/82

02

.0.0

.0/8

20

4.0

.0.0

/82

06

.0.0

.0/8

20

8.0

.0.0

/82

09

.0.0

.0/8

21

2.0

.0.0

/82

16

.0.0

.0/8

22

2.0

.0.0

/82

55

.0.0

.0/8

0

20

40

60

80

100

120

140

1.0

.0.0

/81

0.0

.0.0

/81

04

.0.0

.0/8

11

1.0

.0.0

/81

21

.0.0

.0/8

15

0.0

.0.0

/81

72

.0.0

.0/8

19

2.0

.0.0

/82

02

.0.0

.0/8

20

4.0

.0.0

/82

06

.0.0

.0/8

20

8.0

.0.0

/82

09

.0.0

.0/8

21

2.0

.0.0

/82

16

.0.0

.0/8

22

2.0

.0.0

/82

55

.0.0

.0/8

源地址段

规则数目

目的地址段

Num

ber

of

Rule

s

Destination Source

(exact rules)

10

.0.0

.0/8

16

.1.0

.4/3

21

31

.10

7.3

.43/

32

13

1.1

07.

39.

10/3

21

31

.10

7.3

9.11

/32

17

2.1

6.0

.0/1

21

92

.5.4

.0/2

31

92

.16

8.0

.0/1

61

92

.24

6.1

08.4

/32

19

8.3

2.1

76.

6/3

22

04

.12

3.2

.5/3

22

04

.15

2.1

84.0

/21

20

6.1

59.

213

.125

/32

20

7.1

59.

9.1

18/3

2

0

40

80

120

160

10

.0.0

.0/8

16

.1.0

.4/3

21

31

.10

7.3

.43/

32

13

1.1

07.

39.

10/3

21

31

.10

7.3

9.11

/32

17

2.1

6.0

.0/1

21

92

.5.4

.0/2

31

92

.16

8.0

.0/1

61

92

.24

6.1

08.4

/32

19

8.3

2.1

76.

6/3

22

04

.12

3.2

.5/3

22

04

.15

2.1

84.0

/21

20

6.1

59.

213

.125

/32

20

7.1

59.

9.1

18/3

2

源地址段

规则数目

目的地址段

Destination Source

Num

ber

of

Rule

s

(partially specified rules)

1.0

.0.0

/81

0.0

.0.0

/81

04

.0.0

.0/8

11

1.0

.0.0

/81

21

.0.0

.0/8

15

0.0

.0.0

/81

72

.0.0

.0/8

19

2.0

.0.0

/82

02

.0.0

.0/8

20

4.0

.0.0

/82

06

.0.0

.0/8

20

8.0

.0.0

/82

09

.0.0

.0/8

21

2.0

.0.0

/82

16

.0.0

.0/8

22

2.0

.0.0

/82

55

.0.0

.0/8

0

20

40

60

80

100

120

140

1.0

.0.0

/81

0.0

.0.0

/81

04

.0.0

.0/8

11

1.0

.0.0

/81

21

.0.0

.0/8

15

0.0

.0.0

/81

72

.0.0

.0/8

19

2.0

.0.0

/82

02

.0.0

.0/8

20

4.0

.0.0

/82

06

.0.0

.0/8

20

8.0

.0.0

/82

09

.0.0

.0/8

21

2.0

.0.0

/82

16

.0.0

.0/8

22

2.0

.0.0

/82

55

.0.0

.0/8

源地址段

规则数目

目的地址段Destination Source

Num

ber

of

Rule

s

(partially specified rules)

Topological Analysis of Service Policy (III)

• Forwarding policy vs. Service Policy

– Forwarding Policy

• Entire network view, fine-grained address objects, less rule

overlapping

– Service Policy

• Choke points of subnets, relationship of organization,

wildcard and more rule overlapping

• Implication for packet classification

– Rule distribution is asymmetrical

– Firewall and IP Chain type scenarios have more overlapping

– Either source IP or destination IP is highly separable


Outline

• Background

• Related Work







MBPE

• Principles

– Preserve Forwarding Rules

• On-datapath processing to reduce bandwidth cost introduced by traffic

steering

– Dispatch onto middleboxes

• From path-wise to network-wise to reduce wildcard rules replication

• Requirements

– Correctness

• Each policy or policy partition is processed once and only once

– Efficiency

• As less enforced nodes as possible


MiddleBox Policy Enforcement

Set Cover Problem (SCP)

• Given a set U of n elements and a collection S of subsets of U, find

the smallest collection C of S whose union is U

• 𝑺𝑷 is mapped to U and 𝑺𝒇 is mapped to S

MBPE modeled as SCP


𝑚𝑖𝑛 𝑥𝑗

𝑁

𝑗=1

subject to

∀𝑖 ∈ 1,… ,𝑀 : 𝑆𝑟𝑖𝑗

𝑁

𝑗=1

= 𝑆𝑟𝑖 , 𝑆𝑟𝑖𝑗⊆ 𝑆𝑟𝑖 ∩ 𝑆𝑓

𝑗

∀𝑗1, 𝑗2 ∈ 1,… , 𝑁 , 𝑗1 ≠ 𝑗2, ∀𝑖 ∈ 1,… ,𝑀 : 𝑆𝑟𝑖𝑗1 ∩ 𝑆𝑟𝑖

𝑗2 = ∅

𝑥𝑗 = 1, ∃𝑖 ∈ 1,… ,𝑀 , 𝑠. 𝑡. 𝑆𝑟𝑖

𝑗≠ ∅

0, 𝑜𝑡ℎ𝑒𝑟𝑤𝑖𝑠𝑒

𝑆𝑓 The flow space of all network nodes

𝑆𝑓𝑗 The flow space that the traffic

covers at enforced node 𝑗

𝑆𝑟𝑖 The rule space that 𝑖𝑡ℎ rule

covers

𝑆𝑟𝑖𝑗 The rule space that 𝑖𝑡ℎ rule


Greedy Algorithm

• Iteratively select the network node whose flow space can

cover most of the remaining policy space

• Enforce two types of rules in each iteration

– Bypass rule with high priority: subtraction of the remaining flow

space from the flow space of the selected node

– Enforced rule with low priority: intersection of the remaining

flow space of the selected node and the complete policy space

• Two heuristics to select network nodes

– Policy-oriented: node intersects with the maximum number of

policy rules

– Flow-oriented: the node has the maximum number of hyper-

rectangles


Evaluation

• Enforced nodes and rules


1

1.5

2

2.5

3

膨胀倍数

网络拓扑

基于相交规则数目基于子空间数目面向源端

1.0E+00

1.0E+01

1.0E+02

膨胀倍数

网络拓扑


1.0E+00

1.0E+01

1.0E+02

1.0E+03

部署节点（个）

网络拓扑


1.0E+00

1.0E+01

1.0E+02

1.0E+03

部署节点（个）

网络拓扑

基于相交规则数目基于子空间数目面向源端flow-oriented policy-oriented source-oriented

Network Topology

Num

ber

of

Nodes

Overh

ead o

f R

ule

s

flow-oriented policy-oriented source-oriented

Num

ber

of

Nodes

Network Topology

Network Topology Network Topology

Overh

ead o

f Rule

s

flow-oriented policy-oriented source-oriented flow-oriented policy-oriented source-oriented

# of all enforced rules / # of the original policy rules

exact policies wildcard policies

# of enforced nodes

Extended MBPE

• Practical constraints to Set Cover Problem

– Rule size of forwarding devices

– Processing capability of middleboxes


𝑚𝑖𝑛 𝑥𝑗

𝑁

𝑗=1

subject to

∀𝑖 ∈ 1,… ,𝑀 : 𝑆𝑟𝑖𝑗

𝑁

𝑗=1

= 𝑆𝑟𝑖 , 𝑆𝑟𝑖𝑗⊆ 𝑆𝑟𝑖 ∩ 𝑆𝑓

𝑗

∀𝑗1, 𝑗2 ∈ 1,… ,𝑁 , 𝑗1 ≠ 𝑗2, ∀𝑖 ∈ 1,… ,𝑀 : 𝑆𝑟𝑖𝑗1 ∩ 𝑆𝑟𝑖

𝑗2 = ∅

∀𝑗 ∈ 1,… , 𝑁 : 𝐹(𝑆𝑟𝑖𝑗)

𝑀

𝑖=1

≤ 𝐶𝑗

𝑥𝑗 = 1, ∃𝑖 ∈ 1, … ,𝑀 , 𝑠. 𝑡. 𝑆𝑟𝑖

𝑗≠ ∅

0, 𝑜𝑡ℎ𝑒𝑟𝑤𝑖𝑠𝑒

𝑆𝑓 The flow space of all network nodes

𝑆𝑓𝑗 The flow space that the traffic


𝑆𝑟𝑖 The rule space that 𝑖𝑡ℎ rule

covers

𝑆𝑟𝑖𝑗 The rule space that 𝑖𝑡ℎ rule


𝐹() Cost mapping function

𝐶𝑗 Constraints of node 𝑗

Outline

• Background

• Related Work







Policy Verification

• Distributed data plane policies verification

– Decompose network scope problem into path scope

problems according to forwarding policy

– Operate policies along the specified flow path

– Leverage the set operations of PSA


Policy Verification

• Distributed data plane policies verification

– Consistency

• 𝑆𝑓′ = 𝑆𝑓′ ∩ 𝑆𝑃

– Redundancy

• ∃ 𝑗1 ≠ 𝑗2: 𝑆𝑓′𝑗1 ⊆ 𝑆

𝑓′𝑗2 , 𝑆

𝑓′𝑗1 . 𝑎𝑐𝑡𝑖𝑜𝑛 = 𝑆

𝑓′𝑗2 . 𝑎𝑐𝑡𝑖𝑜𝑛

– Confliction

• ∃ 𝑗1 ≠ 𝑗2: 𝑆𝑓′𝑗1 ∩ 𝑆

𝑓′𝑗2 ≠ ∅, 𝑆

𝑓′𝑗1 . 𝑎𝑐𝑡𝑖𝑜𝑛 ≠ 𝑆

𝑓′𝑗2 . 𝑎𝑐𝑡𝑖𝑜𝑛


𝑓′ flow of one network policy

𝑆𝑓′ flow space

𝑆𝑃 The policy space of all rule space

𝑆𝑓′𝑗= 𝑆𝑟𝑖

𝑗∩ 𝑆𝑓′

𝑀

𝑖=1 rule space on node 𝑗

𝑆𝑓′ = 𝑆𝑓′𝑗𝑡

𝐾

𝑡=1

combined rule space on 𝑝𝑎𝑡ℎ of flow 𝑓′

Summary

• An orthogonal perspective of network

forwarding and network service

• A framework of policy space analysis

definitions and operations

• A few policy enforcement algorithms as

research in progress


June 17, 2016

Thanks


Documents

Distributed Network Service Policy Enforcement