High-Availability Designs for Juniper NetScreen Firewalls

Copyright © 2005 Juniper Networks, Inc. Proprietary and Confidential www.juniper.net 1

High-Availability Designs for Juniper NetScreen

Firewalls

Dan BackmanSenior Systems [email protected]

2Copyright © 2005 Juniper Networks, Inc. Proprietary and Confidential www.juniper.net

Routing and Firewall Functions MergingNew JUNOS Routing platforms (J / M) and AS PIC• Stateful firewall, IPsec and NAT services in JUNOS

Expanded Routing functionality in NetScreen platformsNew solutions possible:• Stateful Firewall, NAT, IPsec VPN termination and

Dynamic Routing

+ !+ =


Routing and Firewall Functions MergingTraditional uses of dynamic routing in firewalls:• Dynamically advertise reachability of connected services

• Statically routed VPNs advertised into IGP/iBGP• Dynamic path calculation

• Firewalls participate in routing (usually RIP)• Limited control plane impacts

• Relatively few prefixes• Limited policy/redistribution

Today:• Deployments require:

• Interchangable routing / firewall features• Juniper delivering integrated feature sets

• AS PIC / J Series SFW/IPsec• Increasing routing functionality in ScreenOS


JUNOS / ScreenOS Routing StrengthsVirtualization• Native support for multiple routing tables

• Multiple VRF and Logical routers in JUNOS• At least two Virtual Routerss in all ScreenOS platforms

– Allows simple split tunneling at edge• Hundreds of VRs in NetScreen Systems• Multiple instances of routing protocols in JUNOS and ScreenOS

Scalable, standards-based routing protocols (OSPF/BGP/RIPv2)PIM-SM and IGMP Proxy for dynamic multicast forwardingDynamic route-based VPNs• Support for policy and route-based VPNs in ScreenOS and JUNOS


ScreenOS Dynamic RoutingScreenOS is designed for integrated Firewall / Routing• Security platform from the ground-up• Integrated static and dynamic routing support• Multiple virtual IPv4 routing tables / Multiple routing instances

Security Features• Screen function

• DoS, IP spoofing, L3/L4 protocol anomaly detection

• Flexible security zone model for all policy• Network interfaces bound to security zones• Sessions / flows bound to zones ,not interfaces• Allows real-time next-hop changes to existing flows• Critical to support dynamic routing in a firewall


High Availability ScenariosFirewalls integral part of routing topology –need redundancy solutions• Border protection (Screen/Policy)

• Inline to forwarding path at network border• Logical progression for integrated IDP

– Add IDP into forwarding path with fewer headaches

VPN Routing Edge• Redundant VPN termination at site• Stateful failover without dynamic

routing impact


Stateful FailoverTrue security boundaries require stateful inspection• Firewalls track individual network flows

• Provide stateful enforcement of policies and DoS protection

Redundancy requires stateful awareness• Firewall Cluster must support state synchronization

Failover without state sync:• Results in loss of existing TCP/UDP sessions• Users must restart existing protocol connections

Traditional firewall state sync does not account for dynamic routing


Classic Firewall HA Scenario“Ten-Pack” of routers, switches, firewalls, switches and routers• HSRP/VRRP/NSRP virtual addresses for

next-hop• Static routing

Pros:• Simple. No dynamic routing• No asymmetric state• Supports all firewall features/functions

Cons:• May require redundant interfaces • No dynamic routing through firewalls• Requires additional devices (L2 switches)

S T A T U S

P O W E R

5 2 0 0

C O M P A C T F L A S H 1T X / R XL I N K

1 0 / 1 0 0

A L A R MS T A T U SH AS E S S I O NF L A S H

C O N S O L EM O D E M

5 0 0 0 - M G T

5 0 0 0 - 8 G

1

2S T A T U S

P O W E R

5 2 0 0


1 0 / 1 0 0



5 0 0 0 - M G T

5 0 0 0 - 8 G

1

2HA Link

UNTRUST

TRUST

Master

Backup


Dynamic Routing / Firewall HA Scenario


Firewalls in a Dynamic Routing Topology: Why?Customer desire to integrate firewalls into existing network topology• Must support dynamic failover

based on OSPF• Contiguous OSPF area

• Full Link State in network edge

• Advertise prefixes betweeninternal network and external routers

• Must support PIM-SM for multicast routing (ScreenOS 5.1)

Interop eNet Design• NSRP VSD-less clusters originally designed

for this topology 2 years ago


NetScreen Redundancy ProtocolOriginally designed to support stateful failover• Never intended to support asymmetric state

VSD – Virtual Security Device• Logical failover domain within firewall• Master / Backup state machine per VSD

VSI – Virtual Security Interface• Shared interface (Virtual IP/MAC pair)• Maps traffic into VSD

RTO Mirror – Real Time Object Mirroring• State sync in NSRP cluster


NSRP: Traditional (L3) DesignVirtual addressing • NSRP VSI and VRRP or HSRP on

routers• All virtual MAC addresses as

next-hop between routers and firewall cluster

• Static routes throughout topology

Single VSD for all trafficAll firewall interfaces are virtual interfaces (VIP/MAC)• Easy to add additional

zones/interfaces (DMZ) • No asymmetric state

S T A T U S

P O W E R

5 20 0

C O M P A C T F L A S H1

T X / R XL I N K

1 0 / 1 0 0



5 0 0 0 - M G T

5 0 0 0 - 8 G

1

2S T A T U S

P O W E R

5 20 0

C O M P A C T F L A S H1

T X / R XL I N K

1 0 / 1 0 0



5 0 0 0 - M G T

5 0 0 0 - 8 G

1

2HA Link

UNTRUST

TRUST

NSRPBackupNSRP Master

VRRPBackupVRRP Master

VRRPBackupVRRP Master

Virtual AddressStatic Routes

Virtual AddressDefault Route

Virtual AddressStatic Routes

Virtual AddressDefault Route


NSRP: Traditional (L2) DesignFirewall operates as logical L2 learning bridge• Backup is in L2 blocking state• Must permit IGP adjacencies

through firewall• No asymmetric state

Topologies• Support for proprietary IGPs• “drop-in” / transparent firewalls

S T A T U S

P O W E R

5 2 0 0


1 0 / 1 0 0

A L A R MS T A T U SH A S E S S I O NF L A S H


5 0 0 0 - M G T

5 0 0 0 - 8 G

1

2S T A T U S

P O W E R

5 2 0 0


1 0 / 1 0 0



5 0 0 0 - M G T

5 0 0 0 - 8 G

1

2


Transparent Mode NSRP (L2) OperationOperate as logical L2 bridge• MAC learning and forwarding• Policy engine and forwarding still based on 5-tuple

Must carefully engineer DMZ topology• ICMP redirect cannot force traffic across

zone boundaryLimited support for VLANs• VLAN tags preserved, but single inspection domain• No current support for VLAN tag rewrite

• Enhancement coming in next major ScreenOS release


NSRP Real-Time Object SyncWhat is synchronized?• Sessions / IPsec SA /

Crypto and VSD Configs• Master Backup replication

in VSD• Bi-Directional replication in

VSD-less cluster

What is not synchronized?• Screens (pre-flow

processing counters)• Application Level Gateways• TCP Setup / Inspection

S T A T U S

P O W E R

52 0 0


1 0 / 1 0 0



5 0 0 0 - M G T

5 0 0 0 - 8 G

1

2S T A T U S

P O W E R

52 0 0


1 0 / 1 0 0



5 0 0 0 - M G T

5 0 0 0 - 8 G

1

2

HA Link(s)

UNTRUST

TRUST

NSRPBackupNSRP Master

RTO MirrorMaster Backup

Normal Traffic Traffic on Failover


NSRP OperationMaster/Backup state machine run per VSD• Priority and tracking (weight-based) determines

master eligibility• Tracking: interface / IP reachability (ping) / Zone

Master assumes virtual IP/MAC addresses for VSI • Physical interfaces in VSD 0• Additional VSI (eg: eth2/1:1)

Master synchronizes state to Backup deviceBackup blocks ports in L2/Transparent mode


NSRP State Control: TrackingNSRP can track various factors to determine master eligibility • Applies per VSD• Administrative weight per tracked object• Failover threshold per VSD

Track:• Multiple IP addresses

• Weight per address• Interfaces• Zones

• Behaves like VLAN on L3 switch • any one interface with link == zone up


OSPF and NSRP (The Wrong Way)VERY slow failover (40-60 sec) when using OSPF and NSRPDoes support NSRP RTO mirror for session sync

• NSRP backup has “down” interfaces in VSD id 0

• OSPF adjacency is “down” when in backup state

• On failover:1. Interface up

2. Reestablish OSPF adj. (must wait OSPF Dead Interval)

3. Database exchange

4. SPF calc

5. Populate routes

• THEN, can begin forwarding traffic

S T A T U S

P O W E R

5 2 0 0


1 0 / 1 0 0



5 0 0 0 - M G T

5 0 0 0 - 8 G

1

2S T A T U S

P O W E R

5 2 0 0


1 0 / 1 0 0



5 0 0 0 - M G T

5 0 0 0 - 8 G

1

2


Dynamic Routing Clusters (1): JustificationDesire to integrate firewall into IGP • Multiple egress paths, integrate into IGP routing• Control advertisement of default or external routes into IGP

based on exterior connectivity• Continuity of IGP routing across firewalls• OSPF-based dynamic route selection• Simplified topology (no L2 switching required)

ScreenOS modified (early 5.0x) to abstract sessions from interface to zone. • Allows route update to new next-hop without invalidating

existing sessionsNew NSRP mode needed to keep routing adjacencies up


Dynamic Routing Clusters (2): OperationDual Masters in VSD id 0 Bi-directional RTO mirroring between cluster members• All physical interfaces remain active and can support

active routing protocol adjacencies• All devices in cluster can actively forward traffic

Same as running OSPF on non-clustered devices, but adds session syncConfig:• Must manually “unset vsd id 0”• “set nsrp rto-mirror session non-vsi”


Primary limitations:• VERY susceptible to asymmetric state issues• Require more complex config (mixed mode) for NAT support• Policy-based VPNs also require

• In both cases, traffic must return to a single address which maybe resident on both devices

Cannot use Data-Path Forwarding as a band-aid• Both nodes are Master: only backup node can perform

data-path forwardingMust use “Mixed-mode” NSRP to address these issues• Unset VSD id 0• Virtual interfaces in VSD id 1 (loopback for VPN, NAT Pool)

Dynamic Routing Clusters (3): Caveats


HA Considerations: Stateful forwardingReal Stateful Inspection requires bidirectional forwarding• Traditional routing protocols do not guarantee

symmetric bidirectional traffic flows• ECMP nearly guarantees asymmetric state

• True stateful load balancing requires reverse hash for returning microflows

• NetScreen firewalls use session/flow state for all forwarding paths

• Required for stateful policy inspection

• J/M/T/E series use stateless forwarding• LPM / J-Tree lookup per-packet on forwarding

and firewall filters


ScreenOS – Session StateAll forwarded traffic must have a session • Contains bidirectional flow information

• Route lookup determines egress zone• Policy lookup from ingress to egress zone

• NetScreen Systems forward traffic like L3/L4 switches5200-17(M)-> get sessionslot 1: sw alloc 3/max 1000064, alloc failed 0, mcast alloc 0, di alloc failed 0slot 2: hw0 alloc 1/max 1048576 slot 2: hw1 alloc 1/max 1048576 id 7267/s**,vsys 0,flag 00000040/0080/23,policy 320002,time 6, dip 011(0601):10.2.4.2/1->224.0.0.5/1,89,000000000000,15,vlan 0,tun 0,vsd 0,route 03(0010):10.2.4.2/1<-224.0.0.5/1,89,000000000000,4,vlan 0,tun 0,vsd 0,route 0id 7268/s**,vsys 0,flag 00000040/0080/23,policy 320002,time 6, dip 07(0601):10.1.4.1/1->224.0.0.5/1,89,000000000000,14,vlan 0,tun 0,vsd 0,route 03(0010):10.1.4.1/1<-224.0.0.5/1,89,000000000000,4,vlan 0,tun 0,vsd 0,route 0

id 7269/s01,vsys 0,flag 10200440/0000/03,policy 1,time 1440, dip 011(0801):10.2.2.2/11033->10.1.255.1/23,6,00a0c96cce14,15,vlan 0,tun 0,vsd 0,route 747(4800):10.2.2.2/11033<-10.1.255.1/23,6,00a0c92490e4,14,vlan 0,tun 0,vsd 0,route 44

Total 3 sessions shown


Asymmetric State: Symptoms“Split-state” environment may appear to work in the lab• BUT: TCP handshake never completed

through same device• Half-open sessions: User sees TCP

sessions establish but freeze (short-lived TCP sessions)

• Can “disable syn checking” but lose effective TCP inspection and protection

• ALG cannot fully inspect control channels• Deep Inspection will fail• Integrated IDP will fail• “pinholes” may not open correctly

• Some screening functions may depend on bidirectional traffic

S T A T U S

P O W E R

5 2 0 0


1 0 / 1 0 0



5 0 0 0 - M G T

5 0 0 0 - 8 G

1

2S T A T U S

P O W E R

5 2 0 0


1 0 / 1 0 0



5 0 0 0 - M G T

5 0 0 0 - 8 G

1

2


IGP Costing Exercise (1)Predictable forwarding path• Ensure bidirectional path through firewalls• Must not allow transit through

multiple firewalls• If ABRs directly connected to firewalls,

make sure there is a valid Intra-Area route between ABRs in firewall area

IGP costing is unidirectional• Must be careful to set IGP costing

bidirectionally (must configure both sides of a link to the same cost)

• Do NOT rely on automatic costing (varies between vendors and equipment types)

S T A T U S

P O W E R

5 2 0 0


1 0 / 1 0 0



5 0 0 0 - M G T

5 0 0 0 - 8 G

1

2S T A T U S

P O W E R

5 2 0 0


1 0 / 1 0 0



5 0 0 0 - M G T

5 0 0 0 - 8 G

1

2


IGP Costing Exercise (2)Predictable failover• Control traffic paths in the

event of a link-down event• This design preseves state

through a firewall in a single link-break

Fast IGP failover:• No split link • Can use aggregated

interfaces between devices • Use /30 p2p links to skip

dead timer / DR election on link-up

S T A T U S

P O W E R

5 2 0 0


1 0 / 1 0 0



5 0 0 0 - M G T

5 0 0 0 - 8 G

1

2S T A T U S

P O W E R

5 2 0 0


1 0 / 1 0 0



5 0 0 0 - M G T

5 0 0 0 - 8 G

1

2


IGP Costing Exercise (3)IGP Costing Dangers:• Routed DMZ Network• Do not allow transit

between firewalls• Carefully control costs within

the OSPF area• Watch out for asymmetric costs• Use separate VR for DMZ

network if necessary• Carefully test all iterations in a

failover topology

S T A T U S

P O W E R

5 2 0 0


1 0 / 1 0 0



5 0 0 0 - M G T

5 0 0 0 - 8 G

1

2S T A T U S

P O W E R

5 2 0 0


1 0 / 1 0 0



5 0 0 0 - M G T

5 0 0 0 - 8 G

1

2

External Router-A

External Router-B

Internal Router-A

Internal Router-B

DMZ Router


NSRP: Data-Path ForwardingNSRP can correct asymmetric state in some situations• 2) BACKUP device receives

packet that matches session from master

• 3) packet is exception-forwarded (CPU forwarded) to master over HA link

• 4) MASTER forwards packet to end node

Do not rely on this behavior• Serious performance impact

for large amounts of forwarded traffic

S T A T U S

P O W E R

5 2 0 0


1 0 / 1 0 0



5 0 0 0 - M G T

5 0 0 0 - 8 G

1

2S T A T U S

P O W E R

5 2 0 0


1 0 / 1 0 0



5 0 0 0 - M G T

5 0 0 0 - 8 G

1

2


Mixed-Mode NSRP (Simple)Medium-sized enterprise• Upstream OSPF to routers• Downstream (Trust)

• Firewall cluster is first-hop router for internal network

• Virtual IP/MAC in Trust VSI• VSI exported to OSPF

Pro:• Simple integration of OSPF and

Firewalls• No Asymmetric State

Cons:• Requires both VSD-less (untrust)

and VSD/VSI (trust)

S T A T U S

P O W E R

5 2 0 0


1 0 / 1 0 0



5 0 0 0 - M G T

5 0 0 0 - 8 G

1

2S T A T U S

P O W E R

5 2 0 0


1 0 / 1 0 0



5 0 0 0 - M G T

5 0 0 0 - 8 G

1

2

HA Link

VSI: Shared Address

OSPF(VSD-less)

UNTRUSTOSPF Area X

TRUST(L2)


Mixed-Mode NSRP (VSD-less + DMZ)Add DMZ network to existing VSD-less NSRP clusterPros:• Allows for DMZ network

connected to OSPF meshed network

Cons:• Must control asymmetric state

with OSPF costing• Requires both VSD-less and

VSD/VSI support

S T A T U S

P O W E R

5 2 0 0


1 0 / 1 0 0



5 0 0 0 - M G T

5 0 0 0 - 8 G

1

2S T A T U S

P O W E R

5 2 0 0


1 0 / 1 0 0



5 0 0 0 - M G T

5 0 0 0 - 8 G

1

2HA Link

UNTRUSTOSPF Area X

TRUSTOSPF Area X

DMZVSI

OSPF Passive


Mixed-mode NSRP ComplicationsMust link NSRP and OSPF failover in mixed mode• OSPF makes path calculations

based on link state information from routers

• NSRP elects master based on tracking information and priority

• Unidirectional feedback• Add VSI as OSPF

passive interface

• Recommend adding NSRP zone tracking or IP ping tracking to control NSRP failover

S T A T U S

P O W E R

5 2 0 0


1 0 / 1 0 0



5 0 0 0 - M G T

5 0 0 0 - 8 G

1

2S T A T U S

P O W E R

5 2 0 0


1 0 / 1 0 0



5 0 0 0 - M G T

5 0 0 0 - 8 G

1

2HA Link

UNTRUSTOSPF Area X

TRUSTOSPF Area X

DMZVSI

OSPF Passive

OSPF Trust-Untrust Transit Path

VSD 1 Backuplo0

VSD 1 Masterlo0

X

OSPF Untrust-DMZ Transit Path

NAT from loopback1:1


Questions?


Thank You

Documents

High-Availability Designs for Juniper NetScreen Firewalls