How To Build a Redundant, Fault Tolerant, Load Balanced,

Traffic Managed Wireless network

a bella mia company

A Little BackgroundA Little Background

• Jeremy Kinsey “Known as Jer” to his friends.

• CEO and Co-Founder of Bella Mia, Inc.

• Been an ISP for 6 years• Been a WISP for 2 + years

So you are a Wireless ISPBut how do you make your

system Redundant?

Some Some AssumptionsAssumptions

• You have at least 2 WPOPS (Wireless Points of Presence)

• You already have a redundant network, ie., are running BGP, have two providers, etc.

• You have a good understanding of Cisco Routers and various routing protocols.

Assuming you have all Assuming you have all that, lets take a look at that, lets take a look at how things break down.how things break down.

Break down of Redundant WPOP Setup

PTPWireless Link to Tower 2

WPOP 1Your Wireless POP/TowerLand line from your NOC

Cisco Router

Cisco RouterBandwidth Manager

PTPWireless Linke to Tower 1

WPOP 2Your other Wireless POP/Tower

Land Line from your NOCCisco Router

NOCYour Network Operations CenterConnected to multiple Providers

Running BGP

Lets Start at the WPOPLets Start at the WPOP

• 3 180 degree Sectors

• 3 runs of Heliax• 4 radios Proxim

COR’s• Cisco 1600 Series

Router• UPS• Switch

Where do the pieces fit?Where do the pieces fit?

• One Sector per side of tower

• One Radio per sector

• UPS, Radio, Switch, Router in enclosure at tower base.

• Router connected to T1 and NOC

The Big PictureThe Big Picture

Internet

NOCRouter

Bandwidth Manager

WPOP 1 WPOP 2T1T1

Wireless PTP

So what do we have?So what do we have?Redundancy at the WPOP (3 separate radios & antennas)

Redundancy in the entire system (multiple paths, traffic re-routed on the fly)

Shared bandwidth (load balanced traffic, and bandwidth delivered where it is needed on demand)

Another Key Point!Another Key Point!

_ Imagine Zero Down Time• Easily add to existing infrastructure• Upgrade Remote Devices• Replace Downed Gear

MaintenanMaintenancece!!

But does it Work?But does it Work?

Well Let’s See it in Action!

Some ExamplesSome ExamplesWPOP 1 Aggregate Traffic

WPOP 2 Aggregate Traffic

Traffic Flow over PTP between WPOP 1 & WPOP 2

What does it look like when What does it look like when a pipe goes down?a pipe goes down?

• 1 Pipe goes down, traffic automatically re-routed over backup link

• Pipe comes back up, PTP traffic will return to normal• Your users never know what happened

PTP under normal conditions

PTP with 1 T1 Down

Where do you go from Where do you go from here?here?

Endless Expansion!Endless Expansion!

Internet

NOCRouter

Bandwidth Manager

WPOP 1 WPOP 2

T1

T1

Wireless PTPWireless PTP

Wireless PTP to NOC

WPOP 4

WPOP 3

Wireless PTP

So How Do I Do It?So How Do I Do It?

• Now for the routing side of the “Keeping It Up”session

• Josh Easton, CTO Bella Mia, Inc.• Routing protocols used and why

OSPFOSPF

• Open Shortest Path First• Allows for redundancy• Smarter routing of traffic

Brief intro to OSPFBrief intro to OSPF

• Every network link (T1, ethernet, etc.) has a cost

• When multiple paths are available, lowest cost path is chosen

• Costs can be manipulated to route your traffic ideally

How does OSPF fit in?How does OSPF fit in?

Internet

NOCRouter

Bandwidth Manager

WPOP 1 WPOP 2T1T1

Wireless PTP

Cost = 64

Cost = 64

Cost = 10

Why these costs?Why these costs?

• T1 has a default cost of 64• 10Mbit ethernet’s default cost is 10• Traffic from WPOP1 to WPOP2 will take

the wireless PTP link if it’s available, else it will use T1s

• Traffic from WPOP1 to Internet goes via the local T1, else via the wireless PTP link to WPOP2 and then via WPOP2’s T1

Sample Cisco router Sample Cisco router configconfig

• Note: Need IP+ image to do OSPF on 1600interface Ethernet0 ip address 10.0.0.1 255.255.255.0 ! Just an example ip address ip ospf message-digest key 1 md5 <Put OSPF password here> exitinterface Serial0 ip address 10.0.1.1 255.255.255.0 ! Just an example ip address ip ospf message-digest key 1 md5 <Put OSPF password here> exit router ospf 100 redistribute connected subnets redistribute static subnets network 10.0.0.1 0.0.0.0 area 0 network 10.0.1.1 0.0.0.0 area 0 default-information originate always area 0 authentication message-digest ospf log-adjacency-changes exit

OSPF in a larger networkOSPF in a larger network

Internet

NOCRouter

Bandwidth Manager

WPOP 1 WPOP 2

T1T1

Wireless PTPWireless PTP

Wireless PTP to NOC

WPOP 4

WPOP 3

Wireless PTPCost = 64

Cost = 64

Cost = 10

Cost = 45

Cost = 10

Cost = 10

Why these costs?Why these costs?• Cost from WPOP1 to WPOP3 is manually

set at 45• Traffic from one WPOP to another WPOP

will use the wireless backbone by default• WPOP3 and WPOP4 use WPOP4’s link to

the NOC to get on the Internet• WPOP1 and WPOP2 use their T1s to get

on the Internet• If any link goes down, traffic will be

routed around the damage

Questions?Questions?