View
215
Download
0
Category
Tags:
Preview:
Citation preview
CPR and GAMMONDeployment Experiences.
Warren Matthews Georgia Institute of
Technology.
Office of Information Technologyhttp://www.oit.gatech.edu
2
Georgia TechGeorgia Tech
• Campus– Backbone group maintain 180 buildings,
1700 switches, 55000 ports.
• Southern Crossroads gigapop (SOX)– Provides connectivity for 20+ Universities
throughout the South East– 10Gbps link to Abilene backbone.
Office of Information Technologyhttp://www.oit.gatech.edu
3
MotivationMotivation
• Measurement Infrastructure typically means WAN monitoring– But problems are LAN and host based
• Network Operations– Single point of view– Catastrophic failure is easier to detect– Little quantitative data to troubleshoot
performance problems
Office of Information Technologyhttp://www.oit.gatech.edu
4
CPRCPR
• Campus-wide Network Performance Monitoring and Recovery
• 50+ hosts on Campus– Collocated with switches in data closets– Multiple views of the network– Especially the users view
• Regular tests across campus network
Office of Information Technologyhttp://www.oit.gatech.edu
5
Gateway Routers
NI SI
Rich
DMSmith
Lyman
Skiles
ET
Servernet
845
811
EDI GLC
SSC
Ajax
Mason
FrenchOHR
Core Routers
FAB OKeefe KingGCATT
SOX
LAWN
IBB
Howey
Admin
SavannahGTL
MRDC
Rich2
OHR
SEB
Neely
MiRC
Cherry-Emerson
Couch
ES&T
MARC
Habersham
Lib-class
French-class
Sc-class
Arch
Daniel
Savant44
Savant
Weber
Boggs
GTRI
YamacrawClassTechSq Classroom
Lyman
505
Rich133
DeploymentDeployment
Office of Information Technologyhttp://www.oit.gatech.edu
6
GAMMONGAMMON
• Georgia Measurement and Monitoring– State-wide initiative
• Distance Learning and Professional Education (DLPE)
• Valdosta State University, Armstrong Atlantic State University, Barrow County School System.
Office of Information Technologyhttp://www.oit.gatech.edu
7
Other DeploymentsOther Deployments
• Local ISPs– Major providers (Level3, Qwest, Charter)– Residential (SpeedFactory, BellSouth,
Charter, Cox)
• Global collaborations– International focus in strategic plan – Metz, Shanghai– Leverage Global PMP Infrastructure and
communicate using emerging standards
Office of Information Technologyhttp://www.oit.gatech.edu
8
Deployment Deployment ExperienceExperience
• On the whole, very smooth• Control of Network
– Firewall– Physical access– Control of satellite server, DNS
• Warmly welcomed– Help is on the way– Pass the buck
Office of Information Technologyhttp://www.oit.gatech.edu
9
Deployment Deployment ExperienceExperience
• Donated hardware– Cheap– Flakey– Scavenging of parts before donation
• GAMMON/International deployment faces more of the traditional challenges
Office of Information Technologyhttp://www.oit.gatech.edu
10
ToolsetToolset
• No in-house development of measurement tools.
• Original plan also didn’t include much visualization.– Inconvenient to click through
numerous graphs
Office of Information Technologyhttp://www.oit.gatech.edu
11
MeasurementsMeasurements• Currently
– Smokeping - roundtrip time and graphs.– Nagios - Services.– Security - nessus and nmap.
• Also available– Iperf (bwctl) - TCP throughput only.– Pathchar, traceroute
• Coming soon– NDT (central, distributed)– Test bed for tools under development
Office of Information Technologyhttp://www.oit.gatech.edu
12
Measurement Measurement ExperiencesExperiences
• Passive Monitoring of border traffic– Bwctl.sox is outside GT.
• OWAMP– NTP
• Alleged network problems are often issues with applications– Added application monitoring using Nagios
• Information Security– CPR as a platform for other groups, research
Office of Information Technologyhttp://www.oit.gatech.edu
13
AnalysisAnalysis
• Analysis– Create base-lines for historical comparison– Use multiple view to detect location– Middleware
• Alarm system– Plateau detector (AMP), RIPE-TT– How should we react to alarms?– Troubleshooting guide
Office of Information Technologyhttp://www.oit.gatech.edu
14
Analysis ExperienceAnalysis Experience
• Initial design was active monitoring– Integrate with passive tools– Wireless tools, NETI– Provide a platform for localized netflow
reports
• Initial design was to avoid looking at graphs– Auto detect and alarms– MyCPR
Office of Information Technologyhttp://www.oit.gatech.edu
15
MyCPRMyCPR
• Configurable alarms and graphs
Office of Information Technologyhttp://www.oit.gatech.edu
16
Case StudiesCase Studies
• CPR has helped solve numerous issues– Firewall– Network slowness for file sharing– Dropped sessions
• Not everything is a network issue
Office of Information Technologyhttp://www.oit.gatech.edu
17
ExperiencesExperiences
• Don’t assume anything– If you don’t measure, you don’t know.
• Don’t expect people to do what they should
• When one door closes, another one opens.
Office of Information Technologyhttp://www.oit.gatech.edu
18
This is the endThis is the end
• Contact– Warren.matthews@oit.gatech.edu
• Project WebSite– http://www.rnoc.gatech.edu/cpr
Recommended