The Technical Network in brief Jean-Michel Jouanigot & all IT/CS

The Technical Network in briefThe Technical Network in brief

Jean-Michel Jouanigot & all

IT/CS

26-March-2003The Technical network in brief - JCOP2

IntroductionIntroduction

• Introduction on Ethernet/IP technologies• The New Technical Network• Networking at the Pits


What is a HUB ? What is a HUB ? (CERN definition)(CERN definition)

• Shared medium (one cable)

• Collisions arbitrate access (CSMA/CD)

• Non deterministic• Shared Bandwidth

« Coax »

Ethernet hub

• A Repeater• UTP5 (structured cabling)• Removes cabling problems

• Wireless is a HUB (CSMA/CA)


What is a switch?What is a switch?

• A Point to Point system (structured cabling only)• Isolation

• One collision domain per port (the switch and the station)

OR

• Full duplex = No Collision (both can send/receive at the same time)• Dedicated bandwith per station, Central Intelligence, auto learning

Ethernet Switch >> 1Gbps

10 100 1000


What is a Router?What is a Router?

• Ethernet does not scale• All stations have a unique 6 bytes address• The switches must auto-learn where the stations are

• Leaking (aging, unknown destinations, etc)• Broadcasts/Multicasts

• Redundant networks difficult to implement (loops)• IP uses Ethernet as a transport; Routers use IP addresses

• Allocates a Topological address to a station• Routers IMPOSE a topology :: « regions » (=networks)• Routers run routing protocols to find the best path to a

(sub)network.• IP addresses and routers implement a topology


IP creates networksIP creates networks

Routers find the best path Routers find the best path

betweenbetween networks networks

Basic resiliencyBasic resiliency

• Ethernet alone does not allow efficient resiliency• No load sharing• No optimum path• Loop avoidance only


Physical

What is a Firewall?What is a Firewall?

• Look deep inside each packet• Applies security policies

• Access control (sources, destinations, etc)• Sessions (port numbers, connection

establishment, etc)• State full

• Understands some Applications• E.g. FTP, HTTP, etc

• Analyses Potential Attacks• Denial of service• Malicious applications

Medium

Network

Transport

Session

Presentation

Application

1

2

3

4

5

6

7H

ub/S

wit

ch

Rou

ter

Fire

wal

l

FirewallINSIDENetwork to protect

OUTSIDE


SummarySummary

Speed

Price

Complexity

The CERN campus NetworkThe CERN campus NetworkIn a nutshell


A large Infrastructure (1)A large Infrastructure (1)

• 90 Gigabit Ethernet Routers; 900 subnets• 390 switches (~10’000 ports)

• 150 (38%) in 513-C

• 860 Ethernet hubs (~20’000 ports)• 15’000 active connections

85% outside 513-C• 32’000 sockets (1’200km of UTP cable)• 160 starpoints (from 20 to 1’000 outlets)• Multi manufacturer site


CERN MULTI-GIGABIT BACKBONECERN MULTI-GIGABIT BACKBONEREDUNDANT STRUCTURE OVERVIEWREDUNDANT STRUCTURE OVERVIEW

Technical Network

CO

MPU

TE

R C

EN

TE

RR

EM

OT

E M

AJO

R ST

AR

POIN

TS

B513-B-1

B513-C-1

B513-C-3

B2-S

B513-C-2

B887-R

B874-R

B40-S2

B376-R

..etc..

B10-1

ServerFarms

B513-B-2

B513-C-4

Firewall

CIXP,Internet13 Xpedition 8600

56 Xpedition 8000

B513-C-5..etc..

New Technical NetworkNew Technical NetworkGeneral OverviewGeneral Overview

The new Technical Networkwill replace the SPS, PS and “service” networks and Integrate LHC controls

All merged into ONE single infrastructure


The New technical NetworkThe New technical Network

• One global infrastructure but two (sub)projects• For LHC

• Redundant infrastructure• Surface topology ready by end 2002 (no full redundancy)

• For SPS, PS, Meyrin, Prevessin• Rejuvenation plan without redundancy and 100 Megabit/s

backbone

• Integrated, with the GPN, into one single 24x24, 365x365 support schema

• One database• One helpdesk• Same tools and procedures

• Taking care of some specificities


TechNet specificitiesTechNet specificities

• Maintenance windows agreed with Technical and Accelerator sectors

• Priority in case of failure because carries security information (“alarmes de niveau 3”)• CSAM (fire detection)

• Access Control/Interlock, etc

• No direct access from Internet (security)

Dedicated active infrastructure


The Global viewThe Global view

Fuse


Rejuvenation: In practice…Rejuvenation: In practice…

• A LOT of work• ~2’000 plugs to adapt, hundreds of equipment to install

• Planning• LHC area: Backbone ready, connections in progress (surface)• PS area: 4Q2002-1Q2003: COMPLETED• TCR, Meyrin, LHC surface: 2Q2003-3Q2003 STARTING• SPS: 4Q2003-1Q2004

• A new network prefix• 128.142 replaced by 172.18

• Fuse between the Technical network and the GPN• Independent DNS, TS, etc managed by CS

• Tools adaptations• Network monitoring (Spectrum will replace OpenView)• Database adaptations (SL network database integrated)• End node monitoring


GeneralPurpose Network

GeneralPurpose Network

SR2

SR1

SR3

SR4

SR5

SR6

SR7

SR8

TECHNICAL NETWORK GIGABIT BACKBONETECHNICAL NETWORK GIGABIT BACKBONEREDUNDANT STRUCTURE OVERVIEWREDUNDANT STRUCTURE OVERVIEW

PCR

CCR

MCR

TCR

PCR

CCR


TECHNICAL NETWORK – LHC TOPOLOGIE OVERVIEWTECHNICAL NETWORK – LHC TOPOLOGIE OVERVIEW

P1 P2

RE18 RE22

US15 US25

SR2SR1

LHC TUNNEL Done!


Networking for Physics in PitsNetworking for Physics in Pits

• The experiments will have large clusters at the Pits• Thousands of (gigabit?) connections…

• The physics will be connected to the GPN:• Very high speed link(s) to the Computer Center

• Possibly redundant via fibers in tunnel

• At least 10 Gbps for data acquisition connected to the computer center farms switching fabrics

• One Gbps for experiment control

From the first discussions with experiments…

The Technical Network in BriefThe Technical Network in BriefQuestions?

Documents

The Technical Network in brief Jean-Michel Jouanigot & all IT/CS