Why Not 100GbE as Service ~ JPIX Perspective ~

Why Not 100GbE as Service~ JPIX Perspective ~

Yoshiki Ishida, Takejiro TakabayashiJPIX

(Japan Internet eXchange Co., Ltd.)

What Motivates 100GbE Service?

• Customer (Member) Requirement• Traffic• Technology & Operation• Cost

Traffic Background(1) ~ Europe ~

• IX points Traffic are keep growing

• The peak traffic of three global IX points are over 2 tera bit per second. – AMS-IX(NL)– DE-CIX(DE)– LINX(UK)

https://ams-ix.net/technical/statisticshttps://www.de-cix.net/about/statistics/https://www.linx.net/pubtools/trafficstats.html

Source:

Captured date: Jan. 28, 2015

100GbE in Europe ~ No Surprise ~

• 100G services are available at these major IX Points – AMS-IX and Limelight Networks Field-Test Brocade 100

Gigabit Ethernet Technology to Stay Ahead of Internet Growth - June 2011 (*1)

– DE-CIX Completes 100G Optical Layer Build for New Flagship Apollon Platform - May 2013 (*2)

– BT expands its 21st Century Network as the London Internet Exchange deploys 100 Gigabit Ethernet across their leading edge IP/MPLS Internet Exchange - July 2012 (*3)

Traffic Background(2) ~ ASIA ~

http://www.hkix.net/hkix/stat/aggt/hkix-aggregate.htmlhttp://www.mfeed.co.jp/jpnap-tokyo-i/traffic.html

Source:

• HKIX ≈400Gbps peak– 100GbE available

• JPNAP ≈400Gbps peak– 100GbE available


Traffic Background(3) JPIX

• JPIX’s traffic is keep growing 　 ≈ 300Gbps

• Some our customers, especially Hypergiants and FTTH providers, use multiple 10G Interfaces. And finally…

http://www.jpix.ad.jp/en/technical/traffic.htmlSource:


Press Release - July 9, 2014

Press Release - July 9, 2014• Japan Internet Exchange Co., Ltd., (hereafter referred to as “JPIX”) an IX

provider that has a largest number of customers in Japan, will launch a new IX port service, 100 Gigabit Ethernet Port, in July. Brocade Communications Systems, K.K., (hereafter referred to as Brocade) announces that it provides the Brocade MLXe Series, deployed as switch to support the new service.

• Dwango Co., Ltd., a company that provides Japan’s largest video sharing service, “niconico”, and a wide range of entertainment contents and distributes vast amounts of content, will become the first user of the service.

• The equipment for the new IX port service, 100 Gigabit Ethernet Port, are the Brocade MLXe Series provided by Brocade.

• Brocade MLXe Series, which include a 100 Gigabit Ethernet (GbE)-enabled model released for the first time in the industry, were selected to equip the 100 GbE Port Service. As a result of Brocade’s feasibility testing for various interfaces, it was proven that the 100 GbE Port Service can be provided through main interfaces, including 10×10, 100G Base-LR4, and 100G Base-ER4.

• JPIX continues to strive to provide effective services and quality leading to stable operation, which can satisfy customers, such as ISPs and content providers, in order to respond to continuously increasing Internet traffic.

100GbE motives PROS & CONS

• Higher Speed– Faster the better

• Less operation– Less LAG, Simplex

• Less consideration– Once you upgrade, you

are free from concerning upgrading the bandwidth

• Opens Opportunity

• Stability/Reliability/RISK– 100G affects too much

traffic

• Need experience…

• COST– CAPEX/OPEX may higher

than 10x 10G

• Opportunity Loss

PROS CONS

Issues for deploying 100G Service~ burdens ~

• Upgrading the IX Architecture– Needed to upgrade our BIG 100GbE supported switch act as

spine and 40GbE supported switch act as leaf switch• Verifying 100GbE Interface

– Optical Switch testing needed• What if one wavelength drops optical power…

• Selection of the Optical Interface and the Optical Module– Shall we wait or go?

• 100G Ethernet Tester– What can we do about it…? Is it CFP or Backplane…?

Upgrading IX Architecture• Distributed Architecture

• Spine and Leaf Architecture

Verifying 100G Interface

100G Ethernet uses Multiple Waves on a FiberVerifying 100G Interface for our Architecture

( eg. Optical A/B Switch)

*4

Optics and Optical Module

• Optics– LR4(IEEE)

• 10Km SMF/4x25Gbps

– ER4(IEEE)• 40Km SMF/4x25Gbps

– 10x10(10x10MSA)• 2KmSMF/10x10Gbps• 10KmSMF/10x10Gbps

• Optical Module– CFP – CFP2/CFP4– CPAK

*5

*6

CFP image courtesy of Finisar

100G Ethernet Tester

• Purpose for 100G Ethernet Tester– Isolate the cause of

failure on a 100GbE port– Sanity check of a CFP– Health Check of IX

Switch– Throughput

Test(RFC2544)

• Specification– Hardware

• Throughput Test with Multiple 100GbE (at least 2) Ports

• No limitation for CFP vendors

• Portability and Battery-powered

– Software• Multiple Stream• CFP Check• Reporting

Tester IX SwitchCFP

CFP

CFP

CFP

courtesy of EXFO

Technology Myth & Truth

• 10x10 MSA is very attractive; cost effective– Really?

• With CFP2 10x10MSA is unsupported… can be dead-stock

• Big Chassis that supports 100GbE can be in the middle– Can’t live with old GbE interface… will drop packets… not

enough buffer…, can’t be in the middle

• Vendor lock-in…– As always

• Timing– If we wait for another year, we could have install double or

quadruple capacity line-card…

Cost Myth & Truth

• 100GbE is getting less expensive– Yes it has. In our case, we see the CFP cost dropped by

50%...• Comparison year 2012 and 2015

• But Still:– For us, it cost 21 times higher cost…. 21X

• 21x SFP+ = 1x 100G CFP PRICE…

• Is it really less expensive?

Why not 100G now?Shall we

deploy 100GNow?

New modules coming

There may be less expensive better product

in market

Expensive interface you buy today can

be legacy

Let’s just wait

Vicious Circle

Summary

• 100G Ethernet is still “on going” technology.– Indecisiveness of making investment decision

• Technology(Optics/Wave Multiplexity)• Cost…

• Cutting Edge Players have already started adopting to 100G because of the necessity– Hypergiant, IX, R&E Community– They needed it, we needed it

• It works fine• When will do? ＞ Timing is everything ＜ ~ mark your calendar ~

Reference*1

http://newsroom.brocade.com/press-releases/ams-ix-and-limelight-networks-field-test-brocade-1-nasdaq-llnw-0765435#.U-rzNGMTNW_

*2http://press.de-cix.net/press-releases/pressemitteilung/article/de-cix-completes-100g-optical-layer-build-for-new-flagship-apollon-platform/

*3 https://www.linx.net/publicity/2012releases/pr2012-04.html

*4 http://www.ntt-at.com/product/optical-switch/

*5 http://www.cisco.com/web/JP/product/hs/optical/ons15454/prodlit/white_paper_c11-727398_031813.html

*6 http://www.epsglobal.com/downloads/Finisar/Finisar40G_100G_Presentation.pdf