Interconnection: Costs, Complexity, Policies

Gaurab Raj Upadhayagaurab@llnw.com

Why Peering in Asia Sucks !

Disclaimer

• Everything in these slides is my own opinion.

Interconnection Markets

• Internet interconnections are concentrated in few hubs around the world– US West Coast (Silicon Valley, Los Angeles)– US East coast (New York, Ashburn)– London, Amsterdam, Frankfurt, Stockholm – Singapore, Hong Kong, Tokyo

Interconnection Markets

• It takes time and lots of efforts to create a viable interconnect market. – Singapore is a very recent entrant into this ‘hub’

status. • It doesn’t always work at scale for everyone– Local relevance may sometimes be more value • Moscow, Miami, Prague, Budapest

• Open and easy access are the key drivers– Why are Rio / Mumbai / Dubai not real hubs yet?

Let’s Dig Deeper into the Asian Hubs

• Singapore– A hub for a lot of other things – transport, banking, logistics due to ease

of doing business– Equinix SG1 IX (287 networks according to peeringdb)

• Hong Kong– Similar to Singapore, traditional hub for logistics – Voice Interconnect hub, so carrier density – HKIX (297 networks according to peeringdb)

• Tokyo– The traditional jumping point to the ‘Internet’ in the US– Japan is one of the largest Internet markets (users, content, everything)– JPIX (184 ), JPNAP (150)

Greg’s Cable Map : www.cablemap.info

But

Layer 1 infrastructure doesn’t translate into Layer 3 Interconnect

Because of

Cost Complexity

Policies

Cost of Backhaul – NA/Europe

• Intra Europe 10G terrestrial wave Costs– 800-1200 USD for 10G between major points – 1000-1500 USD for extended locations– Bucharest to London is ~2500KMs, costs below

2500 USD, or roughly 0.0001/mbps-km. • Intra-US 10G wave costs are similar (actually a

bit higher)– US distances are longer – Large expanse of empty lands

Cost of Backhaul in APAC

• Intra Asia is mostly Sub sea, costs are traditionally higher, but going down drastically– Intra Asian Hub prices range from 12-20K/Month,

depending on locations– Cost of local loop are a key driver, but with POP to

POP design changes, the local loop is not a big consideration for many providers. • Where it does, the cost of CLS to POP backhaul is a big

problem

Cost of terrestrial Backhaul

– North Asia • Japan: Tokyo – Osaka : 10G ~-5-8K/month • Korea: Seoul – Busan : 10G ~ 12K/month

– South Asia • India : Delhi-Mumbai: 10G ~50-60K a month• Bangladesh : Dhaka-Cox’ 10G – 25K/month • Nepal : Kathmandu-Pokhara 10G: - 10K/month

– South East Asia• Malaysia: Kualalumpur – Johor: 30K/month• Thailand: Bangkok – Phuket: 10G ~25-30K/month*Philippines, Indonesia are mostly subsea. Different costs

Asian Transport Pricing

10 Gbps Wavelength Price Multiple over London-New York

Let’s look deeper

• CLS to POP costs– Tuas/ Changi – SG1 : 5-8K/month– Chikura/Maryuama - Otemachi : 5-8K/month– Hongkong : 4K/month premium for anywhere

outside Mega-I. – Korea: it cost more for local backhaul from the CLS to POP, then

the wet segment from Tokyo

– India : Regulated Access charge ($4/mbps ?)

So,

• It is EXPENSIVE to get to peering locations– Transit may be cheaper – Going to LA is likely cheaper

• Operators keep costs artificially high– In Metro Areas – Offer IP Transit over the same transport, which is cheaper

(vs. buying the transport). • Complexity keeps the costs high– SDH – Weak Fiber Infrastructure (i.e, needing 3 routes to build full

redundancy).

Complexity & costs are related

• In Tokyo metro area, metro 10G between certain Datacenters are available as low as $350. – Same transport from other providers range upto 5K/month

• Aggregation for other services are not optimal for data. – One major Indian mobile carrier aggregates into 3 locations,

and those are not close to data gateways. – Network designed based around regulatory boundaries,

and voice optimization• Just easier to buy transit in 3 locations then try to aggregate and

build peering.

• It adds the costs, makes engineering changes difficult.

It’s not just the cost or complexity

• Most Markets are closed except for HK– HKIX had an early start, and is popular even with the

high local transport cost• (imagine HKIX without transport cost.. without M-I)..

– But then you have Dark Fiber in HK being limited to N-Waves

• Domestic Peering fabric are closed user groups or non existent – Korea, Japan, Vietnam, India have closed environment– In most other it’s likely non-existent

Can it be improved

• Singapore has nearly done it, so yes.. Can be done

• Things may move faster in JP, HK, SG then in other locations.

• But something more disruptive needed– Dark Fiber availability or metro waves availability – Metro area IXPs ? (in SG and HK)

Acknowledgement

• Telegeography• Donald Clark• Raphael Ho