Embed Size (px)
Citation preview
Is it hard to build a service for 100M user?
Short answer – yes
/dev/viber
What would you do if you had 1,000,000 users after 3 days?
1. Run away 2. Get drunk3. Pray4. All of the above
/dev/viber
Some numbers…
• 100M registered users• 2.5B text, 1.7B voice minutes monthly• 50 employees in Belarus, about 120
worldwide
/dev/viber
What do we do?
• Free calls, messages and group messages (more coming!)
• Your number is your “user id”• Always On…
/dev/viber
Why Viber became popular
• People love to get something for free• First one to do it “right”• It’s purple :)
/dev/viber
Reception
• 1M after 3 days• Very high growth ratio• Give us more (platforms, features)– “Where is the Blackberry version?”
/dev/viber
What we saw as a result?
PROBLEMS!!!
/dev/viber
Building a service that can handle 100M users in 1.5 years is difficult.
Really.
/dev/viber
Why problems?
• More usage - more load on backend• More platforms:– More development – Can’t deliver all features on all platforms at the
same time.
/dev/viber
Scaling
“Scaling is replacing all components of a car while driving it at 100MPH” [Mike Krieger, Instagram]
/dev/viber
When developers say “fully scalable” they don’t really mean it…
/dev/viber
Backend
• Got to 1M connected users in less than 2 weeks
• Used Amazon EC2, but still you have to design for scalability
• Started from 30 servers (and we thought it was a LOT)
• So we started to rewrite
/dev/viber
Why is it so hard?
• Address books (100M)• Reverse address entries (3B)• 300,000 data requests per second• Active dataset 1.5TB• 15-20M concurrent connected users (TCP)• Over 100,000 concurrent phone calls
/dev/viber
How to handle all the data
• NoSQL (MongoDB, Redis)!• Sharding• MongoDB uses auto-sharding but it’s slow• Simple in-memory solutions require lot of
work
/dev/viber
We are our biggest enemy
• A small mistake on a mobile client can lead to a massive DDOS
• One bug in Android client = 100 times load on backend
/dev/viber
Say goodbye to life
• You are always in the office• Sometime the office is at home• Sometime at the beach• Sometime...
/dev/viber
Shit happens
• Stuff will break• Stay cool and fix• Communicate with your users
/dev/viber
Monitoring
• Collect tons of stats• Look at them• Look at them again• Trending matters. Things don’t just happen.
/dev/viber
Backend now
• Constantly rewriting and optimizing• Runs on 300 servers (and we think it's not too
much)• Using and improving open source software
/dev/viber
Backend - technologies
• Proprietary front-end servers (C++, really fast)• Redis – in memory NoSQL, fast• MongoDB – persistent NoSQL, not so fast• MySQL – registrations, pretty slow• Always looking for alternatives
(CouchBase, DynamoDB, etc)
/dev/viber
Platforms
• Users wants to communicate with their friends, but they keep buying different phones
• Started from iOS and had Android community bashing us for a year
• Now: iOS, Android, Blackberry, WindowsPhone 7
• More coming…
/dev/viber
Client challenges - iOS
• Support numerous iOS versions (we’re finally dropping support for iOS 3)
• Support several device type (3G, 3GS, 4, 4S)• Need to react very fast (iOS upgrade)!• Hard to test on large scale (no beta testing,
although there is a one-time solution)
/dev/viber
iOS Device Breakdown
36.60%
34.77%
11.57%
3.60%
7.16%
6.00%
iPhone 4SiPhone 4iPhone 3GSiPhone 3GiPodiPad
6 users registered with iPhone 5
/dev/viber
iOS versions breakdown
1.10%13.90%
84.80%
iOS3iOS4iOS5
/dev/viber
Mobile upgrade speedDesktop application upgrade Mobile application upgrade
/dev/viber
Client challenges - Android
• Devices/platform fragmentation (we saw more then ??? devices)
• Lot of sound-related bugs caused by it• Different screens and device caps• APIs not always work as documented and not
always documented
/dev/viber
Client challenges - voice & video infrastructure
• Have to choose/write a proper engine• Now we are using and modifying WebRTC• Has to work smoothly on mobile devices• 3G is a pain• Has to optimize for several platforms
/dev/viber
Lessons learned
• Always plan for success• Monitor your system• Hire best people• It’s VERY REWARDING!
/dev/viber
Questions?
