Embed Size (px)
DESCRIPTION
How to scale a rails application on several nodes, including master slave replication and a key value store...
Citation preview
Application
Data Structure
100.000 Users
1 Million Interests
5 Million Status Messages
13 Million Friendships
19.100.000 Data Tuples
Actions
ProfileReads Statuses, Friends and Interests
BefriendWrites 2 Friends
DefriendWrites 2 Friends
StatusWrites 1* Status
LoginWrites 1 User
Simulation
30 Minutes
20 Minutes Ramp Up
3 Thread Groups
20, 10 and 5 concurrent Requests
Hardware
5 Boxes
Dual and Quad Core 2.5 GHZ
2 to 4 GB RAM
100 Mega Bit Network
Setups
4 Setups
Passenger. Rails 3. Capistrano.
Default Rails Installation1 Box. Nginx. MySQL.
Default Rails Installation1 Box. Nginx. MySQL.
AppDatabase
Distributed Rails Installation3 Boxes. 2 Nginx Boxes. 1 MySQL Box.
Distributed Rails Installation3 Boxes. 2 Nginx Boxes. 1 MySQL Box.
LBApp
App
Database
Distributed Rails Installation3 Boxes. 2 Nginx Boxes. 2 Replicated MySQL Boxes.
Distributed Rails Installation3 Boxes. 2 Nginx Boxes. 1 Redis Box.
Distributed Rails Installation3 Boxes. 2 Nginx Boxes. 1 Redis Box.
LBApp
App
Redis
Distributed Rails Installation5 Boxes. 4 Nginx Boxes. 1 Redis Box.
Distributed Rails Installation5 Boxes. 4 Nginx Boxes. 1 Redis Box.
LBApp
AppRedis
App
App
Performance
Default Rails Installation1 Box. Nginx. MySQL.
Default Rails Installation1 Box. Nginx. MySQL.
Default Rails Installation1 Box. Nginx. MySQL.
Default Rails Installation1 Box. Nginx. MySQL.
Default Rails Installation1 Box. Nginx. MySQL.
Default Rails Installation1 Box. Nginx. MySQL.
Default Rails Installation1 Box. Nginx. MySQL.
48.755 Requests / 30 Minutes
27 Requests / Second
Distributed Rails Installation3 Boxes. 2 Nginx Boxes. 1 MySQL Box.
Distributed Rails Installation3 Boxes. 2 Nginx Boxes. 1 MySQL Box.
Distributed Rails Installation3 Boxes. 2 Nginx Boxes. 1 MySQL Box.
Distributed Rails Installation3 Boxes. 2 Nginx Boxes. 1 MySQL Box.
Distributed Rails Installation3 Boxes. 2 Nginx Boxes. 1 MySQL Box.
Distributed Rails Installation3 Boxes. 2 Nginx Boxes. 1 MySQL Box.
Distributed Rails Installation3 Boxes. 2 Nginx Boxes. 1 MySQL Box.
62.138 Requests / 30 Minutes
35 Requests / Second
Distributed Rails Installation3 Boxes. 2 Nginx Boxes. 1 Redis Box.
Distributed Rails Installation3 Boxes. 2 Nginx Boxes. 1 Redis Box.
Distributed Rails Installation3 Boxes. 2 Nginx Boxes. 1 Redis Box.
Distributed Rails Installation3 Boxes. 2 Nginx Boxes. 1 Redis Box.
Distributed Rails Installation3 Boxes. 2 Nginx Boxes. 1 Redis Box.
Distributed Rails Installation3 Boxes. 2 Nginx Boxes. 1 Redis Box.
95.861 Requests / 30 Minutes
53 Requests / Second
Distributed Rails Installation5 Boxes. 4 Nginx Boxes. 1 Redis Box.
Distributed Rails Installation5 Boxes. 4 Nginx Boxes. 1 Redis Box.
Distributed Rails Installation5 Boxes. 4 Nginx Boxes. 1 Redis Box.
Distributed Rails Installation5 Boxes. 4 Nginx Boxes. 1 Redis Box.
Distributed Rails Installation5 Boxes. 4 Nginx Boxes. 1 Redis Box.
Distributed Rails Installation5 Boxes. 4 Nginx Boxes. 1 Redis Box.
Distributed Rails Installation5 Boxes. 4 Nginx Boxes. 1 Redis Box.
397.896 Requests / 30 Minutes
111 Requests / Second
411% faster
Performance Coding
No Joins
user.friend_ids = “234, 2546, 123, 4, ...”
SELECT * FROM user WHERE id IN (user.friend_ids)
Publish / Subscribe Redundancy
friends
user_3_feed
user_4_feed
user_9_feed
user_12_feed
user_43_feed
new status
new status
new status
new status
new status
new status
Read Feed? Lookup Key.
Current State
Fully Hardware Scaleable.
How far?
Done.
https://github.com/volkert/railscale
Volker TietzThomas Fankhauser
