1. 1. Dead Simple Scalability Patterns Vedang Manerikar Platform Architect, Helpshift Fifth Elephant, 16th July 2015
2. 2. Background: Helpshift +
3. 3. Agenda Monitoring 101: Logs! Database Access Patterns Serialization and Deserialization Network Calls Integration Points
4. 4. Agenda Monitoring 101: Logs! Database Access Patterns Serialization and Deserialization Network Calls Integration Points
5. 5. Monitoring 101: Logs!
6. 6. Monitoring 101: Logs! Request URL UUID Resource being queried Time taken (ms) Size of response (bytes) Human Identifiers for Data Store, Type of operation IMP
7. 7. Monitoring 101: Logs! Top 5 slowest DB calls $ sort -k6 -r -n | cut -f3- -d Top 5 popular URLs $ sort -k4,4 -u | sort -k3 | cut -f 3-3 -d ' ' | uniq -c | sort -k1 -n -r Top 5 routes making the maximum number of DB calls. $ sort -k4 | cut -f2-4 -d ' ' | uniq - f1 -c | sort -k1 -n -r
8. 8. Monitoring 101: Logs!
9. 9. Agenda Monitoring 101: Logs! Database Access Patterns Serialization and Deserialization Network Calls Integration Points
10. 10. Unbounded DB Calls Lock up Resources Eat up Memory Discovered when: Data Sludge Unexpected Use-case Fixed by: Batch Sizes Abstractions for Chunked Requests QA < Crazy Users
11. 11. Indiscriminate DB Calls
12. 12. Check Out: Batch requests to the same Data Source Cache previous results
13. 13. Agenda Monitoring 101: Logs! Database Access Patterns Serialization and Deserialization Network Calls Integration Points
14. 14. Serialisation / Deserialisation
15. 15. Agenda Monitoring 101: Logs! Database Access Patterns Serialization and Deserialization Network Calls Integration Points
16. 16. The Network is Wonderful
17. 17. The Network
18. 18. Avoid Network Calls Caching Precomputing Avoid
19. 19. Agenda Monitoring 101: Logs! Database Access Patterns Serialization and Deserialization Network Calls Integration Points
20. 20. Integration Points and Domino Effects App Server DB Network Calls Web Server Cascading Failures QUEUE Analytics Consumer
21. 21. Integration Points and Domino Effects App Server DB Network Calls Web Server Cascading Failures QUEUE Analytics Consumer X X X X
22. 22. Timeouts and Circuit Breakers Closed (everything is operational) Half-Open (Has it recovered?) Open (Resource has failed) Failure Wait for some time, in the meanwhile: - Fail Fast - Gracefully Degrade Attempt Reset Failure Success Resource Timeout
23. 23. Check out:
24. 24. Health Checks Am I Alive? Auto-Scaling
25. 25. Revisiting our Stack App Server DB Network Calls Web Server Cascading Failures QUEUE Analytics Consumer CB, T, GD CB, T BP CB, T You Shall Not Pass CB: Circuit Breaker T: Timeouts BP: Back Pressure GD: Graceful Degradation
26. 26. Take-Aways and Questions Designing Scalability > Testing Scalability Scalability is an important perspective vedang@helpshift.com @vedang