1. Concurrency & GeventScaling Real World Applications

2. Concurrencyhandling a number of things at the same timeExamples: (Incoming) WebServers, Database Servers (Outgoing) SSH Mux 3. SSH MuxExecute a command on a remoteSSH serverHandle concurrent SSH ClientsCommand execution time variesfrom seconds to daysCommand execution happens onremote servers, SSH mux is I/Obound 4. SSH Client1. Init session2. Authenticate3. Get a channel4. Issue command5. Read output 5. Need Concurrency?Process blocks on read()No new connections can be inititatedNeed ability to handle multiple clients at the sametime 6. MultiprocessingOne process is the masterMaster can spawn workersEach worker handles one request at a timePre-forked pool of workers 7. Concurrent SSH Clients 8. SSH Mux Memory Usagessh mux memory usage 600 500 400 Memory (MB) 300 Processes 200 1000 0 5 10 15 20 25 30 35 40 45 50 55 60 65 70 75 80 85 90 95 100# Concurrent Reqs 9. SSH Mux Performace ssh mux performance sheet160140120100Time(s)80 Processes604020 020 40 80 150 200 400 800 1200 1500 1800 2100 # Concurrent Reqs 10. Multiprocessing yayEasy to get startedOS guaranteed process isolation & fairnessCovers up for misbehaving workersAdd more concurrency by adding more workersConvenient when numbers are smaller numbers 11. Multiprocessing nayConcurrency limited by number of processesMemory heavyImplicit schedulingSynchronization is not trivial 12. More Concurrency?Command execution is happening on remote servers, weare mostly blocked on I/OHandle multiple I/O in a single process? 13. Geventgevent is a coroutine-based Python networking library that uses greenlet to provide a high-level synchronous API on top of the libevent event loop. 14. GreenletsLightweight threads - not OS threadsExplicit scheduling - CooperativeMinimal stackApplication decides execution flowEasy to synchronize/ Avoid locksAll run inside one process 15. LibeventUse fastest mechanism to poll (portable)Fast Event loopIn Gevent, event loop runs in a greenlet (event hub)Instead of blocking, greenlets switch to event hubIts all done behind the scene 16. Monkey PatchingMonkey patchingModifies behaviour of blocking calls such as select, sleep tonon-blockingPatches the python standard socket library 17. Gevent Greenlet 1 is running Greenlet 2 and 3 are ready 18. GeventGreenlet 1 has to wait for readGreenlet 1 switches to Event hub 19. GeventEvent hub switches to Greenlet 3 20. Gevent Greenlet 2 runs 21. GeventGreenlet 2 wants to sleepGreenlet 2 switches to Event hub 22. GeventGreenlet 1 data has come, moved to ready stateEventhub switches to Greenlet 3 23. Gevent Greenlet 3 runs 24. Gevent When Greenlet 1 resumes, its from next instruction Its as if it were a blocking call 25. Green SSH Client1. Init session2. Authenticate3. Get a channel4. Issue command5. Read output 26. A closer look 27. Going Concurrent Use pre-forked processes to use all cores 28. Memory usage ssh mux memory usage 45 40 35 30Memory(MB) 25Gevent+Processes 20 15 10 5 00 5 10 15 20 25 30 35 40 45 50 55 60 65 70 75 80 85 90 95 100# Concurrent Reqs 29. SSH Mux Performace ssh mux performace chart 70 60 50 40 Time(s) Gevent+Processes 30 20 10 020 40 80 150 200 400 800 1200 1500 1800 2100 # Concurrent Reqs 30. SSH Mux memory usagessh mux memory usage600500400 Memory(MB)Processes300Gevent200100 00 5 10 15 20 25 30 35 40 45 50 55 60 65 70 75 80 85 90 95 100 # Concurrent Reqs 31. SSH Mux Performancessh mux performance sheet160140120100 ProcessesTime(s)80 Gevent+Processes604020 020 40 80 150 200 400 800 1200 1500 1800 2100 # Concurrent Reqs 32. Gevent yay!Untwist write linear non blocking codeExplicit scheduling, dictate the execution flowTimeoutsEvents, AsyncResults for Synchronizationgevent.wsgiPre-spawned pool of greenlets 33. Gevent beware ofNo multicore supportNot great for CPU bound applicationsThird party libs must be green (non blocking)Misbehaving workers can be lethalNo fairness when it comes to scheduling 34. Take AwayGevent lets you write asynchronous code in asynchronous mannerNo multicore support, still need multiprocessingNot so great for CPU bound applicationsSplit your application into CPU bound and IO boundpartsBe willing to contribute patchesCode available atgit@github.com:aaloksood/pyexamples.git 35. Thank youThats all folks! 36. Countdown TimerCount down from 200000000Split work among workers 37. Threads Multithreading wonder2520151 CoreTime(s)4 cores1050 1 2 3 4 5 6 7 8 9 10 # Workers 38. One coreExecution time One Core 14.514 13.513 Processes 1 CoreTime (s) 12.5 Gevent_112 Gevent_4 11.511 10.51 2 3 4 5 6 7 8 9 10# Workers 39. Four coresExecution time 4 cores 25 20Process 15Threads Time(s)Gevent_1 10Gevent_4 5 01 2 3 4 56 7 8 9 10 # Workers