Olivier Dolbeau, Benjamin De Bernardi "25+ millions members in 22 countries, how to scale with Symfony 2?"

PHP Frameworks Day 2016 - 25+ million members in 22 countries; How to scale with Symfony2 @odolbeau & @genes0r

25+ million members in 22 countries; How to scale with Symfony2?Benjamin De Bernardi - Olivier Dolbeau


@genes0r

Lead Engineer

Benjamin De Bernardi


Olivier Dolbeau

@odolbeau

Web Architect


100€ 25€ 25€ 25€

100€ 25€


15M

20M

10M

5M

1M

2015201420132012201120102009

25M

mem

bers worldw

ide

Over 25 million members and growing

1.5 million+joining each month



1. The past

2. Why symfony?

3. How?

○ How to deal with 22 countries?

○ Why do we love the EventDispatcher?

○ Performance & asynchronous

○ Doctrine

4. Traps

5. Here & now

Talk overview:


The past

1


v1.0

2005-2006



v2.0

2008-2011(2014)



2010

New countries

2011 2012



3 770 lines of code

69 functions

926 lines for the longest one.

lib.trip.php


Homemade framework


NO TESTS


v3.0

2011 - today



2012 2013 2014 2015

New countries


Why Symfony?

2


Huge community.



Attract new people


How?3


22 countries, how can we manage specificities?


➔ One codebase for each country?

➔ Conditions everywhere in the code?

➔ Configuration and Context?

22 countries: how can we manage specificities?


➔ One codebase for each country?

➔ Conditions everywhere in the code?

➔ Configuration and Context?

22 countries: how can we manage specificities?


Web request Listener

Symfony parse & build a Request object

Build Context from configuration and Request

Response

22 countries: Configuration & Context


1 file for each country

100+ configuration keys

Inheritance

Toggling of features

Metadata


Configuration:





Get config from a controller:



Get config from a controller:



Get config from the view:




Lot of code, how to keep events clean?


➔ Symfony built-in EventDispatcher



EventDispatcher how it works:


Event DispatcherEventListener

Listener


User A rates User B:


Event DispatcherNew Rating





Grade Listener

Recalculate the grade of both users





Grade Listener

Moderation Listener

Ask for content verification



Sending the event:

● Sent from anywhere



Registering a listener:

● Update a configuration file



Listening the event:

● Inside a registered listener


+ Perfect to decouple code inside the app.+ Small listeners, easy to understand, to test.+ Easy to dispatch events outside application.

- Sometimes, hard to understand the flow.- Need tools/rules to keep a common logic.

Dispatching events Pros/Cons


■ Avoid Entities

■ Keep listener small and dedicated to a single task

■ Limit dependencies

Dispatching events one last thing:


High traffic, how to improve performance?


Problem:The website is a bit slow


Solution:All side tasks can be done asynchronously


Solution:All side tasks can be done asynchronously

Tasks:

● Find a tool to transmit messages● Create workers ● Use the event dispatcher


Asynchronous jobs everywhere.

● Performance● Job splitting



Dispatch Event with the EventDispatcher.


● Listened with RabbitMQ or internal Listeners


Check eventName & event interface

Dispatch the event in the application

Decorated EventDispatcher.

Publish to RabbitMQ


Send an Email


● Built using the User context in a dedicated worker


+ Perfect to decouple code+ Small workers, easy to understand+ Easy to dispatch events outside application+ Separate projects & deployment+ Scalable!

- Learning curve- Need tools/rules to keep a common logic- Hard to test a complete flow with mock/stubs- Mapping is time consuming

Asynchronous event & process Pros/Cons


129 workers

Mail

SMS

Publication

Push

IndexationImage

resizing

Alerts

Profile completion

I love workers!

Payment


Asynchronous workers:

Infrastructure:

■ PHP

■ Supervisord

Code:

■ Symfony command inside the main app

■ Standalone app + Symfony components


Asynchronous workers, Symfony:

+ Access to Entities+ Every already defined services

- Entire Symfony- Not dedicated- Deployed along the website- Not the main goal of Symfony- High memory usage


Asynchronous workers, Standalone:

+ Dedicated to a single task/type+ Maintainable small code+ Deployed independently+ Easily created & tested

- Component splitting from the codebase- More monitoring


About Doctrine...


It’s not a troll.


Code fast!


Code well?



Queries from the view!

Hard to control cache strategy

No integrity check.


Doctrine Pros/Cons

+ Quick to bootstrap a new project+ Quick to code a small project

- Hard to respect MVC- Hard to deal with cache- Hard to deal with integrity problems in DB- Not made to decouple code (relations).- Not made to scale.


We fell in traps...

4


Never pimp your kernel!


Before

EnvironmentDebug?


Before

After

Branding Media

EnvironmentDebug?


Put everything inthe same place can be

dangerous!



This is our application.

This is our business code.


Don’t mess with your tools!




We fell in traps...


We had parachutes!


Problems solved?

+ Use symfony only as a tool+ Work a lot on onboarding

+ Try to decouple everything with components

- We still have one big application- As everyone, we lack of documentation


5


40 269Commits

There are “only” 18 952 commits for symfony/symfony


561 851NCLOC

There are “only” 278 909 NCLOC for symfony/symfony


~ 20 minutes to run all the unit/functional tests


~ 30minutes to deploy the application


Too big!


We’re working on it! ;)


We’re hiring!In Paris & Warsaw!http://blbl.cr/dreamjobs

Technology

Olivier Dolbeau, Benjamin De Bernardi "25+ millions members in 22 countries, how to scale with Symfony 2?"