Skyscanner Journey - from code jungle to state of the art

Skyscanner is a leading global travel search site offering unbiased, comprehensive and free flight, hotel and car hire comparison.

Unique monthly visitors Over 50 million

App downloads Over 40 million

Languages available Over 30

Global employees Over 700

Number of nationalities Over 50

Global offices 9

A bit of history

hotels meta search website built on top of a Natural Language Processing engine

supported Spanish and English developed a graph-based database to represent the world as

an ontology DDB (Deductive Database) & DQL based in Barcelona (obviously )

A bit of history

Today* in the news

* by today I mean mid-2013

First task

Fogg architecture – simplified edition

Business Logic Service (BLS) aka backend

Frontend(Zend Framework)

JSO

N A

PI

Fogg frontend

5 years of “startup code” team was focused on the backend & DQL MVC(-ish) files between 4000 – 6000 lines (not be opened with Eclipse) no tests / documentation no coding / naming standard

…some blood, sweat & tears later Skyscanner Hotels launches in January 2014 (hooray)

Going mobile

Not really doable on the current code base.

Still going mobile

Develop a separate mobile website tight time constrains not all functionality from the desktop site was implemented no real tests some documentation no separation of concerns

Welcome to code jungle

Desktop applicationZF1 – Apache 8090

Mobile / tablet applicationYii – nginx 8092

Deviceatlas

Since maintaining 2 code bases for the same product was not bad enough, other “features” were included no real tests or useful documentation monolithic architecture – “one class to rule them all” – always expect the unexpected some pages – redirect to the partner site – were shared complicated release process – only for the bravest and brightest

Squadification

Squads & tribes

Hotels Tribe Flights TribeCar hire Tribe

HFE(frontend)

Infrastructure Tribe

HBE(backend)

HD(data)

PE(partners)

…

…

…

…

…

…

…

…

…

…

…

…

Technology stacks

multi-discipline team operate like a startup owns a (micro)service can pick its technology stack based on needs own its internal processes (scrum, kanban) must respect “contracts” with other squads

Does it work? How?

Edge Side Includes

Edge Side IncludesEdge Side Includes or ESI is a small markup language for edge level dynamic web content assembly. The purpose of ESI is to tackle the problem of web infrastructure scaling. It is an application of edge computing.

<section title=“Textbook definition” warning=“boring”>

</section>

Everything is a component

/hotels/search-box

/hotels/results-list

Anatomy of ESI component

/hotels/search-box/

requirements(configuration json)

content(serve html)

style(serve <style> tags)

script(serve <script> tags)

Endpoints

Component

self contained entity that renders a piece of HTML

the endpoint is added to the component name to form the URL: /hotels/search-box/content

all ESI templates are preprocessed by a component called Scaffolding

the requirements endpoint is requested first

components cannot access the URL / cookies / client headers directly (they need to be explicitly requested via the configuration json)

ESI architecture

Varnish retrieve content / style / script endpoints cache according to HTTP headers

ESI components(various technologies)

Scaffolding (.NET) loads the ESI template for the URL retrieves the configuration json injects the requested parameters returns the template to Varnish

http request

Scaffolding response

Pretty cool, right?

You are here!

You are here

Choosing the framework

Requirements: PHP micro-framework no database layer required strong community actively supported fast

Testing methods: apache benchmark on “hello world” number of installs on packagist time since the last commit community vibe And the winner is…

Mustache / Handlebars

widely used 30+ programming languages templates can be easily shared with other squads started with mustache but was *too* logicless

Routing

routing app

search-box

results-list

hotel-details

ESI request

Components

Clear separation of concerns

Routing app detect which component is requested bootstrap common services (SilexServiceProvider)

monolog handlebars i18n & l10n caching static routing BLS connector

no controllers / logic error management

Component bootstrap component specific services render the component endpoints (content /

style / script) controllers internal webroot static files (css & js) proxied via Silex in dev,

packaged and served via nginx otherwise

BLS Connector

BLS connector library models encapsulating the BLS json API shared by multiple applications (internal & external

use) composer vcs repository unit tested separately treated as an external library uses JsonPath to extract data

Business Logic Service(BLS)

BLS connector(PHP library)

App1 App2 App3

BLS consuming applications

Json path

https://github.com/Skyscanner/JsonPath-PHPhttp://goessner.net/articles/JsonPath/

Javascript

Javascript common javascript libraries (shared by

multiple components) component specific served minified in non-development

environments can be minified in dev for debugging

purposes all code is namespaced and encapsulated Backbonejs and jQuery no inline javascript in content nodes

CSS / SaSS

shared repository of common styles component specific styles managed via assetic

Testing & code coverage

aiming for 100% unit test coverage hard limit 95% strictly enforced

Coding standard

custom coding standard PSR-1 & PSR-2 compatible based on Symfony’s actively enforced with php_codesniffer repository with custom rules jshint for javascript coding standard

Git hooks & pipeline

Local git feature branches pre-commit – check coding standard pre-push – run phpunit

Merge request Code review

Wiggum (nodejs application)

Teamcity run all previous tests user journeys layout tests (Sauce labs)

Gitlabdevelop branch

Release procedure

Releasing is the only time we bring value to our users!

automated with ansible only 1 manual step (wip) takes 1h to deploy to 4 DCs around the globe all static resources are packaged and optimized version numbers are added to break the cache we *did* release on Friday (morning, but still) 2-3 times per week

Logging / monitoring

Going forward

Thank you

Tudor Barbu

@motanelu tudor.barbu@skyscanner.netbeer after the conference (preferred way)