Embed Size (px)
Citation preview
Wednesday, March 6, 13
Applying Evolutionary Architecture on a
Popular API
phil calçado@pcalcado
http://philcalcado.com
QCon London Session Code: 1603Wednesday, March 6, 13
Wednesday, March 6, 13
Wednesday, March 6, 13
> 50K registered client apps
~ 180 million users / month
~5 years old
Wednesday, March 6, 13
Wednesday, March 6, 13
API controllers UI controllers
domain logic
Wednesday, March 6, 13
http://bit.ly/building-the-next-soundcloud
Wednesday, March 6, 13
API controllers UI controllers
domain logic
Wednesday, March 6, 13
API controllers
UI controllers
domain logic
internet
Wednesday, March 6, 13
a new platform while keeping the old one
running?
Wednesday, March 6, 13
WARNING:THIS IS
WORK IN PROGRESS
http://bit.ly/work-at-soundcloud and you can help us ;)
Wednesday, March 6, 13
DO NOT BREAK CLIENT APPS
Wednesday, March 6, 13
separate application
from domain
Wednesday, March 6, 13
A set is a collection of
your own sounds
ClassicSoundCloud
NextSoundCloud
A set is a collection of
your own sounds
Wednesday, March 6, 13
A set is a collection of
your own sounds
ClassicSoundCloud
NextSoundCloud
A set is a collection of
your own sounds
A set is a collection of
sounds
SoundCloudAPI
Wednesday, March 6, 13
API controllers
UI controllers
domain logic
internet
Wednesday, March 6, 13
API controllers
UI controllers
domain logic
internet
domain logic domain logic
Wednesday, March 6, 13
applications enforce their own business rules
domain enforces “global” rules
Wednesday, March 6, 13
applications enforce their own business rules
domain enforces “global” rules
(accidents happen...)
Wednesday, March 6, 13
how to add/change features on the fly?
Wednesday, March 6, 13
attempt #1: identify “official” apps
- too easy to spoof- abused by spammers- Android and iOS can’t easily rotate secrets
Wednesday, March 6, 13
attempt #2: namespace endpoints
- “experimental” endpoints would link to regular ones but never be linked back- toyed with “lifted” hyperlinks, not enough time for launch
Wednesday, March 6, 13
attempt #3: just use feature flags
- works ok for logged in users
- system is a little monster now, and ruby is not very good with numbers. rewriting in scala
Wednesday, March 6, 13
SPLITTING THE MOTHERSHIP
Wednesday, March 6, 13
define internal protocols
Wednesday, March 6, 13
micro-services- couple thousands lines of code
- mostly JVM-based, JRuby, Clojure and Scala
- more about this at the SoundCoud pub night tomorrow: http://lanyrd.com/2013/soundcloudpub/ (Walrus Pub Waterloo)
Wednesday, March 6, 13
HTTP+JSON & AMQP for most stuff
- moving AMQP usage for event broadcasting, as opposed to messaging
- no internal HTTP caching (yet?)
Wednesday, March 6, 13
make it easy to get stuff in prod
- heroku-like deployment system
- tending towards twitter stack (Finagle) for I/O
Wednesday, March 6, 13
URNs, not IDs
soundcloud:sounds:123
Wednesday, March 6, 13
URNs, not IDs- everything addressable has a URN, some have URLs
- Mapping depends on context, e.g. given object may have a public URL and an internal URL
Wednesday, March 6, 13
<3 hypermedia
- can easily remove things from the main app without breaking clients, just some HTTP 303s
Wednesday, March 6, 13
OPTIMISE ALL THE THINGS!!11!
Wednesday, March 6, 13
twitter moving back to server rendering...?
Wednesday, March 6, 13
not too bad, actually
- besides load balancers vs. increase in number of requests per page
http://backstage.soundcloud.com/2012/08/evolution-of-soundclouds-architecture/
Wednesday, March 6, 13
the actual problems
- number of requests from Next keeps increasing, page load affected- very different behaviour depending on the client app
Wednesday, March 6, 13
API controllers
UI controllers
domain logic
internet
domain logic domain logic
Wednesday, March 6, 13
API controllers
internet
UI controllers
Wednesday, March 6, 13
attempt #1: optmising proxy for clients
UI controllers
domain logic
internet
domain logic domain logic
API controllers
Proxy
Wednesday, March 6, 13
attempt #1: optmising proxy for clients
- still leverage public API- API has to know about how things behave in different clients
Wednesday, March 6, 13
attempt #2: specialised back-end for official
clients
UI controllers
domain logic
internet
domain logic domain logic
Specialised Back-end API controllers
Wednesday, March 6, 13
attempt #2: specialised back-end for official
clients
- each official app has its own back-end facade- no rendering, just data- public API becomes just one of them- front-end heavy teams can just script a common core using JRuby
Wednesday, March 6, 13
NEXT STEPS
Wednesday, March 6, 13
what we are looking at
- dataflow programming for the specialised back-ends
- ProtocolBuffers for Android
- make the mothership just a back-end service, not exposed to the Internet
Wednesday, March 6, 13
phil calçado
http://philcalcado.com@pcalcado
www.soundcloud.com
Wednesday, March 6, 13
