Project Phoenix - From PHP to the Play Framework in 3 months

@elmanu#playframework

From PHP to the Play Framework in 3 months

Manuel Bernhardt @elmanu http://manuel.bernhardt.io

http://manuel.bernhardt.io


Agenda

• Background

• Building the new platform

• Data migration

• Operations

• Summary & Future


Your speaker

•Freelance software consultant based in Vienna !

•Web, web, web

•Scala, Akka, Play Framework !

•Vienna Scala User Group !

•Open-Source


Agenda

• Background


• Data migration

• Operations



Talenthouse

•www.talenthouse.com !

•based in Los Angeles !

•connecting brands and artists !

•3+ million users


Background

•Problems with the old site

• Slow

• Developed by an external company

• Hard to evolve


Background

•Solution: new site!

• Scala

• Play Framework

• AngularJS

• elasticsearch

• …


Background

•A few interesting aspects

• Source code may not be available

• Data spread out in many places

• Only 3 months time


Background

•A few interesting aspects

• Source code may not be available

• Data spread out in many places

• Only 3 months time


Agenda

• Background


• Data migration

• Operations

• Methodologies & Tools



Building the new platform





• MVC framework !

• Compile as much as possible

• “Reactive”


http://www.reactive-manifesto.org

http://www.reactivemanifesto.org


Load balancer


Load balancer



Data access?



Anorm

Data access?

Squeryl



Anorm

Data access?

Squeryl



Anorm

Data access?

Squeryl

SORM Tiramisu



• Started with Anorm

• Too verbose !

• Switched to Slick

• Plain SQL with neat string interpolation

• DSL for composing queries



def fetchItemWithRelevanceSort(userId: Long, skip: Int, limit: Int)! (implicit s: Session): List[Item] = {! val q = for {! i <- items if !i.deleted! ! && i.visible_on_profile! ! && i.user_id === userId! } yield i! ! q.sortBy(item => (item.selected.desc, item.created_on.desc))! .drop(skip)! .take(limit)! .list!! }

val query = sql"""! with invites_for_market as (! select #$inviteSL, #$inviteMarketSL! from invite i! inner join invite_market im! on i.id = im.invite_id! and im.market_id = ${marketId.id}! where i.art_type = ${artType.toString}! and i.visibility_state = ${InviteVisibilityState.Public}! and i.deleted = false! order by i.submission_start desc, i.id desc! limit $limit! offset $skip! )!! select ifm.*, #$localInviteSL! from invites_for_market ifm! inner join local_invite li! on ifm.id = li.invite_id! and li.completed = true! order by ifm.submission_start desc, ifm.id asc! """! val queryResults = query.as[(Invite, InviteMarket, LocalInvite)].list!



Front-end & UI



Front-end & UI



Front-end & UI



• Recently open-sourced by Facebook !

• Virtual DOM, fast !

• Very easy to integrate in a page (less invasive than Angular)

ReactJS



Front-end & UI



• Build pipeline using gulp.js & bower !

• Assets pipeline

• LESS compilation

• JS minification

• Upload assets to Cloudfront

• Integrates translations from Crowdin

• Washes the dishes


Agenda

• Background


• Data migration

• Operations



Data migration: source system(s)
















Data migration: migration schedule


Data migration

• Akka to the rescue! !

• Concurrent user migration !

• Concurrent item migration

• Using Play’s WS library for async calls to Youtube and friends


Data migration

User migrator

Worker Worker Worker Worker Worker


Data migration

Item migrator


Data migration

Item migrator

User item migrator

User item migrator

User item migrator


Data migration

Item migrator

Item migration

worker

User item migrator

User item migrator

User item migrator

Item migration

worker

Item migration

worker

Item migration

worker

Item migration

worker

Item migration

worker


Data migration

Item migrator

Item migration

worker

User item migrator

User item migrator

User item migrator

Item migration

worker

Item migration

worker

Item migration

worker

Item migration

worker

Item migration

worker

File fetcher

File fetcher


Data migration

Item migrator

Item migration

worker

User item migrator

User item migrator

User item migrator

Item migration

worker

Item migration

worker

Item migration

worker

Item migration

worker

Item migration

worker

File fetcher

File fetcher

File uploader

File uploader


Data migration

Item migrator

Item migration

worker

User item migrator

User item migrator

User item migrator

Item migration

worker

Item migration

worker

Item migration

worker

Item migration

worker

Item migration

worker

File fetcher

File fetcher

File uploader

Soundcloud worker

File uploader


Data migration


Agenda

• Background


• Data migration

• Operations



Operations

Operations


Operations

@YourTwitterHandle#DVXFR14{session hashtag} @elmanu#playframework

Summar

y

& Futu

re


Summary & Future

• Play Framework does not get in the way

• Easy to deploy and operate

• Many, many integrations !

• Many plans for the future

• Live activity feed

• Similarity search !

• Talenthouse is hiring!

@YourTwitterHandle#DVXFR14{session hashtag} @elmanu#playframework

Q & A

Software

Project Phoenix - From PHP to the Play Framework in 3 months