Application Design for the Cloud/AWS

Jonathan Holloway (@jph98)Application Architect @ Brightpearl

A little bit of background

Brightpearl grew out of a company called Lush Longboards in Bristol

Chris Tanner and Andrew Mulvenna identified a need for a single online business management app

First customer on Brightpearl in 2007

Over 1300 customers currently on Brightpearl and growing

The numbers:

Founded 2007Today, we have:

1,300+ customers53 countries

87 employees(in San Francisco and Bristol)

$1.3 billiongmv processed

What Does Brightpearl Do?

- Multi-channel integration (Amazon, Ebay) - Storefront integration - Inventory, orders, and customers - Logistics and shipping - Web and POS interface - App store for third party integrations

Business @ Scale on the Amazon Platform

Over 1,300 customers run on our cloud based platform

Company Culture

Rapidly growingTeam, Product, Technology, Career

Team and PeoplePassionate, Diverse, Agile

The way we workOpen Plan, Open Minded, Autonomous

Great social lifeWe call our colleagues friends and do fun stuff together outside the office.

Career developmentWe invest in our people and support their career aspirations.

Open atmosphereOur managers don’t have offices and openly tell us about the business’ progress.

Multi-cultural workforceWe have employees who come from over 20 different countries.

Tech Culture

General tech lunch and learns (TLAL) for various technology talks (Java, PHP, Javascript)

Ruby project workshop for building apps once every two weeks, encourage use outside development

Friday - half day for personal tech projects, e.g. dashboards, product improvements

Java Meetup Group for external talks with developers from other companies in the Bristol and Bath area. We host PHPSW along with Basekit in Bristol

My Background

Application Architect - Solutions/Technical, Developer

Started with SaaS back in 2010 with email archiving and large scale storage/search solution

- Cassandra, Hadoop, Pig, Lucene and Jersey

Worked in statistical computing with distributed grids (Oracle Grid Engine, LSF). Deployed on large compute clusters internally in pharma, looked at cloud based solution using Starcluster, Python

Java background, Ruby, Python and Javascript

Cloud, Tech Overview

SaaS - Big Product Examples

Lots of big SaaS based products on the web today including:

On-demand movie rental

Customer Relationship Management

Microsoft Office Toolsuite Online

Conference and collaboration

SaaS and PaaS and IaaS

It’s all very confusing… think of it as a pyramid

IaaS - Infrastructure as a service (Tools/services for devops)- Amazon EC2, Windows Azure, Heroku

PaaS - Platform as a service (Tools/services for app devs)- AWS Beanstalk, Heroku, Google App Engine, Redhat

Openshift

SaaS - Software as a service (Apps for end users - yay)- e.g. Netflix, Brightpearl, Salesforce, Office365

How do I know when a product should be SaaS based?

- Have to be careful with data requirements, data at rest, transfer because of the public cloud. VPC and VPN can help.

- I/O and throughput might prohibit movement of files

- Have to be careful don’t just move your application into the cloud… break it down… re-assemble with cloud based application services

- Factor in availability, performance, failover, reliability. Don’t underestimate reliability

SaaS - Archiving Solution

Customer Account Metadata

Postgres/Slony

Search Services

SOLR

SOLR

SOLR

Content Extraction

Tika Tika

Content Storage

Solaris/ZFS

Search Interface

Django

Tika

Mail Archive

Mail Archive

Mail Archive

- Worked on this for a private cloud solution- Why doesn’t it fit the public cloud - Amazon/Rackspace?- Couldn’t move it “as is”

SaaS Distributed Grid - Cloud Bursting Scenario

Oracle Grid Engine

Grid Master

Slave

Slave

Slave

Stats Desktop

SAS

Matlab

Amazon Cloud

Slave(m3 large)

Slave(m3 large)

Slave(m3 large)

Slave(g2 GPU)

Slave(g2 GPU)

Slave(g2 GPU)

1. Provision new instances

2. Extend Capacity Internally

3. Transfer Files

4. Execute Files

5. Shutdown when done

R

SaaS eCommerce Business Management Scenario

Amazon Web Services - Cloud Computing Services

Using AWS for ~ 4 years as a IaaS platform

Brightpearl is designed for use on AWS

Make use of both US and European datacentres, multi-availability zones - approx 90 EC2 instances

All Amazon Linux based images, use Centos 5.x in dev/test

We approximate an environment for development and test

Architectural Overview

Will break it down into the following views:

- Infrastructure and Operating System

- Application Services (Queuing, Data Storage, Load Balancing)

- Software Stack (Brightpearl Application - JS, PHP and Java)

Infrastructure

We build on various Amazon base Images (based on Amazon Linux) with different specifications...

- m1 medium (webserver) - t1 micro (mail relay)- m1 small (messaging)

EC2Instances is useful - http://www.ec2instances.info/

Software is provisioned on top of the base O/S with Chef (we maintain this configuration and keeping it up to date).

Application Services

Elastic Compute Cloud (based off Xen)- Various instance sizes (small, medium large, xlarge)- Basis is an AMI - Centos, RHEL, Windows

Amazon RDS (Relational Database Service) - aka MySQL - Data Storage

Content delivery network, think Akamai- Global delivery of static resources (images, content)

Application Services

Key/value store - useful for storing large data that won’t fit in a relational database

Amazon S3 - storage service for files

ELB (Elastic Load Balancer) - instance load balancing

Languages - Javascript

Javascript for DOM manipulation, data binding, validation

- Functional, oh so functional- JQuery for DOM manipulation and UI elements- Backbone for structure (+ CommonJS)- Mocha for testing- JS on the serverside - Node- Dependency Management (Bower, NPM)

Languages - PHP

PHP for web development in the presentation tier

- Dynamically typed, interpreted

- Single threaded

- Well supported, lots of third party software

- Lightweight, fast and proven

Languages - Java

Java for scaling services out

- Statically and strongly typed

- Good for concurrency and parallelisation

- Good library, framework & IDE support (Intellij)

- Build RESTful API’s for PHP to communicate with

Languages - Ruby

Ruby - for provisioning infrastructure, configuration and test

- Our “devops” and “test engineer” language- Dynamically typed, multi paradigm- Readable, testable, way cool- Great third party library support- Chef by Opscode used for configuring EC2 instances- Cucumber and Webdriver for application testing

Design Problems

AWS Design Considerations

Sometimes we need to be agnostic for performance, cost reasons and also for vendor (Amazon) lockin

Have to build in failover to each individual service

Don’t use SQS (Simple Queueing Service) - instead we roll our own

Roll our own datagrid for cross EC-2 instance data - an in-memory datagrid. Think distributed Java collections.

Make use of a distributed file system for transient file storage

Problems at Scale

Design for redundancy:

- Ephemeral storage by default. Use EBS (Elastic Block Storage)- Multiple copies of services deployed on instances- Multiple instances for failover

Design for scale:

- Partitioning of accounts using separate RDS instances

Problems at Scale

Design for concurrency:

- Immutability is key, Actor Model, STM are useful

Design for performance:

- Profile everything (Yourkit is great for this)- Java is very memory hungry… tune the JVM and GC strategy

Problems at Scale

Design for cost:

- Reserved instances (up-front cost) can save a fair bit

Design for content delivery:

- Server side caching (Varnish, mod_cache)

- Client side caching (expires headers, etags)

- Content Delivery Network - (Cloudfront, Akamai)

Design for failure:

- Internal Services (Redundant Copies)

- External Services

- Protect against them overloading your internal services

- Don’t flood them with your traffic

Problems at Scale

Future Tech

Some future technology we’re interested in...

- Web Components (Polymer)

- Functional JVM languages (JDK 8 Streams, Clojure)

- Docker (LXC containers) for virtualisation

- Interested in Quasar for lightweight threading

Oh by the way… we’re expanding...

Current Open PositionsSenior DevelopersGraduate Developers Test Engineers

Key Events @ Brightpearl5th August - Java Meetup Group20th August - PHPSW Meetup

Questions?