Embed Size (px)
Citation preview
clda.co/aws-‐mostar12/15/2016
Serverless Compu5ng for Machine Learning applica5ons
About Me
@alex_casalboni
clda.co/aws-‐mostar
Bachelor in Computer Science
Master in Sound & Music Engineering
Web Developer (6+ years)
Cloud Evangelist @ Cloud Academy
Today’s Agenda
clda.co/aws-‐mostar
Amazon Web Services Overview
Machine Learning as a Service (MLaaS)
Serverless Machine Learning with AWS Lambda
Amazon Web Services -‐ IaaS
clda.co/aws-‐mostar
VPC
S3
AMIs
Elas5c IP
ECS
ELB
EFS
Route 53EC2
Keypairs
Volumes
Spot
Snapshots
S3
Glacier
Auto Scaling
CloudFront
Amazon Web Services -‐ PaaS
clda.co/aws-‐mostar
Elas5c Beanstalk
EMR
Lambda
API Gateway
CloudSearch
QuickSight
Athena
Kinesis
RDS
Elas5Cache
DynamoDB
RedshiZ
SQS
SNS
SES
AI
Amazon Web Services -‐ SaaS
clda.co/aws-‐mostar
Alfresco
Atlassian
Autodesk
Cloudability
Orbitera
Sumo Logic
Symantec
Trend Micro
CloudCheckr
Freshdesk
Datadog
New Relic
AWS CerBficaBons
clda.co/aws-‐mostar
Solu%ons Architect Associate
Solu%ons Architect Professional
Developer Associate
SysOps Associate
DevOps Professional
Advanced Networking (Beta)
Security (Beta)
Big Data (Beta)
cloudacademy.com/cerBficaBons
clda.co/aws-‐mostar
Cloud CompuBng Numbers
Adop5on (UK) Market Growth ($)
clda.co/aws-‐mostar
The move to the Cloud is inevitable, so don’t be leN behind!
Have you ever met a Data ScienBst?
Bob
Bob is very smart and curious
Bob loves Python & (Big)Data
Bob builds great ML models
Bob likes user-‐centered ML
clda.co/aws-‐mostar
Bob can build a model in ~30 LOC!
clda.co/ML-codeBob
clda.co/aws-‐mostar
Phase 1: Analysis & Modeling
+
+
Bob
Data
Time
clda.co/aws-‐mostar
Phase 2: VisualizaBon & Prototyping
+
+
Bob
Data
Time
ML Model
Data Visualisa5on
Prototype
+
+
clda.co/aws-‐mostar
Phase 3: Disillusion
+
+
Bob
Data
Time
ML Model
Data Visualisa5on
Prototype
+
+
Produc5on Code
clda.co/aws-‐mostar
Phase 4: ProducBon-‐ready
+
+
Bob
Data
Time
ML Model
Data Visualisa5on
Prototype
+
+
WebDev
DevOps
A lot of Time
+
+
clda.co/aws-‐mostar
The lack of ownership
!=
Data Scien5st DevOps
MathemaBcal modeling StaBsBcal analysis
Data mining
(Cloud) OperaBons System administraBon SoNware best pracBces
clda.co/aws-‐mostar
clda.co/aws-‐mostar
Machine Learning as a Service (MLaaS)
AmazonMachine Learning
AzureMachine Learning
GooglePredic5on API
IMBWatson Analy5cs
BigML
clda.co/aws-‐mostar
Amazon Machine Learning
AmazonML
One of the first MLaaS solu5ons (Apr 2015)
Great service for classifica5on and regression
Only linear models (linear & logis5c regression + SGD)
No support for advanced scenarios yet
clda.co/aws-‐mostar
AmazonML @ Cloud Academy
clda.co/7-‐day-‐free(no credit card required!)
Why is deploying ML models a challenge?
Bob’s prototypes are’t produc5on-‐ready
Bob needs Elas5city & High Availability
Bob will deploy hundreds of models
Bobs wants plenty of features
Bob
clda.co/aws-‐mostar
Deployment Strategies
1. Web-‐app controller
how will Bob update his model(s)?
same website auth layer?
shared uWSGI processes?
It’s disarmingly simple, but…
Bob
clda.co/aws-‐mostar
Deployment Strategies
2. Fleet of servers
same problems as before
Bob has many more machines to maintain
s5ll no elas5city (over-‐provisioning)
Bigger capacity and no code changes, but…
Load Balancing
Bob Bob2 Bob3
clda.co/aws-‐mostar
Deployment Strategies
<Bob>
3. Auto Scaling
s5ll shared resources? containers?
even bigger lack of ownership
what about caching, versioning and auth?
Bob achieved elas5city and availability , but…
Load Balancing + Auto Scaling
clda.co/aws-‐mostar
Deployment Strategies
4. Serverless CompuBng
+Amazon API Gateway
AWS Lambda
Happy Bob
Managed infrastructure + BYOC
Transparent elas5city and PAYG
Many useful features for free
Bob doesn’t need (to be) a DevOps anymore!
clda.co/aws-‐mostar
There is always a server somewhere, you just don't have to worry about it :)
clda.co/aws-‐mostar
Benefits of Serverless Machine Learning
Versioning, staging & caching
1 model = 1 microservice
Intui5ve RESTful interface
High Availability (no down5me)
Very likle opera5onal effort
Transparent elas5city (PAYG)
Failure isola5on Offline training phase
Produc5on-‐ready prototypes A/B tes5ng through composi5on
clda.co/aws-‐mostar
A simple example: Serverless SenBment Analysis
clda.co/ML-‐Lambda
clda.co/aws-‐mostar
A real-‐world example
Serverless ML @ Cloud Academy
Mul5-‐model architecture
RESTful interface for each ML model
S3 + RDS for storage
Periodic training (offline)
clda.co/aws-‐mostar
LimitaBons of Serverless ML
AWS Lambda
No real-‐5me models (only pseudo real-‐5me)
Deployment package management: size limit and OS libraries
Not suitable for model training yet (5 min max execu5on 5me)
Cold-‐start 5me is long and hard to avoid
Unit/integra5on tests help, but not enough
clda.co/aws-‐mostar
The Serverless Ecosystem
AWS Lambda
MicrosoZ Azure Func5ons
Google Cloud Func5ons
IBM Bluemix OpenWhisk
hook.io
Cloud Plaforms FuncBon as a Service
iron.io
Auth0 (SSO)
Misc
Firebase (DB)
clda.co/aws-‐mostar
