View
2.679
Download
2
Category
Preview:
Citation preview
WHY STARTUPS ARE (STILL)
ON AMAZON WEB SERVICES
Danilo Poccia | Solutions Architect
Startups?
http://www.flickr.com/photos/klearchos/5857451393/
« A startup is a human institution
designed to create a new
product or service under
conditions of extreme
uncertainty. »
Eric Ries, The Lean Startup
Startups fail?
Assume customer is known
Assume features are known
Assume solution is known
Waterfall approach You know the problem and the solution
TEST LAUNCH BUILD SPEC
Known set of
requirements
Known ways to
satisfy them
Agile methodologies You know the problem, not the solution
TEST LAUNCH BUILD SPEC
Known set of
requirements
Unknown ways
to satisfy them
VALID?
You don't know precisely
what problem you solve
Lean startups:
LEARN & ADAPT
1. Focus on a simple implementation
of your idea
1. Focus on a simple implementation of
your idea
2. Start with a minimal core set of
features
1. Focus on a simple implementation of
your idea
2. Start with a minimal core set of
features
3. Release and listen to your users
1. Focus on a simple implementation of
your idea
2. Start with a minimal core set of
features
3. Release and listen to your users
4. Question your initial assumptions
based on feedback
1. Focus on a simple implementation of
your idea
2. Start with a minimal core set of
features
3. Release and listen to your users
4. Question your initial assumptions
based on feedback
5. Rinse and repeat
MVP Minimum Viable Product
MVP Smallest thing I can do to test my idea?
« If you're not embarrassed
when you ship your first version
you waited too long »
Matt Mullenweg
CEO & Founder of WordPress
Staying lean is creating the
smallest viable product and
then iterate around it You don't know precisely your user's needs
RELEASE EVAL BUILD IDEA
Unknown set of
requirements
Unknown ways
to satisfy them
ITERATE OR
PIVOT
RELEASE EVAL BUILD IDEA ITERATE
OR
PIVOT
YOUR problem you have the idea
you don't have the resources
It should be cheap and validate ideas
SPEED AND AGILITY
Experiment
Often
Fail quickly
at a low cost
More
Innovation
Experiment
Infrequently
Failure is
expensive
Less
Innovation
“ON-PREMISE”
RELEASE EVAL BUILD IDEA ITERATE
OR
PIVOT
you created a fantastic webapp
http://www.flickr.com/photos/scobleizer/3985020876/
people love your app
http://www.flickr.com/photos/grantrobertson/448648883
1
and everyone wants to use it!
http://www.flickr.com/photos/livingos/4480894461/
the traditional way…
invest on infrastructure
capacity planning…
http://www.flickr.com/photos/mutsmuts/4695658106
/
…capacity guessing
once it's deployed…
maintenance?
monitoring?
log analysis?
test environments?
http://www.old-computers.com/news
Unable to
serve
customers
Infrastructure
Cost $
time
Large
Capital
Expenditure
Opportunity
Cost
Predicted
Demand
Traditional
Hardware
Actual
Demand
Automated
Virtualization
with AWS…
“When we started musiXmatch in 2010, we wanted
to focus on building a great user experience for our
users and our customers", says Francesco Delfino,
cofounder, "We choose Amazon Web Services
because it allowed us to freely define and fine tune
our server architecture, while shielding us from
common hardware issues.
During these years, AWS sustained our growth
providing the resources we needed as soon as our
traffic increased: there is no bigger cost for a startup
than missing the opportunity to scale exactly when
you need it.”
“Amazon Web Services gives us the
power to scale our infrastructure
without worrying about capacity limits.
Our infrastructure is distributed and
currently runs on 3 regions.”
www.spreaker.com
RELEASE EVAL BUILD IDEA ITERATE
OR
PIVOT
Data based
decision making
1. Collect as much data as you can
1. Collect as much data as you can
2. Do A/B testing
1. Collect as much data as you can
2. Do A/B testing
3. Drive your development by
user's feedback
Store valuable data sources Server logs, click streams, application events, …
"Hadoop is a reliable storage and data analysis system"
HDFS MapReduce
Deploying a Hadoop cluster is hard
http://eddie.niese.net/20090313/dont-pity-incompetence/
Amazon Elastic
MapReduce Hadoop + The AWS Cloud
Elastic Data Warehouse
Expand to
25 instances
Data Warehouse
(Steady State)
Data Warehouse
(Batch Processing)
Shrink to
9 instances
Data Warehouse
(Steady State)
3.5 billion records, 71 million unique cookies, 1.7 million targeted ads required per day
Targeted Ad
User recently
purchased a
sports movie and
is searching for
video games (1.7 Million per day)
Leveraged AWS and Elastic MapReduce 100 node cluster on demand
Processing time dropped from 2+ days to 8 hours
Increased ROAS by 500%
We use EC2, Auto
Scaling and Elastic Load
Balancing to deliver ads
along side our main
infrastructure.
We automatically absorb
peaks through scripts
monitoring our local farm,
and change entries in the
Route 53 (DNS) when
needed, allowing more
traffic towards AWS load
balancers.
www.dotandmedia.com
RELEASE EVAL BUILD IDEA ITERATE
OR
PIVOT
Iterate Enhance your product, get more feedback
to prepare next releases
The platform grows with you Add or remove components as needed
Scale up – scale down
Pay as you go
Pivot If no traction, pivot to address a different
vertical, a different problem
POSSIBLE
OFFERING
POSSIBLE
OFFERING
POSSIBLE
OFFERING
POSSIBLE
OFFERING
PROBLEM
SPACE
POSSIBLE
OFFERING
POSSIBLE
OFFERING
POSSIBLE
OFFERING
PROBLEM
SPACE
POSSIBLE
OFFERING
Getting the right features matters less
Knowing what's not working,
quickly, matters more
Getting the right features matters less
Knowing what's not working,
quickly, matters more
Experiment and
Innovate
Lean startup goal is to accelerate the cycle
CONTINUOUS
INTEGRATION
DEVELOPER
SOURCE CODE
REPOSITORY
SOURCE CODE
REPOSITORY
PROJECT MANAGEMENT
SERVER
SOURCE CODE
REPOSITORY
PROJECT MANAGEMENT
SERVER
CONTINUOUS
INTEGRATION SERVER
SOURCE CODE
REPOSITORY
PROJECT MANAGEMENT
SERVER
CONTINUOUS
INTEGRATION SERVER
PICK
TASKS
SOURCE CODE
REPOSITORY
PROJECT MANAGEMENT
SERVER
CONTINUOUS
INTEGRATION SERVER
SUBMIT
CODE
SOURCE CODE
REPOSITORY
PROJECT MANAGEMENT
SERVER
CONTINUOUS
INTEGRATION SERVER
SCHEDULE
BUILD
SOURCE CODE
REPOSITORY
PROJECT MANAGEMENT
SERVER
CONTINUOUS
INTEGRATION SERVER
RECURENT
BUILDS
SOURCE CODE
REPOSITORY
PROJECT MANAGEMENT
SERVER
CONTINUOUS
INTEGRATION SERVER
CODE
FETCH
SOURCE CODE
REPOSITORY
PROJECT MANAGEMENT
SERVER
CONTINUOUS
INTEGRATION SERVER
CODE QUALITY
TESTS
TEST
RESULTS
SOURCE CODE
REPOSITORY
PROJECT MANAGEMENT
SERVER
CONTINUOUS
INTEGRATION SERVER
BUILD OUTPUT
SOURCE CODE
REPOSITORY
PROJECT MANAGEMENT
SERVER
CONTINUOUS
INTEGRATION SERVER
DOCS
BINARIES /
PACKAGES
SOURCE CODE
REPOSITORY
DNS
CONTINUOUS
INTEGRATION SERVER
PROJECT
MANAGEMENT SERVER
BUILDS
PAIN POINTS: • UNIT TESTS INCOMPLETE
• MOCKS MAINTENANCE
• TEST ENV EXPENSIVE
• TEST ENV ≠ PROD
KEY = ITERATION
ITERATIVELY MODIFY THE
SYSTEM TO BETTER MEET
THE EXPECTATIONS OF
YOUR USERS
ON-DEMAND
PAY AS YOU GO
ELASTIC
AWS
CLOUDFORMATION STACK-BASED DEPLOYMENT
SERVICE
CLOUDFORMATION
TEMPLATE
{
"Description" : "Create RDS with username and
password",
"Resources" : {
"MyDB" : {
"Type" : "AWS::RDS::DBInstance",
"Properties" : {
"AllocatedStorage" : "500",
"DBInstanceClass" : "db.m1.small",
"Engine" : "MySQL",
"EngineVersion" : "5.5",
"MasterUsername" : "MyName",
"MasterUserPassword" : "MyPassword"
}
}
}
}
"AWS::CloudFormation::Init" : {
"config" : {
"packages" : {
"yum" : {
"mysql" : [],
"mysql-server" : [],
"httpd" : [],
"php" : [],
"php-mysql" : []
}
},
"sources" : {
"/var/www/html" :
"https://s3.amazonaws.com/my-builds/build-
v4.zip"
}
}
}
{
"Parameters" : {
"KeyName" : {
"Description" : "Name of an existing EC2
KeyPair to enable SSH
access to the instance",
"Type" : "String"
}
},
}
CLOUDFORMATION
TEMPLATE
PROCEDURAL
DEFINITION Create it programmatically
KNOWN
CONFIGURATION Store stack configuration in
source control
PARAMETER
DRIVEN Dynamic and user-driven
templates
COLLABORATION Share templates with ease
as just files
Template ELBs to front
secondary cache
~100 Nginx
secondary cache
servers
2-3 Nginx mid-tier
cache servers
Stack
CLOUDFORMATION TEMPLATE
VIDEO CACHING
INFRASTRUCTURE
APPLICATION
VERSIONS
& INFRASTRUCTURE
VERSIONS
CLOUDFORMATION
TEMPLATE
TEST ENVIRONMENTS
30,000 REQUESTS / SECOND
1 TB TRAFFIC / DAY
“…AWS seemed to be the best solution available to allow a small, independent company to rapidly develop and test a completely new infrastructure,
and host it.
We also loved the flexibility that AWS allowed us, when spinning up smaller test environments, for
beta trials, QA, localization, and during development. The low initial cost was also
crucial.”
Alex Evans, CTO
CONTINUOUS
INTEGRATION
CONTINUOUS
DEPLOYMENT
CONTINUOUS
DEPLOYMENT SMALL, FREQUENT CHANGES
CONSTANTLY INTEGRATING INTO
PRODUCTION.
SOFTWARE DEPLOY
≠ PRODUCT LAUNCH
1.5 BILLION PAGE VIEWS
OCTOBER 2012
$83 MILLION IN TRANSACTIONS
4.2 MILLION ITEMS SOLD
30 DEPLOYS PER DAY = 1 DEPLOY EVERY 20 MINUTES
« Complexity arises when the
dependencies among the
elements become important. »
Scott E. Page, John H. Miller,
Complex Adaptive Systems: An Introduction to
Computational Models of Social Life
“Production is truly the only place
you can validate your code.”
HTTPS://GITHUB.COM/ETSY/DEPLOYINATOR
HTTP://CODEASCRAFT.ETSY.COM/
HTTP://SORCERY.SMUGMUG.COM/
AWS OPSWORKS MANAGING THE COMPLETE
APPLICATION LIFECYCLE
A stack represents
your application. One
stack might be used
for staging and
another for production.
A layer defines how to
setup and configure a
set of instances and
related resources
such as volumes
and software.
Tell OpsWorks where
it can find your code
and define any
additional deployment
tasks. OpsWorks will
take care of deploying
your app.
Scale your stack
based on time or
load. Clone your
production stack to a
different region.
Automate workflows
for common tasks.
STACK | LAYER | APP | INSTANCE
GETTING STARTED WITH OPSWORKS
YOUR STACKS IN THE DASHBOARD
STACK OVERVIEW
LAYERS IN A STACK
INSTANCES IN A STACK
APPS IN A STACK
DEPLOYMENTS & COMMANDS
YOU CAN BRING YOUR
OWN CHEF RECIPES OR
LEVERAGE HUNDREDS
OF COMMUNITY-BUILT
CONFIGURATIONS
14 BILLION REQUESTS/MONTH
50 000 DATABASE UPDATES / SEC
NO CACHE
“AWS OpsWorks gives us the tools we
need to automate operations.
We can scale Monster World, one of the
largest Facebook games, to millions of
users without ever needing more than two
backend developers”
Jesper Richter-Reichhelm
Head of engineering – Wooga
A / B TESTING
LOAD TESTING
USING AMAZON EC2 TO SIMULATE
2.4 MILLION PLAYERS
DATA-DRIVEN
ARCHITECTURES
METRICS @ETSY
COST-ORIENTED
ARCHITECTURES
PHP+APACHE+VARNISH
NGINX+NODEJS
11.6s
Mean time
between
deployments
(weekday)
1,079
Max number of
deployments in a
single hour
10,000
Mean number of
hosts
simultaneously
receiving a
deployment
30,000
Max number of
hosts
simultaneously
receiving a
deployment
DEPLOYMENTS AT
AMAZON.COM
CONTINUOUS
DEPLOYMENT
=
CONTINUOUS
EXPERIMENTATION
CONTINUOUS
DEPLOYMENT
=
CONTINUOUS
IMPROVEMENT
FASTEST TIME TO
MARKET
“The AWS Cloud brings business agility as Shell is able to deploy services much more quickly”
Johan Krebbers
Vice President of Architecture
“AWS has enabled Soundtracker to
perform nimble development” says
Daniele Calabrese, Founder and CEO,
“allowing our service to scale quickly,
effectively, and seamlessly.”
focus on your business
Stefano Pochet, founder and CEO of Nealab, explains,
“Amazon Web Services fits the need for flexibility and
reliability required by high-traffic Web applications. In
addition, it makes it easier and cheaper to maintain our
entire infrastructure.”
…
“AWS has allowed us to forget about hardware and
focus on software. For us, AWS has really made Web
development easier.”
Recommended