www.techarchday.fiwww.techarchday.fi
Scaling up to your first 10 million users
Matthijs ten Seldam
Solutions Architect - AWS
www.techarchday.fi
US-WEST (Oregon)
EU (Ireland)
ASIA PACIFIC
(Tokyo)
US-WEST (N. California)
SOUTH
AMERICA (Sao
Paulo)
US-EAST (N. Virginia)
AWS GOVCLOUD (US)
ASIA PACIFIC
(Sydney)
ASIA PACIFIC
(Singapore)
CHINA (Beijing)
ASIA PACIFIC
(Seoul)
RegionsEU (Frankfurt)
INDIA (2016)
UK (2016)
Canada
Ohio
Ningxia
www.techarchday.fi
US-WEST (Oregon)
EU (Ireland)
US-WEST (N. California)
SOUTH
AMERICA (Sao
Paulo)
US-EAST (N. Virginia)
AWS GOVCLOUD (US)
ASIA PACIFIC
(Sydney)
ASIA PACIFIC
(Singapore)
Availability ZonesEU (Frankfurt)
INDIA (2016)
UK (2016)
Canada
ASIA PACIFIC
(Tokyo)
CHINA (Beijing)
ASIA PACIFIC
(Seoul)Ohio
Ningxia
www.techarchday.fi
ENTERPRISE APPS
Streaming Data Collection
Machine Learning
Elastic Search
Virtual Desktops
Sharing & Collaboration
Corporate Email
Backup
Search
Transcoding
Sync
Single Integrated Console
PushNotifications
Application Lifecycle Management
Containers
Triggers
Resource Templates
RegionsAvailability Zones
Points of Presence
INFRASTRUCTURE
CORE SERVICES
ComputeVMs, Auto-scaling, & Load Balancing
StorageObject, Blocks, Archival, Import/Export
DatabasesRelational, NoSQL, Caching, Migration
NetworkingVPC, DX, DNS
CDN
Access ControlIdentity Management
Key Management & Storage
Monitoring & Logs
Assessment and reporting
Resource & Usage Auditing
SECURITY & COMPLIANCE
Configuration Compliance
Web application firewall
IoT
Rules Engine
Device Shadows
Device SDKs
Registry
Device Gateway
Streaming Data Analysis
MobileAnalytics
The AWS Platform
HYBRIDARCHITECTURE
Data Backups
Integrated App Deployments
DirectConnect
IdentityFederation
IntegratedResource Management
Integrated Networking
MARKETPLACE
Business Apps
Business Intelligence
DatabasesDevOps Tools
NetworkingSecurity Storage
DEVELOPMENT & OPERATIONSMOBILE SERVICESAPP SERVICESANALYTICS
DataWarehousing
Hadoop/Spark
Queuing & Notifications
Workflow
One-click App Deployment
Identity DevOps Resource Management
API Gateway
Business Intelligence
www.techarchday.fi
AWS Building Blocks
Inherently scalable, highly available andfault-tolerant services
Highly available with the right architecture
Amazon CloudFront
Amazon Route 53
Amazon S3
Amazon DynamoDB
Elastic Load Balancing
Amazon EFS
AWS Lambda
Amazon SQS
Amazon SNS
Amazon SES
Amazon SWF
…
Amazon EC2
Amazon EBS
Amazon RDS
Amazon VPC
www.techarchday.fi
1 User
• Amazon Route 53 for DNS
• A single Elastic IP• A single Amazon EC2
instance– With full stack on this
single host• Web app• Database• Management• And so on…
Amazon EC2 instance
Elastic IP
UserAmazon Route 53
www.techarchday.fi
“We’re Gonna Need a Bigger Box”
• Simplest approach
• Can now leverage PIOPS
• High I/O instances
• High memory instances
• High CPU instances
• High storage instances
• Easy to change instance sizes
• Will hit an endpoint eventually
c4.8xlarge
m3.2xlarge
t2.micro
X1 (2TB, > 100 vCPU)
www.techarchday.fi
1 User
• We could potentially get to a few hundred to a few thousand depending on application complexity and traffic
• No failover
• No redundancy
• Too many eggs in one basket
EC2 Instance
Elastic IP
UserAmazon Route 53
www.techarchday.fi
Users > 1
First, let’s separate out our single host into more than one.
• Web
• Database Make use of a database
service?
Web Instance
DatabaseInstance
Elastic IP
UserAmazon Route 53
www.techarchday.fi
Self-managed Fully managed
Database server on Amazon EC2
Your choice of database running on
Amazon EC2
Bring Your Own License (BYOL)
Amazon DynamoDB
Managed NoSQL database service using
SSD storage
Seamless scalability Zero administration
Amazon RDS
Microsoft SQL Server OracleMySQL
PostgreSQLMariaDB
Amazon Aurora
BYOL or license Included
Amazon Redshift
Massively parallel, petabyte-scale data warehouse service
Fast, powerful, and easy to scale
Database Options
www.techarchday.fi
Users > 100
First, let’s separate out our single host into more than one:
• Web
• Database Use Amazon RDS to
make your life easier
Web instance
Elastic IP
RDS DB instance
UserAmazon Route 53
www.techarchday.fi
Users > 1000
Next, let’s address our lack of failover and redundancy issues:
• Another web instance– In another Availability
Zone
• RDS Multi-AZ• Elastic Load Balancing
(ELB)
Web Instance
RDS DB InstanceActive (Multi-AZ)
Availability Zone Availability Zone
Web Instance
RDS DB Instance Standby (Multi-AZ)
ELB Balancer
UserAmazon Route 53
www.techarchday.fi
Users > 10,000s–100,000s
RDS DB InstanceActive (Multi-AZ)
Availability Zone Availability Zone
RDS DB Instance Standby (Multi-AZ)
ELB Balancer
RDS DB Instance Read Replica
RDS DB Instance Read Replica
RDS DB Instance Read Replica
RDS DB Instance Read Replica
Web Instance
Web Instance
Web Instance
Web Instance
Web Instance
Web Instance
Web Instance
Web Instance
Amazon Route 53User
www.techarchday.fi
RDS DB InstanceActive (Multi-AZ)
Availability Zone
ELB Balancer
Amazon S3
Amazon CloudFront
Shift Some Load Around
• content to Amazon S3 and Amazon CloudFront
Web Instances
Move…Amazon Route 53
User
www.techarchday.fi
RDS DB InstanceActive (Multi-AZ)
Availability Zone
ELB Balancer
Amazon S3
Amazon CloudFront
Shift Some Load Around
• content to Amazon S3 and Amazon CloudFront
• session/state to Amazon DynamoDB
• DB caching to Amazon ElastiCache
Web Instances
Move…Amazon Route 53
User
DynamoDBElastiCache
www.techarchday.fi
Now that our web tier is much more lightweight, we can revisit the beginning
of our talk…
www.techarchday.fi
Automatic resizing of compute clusters
Define min/max pool sizes
CloudWatch metrics drive scaling
On-demand or Spot instances
aws autoscaling create-auto-scaling-group--auto-scaling-group-name MyGroup--launch-configuration-name MyConfig--min-size 4--max-size 200--availability-zones us-west-2c, us-west-2b
Auto Scaling
www.techarchday.fi
Sunday Monday Tuesday Wednesday Thursday Friday Saturday
Typical Weekly Traffic to Amazon.com
Provisioned capacity
www.techarchday.fi
Users > 500,000+
Availability Zone
Amazon Route 53
User
Amazon S3
Amazon CloudFront
Availability Zone
ELB Balancer
DynamoDB
RDS DB Instance Read Replica
Web Instance
Web Instance
Web Instance
ElastiCache RDS DB Instance Read Replica
Web Instance
Web Instance
Web Instance
ElastiCacheRDS DB Instance Standby (Multi-AZ)
RDS DB InstanceActive (Multi-AZ)
www.techarchday.fi
Service Oriented Architecture
• Move services into their own tiers.– Treat them separately and
scale them independently.
• Amazon and AWS do this extensively!
• It offers flexibility and greater understanding of each component
www.techarchday.fi
Don’t Re-invent the Wheel
Leverage the AWS pre-built services.
• Email• Queuing• Transcoding• Search• Databases• Monitoring• Metrics• Logging• Compute• …
Amazon CloudSearch Amazon SQSAmazon SNS
Amazon Elastic TranscoderAmazon SWFAmazon SES
AWS Lambda
www.techarchday.fi
Users > 1 million+
Reaching a million and above is going to require some bit of all the previous things:
– Multi-AZ
– Elastic Load Balancing between tiers
– Serving content smartly (Amazon S3/CloudFront )
– Caching of DB
– Moving state off tiers that auto scale
– Auto Scaling
– Service Oriented Architecture
www.techarchday.fi
Users > 1 million+
RDS DB InstanceActive (Multi-AZ)
Availability Zone
ELB Balancer
RDS DB Instance Read Replica
RDS DB Instance Read Replica
Web Instance
Web Instance
Web Instance
Web Instance
Amazon Route 53
User
Amazon S3
Amazon CloudFront
DynamoDB
Amazon SQS
ElastiCache
WorkerInstance
WorkerInstance
Amazon CloudWatch
Internal AppInstance
Internal AppInstance Amazon SES
Lambda
www.techarchday.fi
Users > 5 million - 10 million
• You’ll potentially start to run into issues with your database around contention on the write master.
• How can you solve it?– Federation—splitting into multiple DBs based on function
– Sharding—splitting one dataset up across multiple hosts
– Moving some functionality to other types of DBs (NoSQL, Graph)
www.techarchday.fi
In Summary…
• Multi-AZ your infrastructure.
• Make use of self-scaling services—ELB, Amazon S3, Amazon SNS, Amazon SQS, Amazon SWF, Amazon SES, and more.
• Build in redundancy at every level.
• Use Auto Scaling once you’re ready for it.
• Split tiers into individual services (SOA).
• Select the right database architecture.
www.techarchday.fi
Putting all this together means we should now
easily be able to handle 10+ million users!