Creating SaaS Startups that Rock: Scaling to Millions of Users

Creating the SaaS Startups that Scale to Millions of UsersCreating the SaaS Startups that Scale to Millions of Users

Hasan Basri AKIRMAKMentor at ITU Seed, Cloud Evangelist at Ericsson

https://tr.linkedin.com/in/hasanbasriakirmak

https://tr.linkedin.com/in/hasanbasriakirmak

This session was presented in ITU Cekirdek, http://www.itucekirdek.com/enThe University Startup Ecosystem in Istanbul, on December 13th 2016.

http://www.itucekirdek.com/en

Do you often tell your potential customer or investor things like “This is a PoC”,“It only works on a single host”, “We need time to install on another server”,“We need to redesign the code to handle ten thousand users” ?

You need to have solid answers to how your platform scales on growingdemand if you aim for Silicon Valley.

In the first part of this seminar, we’ll focus on non-technical stakeholders likeCEO and CMO co-founders, and we’ll have an overview of designingenterprise grade B2B, B2C or B2G Software as a Service (SaaS) platformsusing cloud technologies. What are the best practices to scalable, highlyavailable and reliable SaaS platforms and the store front, customer supportchannels? Why is an on-demand, elastic IT infrastructure setup necessary?What do scalability and automation have to do with the OPEX and theBusiness Case?

In the second part of this seminar, we’ll go into technical details for CTO co-founders and give examples for realizing platforms that scale from 1 to 1MUser on cloud. You’ll also have an understanding of the products and servicesneeded to realize this. You’ll also have an understanding of the products andservices needed to realize this from Amazon (AWS) Cloud: Shopify (e-shop),Zendesk (Customer Support), AWS API Gateway, Lambda (ServerlessComputing), EC2 (Virtual servers), ASG (Auto Scaling), SNS (NotificationService), CloudWatch (Fault and Performance Monitoring & CentralizedLogging), CloudTrail (Compliance), CloudFormation (Infrastructure Automation).

Motivation

İTÜ Çekirdek - Creating Startups that Rock | Creative Commons CC0 - Attribution License | https://www.linkedin.com/in/hasanbasriakirmak | 2016-12-13 | Page 4

What to expect from this session

› Part 1

– “Ticket to Haydarpasa or Silicon Valley” – Google Ventures Engineering Team

– “I’m gonna beat u Zuckerberg” – Industrialized Production

– “From Illusionist to Magician” – The B2B Perspective

– “From Black Friday to Godzilla” – Defining Business Performance

– “The Business Case for SaaS” – OPEX, CAC, Outsourcing

› Part 2

– “Building Infrastructure at Scale” – Well Architected Framework

– “Running it” – Tracking Performance with KPIs

– “Time vs. Space Tradeoff” – Compute, Storage, Data and Apps: The New Normal

– “Putting Pieces Together” – Scalable SaaS Platform on AWS


The opinions expressed here represent my own and not those of my employer, AWS or

any other trademarks/companies mentioned herein.

DISCLAIMER

Creative Commons License

Licensees may copy, distribute, display and perform the work and make derivative works and remixes based on it only if they give the

author or licensor the credits (attribution).

LICENSE




› Engineering

› “When a startup begins to take off, the technical requirements for data, computing, and networking skyrocket. At GV, we’ve built a team that lives for these challenges. They’re a group of experts with a track record of working at massive scale, and they love to help.”

› — Graham Spencer

› General Partner at GV

Why this session?

Source: Google Ventures


“Adidas’s pilot factory in Ansbach is an example of how robots are revolutionisingmanufacturing.” [Financial Times]

You need to be on the right to beat Silicon Valley.


In Software Context› Single developer vs. Mass production

› Small Workshop vs. Factories– Standardize HW, SW and Operations for Innovation

– Automate (No human interaction) for OPEX

Industrial Production


› “You can sell products to B2C using emotional levers (games, fashion, etc.) but, in B2B, getting customers excited will not get you a check.”

› “B2C consumers are accustomed to ‘what you see is what you get.’ They can choose to use it or not. Businesses have higher expectations (security, customization, integration, performance, etc.)*

Source: Lean B2B: Build Products Businesses Want, Garbugli, Étienne.Image: IMDB

B2B also has higher expectations



› Revenue Drivers– Consider upfront payments

– Contract Length & discounts

– Monthly Recurring Revenues

– Churn effect

› CAC – Cost of Acquiring Customer– Digital marketing to help you

› LTV – Customer’s Life Time Value– Retaining the subscriber

– Add-on sales (new features, or increase consumption)

SaaS Business Case – Revenues

Source: http://www.forentrepreneurs.com/saas-sales-compensation-plan/

http://www.forentrepreneurs.com/saas-sales-compensation-plan/


› Revenue Drivers– Consider upfront payments

– Contract Length & discounts

– Monthly Recurring Revenues

– Churn effect

› CAC – Cost of Acquiring Customer– Digital marketing to help you

› LTV – Customer’s Life Time Value– Retaining the subscriber

– Add-on sales (new features, or increase consumption)

SaaS Business Case – Revenues




SaaS Business Case – Costs› COGS

– IT infrastructure components

› CAPEX: Acquisition Costs

- Server, Storage, Networking

- SW Development: Reliability, Availability, Security

– OPEX: Operation Costs

– Support Costs

› 1st line: Web, mail or Call Center

› 2nd line: Software Development Team

› 3rd line: 3PP (AWS, Shopify, …)

› Risk Mitigation Costs– How much Data can we afford to lose?

– How much downtime can we accept?Diagram Source: AWS


SaaS Business Case – Costs› COGS

– IT infrastructure components

› CAPEX: Acquisition Costs

- Server, Storage, Networking

- SW Development: Reliability, Availability, Security

– OPEX: Operation Costs

– Support Costs

› 1st line: Web, mail or Call Center

› 2nd line: Software Development Team

› 3rd line: 3PP (AWS, Shopify, …)

› Risk Mitigation Costs– How much Data can we afford to lose?

– How much downtime can we accept?Diagram Source: AWS


› Make sure customers are happy after purchase so they will remain long term customers.

› Book as much New Recurring Revenue as possible.

› Collect as much Cash Upfront as possible. (Particularly important in the startup phase)

› Sign Longer Term Contracts (e.g. annual terms vs. monthly, multi-year vs. annual)

› Drive expansion revenue with existing customers to have a revenue retention rate of greater than 100%, even when you lose some customers due to churn.( “Negative churn”)


What are the Key business objectives for a SaaS business?



Identify which one is more profitable: Adding New Users vs. Stimulating Consumption?


For Small Contract size, use Internet as Primary Mode of Distribution

Source: http://www.forentrepreneurs.com/

http://www.forentrepreneurs.com/


› Compare your current on-premises environment to your FUTURE STATE, RE-ARCHITECTED cloud environment. NOT a replica of what exists today.

Cloud is more than a technology

Business Agility

› Rapid experimentation.

› Faster app development

› Faster business decisionsSource: AWS ReInvent 2016

Workforce Productivity

› Self service culture

› Increased automation

› Attracting & retaining talent

Operational Resilience

› Improved Service Performance

› Less Downtime

› Less bugs

Other (In)tangible Benefits you can qualify


Technologies that are difficult to implement can become easier to consume by pushing that knowledge and complexity into the cloud vendor's domain.

For example, NoSQL databases, media transcoding, and machine learning are all technologies that require expertise that is not evenly dispersed across the technical community. In the cloud, these technologies become services that your team can consume while focusing on product development rather than resource provisioning and management.

Cloud Democratizes Advanced Technologies


Virtualization is NOT cloud


› Migration (especially with redesign is complex and costly)

Never say “We move to cloud later!”

Source: AWS Cloud Adoption Framework


Time is Money. Performance Matters


Understanding End to End Performance


Godzilla will visit you one day

Image: IMDB


Acceptable Downtime and Loss of Data for your Business

Identify where your startup should be


› Backup Requirements for each workload

› HA (High Availability) requirements for each workload

› Scalability requirements for each workload

› DR (Disaster Recovery) Requirements for each workload

Performance Requirements: A Nice to Have?


How do we match with demand, adding/removing resources when needed?

“Righscaling”.

Time-based, event-driven approaches

The big question

Part 2

Creating a software system is a lot like constructing a building. If the foundation is not solid there might be structural problems that undermine the integrity and function of the building.

When architecting technology solutions, do not neglect the four pillars of security, reliability, performance efficiency, and cost optimization.


Fallacies of Software Engineering

›Servers don’t fail

›Disks don’t fail

›Networks don’t fail

In the cloud, test how your system fails, and validate your recovery procedures.


SaaS Architecture – Apps vs Ops View

Data Partitioning

Tenant Isolation

Identity

Pro

filing

& A

nalyt

ics

Man

agem

ent &

Mon

itorin

g

Met

erin

g, B

illing

& T

enan

t A

dmin

istra

tion

Operational View

App

Vie

w

Technical and Business Agility


Understanding Multi-tenancy

Silo Pool

vs.

› Silo Model

› Pros– Tenant specific tuning

– Tenant level availability

› Cons– Cost

– Management Complexity

– Deployment Challenges

– Analytics aggregation

› Pool Model

› Pros– Centralized management

– Cost optimization

› Cons– Cross-tenant impacts

– Compliance challenges

– All or nothing available


TENANT ID ITEM ID

T1 230

t2 3434

Data partitioning

Tenant 1 Tenant 2

Schema

Schema

Separate DB for each tenant

Single DB, multiple schemas

Common DB, single schema


› Actively track KPIs– Resource Level: CPU load, memory usage, network traffic

– Business level: Response times, RTO and RPO, Costs …

› Use Automation: Logging in and checking the CPU manually is not scalable!

› Implement self healing– Example Monitoring: Send an alarm when cache is 80%

– Example Action: Purge cache automatically when 90%

– Example Monitoring: Sense list of IP’s generating unsuccessful SSH attempts.

– Example Action: Block those IP’s by adding them to firewall blacklist rules automatically

› Learn to implement your “Infrastructure as Code”

Managing your infrastructure at scaleHow do you monitor your instances post launch to ensure they are performing as expected?

Amazon

CloudWatch

AWS

CloudFormation

Amazon

Elasticsearch

Service


Performance Efficiency in the cloud is composed of four areas:

1 Compute

2. Storage

3. Database

4. Space-time trade-off


› Share info

› Extract insight

› Recognize patterns

› Track performance

› Ultimately make better business, technical and operational decisions.

What Can You Do with all this Data?

Generation

Collection & Storage

Analytics & computation

Collaboration & Sharing


COMPUTE

Manage Storage & Compute Resources Separately in Cloud

STORAGE


Don’t do a Monolithic DB + Search Tier

RDBMS

DB & Search Tier

Apps


Types of Data

› Transactional– DB reads & writes (OLTP__

– Cache

› Search– Logs

– Streams

› File– Log files

› Stream– Clickstream, Sensors, IoT data

Big Data Architectural Principles & Understanding Types of Data


Use the right tool for the job. There are multiple services for DB & Analytics in AWS


Data Structure and Access PatternsACCESS PATTERNS WHAT TO USE in AWS?

Joins, Transaction, SQL SQL: RDS

Put/Get (Key/Value) Cache: ElastiCache

(Memcached, Redis)

NoSQL: DynamoDB

Simple Relationships, No Transaction, Joins NoSQL: DynamoDB

Search Search: ElasticSearch

DATA STRUCTURES WHAT TO USE in AWS?

Fixed Schema SQL: RDS

NoSQL: DynamoDB

Schema Free (JSON) NoSQL: DynamoDB

Search: ElasticSearch

Key Value Cache: ElastiCache

(Memcached, Redis)

NoSQL: DynamoDB


Use the Right Tool for the Job

DB & Search Tier

Apps

Cache

MemCached, Redis

NoSQL

Cassandra, DynamoDB

Hbase, MongoDB

SQL

MySQL, Oracle, SQL Server, Postgres

Search

ElasticSearch, CloudSearch


“Data Lake” Architecture & AWS ServicesAPI & UI

Amazon API

Gateway

AWS Elastic

Beanstalk

Amazon

S3

Amazon

Glacier

Amazon

Kinesis

AWS

Lambda

Amazon

CloudSearchAmazon

DynamoDB

AWS Elastic

Beanstalk

AWS IAM AWS STS

(alternate)

AWS KMS encrypted

data

Users

Storage & Streams Catalogue & Search Entitlements


The optimal storage solution for a particular system will vary based on the:

› Kind of access method Block, file, or object

› Patterns of access Random or sequential

› Throughput required IOPS, SSD vs. HDD

› Frequency of access Online, offline, archival

› Frequency of update WORM, dynamic

› Availability and durability constraints 99 vs. 99.9…9% and more

Understand Storage Parameters


› Use Stateless Design: Support 12-factor stateless apps: 12factor.net

› As Deployment Unit: Use Docker container images

Modern Software Design Goals

12factor.net


Putting Pieces TogetherA Scalable SaaS Platform Implementation

AWS API(order fulfilment triggered by e-commerce platform)

AWS CloudFormationInfrastructure as Code

AWS LambdaSend login and web address information to customerper email using SES

AWS CloudWatchInfra & Application layerFault & Performance Monitoring

AWS CloudTrailCompliance reporting (Log of all requeststo AWS account and API)

AWS ConfigResource Configuration Managementkeeping track of versions of every resource

AWS DynamoDBMetadata repository (file name, size…)

AWS S3Storage with 99.999999999% durabilityTenant isolation by buckets

AWS SNSEmail notifications to Operations Team about alarms

AWS EC2Compute service for Open Source Sync SwSingle tenant for tenant isolation

AWS EBSNAS Storage service, for EC2 cache

AWS ASGReplaces EC2 in case

of sys or health check errors

AWS Route 53DNS Service

Shopify.comSaaS E-Commerce Platform as sales channel

Zendesk.comSaaS Customer Support Channel

: A Marketplace for Partner

Products, Solutions or SaaS running on AWS

AWS IAMMFA & Cross Account Access to Confounders

Bitnami.com3PP Baseline AMIAWS CloudFront

CDN for Joomla Microsite


SaaS Fueled by Digital Marketing

ReferencesThe Best Practices explained here and used in the scalable SaaS platform implementation are based on several advanced AWS Seminars and whitepapers.

AWS ReInvent 2015 Sessions available on YouTube1. DVO315 Log Monitor Events with AWS CloudWatch, Lambda and Dashboards. Link2. BDT317 Building a Data Lake on AWS. Link3. WEB401 Optimizing your Web Server on AWS. Link4. SEC313 Security by Design. Link5. DEV203 AWS API Gateway and Lambda. Link6. DVO303 Scaling Infrastructure Operations AWS Config and AWS CloudTrail. Link7. ISM402 Cost Optimization at Scale. Link8. ARC302 Running Lean Architectures: Optimizing for Cost Efficiency. Link

AWS ReInvent 2016 Sessions (Not available on YouTube yet)

1. ARC301 Architecting Next Generation SaaS 2. ENT308 Building a Solid Business Case for Cloud Migration

APN (AWS Partner Network) Webcasts (You need to be an AWS Partner to access this content)1. SaaS Architecture ()2. Amazon EC2 Masterclass Webcast

AWS Whitepapers 1. AWS Well Architected Framework. Link2. Managing Infrastructure at Scale. Link

http://bit.ly/2hPoAnE

http://bit.ly/2in6Yga

http://bit.ly/2in99jO

http://bit.ly/2imWAVw

http://bit.ly/2hji5up

http://bit.ly/2i9gDrx

http://bit.ly/2i9a8Vy

http://bit.ly/2i9aFGV

http://bit.ly/2hZIBGg

http://bit.ly/2hZE9Yi


›Lessons Learned from Creating a B2B Cloud SaaS Startup in 90 Days: From Ideation to Launch

›Designing an End to End Security Architecture in Cloud

›20 AWS Services in 120 Minutes

FOR MORE INFO ABOUT LEARNINGS & OUTCOMES, JOIN THE UPCOMING SEMINARS:


If you found this session useful, please “endorse” me in LinkedIn for areas like: Business Development, Business Planning, Go-To-Market Strategy, Consulting,

Product Management, Product Strategy, Cloud Computing, Entrepreneurship.

THANK YOUHasan Basri AKIRMAK

Differentiate in Information Technology