Living on Cloud 1001 Architecture - Allen Holub · 2019-08-28 · – Lets you talk to other services • E.g., eternal mailer Mail – Incoming: handled by Servlet doPost() – Outgoing:

©2011 Allen I. Holub www.holub.com 1

Allen I. Holub Holub Associates

www.holub.com [email protected]

Living on Cloud 1001

1

©2011 Allen I Holub

www.holub.com 2

Architecture

2


www.holub.com 7

Web 2.0

3


www.holub.com 15

The Big Negative

Odds are that only one data center will go down.

4 LivingOnCloud1001.key - March 28, 2014


www.holub.com 8

Practical Definitions

AJAX – UI Rendered by JavaScript in Browser – Talk to Server via HTTP w/o reloading page

Web Application, Software as a Service (SaaS) – Application is web based. Access via browser

Cloud – an amorphous mass of machines and data, accessed via a

“gateway” (your ISP). Client/Server Application – runs on known machines. – Server:Client => 1:*

Cloud Application – distributed throughout the cloud, with processing and data

happening on several anonymous machines.

5


www.holub.com 9

The Cloud Architecture

6


www.holub.com 10

Virtual Data Center (Amazon and Google)

7


www.holub.com 11

Virtual Data Center

You have complete control over your VM – Load whatever software you want – Write in any language – Use any Database Server or choose Amazon Service – Can host top-level domain (X.com)

You have complete control over your VM – You must do all the setup/installation/maintenance – You must do all the system administration – You must deploy manually and handle hot updates – You must write your application to handle scaling

manually. (No difference from your own data center)



www.holub.com 12

Virtual App Server (Google)

Java Web App (Servlets + static content)

JDO & Buckets

9


www.holub.com 13

Google AppEngine

It’s painless and automatic – No setup/installation/maintenance/sys-admin – Painless deployment (just upload .war file) – Standards based, easy to move to other platform. –SCALING IS COMPLETELY AUTOMATIC!

!You have to play by Google Rules – Java • Python, Groovy works. Java (servlets) is easier • No PHP – Tomcat-like container (Servlets/JSP. No EJB, JMS, …) – JDO/JPA • No viable SQL support outside of in-house enterprise apps

10

!11

11

!12

Amazon Web Services (AWS)


More Amazon Issues

Support provided (but it costs $) Good monitoring/report services PAINFUL DEPLOYMENT/MANAGEMENT

– VMWare “Cloud Foundry” solves problem: http://www.cloudfoundry.com/

–Only for Java (and any JVM-targeted language), Spring, Rails, Sinatra/, Ruby

!13

13

Static Web Site (S3)

!14

http://docs.aws.amazon.com/gettingstarted/ latest/swh/website-hosting-intro.html

14

!15

15

!16


!17

17

!18

Deployment load-balancing auto-scaling health-mgmt

In memory. Reddis, Mem- cached

MySQL

Petabyte data ware- house

Amazon Machine Image

EBS Storage

EC2 Elastic Cloud Computing

NoSQL (key/value. No nesting)

18

Preconfigured AMIs (+others)

!19

Debian

BitNami+ Ubuntu

19

Compute/Networking

Direct Connect

– Direct connection to Amazon to increase transfer bandwidth

VPC

– Put a LAN on Amazon and connect to it using a VPN.

!20


Storage

Glacier

– Archiving/backup service. High reliability. Storage Gateway

– Integrate AWS storage to your own local system.

!21

21

Analytics

Data Pipeline

– workflow engine Elastic MapReduce

– Hadoop for Amazon—big-data analysis Kenesis

– Streaming big-data analyzer

!22

22

Deployment/Management

CloudFormation – Create AWS configurations from scripts

CloudTrail – Record API calls from web applications

CloudWatch – Resource monitoring

IAM – Identiy/Access Management (PKI)

OpsWorks – DevOps Platform

!23

23

Services

CloudSearch

– website search Elastic Transcoder

– coverts media files SES

– email service SNS

– Publish SNS messages SQS

– Message queueing

!24


SWF (Simple

Workflow Service)

!25

25

!26

Cloud Platform

26

!27

App Engine

• Virtual App Server (eg. tomcat) • Python, Java, PHP, Go • Implements JDO/JPA, JCache,

java.net.URL, JavaMail, Logging

27


www.holub.com 14

App-Engine Negatives

500 outgoing email per day limit Can’t support HTTPS within your own domain – https://xxx.appspot.com works. Can’t host top-level domain, only subdomains – You must use ISP & forward to Google – www.xxx.com is a subdomain. Tech Support nonexistent App server configuration impossible. 30-second limit on service requests.



www.holub.com 27

AppEngine Service APIs

Memcache – Caches in memory – Deals with the fact that the app might move!

URL Fetch – Encapsulated by java.net.URL and related classes – Lets you talk to other services • E.g., eternal mailer

Mail – Incoming: handled by Servlet doPost() – Outgoing: simplified javax.mail.*

Images – Simple image manipulation on server • Dim, rotate, resize, flip

29


www.holub.com 28

AppEngine Service APIs

Google Accounts – Log in using Google’s login system – DON’T DO IT! • Locks you in to the AppEngine • Gives control of your customers to Google

Task Queues – Queue up POST to be issued at a future time/date. – Replaces chron processes.

BlobStore – Upload large objects (videos?) – Access later using URL.

Capabilities – See what Google services are currently running

Channel – Persistant “push” API – Let’s server push data to client without polling – 30-second max on HTTP request, so you can’t roll your own.

30

!31

Compute Engine

Like EC2 Cheaper Closed ecosystem Debian/CentOS only Scaling, load balancing,etc. built in

31

!32

MySQL

100GB MaxUses VM instance

Like S3: Bucket- based object store

NoSQL Key/ValueBigTable, typed.


!33

Builds REST client libraries: Android/IOS/etc.Big-data analysis.

Natural Language

Machine Learning

33


www.holub.com 21

Other Eclipse Integration

Makes it easy to build Gdata Apps Eases RPC creation – otherwise an annoyingly persnickety operation

Allows testing of browser-side code in Eclipse – Uses a IE/Firefox/Chrome/Safari plug-in so GWT app runs in real

browser. – But, you can set breakpoints, log to Eclipse console, etc.

For AppEngine projects – simulates deployed environment when testing – disallows Java code that won’t work when deployed.

Client/Server-side profiling (SpeedTracer)

34

!35

Google’s war on

JavaScript

35

!36

Native VM for Chrome Compiles to JavaScript…

…but interfacing to JavaScript is primitive.


!37

import 'dart:math' show Random; // Import class!void main() { // Starts executing here! print(new Die(n: 12).roll()); // Print value. Chaining!}!!class Die { // Define class! static Random shaker = new Random(); // Define class variable! int sides, value; // Define instance var!! String toString() => '$value'; // Define method shorthand!! Die({int n: 6}) { // Constructor! if (4 <= n && n <= 20) {! sides = n;! } else {! throw new ArgumentError(/* */); // Errors & exceptions! }! }! int roll() { // Instance method! return value = shaker.nextInt(sides);// Get a random number! }!}

37


www.holub.com 22

Java (compiles-to/replaces HTML/JavaScript) Works well in Eclipse (client &server-side debugging, JUnit). CSS for look & feel.

38


www.holub.com 23

Uses it’s own widgets (HTML integration marginal)

39


www.holub.com 24

Panel-based layout



www.holub.com 25

GWT RPC

41


www.holub.com 17

Development Tools

Google supports both command-line (raw or ANT) and Eclipse

42


www.holub.com 20

Visual UI Designer

43

!44

Get over it!JavaScript

works fine.

Anybody remember ColdFusion?



www.holub.com 30

Analytics

Apps

Base Data

Blogger

Booksearch

Calendar

Code Search

Contacts

Documents

Finance

Health

Maps

Picasa

Project

Sidewiki

Sites

Spreadsheets

Translator

Webmaster

YouTube

https://developers.google.com/gdata/docs/directory

GData

45

© 2014. Allen I. Holub www.holub.com !46

JavaScript

Objective C

REST+wrappers

46


www.holub.com 31

Oauth and AuthSub

!!AuthSub is google proprietary. Use OAuth. Secure access without exposing Google password – User logs on to Google login page – On success, Google sends “session key” or

“token” to URL you specify – Token allows access to subset of Google

facilities • One or more calendars, but not all of them.

47


www.holub.com 33

Q&A

Allen [email protected]
