Understanding the Windows Azure platform - june

Understanding the

Windows Azure Platform

David Gristwood

Application Architect, Microsoft

blogs.msdn.com/david_gristwood

Why a Cloud Platform?

Reduce capital & operations costs

Simplify application deployment & management

Simplify scaling to internet scale

Cost effectively handle peak loads

Focus on new functionality & not infrastructure

The Azure Philosophy

Windows Azure is about Scale

• Windows Azure is

designed from the ground

up for true Internet scale

• Global presence

• Ride the “network effect” • LinkedIn: 16 months for first

million users, 11 days for most

recent million

• Facebook: 5 years to hit 250m

users, 8 months to double

• Scale up and scale down

• Supports the “Scale Fast,

Fail Fast” model

www.flickr.com/photos/ScroffTheBad

Application Characteristics “Scale & Elasticity” • Need massive scale

• Social networking, web 2.0

• Need high reliability

• LOB, software as a service

• Have variable load / unpredictable lifetime

• Ticketing, marketing

• Parallel processing

• Financial applications

• Don’t fit in well with a company’s data center

• Avoid IT dept, joint ventures

• Benefit from external storage

• Archive

http://www.flickr.com/photos/39919537@N06/3666813048/

Source: Alexa

~4x normal load (Holiday shopping)

• target.com • walmart.com

• toysrus.com • barnesandnoble.com

Jan 2009 Jan 2010

Helps your Systems Scale

www.flickr.com/photos/39818355@N00/2128626176

• Most architecture is

hidden inside code

• Windows Azure

encourages you to

define your application

into constituent parts

• Windows Azure can

scale the parts of your

application as

appropriate

Flexible Architecture

• Systems evolve over

time

• Cloud is not an “all

or nothing”

proposition

• Windows Azure

makes it easier to

extend to the cloud

www.flickr.com/photos/tomkpunkt/3695066577

Opening up the Cloud

www.flickr.com/photos/h19/2306213399/in/set-72157594224557858

• Allow developers to

apply their existing

skills to the cloud

• Support for .NET,

Ruby, PHP, Java

• Interoperability with

any platform, tools or

technology

Natural Platform Evolution

• Windows Server 2008

• Virtualisation

• Windows Services,

such as IIS, logging,

diagnostics, etc

• Visual Studio & .NET

Flexible Business Model

• No up front

hardware

purchase

• Pay as you go

costing model

• Ideal in current

economical

climate

www.flickr.com/photos/cleebster/2508660124

Azure Adoption

Azure is Live…..

http://www.liveonazure.com/

Managing Data Centric Applications with Capacity Bursts

Elastic scale – database as a service Pay as you grow and shrink Easy to provision and manage database No hardware, no manual database administration required

Promotions, events, ticket selling businesses are “bursts – bound” by nature Capacity constraints limit business agility High costs of entry into new business Difficult to roll out extra capacity quickly Idle capacity “off-bursts” is cost prohibitive

Cap

acit

y

Time

“Capacity Bursting“

Average Usage

Average Usage Co

mp

ute

#

of

Hr’

s

Challenges Today

Data Centers

North Central USA

South Central USA

Northern Europe

Western Europe

Eastern Asia

Southeast Asia

Regional hosting

200ms Latency from 2 regional hosting

San Antonio

Chicago

Dublin

Singapore

Amsterdam

Hosting locations within 100ms of 100ms of the customer

At least one hosting location can can be reached within 100ms, but but not two

No points to test from or greater greater than 200ms latency

Hong Kong

Microsoft has run Online Services since 1994 1st MSFT datacentre built in 1989 Running Online Services 24x7x365 Global service delivery to 59 markets/36 languages

∙ 30B Live ID authentications/month ∙ 2B Bing queries/month

∙ 10B MSN page views/month ∙ 240B Messenger messages/month

Global Infrastructure to Run Your Services World Wide Quincy, WA: Approx 500K sq ft, hydro-powered San Antonio, TX: Approx 475K sq ft, recyclable water Chicago, IL: Approx 700K sq ft, water economization Dublin, Ireland: Approx 303+K sq ft, air economization

http://www.globalfoundationservices.com/

One of world’s largest datacentres using containers, on-line on July 2009

Cost $500 million,

700k sq ft facility (approx 16 football fields)

30 MegaWatts today, 60 MW for future use

Security

ISO 27001:2005 Accreditation - Security Management

SAS 70 Type I and II Attestations - Audit standard for our internal controls system

Containers

40 ft containers with 1800-2500 servers

10 x density traditional datacentres

Optimizes efficiency, reduces wastes & carbon footprint

Delivers an average PUE of 1.22

First Mega Datacentre built outside the

US, on-line on July 2009

Cost $500 million, 303+sq ft facility and

growing

EU Datacenter Best Practice Award Winner

Stand-alone Server Pods

Environmentally Sustainable

Free Air-Cooling via Air-Side

Economization

50% less energy use vs. traditional

facilities

1 % water use vs. traditional facilities

5.4 MegaWatts today, 22.2 MW for future use

ISO 27001:2005 Accreditation

SAS 70 Type I and II Attestations

Windows Azure

Windows Azure

Applications

Windows Azure

AppFabric

SQL Azure

Applications

Others Mobile Desktop Server

The Windows Azure Platform

The Big Picture

Fabric

Storage

Config

Compute

Application

AppFabric

Windows Azure

Applications

Applications

SQL Azure

Others Mobile

Desktop

Server

Azure Building Blocks

Azure Compute Building Blocks

Windows Azure Roles

Your role class typically derives from RoleEntryPoint represents role

Do initialization in OnStart()

Do work in Run()

You call into RoleEnvironment to enumerate roles, endpoints, etc

You get called in response to changes in the number of instances in the service

Can force restart

Windows Azure Storage Building Blocks

Each storage account can hold up to 100 TB

Windows Azure Blobs Provide simple interface for storing named files

along with metadata for the file

Windows Azure Blob features

REST based API

ouraccount.blob.core.windows.net/thecontainer/

myblob

PutBlob, GetBlob, DeleteBlob, CopyBlob,

SnapshotBlob, LeaseBlob, etc

Block Blob for streaming + commit-based

writes

Page Blob for random read/write

100 Tb per storage account

Windows Azure Drives Page Blob formatted as a NTFS single volume

Virtual Hard Drive (VHD)

NTFS APIs providing durable file storage on Page

Blobs

Drives can be up to 1TB, up to 16 can be mounted

Can upload VHD via Page Blob then mount

Ease migration of existing Windows

applications to Azure

Durability and survival of data on application

failover or hardware failure

Windows Azure Tables Structured storage via entities, containing sets of

properties

Windows Azure Tables Provides Massively Scalable Structured Storage

Billions of entities (rows) and TBs of data

Can utilise thousands of servers as it scales up

Triple replicated for durability

Familiar and easy to use REST based API

Each entity can have up to 255 properties

PartitionKey & RowKey as key and index

Timestamp for optimistic concurrency

No fixed schema for all other properties

Property is stored as a <name, typed value> pair

Windows Azure Tables

Entity: Sports Car

MyTable1

Entity: Customer

Entity: Car

Table Partitions and Rows PartitionKey

(Category)

RowKey

(Title)

Timestamp ReleaseDate ………..

Action Clash of the Titans … 2010

Action The Bourne

Ultimatum

… 2007

… … … …

Animation Avatar … 2009

Animation How to Train Your

Dragon

… 2010

… … … …

Comedy What Happens in

Vegas

… 2008

… … … …

SciFi Alien 1979

… … … …

Horror Halloween … 1978

Windows Azure Queues Provide reliable storage and delivery of messages

Windows Azure Queues Provide reliable storage and delivery of

messages

Max size of message 8 Kb

Store URI to blog / table for larger data

No limit to number of messages in a queue

Messages must explicitly be deleted

Message expiration timeout

Message de-queue count for poison messages

Common Azure Pattern

Azure Features

Service Management API

Provide Lights-Out Service Management

Manage services programmatically via

REST-based API

X509 client certificates for authentication

Viewing, creating, deleting, swapping,

modifying configuration settings, etc on

deployments

Windows Azure Diagnostics

41

Cloud is much harder than single server

Dynamic environment, no local access

Azure Diagnostics built for monitoring & data

collection

Focus on what to collect and when

Designed for Azure

Based on standard APIs

Traces, logs, crash dumps, IIS logs,

perf counters, ETW, etc

Upload to Azure storage as required

Logging &

Diagnostics

Service Management CmdLets

PowerShell cmdlets wrapping Azure Service

Management and Diagnostics API

Simple to script out deployments, upgrades,

scaling

• Deploy new services

• Upgrade services

• Manage storage

accounts

• Transfer diagnostics

information

Service Upgrade Models Rolling upgrade (aka “In-place”)

Roles are updated across update domains

one at a time

Service remains available during upgrade

Real-time hot swap (aka “VIP Swap”)

Virtual IP swap between staging &

production

Swap service endpoints of old/new

version

Enables complex architectural changes

Planned downtime

Stop and replace service with new version

Content Delivery Network

Better performance and user experience by caching

Azure blobs at strategically placed locations

19 locations globally (United States, Europe, Asia,

Australia and South America) and growing

Windows Azure

Blob Service

pic1.jpg

Content Delivery Network Edge Location

Edge Location Edge Location pic1.jpg

The Fabric

Fault Domains

Azure Development

Windows Azure for Application Developers

Software Development Kit

Develop Run

Code

Data

Relational

Data

Portal

Deploy

API

Development Fabric and Storage

Visual Studio 2010

Visual Studio 2010

Visual Studio 2010

Azure Portal

SQL Azure

account servers

server

database

Account

Server

Datab

ase

Application

Internet

Azure Cloud

LB

TDS (tcp)

TDS (tcp)

TDS (tcp)

Applications use standard SQL

client libraries: ODBC, ADO.Net, …

Load balancer forwards ‘sticky’

sessions to TDS protocol tier

Gatewa

y

Gatewa

y

Gatewa

y

Gatewa

y

Gatewa

y

Gatewa

y

Scalability and Availability: Fabric, Failover, Replication, and Load balancing

SQL SQL SQL SQL SQL SQL

Gateway: TDS protocol gateway, enforces AUTHN/AUTHZ policy; proxy to backend

SQL

SQL Azure & SQL Server

Similarities

“Just change the connection string”

Same SQL Server core database service &

TDS

Differences

Supports subset of SQL Server 2008

Limited to 5Gb or 50Gb database

Different pricing model

No Reporting Services, Analysis Services, etc

SQL Azure Architectures

T-SQL (TDS)

Application / Tools

SQL Azure access from

outside the cloud

“Code Far”

SQL Azure

SOAP/REST

HTTP/S

SQL Azure access from

within the cloud

Windows Azure

T-SQL (TDS)

Web Role

SQL Azure

Application/ Browser

“Code Near”

Offline-Capable Cached Mode Applications

On-Premises Applications

Sync Group

SQL Azure

SQL Azure

SQL Azure

Remote Offices & Retail Stores

SQL Azure Roadmap

Service Update 2 Multiple Active Result Sets (MARS)

Service Update 3 50 Gb Databases, Spatial support

Incubation Project “Houston” – new SQL Azure web administration tool

Codename “Dallas” – information as a service

OData service on top of SQL Azure

OData “ODBC for the Internet”

• Uniform way of representing data, URL conventions and

operations

• Auto generation of OData for SQL Azure

Open Data

Protocol

OData Consumers

Session

OData Producer

Reports

Format

Metadata + +

Dashboard

Query&[Uri]

Forms

R/W

+

Apps

Batching, Versions

Browser

Codename “Dallas”

I need data!

Marketplace

Billing

Analytics

Se

rvic

es fo

r

Co

nte

nt P

rovid

ers

Standard Data

Access (OData)

Office,

Dynamics,

SQL Server

ISV Apps

Custom

Apps

Information as a Service

68

Data or functionality that is “of value to many” – enabling applications, reports, BI analysis, etc… Examples include GIS/Spatial, traffic, movie show times,

crime, real-estate sales, financial data, navigation, census data, reviews, etc…

Various Classifications: Commercial: clean, supported, and regularly updated from

ISVs and Content Providers

Trusted Public Domain: clean, unsupported data from academia and governments (Census, FDA, …)

Crowd Sourced: unreliable data in the public domain from anyone and everyone

http://apps.facebook.com/crimestats http://silverlight.onterrasys.com/CrimeDemo/

A quick way to explore “Dallas” datasets

Available for every dataset

Fill in parameter values using guided help

View results in various formats

Click “Analyze” to continue exploring in PowerPivot and Excel to create PivotTables, graphs, etc.

Learn about data format and perform quick queries

AppFabric

“Infrastructure to help build and manage applications more easily”

Windows Server AppFabric

Caching capabilities (“Velocity”)

Workflow + service hosting (“Dublin”)

Windows Azure Platform AppFabric

Service Bus

Access Control

Service Bus

75

Exposing internal applications on the

Internet isn’t easy

Network address translation (NAT) and

firewalls get in the way

The Service Bus:

Provides a cloud-based intermediary

between clients and internal applications

Provides a service registry that clients can

use to find the services they need

App #1: Web,

Mobile, Desktop,

LOB, RIA, ...

without altering network topology

App #2: Web,

Mobile,

Desktop, LOB,

RIA, ...

Send Send Receive

Service

Bus

App 1 App 2

Receive

Access Control

77

Different organizations identify users

through many different techniques

Applications can be faced with a confusing

mess of security related code

The Access Control Service:

Implements a security token service (STS) in the

cloud

It accepts one token and issues another

An administrator can define rules for how this

claims transformation is done

• Many identity providers, one code base

March ‘10 V-Next

Silverlight support Multicast with Message

Buffers

Advanced messaging features (ie. durability) BizTalk Server integration Windows Server AppFabric integration Web SSO and service federation - WS-Federation Considering additional identity providers

Windows Live ID, Open ID, Facebook Connect Management UI

Labs

Commercial Information

Windows Azure Platform Consumption Prices

$0.12/hour

+ Variable Instance Sizes

$9.99/month

(scale up to 5 GB)

$99.99/month

(scale up to 50 GB)

$0.15 GB/month

$0.01/10K transactions

Prices shown in USD only

$1.99/100k Message Operations

Per Message Operation

$3.99/connection

Per Connection

International prices are available

Windows Azure Instance Sizes

Unit of Compute Defined

$0.12 $0.24 $0.48 $0.96

1 x 1.6Ghz

2 x 1.6Ghz

4 x 1.6Ghz

8 x 1.6Ghz

1.75 GB memory 3.5 GB memory 7.0 GB memory 14 GB memory

250 GB storage (instance storage)

500 GB storage (instance storage)

1000 GB storage (instance storage)

2000 GB

(instance storage)

North America Region

S. Central - US

Sub-region

N. Central – US

Sub-region

Europe Region

W. Europe

Sub-region

S.E. Asia

Sub-region

Asia Pacific Region

E. Asia

Sub-region

N. Europe

Sub-region

Windows Azure Platform Data Transfer

On-board to Windows Azure platform at no charge Off peak times defined as: 10pm-6am Mon-Fri & from 10pm-Fri to 6am-Mon for

weekends in each designated regional time zones below

North America PST = UTC-8

Europe WET = UTC

Asia Pacific SST = UTC+8

Enabling customers to deliver content closest to end-users. Leads to better user

experience, global reach, & engagement

Development Accelerator

Biz Spark One

Investing in high potential

start-ups; receive full platform

services at no charge

MSDN Premium

Subscribers receive monthly

allocation of platform services

at no additional charge

Introductory Special

Anyone can try the platform

at no charge (monthly

service allocations)

UK Azure Pricing – Standard Rates Windows Azure

Compute

Small instance (default): £0.0728 per hour

Medium instance: £0.1455 per hour

Large instance: £0.291 per hour

Extra large instance: £0.5819 per hour

Storage

£0.091 per GB stored per month

£0.0061 per 10,000 storage transactions

Content Delivery Network (CDN)

£0.09 GB CDN Egress

£0.06/100K CDN Storage X-actions

SQL Azure

Web Edition – Up to 1 GB relational database

£6.055 per database per month

Business Edition – Up to 10 GB relational database

£60.604 per database per month

AppFabric

Access Control

£1.2062 per 100,000 transactions*

Service Bus

£2.4184 per connection on a “pay-as-you-go” basis*

Pack of 5 connections £6.0307*

Pack of 25 connections £30.1535*

Pack of 100 connections £120.6139*

Pack of 500 connections £603.0695*

Data Transfers

North America and Europe regions

£0.0607 per GB in

£0.091 per GB out

Asia Pacific Region

£0.1819 per GB in

£0.2728 per GB out

Inbound data transfers during off-peak times through June 30, 2010 are at no charge. Prices revert to our normal inbound data transfer rates after June 30, 2010

£100 per month* - Highly Available 2 x

Instance Web Application + 0.5TB data

£6 per month* - Highly Available 1Gb

SQL Server database

* Lots of assumptions and caveats

Some Final Thoughts….

© 2010 Microsoft Corporation. All rights reserved. Microsoft, Windows, Windows Vista and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries.

The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it

should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO

WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.