Transcript
Page 1: Building services using windows azure

Training Workshop

Windows Azure Platform

Page 2: Building services using windows azure

3

Building Services using Windows Azure

NameTitleOrganizationEmail

Page 3: Building services using windows azure

4

Windows Azure

Windows Azure is the foundation of Microsoft’s Cloud PlatformIt is an “Operating System in the Cloud” and provides Essential Services for the cloud

Virtualized ComputationScalable StorageAutomatic ManagementDeveloper SDK

Page 4: Building services using windows azure

5

Role Programming Model

Inherits RoleEntryPointOnStart() Method

Called by Fabric on startup, allows you to perform initialization tasks.Reports Busy status to load balancer until you return true.

OnStop() MethodCalled when role is to be shutdown, graceful exit.

Run() MethodMain logic is here – can do anything, typically loop and never exit.

Page 5: Building services using windows azure

6

Web Role

ASP.NET 3.5 SP1 – 64bitIIS7 Hostable Web CoreHosts

Webforms or MVCFastCGI applications (e.g. PHP)

Http(s)/TCPWebRole

New web.roleConfig to support FastCGISupport same Role model semantics as Worker Role (e.g. OnStart, Run, etc.)

Page 6: Building services using windows azure

7

Packaging & Deployment

ServicePackage

ServiceConfiguration

Page 7: Building services using windows azure

8

Building and Deploying a Service

demo

Page 8: Building services using windows azure

9

Configuration

Service ConfigurationServiceconfiguration.csdef – Service ModelServiceConfiguration.cscfg – instance data

RoleEnvironment.GetConfigurationSettingValue()

Don’t use web.config for values you wish to change at runtime

Web.config change requires re-deploy

Page 9: Building services using windows azure

10

MonitoringNo Debugging in CloudInstrument your application using Trace, Debug

DiagnosticMonitorTraceListenerUse Diagnostics API to Configure and Collect

Event LogsPerformance CountersTrace/Debug information (logging)IIS Logs, Failed Request LogsCrash Dumps or Arbitrary files

Request data on demand or scheduledTransferred into your table and/or blob storage

Everything is remotely configurable

Page 10: Building services using windows azure

11

Logging and Configuration

demo

Page 11: Building services using windows azure

12

Using Queues to Decouple Roles

Use Queue to DecoupleDesign for Idempotency

multiple applications of the operation does not change the result

Web Role Worker Role

StorageQueue

LB LB

Worker RoleWorker RoleWeb RoleWorker Role

Page 12: Building services using windows azure

13

Using the Worker Roles

demo

Page 13: Building services using windows azure

14

Upgrading Your Application

Two Models: VIP Swap and In-Place UpgradeVIP Swap:

Uses Staging and Production environments.Allows to quickly swap environments.Production: v1 Staging: v2, after swap then Production: v2 Staging: v1.

In-Place UpgradePerforms a rolling upgrade on live service.Entire service or a single roleManual or Automatic across update domains

Page 14: Building services using windows azure

15

Storage

Blobs – Provide a simple interface for storing named files along with metadata for the file Drives – A durable NTFS volume backed by blob storageTables – Provide structured storage; A Table is a set of entities, which contain a set of propertiesQueues – Provide reliable storage and delivery of messages for an application

Page 15: Building services using windows azure

16

Storage in the Dev Fabric

Provides a local “Mock” storageEmulates storage in cloudAllows offline developmentRequires SQL Express 2005/2008

Page 16: Building services using windows azure

17

Storage in the cloud

Create a Storage AccountEndpointsAccess KeysCDN

Page 17: Building services using windows azure

18

Windows Azure Storage Account

User creates a globally unique storage account name

Can choose geo-location to host storage account“US Anywhere”, “US North Central”, “US South Central”,

Can co-locate storage account with compute accountReceive a 256 bit secret key when creating account

Storage Account Capacity at Commercial Availability

Each storage account can store up to 100 TB Default limit of 5 storage accounts per subscription

Page 18: Building services using windows azure

19

Blob Containers

Number of Blob ContainersCan have has many Blob Containers that will fit within the storage account limit

Blob ContainerA container holds a set of blobsSet access policies at the container level

Private or Public accessibleAssociate Metadata with Container

Metadata are <name, value> pairsUp to 8KB per container

List the blobs in a containerCan now include Blob Metadata and MD5 (new)

Page 19: Building services using windows azure

20

Blob Features and Functions

Store Large Objects (100s of GBs in size)

Associate Metadata with BlobMetadata is <name, value> pairs, Up to 8KB per blobSet/Get with or separate from blob data bits

Standard REST InterfacePutBlob

Inserts a new blob, overwrites the existing blobGetBlob

Get whole blob or a specific rangeDeleteBlobCopyBlob (new)SnapshotBlob (new)LeaseBlob (new)

Page 20: Building services using windows azure

21

Blocks or Pages

Blob Storage ConceptsKey concepts account, container, blob and blocks/pages

BlobContainerAccount

Account

Pictures

IMG001.JPG

IMG002.JPG

Movies MOV1.AVI

Block/Page 1

Block/Page 2

Block /Page 3

Page 21: Building services using windows azure

22

Two Types of Blobs Under the Hood

Block Blob Targeted at streaming workloadsEach blob consists of a sequence of blocks

Each block is identified by a Block IDSize limit 200GB per blob

Page Blob (new)Targeted at random read/write workloadsEach blob consists of an array of pages

Each page is identified by its offset from the start of the blobSize limit 1TB per blob

Page 22: Building services using windows azure

23

Block Blob Details

Block can be up to 4MB eachEach block can be variable sizeEach block has a 64 byte ID

Scoped by blob name and stored with the blob

Block operationPutBlock

Puts an uncommitted block defined by the block ID for the blob

Block List OperationsPutBlockList

Provide the list of blocks to comprise the readable version of the blobCan use blocks from uncommitted or committed list to update blob

GetBlockListReturns the list of blocks, committed or uncommitted for a blob

Block ID and Size of Block is returned for each block

Page 23: Building services using windows azure

24

Page Blob DetailsPage Blob is created with a Max Blob Size

Can change the max size of the blob at anytime.

Address space is broken up into fixed sized 512 byte pages for updates

Page update operations – must be page alignedPutPage - limited to 4MB

Overwrite range of pages starting at the specified offsetClearPage - can specify up to max size of blob

Clear range of pages at the offset

Reading a Page BlobGetBlob – can read from any byte offset for any valid range

What parts of the Page Blob have stored pages in themGetPageRange

Get valid page ranges in the blob

Only charged for pages with data stored in them

Page 24: Building services using windows azure

25

Choosing Between Block and Page Blob

Block BlobTargeted at streaming workloadsUpdate semantics

Upload a bunch of blocks. Then commit change.Concurrency: ETag Checks

Page BlobTargeted at random read/write workloadsUpdate Semantics

Immediate updateConcurrency: Leases

Page 25: Building services using windows azure

26

Using Blobs

demo

Page 26: Building services using windows azure

27

Summary Of Windows Azure Blobs

Blob Access PatternsBlock Blobs – streamingPage Blobs – random read/write (new)

New Blob OperationsCopy, Snapshot, and Lease work for both types

New Ways of Accessing and Serving Blob Content (see MSDN)

Content Delivery Network accessCustom Domain NamesRoot Blob ContainerShared Access Signatures

Additional new featuresListBlob, GetBlob, Blob Properties (see MSDN docs)

Page 27: Building services using windows azure

28

Queues

Simple asynchronous dispatch queueCreate and delete queues

Message:Retrieved at least onceMax size 8kbOperations:

putgetdelete

Page 28: Building services using windows azure

29

Queue Storage ConceptsAccount, queue and message

MessageQueueAccount

Account

Thumbnail Jobs

128x128, http://…

256x256, http://…

Indexing Jobs

http://…

http://…

Page 29: Building services using windows azure

30

Using Queues

demo

Page 30: Building services using windows azure

31

Summary Of Queues

Provide reliable message deliveryAllows Messages to be retrieved and processed at least once

No limit on number of messages stored in a Queue

Message size is <=8KB

Page 31: Building services using windows azure

32

Tables

Entities and properties (rows & columns)Tables scoped by accountDesigned for billions+Scale-out using partitions

Partition key & row keyOperations performed on partitionsEfficient queriesNo limit on number of partitions

Use ADO.NET Data Services

Page 32: Building services using windows azure

33

Table Storage ConceptsAccount, table and entity

EntityTableAccount

Account

Users

Name=…hash=…

Name=…hash=…

PhotoIndex

Tag=…id=…

Tag=…,id=…

Page 33: Building services using windows azure

34

Entities and Properties

Each Entity can have up to 255 propertiesEvery Entity has fixed key properties

Partition keyRow keyTimestamp

No fixed schema for rest of properties2 entities in the same table can have different propertiesProperties stored as <Name, TypedValue> pairs

Each entity has a system maintained version

Page 34: Building services using windows azure

35

Property Types

Partition key and Row keyString (up to 1KB)

Other propertiesString (up to 64KB)Binary (up to 64KB)BoolDateTime GUIDIntInt64Double

Page 35: Building services using windows azure

36

Partition Key And Partition

Every Entity has a partition keyAll entities in a table with the same partition key value live in the same partition

Need to choose partitioning scheme to make data access scalable

Page 36: Building services using windows azure

37

Partitioning Guidelines

PerformanceUse a PartitionKey that is common in your queries

Always try to specify the partition key in the queryEntities with same partition key value are clusteredBatch capabilities and transaction are supported within a partition only.

ScalabilityWe monitor partition trafficAutomatically load balance partitions

Each partition can potentially be served by a different storage nodeScale to meet the traffic needs of your application

More partitions – makes it easier to balance load

Page 37: Building services using windows azure

38

Using Tables

demo

Page 38: Building services using windows azure

Client A ClientB

5 : Ch9, Jan-1, 3

1 : Ch9, Jan-2, 21 : Ch9, Jan-2, 21 : Ch9, Jan-2, 22: Ch9, Jan-2, 5

Use standard HTTP mechanisms – Etag and If-MatchGet entity – get system maintained version as ETagUpdate Entities Locally – change ratingSend Update with version check - IF-Match with EtagSuccess if version matches, and update version on Client-APrecondition failed (412) if version does not match

Concurrent Updates

9 : Ch9, Jan-3, 6

If-Match: 1 Ch9, Jan-2, 5If-Match: 1 Ch9, Jan-2, 4

Version Rating 1: Ch9, Jan-2, 41: Ch9, Jan-2, 5Error: 412

2: Ch9, Jan-2, 5

Page 39: Building services using windows azure

.NET: LINQ Take(N) function

Getting the Top N entities

serviceUri = new Uri("http://<account>.table.core.windows.net");DataServiceContext context = new DataServiceContext(serviceUri);

var allMessages = context.CreateQuery<Message>("Messages");foreach (Message message in allMessages.Take(100)){ Console.WriteLine(message.Name);}

GET http://<serviceUri>/Messages?$top=100

REST: $top=N query string option

Page 40: Building services using windows azure

41

GET http://<Uri>/Messages?$filter=...&$top=100

&NextPartitionKey=xxxxxxx &NextRowKey=yyyyyy

Pagination – Continuation Tokens

Send a requestGET http://<serviceUri>/Messages?$filter=...&$top=100

Set HTTP query parameters

Get continuation token in response headerx-ms-continuation-NextPartitionKey: xxxxxxxx-ms-continuation-NextRowKey: yyyyyy

Messages

Ch9, Date1,

Ch9, Date2,

Ch9, …

Ch9,Date100,

Ch9,Date101,

Ch9, …

100

Page 41: Building services using windows azure

42

Single Table Consistency

ACID transactions for single entity CUDInsert/update/delete

Snapshot isolation for query within a single partition

Consistent view from start time of the queryNo dirty (uncommitted) readsDoes not block concurrent updates

No snapshot isolation across partitionsNo snapshot isolation across different continuations of a queryBatch transactions within partition only

Page 42: Building services using windows azure

43

Cross Table Consistency

Application is responsible for maintaining consistency

ExampleWhen a channel is deleted, delete all the messages for that channel

Failures can occur in the middleExample - Application fails after deleting some messagesUse Windows Azure Queues to help ensure completion of operation

Page 43: Building services using windows azure

44

1. Dequeue DelCh12. Delete Ch1 from Channels3. Delete from Messages4. Delete queue entry

Messages

Channels

QueueWorker

Cross Table Consistency

Del

Ch1

2

Delete channel Delete messages workerFront end

Front End

Del

Ch5

Del

Ch

11Ch1, Msg1

Ch1, Msg2

Ch1, Msg3

Ch2, Msg1

Ch2, Msg2

Ch3, Msg1Ch1,…

Ch2,…

Del

Ch1

Page 44: Building services using windows azure

45

1. Dequeue DelCh1 and start delete2. Fails after deleting Ch1 and Msg13. DelCh1 is visible again4. Dequeue DelCh1 again5. Repeat delete operations

Messages

Channels

QueueWorker 1

Resuming After Failure

Del

Ch1

2

Delete channel Delete messages workerFront end

Front End

Del

Ch5

Del

Ch

11Ch1, Msg1

Ch1, Msg2

Ch1, Msg3

Ch2, Msg1

Ch2, Msg2

Ch3, Msg1Ch1,…

Ch2,…

Worker2

Del

Ch1

Del

Ch

1h

Page 45: Building services using windows azure

46

Table Summary

Windows Azure tables areMassively ScalableHighly Available

Simple familiar APIUse .NET –- ADO.NET Data Services and LINQOr use RESTLeverage your .NET expertise

Page 46: Building services using windows azure

47

Learning Windows Azure

www.windowsazure.comhttp://channel9.msdn.com/learnDownload the SDK

You don’t need cloud access to develop!Look at the samples in the SDKWindows Azure Platform Training Kit

3 Windows Azure labsFollow the team bloggers

Page 47: Building services using windows azure

48

Q & A

Page 48: Building services using windows azure

49

© 2008 Microsoft Corporation. All rights reserved. Microsoft, Windows, Windows Vista and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries.The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED

OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.