Training Workshop
Windows Azure Platform
3
Building Services using Windows Azure
NameTitleOrganizationEmail
4
Windows Azure
Windows Azure is the foundation of Microsoft’s Cloud PlatformIt is an “Operating System in the Cloud” and provides Essential Services for the cloud
Virtualized ComputationScalable StorageAutomatic ManagementDeveloper SDK
5
Role Programming Model
Inherits RoleEntryPointOnStart() Method
Called by Fabric on startup, allows you to perform initialization tasks.Reports Busy status to load balancer until you return true.
OnStop() MethodCalled when role is to be shutdown, graceful exit.
Run() MethodMain logic is here – can do anything, typically loop and never exit.
6
Web Role
ASP.NET 3.5 SP1 – 64bitIIS7 Hostable Web CoreHosts
Webforms or MVCFastCGI applications (e.g. PHP)
Http(s)/TCPWebRole
New web.roleConfig to support FastCGISupport same Role model semantics as Worker Role (e.g. OnStart, Run, etc.)
7
Packaging & Deployment
ServicePackage
ServiceConfiguration
8
Building and Deploying a Service
demo
9
Configuration
Service ConfigurationServiceconfiguration.csdef – Service ModelServiceConfiguration.cscfg – instance data
RoleEnvironment.GetConfigurationSettingValue()
Don’t use web.config for values you wish to change at runtime
Web.config change requires re-deploy
10
MonitoringNo Debugging in CloudInstrument your application using Trace, Debug
DiagnosticMonitorTraceListenerUse Diagnostics API to Configure and Collect
Event LogsPerformance CountersTrace/Debug information (logging)IIS Logs, Failed Request LogsCrash Dumps or Arbitrary files
Request data on demand or scheduledTransferred into your table and/or blob storage
Everything is remotely configurable
11
Logging and Configuration
demo
12
Using Queues to Decouple Roles
Use Queue to DecoupleDesign for Idempotency
multiple applications of the operation does not change the result
Web Role Worker Role
StorageQueue
LB LB
Worker RoleWorker RoleWeb RoleWorker Role
13
Using the Worker Roles
demo
14
Upgrading Your Application
Two Models: VIP Swap and In-Place UpgradeVIP Swap:
Uses Staging and Production environments.Allows to quickly swap environments.Production: v1 Staging: v2, after swap then Production: v2 Staging: v1.
In-Place UpgradePerforms a rolling upgrade on live service.Entire service or a single roleManual or Automatic across update domains
15
Storage
Blobs – Provide a simple interface for storing named files along with metadata for the file Drives – A durable NTFS volume backed by blob storageTables – Provide structured storage; A Table is a set of entities, which contain a set of propertiesQueues – Provide reliable storage and delivery of messages for an application
16
Storage in the Dev Fabric
Provides a local “Mock” storageEmulates storage in cloudAllows offline developmentRequires SQL Express 2005/2008
17
Storage in the cloud
Create a Storage AccountEndpointsAccess KeysCDN
18
Windows Azure Storage Account
User creates a globally unique storage account name
Can choose geo-location to host storage account“US Anywhere”, “US North Central”, “US South Central”,
Can co-locate storage account with compute accountReceive a 256 bit secret key when creating account
Storage Account Capacity at Commercial Availability
Each storage account can store up to 100 TB Default limit of 5 storage accounts per subscription
19
Blob Containers
Number of Blob ContainersCan have has many Blob Containers that will fit within the storage account limit
Blob ContainerA container holds a set of blobsSet access policies at the container level
Private or Public accessibleAssociate Metadata with Container
Metadata are <name, value> pairsUp to 8KB per container
List the blobs in a containerCan now include Blob Metadata and MD5 (new)
20
Blob Features and Functions
Store Large Objects (100s of GBs in size)
Associate Metadata with BlobMetadata is <name, value> pairs, Up to 8KB per blobSet/Get with or separate from blob data bits
Standard REST InterfacePutBlob
Inserts a new blob, overwrites the existing blobGetBlob
Get whole blob or a specific rangeDeleteBlobCopyBlob (new)SnapshotBlob (new)LeaseBlob (new)
21
Blocks or Pages
Blob Storage ConceptsKey concepts account, container, blob and blocks/pages
BlobContainerAccount
Account
Pictures
IMG001.JPG
IMG002.JPG
Movies MOV1.AVI
Block/Page 1
Block/Page 2
Block /Page 3
22
Two Types of Blobs Under the Hood
Block Blob Targeted at streaming workloadsEach blob consists of a sequence of blocks
Each block is identified by a Block IDSize limit 200GB per blob
Page Blob (new)Targeted at random read/write workloadsEach blob consists of an array of pages
Each page is identified by its offset from the start of the blobSize limit 1TB per blob
23
Block Blob Details
Block can be up to 4MB eachEach block can be variable sizeEach block has a 64 byte ID
Scoped by blob name and stored with the blob
Block operationPutBlock
Puts an uncommitted block defined by the block ID for the blob
Block List OperationsPutBlockList
Provide the list of blocks to comprise the readable version of the blobCan use blocks from uncommitted or committed list to update blob
GetBlockListReturns the list of blocks, committed or uncommitted for a blob
Block ID and Size of Block is returned for each block
24
Page Blob DetailsPage Blob is created with a Max Blob Size
Can change the max size of the blob at anytime.
Address space is broken up into fixed sized 512 byte pages for updates
Page update operations – must be page alignedPutPage - limited to 4MB
Overwrite range of pages starting at the specified offsetClearPage - can specify up to max size of blob
Clear range of pages at the offset
Reading a Page BlobGetBlob – can read from any byte offset for any valid range
What parts of the Page Blob have stored pages in themGetPageRange
Get valid page ranges in the blob
Only charged for pages with data stored in them
25
Choosing Between Block and Page Blob
Block BlobTargeted at streaming workloadsUpdate semantics
Upload a bunch of blocks. Then commit change.Concurrency: ETag Checks
Page BlobTargeted at random read/write workloadsUpdate Semantics
Immediate updateConcurrency: Leases
26
Using Blobs
demo
27
Summary Of Windows Azure Blobs
Blob Access PatternsBlock Blobs – streamingPage Blobs – random read/write (new)
New Blob OperationsCopy, Snapshot, and Lease work for both types
New Ways of Accessing and Serving Blob Content (see MSDN)
Content Delivery Network accessCustom Domain NamesRoot Blob ContainerShared Access Signatures
Additional new featuresListBlob, GetBlob, Blob Properties (see MSDN docs)
28
Queues
Simple asynchronous dispatch queueCreate and delete queues
Message:Retrieved at least onceMax size 8kbOperations:
putgetdelete
29
Queue Storage ConceptsAccount, queue and message
MessageQueueAccount
Account
Thumbnail Jobs
128x128, http://…
256x256, http://…
Indexing Jobs
http://…
http://…
30
Using Queues
demo
31
Summary Of Queues
Provide reliable message deliveryAllows Messages to be retrieved and processed at least once
No limit on number of messages stored in a Queue
Message size is <=8KB
32
Tables
Entities and properties (rows & columns)Tables scoped by accountDesigned for billions+Scale-out using partitions
Partition key & row keyOperations performed on partitionsEfficient queriesNo limit on number of partitions
Use ADO.NET Data Services
33
Table Storage ConceptsAccount, table and entity
EntityTableAccount
Account
Users
Name=…hash=…
Name=…hash=…
PhotoIndex
Tag=…id=…
Tag=…,id=…
34
Entities and Properties
Each Entity can have up to 255 propertiesEvery Entity has fixed key properties
Partition keyRow keyTimestamp
No fixed schema for rest of properties2 entities in the same table can have different propertiesProperties stored as <Name, TypedValue> pairs
Each entity has a system maintained version
35
Property Types
Partition key and Row keyString (up to 1KB)
Other propertiesString (up to 64KB)Binary (up to 64KB)BoolDateTime GUIDIntInt64Double
36
Partition Key And Partition
Every Entity has a partition keyAll entities in a table with the same partition key value live in the same partition
Need to choose partitioning scheme to make data access scalable
37
Partitioning Guidelines
PerformanceUse a PartitionKey that is common in your queries
Always try to specify the partition key in the queryEntities with same partition key value are clusteredBatch capabilities and transaction are supported within a partition only.
ScalabilityWe monitor partition trafficAutomatically load balance partitions
Each partition can potentially be served by a different storage nodeScale to meet the traffic needs of your application
More partitions – makes it easier to balance load
38
Using Tables
demo
Client A ClientB
5 : Ch9, Jan-1, 3
1 : Ch9, Jan-2, 21 : Ch9, Jan-2, 21 : Ch9, Jan-2, 22: Ch9, Jan-2, 5
Use standard HTTP mechanisms – Etag and If-MatchGet entity – get system maintained version as ETagUpdate Entities Locally – change ratingSend Update with version check - IF-Match with EtagSuccess if version matches, and update version on Client-APrecondition failed (412) if version does not match
Concurrent Updates
9 : Ch9, Jan-3, 6
If-Match: 1 Ch9, Jan-2, 5If-Match: 1 Ch9, Jan-2, 4
Version Rating 1: Ch9, Jan-2, 41: Ch9, Jan-2, 5Error: 412
2: Ch9, Jan-2, 5
.NET: LINQ Take(N) function
Getting the Top N entities
serviceUri = new Uri("http://<account>.table.core.windows.net");DataServiceContext context = new DataServiceContext(serviceUri);
var allMessages = context.CreateQuery<Message>("Messages");foreach (Message message in allMessages.Take(100)){ Console.WriteLine(message.Name);}
GET http://<serviceUri>/Messages?$top=100
REST: $top=N query string option
41
GET http://<Uri>/Messages?$filter=...&$top=100
&NextPartitionKey=xxxxxxx &NextRowKey=yyyyyy
Pagination – Continuation Tokens
Send a requestGET http://<serviceUri>/Messages?$filter=...&$top=100
Set HTTP query parameters
Get continuation token in response headerx-ms-continuation-NextPartitionKey: xxxxxxxx-ms-continuation-NextRowKey: yyyyyy
Messages
Ch9, Date1,
Ch9, Date2,
Ch9, …
Ch9,Date100,
Ch9,Date101,
Ch9, …
100
42
Single Table Consistency
ACID transactions for single entity CUDInsert/update/delete
Snapshot isolation for query within a single partition
Consistent view from start time of the queryNo dirty (uncommitted) readsDoes not block concurrent updates
No snapshot isolation across partitionsNo snapshot isolation across different continuations of a queryBatch transactions within partition only
43
Cross Table Consistency
Application is responsible for maintaining consistency
ExampleWhen a channel is deleted, delete all the messages for that channel
Failures can occur in the middleExample - Application fails after deleting some messagesUse Windows Azure Queues to help ensure completion of operation
44
1. Dequeue DelCh12. Delete Ch1 from Channels3. Delete from Messages4. Delete queue entry
Messages
Channels
QueueWorker
Cross Table Consistency
Del
Ch1
2
Delete channel Delete messages workerFront end
Front End
Del
Ch5
Del
Ch
11Ch1, Msg1
Ch1, Msg2
Ch1, Msg3
Ch2, Msg1
Ch2, Msg2
Ch3, Msg1Ch1,…
Ch2,…
Del
Ch1
45
1. Dequeue DelCh1 and start delete2. Fails after deleting Ch1 and Msg13. DelCh1 is visible again4. Dequeue DelCh1 again5. Repeat delete operations
Messages
Channels
QueueWorker 1
Resuming After Failure
Del
Ch1
2
Delete channel Delete messages workerFront end
Front End
Del
Ch5
Del
Ch
11Ch1, Msg1
Ch1, Msg2
Ch1, Msg3
Ch2, Msg1
Ch2, Msg2
Ch3, Msg1Ch1,…
Ch2,…
Worker2
Del
Ch1
Del
Ch
1h
46
Table Summary
Windows Azure tables areMassively ScalableHighly Available
Simple familiar APIUse .NET –- ADO.NET Data Services and LINQOr use RESTLeverage your .NET expertise
47
Learning Windows Azure
www.windowsazure.comhttp://channel9.msdn.com/learnDownload the SDK
You don’t need cloud access to develop!Look at the samples in the SDKWindows Azure Platform Training Kit
3 Windows Azure labsFollow the team bloggers
48
Q & A
49
© 2008 Microsoft Corporation. All rights reserved. Microsoft, Windows, Windows Vista and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries.The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED
OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.