MSDN EventMSDN EventMSDN EventMSDN Event

WINDOWS AZURE STORAGEWINDOWS AZURE STORAGEWINDOWS AZURE STORAGEWINDOWS AZURE STORAGE

Windows Azure StorageWindows Azure StorageWindows Azure StorageWindows Azure Storage

• Storage in the Cloud– Scalable, durable, and available

– Anywhere at anytime access

– Only pay for what the service uses

• Exposed via RESTful Web Services– Use from Windows Azure Compute

– Use from anywhere on the internet

• Various storage abstractions– Tables, Blobs, Queues, Drives

The Storage Client APIThe Storage Client APIThe Storage Client APIThe Storage Client API

• The Main API are RESTfuls– Can call these from any HTTP client

e.g. Flash, Silverlight, etc…

• For easier .NET access, a Client API from SDK Microsoft.WindowsAzure.StorageClient– Provides a strongly typed wrapper around REST services

Windows Azure Storage AccountWindows Azure Storage AccountWindows Azure Storage AccountWindows Azure Storage Account• User specified globally unique account name

– Can choose geo-location to host storage account• US – “North Central” and “South Central”

• Europe – “North” and “West”

• Asia – “East” and “Southeast”

– Can CDN Enable Account• Blobs delivered via 18 global CDN nodes

– Can co-locate storage account with compute account• Explicitly or using affinity groups

• Accounts have two independent 512 bit shared secret keys

• 100TB per account

Storage in the Development FabricStorage in the Development FabricStorage in the Development FabricStorage in the Development Fabric

• Provides a local “Mock” storage

• Emulates storage in cloud

• Allows offline development

• Requires SQL Express 2005/2008 or above

There are some differences between Cloud and Dev Storage.There are some differences between Cloud and Dev Storage.http://msdn.microsoft.com/dd320275 http://msdn.microsoft.com/dd320275

A good approach for developers:A good approach for developers:To test pre-deployment, push storage to the cloud firstTo test pre-deployment, push storage to the cloud first

Use Dev Fabric for compute connect to cloud hosted storage.Use Dev Fabric for compute connect to cloud hosted storage.Finally, move compute to the cloud.Finally, move compute to the cloud.

Windows Azure Storage AbstractionsWindows Azure Storage Abstractions

Windows Azure Storage AccountWindows Azure Storage Account

Blob Storage ConceptsBlob Storage Concepts

BlobContainerAccount

cohowinerycohowinery

imagesimages

PIC01.JPGPIC01.JPG

PIC02.JPGPIC02.JPG

videosvideos VID1.AVIVID1.AVI

http://<account>.blob.core.windows.net/<container>/<blobname>http://<account>.blob.core.windows.net/<container>/<blobname>

Blob Features and FunctionsBlob Features and Functions

Blob Client LibraryBlob Client LibraryCloudStorageAccount account = CloudStorageAccount.FromConfigurationSetting("CloudStorageAccount");CloudBlobClient blobClient = new CloudBlobClient(

account.BlobEndpoint, account.Credentials); // Create ContainerCloudBlobContainer cloudContainer = blobClient.GetContainerReference(containerName);bool hasCreated = cloudContainer.CreateIfNotExist();

// Access Blob in the ContainerCloudBlob cloudBlob = cloudContainer.GetBlobReference(blobName);

//BlobRequestOptions has retry policy, timeout etc.BlobRequestOptions options = new BlobRequestOptions(); //Upload the local file to Blob servicecloudBlob.UploadFile(uploadFileName, options);

//Download to local file namecloudBlob.DownloadToFile(downloadFileName, options);

Two types of Blobs under the hoodTwo types of Blobs under the hood

Block BlobBlock BlobStreaming Workload w/ Random Reads + Committed Streaming Workload w/ Random Reads + Committed WritesWrites

10 GB Movie10 GB Movie

Windows Azure

Storage

Windows Azure

Storage

blobName = “blob.wmv”;PutBlock(blobName, blockId1, block1Bits);PutBlock(blobName, blockId2, block2Bits);

…………PutBlock(blobName, blockIdN, blockNBits);

PutBlockList(blobName, blockId1, blockId2…,blockIdN);

blobName = “blob.wmv”;PutBlock(blobName, blockId1, block1Bits);PutBlock(blobName, blockId2, block2Bits);

…………PutBlock(blobName, blockIdN, blockNBits);

PutBlockList(blobName, blockId1, blockId2…,blockIdN);

blob.wmvblob.wmvblob.wmv

Page Blob – Random Read/WritePage Blob – Random Read/Write

10 GB

10 GB

Address S

pace10 G

B A

ddress Space

512

1024

1536

2048

2560

Windows Azure Content Delivery NetworkWindows Azure Content Delivery Network

Blobs TipsBlobs Tips

Windows Azure DriveWindows Azure Drive

Windows Azure Drives CapabilitiesWindows Azure Drives Capabilities

Cloud Library SampleCloud Library Sample//Create Local Storage resource and initialize the local cache for drivesCloudDrive.InitializeCache(localCacheDir, cacheSizeInMB);

CloudStorageAccount account = CloudStorageAccount.FromConfigurationSetting("CloudStorageAccount");

//Create a cloud drive (PageBlob)CloudDrive drive = account.CreateCloudDrive(pageBlobUri);drive.Create(1000 /* sizeInMB */);

//Mount the network attached drive on the local file systemstring pathOnLocalFS = drive.Mount(cacheSizeInMB, DriveMountOptions.None);

//Use NTFS APIs to Read/Write files to drive…

//Snapshot drive while mounted to create backupsUri snapshotUri = drive.Snapshot();

//Unmount the drivedrive.Unmount();

Windows Azure TablesWindows Azure Tables

Table Storage ConceptsTable Storage Concepts

EntityTableAccount

cohowinery

customers

Name =…

Email = …

Name =…

Email = …

winephotos

Photo ID =…

Date =…

Photo ID =…

Date =…

Entity PropertiesEntity Properties

PartitionKey and PartitionsPartitionKey and Partitions

Purpose of the Partition KeyPurpose of the Partition Key

PartitionKey(Category)

RowKey(Title)

Timestamp ReleaseDate

Action Fast & Furious … 2009

Action The Bourne Ultimatum … 2007

… … … …

Animation Open Season 2 … 2009

Animation The Ant Bully … 2006

PartitionKey(Category)

RowKey(Title)

Timestamp ReleaseDate

Comedy Office Space … 1999

… … … …

SciFi X-Men Origins: Wolverine … 2009

… … … …

War Defiance … 2008

PartitionKey(Category)

RowKey(Title)

Timestamp ReleaseDate

Action Fast & Furious … 2009

Action The Bourne Ultimatum … 2007

… … … …

Animation Open Season 2 … 2009

Animation The Ant Bully … 2006

Comedy Office Space … 1999

… … … …

SciFi X-Men Origins: Wolverine … 2009

… … … …

War Defiance … 2008

Partitions and Partition Partitions and Partition RangesRanges

Server BTable = Movies

[Comedy - MaxKey)

Server ATable = Movies

[MinKey - Comedy)

Server ATable = Movies

Table OperationsTable Operations

Hello Azure ArchitectureHello Azure ArchitectureHello Azure ArchitectureHello Azure Architecture

Cloud

Hello Azure Web App

TwitterAggragator

Web Role

Web Role

Twitter

Aggregator Instance 1

Client Client

Client

Aggregator Instance 0

TwitterDispathcer

Dispatcher Instance 0

Dispatcher Instance 1

Table Storage

Queue

DEMODEMODEMODEMO

Twitter Table

Table TipsTable Tips

Table Tips – cont.Table Tips – cont.Avoid large table scans when performance is critical

Restructure your schema if required

Concatenate different keys to form appropriate index

Most Optimal:

PartitionKey == “SciFi” and RowKey == “Star Wars”

Scans: Expect continuation tokens (REST)

PartitionKey == “SciFi” and “Sphere” ≤ RowKey ≤ “Star Wars”

“Action” ≤ PartitionKey ≤ “Thriller”

PartitionKey == “Action” || PartitionKey == “Thriller” - currently scans entire table

“Cars” ≤ RowKey ≤ “Star Wars” - scans entire table

Queue Storage ConceptsQueue Storage Concepts

MessageQueueAccount

order processing

customer ID order ID http://…

customer ID order ID http://…

cohowinery

Loosely Coupled interaction with QueuesLoosely Coupled interaction with Queues

Azure Queue

Input Queue (Work Items)

Queue’s Reliable DeliveryQueue’s Reliable Delivery Azure QueueInput Queue (Work Items)

DEMODEMODEMODEMO

Twitter Queue

Queue TipsQueue Tips