1

Windows AzureLARISA KOCSIS

PRIYA RAGUPATHY

Windows Azure 2

Windows Azure Microsoft’s cloud computing platform

Operating system for the cloud

Provides three Core services: ◦ Compute ◦ Storage◦ Management

3

High level Architecture

High level Architecture 4

High level Architecture

High level Architecture 5

High level Architecture

High level Architecture 6

High level Architecture

High level Architecture 7

WAS 8

Windows Azure Storage

WAS 9

Windows Azure Storage (WAS) Offers scalable disk allocation

Responsibility of Distributed file system

It implement different data abstraction ◦ Tables◦ Blobs◦ Queues

10

Logical components of WAS Storage Stamps

◦ Cluster of racks ~ 10 to 20 ◦ Each rack contains 18 storage node◦ Three Layers

◦ Front End Layer ◦ Partition Layer◦ Stream Layer

WAS

WAS 11

Logical components of WAS Location Service

◦ Manages the Storage Stamps◦ Data replication◦ Load balancing◦ Ensures the VIP is mapped to

AccountName

WAS 12

Intra-Stamp and Inter-Stamp Replication

Table service 13

The Table Storage Service• Structured Data• NoSQL• Quick Query

Table Service 14

COMPONENTS OF TABLE SERVICE

15

Component: Storage Account

TABLE SERVICE

16

Components: Tables

TABLE SERVICE

17

Components: Entity

TABLE SERVICE

TABLEs 18

How client can Access these table?Access to windows Storage Account via REST APIhttp://<storageaccount>.table.core.windows.net/<table>

Operations

Table Operations:

Create Table (POST), Delete Table (DELETE), Query (GET)

Entity Operation:

Query (GET), update (PUT), merge (MERGE), insert (POST), DELETE

Table service 19

Azure Storage Explorer

20

Related Research: Performance of Azure Table

◦ Maximum Throughput in Operation/ second

◦ Operations from Azure Table API: Insert, Query, Update, and Delete

◦ Different entity sizes: 1 KB, 4 KB, 16 KB and 64 KB

◦ Concurrent Clients : 1 to 192

TABLE SERVICE

Table service 21

Findings

22

Challenge

TABLE SERVICE

Windows Azure 23

The Blob Service Provides scalable and highly available storage for any kind of content (documents,

social data, backups of files/computers/databases/devices, images and text for web applications, configuration data for cloud applications, big data)

Distributes blob files across multiple servers and replicates them at least three times.

Every blob is organized into a container. A storage account can contain any number of containers, and a container can have any number of blobs.

Windows Azure 24

The Blob ArchitectureHierarchical structure:

account

containers

blobs

blocks and pages

Users’ Windows Azure storage account is the entry point to the Blob service via the REST API.

Windows Azure 25

Containers Containers are logical groupings for sets of blobs. Can have metadata in the form of name-value pairs. They can be created as public or private. Blob service is the only storage type that supports public and private access.

URI scheme to access a container:

<http|https>://<account name>.blob.core.windows.net/<container>

Windows Azure 26

Blobs Are the actual entities in the Blob service, stored in containers. A blob name must be unique within the scope of a container. A blob can also have metadata in the form of name-value pairs. The Access Control List (ACL) is set only at the container level, so all the blobs in a

public container are visible to everyone for read-only access.

URI scheme to access a blob:

<http|https>://<accountname>.blob.core.windows.net/<container>/<blob>

Windows Azure 27

Block Blobs vs. Page Blobs Block blobs are optimized for

streaming and storing cloud objects. Are a good choice for storing

documents, media files, backups etc. A block blob can be up to 200 GB in

size.

Page blobs are optimized for representing IaaS disks and supporting random writes.

An Azure virtual machine disk is a VHD stored as a page blob.

May be up to 1 TB in size.

Windows Azure 28

Block Blobs If a file is more than 64MB in size, it can't be uploaded to the Blob service using

the PUT blob function. Large blob files must be broken into contiguous blocks and uploaded in the form

blocks of maximum of 4MB each. Uploading and committing blocks to a blob are two separate operations; upload

can take place in any sequence or in parallel, but the commit sequence represents the readable blob.

After all the blocks are uploaded, they can be committed to a blob. After blocks are committed to a blob, they can only be retrieved as a complete

blob; the GET operation is only executed at the blob level.

Windows Azure 29

Block Blobs

Windows Azure 30

Page Blobs Page blobs are collections of 512-byte pages. A page is represented by its start offset from the start of the blob. Writes to page blobs are immediately committed to the blob storage. The Windows Azure Storage Client API provides two specific operations on page

blobs: Put Page and Get Page Regions.

Windows Azure 31

Blob service performanceThe blob service is scalable.

For an increasing #workers:

download time increases

download throughput increases

upload time decreases

upload throughput increases

Sequential blockwise download is faster than random pagewise download.

Windows Azure 32

Limitations of the Blob Service Files can only be stored in a flat structure; the Blob API provides filtering

capabilities based on a delimiter to retrieve only the log files in a particular virtual structure.

Containers do not allow nesting; a virtual folder structure is created by adding a delimiter to a blob's name.

Applications need to parse the tree structure for visual representation. Any blocks that remain uncommitted due to application errors or network failures

are garbage-collected after seven days of inactivity.

Windows Azure 33

The Queue Service Provides a highly scalable and available messaging solution for asynchronous

communication between application components. A queue can contain any number of messages. Individual messages may be up to 64 KB in size. Messages aren't guaranteed to follow the First In First Out (FIFO) pattern. Provides a REST API for applications to use the large-scale Queue service

infrastructure.

Windows Azure 34

The Queue ArchitectureThree-level hierarchy:

account

queues

messages

Users’ Windows Azure storage account is the entry point to the Queue service via the REST API.

Windows Azure 35

Queues Are a logical destination for sending messages. A queue stores messages and makes them available to applications via the REST

API. Queues can have metadata in the form of name-value pairs up to 8KB in size per

queue. The Queue service support only private access.

URI scheme to access a queue:

<http|https>://<account name>.queue.core.windows.net/<queue name>

Windows Azure 36

Messages An unlimited number of messages that can be stored in a queue, but the size of

each individual message can't exceed 64KB. To communicate large object messages, large objects can be put in a Blob and

then the URI of that object can be sent as a message to a queue. Sent messages can be in either text or binary format; but when a message is

received from the queue, it's always in Base64-encoded format. A GUID MessageID assigned by the Queue service uniquely identifies a message

in the queue.

Windows Azure 37

MessagesA message has the following attributes: MessageID = unique message identifier PopReceipt = attribute of the message when deleted/retrieved Visibilitytimeout = int value, 30 seconds by default Messagettl = time-to-live value in seconds

URI scheme to access messages:

<http|https>://<account name>.queue.core.windows.net/<queue name>/messages

Windows Azure 38

Queue serviceperformance

Scenario 1 (on left):

each worker accesses its own dedicated queue

Scenario 2 (on right):

all workers access the same queue

Windows Azure Queue storage scales with both scenarios.

Windows Azure 39

Limitations of the Queue Service Individual messages can't be more than 64KB in size. Larger objects can be stored

in Blob or Table storage and their URI sent as a queue message. The FIFO behavior of the messages sent to the Queue service isn't guaranteed,

messages can be received in any order. The Queue service doesn't offer guaranteed-once delivery; a message may be

received more than once. Messages format varies between sent and received messages. The expiration time for messages stored in the Queue service is seven days, after

which the messages are garbage-collected

Windows Azure 40

Conclusion Windows Azure is scalable and highly available (99.95%).

Standard accounts can store up to 500TB of data, for a maximum of 50 accounts, pricing is per-use.

Data is stored in three different physical locations by default, to avoid a single point of failure.

Users are responsible for the security and the flow of data from data sources to cloud storage and back.

Users can replicate data to an on-premise storage server, or to a different datacenter.

Once is in the cloud, any application from anywhere can access this data securely.

The data uploads are limited by the bandwidth from user’s datacenter to Windows Azure datacenters.

Windows Azure 41

References Windows Azure Platform, SecondEdition, Tejaswi Redkar

AzureBench: Benchmarking the Storage Services of the Azure Cloud Platform, Dinesh Agarwal and Sushil K. Prasad

Introduction to Microsoft Azure Storage - azure.microsoft.com

Be Sure With Azure - lockmedown.com/category/azure

Early Observations on the Performance of Windows Azure, Zach Hill et al