mrys@microsoft.com

Management

Social Network

s

Social Networks

Data Backend

OpsManagement

Services

Data Backend S i

Data Backend Services

s

Data Backend Services

MSN Games l

MSN Games

Web Portal

Services

WLM Games

GameBarHostWLM

Games W b P t l

Front Door Router

Services

Bing

Games

Auth

Web Portal

Bing G STSSTS

gGames

Authl

Games Web Portal

STS

Azure Data Centers

•

••

DBUser …

Partitioned over 100 SQL Azure DBs

Social Service

Social Services

Fi d F i d ’ P fil

Find Friends’ ProfilesGet my ProfilePublish feed, read feedDBUser ServiceFind Friends’ Profiles Publish feed, read feed

Last PlayedGet Friends highscores

STSSTS

Gamer Services

Gamer Services

DBLeaderboard

…

Last PlayedFavoritesGame PreferencesSocial Leaderboards

Front Door R t

Game Ingestion

Game Ingestion

Partitioned over 298 SQL Azure DBs Disable/Enable Games from accessing servicesWrite user specific game infos

Router Services

i

Game Catalog

g

Game binariesGame metadataDBUser …

250 instances Game metadata

Partitioned over 100 SQL Azure DBs 250 instances

• Fanout: Parallel calls to multiple databasemultiple database partitions

Q Abl t• Quorum: Able to tolerate a percentage of request failures during F tFanout

• Retry: Retry on databaseRetry: Retry on database requests error

Statistics per game

Azure DB with Federation Root

Federation Directories, Federation Users, Federation Distributions

Federation− Represents the data being sharded

Federation Root− Database that logically houses federations,

contains federation meta data

Federation “Orders_Fed”

Federation Distributions, …contains federation meta dataFederation Key− Value that determines the routing of a piece

of data (defines a Federation Distribution)Federation Member (aka Shard)

A h i l t i f t f f d t d (Federation Key: CustomerID)− A physical container for a set of federated tables for a specific key range and reference tables

Federated Table− Table that contains only atomic units for the

member’s key range

Member: PK [min, 100)

(Federation Key: CustomerID)

AUPK=5

AUPK=25

AUPK=35member s key range

Reference Table− Non-sharded table

Atomic Unit− All rows with the same federation

key value: always together!

Member: PK [100, 488)

PK=5 PK=25 PK=35

AU AU AUkey value: always together!

Member: PK [488, max)

AUPK=105

AUPK=235

AUPK=365

ConnectioG

ateway

17

ShardedApplication

AUPK=555

AUPK=2545

AUPK=3565

on y

••

•

http://www.microsoft.com/casestudies/Case Study Detail.aspx?CaseStp // / / _ y_ pudyID=4000008310

http://cacm.acm.org/magazines/2011/6/108663-l bl lscalable-sql

http://download.microsoft.com/download/9/E/9/9E9F240D-0EB6-472E-B4DE-6D9FCBB505DD/Windows%20Azure%20No%20SQL%20White%20Paper.pdf

http://blogs msdn com/b/cbiyikoglu/archive/2011/03/03/nosql-genes-in-http://blogs.msdn.com/b/cbiyikoglu/archive/2011/03/03/nosql genes insql-azure-federations.aspx

mrys@microsoft.comy

http://sqlblog.com/blogs/michael_rys/default.aspx

CREATE FEDERATIONExisting Databaseg

CREATE FEDERATION sales i bi i

way

(customer_id bigint RANGE)

Gatew

Conn

ectio

n:dows.net;

l321.db.wind

DB;

r;

;

Server=az1cl

Database=MyD

User=AppUser

Passwd=****;

22

Federation with a Single ShardExisting Databaseg

CREATE FEDERATION sales (customer_id bigint RANGE)

way

salesDatabase root contains:• Federation root = DB level object containing federation scheme

• Federation users

Gatew • Federation metadata incl. federation 

map

Conn

ectio

n:net;

Range: Min...Max

db.windows.n

er=az1cl321.

base=MyDB;

=AppUser;

wd=****;

Federation MemberServe

Datab

User=

Passw

23

Introducing Two Connection Modes

• Filtered Connection– Guarantees that any queries or DML will produce the same results independent of changes to the physical layout of the federation membersS d “A i U i ”– Scoped to an “Atomic Unit”

• Unfiltered Connection– Scoped to a Federation Member– Management Connection

24

Create Schema on Member: Management ConnectionExisting Databaseg

way

sales

Gatew

Conn

ectio

n:net;

USE FEDEDERATION sales (customer_id=0) WITH FILTERING=OFF, RESET;

CREATE TABLE …

Range: Min...Max

db.windows.n

Customer Order Product

er=az1cl321.

base=MyDB;

=AppUser;

wd=****;

Customer Order

© 2011 Microsoft Corporation.   Microsoft Materials ‐ Confidential.  All rights reserved. CITA # 

MSFT101120_A

Federation MemberServe

Datab

User=

Passw

25

DDL

CREATE TABLE customer (CREATE TABLE customer (c_id bigint PRIMARY KEY,…)FEDERATED ON (customer id=c id);( _ _ )

CREATE TABLE order (item_num int,customer_id bigint, date_sold datetime2,…,CONSTRAINT PK_Order PRIMARY KEY (item_num, customer_id, date_sold),CONSTRAINT FK_Cust FOREIGN KEY customer_id REFERENCEScustomer (customer id)customer (customer_id))FEDERATED ON (customer_id=customer_id);

CREATE TABLE product (CREATE TABLE product (product_name varchar(100) NOT NULL,unit_price money,item_num int PRIMARY KEY,…);

26

More Detail

• Supported data types for federation key : bigint int GUID and varbinary• Supported data types for federation key : bigint, int, GUID,  and varbinary(900)– Only range partitioning

• Federation key must be part of unique index• Foreign key constraints only allowed between federated tables and from 

federated table to reference table• Not all Azure programmability features supported

Sequence timestamp– Sequence, timestamp• Additional surface area restrictions

– Indexed views, drop database (members)• Schemas are allowed to diverge over timeg

– Furthermore, in v1, schema updates to existing members must be done in each member (where the change is needed)

• USE FEDERATION “rewires a connection”Connection is reestablished– Connection is reestablished

– All existing settings and context of the connection is lost (sp_reset_connection)– Must be in a batch by itself

27

Connect to Atomic Unit: FilteredExisting Databaseg

When using into a specific key way

salesvalue, SELECT will only return records from federated tables that match that value. It will still return all records from non‐federated 

Gatew

Conn

ectio

n:net;

USE FEDEDERATION sales (customer_id=3) WITH FILTERING=ON, RESET;

tables.Inserts and UPDATES operating outside of the value will fail.

Range: Min...Max

customer order productdb.windows.n

33

er=az1cl321.

base=MyDB;

=AppUser;

wd=****; SELECT * from customer

SELECT * from order

SELECT * from product

Federation MemberServe

Datab

User=

Passw

28

More on Connection Filtering 

• Most operations behave differently in filtered vsunfiltered connectionsC ti filt i i t f th i• Connection filtering is a property of the session– Filter injected dynamically at runtime– Cannot inspect source code to determine how it behavesCannot inspect source code to determine how it behaves

• E.g., running stored proc written for filtered mode on unfiltered connection could lead to unintended results

• There are several operations that will not work in• There are several operations that will not work in filtered connection in v1– DDL, DML on reference tables, …

• Fan‐out, bulk operations not efficient in filtered mode– For now, filter=off is our best offer

29

Support Matrix

Connection Type Filtered Unfiltered Named (unfiltered)Operation

Dynamic SELECT DML* (federated tables)DML  (federated tables) DML* (reference tables)  X

DDL  X

Views (not indexed) UDF ‐ activate

Stored Proc ‐ activateTrigger (all modes) ‐ activate

CREATE/UPDATE Stats XCREATE/UPDATE Stats XBulk Ops

openrowset bulk, bcp, bulk insert X

*  not including SELECT & modules

^ autostats will work on all connections

System stored procs, intrinsics will be unaffected (run unfiltered)

30

Splitting a MemberExisting Databaseg

ALTER FEDERATION l

way

salesALTER FEDERATION salesSPLIT AT (customer_id=50)

Gatew

Conn

ectio

n:net;

USE FEDERATION ROOT WITH RESET

Using to the federation ROOT 

db.windows.n

will pop you out of a member back into the database that hosts the 

Range: Min...Max

customer order product

er=az1cl321.

base=MyDB;

=AppUser;

wd=****;

federation 3

58

3

58

58

40

Federation MemberServe

Datab

User=

Passw

31

Two New MembersExisting Databaseg

ALTER FEDERATION l

way

sales

USE FEDEDERATION ROOT

ALTER FEDERATION salesSPLIT AT (customer_id=50)

Gatew

Conn

ectio

n:net;

USE FEDEDERATION ROOT WITH RESET

Range: Min...50

customer order productdb.windows.n

Range: 51...Max

customer order product

33

er=az1cl321.

base=MyDB;

=AppUser;

wd=****;

5858

5840

Federation MemberServe

Datab

User=

Passw

Federation Member

32

Two New MembersExisting Databaseg

sales

way

Conn

ectio

n:net;

USE FEDEDERATION sales (customer_id=40) WITH FILTERING=ON, RESET;G

atew

Range: Min...50

customer order productdb.windows.n

Range: 51...Max

customer order product

40

er=az1cl321.

base=MyDB;

=AppUser;

wd=****;

5858

5840

SELECT *from customer

SELECT * from order

Serve

Datab

User=

Passw

33