Project "Velocity"- A First Look

Muralidhar KrishnaprasadPrincipal ArchitectMicrosoft Corporation

TL14

Distributed Caching Types of Cached data

What is Velocity? When can I use it?

How does it help my application? Performance Scale High Availability

Features of Velocity

This session..

Most composite applications designed for serious scale and availability are using mid-tier, distributed caching technologies today.

Evolving

Application

Requirements

Underlying

Hardware

Trends

Evolving

Application

Architectures

Evolving

Business

Requirements

Caching Reference Data

Web Tier

Clients

Local Cache (in Proc)

Distributed Cache Servers

Data Tier

Usernames, Name-> ID Mapping

Friend Lists Usernames

Scenario: Social Networking

Caching Activity-Oriented Data

Data Tier

Application Tier

Distributed Cache

IntegrationHub

Clients

Web Tier

Vendor services Pricing

VendorSources

Order, Invoice, Payment

Aggregated Vendor

Catalogs

Scenario: Enterprise LOB Application

Caching Resource-Oriented DataScenario: Flight Inventory and Pricing

Application Tier

American

Distributed Cache

Flight Itinerary

Flight Segment Flight Price

Airli

nes

Seat Inventory

Booking Service

Sample Application

Shopping Cart

Catalog

Inventory

Unified Cache View

What Is "Velocity"?

An explicit, distributed, in-memory application cache for all kinds of data (CLR objects, rows, XML, Binary data etc.) Fuse "memory" across machines into a unified cache

Clients can be spread across machines or

processes

Clients Access the Cache as if it

was a large single cache

Cache Layer distributes data

across the various cache

nodes

Where Does It Fit?

…Application/ Web Tier

Users

Database

Cloud Data Tier

Application

Application

Application

Velocity ClientLocal Cache

Velocity ClientLocal Cache

Velocity ClientLocal Cache

Server 1 Server 2 Server 3

Velo

city

Ser

vice

Velo

city

Ser

vice

Velo

city

Ser

vice

Server Side Callbacks

Cache Tier

Why Velocity?

Increase PerformanceCache Objects close to

Application

Scale OutAdd More Machines to Scale out

your Application

Make Application AvailableProtect from machine failures

Do all this at low costUse Cheap Commodity Hardware

Velocity V1 will be FREE out of band release for the .Net framework

When Can I Get Velocity?

CTP1• Teched

CTP2• Now

CTP3• Mix 2009

RTM• Mid 2009

Velocity Concepts

Velocity Logical Hierarchy

Host Physical processes hosting Velocity

instance. Named Caches

Can span across machines Defined in the configuration file

Regions Physically co-located Container of

Cache Items May be implicit or explicitly created

Cache Item Key, Payload (Object ), Tags, TTL,

Timestamps, Version

Regions Region A

Key Payload Tags Key Payload Tags 121 xxxx “Toy” “Child”

123 yyyy “Toy” “Chair”..

Machine -> Cache Host -> Named Caches -> Regions -> Cache Items -> Objects

Velocity Service

Named Cache : Product Catalog

Named Cache : Electronics Inventory

Velocity Service

Velocity Service

Velocity Service

Example Usage Of Velocity API

// Create instance of cachefactory (reads appconfig)CacheFactory fac = new CacheFactory();

// Get a named cache from the factoryCache catalog = fac.GetCache("catalogcache");

// Simple Get/Putcatalog.Put("toy-101", new Toy("Puzzle", .,.));

// From the same or a different clientToy toyObj = (Toy)catalog.Get("toy-101");

// Region based Get/Putcatalog.CreateRegion("toyRegion");

// Both toy and toyparts are put in the same region catalog.Put("toyRegion", "toy-101", new Toy( .,.));Catalog.Put("toyRegion", "toypart-100", new ToyParts(…));

Toy toyObj = (Toy)catalog.Get("toyRegion", "toy-101");

Install Demo

demo

Velocity V1 Features

Simple Access Patterns – Get/Add/Put/Remove Tag Searching Partitioned & Local Client Cache High Availability Multiple Concurrency Control Options ASP.Net Integration Management & Monitoring Security Cache Event Notifications Server Side Callbacks

Post CTP2

Access APIs – Tagging Items

Hot or Discount Items in catalog Use Tagging

List All Items / Hot Items - Enumeration Use Enumeration

// Add TagsTag hotItem = new Tag("hotItem");Tag discItem = new Tag("discountItem");

catalog.Put("toyRegion", "toy-101", new Toy("Puzzle"), new Tag[]{hotItem});catalog.Put("toyRegion", "toy-102", new Toy("Bridge"), new Tag[]{hotItem,discItem});

// From the same or a different clientList<KeyValuePair<string, object>> toys = catalog.GetAnyMatchingTag("toyRegion", hotItem);

Cache Types

Scale the Catalog and Shopping CartPartitioned Cache

Data Partitioned across all nodes in the named cache Backups for shopping cart for Availability

Client Side Local Cache for Higher Performance Use Local Cache in client to speed up catalog access Payload kept in object form Synchronization using expiry or notifications

*- Post CTP2

Partitioned Cache

…Application/ Web Tier

Cache Tier

Velo

city

Ser

vice

Velo

city

Ser

vice

Users

ApplicationVelocity Client

Local Cache

Velo

city

Ser

vice

ApplicationVelocity Client

Local Cache

ApplicationVelocity Client

Local Cache

A C EDB G IHF

Server 1 Server 2 Server 3

Partitioning Increases Scale And Performance

Scale on Data Size More machines => More memory to cache

Scale on Cache Throughput Cache Throughput: Operations/sec from

entire cache More machines => keys distributed across

more machines => better throughput Performance

High Performance by scaling out data and processing

Increased memory Increased processing

Workload distributed across multiple cache nodes

Scale Demo

demo

Basic Layout

Load Test Controller

…

Server 1 Server 2 Server 3

Cache Client1 Cache Client 2 Cache Client n

Load Controller Spawns Test

Clients

Client Machines run

test

Cache Service Cluster

Partitioned Cache

Demo Output

Single ServerThroughput Increases with Increasing Load

Until Server Saturation

Load

Throughput

Latency

Server 2 AddedThroughput Increases

Latency DecreasesUntil Server Saturation

Server 3 AddedThroughput Increases

Latency Decreases

Partitioned Cache + HA Option

…Application/ Web Tier

Cache Tier

Velo

city

Ser

vice

Velo

city

Ser

vice

Users

ApplicationVelocity Client

Local Cache

Velo

city

Ser

vice

ApplicationVelocity Client

Local Cache

ApplicationVelocity Client

Local Cache

A C EDB G IHF

Server 1 Server 2 Server 3AC

E

DB G

I HF

HA Option – Failure Scenario

…Application/ Web Tier

Cache Tier

Velo

city

Ser

vice

Velo

city

Ser

vice

Users

ApplicationVelocity Client

Local Cache

Velo

city

Ser

vice

ApplicationVelocity Client

Local Cache

ApplicationVelocity Client

Local Cache

AC

E

D B GI H

F Server 1 Server 2 Server 3

AC

E DBG I

HF DA H

CF E

Session Cart Integration With ASP.Net

…Session State

stored in Velocity

Velo

city

Ser

vice

Velo

city

Ser

vice

Load Balance RequestsNo more sticky routing

ApplicationVelocity

SessionStoreProvider

ApplicationVelocity

SessionStoreProvider

ApplicationVelocity

SessionStoreProvider

Velo

city

Ser

vice

Scale your Session StoreDynamically

Highly Available

Drop in Velocity SessionStoreProvider

<sessionState mode="Custom" customProvider="SessionStoreProvider"><providers> <add name="SessionStoreProvider" type="System.Data.Caching.SessionStoreProvider, ClientLibrary" cacheName="<YourNamedCache>"/></providers></sessionState>

Availability Demo

demo

Availability Demo

Cache ClusterVelocity

Embedded in ASP.Net

Application

NorthwindApplication

ASP.net SessionCache

VELOCITYVelocity SessionStore Provider

NorthwindApplication

ASP.net SessionCache

VELOCITYVelocity SessionStore Provider

CacheViewerApplication

VELOCITY

Partitioned Cache

Administration And Monitoring

Installation MSI to install on each box Silent/Remote Install

Logging Provider model; Default ETW, support for file logs

Administration Powershell Commandlets

Start and stop cluster or service, configure named caches .. Monitor Cache statistics

Perfmon integration

* Post CTP2

Velo

city

Ser

vice

Configuration Store

Used to store Dynamic Global Partitioning Map

Enables Global Partition Manager to restart fast Velocity Configuration & Policies

Enables global change of policies Options - Network Share or SQL Server

Server 1 Server 2 Server 3

Cache Servers

Configuration Store (Can be database , File share etc.)Stores Named Cache PoliciesStores Global Partitioning Map

Glo

bal P

artiti

on

Man

ager

Velo

city

Ser

vice

Velo

city

Ser

vice

One of the Velocity

Service Hosts the Global Partition Manager

Putting It All Together

Web Tier

ASP.Net Application Farm

Clients

Velocity Cache Service

Data Tier

Friend Lists Usernames

Velocity Local Cache

Caching Reference Data

Scenario: Social NetworkingPartitioned Cache

Scale

Usernames, Name-> ID Mapping Local Client Cache

Higher Performance

Application TierASP.Net Application Farm

Caching Activity-Oriented Data

Data Tier

Velocity (with HA on)

IntegrationHub

Clients

Web Tier

Vendor services Pricing Vendor Sources

Order, Invoice, Payment

Aggregated Vendor

Catalogs

Scenario: Enterprise LOB Application

Partitioned Cache + HA

Scale + Availability

Session Store Provider

ASP.Net IntegrationSession Load

Balancing

Caching Resource-Oriented DataScenario: Flight Inventory and Pricing

Application Tier

American

Velocity

Flight Itinerary

Flight Segment Flight Price

Airli

nes

Seat Inventory

Booking ServicePartitioned Cache + HA + Concurrency

ControlScale + Availability +

Performance

Looking Just Ahead

More PerformancePost CTP2

CTP2 delivers 22K Ops per second on Quad Core for 4K sized Objects @ 0.3 – 0.5ms per access without serialization (byte[]) @ around 2 ms with serialization overheads

Session Store 50K sessions stored and accessed Same as SQLServer temporary table performance

Bulk Access APIs Bulk fetch or puts to improve throughput

SecurityPost CTP2

Open Security Option No security enforced

Token Based Security Option Register Simple Token Id at named cache level Application must call GetCache() with that token

App-id Based Security Option Domain Account based Authentication Register the domain account with the named cache

Transport Level Security Use Standard WCF security options E.g., Encryption

June 2008 (CTP1)(TechEd 2008)

Oct 2008 (CTP2)(PDC 2008)

CTP3(Mix 2009)

• Mid 2009RTM

Schedule

Distributed, partitioned cache service

Regions, Tags

Local Cache ASP.Net Integration

Availability Configuration Options

Powershell

Bulk Access APIs Security

Read-Through/Write-Behind Cache Event Notifications

Recap

Make it available

Make it faster

Scale your app

Velocity

Http://msdn.microsoft.com/data

Evals & Recordings

Please fill

out your

evaluation for

this session at:

This session will be available as a recording at:

www.microsoftpdc.com

Please use the microphones provided

Q&A

© 2008 Microsoft Corporation. All rights reserved. Microsoft, Windows, Windows Vista and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries.The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market

conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.