AppFabric Velocity

Session Code: NE.13

Velocity, distributed caching

Dennis van der Stelt

Introduction

• Class-A– Kennisprovider– Microsoft development– Training / Coaching– www.class-a.nl

• Dennis van der Stelt– Trainer / Coach– Weblog at http://bloggingabout.net/blogs/dennis/

Agenda

• Challenges for webfarms• What is Velocity?• Options in caching in Velocity• Velocity features• Caching strategies• Best practices

Challenges for webfarms

Load balancer

Web server A

Web server B

Web server C

Solution : sticky sessions

Load balancer

Web server A

Web server B

Web server C

1. Uneven load balancing2. Lost sessions3. Single point of failure4. Need to drain state

Solution : Use DBMS

Load balancer

Web server A

Web server B

Web server C

1. Doesn’t scale2. Creates a performance

bottleneck3. Clustering is costly and fault

sensitive

What is Velocity?

What is Velocity?• An explicit, distributed, in-memory application

cache for all kinds of data (CLR objects, XML, Binary data, etc.)– Flows "memory" across machines into a unified

cache

Unified Cache View

Clients can be spread across machines or

processes

Clients Access the Cache as if it was a large single cache

Cache Layer distributes data

across the various cache nodes

Velocity in a webfarm

Load balancer

Web server A

Web server B

Web server C

Why Velocity?

Operation Throughput

Latency

Read 2k 30,000 / sec

3 – 4 ms

Write 2k 18,000 / sec

3 ms

Operation Servers

Throughput

Read 2k 1 30,000 / sec

Read 2k 2 58, 600 / sec

Read 2k 3 85, 500 / sec

Velocity

* Share data across applications No more sticky routing

* Peformance

* Scale out

MSDN Forums5 Velocity Servers, 40GB Cache

98% of all calls to database come from cacheResult is from 60% load on SQL Server, down to 2%

Key features of Velocity

• Caches any serializable CLR object• Enterprise scale

– Up to hundreds of computers– Dynamic scaling

• Runs as a service• Automatic load balancing

Key features of Velocity

• Tight integration with ASP.NET– Use as session store– Cache application data

• Integration with administration and monitoring tools as System Center, etc.

• Cache-aside architecture for version 1• Support for multiple client languages

– PHP, C#, C++, etc.

Velocity installation

• Needed for installation– .NET Framework 3.5– Windows XP/Vista/Server (2003/2008)

• Firewall exceptions on port 22233-22235• Configuration

– SQL Server– Network share

single-point-of-failure?10 concurrent connections?

Basic terminology in Velocity

• Cache host• Cache cluster• Cluster configuration storage• Named cache• Region• Cache item• Tags

Machine 1Cache host A

Cache host B

Machine 2Cache host C

Machine 2Cache host DNamed cache : Product catalog

Named cache : ShoppingCart

Region A

Working with Velocity

// Create instance of CacheFactory, which reads app.config DataCacheFactory factory = new DataCacheFactory();        

// Get a named cache from the factory       DataCache cache = factory.GetCache("default");

// Cache.Put(string key, object value)cache.Add("SDC", new SDNConference()); // Cache.Get(string key);var meeting = (SDNConference)cache.Get("SDC");

// Via indexers is also an optioncache["what"] = new Object();Object o = cache["what"];

Types of cache

•Data partitioned across all nodes

•Used for scale & availability

Partitioned

•Data replicated across all nodes

•Used for high availability

•All reads occur on primary host

Replicated

•Payload stays in object form•No

serialization

•No extra network hops

Local

Partitioned cache

Application 2Application 1

Cache2Cache1

Primary RegionsPrimary Regions

T1, “A”

Cache3

Primary Regions

T3, “A”

Routing table Routing table

Put(“T5”, “D”)

T5, “D”

Routing table Routing table Routing table

33% 33% 33%

Replicated cache (high availability)

Application 2Application 1

Cache2Cache1

Primary RegionsPrimary Regions

T1, “A”

Cache3

Primary Regions

T3, “A”

Routing table Routing table

Put(“T5”, “D”)

Secondary Regions Secondary Regions Secondary Regions

T5, “D”

T5, “D” T3, “A” T1, “A”

Get(“T5”)

Routing table Routing table Routing table

Cache clients

Simple client

• Has to request where objects live

Routing Client

• Has knowledge of where objects live

• Keeps in sync with hosts

• Performance benefits

• Extra network communication

Local cache

• No pre-installed host needed

• Your application is the host

• Configurable• Stored in

deserialized state!

Partitioned cache fronted by local cache

Velocity Client2Velocity Client1

Cache2Cache1

Primary RegionsPrimary Regions

T1, “A”

Cache3

Primary Regions

T3, “A”

Put(T5, “D”)

Get(T5)

T5, “D”

Local cache

Routing layer

Local cache

Routing layer

T5, “D” T5, “D”

Velocity features

Regions

// Without regionscache.Put("MyKey", sdnConference); // With regionscache.CreateRegion("MyRegion", true);cache.Put("MyKey", sdnConference, "MyRegion"); var result = (SDNConference)cache.Get("MyKey", "MyRegion");

• Regions don’t distribute• But will replicate when high availability is on!

Tags

var starWarsTag = new DataCacheTag("StarWars");

var tags = new List<DataCacheTag>();tags.Add(starWarsTag);tags.Add(new DataCacheTag("Force"));tags.Add(new DataCacheTag("Sith")); cache.Add("MyKey", "A New Hope", tags, "StarWarsRegion"); var result = cache.GetObjectsByTag(starWarsTag, "StarWarsRegion"); foreach (var item in result){  Console.WriteLine("{0} has value of {1}", item.Key, item.Value);}

Optimistic locking

DataCacheItemVersion versionWillChange;DataCacheItemVersion versionWithError; // First get the current version 2 timescache.Get("MyKey", out versionWillChange);cache.Get("MyKey", out versionWithError); // We change the key, version hasn't changed in Velocity yet.cache.Put("MyKey", "MyNewValue", versionWillChange); // Version has changed with previous update, this will #failcache.Put("MyKey", "MyErrorValue", versionWithError);

Pessimistic locking

DataCacheLockHandle lockHandle = null;DataCacheLockHandle secondLockHandle = null; // Lock our objectcache.GetAndLock("MyKey", new TimeSpan(0, 0, 10), out lockHandle); // This will still workstring result = (string)cache.Get("MyKey"); // Try to lock for 2nd time -> #failcache.GetAndLock("MyKey", new TimeSpan(0, 0, 10), out secondLockHandle); // This will break the lock!!!cache.Put("MyKey", "MyNewValue");

Notification callback

DataCacheOperation filter = DataCacheOperation.AddItem;cache.AddItemLevelCallback("MyKey", filter, callback);cache.AddRegionLevelCallback("Region", filter, callback);cache.AddFailureNotificationCallback(failCallback); cache.Add("MyKey", "MyInitialValue");

public static void callback (string myCacheName,    string myRegion, string myKey, DataCacheItemVersion itemVersion, DataCacheOperation OperationId,    DataCacheNotificationDescriptor nd){  //display some of the delegate parameters  Console.WriteLine("Region : " + myRegion);  Console.WriteLine("Key : " + myKey);}

Cluster configuration

<dataCacheClient deployment="routing">  <localCache    isEnabled="true"    sync="TTLBased"    objectCount="100000"    ttlValue="300" />   <clientNotification pollInterval="300" />   <!-- cache host(s) -->  <hosts>    <host       name="localhost"       cachePort="22233"       cacheHostName="DistributedCacheService"/>  </hosts></dataCacheClient>

ASP.NET Session integration

• SessionStoreProvider class– Plugs into ASP.NET Session store– Stores session state in Velocity

• Scale– Session information available at all ASP.NET

Nodes• High availability

– Session data is backed up on addditional machines– Resilient to machine or process failures

Performance

Achieving scale and performance

• Scale on data size– More machines => more memory to cache

• Scale on cache throughput– Throughput : Operations/sec from entire cache– More machines : keys distributed across more

machines => better throughput• Performance

– High performance by scaling out data and processing– Increased memory– Increased processing

• Workload distributed across multiple cache nodes

Achieving scale and performance

Single ServerThroughput Increases with Increasing Load

Until Server Saturation

Load

Throughput

Latency

Server 2 AddedThroughput Increases

Latency DecreasesUntil Server Saturation

Server 3 AddedThroughput Increases

Latency Decreases

Caching strategies

Velocity

• Is Velocity next-next-finish?– Think about the consequences!– From in-proc. cache to out-of-process cache– Multiple network cards for higher bandwith

• Don’t disturb your normal web visitors• Velocity can bring network card to its knees

– Computers are communicating data• Can you transfer lots of data, allowed to do so?• What about security?

Best practices

• Develop a caching strategy– When and what to cache– Selection of cache keys

• Create guidelines for caching– Make sure developers live by the strategy– Caching is hard to do right

• Code once, measure twice

Best practices

• Cache close to where you need the items– Usually presentation or service layer

• Cache wisely– Think about cache item lifetimes, eviction– Don’t cache to get performance, but to

improve performance• Define a key naming convention

– Avoid conflicts of cache entries

Velocity and the future

• Available mid 2009 (huh?)• Velocity is cache provider in .NET 4.0• Version 2

– Will be available in the cloud– LINQ over Velocity– Output caching provider– Cache through

• Read through & Write behind

– Grid computing

Review

• Distributed cache is the future for performance demanding enterprises– Linkedin, Slashdot, Facebook, Flickr,

Wikipedia• Velocity makes this possible• A giant hashtable with performance,

scalability and failover.

Thank you for your attention

• Dennis van der Stelt– Dennis@BloggingAbout.NET– http://twitter.com/dvdstelt/– http://bloggingabout.net/blogs/dennis/

Resources to more information can be found here, incl. this slidedeck.

Evaluation form

Vul je evaluatieformulier in en maak kans op een van de prachtige prijzen!!

Fill out your evaluation form and win one of the great prizes!!

Session Code: NE.13