Hopscotch

Hopscotch hashing, a scalable,concurrent, resizable hash map

implementationor, how to write a linearly scalable hash map up to 64 cores

Paul ADENOT, <paul@paul.cx>

KTH, CSC Department

November 18, 2011

Hopscotch hashing 1

Table of contents :

Introduction

Hopscotch algorithm implementationA simplified algorithmThe real implementationConcurrency studyPerformances analysis

Conclusion

Hopscotch hashing 2

PlanIntroductionHopscotch algorithm implementationA simplified algorithmThe real implementationConcurrency studyPerformances analysis

Conclusion

Hopscotch hashing 3

In a few wordsI Maurice Herlihy, Brown University, Providence, RII Nir Shavit, Sun Microsystems, Burlington, MAI Moran Tzafrir, Tel-Aviv University, Tel-Aviv, Israel

Presented at the DISC’08 (DIStributed Computing 2008 atArcachon, France).First linearly scalable concurrent hash map, efficient bothin single and multiple thread applications.

Hopscotch hashing 4

Hash map?

I Data structure, capable of associating a key to avalue, thus implementing an associative array

I Supports four operations :I get(key) : retrieve the data associated to the keyI contains(key) : test if a key is present in the hash mapI insert(key, value) : insert a <key,value> pairI remove(key) : remove the <key,value> pair

I Widely used in software you use every day :I Resource caching (key : URL, value : resource)I Object implementation (Perl, Python, Javascript, Ruby,

etc.)I CSS rule matching in browsers rendering engineI Database indexing

Hopscotch hashing 5

Main goals?

I The immense majority of calls are contains(key) &get(key)

I Ability to be able to fetch a value from a key as fastas possible

I Several attack angle possible :I Use an appropriate hash function for a particular

datasetI Use better algorithmsI Leverage hardware specificityI Use multithreading

Hopscotch hashing 6

State of the artAt the moment of the publication of the paper, severalimplementation families exist :

I Chained HashingI Linear probingI Cuckoo hashingI . . .

Hopscotch hashing is a combination of these techniques,and avoids the limitations of these algorithms.

Hopscotch hashing 7

Hardware prerequisiteWhat is a cache line, and why does it matter?

I The minimum amount of data transferable from mainmemory to the cache (64 bytes on my Intel Core i7)

I False sharing problem (on multicore/multiprocessorCPU)

Cache access speed

I Register : 1 cycleI L1 cache : 4 cyclesI L2 cache : 10 cyclesI L3 cache : 40-75 cyclesI Memory : 60-100nsI Disk : 4msI Network : dozens of milliseconds

Hopscotch hashing 8

Chained Hashing

0x0000

0x0004

0x0008

0x000C

0x0010

0x0014

0x0018

value Bucket 0

1

2

3

4

5

6

key value

index=hash(key);

value value

value value value

value value value

value value value

value value value

value value value

I ExtensibleI Not cache friendly : closed addressingI Trivial to implementI Pointer space overheadI Need to allocate memory on insertion (or to use a

pool, . . . )Hopscotch hashing 9

Linear probing

Bucket for one item

K

V

K

V

K

V

K

V

K

V

1 - Value hashed for this bucket

3 - Actual insertion point2 - Linear probing

K

V

K

V

K

V

K

V

K

V

K

V

K

V

K

V

Already occupied

I Cache friendly : closed addressingI Inefficient when the table is rather full (over 75%, most

of the implementation reallocate and rehash thetable)

Hopscotch hashing 10

PlanIntroductionHopscotch algorithm implementationA simplified algorithmThe real implementationConcurrency studyPerformances analysis

Conclusion

Hopscotch hashing 11

Main characteristics of Hopscotch hash map

I Concurrent, and highly scalableI Cache friendly (unlike open addressing)I Good behavior when the hash table is fullI Resizable

Hopscotch hashing 12

General idea (not the real-world implementation) (1)

I Closed addressing, like the linear probingI Item are hashed into a location using a single hash

functionI Entries have a bit table of width H (H is equal to the

size of a machine word)I When inserting, and the location resulting from the

hashing is empty, insert to this location.I When inserting, and the location resulting from the

hashing is not empty, the <key, value> pair isdisplaced to another close and empty location, andthe displacement is written in the bit table

Hopscotch hashing 13

General idea (not the real-world implementation) (2)

I If there is no close and empty location, we find anitem y which hashes between the first empty location(j), and the current entry (i), but within H − 1 elements.We displace y to j, thus we create an empty locationclose to i. If no solution exist, resize and rehash thetable (unlikely, but possible, about Θ(1/H!) ofprobability, that is 3.8 × 10−36 for H = 32 and a goodhashing function)

I When searching, hash the key, and lookup the entryI If the entry found does not match the key, follow the

displacement sequence until we get to the valueI If we can’t find the value at the end of the

displacement sequence, return false

Hopscotch hashing 14

Example

Hopscotch hashing 15

How concurrency is handledThe strategy used is rather simple

I Fine grained locks are protecting the buckets fromconcurrent mutation

I A lock is therefore mapped to each bucketI contains(key) is obstruction-free, and relies on

timestamps to handle concurrency

Hopscotch hashing 16

The real Hopscotch hashing implementation

I In place of the displacement in the array, we useregular pointers and a linked list, still having thelocality constraint (i.e. location of linked-list node areenforced to be within H = hop_range of the initialnode)

I The remove(key) method tries to optimize for cacheline alignment, to rely on a minimal number of cacheline, to avoid having to fetch from main memory.

Hopscotch hashing 17

Handling concurrency, finding the hot spotAnalysis of the data structure utilisation :

I Most of the operations are read-only (contains(key),search(key))

I Each group of buckets (called Segment) has atimestamp field to easily handle concurrent reads andwrites (i.e. contains(key) and remove(key))

I Each Segment also has a lock, to prevent concurrentmutation of data structure.

Hopscotch hashing 18

Linearization pointsadd(pair) and remove(key) use locks, and aredeadlock-free, but not livelock-free, contains(key) isobstruction-freeadd(pair) : linearized when finding the key when it exists,

when adding the bucket to the list of bucketsin the linked list (updating the pointers)

remove(key) : linearized when failing at finding the key (ifthe key does in fact no exist), or when the keystable entry is overwritten

contains(key) : linearized when it finds the key, or when itreaches the end of the list (on an unsuccessfulcontains(key)); and the timestamp isunchanged.

Hopscotch hashing 19

Pseudocode for the obstruction-free contains(key) :� �1 bool contains(KeyType key) {2 int hash = hash(key);3 Segment segment = get_segment(hash);4 Bucket current_bucket = get_bucket(segment, hash);5 int start_timestamp;6 do {7 start_timestamp = segment.timestamp[hash];8 short next_delta = current_bucket.first_delta;9 while (NULL != next_delta) {

10 current_bucket.first_delta += next_delta;11 if (key == current_bucket.key) {12 return true;13 }14 next_delta = current_bucket.next_delta15 }16 } while (start_timestamp != bucket.timestamp[hash])17 return false;18 }� �

Hopscotch hashing 20

Performances analysis

I Most important property : expected constant timeperformance

I In the common case, there is very few items in thebuckets, which is : (α is the density, <= 0)

Number of items in a bucket = 1 +e2α − 1 − 2α

4

I add(pair), remove(key), contains(key) complete inO(1)

I resize() completes in O(n) (n being the number ofelements in the hash map)

Hopscotch hashing 21

Performances, mainly contains(key)

Hopscotch hashing 22

Performances, various operation

Hopscotch hashing 23

A beginning of an explanation?

Hopscotch hashing 24

PlanIntroductionHopscotch algorithm implementationA simplified algorithmThe real implementationConcurrency studyPerformances analysis

Conclusion

Hopscotch hashing 25

ConclusionI Always have hardware consideration when designing

a data structureI The rather low concurrency guarantees can lead to

very good performancesI Simple is not always betterI A 300 line data structure implementation can bring a

tremendous speedup to a program

Hopscotch hashing 26

Questions?

Slides available at http://paul.cx/public/hopscotch.pdf

Hopscotch hashing 27

ReferencesThe paper itself : http:

//www.springerlink.com/content/u710121187m65436/

Obstruction-free introduction paper : Herlihy, M.; Luchangco, V.; Moir,M.,, Distributed Computing Systems, 2003,

Conference from Intel : http://www.gdcvault.com/play/1014645/-SPONSORED-Hotspots-FLOPS-and

Wikipedia pages : CPU cache, Locality of reference, Openaddressing, Hash table.

Master thesis : Sae-eung, Suntorn, "Analysis of False Cache LineSharing Effects on Multicore CPUs" (2010). Master’sProjects. Paper 2.http://scholarworks.sjsu.edu/etd_projects/2

The course’s book

Hopscotch hashing 28