CS235102 CS235102 Data StructuresData Structures

Chapter 8 HashingChapter 8 Hashing

Chapter 8 Hashing: OutlineChapter 8 Hashing: Outline

The Symbol Table Abstract Data TypeThe Symbol Table Abstract Data Type Static HashingStatic Hashing

Hash TablesHash Tables Hashing FunctionsHashing Functions

Mid-squareMid-square DivisionDivision FoldingFolding Digit AnalysisDigit Analysis

Overflow HandlingOverflow Handling Linear Open Addressing, Quadratic probing, RehashingLinear Open Addressing, Quadratic probing, Rehashing ChainingChaining

The Symbol Table ADT (1/3)The Symbol Table ADT (1/3) Many example of Many example of dictionariesdictionaries are found in many are found in many

applications, Ex. spelling checkerapplications, Ex. spelling checker In computer science, we generally use the term In computer science, we generally use the term

symbol tablesymbol table rather than dictionary, when rather than dictionary, when referring to the ADT.referring to the ADT.

We define the symbol table as a We define the symbol table as a set of set of name-name-attribute pairsattribute pairs..

Example: In a symbol table for a compilerExample: In a symbol table for a compiler the name is an identifierthe name is an identifier the attributes might include an initial valuethe attributes might include an initial value a list of lines that use the identifier.a list of lines that use the identifier.

The Symbol Table ADT (2/3)The Symbol Table ADT (2/3) Operations on symbol table:Operations on symbol table:

Determine if a particular name is in the tableDetermine if a particular name is in the table Retrieve/modify the attributes of that nameRetrieve/modify the attributes of that name Insert/delete a name and its attributesInsert/delete a name and its attributes

ImplementationsImplementations Binary search tree: the complexity is O(Binary search tree: the complexity is O(nn)) Some other binary trees (chapter 10): O(log Some other binary trees (chapter 10): O(log nn).).

HashingHashing A technique for A technique for search, insert, and deletesearch, insert, and delete operations t operations t

hat has very good expected performance.hat has very good expected performance.

The Symbol Table ADT (3/3)The Symbol Table ADT (3/3)

Search TechniquesSearch Techniques Search tree methodsSearch tree methods

Identifier comparisonsIdentifier comparisons

Hashing methodsHashing methods Relies on a formula called the hash function.Relies on a formula called the hash function.

Types of hashingTypes of hashing Static hashingStatic hashing Dynamic hashingDynamic hashing

Hash Tables (1/6)Hash Tables (1/6) In static hashing, we store the identifiers in a fixeIn static hashing, we store the identifiers in a fixe

d size table called a d size table called a hash tablehash table Arithmetic function, Arithmetic function, ff

To determine the address of an identifier, To determine the address of an identifier, xx, in the tabl, in the tablee

ff((xx) gives the hash, or home address, of ) gives the hash, or home address, of xx in the table in the table

Hash table, Hash table, htht Stored in sequential memory locations that are partitioStored in sequential memory locations that are partitio

ned into ned into bb buckets, buckets, htht[0], …, [0], …, htht[[bb-1].-1]. Each bucket has Each bucket has ss slots slots

Hash Tables (2/6)Hash Tables (2/6)

. . . . . ... ... ... . . . .

012..

b-2b-1

1 2 ………. s

hash table (ht) f(x): 0 … (b-1)

s slots

b buckets

Hash Tables (3/6)Hash Tables (3/6)

The The identifier densityidentifier density of a hash table of a hash table is the ratio is the ratio n/Tn/T nn is the number of identifiers in the table is the number of identifiers in the table TT is possible identifiers is possible identifiers

The The loading densityloading density or or loading factorloading factor of a hash table is of a hash table is = = nn/(/(sbsb)) ss is the number of slots is the number of slots bb is the number of buckets is the number of buckets

Hash Tables (4/6)Hash Tables (4/6)

Two identifiers, Two identifiers, ii11 and and ii22 are are synonymssynonyms with with

respect to respect to ff if if ff((ii11) = ) = ff((ii22)) We enter distinct synonyms into the same bucket as We enter distinct synonyms into the same bucket as

long as the bucket has slots availablelong as the bucket has slots available

An An overflowoverflow occurs when we hash a new occurs when we hash a new identifier into a full bucketidentifier into a full bucket

A A collisioncollision occurs when we hash two occurs when we hash two non-identical identifiers into the same bucket.non-identical identifiers into the same bucket.

When the bucket size is When the bucket size is 11, collisions and , collisions and overflows occur simultaneously.overflows occur simultaneously.

Hash Tables (5/6)Hash Tables (5/6) Example 8.1: Hash tableExample 8.1: Hash table

b b = 26 buckets and = 26 buckets and ss = 2 slots. Distinct identifiers = 2 slots. Distinct identifiers nn = 10 = 10 The loading factor, The loading factor, , is 10/52 = 0.19., is 10/52 = 0.19. Associate the letters, Associate the letters, a-za-z, ,

with the numbers, 0-25, with the numbers, 0-25, respectivelyrespectively

Define a fairly simple hash Define a fairly simple hash function, function, ff((xx), as the ), as the first character of first character of xx..

Synonyms

Synonyms

C library functions (f(x)): acos(0), define(3), float(5), exp(4), char(2), atan(0), ceil(2), floor(5), clock(2), ctime(2)

overflow: clock, ctime

Synonyms

Hash Tables (6/6)Hash Tables (6/6)

The time required to enter, delete, or search for The time required to enter, delete, or search for identifiers does not depend on the number of identifiers does not depend on the number of identifiers identifiers nn in use; it is in use; it is OO(1).(1).

Hash function requirements: Hash function requirements: Easy to compute and produces few collisions.Easy to compute and produces few collisions. Unfortunately, since the ration Unfortunately, since the ration bb//TT is usually small, we is usually small, we

cannot avoid collisions altogether. cannot avoid collisions altogether. => Overload handling mechanisms are needed=> Overload handling mechanisms are needed

Hashing Functions (1/8)Hashing Functions (1/8) A hash function, A hash function, ff, transforms an identifier, , transforms an identifier, xx, int, int

o a bucket address in the hash table.o a bucket address in the hash table. We want a hash function that is easy to compute We want a hash function that is easy to compute

and that minimizes the number of collisions.and that minimizes the number of collisions. Hashing functions should be unbiased.Hashing functions should be unbiased.

That is, if we randomly choose an identifier, That is, if we randomly choose an identifier, xx, from th, from the identifier space, the probability that e identifier space, the probability that ff((xx) = i is 1/) = i is 1/bb for for all buckets all buckets ii..

We call a hash function that satisfiesWe call a hash function that satisfies unbiased propert unbiased propertyy a a uniform hash functionuniform hash function..Mid-square, Division, Folding, Digit AnalysisMid-square, Division, Folding, Digit Analysis

Hashing Functions (2/8)Hashing Functions (2/8) Mid-squareMid-square ffmm((xx)=)=middlemiddle((xx22):):

Frequently used in symbol table applications.Frequently used in symbol table applications. We compute We compute ffmm by squaring the identifier and then usi by squaring the identifier and then usi

ng an appropriate number of bits from the middle of thng an appropriate number of bits from the middle of the square to obtain the bucket address.e square to obtain the bucket address.

The number of bits used to obtain the bucket address The number of bits used to obtain the bucket address depends on the table size. If we use depends on the table size. If we use rr bits, the range bits, the range of the value is 2of the value is 2rr..

Since the middle bits of the square usually depend upSince the middle bits of the square usually depend upon all the characters in an identifier, there is high probon all the characters in an identifier, there is high probability that different identifiers will produce different haability that different identifiers will produce different hash addresses.sh addresses.

Hashing Functions (3/8)Hashing Functions (3/8)

DivisionDivision ffDD((xx) = ) = xx % % M M :: Using the modulus (%) operator.Using the modulus (%) operator. We divide the identifier We divide the identifier xx by some number by some number MM and use and use

the remainder as the hash address for the remainder as the hash address for xx.. This gives bucket addresses that range from 0 to This gives bucket addresses that range from 0 to MM - 1, wher - 1, wher

e e MM = that table size. = that table size.

The choice of The choice of MM is critical. is critical. If If MM is divisible by 2, then odd keys to odd buckets an is divisible by 2, then odd keys to odd buckets an

d even keys to even buckets. (biased!!)d even keys to even buckets. (biased!!)

Hashing Functions (4/8)Hashing Functions (4/8) The choice of The choice of MM is critical (cont’d) is critical (cont’d)

When many identifiers are permutations of each other, a biased use oWhen many identifiers are permutations of each other, a biased use of the table results.f the table results.

Example: Example: X=x1x2 and Y=x2x1

Internal binary representation: x1 --> C(x1) and x2 --> C(x2)Each character is represented by six bits

X: C(x1) * 26 + C(x2), Y: C(x2) * 26 + C(x1)

(fD(X) - fD(Y)) % p (where p is a prime number)

= (C(x1) * 26 % p + C(x2) % p - C(x2) * 26 % p - C(x1) % p ) % pp = 3, 26=64

(64 % 3 * C(x1) % 3 + C(x2) % 3 - 64 % 3 * C(x2) % 3 - C(x1) % 3) % 3

= C(x1) % 3 + C(x2) % 3 - C(x2) % 3 - C(x1) % 3 = 0 % 3

The same behavior can be expected when The same behavior can be expected when p p = 7= 7 A good choice for A good choice for MM would be : would be : MM a prime number such that a prime number such that MM does n does n

ot divide ot divide rrkkaa for small for small kk and and aa..

Hashing Functions (5/8)Hashing Functions (5/8) FoldingFolding

Partition identifier Partition identifier xx into several parts into several parts All parts except for the last one have the same lengthAll parts except for the last one have the same length Add the parts together to obtain the hash addressAdd the parts together to obtain the hash address

Two possibilities (divide Two possibilities (divide xx into several parts) into several parts) Shift foldingShift folding: :

Shift all parts except for the last one, so that the least Shift all parts except for the last one, so that the least significant bit of each part lines up with corresponding significant bit of each part lines up with corresponding bit of the last part.bit of the last part. xx11=123, =123, xx22=203, =203, xx33=241, =241, xx44=112, =112, xx55=20, address=699=20, address=699

Folding at the boundariesFolding at the boundaries: : reverses every other partition before addingreverses every other partition before adding xx11=123, =123, xx22==302302, , xx33=241, =241, xx44==211211, , xx55=20, address=897=20, address=897

P1 P2 P3 P4 P5

123 203 241 112 20

shift folding 123203

241112

20

699

folding at the boundaries

MSD ---> LSDLSD <--- MSD

123 203 241 112 20

Hashing Functions (6/8)Hashing Functions (6/8) Folding example:Folding example:

Hashing Functions (7/8)Hashing Functions (7/8)

Digit AnalysisDigit Analysis Used with static filesUsed with static files

A A static filesstatic files is one in which all the identifiers are known in is one in which all the identifiers are known in advance.advance.

Using this method,Using this method, First, transform the identifiers into numbers using some radix, First, transform the identifiers into numbers using some radix,

rr.. Second, examine the digits of each identifier, deleting those Second, examine the digits of each identifier, deleting those

digits that have the most skewed distribution.digits that have the most skewed distribution. We continue deleting digits until the number of remaining We continue deleting digits until the number of remaining

digits is small enough to give an address in the range of the digits is small enough to give an address in the range of the hash table.hash table.

Hashing Functions (8/8)Hashing Functions (8/8) Digital Analysis example:Digital Analysis example:

All the identifiers are known in advance, All the identifiers are known in advance, MM=1~=1~999999

XX11:d:d1111 dd1212 …… dd1n1nXX22:d:d2121 dd2222 …… dd2n2n……XXmm:d:dm1m1 ddm2m2 …… ddmnmn

Select 3 digits from Select 3 digits from nn Criterion:Criterion:

Delete the digits having the most skewed distributionsDelete the digits having the most skewed distributions

The one most suitable for general purpose appliThe one most suitable for general purpose applications is the division method with a divisor, cations is the division method with a divisor, MM, s, such that uch that MM has no prime factors less than 20. has no prime factors less than 20.

Overflow Handling (1/8)Overflow Handling (1/8)

Linear open addressing (Linear probing)Linear open addressing (Linear probing) Compute Compute ff((xx) for identifier ) for identifier xx Examine the buckets:Examine the buckets:

htht[([(ff((xx)+)+jj)%)%TABLE_SIZETABLE_SIZE], 0 ], 0 jj TABLE_SIZETABLE_SIZE The bucket contains The bucket contains xx.. The bucket contains the empty string (insert to it)The bucket contains the empty string (insert to it) The bucket contains a nonempty string other than The bucket contains a nonempty string other than xx

(examine the next bucket) (circular rotation)(examine the next bucket) (circular rotation) Return to the home bucket Return to the home bucket htht[[ff((xx)], )],

if the table is full we report an error condition and exitif the table is full we report an error condition and exit

Overflow Handling (2/8)Overflow Handling (2/8) Additive transformation and DivisionAdditive transformation and Division

Hash table with linear probing (13 buckets, 1 slot/bucket)

insertion

Overflow Handling (3/8)Overflow Handling (3/8) Problem of Linear ProbingProblem of Linear Probing

Identifiers tend to cluster togetherIdentifiers tend to cluster together Adjacent cluster tend to coalesceAdjacent cluster tend to coalesce Increase the search timeIncrease the search time Example: suppose we enter the Example: suppose we enter the

C built-in functions into a C built-in functions into a 26-bucket hash table in order. 26-bucket hash table in order. The hash function uses the first The hash function uses the first character in each function namecharacter in each function name

acosacos, , atoiatoi, , charchar, , definedefine, , expexp, , ceilceil, , coscos, , floatfloat, , atolatol, , floorfloor, , ctimectime

Hash table with linear probing (26 buckets, 1 slot/bucket)

Enter:

Enter sequence:

acosacosatoiatoicharchardefinedefineexpexpceilceilcoscosfloatfloatatolatolfloorfloorctimectime

# of key comparisons=35/11=3.18

Overflow Handling (4/8)Overflow Handling (4/8)

Alternative techniques to improve open Alternative techniques to improve open addressing approach:addressing approach: Quadratic probingQuadratic probing rehashingrehashing random probingrandom probing

RehashingRehashing Try Try ff11, , ff22, …, , …, ffmm in sequence if collision occurs in sequence if collision occurs

disadvantagedisadvantage comparison of identifiers with different hash valuescomparison of identifiers with different hash values use chain to resolve collisionsuse chain to resolve collisions

Overflow Handling (5/8)Overflow Handling (5/8) Quadratic ProbingQuadratic Probing

Linear probing searches buckets (Linear probing searches buckets (ff((xx)+)+ii)%)%bb Quadratic probing uses a quadratic function of Quadratic probing uses a quadratic function of ii as the as the

incrementincrement Examine buckets Examine buckets ff((xx), (), (ff((xx)+)+ii22)%)%bb, (, (ff((xx)-)-ii22)%)%bb, for 1<=, for 1<=ii

<=(<=(bb-1)/2-1)/2 When When bb is a prime number of the form is a prime number of the form

44jj+3, +3, jj is an integer, the quadratic search is an integer, the quadratic search examines every bucket in the tableexamines every bucket in the table

Prime j Prime j

3 0 43 10

7 1 59 14

11 2 127 31

19 4 251 62

23 5 503 125

31 7 1019 254

Overflow Handling (6/8)Overflow Handling (6/8) ChainingChaining

Linear probing and its variations perform poorly becauLinear probing and its variations perform poorly because inserting an identifier requires the comparison of idse inserting an identifier requires the comparison of identifiers with different hash values.entifiers with different hash values.

In this approach we maintained a list of synonyms for In this approach we maintained a list of synonyms for each bucket.each bucket.

To insert a new elementTo insert a new element Compute the hash address Compute the hash address ff ( (xx)) Examine the identifiers in the list for Examine the identifiers in the list for ff((xx).).

Since we would not know the sizes of the lists in advaSince we would not know the sizes of the lists in advance, we should nce, we should maintain them as lined chainsmaintain them as lined chains

Overflow Handling (7/8)Overflow Handling (7/8) Results of Hash ChainingResults of Hash Chaining

acos, atoi, char, define, exp, ceil, cos, float, atol, floor, ctimef (x)=first character of x

# of key comparisons=21/11=1.91

Overflow Handling (8/8)Overflow Handling (8/8) Comparison:Comparison: In Figure 8.7, The values in each column give the average In Figure 8.7, The values in each column give the average

number of bucket accesses made in searching eight number of bucket accesses made in searching eight different table with 33,575, 24,050, 4909, 3072, 2241, different table with 33,575, 24,050, 4909, 3072, 2241, 930, 762, and 500 identifiers each.930, 762, and 500 identifiers each.

Chaining performs better than linear open addressing.Chaining performs better than linear open addressing. We can see that division is generally superiorWe can see that division is generally superior

Average number of bucket accesses per identifier retrieved

Dynamic HashingDynamic Hashing

Dynamic hashing using directoriesDynamic hashing using directories Analysis of directory dynamic hashingAnalysis of directory dynamic hashing

simulationsimulation

Directoryless dynamic hashingDirectoryless dynamic hashing

Dynamic Hashing Using DirectoriesDynamic Hashing Using Directories

Dynamic Hashing Using DirectoriesDynamic Hashing Using Directories

Dynamic Hashing Using DirectoriesDynamic Hashing Using Directories

Program8.5 Dynamic hashingProgram8.5 Dynamic hashing

Analysis of Directory Dynamic HashingAnalysis of Directory Dynamic Hashing

Directoryless Dynamic HashingDirectoryless Dynamic Hashing

Directoryless Dynamic HashingDirectoryless Dynamic Hashing

Directoryless Dynamic HashingDirectoryless Dynamic Hashing

Directoryless Dynamic HashingDirectoryless Dynamic Hashing