Advanced Data Structures

PETER BRASS

City College of New York

CAMBRIDGE UNIVERSITY PRESS

Cambridge, New York, Melbourne, Madrid, Cape Town, Singapore, São Paulo

Cambridge University Press

The Edinburgh Building, Cambridge CB2 8RU, UK

First published in print format

ISBN-13 978-0-521-88037-4

ISBN-13 978-0-511-43685-7

© Peter Brass 2008

2008

Information on this title: www.cambridge.org/9780521880374

This publication is in copyright. Subject to statutory exception and to the

provision of relevant collective licensing agreements, no reproduction of any part

may take place without the written permission of Cambridge University Press.

Cambridge University Press has no responsibility for the persistence or accuracy

of urls for external or third-party internet websites referred to in this publication,

and does not guarantee that any content on such websites is, or will remain,

accurate or appropriate.

Published in the United States of America by Cambridge University Press, New York

www.cambridge.org

eBook (EBL)

hardback

Contents

Preface page xi

1 Elementary Structures 11.1 Stack 11.2 Queue 81.3 Double-Ended Queue 161.4 Dynamical Allocation of Nodes 161.5 Shadow Copies of Array-Based Structures 18

2 Search Trees 232.1 Two Models of Search Trees 232.2 General Properties and Transformations 262.3 Height of a Search Tree 292.4 Basic Find, Insert, and Delete 312.5 Returning from Leaf to Root 352.6 Dealing with Nonunique Keys 372.7 Queries for the Keys in an Interval 382.8 Building Optimal Search Trees 402.9 Converting Trees into Lists 472.10 Removing a Tree 48

3 Balanced Search Trees 503.1 Height-Balanced Trees 503.2 Weight-Balanced Trees 613.3 (a, b)- and B-Trees 723.4 Red-Black Trees and Trees of Almost Optimal Height 893.5 Top-Down Rebalancing for Red-Black Trees 1013.6 Trees with Constant Update Time at a Known Location 1113.7 Finger Trees and Level Linking 114

vii

viii Contents

3.8 Trees with Partial Rebuilding: Amortized Analysis 1193.9 Splay Trees: Adaptive Data Structures 1223.10 Skip Lists: Randomized Data Structures 1353.11 Joining and Splitting Balanced Search Trees 143

4 Tree Structures for Sets of Intervals 1484.1 Interval Trees 1484.2 Segment Trees 1544.3 Trees for the Union of Intervals 1624.4 Trees for Sums of Weighted Intervals 1694.5 Trees for Interval-Restricted Maximum Sum Queries 1744.6 Orthogonal Range Trees 1824.7 Higher-Dimensional Segment Trees 1964.8 Other Systems of Building Blocks 1994.9 Range-Counting and the Semigroup Model 2024.10 kd-Trees and Related Structures 204

5 Heaps 2095.1 Balanced Search Trees as Heaps 2105.2 Array-Based Heaps 2145.3 Heap-Ordered Trees and Half-Ordered Trees 2215.4 Leftist Heaps 2275.5 Skew Heaps 2355.6 Binomial Heaps 2395.7 Changing Keys in Heaps 2485.8 Fibonacci Heaps 2505.9 Heaps of Optimal Complexity 2625.10 Double-Ended Heap Structures and Multidimensional

Heaps 2675.11 Heap-Related Structures with Constant-Time Updates 271

6 Union-Find and Related Structures 2786.1 Union-Find: Merging Classes of a Partition 2796.2 Union-Find with Copies and Dynamic Segment Trees 2936.3 List Splitting 3036.4 Problems on Root-Directed Trees 3066.5 Maintaining a Linear Order 317

7 Data Structure Transformations 3217.1 Making Structures Dynamic 3217.2 Making Structures Persistent 330

Contents ix

8 Data Structures for Strings 3358.1 Tries and Compressed Tries 3368.2 Dictionaries Allowing Errors in Queries 3568.3 Suffix Trees 3608.4 Suffix Arrays 367

9 Hash Tables 3749.1 Basic Hash Tables and Collision Resolution 3749.2 Universal Families of Hash Functions 3809.3 Perfect Hash Functions 3919.4 Hash Trees 3979.5 Extendible Hashing 3989.6 Membership Testers and Bloom Filters 402

10 Appendix 40610.1 The Pointer Machine and Alternative Computation

Models 40610.2 External Memory Models and Cache-Oblivious

Algorithms 40810.3 Naming of Data Structures 40910.4 Solving Linear Recurrences 41010.5 Very Slowly Growing Functions 412

11 References 415

Author Index 441

Subject Index 455

Preface

This book is a graduate-level textbook on data structures. A data structure isa method1 to realize a set of operations on some data. The classical exampleis to keep track of a set of items, the items identified by key values, so thatwe can insert and delete (key, item) pairs into the set and find the item with agiven key value. A structure supporting these operations is called a dictionary.Dictionaries can be realized in many different ways, with different complexitybounds and various additional operations supported, and indeed many kinds ofdictionaries have been proposed and analyzed in literature, and some will bestudied in this book.

In general, a data structure is a kind of higher-level instruction in a virtualmachine: when an algorithm needs to execute some operations many times, itis reasonable to identify what exactly the needed operations are and how theycan be realized in the most efficient way. This is the basic question of datastructures: given a set of operations whose intended behavior is known, howshould we realize that behavior?

There is no lack of books carrying the words “data structures” in the title, butthey merely scratch the surface of the topic, providing only the trivial structuresstack and queue, and then some balanced search tree with a large amount ofhandwaving. Data structures started receiving serious interest in the 1970s, and,in the first half of the 1980s, almost every issue of the Communications of theACM contained a data structure paper. They were considered a central topic,received their own classification in the Computing Subject Classification,2

1 This is not a book on object-oriented programming. I use the words “method” and “object” intheir normal sense.

2 Classification code: E.1 data structures. Unfortunately, the Computing Subject Classification istoo rough to be useful.

xi

xii Preface

and became a standard part of computer science curricula.3 Wirth titled abook Data Structures + Algorithms = Programs, and Algorithms and DataStructures became a generic textbook title. But the only monograph on an al-gorithmic aspect of data structures is the book by Overmars (1983) (which isstill in print, a kind of record for an LNCS series book). Data structures re-ceived attention in a number of application areas, foremost as index structuresin databases. In this context, structures for geometric data have been studied inthe monographs of Samet (1990, 2006); the same structures were studiedin the computer graphics context in Langetepe and Zachmann (2006). Re-cently, motivated by bioinformatics applications, string data structures havebeen much studied. There is a steady stream of publications on data structuretheory as part of computational geometry or combinatorial optimization. Butin the numerous textbooks, data structures are only viewed as an example ap-plication of object-oriented programming, excluding the algorithmic questionsof how to really do something nontrivial, with bounds on the worst-case com-plexity. It is the aim of this book to bring the focus back to data structures as afundamental subtopic of algorithms. The recently published Handbook of DataStructures (Mehta and Sahni 2005) is a step in the same direction.

This book contains real code for many of the data structures we discuss andenough information to implement most of the data structures where we do notprovide an implementation. Many textbooks avoid the details, which is onereason that the structures are not used in the places where they should be used.The selection of structures treated in this book is therefore restricted almosteverywhere to such structures that work in the pointer-machine model, withthe exception of hash tables, which are included for their practical importance.The code is intended as illustration, not as ready-to-use plug-in code; there iscertainly no guarantee of correctness. Most of it is available with a minimaltesting environment on my homepage.

This book started out as notes for a course I gave in the 2000 winter semesterat the Free University Berlin; I thank Christian Knauer, who was my assistantfor that course: we both learned a lot. I offered this course again in the fallsemesters of 2004–7 as a graduate course at the City College of New Yorkand used it as a base for a summer school on data structures at the KoreanAdvanced Institute of Science and Technology in July 2006. I finished thisbook in November 2007.

3 ABET still lists them as one of five core topics: algorithms, data structures, software design,programming languages, and computer architecture.

Preface xiii

I thank Emily Voytek and Günter Rote for finding errors in my code ex-amples, Otfried Cheong for organizing the summer school at KAIST, andthe summer school’s participants for finding further errors. I thank ChristianKnauer and Helmut Brass for literature from excellent mathematical libraries atthe Free University Berlin and Technical University Braunschweig, and JánosPach for access to the online journals subscribed by the Courant Institute. Aproject like this book would not have been possible without access to goodlibraries, and I tried to cite only those papers that I have seen.

This book project has not been supported by any grant-giving agency.

Basic Concepts

A data structure models some abstract object. It implements a number ofoperations on this object, which usually can be classified into

– creation and deletion operations,– update operations, and– query operations.

In the case of the dictionary, we want to create or delete the set itself, update theset by inserting or deleting elements, and query for the existence of an elementin the set.

Once it has been created, the object is changed by the update operations.The query operations do not change the abstract object, although they mightchange the representation of the object in the data structure: this is called anadaptive data structure – it adapts to the query to answer future similar queriesfaster.

Data structures that allow updates and queries are called dynamic datastructures. There are also simpler structures that are created just once forsome given object and allow queries but no updates; these are called staticdata structures. Dynamic data structures are preferable because they are moregeneral, but we also need to discuss static structures because they are usefulas building blocks for dynamic structures, and, for some of the more complexobjects we encounter, no dynamic structure is known.

We want to find data structures that realize a given abstract object and arefast. The size of structures is another quality measure, but it is usually of lessimportance. To express speed, we need a measure of comparison; this is thesize of the underlying object, not our representation of that object. Notice thata long sequence of update operations can still result in a small object. Our

xiv Preface

usual complexity measure is the worst-case complexity; so an operation in aspecific data structure has a complexity O(f (n)) if, for any state of the datastructure reached by a sequence of update operations that produced an object ofsize n, this operation takes at most time Cf (n) for some C. An alternative butweaker measure is the amortized complexity; an update operation has amortizedcomplexity O(f (n)) if there is some function g(n) such that any sequence ofm of these operations, during which the size of the underlying object is neverlarger than n, takes at most time g(n) + mCf (n), so in the average over a longsequence of operations the complexity is bounded by Cf (n).

Some structures are randomized, so the data structure makes some randomchoices, and the same object and sequence of operations do not always leadto the same steps of the data structure. In that case we analyze the expectedcomplexity of an operation. This expectation is over the random choices of thedata structure; the complexity is still the worst case of that expectation over allobjects of that size and possible operations.

In some situations, we cannot expect a nontrivial complexity bound of typeO(f (n)) because the operation might give a large answer. The size of the answeris the output complexity of the operation, and, for operations that sometimeshave a large output complexity, we are interested in output-sensitive methods,which are fast when the output is small. An operation has output-sensitivecomplexity O(f (n) + k) if, on an object of size n that requires an output ofsize k, the operation takes at most time C(f (n) + k).

For dynamic data structures, the time to create the structure for an emptyobject is usually constant, so we are mainly interested in the update and querytimes. The time to delete a structure of size n is almost always O(n). For staticdata structures we already create an object of size n, so there we are interestedin the creation time, known as preprocessing time, and the query time.

In this book, loga n denotes the logarithm to base a; if no base is specified,we use base 2.

We use the Bourbaki-style notation for closed, half-open, and open intervals,where [a, b] is the closed interval from a to b, ]a, b[ is the open interval, andthe half-open intervals are ]a, b], missing the first point, and [a, b[, missing thelast point.

Similar to the O(·)-notation for upper bounds mentioned earlier, we also usethe �(·) for lower bounds and �(·) for simultaneous upper and lower bounds.A nonnegative function f is O(g(n)), or �(g(n)), if for some positive C and allsufficiently large n holds f (n) ≤ Cg(n), or f (n) ≥ Cg(n), respectively. Andf is �(g(n)) if it is simultaneously O(g(n)) and �(g(n)). Here “sufficientlylarge” means that g(n) needs to be defined and positive.

Preface xv

Code Examples

The code examples in this book are given in standard C. For the readers usedto some other imperative programming language, most constructs are self-explanatory.

In the code examples, = denotes the assignment and == the equality test.Outside the code examples, we will continue to use = in the normal way.

The Boolean operators for “not,” “and,” “or” are !, &&, ||, respectively,and % denotes the modulo operator.

Pointers are dereferenced with *, so if pt is a pointer to a memory location(usually a variable), then *pt is that memory location. Pointers have a type todetermine how the content of that memory location should be interpreted. Todeclare a pointer, one declares the type of the memory location it points to, so“int *pt;” declares pt to be a pointer to an int. Pointers are themselvesvariables; they can be assigned, and it is also possible to add integers to apointer (pointer arithmetic). If pt points to a memory object of a certain type,then pt+1 points to the next memory location for an object of that type; this isequivalent to treating the memory as a big array of objects of that type. NULLis a pointer that does not point to any valid memory object, so it can be used asa special mark in comparisons.

Structures are user-defined data types that have several components. Thecomponents themselves have a type and a name, and they can be of any type,including other structures. The structure cannot have itself as a type of acomponent, because that would generate an unbounded recursion. But it canhave a pointer to an object of its own type as component; indeed, such structuresare the main tool of data structure theory. A variable whose type is a structurecan be assigned and used like any other variable. If z is a variable of type C,and we define this type by

typedef struct { float x; float y; } C,

then the components of z are z.x and z.y, which are two variables of typefloat. If zpt is declared as pointer to an object of type C (by C *zpt;),then the components of the object that zpt points to are (*zpt).x and(*zpt).y. Because this is a frequently used combination, dereferencing apointer and selecting a component, there is an alternative notation zpt->xand zpt->y. This is equivalent, but preferable, because it avoids the operatorpriority problem: dereferencing has lower priority than component selection,so (*zpt).x is not the same as *zpt.x.

We avoid writing the functions recursively, although in some cases this mightsimplify the code. But the overhead of a recursive function call is significant

xvi Preface

and thus conflicts with the general aim of highest efficiency in data structures.We do not practice any similar restrictions for nonrecursive functions; a goodcompiler will expand them as inline functions, avoiding the function call, orthey could be written as macro functions.

In the text we will also frequently use the name of a pointer for the objectto which it points.

1

Elementary Structures

Elementary data structures usually treated in the “Programming 2” class arethe stack and the queue. They have a common generalization, the double-ended queue, which is also occasionally mentioned, although it has far fewerapplications. Stack and queue are very fundamental structures, so they willbe discussed in detail and used to illustrate several points in data structureimplementation.

1.1 Stack

The stack is the simplest of all structures, with an obvious interpretation: puttingobjects on the stack and taking them off again, with access possible only to thetop item. For this reason they are sometimes also described as LIFO storage:last in, first out. Stacks occur in programming wherever we have nested blocks,local variables, recursive definitions, or backtracking. Typical programmingexercises that involve a stack are the evaluation of arithmetic expressions withparentheses and operator priorities, or search in a labyrinth with backtracking.

The stack should support at least the following operations:

{ push( obj ): Put obj on the stack, making it the top item.{ pop(): Return the top object from the stack and remove it from the stack.{ stack empty(): Test whether the stack is empty.

Also, the realization of the stack has, of course, to give the right values,so we need to specify the correct behavior of a stack. One method would bean algebraic specification of what correct sequences of operations and returnvalues are. This has been done for simple structures like the stack, but eventhen the specification is not very helpful in understanding the structure. Instead,we can describe a canonical implementation on an idealized machine, whichgives the correct answer for all correct sequences of operations (no pop on an

1

2 1 Elementary Structures

empty stack, no memory problems caused by bounded arrays). Assuming thatthe elements we want to store on the stack are of type item t, this could lookas follows:

int i=0;item_t stack[∞];

int stack_empty(void){ return( i == 0 );}

void push( item_t x){ stack[i++] = x ;}

item_t pop(void){ return( stack[ --i] );}

This describes the correct working of the stack, but we have the problemof assuming both an infinite array and that any sequence of operations will becorrect. A more realistic version might be the following:

int i=0;item_t stack[MAXSIZE];

int stack_empty(void){ return( i == 0 );}

int push( item_t x){ if ( i < MAXSIZE )

{ stack[i++] = x ; return( 0 );}else

return( -1 );}

item_t pop(void){ return( stack[ --i] );}

1.1 Stack 3

This now limits the correct behavior of the stack by limiting the maximumnumber of items on the stack at one time, so it is not really the correct stackwe want, but at least it does specify an error message in the return value ifthe stack overflow is reached by one push too many. This is a fundamentaldefect of array-based realizations of data structures: they are of fixed size,the size needs to be decided in advance, and the structure needs the full sizeno matter how many items are really in the structure. There is a systematicway to overcome these problems for array-based structures, which we will seein Section 1.5, but usually a solution with dynamically allocated memory ispreferable.

We specified an error value only for the stack overflow condition, but notfor the stack underflow, because the stack overflow is an error generated bythe structure, which would not be present in an ideal implementation, whereasa stack underflow is an error in the use of the structure and so a result in theprogram that uses the stack as a black box. Also, this allows us to keep thereturn value of pop as the top object from the stack; if we wanted to catchstack underflow errors in the stack implementation, we would need to returnthe object and the error status. A final consideration in our first stack versionis that we might need multiple stacks in the same program, so we want tocreate the stacks dynamically. For this we need additional operations to createand remove a stack, and each stack operation needs to specify which stack itoperates on. One possible implementation could be the following:

typedef struct {item_t *base; item_t *top;int size;} stack_t;

stack_t *create_stack(int size){ stack_t *st;

st = (stack_t *) malloc( sizeof(stack_t) );st->base = (item_t *) malloc( size *

sizeof(item_t) );st->size = size;st->top = st->base;return( st );

}

int stack_empty(stack_t *st){ return( st->base == st->top );}

4 1 Elementary Structures

int push( item_t x, stack_t *st){ if ( st->top < st->base + st->size )

{ *(st->top) = x; st->top += 1; return( 0 );}else

return( -1 );}

item_t pop(stack_t *st){ st->top -= 1;

return( *(st->top) );}

item_t top_element(stack_t *st){ return( *(st->top -1) );}

void remove_stack(stack_t *st){ free( st->base );

free( st );}

Again, we include some security checks and leave out others. Our policyin general is to include those security checks that test for errors introducedby the limitations of this implementation as opposed to an ideal stack, butto assume both that the use of the stack is correct and that the underlyingoperating system never runs out of memory. We included another operationthat is frequently useful, which just returns the value of the top element withouttaking it from the stack.

Frequently, the preferable implementation of the stack is a dynamicallyallocated structure using a linked list, where we insert and delete in front ofthe list. This has the advantage that the structure is not of fixed size; therefore,we need not be prepared for stack overflow errors if we can assume that thememory of the computer is unbounded, and so we can always get a new node.It is as simple as the array-based structure if we already have the get nodeand return node functions, whose correct implementation we discuss inSection 1.4.

typedef struct st_t { item_t item;struct st_t *next; } stack_t;

1.1 Stack 5

stack_t *create_stack(void){ stack_t *st;

st = get_node();st->next = NULL;return( st );

}

int stack_empty(stack_t *st){ return( st->next == NULL );}

void push( item_t x, stack_t *st){ stack_t *tmp;

tmp = get_node();tmp->item = x;tmp->next = st->next;st->next = tmp;

}

item_t pop(stack_t *st){ stack_t *tmp; item_t tmp_item;

tmp = st->next;st->next = tmp->next;tmp_item = tmp->item;return_node( tmp );return( tmp_item );

}

item_t top_element(stack_t *st){ return( st->next->item );}

void remove_stack(stack_t *st){ stack_t *tmp;

do{ tmp = st->next;

return_node(st);st = tmp;

}while ( tmp != NULL );

}

Notice that we have a placeholder node in front of the linked list; even anempty stack is represented by a list with one node, and the top of the stack is

6 1 Elementary Structures

only the second node of the list. This is necessary as the stack identifier returnedby create stack and used in all stack operations should not be changed bythe stack operations. So we cannot just use a pointer to the start of the linkedlist as a stack identifier. Because the components of a node will be invalid afterit is returned, we need temporary copies of the necessary values in pop andremove stack. The operation remove stack should return all the remain-ing nodes; there is no reason to assume that only empty stacks will be removed,and we will suffer a memory leak if we fail to return the remaining nodes.

next

item

next

item

next

item

next

item

placeholder top of stack

Stack Realized as List, with Three Items

The implementation as a dynamically allocated structure always has theadvantage of greater elegance; it avoids stack overflow conditions and needsjust the memory proportional to the actually used items, not a big array of a sizeestimated by the programmer as upper bound to the maximum use expectedto occur. One disadvantage is a possible decrease in speed: dereferencing apointer does not take longer than incrementing an index, but the memorylocation accessed by the pointer might be anywhere in memory, whereas thenext component of the array will be near the previous component. Thus, array-based structures usually work very well with the cache, whereas dynamicallyallocated structures might generate many cache misses. So if we are quite certainabout the maximum possible size of the stack, for example, because its size isonly logarithmic in the size of the input, we will prefer an array-based version.

If one wants to combine these advantages, one could use a linked list ofblocks, each block containing an array, but when the array becomes full, wejust link it to a new node with a new array. Such an implementation could lookas follows:

typedef struct st_t { item_t *base;item_t *top;int size;

struct st_t *previous;} stack_t;

stack_t *create_stack(int size){ stack_t *st;

st = (stack_t *) malloc( sizeof(stack_t) );st->base = (item_t *) malloc( size *

sizeof(item_t) );st->size = size;st->top = st->base;

1.1 Stack 7

st->previous = NULL;return( st );

}

int stack_empty(stack_t *st){ return( st->base == st->top &&

st->previous == NULL);}

void push( item_t x, stack_t *st){ if ( st->top < st->base + st->size )

{ *(st->top) = x; st->top += 1;}else{ stack_t *new;

new = (stack_t *) malloc( sizeof(stack_t) );new->base = st->base;new->top = st->top;new->size = st->size;new->previous = st->previous;st->previous = new;st->base = (item_t *) malloc( st->size *

sizeof(item_t) );st->top = st->base+1;*(st->base) = x;

}}

item_t pop(stack_t *st){ if( st->top == st->base )

{ stack_t *old;old = st->previous;st->previous = old->previous;free( st->base );st->base = old->base;st->top = old->top;st->size = old->size;free( old );

}st->top -= 1;return( *(st->top) );

}

item_t top_element(stack_t *st){ if( st->top == st->base )

return( *(st->previous->top -1) );

8 1 Elementary Structures

elsereturn( *(st->top -1) );

}

void remove_stack(stack_t *st){ stack_t *tmp;

do{ tmp = st->previous;

free( st->base );free( st );st = tmp;

}while( st != NULL );

}

In our classification, push and pop are update operations andstack empty and top element are query operations. In the array-basedimplementation, it is obvious that we can do all the operations in constanttime as they involve only a constant number of elementary operations. For thelinked-list implementation, the operations involve the external get node andreturn node functions, which occur in both push and pop once, so theimplementation works only in constant time if we can assume these functionsto be constant-time operations. We will discuss the implementation of thisdynamic node allocation in Section 1.4, but we can assume here (and in all laterstructures) that this works in constant time. For the block list we allocate largeparts of memory for which we used here the standard memory managementoperations malloc and free instead of building an intermediate layer, asdescribed in Section 1.4. It is traditional to assume that memory allocation anddeallocation are constant-time operations, but especially with the free thereare nontrivial problems with a constant-time implementation, so one shouldavoid using it frequently. This could happen in the block list variant if thereare many push/pop pairs that just go over a block boundary. So the smalladvantage of the block list is probably not worth the additional problems.

The create stack operation involves only one such memory alloca-tion, and so that should be constant time in each implementation; but theremove stack operation is clearly not constant time, because it has to de-stroy a potentially large structure. If the stack still contains n elements, theremove stack operation will take time O(n).

1.2 Queue

The queue is a structure almost as simple as the stack; it also stores items,but it differs from the stack in that it returns those items first that have been

1.2 Queue 9

entered first, so it is FIFO storage (first in, first out). Queues are useful if thereare tasks that have to be processed cyclically. Also, they are a central structurein breadth-first search; breadth-first search (BFS) and depth-first search (DFS)really differ only in that BFS uses a queue and DFS uses a stack to store thenode that will be explored next.

The queue should support at least the following operations:

{ enqueue( obj ): Insert obj at the end of the queue, making it the lastitem.

{ dequeue(): Return the first object from the queue and remove it from thequeue.

{ queue empty(): Test whether the queue is empty.

The difference between queue and stack that makes the queue slightlymore difficult is that the changes occur at both ends: at one end, there areinserts; at the other, deletes. If we choose an array-based implementation forthe queue, then the part of the array that is in use moves through the array. Ifwe had an infinite array, this would present no problem. We could write it asfollows:

int lower=0; int upper=0;item_t queue[∞];

int queue_empty(void){ return( lower == upper );}

void enqueue( item_t x){ queue[upper++] = x ;}

item_t dequeue(void){ return( queue[ lower++] );}

A real implementation with a finite array has to wrap this around, usingindex calculation modulo the length of the array. It could look as follows:

typedef struct {item_t *base;int front;int rear;int size;} queue_t;

10 1 Elementary Structures

queue_t *create_queue(int size){ queue_t *qu;

qu = (queue_t *) malloc( sizeof(queue_t) );qu->base = (item_t *) malloc( size *

sizeof(item_t) );qu->size = size;qu->front = qu->rear = 0;return( qu );

}

int queue_empty(queue_t *qu){ return( qu->front == qu->rear );}

int enqueue( item_t x, queue_t *qu){ if ( qu->front != ((qu->rear +2)% qu->size) )

{ qu->base[qu->rear] = x;qu->rear = ((qu->rear+1)%qu->size);return( 0 );

}else

return( -1 );}

item_t dequeue(queue_t *qu){ int tmp;

tmp = qu->front;qu->front = ((qu->front +1)%qu->size);return( qu->base[tmp] );

}

item_t front_element(queue_t *qu){ return( qu->base[qu->front] );}

void remove_queue(queue_t *qu){ free( qu->base );

free( qu );}

1.2 Queue 11

Again this has the fundamental disadvantage of any array-based structure –that it is of fixed size. So it possibly generates overflow errors and does notimplement the structure correctly as it limits it this way. In addition, it alwaysreserves this expected maximum size for the array, even if it never needs it. Thepreferred alternative is a dynamically allocated structure, with a linked list. Theobvious solution is the following:

typedef struct qu_n_t {item_t item;struct qu_n_t *next; } qu_node_t;

typedef struct {qu_node_t *remove;qu_node_t *insert; } queue_t;

queue_t *create_queue(){ queue_t *qu;

qu = (queue_t *) malloc( sizeof(queue_t) );qu->remove = qu->insert = NULL;return( qu );

}

int queue_empty(queue_t *qu){ return( qu->insert ==NULL );}

void enqueue( item_t x, queue_t *qu){ qu_node_t *tmp;

tmp = get_node();tmp->item = x;tmp->next = NULL; /* end marker */if ( qu->insert != NULL ) /* queue nonempty */{ qu->insert->next = tmp;

qu->insert = tmp;}else /* insert in empty queue */{ qu->remove = qu->insert = tmp;}

}

item_t dequeue(queue_t *qu){ qu_node_t *tmp; item_t tmp_item;

tmp = qu->remove; tmp_item = tmp->item;qu->remove = tmp->next;if( qu->remove == NULL ) /* reached end */

qu->insert = NULL; /* make queue empty */return_node(tmp);

12 1 Elementary Structures

return( tmp_item );}

item_t front_element(queue_t *qu){ return( qu->remove->item );}

void remove_queue(queue_t *qu){ qu_node_t *tmp;

while( qu->remove != NULL){ tmp = qu->remove;

qu->remove = tmp->next;return_node(tmp);

}free( qu );

}

Again we assume, as in all dynamically allocated structures, that the op-erations get node and return node are available, which always workcorrectly and in constant time. Because we want to remove items from the frontof the queue, the pointers in the linked list are oriented from the front to the end,where we insert items. There are two aesthetical disadvantages of this obviousimplementation: we need a special entry point structure, which is different fromthe list nodes, and we always need to treat the operations involving an emptyqueue differently. For insertions into an empty queue and removal of the lastelement of the queue, we need to change both insertion and removal pointers;for all other operations we change only one of them.

remove insert

next

item

next

item

next

item

next

item

Queue Realized as List, with Four Items

The first disadvantage can be avoided by joining the list together to make ita cyclic list, with the last pointer from the end of the queue pointing again tothe beginning. We can then do without a removal pointer, because the insertionpoint’s next component points to the removal point. By this, the entry point tothe queue needs only one pointer, so it is of the same type as the queue nodes.

The second disadvantage can be overcome by inserting a placeholder nodein that cyclic list, between the insertion end and the removal end of the cycliclist. The entry point still points to the insertion end or, in the case of an empty

1.2 Queue 13

list, to that placeholder node. Then, at least for the insert, the empty list is nolonger a special case. So a cyclic list version is the following:

typedef struct qu_t { item_t item;struct qu_t *next; } queue_t;

queue_t *create_queue(){ queue_t *entrypoint, *placeholder;

entrypoint = (queue_t *) malloc( sizeof(queue_t) );placeholder = (queue_t *) malloc( sizeof(queue_t) );entrypoint->next = placeholder;placeholder->next = placeholder;return( entrypoint );

}

int queue_empty(queue_t *qu){ return( qu->next == qu->next->next );}

void enqueue( item_t x, queue_t *qu){ queue_t *tmp, *new;

new = get_node(); new->item = x;tmp = qu->next; qu->next = new;new->next = tmp->next; tmp->next = new;

}

item_t dequeue(queue_t *qu){ queue_t *tmp;

item_t tmp_item;tmp = qu->next->next->next;qu->next->next->next = tmp->next;if( tmp == qu->next )

qu->next = tmp->next;tmp_item = tmp->item;return_node( tmp );return( tmp_item );

}

item_t front_element(queue_t *qu){ return( qu->next->next->next->item );}

void remove_queue(queue_t *qu){ queue_t *tmp;

tmp = qu->next->next;while( tmp != qu->next ){ qu->next->next = tmp->next;

return_node( tmp );

14 1 Elementary Structures

tmp = qu->next->next;}return_node( qu->next );return_node( qu );

}

next

item

next

item

next

item

next

item

next

item

placeholder

entrypoint

front of queue

Queue Realized as Cyclic List, with Three Items

Or one could implement the queue as a doubly linked list, which requires nocase distinctions at all but needs two pointers per node. Minimizing the numberof pointers is an aesthetic criterion more justified by the amount of work thathas to be done in each step to keep the structure consistent than by the amount ofmemory necessary for the structure. Here is a doubly linked list implementation:

typedef struct qu_t { item_t item;struct qu_t *next;struct qu_t *previous; } queue_t;

queue_t *create_queue(){ queue_t *entrypoint;

entrypoint = (queue_t *) malloc( sizeof(queue_t) );entrypoint->next = entrypoint;entrypoint->previous = entrypoint;return( entrypoint );

}

int queue_empty(queue_t *qu){ return( qu->next == qu );}

void enqueue( item_t x, queue_t *qu){ queue_t *new;

new = get_node(); new->item = x;new->next = qu->next; qu->next = new;new->next->previous = new; new->previous = qu;

}

item_t dequeue(queue_t *qu){ queue_t *tmp; item_t tmp_item;

tmp = qu->previous; tmp_item = tmp->item;

1.2 Queue 15

tmp->previous->next = qu;qu->previous = tmp->previous;return_node( tmp );return( tmp_item );

}

item_t front_element(queue_t *qu){ return( qu->previous->item );}

void remove_queue(queue_t *qu){ queue_t *tmp;

qu->previous->next = NULL;do{ tmp = qu->next;

return_node( qu );qu = tmp;

}while ( qu != NULL );

}

next

previous

item

next

previous

item

next

previous

item

next

previous

item

next

previous

item

insertionend

deletionend

entry point

Queue Realized as Doubly Linked List, with Four Items

Which of the list-based implementations one prefers is really a matter of taste;they are all slightly more complicated than the stack, although the two structureslook similar.

Like the stack, the queue is a dynamic data structure that has the updateoperations enqueue and dequeue and the query operations queue emptyand front element, all of which are constant-time operations, and theoperations create queue and delete queue, which are subject to thesame restrictions as the similar operations for the stack: creating an array-based queue requires getting a big block of memory from the underlying systemmemory management, whereas creating a list-based queue should require onlysome get node operations; and deleting an array-based queue just involves

16 1 Elementary Structures

returning that memory block to the system, whereas deleting a list-based queuerequires returning every individual node still contained in it, so it will take O(n)time to delete a list-based queue that still contains n items.

1.3 Double-Ended Queue

The double-ended queue is the obvious common generalization of stackand queue: a queue in which one can insert and delete at either end. Itsimplementation can be done as an array, or as a doubly linked list, just like aqueue; because it does not present any new problems, no code will be givenhere. The double-ended queue does not have many applications, but at leasta “one-and-a-half ended queue” sometimes is useful, as in the minqueue dis-cussed in Section 5.11.

1.4 Dynamical Allocation of Nodes

In the previous sections we used the operations get node and return nodeto dynamically create and delete nodes, that is, constant-sized memory objects,as opposed to the generic operations malloc and free provided by the stan-dard operating-system interface, which we used only for memory objects ofarbitrary, usually large, size. The reason for this distinction is that although theoperating-system memory allocation is ultimately the only way to get memory,it is a complicated process, and it is not even immediately obvious that it isa constant-time operation. In any efficient implementation of a dynamicallyallocated structure, where we permanently get and return nodes, we cannotafford to access this operating-system-level memory management in each op-eration. Instead, we introduce an intermediate layer, which only occasionallyhas to access the operating-system memory management to get a large memoryblock, which it then gives out and receives back in small, constant-sized pieces,the nodes.

The efficiency of these get node and return node operations is reallycrucial for any dynamically allocated structure, but luckily we do not haveto create a full memory management system; there are two essential simpli-fications. We deal only with objects of one size, as opposed to the mallocinterface, which should provide memory blocks of any size, and we do not re-turn any memory from the intermediate level to the system before the programends. This is reasonable: the amount of memory taken by the intermediate layerfrom the system is the maximum amount taken by the data structure up to that

1.4 Dynamical Allocation of Nodes 17

moment, so we do not overestimate the total memory requirement; we only failto free it earlier for other coexisting programs or structures.

This allows us to use the free list as a structure for our dynamical allocationof nodes. The free list contains all the nodes not currently in use; whenevera return node is executed, the node is just added to the free list. For theget node, the situation is slightly more complicated; if the free list is notempty, we may just take a node from there. If it is empty and the currentmemory block is not used up, we take a new node from that memory block.Otherwise, we have to get a new memory block with malloc and create thenode from there.

An implementation could look as follows:

typedef struct nd_t { struct nd_t *next;/*and other components*/ } node_t;

#define BLOCKSIZE 256node_t *currentblock = NULL;int size_left;node_t *free_list = NULL;

node_t *get_node(){ node_t *tmp;

if( free_list != NULL ){ tmp = free_list;

free_list = free_list -> next;}else{ if( currentblock == NULL || size_left == 0)

{ currentblock =(node_t *) malloc( BLOCKSIZE *

sizeof(node_t) );size_left = BLOCKSIZE;

}tmp = currentblock++;size_left -= 1;

}return( tmp );

}

void return_node(node_t *node){ node->next = free_list;

free_list = node;}

18 1 Elementary Structures

Dynamical memory allocation is traditionally a source of many program-ming errors and is hard to debug. A simple additional precaution to avoid somecommon errors is to add to the node another component, int valid, andfill it with different values, depending on whether it has just been receivedback by return node or is given out by get node. Then we can checkthat a pointer does indeed point to a valid node and that anything received byreturn node has indeed been a valid node up to that moment.

1.5 Shadow Copies of Array-Based Structures

There is a systematic way to avoid the maximum-size problem of array-basedstructures at the expense of the simplicity of these structures. We simultaneouslymaintain two copies of the structure, the currently active copy and a larger-sizedstructure which is under construction. We have to schedule the construction ofthe larger structure in such a way that it is finished and ready for use beforethe active copy reaches its maximum size. For this, we copy in each operationon the old structure a fixed number of items from the old to the new structure.When the content of the old structure is completely copied into the new, largerstructure, the old structure is removed and the new structure taken as the activestructure and, when necessary, construction of an even larger copy is begun.This sounds very simple and introduces only a constant overhead to converta fixed-size structure into an unlimited structure. There are, however, someproblems in the details: the structure that is being copied changes while thecopying is in progress, and these changes must be correctly done in the stillincomplete larger copy. To demonstrate the principle, here is the code for thearray-based stack:

typedef struct { item_t *base;int size;int max_size;item_t *copy;int copy_size; } stack_t;

stack_t *create_stack(int size){ stack_t *st;

st = (stack_t *) malloc( sizeof(stack_t) );st->base = (item_t *) malloc( size *

sizeof(item_t) );st->max_size = size;st->size = 0; st->copy = NULL; st->copy_size = 0;return( st );

}

1.5 Shadow Copies of Array-Based Structures 19

int stack_empty(stack_t *st){ return( st->size == 0);}

void push( item_t x, stack_t *st){ *(st->base + st->size) = x;

st->size += 1;if ( st->copy != NULL ||st->size >= 0.75*st->max_size ){ /* have to continue or start copying */

int additional_copies = 4;if( st->copy == NULL )/* start copying: allocate space */{ st->copy =

(item_t *) malloc( 2 * st->max_size *sizeof(item_t) );

}/* continue copying: at most 4 items

per push operation */while( additional_copies > 0 &&

st->copy_size < st->size ){ *(st->copy + st->copy_size) =

*(st->base + st->copy_size);st->copy_size += 1; additional_copies -= 1;

}if( st->copy_size == st->size)/* copy complete */{ free( st->base );

st->base = st-> copy;st->max_size *= 2;st->copy = NULL;st->copy_size = 0;

}}

}

item_t pop(stack_t *st){ item_t tmp_item;

st->size -= 1;tmp_item = *(st->base + st->size);if( st->copy_size == st->size) /* copy complete */{ free( st->base );

st->base = st-> copy;st->max_size *= 2;st->copy = NULL;st->copy_size = 0;

}

20 1 Elementary Structures

return( tmp_item );}

item_t top_element(stack_t *st){ return( *(st->base + st->size - 1) );}

void remove_stack(stack_t *st){ free( st->base );

if( st->copy != NULL )free( st->copy );

free( st );}

For the stack, the situation is especially easy because we can just copyfrom the base until we reach the current top; in between, nothing changes.The threshold when to start copying (here, at 0.75*size), the size of the newstructure (here, twice the previous size), and the number of items copied ineach step (here, four items) must, of course, be chosen in such a way thatcopying is complete before the old structure overflows. Note that we can reachthe situation when the copying is finished in two ways: by actual copying inthe push and by deleting uncopied items in the pop.

In general, the connection between copying threshold size, new maximumsize, and number of items copied is as follows:

{ if the current structure has maximum size smax,{ and we begin copying as soon as its actual size has reached αsmax (with

α ≥ 12 ),{ the new structure has maximum size 2smax, and{ each operation increases the actual size by at most 1,

then there are at least (1 − α)smax steps left to complete the copying of at mostsmax elements from the smaller structure to the new structure. So we need tocopy � 11−α � elements in each operation to finish the copying before the smallerstructure overflows. We doubled the maximum size when creating the newstructure, but we could have chosen any size βsmax, β > 1, as long as αβ > 1.Otherwise, we would have to start copying again before the previous copyingprocess was finished.

In principle, this technique is quite general and not restricted to array-basedstructures. We will use it again in Sections 3.6 and 7.1. We can always try toovercome the size limitation of a fixed-size structure by copying its content to alarger structure. But it is not always clear how to break this copying into many

1.5 Shadow Copies of Array-Based Structures 21

small steps that can be executed simultaneously with the normal operations onthe structure, as in our example. Instead, we have to copy the entire structurein one step, so we cannot get a worst-case time bound, but only an amortizedbound.

A final example of this technique and its difficulties is the realization of anextendible array. Normal arrays need to be declared of a fixed size, they areallocated somewhere in memory, and the space that is reserved there cannot beincreased as it might conflict with space allocated for other variables. Accessto an array element is very fast; it is just one address computation. But somesystems also support a different type of array, which can be made larger; forthese, accessing an element is more complicated and it is really an operationof a nontrivial data structure. This structure needs to support the followingoperations:

{ create array creates an array of a given size,{ set value assigns the array element at a given index a value,{ get value returns the value of the array element at a given index,{ extend array increases the length of the array.

To implement that structure, we use the same technique of building shadowcopies. There is, however, an additional problem here, because the structurewe want to model does not just grow by a single item in each operation; theextend array operation can make it much larger a single operation. Still,we can easily achieve an amortized constant time per operation.

When an array of size s is created, we allocate space for it, but more thanrequested. We maintain that the size of the arrays we actually allocate is alwaysa power of 2, so we initially allocate an array of size 2�log s� and store thestart position of that array, as well as the current and the maximum size, in astructure that identifies the array. Any access to an array element first has tolook up that start position of the current array. Each time an extend arrayoperation is performed, we first check whether the current maximum size islarger than the requested size; in that case we can just increase the current size.Else, we have to allocate a new array whose size is the next number 2k largerthan the requested size, and copy every item from the old array to the newarray. Thus, accessing an array element is always done in O(1) time; it is justone in the direction of the pointer; but extending the array can take linear timein the size of the array. But the amortized complexity is not that bad; if theultimate size of the array is 2�log k�, then we have at worst copied arrays of size1, 2, 4, . . . , 2�log k�−1, so we spent in total time O(1 + 2 + · · · + 2�log k�−1) =O(k) with those extend array operations that did copy the array, and O(1)

22 1 Elementary Structures

with each extend array operation that did not copy the array. Thus, wehave the following complexity:

Theorem. An extendible array structure with shadow copies performs anysequence of n set value, get value, and extend array operations onan array whose final size is k in time O(n + k).

If we assume that each element of the array we request is also accessedat least once, so that the final size is at most the number of element accessoperations, this gives an amortized O(1) complexity per operation.

It would be natural to distribute the copying of the elements again overthe later access operations, but we have no control over the extend arrayoperations. It is possible that the next extension is requested before the copyingof the current array is complete, so our previous method does not work for thisstructure. Another conceptual problem with extendible arrays is that pointersto array elements are different from normal pointers because the position of thearray can change. Thus, in general, extendible arrays should be avoided evenif the language supports them. A different way to implement extendible arrayswas discussed in Challab (1991).

2

Search Trees

A search tree is a structure that stores objects, each object identified by a keyvalue, in a tree structure. The key values of the objects are from a linearlyordered set (typically integers); two keys can be compared in constant time andthese comparisons are used to guide the access to a specific object by its key.The tree has a root, where any search starts, and then contains in each nodesome key value for comparison with the query key, so one can go to differentnext nodes depending on whether the query key is smaller or larger than thekey in the node until one finds a node that contains the right key.

This type of tree structure is fundamental to most data structures; it allowsmany variations and is also a building block for most more complex datastructures. For this reason we will discuss it in great detail.

Search trees are one method to implement the abstract structure calleddictionary. A dictionary is a structure that stores objects, identified by keys, andsupports the operations find, insert, and delete. A search tree usually supports atleast these operations of a dictionary, but there are also other ways to implementa dictionary, and there are applications of search trees that are not primarilydictionaries.

2.1 Two Models of Search Trees

In the outline just given, we supressed an important point that at first seemstrivial, but indeed it leads to two different models of search trees, either ofwhich can be combined with much of the following material, but one of whichis strongly preferable.

If we compare in each node the query key with the key contained in thenode and follow the left branch if the query key is smaller and the right branch

23

24 2 Search Trees

if the query key is larger, then what happens if they are equal? The two modelsof search trees are as follows:

1. Take left branch if query key is smaller than node key; otherwise take theright branch, until you reach a leaf of the tree. The keys in the interior nodeof the tree are only for comparison; all the objects are in the leaves.

2. Take left branch if query key is smaller than node key; take the right branchif the query key is larger than the node key; and take the object containedin the node if they are equal.

This minor point has a number of consequences:

{ In model 1, the underlying tree is a binary tree, whereas in model 2, eachtree node is really a ternary node with a special middle neighbor.

{ In model 1, each interior node has a left and a right subtree (each possibly aleaf node of the tree), whereas in model 2, we have to allow incompletenodes, where left or right subtree might be missing, and only thecomparison object and key are guaranteed to exist.

So the structure of a search tree of model 1 is more regular than that of a treeof model 2; this is, at least for the implementation, a clear advantage.

{ In model 1, traversing an interior node requires only one comparison,whereas in model 2, we need two comparisons to check the threepossibilities.

Indeed, trees of the same height in models 1 and 2 contain at most approximatelythe same number of objects, but one needs twice as many comparisons in model2 to reach the deepest objects of the tree. Of course, in model 2, there are alsosome objects that are reached much earlier; the object in the root is foundwith only two comparisons, but almost all objects are on or near the deepestlevel.

Theorem. A tree of height h and model 1 contains at most 2h objects.A tree of height h and model 2 contains at most 2h+1 − 1 objects.

This is easily seen because the tree of height h has as left and right subtrees atree of height at most h − 1 each, and in model 2 one additional object betweenthem.

{ In model 1, keys in interior nodes serve only for comparisons and mayreappear in the leaves for the identification of the objects. In model 2, eachkey appears only once, together with its object.

2.1 Two Models of Search Trees 25

It is even possible in model 1 that there are keys used for comparison thatdo not belong to any object, for example, if the object has been deleted. Byconceptually separating these functions of comparison and identification, thisis not surprising, and in later structures we might even need to define artificialtests not corresponding to any object, just to get a good division of the searchspace. All keys used for comparison are necessarily distinct because in a model1 tree, each interior node has nonempty left and right subtrees. So each keyoccurs at most twice, once as comparison key and once as identification key inthe leaf.

Model 2 became the preferred textbook version because in most textbooksthe distinction between object and its key is not made: the key is the object.Then it becomes unnatural to duplicate the key in the tree structure. But inall real applications, the distinction between key and object is quite important.One almost never wishes to keep track of just a set of numbers; the numbersare normally associated with some further information, which is often muchlarger than the key itself.

In some literature, where this distinction is made, trees of model 1 are calledleaf trees and trees of model 2 are called node trees (Nievergelt and Wong1973). Our preferred model of search tree is model 1, and we will use it for allstructures but the splay tree (which necessarily follows model 2).

5

3 8

2 4 7 9

61 2 3 4

5 6

7 8 9obj1 obj2 obj3 obj4

obj5 obj6

obj7 obj8 obj9

1 8

2 4 6 9

3 7

5

obj1 obj8

obj2 obj4 obj6 obj9

obj3 obj7

obj5

Search Trees of Model 1 and Model 2

A tree of model 1 consists of nodes of the following structure:

typedef struct tr_n_t {key_t key;struct tr_n_t *left;struct tr_n_t *right;

/* possibly additional information */} tree_node_t;

We will usually need some additional balancing information, which will bediscussed in Chapter 3. So this is just an outline.

26 2 Search Trees

From nodes of this type, we will construct a tree essentially by the followingrecursive definition: each tree is either empty, or a leaf, or it contains a specialroot node that points to two nonempty trees, with all keys in the left subtreebeing smaller than the key in the root and all keys in the right subtree being largerthan or equal to the key in the root. This still needs some details; especiallywe have to specify how to recognize leaves. We will follow here the followingconvention:

{ A node *n is a leaf if n->right = NULL. Then n->left points to theobject stored in that leaf and n->key contains the object’s key.

We also need some conventions for the root, especially to deal with emptytrees. Each tree has a special node *root.

{ If root->left = NULL, then the tree is empty.{ If root->left �= NULL and root->right = NULL, then root is a

leaf and the tree contains only one object.{ If root->left �= NULL and root->right �= NULL, then

root->right and root->left point to the roots of the right and leftsubtrees. For each node *left node in the left subtree, we haveleft node->key < root->key, and for each node *right node inthe right subtree, we have right node->key ≥ root->key.

Any structure with these properties is a correct search tree for the objects andkey values in the leaves.

With these conventions we can now create an empty tree.

tree_node_t *create_tree(void){ tree_node_t *tmp_node;

tmp_node = get_node();tmp_node->left = NULL;return( tmp_node );

}

2.2 General Properties and Transformations

In a correct search tree, we can associate each tree node with an interval, theinterval of possible key values that can be reached through this node. Theinterval of root is ]–∞,∞[, and if *n is an interior node associated withinterval [a, b[, then n->key ∈ [a, b[, and n->left and n->right have asassociated intervals [a, n->key[ and [n->key, b[. With the exception of theintervals starting in −∞, all these intervals are half-open, containing the left

2.2 General Properties and Transformations 27

endpoint but not the right endpoint. This implicit structure on the tree nodes isvery helpful in understanding the operations on the trees.

3 ]-∞,4[ 4 [4,5[

4 ]-∞,5[ 7 [5,10[

5 ]-∞,10[

10 ]-∞,∞[

11 [10,13[ 13 [13,16[

13 [10,16[

16 [16,17[ 17 [17,18[

17 [16,18[ 19 [18,20[

18 [16,20[ 30 [20,∞[

20 [16,∞[

16 [10,∞[

obj3 obj4

obj7

obj11 obj13

obj16 obj17

obj19

obj30

Intervals Associated with Nodes in a Search Tree

The same set of (key, object) pairs can be organized in many distinct correctsearch trees: the leaves are always the same, containing the (key, object) pairs inincreasing order of the keys, but the tree connecting the leaves can be very differ-ent, and we will see that some trees are better than others. There are two opera-tions – the left and right rotations – that transform a correct search tree in a differ-ent correct search tree for the same set. They are used as building blocks of morecomplex tree transformations because they are easy to implement and universal.

Suppose *n is an interior node of the tree and n->right is also aninterior node. Then the three nodes n->left, n->right->left, andn->right->right have consecutive associated intervals whose union isthe associated interval of *n. Now instead of grouping the second and thirdintervals (of n->right->left and n->right->right) together in noden->right, and then this union together with the interval of n->left in*n, we could group the first two intervals together in a new node, and thatthen together with the last interval in *n. This is what the left rotation does:it rearranges three nodes below a given node *n, the rotation center. This isa local change done in constant time; it does not affect either the content ofthose three nodes or anything below them or above the rotation center *n. Thefollowing code does a left rotation around *n:

void left_rotation(tree_node_t *n){ tree_node_t *tmp_node;

key_t tmp_key;tmp_node = n->left;tmp_key = n->key;n->left = n->right;

28 2 Search Trees

n->key = n->right->key;n->right = n->left->right;n->left->right = n->left->left;n->left->left = tmp_node;n->left->key = tmp_key;

}

Note that we move the content of the nodes around, but the node *n stillneeds to be the root of the subtree because there are pointers from higher levelsin the tree that point to *n. If the nodes contain additional information, thenthis must, of course, also be updated or copied.

The right rotation is exactly the inverse operation of the left rotation.

void right_rotation(tree_node_t *n){ tree_node_t *tmp_node;

key_t tmp_key;tmp_node = n->right;tmp_key = n->key;n->right = n->left;n->key = n->left->key;n->left = n->right->left;n->right->left = n->right->right;n->right->right = tmp_node;n->right->key = tmp_key;

}

[a,b[ [b,c[

[c,d[

left right

key b

left right

key c

[a,b[

[b,c[ [c,d[

left right

key b

left right

key c

right rotation

left rotation

Left and Right Rotations

Theorem. The left and right rotations around the same node are inverse oper-ations. Left and right rotations are operations that transform a correct searchtree in a different correct search tree for the same set of (key, object) pairs.

2.3 Height of a Search Tree 29

The great usefulness of the rotations as building blocks for tree operationslies in the fact that they are universal: any correct search tree for some set of(key, object) pairs can be transformed into any other correct search tree by asequence of rotations. But one needs to be careful with the exact statement ofthis property because it is obviously false: in our model of search trees, we canchange the key values in the interior nodes without destroying the search treeproperty as long as the order relation of the comparison keys with the objectkeys stays the same. But the rotations, of course, do not change the key values.The important structure is the combinatorial type of the tree; any system ofcomparison keys is transformed correctly together with the tree.

Theorem. Any two combinatorial types of search trees on the same systemof (key, object) pairs can be transformed into each other by a sequence ofrotations.

But this is easy to see: if we apply right rotations to the search tree as long asany right rotation can be applied, we get a degenerate tree, a path going to theright, to which the leaves are attached in increasing order. So any search treecan be brought into this canonical shape using only right rotations. Becauseright and left rotations are inverse, this canonical shape can be transformed intoany shape by a sequence of left rotations.

The space of combinatorial types of search trees, that is, of binary trees withn leaves, is isomorphic to a number of other structures (a Catalan family). Therotations define a distance on this structure, which has been studied in a numberof papers (Culik and Wood 1982; Mäkinen 1988; Sleator, Tarjan, and Thurston1988; Luccio and Pagli 1989); the diameter of this space is known to be 2n − 6for n ≥ 11 (Sleator et al. 1988). The difficult part here is the exact value of thelower bound; it is simple to prove just �(n) bounds (see, e.g., Felsner 2004,Section 7.5).

2.3 Height of a Search Tree

The central property which distinguishes the different combinatorial types ofsearch trees for the same underlying set and which makes some search treesgood and others bad is the height. The height of a search tree is the maximumlength of a path from the root to a leaf – the maximum taken over all leaves.Usually not all leaves are at the same distance from the root; the distance ofa specific tree node from the root is called the depth of that node. As alreadyobserved in Section 2.1, the maximum number of leaves of a search tree ofheight h is 2h. And at the other end, the minimum number of leaves is h + 1

30 2 Search Trees

because a tree of height h must have at least one interior node at each depth0, . . . , h − 1, and a tree with h interior nodes has h + 1 leaves. Together, thisgives the bounds.

Theorem. A search tree for n objects has height at least �log n� and at mostn − 1.

It is easy to see that both bounds can be reached.The height is the worst-case distance we have to traverse to reach a specific

object in the search tree. Another related measure of quality of a search treeis the average depth of the leaves, that is, the average over all objects of thedistance we have to go to reach that object. Here the bounds are:

Theorem. A search tree for n objects has average depth at least log n and atmost (n−1)(n+2)2n ≈ 12n.

To prove these bounds, it is easier to take the sum of the depths instead of theaverage depth. Because the sum of depths can be divided in the depth of thea leaves to the left of the root and the depth of the b leaves to the right ofthe root, these sums satisfy the following recursions:

depthsummin(n) = n + mina,b≥1a+b=n

depthsummin(a) + depthsummin(b)

and

depthsummax(n) = n + maxa,b≥1a+b=n

depthsummax(a) + depthsummax(b);

with these recursions, one obtains

depthsummin(n) ≥ n log n

and

depthsummax(n) = 12

(n − 1)(n + 2)

by induction. In the first case, one uses that the function x log x is convex, soa log a + b log b ≥ (a + b) log (a + b)/2.

2.4 Basic Find, Insert, and Delete 31

2.4 Basic Find, Insert, and Delete

The search tree represents a set of (key, object) pairs, so it must allow someoperations with this set. The most important operations that any search treeneeds to support are as follows:

{ find( tree, query key): Returns the object associated withquery key, if there is one;

{ insert( tree, key, object ): Inserts the (key, object) pair in thetree; and

{ delete( tree, key): Deletes the object associated with key fromthe tree.

We will now describe here the basic find, insert, and delete operationson the search trees, which will be extended in Chapter 3 by some rebalancingsteps. The simplest operation is the find: one just follows the associatedinterval structure to the leaf, which is the only place that could hold the rightobject. Then one tests whether the key of this only possible candidate agreeswith the query key, in which case we found the object, or not, in which casethere is no object for that key in the tree.

37

34 50

9 35 47 60

5 11 34 35 40 47 53 60

3 7

2 3 5 8

10 13

13 21

37 43

41 45

51 55

50 51 53 57

Search Tree and Search Path for Unsuccessful find(tree, 42)

object_t *find(tree_node_t *tree,key_t query_key)

{ tree_node_t *tmp_node;if( tree->left == NULL )

return(NULL);else{ tmp_node = tree;

while( tmp_node->right != NULL ){ if( query_key < tmp_node->key )

tmp_node = tmp_node->left;

32 2 Search Trees

elsetmp_node = tmp_node->right;

}if( tmp_node->key == query_key )

return( (object_t *) tmp_node->left );else

return( NULL );}

}

The descent through the tree to the correct level is frequently writtenas recursive function, but we avoid recursion in our code. Even with goodcompilers, a function call is much slower than a few assignments. Just asillustration we also give here the recursive version.

object_t *find(tree_node_t *tree,key_t query_key)

{ if( tree->left == NULL ||(tree->right == NULL &&

tree->key != query_key ) )return(NULL);

else if (tree->right == NULL &&tree->key == query_key )

return( (object_t *) tree->left );else{ if( query_key < tree->key )

return( find(tree->left, query_key) );else

return( find(tree->right, query_key) );}

}

The insert operation starts out the same as the find, but after it finds thecorrect place to insert the new object, it has to create a new interior node anda new leaf node and put them in the tree. We assume, as always, that there arefunctions get node and return node available, as described in Section 1.4.For the moment we assume all the keys are unique and treat it as an error if thereis already an object with that key in the tree; but in many practical applicationswe need to deal with multiple objects of the same key (see Section 2.6).

2.4 Basic Find, Insert, and Delete 33

int insert(tree_node_t *tree, key_t new_key,object_t *new_object)

{ tree_node_t *tmp_node;if( tree->left == NULL ){ tree->left = (tree_node_t *) new_object;

tree->key = new_key;tree->right = NULL;

}else{ tmp_node = tree;

while( tmp_node->right != NULL ){ if( new_key < tmp_node->key )

tmp_node = tmp_node->left;else

tmp_node = tmp_node->right;}/* found the candidate leaf. Test whether

key distinct */if( tmp_node->key == new_key )

return( -1 );/* key is distinct, now perform the insert */{ tree_node_t *old_leaf, *new_leaf;

old_leaf = get_node();old_leaf->left = tmp_node->left;old_leaf->key = tmp_node->key;old_leaf->right = NULL;new_leaf = get_node();new_leaf->left = (tree_node_t *)new_object;new_leaf->key = new_key;new_leaf->right = NULL;if( tmp_node->key < new_key ){ tmp_node->left = old_leaf;

tmp_node->right = new_leaf;tmp_node->key = new_key;

}else{ tmp_node->left = new_leaf;

tmp_node->right = old_leaf;}

}}return( 0 );

}

34 2 Search Trees

left right

key keyold

objold NULL

insertion

deletion or

left right

key keyold

left right

key keynew

left right

key keyold

objnew NULL objold NULL

if key < keynew old

left right

key keynew

left right

key keyold

left right

key keynew

objold NULL objnew NULL

if key < keyold new

Insertion and Deletion of a Leaf

The delete operation is even more complicated because when we are deletinga leaf, we must also delete an interior node above the leaf. For this, we needto keep track of the current node and its upper neighbor while going down inthe tree. Also, this operation can lead to an error if there is no object with thegiven key.

object_t *delete(tree_node_t *tree,key_t delete_key)

{ tree_node_t *tmp_node, *upper_node,*other_node;object_t *deleted_object;if( tree->left == NULL )

return( NULL );else if( tree->right == NULL ){ if( tree->key == delete_key )

{ deleted_object =(object_t *) tree->left;

tree->left = NULL;return( deleted_object );

}else

return( NULL );}else{ tmp_node = tree;

2.5 Returning from Leaf to Root 35

while( tmp_node->right != NULL ){ upper_node = tmp_node;

if( delete_key < tmp_node->key ){ tmp_node = upper_node->left;

other_node = upper_node->right;}else{ tmp_node = upper_node->right;

other_node = upper_node->left;}

}if( tmp_node->key != delete_key )

return( NULL );else{ upper_node->key = other_node->key;

upper_node->left = other_node->left;upper_node->right = other_node->right;deleted_object = (object_t *)tmp_node->left;return_node( tmp_node );return_node( other_node );return( deleted_object );

}}

}

If there is additional information in the nodes, it must also be copied orupdated when we copy the content of the other node into the upper node.Note that we delete the nodes, but not the object itself. There might be otherreferences to this object. But if this is the only reference to the object, this willcause a memory leak, so we should delete the object. This is the responsibilityof the user, so we return a pointer to the object.

2.5 Returning from Leaf to Root

Any tree operation starts at the root and then follows the path down to theleaf where the relevant object is or where some change is performed. In allthe balanced search-tree versions we will discuss in Chapter 3, we need toreturn along this path, from the leaf to the root, to perform some update or

36 2 Search Trees

rebalancing operations on the nodes of this path. And these operations need tobe done in that order, with the leaf first and the root last. But without additionalmeasures, the basic search-tree structure we described does not contain anyway to reconstruct this sequence. There are several possibilities to save thisinformation.

1. A stack: If we push pointers to all traversed nodes on a stack duringdescent to the leaf, then we can take the nodes from the stack in the correct(reversed) sequence afterward. This is the cleanest solution under thecriterion of information economy; it does not put any additionalinformation into the tree structure. Also, the maximum size of the stackneeded is the height of the tree, and so for the balanced search trees, it islogarithmic in the size of the search tree. An array-based stack for 200items is really enough for all realistic applications because we will neverhave 2100 items. This is also the solution implicitly used in any recursiveimplementation of the search trees.

2. Back pointers: If each node contains not only the pointers to the left andright subtrees, but also a pointer to the node above it, then we have a pathup from any node back to the root. This requires an additional field in eachnode. As additional memory requirement, this is usually no problembecause memory is now large. But this pointer also has to be corrected ineach operation, which makes it again a source of possible programmingerrors.

3. Back pointer with lazy update: If we have in each node an entry for thepointer to the node above it, but we actually enter the correct value onlyduring descent in the tree, then we have a correct path from the leaf we justreached to the root. We do not need to correct the back pointers during alloperations on the tree, but then the back pointer field can only be assumedto be correct for the nodes on the path along which we just reached theleaf.

Any of these methods will do and can be combined with any of the balancingtechniques. Another method that requires more care in its combination withvarious balancing techniques is the following:

4. Reversing the path: We can keep back pointers for the path even without anextra entry for a back pointer in each node by reversing the forwardpointers as we go down the tree. While going down in each node, if we goleft, the left pointer is used as back pointer and if we go right, the rightpointer is used as back pointer. When we go up again, the correct forwardpointers must be restored.

2.6 Dealing with Nonunique Keys 37

This method does not use any extra space, so it found interest when spacelimitations were an important concern. In the early years of data structures,methods to work with trees without space for either back pointers or a stackhave been studied in a number of papers (Lindstrom 1973; Robson 1973; Dwyer1974; Burkhard 1975; Clark 1975; Soule 1977; Morris 1979; Chen 1986; Chenand Schott 1996). But this method causes many additional problems becausethe search-tree structure is temporarily destroyed. Space is now almost nevera problem, so we list this method only for completeness, but advise againstits use.

2.6 Dealing with Nonunique Keys

In practical applications, it is not uncommon that there are several objects withthe same key. In database applications, we might have to store many objectswith the same key value; there it is a quite unrealistic assumption that eachobject is uniquely identified by each of its attribute values, but there are queriesto list all objects with a given attribute value. So any realistic search tree has todeal with this situation. The correct reaction is as follows:

{ find returns all objects whose key is the given query key inoutput-sensitive time O(h + k), where h is the height of the tree and k is thenumber of elements that find returns.

{ insert always inserts correctly in time O(h), where h is the height of thetree.

{ delete deletes all items of that key in time O(h), where h is the height ofthe tree.

The obvious way to realize this behavior is to keep all elements of the samekey in a linked list below the corresponding leaf of the search tree. Then findjust produces all elements of that list; insert always inserts at the beginningof the list; only delete in time independent of the number of deleted itemsrequires additional information. For this, we need an additional node betweenthe leaf and the linked list, which contains pointers to the beginning and to theend of the list; then we can transfer the entire list with O(1) operations to thefree list of our dynamic memory allocation structure. Again, this way we onlydelete the references to the objects contained in this tree. If we need to deletethe objects themselves, we can do it by walking along this list, but not in O(1)time independent of the number of objects.

38 2 Search Trees

2.7 Queries for the Keys in an Interval

Up to now we have discussed only the query operation find, which, for agiven key, retrieves the associated object. Frequently, a more general type ofquery is useful, in which we give a key interval [a, b[ and want to find all keysthat are contained in this interval. If the keys are subject to small errors, wemight not know the key exactly, so we want the nearest key value or the nextlarger or next smaller key. Without such an extension, our find operation justanswers that there is no object with the given key in the current set, which iscorrect but not helpful.

There are other types of dictionary structures, which we will discuss inChapter 9 on hash tables that cannot support this type of query. But for searchtrees, it is a very minor modification, which can be done in several ways.

1. We can organize the leaves into a doubly linked list and then we can movein O(1) time from a leaf to the next larger and the next smaller leaf. Thisrequires a change in the insertion and deletion functions to maintain thelist, but it is an easy change that takes only O(1) additional time. The querymethod is also almost the same; it takes O(k) additional time if it lists atotal of k keys in the interval.

2. An alternative method does not change the tree structure at all but changesthe query function: we go down the tree with the query interval instead ofthe query key. Then we go left if [a, b[< node->key; right ifnode->key ≤ [a, b[; and sometimes we have to go both left and right ifa < node->key ≤ b. We store all those branches that we still need toexplore on a stack. The nodes we visit this way are the nodes on the searchpath for the beginning of the query interval a, the search path for its end b,and all nodes that are in the tree between these paths. If there are i interiornodes between these paths, there must be at least i + 1 leaves betweenthese paths. So if this method lists k leaves, the total number of nodesvisited is at most twice the number of nodes visited in a normal findoperation plus O(k). Thus, this method is slightly slower than the firstmethod but requires no change in the insert and delete operations.

Next we give code for the stack-based implementation of interval find.To illustrate the principle, we write here just the generic stack operations; theseneed, of course, to be filled in. The output of the operation is potentially long,so we need to return many objects instead of a single result. For this, we createa linked list of the (key, object) pairs found in the query interval, which is linkedhere by the right pointers. After use of the results, the nodes of this list needto be returned to avoid a memory leak.

2.7 Queries for the Keys in an Interval 39

tree_node_t *interval_find(tree_node_t *tree,key_t a, key_t b)

{ tree_node_t *tr_node;tree_node_t *result_list, *tmp;result_list = NULL;create_stack();push(tree);while( !stack_empty() ){ tr_node = pop();

if( tr_node->right == NULL ){ /* reached leaf, now test */

if( a key &&tr_node->key < b )

{ tmp = get_node();/* leaf key in interval */tmp->key = tr_node->key; /*copy to output list */tmp->left = tr_node->left;tmp->right = result_list;result_list = tmp;

}} /* not leaf, might have to follow down */else if ( b key )/* entire interval left */

push( tr_node->left );else if ( tr_node->key right );else /* node key in interval,

follow left and right */{ push( tr_node->left );

push( tr_node->right );}

}remove_stack();return( result_list );

}

Listing the keys in an interval is a one-dimensional range query. Higher-dimensional range queries will be discussed in Chapter 4. In general, a range

40 2 Search Trees

query gives some set, the range, of a specific type, here intervals, and asks for all(key, object) pairs whose key lies in that range. For more complex ranges, suchas rectangles, circles, halfplanes, and boxes, this is an important type of query.

2.8 Building Optimal Search Trees

Occasionally it is useful to construct an optimal search tree from a given setof (key, object) pairs. This can be viewed as taking search trees as static datastructure: there are no inserts and deletes, so there is no problem of rebalancingthe tree, but if we build it, knowing the data in advance, then we should build itas good as possible. The primary criterion is the height; because a search treeof height h has at most 2h leaves, an optimal search tree for a set of n items hasheight �log n�, where the log, as always, is taken to base 2.

We assume that the (key, object) pairs are given in a sorted list, orderedwith increasing keys. There are two natural ways to construct a search tree ofoptimal height from a sorted list: bottom-up and top-down.

The bottom-up construction is easier: one views the initial list as list ofone-element trees. Then one goes repeatedly through the list, joining twoconsecutive trees, until there is only one tree left. This requires only a bitof bookkeeping to insert the correct comparison key in each interior node.The disadvantage of this method is that the resulting tree, although of optimalheight, might be quite unbalanced: if we start with a set of n = 2m + 1 items,then the root of the tree has on one side a subtree of 2m items and on the otherside a subtree of 1 item.

Next is the code for the bottom-up construction. We assume here that thelist items are themselves of type tree node t, with the left entry pointingto the object, the key containing the object key, and the right entry pointingto the next item, or NULL at the end of the list. We first create a list, where allthe nodes of the previous list are attached as leaves, and then maintain a listof trees, where the key value in the list is the smallest key value in the treebelow it.

tree_node_t *make_tree(tree_node_t *list){ tree_node_t *end, *root;

if( list == NULL ){ root = get_node(); /* create empty tree */

root->left = root->right = NULL;return( root );

}

2.8 Building Optimal Search Trees 41

else if( list->right == NULL )return( list ); /* one-leaf tree */

else /* nontrivial work required: at leasttwo nodes */

{ root = end = get_node();/* convert input list into leaves below

new list */end->left = list;end->key = list->key;list = list->right;end->left->right = NULL;while( list != NULL ){ end->right = get_node();

end = end->right;end->left = list;end->key = list->key;list = list->right;end->left->right = NULL;

}end->right = NULL;/* end creating list of leaves */{ tree_node_t *old_list, *new_list, *tmp1,

*tmp2;old_list = root;while( old_list->right != NULL ){ /* join first two trees from

old_list */tmp1 = old_list;tmp2 = old_list->right;old_list = old_list->right->right;tmp2->right = tmp2->left;tmp2->left = tmp1->left;tmp1->left = tmp2;tmp1->right = NULL;new_list = end = tmp1;/* new_list started */while( old_list != NULL )/* not at end */{ if( old_list->right == NULL )

42 2 Search Trees

/* last tree */{ end->right = old_list;

old_list = NULL;}else /* join next two trees of

old_list */{ tmp1 = old_list;

tmp2 = old_list->right;old_list =

old_list-> right->right;tmp2->right = tmp2->left;tmp2->left = tmp1->left;tmp1->left = tmp2;tmp1->right = NULL;end->right = tmp1;end = end->right;

}} /* finished one pass through

old_list */old_list = new_list;

} /* end joining pairs of treestogether */

root = old_list->left;return_node( old_list );

}return( root );

}}

Theorem. The bottom-up method constructs a search tree of optimal heightfrom an ordered list in time O(n).

The first half of the algorithm, duplicating the list and converting all the orig-inal list nodes to leaves, takes obviously O(n); it is just one loop over thelength of the list. The second half has a more complicated structure, but ineach execution of the body of the innermost loop, one of the n interior nodescreated in th