Data structures and algorithm analysis (3rd ed) by mark allen weiss (p1)

Bismillah hir Rehman nir Raheem ------------^^^^^^^^^^----------Assalat o Wasalam o Allika Ya RasoolALLAH

To Read Online & Download: WWW.ISSUU.COM/SHEIKHUHASSAN

Data Structures and Algorithm

Analysis in C++

(3rd Ed)

Mark Allen Weiss

Preface + Ch. #1 to Ch. #6

Published By: Muhammad Hassan Riaz Yousufi

Third Edition

Data

++ Mark Allen Weiss Florida International University

BOSj-OlI San Francisco New York

London Toronto Sydney 'j()kyo SingJ.pore fvl:!drid

Mexico City Munich P:lris C:lpC Ti.)wll Hong Kong Montreal

Puhlisher Greg ]()bin Sellior ;\cquhiliolls Fe/itm lVlichacl llirsch Prodllctioll SHptTVisOl I\'tarilyn Lloyd Ldilorial Assis/m1/ ljnclscy Triche! Cover Di'si,~11 M£ll1agcl Joyce (j)scntino 'vVclls Cover Design Night & Day Design Cover Ima)!.e (.0 Pele (~ardnLT/Digital Vision fdarl1eling Jvlal1(/)!.Cf Michelle Brown Mar]u>lillg i\SSlsttllll Dana! .opreato Project Mal1ag,emclll. vVindfal1 Software Composition \Vindfall Software, using Zill:.X Tcc/miwlllluslralioll Ccorge Nichols J>r()(~rrc(/(lfl tVIaryEllen N. Oliver 1m/excr Ted LUllX

PrCfm:ss and Mmudtic/liring, Caroline j;cll Prinler Courier Stoughton, Ine.

Access the latest information about Addison-\Veslcy liLIes from Our \Vorld \Vide Vv'ch sitt: http://www.aw-hc.com/computing

rvlany of the designations used by manufacturers and sellers to distinguish their products arc claimed as trademarks. \Vhcrc those designalions appear in this book, and Addison-\Vcslcy was aware of a trademark claim, the designations have bcen printed in initial caps or all caps.

J.ibrary of Congress Cataloging-in-Pu\)liGllion l)ala

\\leiss, Mark Allen. Data structures and algorithm analysis ill C++ I ivlark Allen \Vdss.~Jrd cd.

p. Clll.

Includes bibliographical rderences and index. ISBN 0-32 I A4146-X (aIle paper)

1. C++ (Computer program language) 2. Data structures (Computer sciencr) 3. Computer algorithms. l. Title.

QA76.Tl.USlW46200') 00').1 fl-den

Copyright <D 2006 by Pearson Education, Inc.

2005023534

For information on obtaining permiSSion for llse of material ill this work, please suhmil a written n:::quest to Pearson Fducation, Inc., Rights and Contract Department, 7') Arlington Street, Suite '}OO, Boston, MA 021.16 or f~\x your requcst to (617) 848~ 7047.

AI! rights reserved. No pan of this publication may be reproduceel, stored ill a relrieval system, or transmitted, in any forrn or by any means, electronic, mechanical, phutocopying, recording, or any other rncdia embodiments now known or hereafter to become known, without tbe prior written permission of the publisher. Prillted in the United States of America.

ISBN 0-.121 -44146-X 3 4 'j 6 7 8 9 10-CRS-08 OJ

Bismillah hiir Rehman nir Raheem-----------------------------Assalat o Wasalam o Alika Ya RasoolALLAH


Preface xv

Chapter 1 Introduction 1.1 What's the Book About'

1,2 Mathematics Review 2

1.2.1 Exponents >3

12.2 Logarithms >3

1.2.3 Series 4 1.2.4 Modular Arithmetic ')

1.2.5 The P Word 6

1,3 A Brief Introduction to Recursion 7

I .4 C++ Classes II 1.4.1 Basic class Syntax 12 1.4.2 Extra Constructor Syntax and Accessors 12 1.4.3 Separation of Interf~Kc and Implementation 15 1.4.4 vector and str; ng 17

1.5 C++ Details 19 1.5.1 Pointers 19 1.5.2 Paramerer Passing 21 1.5.3 Return Passing 22 1.5.4 Reference Variables 23 1.'5."5 The Big Three: Destructor, Copy Constructor, operator'" 23 1.5.6 C-stylc Arrays and Strings 26

1.6 Templates 29

1.6.1 function Icmplates 29 1.6.2 Class Templates 30 1.6.3 Object, Comparable, and an Example 32 1.6.4 function Objects 34

1.0.5 Separate Compilation of Class Templates 35

1.7 Using Matrices 37 1.7.1 The Data Members, Constructor, and Basic Accessors 37 1.7.2 operator[] 37 1.7.3 Destructor, Copy Assignment, Copy Constructor 39

1

vii



viii Contents

Summary 39

Exercises '_39

References 4 ]

Chapter 2 Algorithm Analysis 2.1 Mathematical Background 43

2.2 Model 46

2.3 What to Analyze 46

2.4 Running Time Calculations 2.4.1 A Simple Example 2.4.2 Cencral Rules 50

49 49

2.4.3 Solutions for the Maximum Subsequence Sum Prohlem 52 2.4.4 Logarithms in the Running Time 58 2.4.5 Checking Your Analysis 62 2.4.6 A C;rain 01 Salt 63

Sumrn<uy 63

Exercises 64

References 69

Chapter 3 Lists, Stacks, and Queues 3.1 Abstract Data "Iypes (ADTs) 71

3.2 The List ADT 72 3.2.1 Simple Array lmpiemenlalion of Lists 72 3.2.2 Simple Linked Lists 73

3.3 vector and list in the SIt 74 3.3.1 Iterators 75 3.3.2 Example: Using erase on a List 77

3.3.3 canst iterators 77

],4 1mplementalion of vector 79

3.5 Implementation of list 83

3.6 The Stack ADT 94 3.6.1 Stack Model 94 3.6.2 Implementation of Stacks 95 3.6.3 Applications 96

3.7 The Queue ADT 104 3.7.1 Queue Model 104 3.7.2 Array Implementation of Queues 104 3.7.3 Applications 01 Queues 106

Summary 107

Exercises 108

43

71



Chapter 4 Trees 4,1 Preliminaries 11'3

4. L I Implementation of]h'es 114 4.1.2 Tree Traversals \-\lith an Applicalion 11')

4.2 Binary'lin's 119 4.2. I Implementation 120

4.2.2 An Example: Expression Trees 121

4.3 The Search '1\'ee ADT-Binary Search Trees 124 43.1 contains 125 4.3.2 findMinandfindMax 125

4.3.3 insert 129 4.3.4 remove 130 4.3.5 Destructor and Copy Assignment Operator 132

4.3.6 Averag('~Casc Analysis 133

4.4 AVL Trees J 36 4.4.1 Single Rotation 139

4.4.2 Double Rotation 142

4.5 Splay'hees 149

4,5,1 A Simple Idea (That Docs Not Work) 150

4.6

4.7

4.R

4'5.2 Splaying 1'52

Tree 'haversals (I<evisited) 158

[l·'lrees 159

Sets and Maps in the Standard Library 4.8.1 Sets 16'5

4.8.2 Maps 166

165

4.83 Implementation of set and map 167 4.8.4 An Example That Uses Several Maps 168

Summary 174

Exercises 1 74

References IH 1

Chapter 5 Hashing 5.1 General Idea 185

5.2 Hash Function 186

5.3 Separate Chaining 188

5.4 Hash Tables Without Linked Lists 192 '5.4.1 Linear Probing 193 '5.4.2 Quadratic Probin~ 195

5.4.3 Double fl3shing 199

'5.5 Rehashing 200

Contents ix

113

185



x Contents

5.6 Hash 1ilbles in the Standard Library 204

5.7 Extendible Hashing 204

Summary 207

Exercises 20S

References 211

Chapter 6 Priority Queues (Heaps) 0.1 Model 213

6.2 Simple Implementations 214

6.3 Binary Hear 215

6.4

6.5

6.6

6.7

6.3.1 Structure Property 21 'j 6.3.2 Heap-Order Property 216 6.3.3 Ilasie Heap Operations 217 6.3.4 Other Heap Operations 220

Applications of Priority Queues 6.4.1 The Selection Prohlem 6.4.2 Event Simulation 227

d-Heaps 228

Leftist Heaps 229 6.6.1 Leftist Heap Property 6.6.2 Leftist Heap Operations

Skew Heaps 235

225 226

229 230

6.S Binomial Queues 239 6.8.1 Binomial Queue Structure 240 6.8.2 Binomial Queue Operations 241 6.S.3 Implementation of Binomial Queues 244

6.9 Priority Queues in the Standard Library 251

Summary 251

Exercises 251

References 257

Chapter 7 Sorting 7.1 Preliminaries 261

7.2 Insertion Son 262 7.2.1 The Algorithm 262 7.2.2 STI. Implementation of Insertion Sort 263 7.2.3 Analysis of Insertion Sort 264

7.3 A Lower Bound for Simple SOIling AlgOrithms 265

213

261



Contents

7'j ShcllsorL 266

7.4.1 Worst-Case Analysis of Shcllsort 268

7.5 1 lcapson 270

7.5.1 ;\nalysis of Ilcapsort 272

7.6 TV!crgcsort 274

7.6.1 Analysis ofiv1crgcsort 276

7.7 Quicksort 279

7.7.1 Picking the Pivot 280

7.7.2 Partitioning Strategy 2B2

7.7.3 Small Arrays 2B4

7.7.4 J\clLlai Quicksort P.outincs 2B4

7.7. 'j AnalysIs of Quicksort 287

7.7.6 A Lillcar-FxpcC1.cd-Timc Algorithm for Selection 2YO

7.B Indirect Sorting 292

7.R.l vector<Comparable*> Docs Not \Vork 295

7.8.2 Smart Pointer Class 20'5 7.8.3 Overloading operator< 29'3

7.8.4 Dcrcferellcing a Pointer \Vilh * 29'5

7.8. '5 ()verloading the -TYpe Conversion Operator 29.'5

7.8.6 Implicit ·rypc Conversions Arc Evcl")!\vhcrc 296

7.B.7 Dual-Direction Implicit Conversiulls em Cause Ambiguities 296

7.8.8 Pointer Subtraction Is Legal 207

7.1) A Ceneral Lower Bound for Sorting 297

7.9.1 Decision'ih:cs 297

7.10 Gucket SOrL 299

7.1 1 External Sorting JOO

7.11. I \Vhy \Vc Need New Algorit hms JOO

7.11.2 ivlodel for External SOrLinglOO

7.1 Ll The Simple Algorithm lOI

7.11.4 Multiway Mergc ·)02

7.11.5 Polyphase Merge ·303

7.1l.6 Replacement Selection 304

SUlllrnary 30'5

Exercises 306

References 31 I

Chapter 8 The Disjoint Set Class B. I Equivalence Febt iOlls ~31 '5

B.2 The Dynamic Equi"alence Prohkrn :, t 6

R.J Basic Data Structure ")17

315

xi



xii Contents

8.4 Smart Union Algorithms 321

8. '5 Path Compression 324

8.6 Worst Case for Union-by-Rank and Path Compression 32'5 8.6. I Analysis 01 the Union/Find Algorithml26

8.7 An Application 331

Summary 334

Exercises 33'5

References 336

Chapter 9 Graph Algorithms 9. [ Definitions 339

9.1.1 Representation of Graphs 340

9.2 Topological Sort 342

9.3 Shortest-Path Algorithms 345 9.3.1 Unweighted Shortest Paths'l47 9.3.2 Dijkstras Algorithm 351 9.3.3 Graphs with Negative Edge Costs 360 'U.4 Acyclic Graphs 360 9.3.5 All-Pairs Shortest Path ')64 9. ').6 Shortest Path Example16'j

9.4 Network Flow Problems 367 9.4.1 A Simple Maximum-Flow Algorithm 367

9_5 Minimum Spanning Tree 372 9.5. I Prilll' Algorithm 373 9.5.2 Kruskals Algorithm 376

9.6 Applications o{ Depth-First Search 378 9.6.1 Undirected Graphs 379 9.6.2 I3iconncctivity 381

9.6.3 Euler Circuits 385 9.6.4 Directed Craphs 388 9.6.5 f'inding Strong Componenls 390

9.7 Introduction to NP-Completcness 392 9.7.1 Easy vs. Hard 392 9.7.2 The Class NP 393 9.7.3 NP-Complete Problems 394

Summary 396

Exercises ]96

References 404

339



Chapter 10 Algorithm Design Techniques 10.1 C;reedy Algorithms 409

10. LI A Simple Scheduling Problem 410 10.1.2 IlulTman Codes 411 ! 0.1.3 Approximate Bin Packing 4 J 9

10.2 Divide and Conquer 427 10.2.1 Running Time of Divide and Conquer Algorithms 428 10.2.2 Closest-Points Problem 430 10.2.3 The Selection Problem 435 10.2.4 Theoretical Improvements for Arithmetic Problems 438

[0.:3 Dynamic Pmgramming 442 H1.'3..\ Using a ~1~lblc Instead of Recursion 442

lO.l.2 Ordering Matrix Multiplications 444 10.1.1 Optimal Binary Searehli·ec 447

10.3.4 Ail-PaIrS Shortest Path 451

10.4 Randomized Algorithms 4')4 ! O.4.l Random Numher CeneraLors 4'5'5

10.4.2 Skip Lists 459 10.4.1 Primality Testing 461

10. 'j Backt racking Algorithms 464 10.5.1 The Turnpike ReCOilS! ruction Problem 465 10.5.2 Cames 469

Summary 475

Exercises 475

References 485

Chapter 11 Amortized Analysis Il.l

11.2

11.3

11.4

1 j')

An Unrelated Puzzle 492

Binomial Queues 492

Skew Heaps 497

Fibonacci lleaps 499 11.4.1 Cultmg Nodes in Leftist Heaps 500 11.4.2 Lazy Merging for Binomial Queues 502 11.4.3 The Fibonacci 1 !cap Operations 506 1 1.4.4 Proof of the. Time Bound 506

Splay lilTS 509

SUlnmary 513

Exercises SU

ReiCrences 5t')

Contents XIII

409

491



XIV Lomems

Chapter 12 Advanced Data Structures and Implementation

12.1 ·l(lp·DoWIl Splay ·11\:c5 'j 17

122 Rcd·Black ·Ii·ccs 525 J 2.2.1 Bottom-Up Insertion '526

122.2 Top·Down Reel·G\ack ·11·ce5 '527

12.2. 3 Top~f)o\Vn Deletion "531

12.3 Deterministic Skip Lists .'535

12.4 ;\i\;liTes '340

1 2. '5 ·/ITap5'54 7

12.6 h·el Trecs 549

12.7 Fairing lIeaps '553

Summary '558

Exercises 558

References '56:)

Appendix A Separate Compilation of Class Templates

AI Everything in the Header 568

1\.2 Explicit instantiation ')(jt)

A.3 The cxport Directive ')70

Index 571

517

567



PREFACE

Purpose/Coals The third edition of Dala Structures alld Algorithm Analysis in C++ describes data structures, methods of organizing large amounts of data, and algorithm analysis, the estimation of the running time of algorithms. As computers become faster and faster, the need for programs that can handle large amounts of input hecomes more acutc. Paracioxically, this requires more careful attention to efficiency, since inefficiencies in programs become most obviolls when input sizes are large. By analyzing an algorithm before it is actually coded, students can decide if a particular solution will be feasible. For example, in this text students look at spccihc problems and see how careful implementations can reduce the time constraint for large amounts of data from] 6 years to less than a second, Therefore, no algorithm or data structure is presenLed without an explanation of its running time. In some cases, minute dctails that affect the running time of the implementation arc explored.

Once a solution method is determined, a program must still be wrillcn. As computers have become more powerful, the problems they must solve have become larger and more complex, requiring development or more intricate programs. The goal of this text is to teach students good programming and algorithm analysis skills simultaneously so that they can develop sllch programs with the maximum amount of efficiency

This book is suitable for either an advanced data structures course or a first-year graduate course in algorithm analysis. Students should have some knowledge of intermediat.e programming, including such topics as pointers, recursion, and object.-based programming, and some background in discrele math.

Approach Although the material in this text is largely language independent, programming requires the use of a specific language. As the title implies, we have chosen C++ for this book.

C++ has become a leading systems programming language. In addition to fixing many of the sy11lactic flaws of C:, C++ provides direct constructs (the ci(!ss and template) to implement generic data structures as abstract data types.

The most difficult part of writing the book was deciding on the amount. of C++ to include. Usc too many features of C++, and one gets an incomprehensible text; usc 1.00 few and you have little more than a C text that supports classes.

The approach we take is to present the material in an ohject-hased approach. As such, there is almost no usc of inheritance in the text. We use class templates to describe generic data structures. 'vVc generally avoid esolnic C++ features, and usc the vector and stri ng classes that are now part or the C++ standard. Using these first-class versions, instead of the second-class counterparts that were used in the first eclition, simplifies much of the code. Previous editions have implemented class templates by separating the class template xv



XVI

inler/ice from its impkmentation. Although this is undoubtedly the prcfcrred approach, il exposes compiler problems that have made it difficult tor readers to actually usc the cock. As a result, in this edilion, the online code represents class Lcmpialcs as a single unit, with no separation of imuface and implementation. Chapler I provides a revicw of the C++ features that arc used throughout the lext and descrihes our approach !o class templates. Appendix A describes hmv the delSS [em plates could be rcwritten to usc separate compilation.

Complete versions of the data structures, in both C++ and Java, arc available on the Internet. Vk use similar coding conventions to make the parallels hel wcen the 1 \\'0 languages more evident.

Summary of the Most Significant Changes in the Third Edition The third edition incorporates numerous hug fixes, and mallY pans of the book haw undergone revision to increase clarity of presentalion. In addition:

Throughout the Lext, the code has been updated as appropriate to usc modern C++ features.

ChapLer 3 has been signif-icantly revised and contains a discussion or the W:iC of the standard vector and 1 i st classes (and their itcfaLors) as well as an implClllcntalion of the standard vector and 1 ist classes. The standard vector class is used throughout the lexl in the implementation of 01 her data structures.

Chapter 4 has heen revised to include a discussion of the set and map classes, along with an extensive example illustrating their usc ill the dcsign of dlJeiCi1l algorithms. Chapter 9 also includes an example that makes usc of tbe standard map to implement a shortest path algorithm.

Chapter 7 contains a discussion of the standard sort algorithms, including an illustration of the techniques involved in implementing the ovcrloaclrd standard sort algorithms.

The presentation of the code has been simplified to avoid using the complicated syntax associated with the separate compilation or class templates. The revised code will allow the reader to focus on the: algorithmiC details, rather than getting hogged down in C++.

Overview Chapter t contains review material on discreLe math and recursion. 1 bclin'C the only way to be comfortable with recursion is to sec good uses over and over. Therefore, recursion is prevalent in this text, with examples in every chapter except Chapter '5. Chapler 1 also includes material that serves as a review of basic C++. Included is a discussion of templates and important constructs in C++ class design.

Chapter 2 deals with algorithm analysis. This chapler explains asymptotic analysis and its major weaknesses. Many examples arc provided, including an in-depth explanation of logarithmic running time. Simple recursive programs arc analyzed by intuitively converting them into iterative programs. More cornplicau:cl clivic1e-and-conqucr programs afC introduced, but some of the analysls (solving recurrence relations) is implicitly delayed until Chapter 7, where it is pcrfnrmccl in detail.



Preface

Chaptcr 3 co\'(']'s liSIS, slacks, and queues. This chapter has been significantly revised from prior edilions. TI now includes a discussion of the STI. vector and 1 i st classes, including material un iteralors, and it provides impicl1ll'lll,lIions of a significant suhset 01 the STL vector and '1 ist classes,

Chapter 4 covers trees, with an emphasis on search trees, including external search trees (B-trcrs). The IJ'\I.\ file system and expression \rees are used as examples. AVL trees and splay trees arc introduced. l\,'-lorc careful treatment uf search tree implementation details is

found in Chapter 1 2. Additional coverage of trees, such as file com pression and game I re('s, is deferred until Chapter 10. Data structures for an external medium arc considcn:d as the final topic in several chapters. New 10 this edilion is (l cliscussinn of the STL set and map classes, including a significant example that illustrates the usc of three separate maps to efficicntly SU]\T a problem.

Chapter '; is a relatively short chapter concerning hash tables. Some i.l!l,tlysis is performed, and extendible hashing is covered at lhe end of the chapter.

Chapter 6 is ahout priority quelles. Binary heaps arc covered, and there is additional material OJ] sOllle of the theoretically inlCl"esting irnplcllH'ntations of priority queues. The fibonacci heap is discussed ill Chapter I -1, ,md the pairing heap is discussed in Chapter] 2.

Chapter 7 co\'ers sorting. It is vcry specific with respect to coding details and analysis. All the important gencral-purpose sorting algorithms arc covered and compared. Four algorithms arc analyzed in detail: insert ion sort, Sbdls(JrL, heapsorl, and quicksort. External sorting is covCl"ed at the end of the chapteT

Chapter K dbcllSSCS the disjoint scI algorithlll with proof of the running time. This is C\

shon and specific chapter 11'1;-)( can he skipped if Kruskal's algtH"ithm is not discllssed.

Chapter 9 c()\'er::> graph algorithm:... Algorithms on graphs arc interesting, not only because they frequently occur ill practice hut "Iso beulUsc their running time is so heavily dcpendelll on the proper usc of data structures, Virtually all of the standard algorithms arc presented along with appropriate data structures, pseudocode, ,mel analysis of running time. To place these prohlems in a proper contexl, a short discussion on complexity theory (including NP-completcncss alld undecidability) is provided.

Chapter lO covers algorithm design hy examining cOlllmon problem-solving tech

niques. This chapter is hC<l\'ily fortified with examples. Pseudocode is used in these bter chapters so tt-wt the student's appreciation of an example algorithm is not obscured hy implementation detaik

Chapter 11 deals with amortized analysis. Three dma strul"lures ["rom Chapters 4 and 6 and the Fibo11acci heap, introduced in this chapter, ,\f(:.'" analyzed.

Chapter 12 covers search tlTe algorithms, the /?-d liTe, alld the pairing heap. This chapter departs from the rest of the text by providing complete and careful implementations for the search trees and pairing heap. The material is structured so tlMI the instructor can illlegrate sections into discussions from other chapter::>. For example, the top-down red-black tree in Chapler ! 2 can he discu~sed under AVL trees (in Chapter Lt).

Ol<1pters 1-0 proYidc enough material fur \])ost one-scmester data structures courses.

If time permits, then Chapter 10 can be covered. A graduate course on algorithm analysis could cover (~hapters 7-11. '!-he adv;mcccl data stnlClurcs <1!l<llyzed in C:iwptcr II (';111 easily

be rcl"crrcd to ill the earlier chapters. The dbcussint1 of NP-complrtcllcss ill Chapler 0 is I;\r too brief to he Llsed in such it course. You might find it useful to usc an additiollJl work Oil

NP-col1lpleteness to augmellt this text.

XVII



Exercises Exercises, provided at the end of each chapter, match the order in which material is prescnlCcl. The last exercises may address the chapter as a whole rather than a specific section. Difficult exercises arc marked with an asterisk, and more challenging exercises have two asterisks.

References References are placed at the end of each chapter. Generally the references either arc historical, representing the original source of the material, or they represent extensions and improvements to the results given in the text. Some references repn.'scnt solutions to exercises.

Supplements The following supplements arc available to all readers at www.aw.com!cssupport:

• Source code for example programs

In addition, the following material is available only to qualified instruclors m AddisonVvesley's Instructor Resource Center (www.aw.comlirc). Visit the IRe or contact your campus A-W representative for access.

Solutions to selected exercises

hgures from the book

Acknowledgments Many, many people have helped me in the preparation of books in this series> Some arc listed in other versions of the book; thanks to all.

As usual, the vvriting process was made easier by the professionals at Addison-Wesley. I'd like to thank my editor, Michael Hirsch, and production editor, Marilyn Lloyd. J'J also like 10 lhank Paul AnagnosLopoulos and his staff at vVindfall Software for their fine work putting lhe final pieces together. My wonderful wife jill deserves extra special thanks for everything she docs.

hnally, l'd like to thank the numerous readers who have sent e-mail messages and pointed out errors or inconsistencies in earlier versions. My \A/orld Wide \A/cb page www.cis.liu.cdu/~weisswillalsocontainupdatedsourcecode(inC++.C.andJava).an

errata list, and a link to submit bug reports,

MAW Miarni, Florida



Third Edition

Data Structures and Algorithm Analysis in

++





CHAPTER 1

Introduction

In this chapter, we discuss the aims and goals of 1 his text and briefly review programming concepts and discrete mathematics. \Vc will

Sec that how a program performs for reasonahly large input is just <IS imponant as its pcrfonnance on moderate arnoutlts of input.

Summarize the basic mathematical background needed for the rest of the book.

Briefly review recursion,

Summarize some important features of C++ that are used throughout the text.

1.1 What's the Book About? Suppose you have a group of N numbers and would like to determine the hlh largest. This is known as the selection problem. Most students who have had a programming course or two would hJVC no difficulty writing a program to solve this problem. There arc quite a few "obvious" solutions.

One \vay to solvc this problem would he 10 read the N numbers into an array, sort the array in decreasing order by some simple algorithm such as hubblesort, and then return the clement in position h.

A somewhat helter algorithrn might be to read the lirst h clements into an array and SOrlthem (in decreasing order). Next, each rcrnaining clement is read one by one. As a new elemf.'nt arrives, it is ignored if it is smaller than the hth clemenl in the array. Otherwisc, it is placed in its correct spot in the array, bumping one clement out of the array. Vlhen the algorithm ends, the clement in Ihe hlh posiLion is returned as the answer.

Both algorithms afC simple to code, and you are encouraged 10 do so. The natural questions, then, are which algorithm is betler and, more important, is either algorithm good enough? A simulation using a random file or 10 million clements and k = .'5,000,000 will show that neither algorithm Gnishcs in a reasonable amount of time; each requires several days of computer processing to terminate (alheit eventually wi1h a correct ansINer). An alternative method, discussed in Chapler 7, gives a solution in about a second. Thus, although our proposed algorithms work, they cannOl be considered good algorithms, becausc they are cntirely impractical ror inplll sizes that a third algorithm can handle in a reasonable amount of time.

1



'--"UP''-'' "",v'"'u'""v,'

2 3 4

i1

2 w a s ) " a i1 " h

4 g d

Figure 1.1 Sample word puzzle

;\ second prohlem is to solve a popular word puzzle. The input consists of a \wodimensional alT(l), of kilns and a list of words. The object is to find the words ill the puzzle. These words may he horizontal, vertical, or diagonal in any direetioll. As an example, the puzzle shown in Figure J .1 contains the words [his, Iwo, fal, and lIwl. The word this begins at row I, column 1, or (I) 1), and extends to (1,4); two goes from (l,J) to (.3,1); Jut goes from (4,\) 10 (2,3); and tilat goes from (4if) 10 (I ,\)

Again, there arc at least two straightforward algorithms tllJl so!\'(, the prohlem. For each

word in the word list, we check cach ordered triple (nl\\~ cO/limn, oricni(Jlion) for t he presence or the word. This amounts to luts or nested for loops hut is hasicaUy straightforward.

Alternati\'t'ly, for each ordered quadruple (row, column, oriell/a/ioll, IllimiJer (!f c/)(//"(/clcrs)

lhat doesn't run off an end of the puzzle, we can test whether the word indicated is in the

word lis!. Again, this amounts to lots or nested for loops. It is possible to smT some lime j[

the maximum number of characters in any word is known. II is relati\'(~ly eas)' to code up either method or solution and solve tll,lIly of the real,-iifc

puzzles common I)' pUhlisiwc\ in mJgazines. These typically have 16 rows, 16 columns, and 40 or so words. Suppose, however, we consider the variation where ol1ly the puzzle hoard is given and the word list is cssentially an English dictionary Both of the solutions proposed require considerJhlc time to solvc this problem and therefore arc not acceptable. Howcver,

it is possihle, evcll wilh a large word list, to solve the problem in a matter of seconds. An importam cOllCqJt is thaI, in mallY prohlems, writing" working program is not good

c-nough. If thc program is to be rull Oil a large data set, then the running tiJlle hecomes an issue Throughout this book we will sec how 10 eSlimate the running time of a program for large inputs and, more important, how to compare t he running limes of two programs

without actually coding them. \Ne will sec techniques for drastically improving the speed of a program and for drll'rmining program hottlenecks. These techniqucs will enable us to {lnd the section of the code on which to concentrate our optimization crrorts.

1.2 Mathematics Review This scction1isls some of the hasic formulas you need to memorize or he ahle to derive and

reviews basic proof tcchlliques.



1.2.1 Exponents

1.2.2 Logarithms

XAX Il =XA -l-- Il

x:" , _ )((\--B XIl -,

(XA)" =X,\R

x' + X N = 2XN i' X lN

2N + 2N = 2N j-I

1.2 Mathematics l<eVlew

In computer science, all logarithms arc to the- bast' 2 unless specifled otherwise.

Definition 1.1.

XA = IJ if and only if logx B =;\

Several convenient equalities follow from this deJinilion.

Theorem 1.1.

Proof. Let X = loge 13, Y = loge A, and Z = logA B. Then, hy the definition of logarithms, eX = 13, CY = A, and ilz = B. Combining these three equalities yields B = eX = (CYf. Therefore, X = YZ, which implies Z = X/Y, proving the theorem.

Theorem 1.2.

log All = log A + log B; A, IJ > 0

Proof. Let X = log A, Y = log 13, and Z = log AB. Theil, assuming the default base of 2, 2x = A,

2Y = fJ, and 22 = AB. Combining the las! three equalities yields 2x2)' = AH = i/. Therefore, X + Y = Z, which proves the theorem.

Some other useful formulas, which can all he derived in a similar manner, follow.

log 1=0,

logA//3 = log A - log B

log (A") = Illog A

log X < X for all X > 0

Jog2= I, Jog 1,024 = 10, log \,048,"576=20

;}



.. Uld(Jl~1 I IIIIIUUULlIUl1

1.2.3 Series The easiest rormulas to remember afC

and the companion,

N

2::2' =2N1'_ I

i=O

In the latler rormula, if (} < A < I) then

and as N Lenels to co, the sum approaches 1/(1 ~ A). These are the "geometric series" formulas.

Vic can derive the last forrnula for L~o Ai (0 < A < I) in the following manner. Let S he I he sum. Then

Then

/\5 =A +A' +/\1 +A4 +A' +.

If we subtract these two equations (which is permissible only for a convergent series), virlually all the terms on the right side cancel, leaving

which implies thm

S-AS= I

. I 5=~'-

I-A

Ilk can use this same technique 10 compute L~l il2i, a sum that occurs frequently.

\Vc write

I 2 3 4 5 5= - + - + - + - + -:c +.

2 2' 23 24 2'

and multiply by 2, obtaining

2 3 4 5 6 25= 1+ - + - + --c + - + - +.

2 22 2' 2" 2'

Subtracting these two equations yields

Thus, S = 2.

1 I 1 I I 5=1+-+-+-+-+-:c+·

2 22 21 24 2'



1.2 Mathematics Review

Another type of common series ill analysis is the arithmetic series. Any such series call he evaluated from the basic formula.

for instance, to find the sum 2 + 5 + 8 + ... + (3h - 1), rewrite it as 3(1 -+ 2 + 3+ ... + 11) - (I + 1 + 1 + ... + I), which is clearly 311(11 + 1)/2 - luI not her way to remem· ber this is to add the first and last terms (total 3k + 1), the second and next !O last terms (tolal 3" -+ I), and so on. Since there ,Ire /,/2 of these pairs, the total sum is h(3k + !)/2, which is the same answer as hefore.

The next two formu!;:ls pop up now anclthen but arc fairly uncommon.

t;2 = N(N + 1)(2N + I) '" N l

6 3 jo:--=l

When h = -1, the lalter formula is not valid. We then need the following, formula, which is used far more in computer science than in other mathcmaticnl disciplines. The numbers HN are known ;;lS the harmonic numbers, and the sum is known as a harmonic sum. The error in the following approximallon tends to y :::::::: 0.57721566, \vhieh is known as Euler's constant.

These two formulas arc just general algebraic manipulations.

N

LI(N) = Nf(N) i=-d

]\I N 110-1

LI(i) = Lf(;) - Lf(iJ i=l

1.2.4 Modular Arithmetic \;Ve say that A is congruent to B modulo N, written A::::: B (mod N), if N divides A - R. Intuitively, this means that the remainder is the same when either A or B is divided by N. Thus, 81"" 61 "" 1 (mod 10). As with equality, if 11£ Il (mod N), then A + C = II + C (mod N) and AD £ lllJ (mod N).

There are many theorems that apply to modular arithmetic, and some of them require extraordinary proofs in number theory. \;Vc will usc modular arithmetic sparingly, and the precedin.s theorems will suffice.

5 Bismillah hiir Rehman nir Raheem-----------------------------Assalat o Wasalam o Alika Ya RasoolALLAH


6 Chapter 1 Introduction

1.2.5 The P Word The two most common ways of proving statements in data structure analysis are proof by induction and proof by contradiction (and occasionally proof by intimidation, used by professors only). The best way of proving thaL a theorem is false is by exhibiting a coumcrexamplc.

Proof by Induction A proof by induction has 1 wo standard parts. The first step is proving a hase cast', that is, rstablishing that a theorem is true for some small (usually degenerate) value(s); this step is almost always trivial. Next, all inductive hypothesis is assumed. Generally this means that the theorem is assumed Lo be true for all cases up to some limit h. Using this assumption, the theorem is then shown to hc true for the next value, which is typically h + L This proves the theorem (as long as h is flnite).

As an example, Wf prove that the Fihonacci numbers, Fo = 1, F j = 1, h = 2, F-') = 3, F.f = 5, . ,Fj = Fi -_- 1 + Fi _ -2, satisfy Fi < ('5/3)i, for i 2:. 1. (Some definitions have /<"0 = 0, which shifts the series.) To do this, we first vcrify that the theorem is true for the trivial cases. It is easy to verify that F] = 1 < 5/3 and F2 = 2 < 2'5/9; this proves the basis. \Ve assume that the theorem is true for i = 1,2, . , Il; this is the inductivc hypothesis. To prove the theorel11, we need to show that Fh-t-l < (5/3)1<+1. \Ve have

[)/ 1- I = Fh + 1'"11 ___ I

by the definition, and we can usc the inductive hypothesis on the right-hand side, obtaining

Fid I < ('5/3)" + ("13),,-1

which simplif-ies to

proving the theorem.

< 0/5)(5/3)",1 + (3/5)'('5/3)'" I

< Cl/5)(5j3)h+1 + (9/25)('515/' "

PHI < 0/5 + 9/2'5)('5/3)1<11

< (24/2 '»)(5/3)1<11

< (5/3)"+1

;\s a second example, we establish the following theorem,

Theorem l.l.

II·' 1 I "N2 N(N + 1)(2N + I) N 2:. t len L..-j= I I = 6

Proof. The proof is hy induction. For the basis, it is readily seen that the theorem is true when N = 1. For the inductive hypothesis, assume that the theorem is true for 1 ::: It :::: N. V./c wilil'stahlish that, under this assumption, the theorem is true for 1\1 + 1. 'We have

N-+l N

2..>2 = I>' + (N + 1)2 i=l i=]



1.3 A Brief Introduction to Recursion

Applying the indLKli\'C hypothesis, we obtaill

:\' 1-1 .L ;' = l'Jii'i_+_1 )(2J'!_~ _I) + (N + I) 1

h,1 6

Thus,

=(N+ I) ~~~~+(N+ I) [1'1(21'1 + I) ]

= (N +

I>

2N 2 +7N+() I) ~--'---'---'

6

(1'1 + I)(N + 2)(21'1 + j)

6

'\-j-1 '\';' = (1'1 + 1)1(1'1 + I) + 1112(1'1 + 1) + II L. 6 Ic_l

proving the theorem.

proof by Counterexample 'I'll(> statement Fh S J?l is hllst'. The easiest way to prove this is to compute F j I = 144 > Ill.

Proof by Contradiction Proof by cOlllradiction proceeds hy <lssurning th;n the theorem is false ,md shnwillg! hat this

assumption implies that SOllle known properly is Ltlse, and bence the original assulilption was erroneous. ;\ classic example is the proof that there is an infinite number of primcs. To

prm'c Ihis, we aSSUlIll' that the theorem is falsc, so that there is some largest prime Ph' Lt't P j , P2, ... , 1\ he all the primes in order and consider

N=PrP2P)"'!)h+ 1

Clearly, N is larger than PII , so by assumption N is not prime. However, none or PI, Pl , ... ,PI, divides N exactly, bec,!L1sc lhere will always he a rcmaindcr of J _ This is a contradiction, hccause evcry numher is dlher prime or a product of primes. Hence, the original assumption, lhml\ is the largest prime, is false, which implies lhat the thc.orcmls

true.

1.3 A Brief Introduction to Recursion l\,lost lllathematical functions I hal we arc tmliliar wit h afC described hy a simple formula. I :01'

instance, we can COnYlTt temperatures from Fahrenheit to Celsius by applying the formula

C = )(1 12)/9

(;iven this formula, it is nivial to write a C++ fUllction; with declaralions and hraces

removed, the one-line formula translates to one line of C++. i'v1alhematical fUllctions arc sometimes defilled in a less standard form. As an eXilmple,

wc (all c1efine ;,1 fUllction f, valid on nOllllegativc integers, that satisfies ICO) = 0 and



8 Chapter 1 introduction

j int f{ int x

2 .1 if(x""O)

4 return 0;

.5 else

(i return 2 * f( x - 1 ) + x * x; 7

Figure 1.2 A recursive function

I(x) = 2f(x - 1) + x, From Ihis clefil1llioll we sec thatf( i) = 1,[(2) = 6,[(3) = 2i, and ],(4) = 58. A function Lhal is defined in terms of itself is called recursive. C++ allows functions to he recursive] It is important to rememher thm what C++ provides is merely an attempt to follow the recursive spirit. Not all mathematically recursive functions are efficiently (or correctly) implemented hy C++'s simulation of recursion. The idea is thai the recursive function I ought to be expressible in only a few lines, just like a non recursive function. Figure 1.2 shows the recursive implementation olJ.

Lines 3 and 4 handle what is known as the hase case, tlwt is, the value for 'which thc function is directly knO\vn without resorting to recursion. Just as declaring I(x) = ~f(x - 1) + x2 is meaningless, mathematically, without including the fact that [(0) = 0, the recursive C++ function doesn't make sense without a base case. Line 6 makes the recursive call.

There arc several important and possihly confusing points ahout recursion. A coml1lon qucstion is: Isn't this just circular logic? The answer is thaI although we arc defIning a function in tenns of itself, \-ve arc not dcfining a particular instance of the fUllction in terms of itsclf. In other words, cvaluatingI(5) by computingIO) would be circular. Evaluating I(S) by computingI(4) is not circular ..... -unless, of coufsc,f(4) is evaluated by eventually computingj(5). The two most important issues arc probably the how and why questions. In Chapter J, the how and why issues are formally resolved. Vic will give an incomplete description here.

It turns out that recursive calls are handled no dilfcrcntly from any others. HI is called with the value of 4, then line 6 requires the computation of 2 *.f0) + 4 * 4. Thus, a call is made to computcf(3). This requires the computation of2 *f(2) + 3 * 3. Therefore, another call is made to compute f(2). This means thaI 2 *I(l) + 2 * 2 must bc evaluated. To do so,fO) is computed as 2 *.1(0) +"1 * 1. Now,I(O) must be evaluated. Since this is a base case, \VC know a pliO!"! that feO) = O. This enables the compktion of the calculation for f (1), winch is now scen to be i. Thenf(2),jCl), and finally J (4) can he clClenninec1. All .he bookkeeping needed to keep track of pending fUllction calls (those started hut wailing for a recursive call to complete), along with thcirvariablcs, is done by the computer automatically. An important. pOint, however, is that recursive calls will keep on heing made until a base case is reached. For instance, an attempt to evaluate I( - 1.) will result in calls to I( ~ 2),

t Using recursioll for numerical calculations IS usually :\ had idea \Ve have d()ne so \0 illustrate the basic

pnlnts.



1.3 A Brief Introduction to Recursion

i nt bade int n

2

3 if(nooO

4 return 0;

5 else (, return bad( n/3+1) + n - I-

T

Figure 1.3 A llontcnninating rccur~i\'c function

I(~3), and so on. Since this wilt never get to ,l base case, the program won't be able to compute the answer (which is undefined anyway). Occasionally, a much more subtle error is made, which is exhibited in Figure 1.:1. The error in rigurc 1.3 is that bad(1} is defined, h)1 line 6, to be bad(l). Obviously, this doesn't give any clue as to what bad(1} actually is. The computer will thus repeatedly make calls to bad(1) in an <lUCl1lpt to resolve its values. EvcnLually, its bookkeeping system will run out of" space, and the program will terminate abnormally. Cenerally, we would say that this function doesn't work for olle special casc but is correct otherwise:. This isn't true here, since bad(2} calls bad(1). Thus, bad(2) cannot be evaluated either. furthermore, bad(3}, bad(4}, and bad(5) all make calls to bad(2). Since bad (2) is uncvaluahlc, none of these values arc either. In bct, this program doesn't work for

any nonnegative value of n, except o. \Vith recursive programs, there is no such thing as a "special case"

These considerations lead to the (irst two fundamental rules of recursion:

1. 13a:,;c cases. You must always havc some base cases, which can be solved without recursioll.

2. lv/ai?il1g progress. for the cases thm arc to be solved rrcursively, the recursivc call must always he 10 a GISe Ihat makes progress toward a base case.

Throughout this book, we will use recursion lO solve problems. As an example of a nonmathcmatical usc, consider alargc dictionary. \IVords in dictionaries arc defined in terllls

of other words. \Vhen we look up a word, we might not always understand the definitioll, so we might havc to look up words in the ddinitiorL Likewise, we might not understand some of those, so we might have to continue this search for a while. Because the dictionary

is finite, eventually either (I) we will come to a point where we understand all of the words in some definition (;md tbus understand that definition and rctracc our path through the other definitions) or (2) we will find thal the definitions are circular and we are stuck, or that some word we need to understand for a dcflnition is not in the dictionary.

Our recursive st rategy to underst.and words is as follows: If we know the mcaning of a word, then we arc done; otherwise, we look the word up in lhe dictionary. If wc understand all thc words in the definition, wc arc done; otherwist', we figure out wha1 the definition means by recursively lookillg up the words we: don't know. This procedure will terminate if thc dictionary is well defined hut can loop indefinitely if a word is either not defined or

circularly deflned.

9




I void printout( 'int n) II Print nonnegative n

2 3 if ( n >e 10 )

'I printOut( n / 10 );

5 printDigit( n % 10 ); 6

Figure 1.4 Recursive routine to print an intcger

Printing Out Numbers Suppose \vc have a positive integer, 11, that we wish to print out. Our rou1ine will have the heading printOut(n). Assume that the only 110 routines available will take a single-digit number and OlllPUl it to the terminal. \\.'1: will cal! Ibis routine printDigit; for example, printDigit(4) will Olltput a 4 to the terminaL

Recursion provides a \-Try clean sulution \0 Ihis problem. To print nut 76234, we need In fIrst print oul 7623 and then print out 4. The second step is easily accomplished with the slatement printDigit(n%10), hut the first doesn't seem any simpkr than the original problem. Indeed i! is virtually the same problem, ~o we can ~oJve it recursively with the statnncnt pri ntOut (njlO).

Ihis tdls us how to solve the general prohlem, but we still necd 10 make sure lh~1t

the program docsn'L loup indefinitely. Since we haven't defined a base case yet, it is clear that we still have something tn do. Our base case will be printOigit(n) if 0 :s n < 10. Now pri ntOut{n) is defined for evcry positive number from () to '1, and larger numbers arc defined in \crms of a smaller positive numher. Thus, there is no cycle. The entire function is showll in Figure 1.4.

vVc have made no effort to do Ihis efficicntly. vVe could have avoided using the mod rmlline (which can be vcry expensive) hecause 11% 10 = n - LIlJ l.OJ * 10.2

Recursion and Induction Let us prove (somewhat) rigorously that the recursive number printing program works. To do SO, \\,c'llusc a proof by induction.

Theorem 1.4.

The recursive number-printing algorithm is correct for 11 ::::. o.

Proof. (By induction on the number of digits in n) First, if 11 has one digit, then the program is trivially correct, since it merely makes i.1 call to printDigit. Assumc then that printOut works for all numbers of h or fewer digits. A number of h + 1 digits is expressed by its first h digits followed by its least Significant digit. HUI thc number formed by the first h digits is exactly LnJIOJ) which, hy the inductive hypothesis, is correeily printed, and the last digit is n mod 10, so the program prints OUl any (I< + I)-digit numher correctly. Thus, by induction, allnumhers <Ire correctly printed.

2 LxJ is [he largest integer thai is less lh(ln \lr equal Iu x



1.4 C++ Classes

This proof prohably seems a little strange in [hat it is virtually identical w the algorithm description. It illustrates that in designing a recursive program, all smaller instances of the same problem (which are on the path to a base case) may be assumed to work correctly The recursive program needs only to combine solutions to smaller prohlems, which arc "magically" ohtained hy recursion, into a solution for the current problem. The mathematical justification for this is proof by induction. This gives the third rule of recursion:

3. Design rule. Assume that all the recursive calls work.

This mle is important because it means that when designing recursive programs, you generally don't need to know the details of the bookkeeping arrangements, and you don't have to try to trace through the myriad of recursive calls. frequently, it is ext rcmdy diITicult to track down the actual sequence of recursive calls. Of course, in many cases this is an indication of a good use of recursion, since the computer is being allowed to work out the complicated details.

The main problem with recursion is the hidden bookkeeping costs. Although these costs afe almost always justifiable, because recursive programs not only simplify the algorithm design but also tend to give cleaner code, recursion should never be used as a substit ute for a simple for loop. We'll discuss the overhead involved in recursion in more detail in Section ~~.6.

When writing recursive routines, it is crucial to keep in mind the four basic rules of recursion:

1. Ra5e cases. You must always have some hase cases, which can be solved without recursion.

2. Mal~il1g progress. for the cases that are to be solved recursively, the recursive call must always be to a case that makes progress toward a base case.

3. Design rule. Assume that all the recursive calls work.

4. Compotwd illl.crest rule. Never duplicate work by solving the same instance of a problem in separate recursive calls.

The f()urth rule, \vhich will be justified (along with its nickname) in later sections, is the reason that it is generally a bad idea to usc recursion to evaluate simple mathematical functions, such as the fibonacci numhers. As long as you keep these rules in mi.nd, recursive programming should be straightforward,

1.4 C++ Classes In this texl, we will write many data structures. All of the data structures will be objects that store data (usually a collection of idcntically typed items), and provide functions that manipulate the collection. in C++ (and other languages), this is accomplished by Llsing a

class. This section describes the C++ class.




1.4.1 Basic c 1 ass Syntax A class in C++ consists of ilS memhers. These mcmhers can he either data or funClions. The ["unctions afC called mcmher fUllctions. L;:lCh instance or a class is an ()I~iccl. Each nbjcct contains the data components specified in lht class (l\n!c~s lhc data cornponcnts arc static,

J dctailllwt c;:m be safely ignllful for now). i\ member fUl1Clion is llsed [0 act on <-111 OhjcCl OftCll memher rUllclions are called methods.

As an example, Figure 1.5 is the IntCell class. llllhc IntCell class, each instance of the I ntCe 11---(ln I ntCe 11 objCCI-col11ains a single data I11cm her named s toredVa 1 ue. Everything

else in this panicularclass is a tnnhnd. In our l'xamplc, there arc four rnclhods. Two of these rnctllocls arc read and write. The other two arc special methods known as constructors. Let us descrihe some key features.

Firs\, notice the two lahels pub 1 i c and pri vate. ·rhese labels deteruline visihility of

class members. In this cX;1.mplc, cverythillg except the storedValue data I11Clllhcr is publ ic.

storedValue is private. 1\ member that is public rnay he accessed hy any method ill any class. i\ member thi.1l is private BUY only he accessed hy methods in its chtss. '[\,pically, data memhers art: declared private, thus restricting aCtTSS 10 internal details or dIe class, while methods intcnded for gencral usc arc made publ ie. This is known as information hiding. By using private data lllembers, we call change the internal representation of the object withollt having all efrect on other parts of (bc program that use the object. This is because the object is accessed througll the public member flll1L"lions, whose viewable hch;\vior remains unchanged. The users or 1he class do not need to know internal details of how the class is implementcd. In many cases, having this access leads to trouble. For instance, in a class that stores dales using month, day, and year, hy making the mOllth, day, and year private, wc prohihit an outsider fm[1l selling these data members to illegal elates, sllch as Feh 29, 200 I. However, some lllcthods Oldy he for intcrnal usc, and call he pri vate.

III a class, all members arc private by default, so the initial publ ie is not optional. Second, wc see two constructors. ;\ constructor is a method that describes how an

instance or the class is construct cd. if no elms! ruClor is exp] ieitly defined, one that initi<llizc:. !Ill' data 11lL'lllhCl"s using langudgc defaults is aUlomatically generated. Thc IntCell class defines two constructors. The first is called if no parameter is specified. The sccond is called jf an i nt parameter is provided, and uses that i nt to initialize till' storedVa 1 ue 1lH.:rnlwr.

1.4.2 Extra Constructor Syntax and Accessors Although thc class works ;15 written. there is SOl11e extra syntax that makes for hetter code. Four changes arc shown ill Figure!.6 (wr omit. COl11lllents for brcvity). Tbe dillcrcnces arc as follows:

Default Parameters The IntCell constructor illustrates tht' default parameter. As a result, there arc still two IntCell constructors defilled. One acccplS an i ni t i alVa 1 ue. The other is the zero-parameter constructor, which is implied because the onc-pantmcler eonst ructor says that i ni ti al Va lue

is optional. The derault value or 0 ~ignifies lilat 0 is used if" no parametCJ" is provided. Default parameters c<ln be used in any fUllctioll, hUllhcy arc most commonly used in constructors.



I /** 2 * A class for simulating an integer memory cell .

.J */ 4 class IntCel'

5

6 public:

7 j'* 8 * Construct the IntCell.

9 * Initial value is 0,

JO */ J1 IntCell ( )

12 I storedValue O;}

I.l

14 /** 15 * Construct the IntCell.

/6 * Initial value is initial Value.

17 */ 18 IntCell( int initialValue)

19 { storedValue = initialValue;

20 21 j** 22 * Return the stored value.

23 */ 24 int read(

25 return storedVa 1 ue;

26 27 /** 28 * Change the stored value to x.

29 */ 30 void I'Jrite( lnt x )

31 { storedValue = x;

32 33 private:

.34 int storedValue;

35 };

Figure 1.5 A complete declaraLion of an IntCell class

Initializer List

1.4 C++ Classes

The IntCe 11 cons! ructOf uses an initiaHzer list (figure I .6, line 8) prior [0 the body of the constructor. The initializer list is used to initialize the data members directly. In Figure 1.6, there's hardly a difference, but using initializcr lists instead of an assignment statement in the body s,wes time in the case where the data members are class typcs that have complex initializations. In somc cases it is required. for instance, if a data memher is canst (meaning that it is not changeable after the object has been constructed), then the clata member's value

13



14 Lnapter I ImrooUUlon

/** 2 * A class for simulating an integer memory cell. 3 *1 4 class IntCell

-' 6 publ ic:

7 explicit IntCell( int initialValue = 0 } 8 storedValue{ initial Value } { }

9 int read( ) canst

10 { return storedValue;

11 void write( int x

12 { storedValue '" X; }

13 14 private: 15 int storedValue;

16 };

Figure 1.6 JntCe 11 class with revisions

can only be initialized in the initializer list. Also, if a data member is itself a class type that does not have a zero-parameter constructor, then it must be initialized in the initializer list.

expl i cit Constructor The IntCell constructor is expl icit. You should make all one-parameter constructors expl i ci t to avoid bchincl-tht>scenes type conversions. Otherwise, there are somewhat lenient rules that will allow type conversions without explicit casting operations. Usually, this is unwanted behavior that destroys strong typing and can lead to hard-to-find bugs. As an example, consider the following:

I ntCe 11 obj;

obj " 37;

Ilobj is an IntCeII

/1 Shou 1 d not compil e: type mi smatch

The code fragment above constructs an IntCell object obj and then performs an assignment statement. But the assignment statement should not work, because the right-hand side of the assignment operator is not another IntCell. obj's wri te method should have been used instead. However, C++ has lenient rules. Normally, a one-parameter constructor defines an implicit type conversion, in which a temporary ol~iect is created that makes an assignment (or parameter to a function) compatible. In this case, the compiler would attempt 10 convert

obj " 37; II Should not compile: type mismatch

into

IntCell temporary 37;

obj '" temporary;

Notice that the construction of the temporary can be performed by using the Oncparameter constructor. The use of expl iCi t means that a one-parameter constructor cannot



1A C++ Classes

be used to generate an implicit temponuy Thus, since IntCell's constructor is declared exp 1; cit, the compiler will correctly complain that there is a type mismatch.

In Section 7.8, we'll sec an eXJmplc in which the lenient rules arc helpful, hU! this is the exception, rather than the rule.

Constant Member Function A member function that examines but does not change the state of its ohject is an accessor. A member function that changes the state is a mutator (because it mutates the state of the object). In the typical collection class, for instance, isEmpty is an accessor, while makeEmpty is a mutator.

In C++, we can mark (Jch member function as being an accessor or a mutator. Doing so is an important parl of the design process and should not be viewed as simply a comment. Indeed, there are importam semantic consequences. r;or instance, mutators cannot he applied to constant ohjects. By default, all member functions are mUl<ltors. To make a member function an accessor, we must add the keyword const after the closing parenthesis that ends the parameter type list. The const-ness is part of the signature. canst can be used ·with many different meanings. The function declaration can have canst in three different contexts. Only the canst <.lfter a closing parenthesis signifies an accessor. Other uses are described in Sections 1.'5.2 and 1.'5.3.

In the IntCell class, read is clearly an accessor: il docs not change the state of the IntCell. Thus it is made a constant member function at line 9. If a member fUllction is marked as an accessor, but has an implementation that changes the value of Hlly claw member, a compiler error is generated. 1

1.4.3 Separation of Interface and Implementation The class in figure 1.6 COTllains all the correct syntactic constructs. However, in C++ it is more common to separate the class interface from its implementation. The interl~lCc lists the class and its memhers (data and functions). The implementation provides implementations of the functions.

Figure 1.7 shows the class interface for IntCell, figure I.B shows the implemetw.Hion, and figure 1.9 shows a main routine that uses the IntCel1. Some importcU1t points follow.

Preprocessor Commands The interface is typically placed jn a file that ends with .h. Source code that requires knowledge of the interface must #include the il1lerface file. In our case, this is both the implementation file and the file that contains mai n. Occasionally, a complicated project will have files including other files, and there is the danger that an interLtce might be read twice in the courSe of compiling a file. This can be illegal. To guard against this, each header file uses the preprocessor to define a symbol when lhe dass interface is read. This is shown on the first two lines in Figure .1.7. The symbol name, IntCell_H, should not appear in any other file; usually, we const ruct it from the filename. The first line of the interface file

I Data members can be marked mutable to indicate that (Ol1st-l1ess should not apply tn them

15




lilndel IntCell_H 2 #define IntCell H 3 4 lid :5 * A class for simulating an integer memory cell.

6 * / 7 class IntCell 8 9 public:

10 explicit JntCell( int initialValue 0); 11 int read( ) const; 12 void write( int x ); U J4 private: 15 int storedValue;

16 };

17

1R lendil

Figure 1.7 IntCell class interface in hIe IntCel!.h

tests whether the symbol is undefined. If so, we can process the file. Otherwise, we do nol process the file (by skipping to the #endif), because we knmv Ih;:H we have already read the lile.

Scaping Operatar In the implementation file, which typically ends in .cpp, .ce, or .C, each member function must idemify the class that it is part of. Otherwise, it would be assumed thaI the function is in global scope (and zillions of errors would resulL). The syntax is ClassName: :member.

The:: is called the scoping operator.

Signatures Must Match Exactly The signature of an impkmented member function must match exactly the Signature listed

in the class interface. Recall that whether a member function is an accessor (via the canst at the end) or a mutator is part of the signature. Thus an error would result if, for example, the canst was omitted from exactly one of the read signatures in Figures 1..7 and] .S. Note that default parameters arc specified in the jnterface only. They are omitted in the

implementation.

Objects Are Declared Like Primitive Types In C++, an object is declared just like a primitive type. Thus the following are legal

declarations of an IntCell object:

IntCell objl; IntCel1 obj2! 12 );

1/ Zero parameter constructor /1 One parameter constructor



#include "IntCell.h '! 2

J j** 4 * Construct the IntCell with initial Value 5 */ 6 IntCell;~IntCell( int inltialValue) : storedValue( initialValue)

7 (

8 )

9

10 /** 11 * Return the stored value. 12 */ 13 int IntCe'1::read( ) canst

14

15 return storedValue;

/6

17

18 /** 19 * Store x. 20 */ 21 void IntCell::write( int x)

22 ( 23 storedValue = X;

24

Figure 1.8 IntCell class implementation in file IfltCcll.cpp

On the other hand, the following are incorrect:

rntCe1l obj3 = 37;

rntCell obj4( );

II Constructor is explicit /1 Function declaration

1.4 (++ (lasses

The declaration of obj3 is illegal because the one-parameter construClor is explicit.

It would ix legal otherwise. (Ill other words, a declaration that uses the onc-parameter constructor must use the parentheses to signify the initial value.) The declaration [or obj4 states that it is a function (defined elsewhere) Ih:l! takes no parameters and returns an InlCe 11.

1.4.4 vector and stri ng

The C++ standard defines two classes: the vector and string. vector is intl'nucd to replace the built-in C++ array, which causes no end of trouble. The problem with the built-in C++ array is that it docs not behave like a first-class object. For instance, built-in arrays cannot be copied with "', a built-in array docs not remember how many items it can store, and its indexing operator does not check that the index is valid. The huilt-in string 1S simply an array of characters, and t.hus has the li~,bi1ities of arrays plus a few more. for instance, "'"" does not correctly compare two built-in strings.



18 Chapter I Introduction

I #include <iostream> 2 #include "lntCell .h" 3 using namespace std: 4 .') int main{ ) (,

7 IntCell m; II Or, IntCell m( 0); but not IntCell m();

8 9 m.write(5);

10 cout« "Cell contents: " «m.read( ) «endl;

II

12 return 0;

13

Figure 1.9 Program that lIses IntCell in file Ji:si1nlCell.cpp

#include <iostream> 2 Uinclude <vector> .'3 using namespace std;

'I :5 int main( )

(, I 7 vector<int> squares( 100 ); H

9 fore int i = 0; i < squares,size( ); i++ )

10 squares [ i ] = i * ;; I I

/2 fore int i = 0; i < squares,size( ); i++)

13 cout « i « " 10 « squares[ ; ] « endl;

14

15 return 0;

16

Figure 1.10 Using the vector class: stores 100 squares and outputs them

The vector and 5 t ri n9 classes in 1 he STL treat arrays and strings as hrst ~class objects. A vector knmvs how large it is. Two string objects can be compared with =OO, <, and so on. Both vector and string can be copied with "'. lfpossihlc, you should a\'oid using the builtin C++ array and Siring. We discuss the huilt~in alTay in Chapter 3 in the context of showing how vector can be implemented.

vector and stri ng are easy to usc. The cock in Figure 1.10 creates a vector that stores one hundred perfect squares and outputs thel11. Notice also that size is a method that returns the size of the vector. A nice feature- of the vector that we explore in Chapter .3 is that it is easy to change its sizc. In many cases, the initial sizc is 0 and tht' vector grows as needed.



1.5 C++ Delaiis

string is also casy to usc and has all the relational and equality operators to compare the states of two strings. Thus strl"'''''str2 is true if the value of the strings arc the same. It also has a 1 ength meLhod that returns the sIring length.

1.S C++ Details Like any language, C++ h8:5 its share of details and language features. Some of these are discussed in this section.

1.5.1 Pointers A pointer variable is a variable that stores the address where another object resides. It is the fundamental mech,mism used in many data structures. for instance, to store a list of items, \ve could use a contiguous array, but insertion into the middle of the contiguous array requires relocation of many items. Rather than SLOre the collection in an array, it is common LO store each item in a separate, noncontiguous piece of memory, which is allocated as the program runs. Along with each object is a link to the next object. This link is a pointer variable, because it stores a memory location of another object. This is the classic linked list that is discussed in more detail in Chapter 3.

To illustrate the operations that apply to pointers, we rewrite Figure 1.9 to dynamically allocate the IntCell. It must be emphaSized that for a simple IntCell class, there is no good reason to write the C++ code this way. We do it only to illustrate dynamic memory allocation in a simple context. Later in the text, we will see more complicated classes, where this technique is useful and necessary. The new version is shO\vn in Figure 1.11.

Declaration Line 3 illustrates the declaration of m. The * indicates that m is a pointer variable; it is allowed to point at an I ntCe 11 object. The value of m is the address of the object [hat it points at. m is uninitialized at this point. In C++, no such check is performed to verify that m is assigned a value prior to being used (however, several vendors make products that do additional checks, including this one). The use or uninitialized pointers typically crashes programs, because they result in access of memory locations that do not exisl. In general, it is a good idea to provide an initial value, either by combining lines 3 and 5, or by initialiZing m to the NULL poinler.

Dynamic Object Creation Line 5 illustrates how objects can be created dynamically. In C++ new returns a pointer to the newly created object. In C++ there are two ways to create an object using its zero~parameter constructor. Both of the folloWing would be legal:

m " new IntCell ( ); m '" new IntCell;

II OK II Preferred in this text

We generally usc the second form because of the problem illustrated by obj4 in Section 1.41

19




1 int main( )

2 .3 IntCell *m;

4 5 m o nel1lntCel1(O);

6 m->wri te ( 5 );

7 cout« "Cell contents: " «m->read( ) «endl;

8

9 delete m; 10 return 0;

11

Figure 1.11 Program that uses pointers to IntCell (there is 110 compelling reason to do .his)

Garbage Collection and de 1 ete in some languages, when an object is no longer rderencecl, it is subject to automatic garbage collection. The programmer does not have to worry about it. C++ does not have garbage collection. \Vhcn an object that is allocatcd by new is no longer referenced, the delete operation must be applied 10 the object (through a pointer). Otherwise, the memory that it consumes is lost (until the program terminates). This is known as fl memory leak Memory leaks arc, unfortuI1aLdy, common occurrences in many C++ programs. fortunately, many sources of memory leaks can be automatically removed v·/ith care. One important rule is to

not use new when an automatic variahle can be used instead. In the original program, the IntCell is no! allocated by new but instead is allocated as a local variable. In that case, the memory for the IntCell is automatically reclaimed when the function in which it is declared returns. The del ete operator is illustrated at line 9 of I~igurc 1.11.

Assignment and Comparison of Pointers Assignment and comparison of pointer variables in C++ is hased on the value of the pointer, meaning the memory address that it stores. Thus two pointer variables are equal if they point at rhe same object. If they point at different objects, the pointer variables afe not equal, even if the objects being pointed al arc themselves equal. If 1 hs and rhs arc pointer variables (of compatible types), then 1 hs"'rhs makes 1 hs point at the same object that rhs points a[."}

Accessing Members of an Object through a Pointer If a pointcrvariablc points at a class t.ype, then a (visible) member oflhl'. object heing pointed at can be accessed via the -> operator. This is illustrated at lines 6 and 7 of Figure 1.11.

Other Pointer Operations c++ allows all sorts of bizarre operations un pointers which are occasionally useful. FOI

instance, < is defined. For pointers lhs and rhs, lhs<rhs is true if the object pointed at by

-1 Throughout Ihis lext, we use 1 hs and rhs 10 signify Icjt-lumd sde <.Illd right-hand side 01 a biliary operator



1.5 C++ Details

lhs is stored at a lower memory location than the object pointed at by rhs. There is rarely a good reason 10 use this construct. I--lO\vcvcr, one example of an equally unusual operation is illustrated in Section 7.B.

One important operator is the address-of operator &. This operator returns the memory location where an object resides and is useful for implementing an alias test that is discussed in Section 1.5.'5.

1.5.2 Parameter Passing 1\ll3ny languages, C and Java included> pass all parameters using call by value: the actual argument is copied into the formal parameter. However, parameters in C++ could be large complex ohjects for which copying is inefficient- Additionally, sometimes it is desirable to be able to alter the value being passed in. As a result of this, C++ has three different ways to pass parameters. However, there. is a simple rule 10 decide which method to use.

The three parameter passing mechanisms arc illustrated in the following function declaration that returns the average oflh{' fIrst n integers in arr, and sets errorFlag to true if n is larger than arr.sizeO or smaller than].

double avg( canst vector<int> & arr, int n, bool & errorFlag );

Here, arr is of typc vector and is passed using call by constant reference, n is of type i nt and is passed using call by value, and errorFl ag is of t yre boo 1 and is passed using call by reference. The panunetcr-passing mechanism can generally be decided by a two-part test:

1. If the formal parameter should be able to change the value of the actual argument, then you must usc call hy reference.

2. Othenvisc, the value or the actual argument cannot he changed hy the formal parameter. If the type is a primitive type, usc call hy value. Otherwise, the type is a class type and would generally be passed using call by constant reference.'

In the declaration of avg, errorFl ag is passed by reference so that the new value of errorFl ag

will be reflcclccl in thc actual argument. arr and n will not be changed hy avg. arr is P;.lsscc! by constant reference because it is a class type, and making a copy would be. too expensive. n is passed by value because it is a primitive type and is cheaply copied.

To summarize the parameter-passing options:

L Call hy value is appropriate ror small objecls that should not be altered by the function.

2. Call by constant reference is appropriate for large objects that should not be altered by the function.

3. Call by reference is appropriate for all objects that may he altered by the fUllction.

') Ho\Vcwr, clnss types that arC small (for inslance, I hose IhaL Slore only a single built-in Iypc) can be passed

using, call by value instead of call hy constant reference

21




canst string & findMax( canst vector<string> & arr ) 2 (

3 i nt maxIndex '" 0; 4 :5 for( int i '" 1; i < arr.size( ); i++

6 if( arr[ maxlndex ] < arr[ i ] )

7 8

rnaxlndex '" ;;

9 return arr[ maxlndex ];

10

II

12 canst string & f;nd~laxWrong{ canst vector<string> & arr )

Ll

14 string maxValue arr[ 0 ]; 15 16 for( 1nt i '" 1; i < arr.size( ); i++ )

17 if( maxValue < arr[ ; ] ) 18 maxValue = arr[ i J; 19 20 return maxValue; 21

Figure 1.12 '[wo versions to find the maximum string; only the first is correct

1.5.3 Return Passing Objects can also be returned using return by value, return by constant reference, and, occasionally, return by reference. For the most part, do not use return by reference. In Section 1.7.2, we will see onc example where it is useful, but this is rarc.

It is ahvays safe to use return hy value. IIowevcr, if the object heing returned is a class lype, it may be better to use rcturn by constant reference, to avoid the overhead of a copy6 However, this is only possible if it is guaranteed that the expression in the return statement has a lifetime that extends past the return of the function. This is a very tricky part of C++, and many compilers will fail to give a warning message for incorrect use.

As an example, consider the code in Figure] .12, which contains two nearly identical functions, to find the largest (alphahetically) string in an array. Both attempt to return the value hy constant reference. The first version, fi ndMax, shows acceptable usc: the expression a[maxlndex] indexes a vector that already exists outside of fi nd~1ax, and will exist long after the call returns. The second version is wrong. maxValue is a local variable that does not exist when the function returns. Thus it is improper to return without making a copy of it. If the compiler fails to complain, then the return value mayor may not contain useful

6 The const here means th<lt the ohject heing returned cannot ilse\fbc modified later on. IL is dilrerent from

the const in the parameter list and the canst thai signifies an accessor.



I.) L ++ Uetalls

information, depending on how quickly the compiler decides to reclaim the memory that was used by maxValue. This makes for (1 diHicult clebuggingjob.

1.5.4 Reference Variables Reference and constant reference variables arc commonly used for parameter passing. But they can also be used as local variables or as class data rnernbers. In these cases, the variable names become synonyms for the objects thal they reference (much as the formal parameters hecome synonyms for actLIal argumcnts in call hy reference). As 10("<11 variables, they avoid the cost of a copy and thus are useful when querying a data structure that conwins a collection of d,lSS types. Thus. in many cases, client code such as

string x = findMax( a );

cout « X « endl ;

is better written as

canst string & x findMax( a );

cout « X « endl;

;\ second usc, which we will see in Chapter .'J, is to usc a local rderence variahle solely for the purpose of renaming an ohject thm is known by a complicated expression. The code we \vill sec is similar to the following:

list<T> & \vhichList = theLists[ hash( x, theLists.size( ) ) ];

if( find( whichList.begin( ). whichList.end( ), x) != whichList.end( ) }

return false;

el se

I'IhichList.push_back( x );

A reference variable is used so tbat the considerably more complex expression, theL i sts [hash (x, theL i sts. s i zeO)], docs not. have to be \vriLten (and then evaluated) four lirnc5.

Reference variables can he lIsed as class data members, thougb we do not do Ihis in the text. References must be initialized (to the object that they will reference).

1.5.5 The Big Three: Destructor, Copy Constructor, operator=

In c++, classes come with three special runctions that arc already written for you. These afC the destructor, copy constructor, and operator"'. In many cases, you can accept the default behavior provided by the compiler. Sometimes you cannot.

Destructor The destructor is called whenever an object goes out of scope or is sul~iected to a de 1 ete.

Typically, the only responsibility of the destructor is to free up any resources that were



.... U Idlll!:!1 I II Hf UUUl,.lIUIt

allocated during the usc of the object. This includes GIBing delete for any corresponding news, closing any files that were opened, and so on. The ckfault simply applies the destructor on each data member.

Copy Constructor There is a special constructor that is required to construct a new object, initialized to a copy of the same type of object. This is the copy constructor. for any object, such as an IntCel1 ohject, a copy constructor is called in the following instances:

a declaration with initialization, such as

IntCell B " C;

IntCell B( C );

hut not

B '" C; /1 Assignment operator, discussed later

an object passed using call hy value (instead of by & or canst &), which, as mentioned carlier, should rarely be done, anyway

an object returned by value (instead of by & or const &)

The first case is the simplest to understand hecause the constructed objects were explicitly requested. The second and third cases construct temporary objects that arc never seen by the user. Even so, a construction is a construction, and in both cases, we are copying an object into a newly created object.

By default the copy constructor is implemented by applying copy constructors to each data member in turn, For data members that arc primitive types (for instance, int, double, or pointers), simple assignmcm is done. This would be the case for the storedValue data member in our IntCell class. for data members that are themselves class objects, the copy constructor for each data member's class is applied to that data member.

operator= The copy assignment operator, operator"', is called when'" is applied to lwo ohjects after they have hoth been previously constructed. lhs=rhs is intended to copy the state of rhs into 1 hs. By default, the operator'" is implemented by applying operator'" to each data member in turn.

Problems with the Defaults If we examine the IntCell class, we sec that the defaults arc perfectly acceptable, and so we do not have to do anything. This is often the case:. 11" a class consists of data members that are exclusively primitive types and objects for which the defaults makc sense, the class defaults will usually make sense. Thus a class whose data members arc int, double, vector<int>, string, and cven vector<string> can accept the defaults.

The main problem occurs in a class that contains a data member that is a pointer. We will describe the problem and solutions in detail in Chapter 3; for now, we can sketch the prohlem. Suppose the class contains a single data member that is a pointer. This pointer points at a dynamically allocated objcct. The default destructor for pointers does nothing (for good reason~recall that we must del ete ourselves). funhermore, the: copy constructor



1.3 l.."""" ueralls

IntCell::-IntCell( )

2 (

3 II Does nothing, since IntCell contains only an lnt data

4 II member. If IntCell contained any class objects, their

5 II destructors would be called.

i>

7 8 IntCe 11 : : I ntCe 11 ( const IntCe 11 & rhs) s toredVa 1 ue ( rhs. s toredVa 1 ue )

9 (

10 )

11

12 const IntCell & IntCell::operator"( const IntCel1 & rhs)

13

14 if( this !"' &rhs} II Standard alias test

15 storedValue '" rhs.storedValue;

16 return *this;

17

Figure 1.13 The ddau1!s for the big three

and operator= hoth copynollhe objects being pointed ill, but simply the value of the pointer. Thus we will simply hJVC two class instances that contain pointers Ihat point to the same ohject. This is a so-called shallow copy. "Iypically, we would expecl a deep copy, in which a clone of the entire object is macir. Thus, when a class contains pointers as data members, and deep semantics arc important, we typically must implement the destructor, operator"',

and copy construclOr ourselves. For IntCell, the signatures ofthesc operations are

-IntCell( ); II destructor

IntCell ( canst IntCell & rhs ); II copy constructor

const IntCell & operator""( const IntCell & rhs );

Although the dehllllts for JntCell are acceptable, we can \vrite the implementations anyv·/3Y, as shO\vn in Figure 1.13. For the destructor, after the body is executed, the destructors arc automatically called for the data members. So t.he ckhmlt is an empty body. For the. copy constructor, the default is an initializer list of copy constructors, followed by execution of the body. Notice that if nothing is in the inilializer list, rather than getting a cop)" each data member gets a dej~lUlt Czero-parameter) initialization.

operator'" is the mosl interesting. Line 14 is an alias test, to make sure we arc not copying to ourselves. Assuming we are not, we apply operator'" to each data mernher Cat line 15).

We then return a reference to the current object, at line 16, so assignments can be chained, as in a=b"'c.

In the routines Ihat we write, if the defaults make sense, we will always accept them. I [owever, if the defaults do not make sense, we will need 10 impkmeI1l tbe destructor, and operator"', and the copy constructor. \Vhen the default docs not work, the copy constructor can generally be implemented by mimicking normal construction and then



.. D Lndpler I IIHruUUlUUfl

I class IntCell 2 3 public:

4 explicit IntCell ( int initialValue '" 0 )

5 storedValue '" new int( initialValue }; } [,

7 int read{ } const 8 return *storedValue;

I) void write( int x )

10 { *storedValue '" x; } } I private:

12 int *storedValue;

Ll };

Figure 1.14 Dala memher is a pointer; defaults afC no good

calling operator"'. Anotheroftcn-uscc1 option is to give a reasonable workingimplcmentation of the copy constructor, but then place it in the private section to disallow call by value.

When the Defaults Do Not Work The most common situation in which the defaults do not work occurs when a data member is a pointer type, and the pointee is allocated by some ohject member function (such as the constructor). As an example, suppose we implement the IntCell by dynamically allocating an i nt, as shown in Figure 1.14. For simplicity, we do not separate the interface and implementation.

There are now numerous problems that arc exposed in Figure 1.1 S. First, the output is three 4s, even lhough logically only a should be 4. The problem is that the dd~1Ult operator'" and copy constructor copy the pointer storedVal ue. Thus a. storedVa 1 ue, b. storedVa 1 ue, and c.storedValue all point at the same int value. These copies are shallow: the pointers rather than the pointers afC copied. A second, less obvious problem is a memory leak The int initially allocated by a's constructor remains allocated and needs to be reclaimed. The int allocated by d; constructor is no longer referenced by any pointer variable. It also needs to

he: reclaimed, but we no longer have a pointer to il. To fix these problems, we implement the big 1hree. The result (with the interface and

implementation separated) is shown in Figure 1.16 (on page 28). Generally speaking, if a destructor is necessary to reclaim memory, then the defaults for copy assignment and copy construcLion are not acceptable.

U the class contains data members that do not have the ability to copy themselves, then the default operator= will not work We will see some examples of this later in the tex\.

1.5.6 (-style Arrays and Strings The C++ language provides a built-in C style array type. To declare an array, arrl, of 10 integers, one \Vri1es:

int arrl[ 10 l;



1.5 (++ Details

i nt f( )

2

3 IntCella(2); 4 IntCel1 b • a; 5 IntCell c; (,

7 c '" b;

8 a.write( 4 );

9 cout« a.read( ) «endl «b.read( ) «endl «c.read( ) «endl; 10 return 0; 11

Figure 1.15 Simple function that exposes problems in Figure 1.14

arr1 is actually a pointer to memory that is large enough to store 10 ints, rather than a first~c1ass array type. Applying'" to arrays is thus an attempt to copy two pointer values, rather than the entire array, and with the declaration above, it is illegal, because arr1 is a constant pointer. When arrl is passed to a function, only the value of the pointer is passed; information about the size of the array is lost. Thus the size must be passed as an additional parameter. There is no index range checking, since the size is unknown.

In the declaration above, the size of the array must be known at compile time. 10 cannot he replaced hy a variable. 1f the size is unknown, we must explicitly declare a pointer and allocate memory via new[]. For instance,

int *arr2 '" new int[ n ];

Now arr2 behaves like arrl, except that it is not a constant pointer. Thus it can he made to point at a larger block of memory. I-Im..vcver, because memory has been dynamically allocated, at some point it must be freed with de 1 ete [] :

delete [ 1 arr2;

Otherwise, a memory leak would result, and the leak could be significant, if the array is large.

Built-in C-styk strings are implemented as an array of characters. To avoid having to pass the length of the string, the special null~terminator I \0 I is used as a character that signals the logical end of the string. Strings arc copied by strcpy, compared with strcmp, and their length can he determined by strlen. Individual characters can be accessed by the array indexing operator. These strings have all the problems associated with arrays, including difficult memory management, compounded by the fact that when strings are copied, it is assumed that the target array is large enough to hold the result. When it is not, difficult debugging ensues, often because room has not been left for the null terminator.

The standard vector class and string class are implemented by hiding the behavior of the huilt-in C-sty1e array and string. Chapter 3 discusses the vector class implementation. It is almost always bener to usc the vector and stri ng class, but you may be forced to usc the C-styk when interacting with library routines that arc designed to work with both C

27



1:6 Lnapter I mUOQualOn

class IntCell 2

3 public: 4 explicit IntCell{ iot initialValue cO);

5 6 IntCell ( const IntCel1 & rhs );

7 -lntCell( );

8 const IntCell & operator=( const lotCell & rhs ); 9

10 int read( ) const; 11 void write( int x );

12 private: 13 int *storedValue;

14 };

15 16 IntCell::IntCell( int initialValue)

17 ( 18 storedValue '" new int( initialValue );

19

20 21 IntCell:: IntCell ( const IntCell & rhs )

22 (

2.3 storedValue '" new int( *rhs.storedValue );

24 25 26 IntCell ::-lntCell (

27 (

28 delete storedValue; 29 30 31 const IntCel1 & IntCell: :operator={ const IntCell & rhs )

32 33 if( this !" &rhs ) 34 *storedValue = *rhs.storedValue;

35 return *thi S;

36 37 38 int IntCell::read( ) const

39 40 return *storedValue; 41

42 43 void Inteell: :write( int x )

44 45 *storedValue'" X;

46

Figure 1.16 Data member is a pointer; hig three needs to be written



1.6 Templates

and C++. It also is occasionally necessary (hul Ihis is rare) to usc the C-stylc ill a section of code that must he optimizrd for speed.

1.6 Templates Consider the problem of finding the largest item ill an array of ilons. A simple algorithm is the sequential scan, in which we examine each item in order, keeping track of the maximum.

As is typical of mallY algorithms, the sequential scan algorithm is type independent. By type independellt, we mean that the logic of this <I\gorithrn does 1101 depend on the type of items that arc stored in the array. The same logic works for an array uf integers, floating-point THunhcrs, or any type for \vhich comparison can be meaningfully defined.

Throughout this text, \VC wi\1 describe algorithms and data structures that are type independent. vVhcn we write C++ code for a type-independent algorithm or data structure,

we \vould prefer to write the code once, ralher than recode it for each differcnt type In this section, we \\'ill descrihe how type-independent algorithms (also known as

generic algorithms) arc written in C++ Llsing the template. \Ve begin by discussing function templates. Then we examine class templates.

1.6.1 Function Templates FUllction templates arc gcnerally vcry easy to \\lritc. A function template is not an aClUa] function, but instead is a pattnn for what could hecome a function. HgUIT J. J 7 illustrates a function template fi ndMax that is virtually identical [0 the routine for stri ng shown in Figure 1.12. Thc line containing the temp1 ate declaration indicates [hal Comparable is the templatc

argument: it can be replaced hy any type to generate a fUllction. for instance, if a GIll to

findMax is made with a vector<string> as parameter, then a function will he g,enenned hy replacing Comparable with string.

Figure 1.18 illustrates that function templates arc expanded automatically as needed. It should he noted that an exp;msion for each nnv type generates additional code; this is known as code bloat, when it occurs in large projects. Note also that the ('illl findMax(v4)

will result in a compile-limE error. This is because when Comparable is feplaced hy IntCel 1, line 12 in Figure 1.l71JCcomes illegal: there is no < fUflClion defined for IntCell. Thus it is customary to include, prior 10 allY tcmplate, comlllents that explain what assumptions arc made about the template argumcnt(s). This includes assumptions about what kinds of constructors are required.

Because template argumcnts can assume any class type, when deciding on paramctfrpassing and return-passing conventions, it should he assumed that template arguments arc not primitive types. That is why we have returned by constant reference.

Not surprisingly, there an: mom)' arcane rules that deal \vith function templates. 1\:10st of

the problems occur when the template canl101 providc an exact match for the paramcters, hut can come close (through impliCit-type conversions). There must be ways to resolve ambiguities and the rules arc quite complex. Note that if there is a nonternplatc and a template, and both match, lhcnlhc nontcmplatc gl'ts priority. Also note that if there arc two

29



LnapIer I muoaualon

/** 2 * Return the maximum item in array a. 3 * Assumes a.size( ) > O.

4 * Comparable objects must provide operatar< and operator'"

j */ () template <typename Comparable> 7 canst Comparable & findMax( canst vector<Comparable> & a ) 8 { 9 i nt maxlndex '" 0;

10

11 fore lnt i '" 1; i < a.size( ); i++

12 if( a[ maxindex 1 < a[ i 1 /3 maxIndex "" i; 14 return a[ maxlndex ]; J5

Figure 1.17 findMax function template

2

J

lnt maine )

vector<int> vI ( 37 );

4 vector<doub 1 e> v2 ( 40 );

5 vector<string> v3( 80 );

6 vector<IntCell> v4( 75 );

7 8 II Additional code to fill in the vectors not shown 9

10 cout « find~lax( vI « endl; // OK: Comparable = int 11 cout « findMax( v2 « endl; // OK: Comparable'" double /2 cout « findMax( v3 « endl; // OK: Comparable = string 13 cout « findMax( v4 « endl; II Illegal; operator< undefined 14 15 return 0;

16

Figure 1.18 Using fi ndMax function template

equally close approximate matches, then the code is illegal and the compiler will declare an ambiguity

1.6.2 Class Templates In the simplest version, a class template works much like a [unction template. Figure 1.19 shows the MemoryCell template. MemoryCell is like the IntCell class, but works [or any type



I /** 2 * A class for simulating a memory cell.

3 */ 4 template <typename Object> 5 class MemoryCell 6 I 7 public:

8 explicit MemoryCell( canst Object & initialValue Object(» 9 : storedValue( initialValue ) { }

10 const Object & read( } canst 11 { return storedValue; }

12 void write( const Object & x

13 { storedValue "" x; }

14 private:

15 Object storedValue; 16 };

Figure 1.19 MemoryCe 11 class template without separation

1 int maine ) 2 I 3 MemoryCe 11 <; nt> ml;

4 MemoryCel1<string> m2;( "hello" ); 5 6 ml.write( 37 ); 7 m2.write( m2.read( ) + "world" }; R cout «ml.read( «endl« m2.read( ) « endl;

9

10 return 0:

11

Figure 1.20 Program that uses MemoryCe 11 class template

1.6 Templates

Object, provided that Object has a zero-parameter constructor, a copy constlUctor, and a copy-assignment operator.

Notice that Obj ect is passed by constant reference. Also, notice that the default parameter for the constructor is not 0, because 0 might not be a valid Object. Instead, the default parameter is the result of constructing an Object with its zero-parameter constructor.

Figure 1.20 shows how the MemoryCell can be used to store objects of both primitive and class types. Notice that MemoryCe 11 is not a class; it is only a class template. MemoryCe 11 <; nt>

and MemoryCell<string> are the actual classes. If we implement class templates as a single unit, then there is very little syntax baggage.

Many class templates are, in fact, implement.ed this way because, currently, separate compilation of templates does not work well on many platforms. Therefore, in many cases, I he

31



\..llQfJlt:1 I IIIl1UUUl..lIUl1

entire class, with its implementation, must be placed ill <I .h file. Popular implementations o/" the STL follow this strategy.

An alternal i\'(', discussed in Appendix A, is 10 separate t he interface and irnplcmel1tcHion

of the class templates. This adds extra syntax and haggage and historically has been difficult for compilers to handle cleanly To avoid the extra syntax thmughout this texl, when needed we provide, in the online cock, class tcmplmcs with no separation of interface and implementation. In the figures, the interrace is shown as if separate compilation was used, but the member function implementations arc shown as if separale compilation was avoided. This allows us to avoid focusing on syntax.

1.6.3 Obj ect, Comparab 1 e, and an Example In this text, we repeatedly usc Object and Comparable as generic types. Object is assumed to have a zero~pararnctcr constructor, an operator"', and a copy constructor. Comparable, as suggested in the fi ndMax cxmnplc, has additional functionality in the form of operator< that can be used to provide a total mete!".-(

figure 1.21 shows an example of J class type thai implements the functionality required or Comparable, and illustrates operator overloading. Operator overloading allows us to define the meaning of a huilt~in operator. The Employee class contains a name and a salary, and defines operator< on the basis of salary. ;\ more complicated operator< is possible; for instance, we could break a tie in salary hy using the name data member. The Employee class also provides a zcro~paramelcr constructor, operator"", and copy constructor (all by clebuh). Thus it has enough to be used '-15 a Comparab 1 e in fi ndMax.

To have practical utility, either its data members musl be public, or we must provide additional accessors and mutators. hgurc 1.21 shows a setVal ue member function and also illustrates the widely used idiom for providing an output function for a new class IYPL. The idiom is to provide a pub 1 i c rnernber function, named pri nt, that takes an ostream as a parameter. That publ i c member function can then be called by a global, nonclass function, operator«, that accepts an ostream and an object to oLltPLlt.8

7 :,nme of the data strUC[1I1TS in Chapter 12 usc operator~'" in addition to operator<. NOll' thal [or the purpose

of proViding a Lotal order, a=oob if hoth a<b and b<a arc false; thus the lise of operator== is simply for

conveniL'llCC".

N An alternative \0 this idiom is to have operator« directly implement the logic jll pri nt. Ikcallse operator«

is not a class member, it would !Iced to bc madc a friend function of the Employee cluss, requiring the

llltrociuCl ion of even more C++ syntax_ This alternative has thc additional disw!vantagc of 110\ working on

older compilers that do llot correctly mix fri end declarations with global template functions. It also has the

disK!vamage Ill' not working correctly in more complex cOlltexls involving inheritance, which <Ire beyond

the scope of this lexL



I.b lemplates

class Employee

2 (

.J pub] ie:

4 void setValue( const string & n. double s )

5 { name::: n; salary" s;

6 7 const string & getName( ) const

8 return name;

9 void print( ostream & out ) const

10 { out « name « II (" « salary « ")"; }

11 bool operator< ( const Employee & rhs ) canst

12 { return salary < rhs.salary; }

1.l

14 I lather general accessors and mutators. not shown

15 private:

16 string name;

17 double salary;

18 };

19

20 II Define an output operator for Employee

21 ostream & operator« ( ostream & out, const Employee & rhs )

22 (

23 rhs.print( out);

24 return out;

25 26 27 int maine

28 29 vector<Emp 1 oyee> v ( 3 );

30 31 v[OJ.seIVa]ue( "George Bush", 400000.00);

32 v[I].setValue( "Bill Gates", 2000000000.00);

33 v[2].setVa]ue( "Or. Phil", 13000000.00);

34 cout « findMax( v ) « endl;

35 36 return 0;

37

Figure 1.21 Comparable can be a class type, such as Employee



\...'IUPlU' "'UYUU"'''Y''

1.6.4 Function Objects In Section 1J).1, we showcd how function tC'mplates c<lnl)(' used to write gellcric algorithms.

As an example, tbe fUllction template ill Figure 1.17 call he lIsed to find the maximum item

ill an array. However, the template has;:1n important limitation: 11 works onl).' for objects that have

an operator< fUllction defined, and it uses that operator< as the hasis for all comparison

decisions. III many situatiolls, this approach is not feasible. For instance, it is a stretch to presume Lhat an Employee class, which is likely to have Jaw members such as name, salary, age, and years of scniority, will have an operator<, and evcn if it docs, as in the case of the Employee class in Figure 1.17, the operator< function that it has might not be the one we want. Perhaps instead of the maximum salary, we walll the maximum seniority. As a second

example, if we wLlnled !o find the maximum string (alphabet ieally last) in an array of strings, the default operator< docs nol ignore case distinctions, so '?r:I3RA' would he considered to precede "alligator" alphahetically, wbich is prohably not what we wan!. 1\ third example would occur if we had an array of pointcrs (0 ohjects (which \vOldd be common in advanced C++ programs that make usc ofa reature known as inheritance, which we do not make much

usc of in this text). The solution, in these cases, is to rewrite findMax to accept as parameters all array 01

objects and a comparison function that explains how to decide which of [\,,/0 objects is the larger and which is the smaller. In e["feCl, the array objects no longer know 110\"/ 10 compare themselves; instead, this information is complctdy clccoupled frolll the l)hjects in the array

An ingenious way to pass fUllctiolls as parameters is to notice that an object contains hOlh data and memher fUllctions, so we can define a class wilh no data and one member

fUIlCl ion, and pass an instance of the class, In effect, a rUIlCI ion is hdng passed hy placing it inside an ol.~jCCI. This ohject is coml1wJlly known as a function ol~jcct.

rigure 1 .11 shows the simplest im plcmentat ion of the function object idea_ f i ndMax takes a second parameter, which is a generic type. In order for the findMax template to expand without error, the generic type must have a member fUllction named isLessThan, which takes two parameters of the firs\. generic type (Object) and returns a bool. Otherwise, an

error williX' gellerated at line 9, when the template expansiotJ is aLtcmptcd hy the compiler. At line 2h, we can scc thai fi ndMax is called hy passing an array of string ,mc] an object that cOlltains all isLessThan method wilh two sLrings as parameters.

C++ function ohjects arc implcmcnlcd using this basic idea, but with some Lmcy

syntax. First, instead of using a function with a namc, we u;:,c operator overloading. 1 nSlcad

of Ihe fUlletion heing isLessThan, it is operator(). Second, when invoking operator(),

cmp.operatorO (x,y) can he shortened to cmp(x,y) (in other words, it looks like a function catl, and conscquenLly operatorO is known tiS the function call operator). As a result, the name of the parameter can be changed to the more meaningful isLessThan, ancllhe

call is isLessThan(x,y). Third, we can provide a version of find~lax that works without a function object. The implementation lIses the SloJnc.lard Li!wary function object template

less (defined in header filcIunclional) 10 generate a function object Ihat imposes Ihe normal c.lcbult ordering. Figure! .23 shows the implementation using the more typical, somewhat

cryptic, C++ idioms, In Chapler 4, we will givc an example of a class that needs to order the items it stores.

\A/e will write 1110S\ ofthc codc using Comparable, ,md show the adjustments needed to usc



II Generic findMax, with d function object. Version #1.

2 II Precondition: a.size( ) > O . .3 template <typename Object, typename Comparator> 4 canst Object & findMax( canst vector<Object> & arr, Comparator cmp ) 5 ( 6 int maxIndex '" 0;

7 8 fore int i '" 1; i < arr.size( ); i++ )

9 if( cmp.;sLessThan( arr[ maxIndex J. arr[ i ] ) )

/0 maxlndex '" i; ] ]

12 return arr[ maxIndex ];

13 ]4

15 class CaselnsensitiveCompare 16

17 public:

18 bool isLessThan( canst string & lhs. canst string & rhs ) canst 19 ( return stricmp( lhs.c_str( ), rhs.c_str( ) ) < 0; ) 20 );

21

22 int maine ) 21 24 vector<stri ng> arr( 3 );

1.6 Templates

25 arr[ 0 ] = "ZEBRA"; arr[ 1 ] '" "alligator"; arr( 2 ] '" "crocodile";

20 cout « findMax( arr, CaselnsensitiveCompare( ) ) « endl; 27

28 return 0;

29

Figure 1.22 Simplest idea of using a function object as a second parameter 10 fi ndMax; output is ZEBRA

the function objects. Elsewhere in the book, \ve will avoid the detail of rUllCI ion objects to

keep the code as simple as possihle, kmwving that it is not difficult 10 add function ohjects laler.

1,6.5 Separate Compilation of Class Templates Like regular classes, class templates call be implemented either entirely in their declarations, or we can separate the interface from the implementation. However, compiler support for separate compilation of templates historically has heen weak ~md platform specific Thus, in many cases, the entire class template with its implementation is placed in a single header file. Popular implementations of the Standard Jjbrary follow this strategy to implement class templates.

35



Lnapler I ImroaUUlun

1 II Generic findt'lax, \'lith a function object, C++ style.

2 II Precondition: a.size( ) > O. 3 template <typename Object, typename Comparator>

4 const Object & findMax( const vector<Object> & arr, Comparator isLessThan )

5 ( 6 int maxlndex = 0;

7

8 fore int i 0: 1; i < arr.size( ); i++ )

9 if( isLessThan( arr[ maxIndex L arr[ i ] ) )

10 maxIndex = i;

11

12 return arr[ maxIndex ];

I.l

14

15 II Generic findMax, using default ordering.

16 #i nc 1 ude <functi ona 1 >

17 template <typename Object>

12 const Object & find~lax( canst vector<Object> & arr )

19 (

20 return findMax( arr, less<Object>( ) );

21

22 2J class CaseInsensitiveCompare

2'1 25 public:

26 bool operator( )( const string & lhs, canst string & rhs ) canst

27 { return stricmp( lhs.c_str( ), rhs.c_str( ) ) < 0; }

2R }; 29 30 int maine )

.11 32 vector<string> arr( 3 ); 33 arr[ 0 ] = "ZEBRA"; arr[ 1 ] = "alligator"; arr[ 2 ] '" "crocodile";

,"3'1 cout « find~lax( arr, CaseInsensitiveCompare( ) ) « endl;

35 cout « find~lax( arr ) « endl;

36 37 return 0;

32

Figure 1.23 Using a function ohject C++ style, with a second version of fi ndMax; output is 7EBRA then crocodile



1.7 Using Matrices

Appendix A describes the mechanics involved in the separate compilation of templates. The declaration of the intcrf~1Cc for a template is exactly what you would expcCl: the member fUllctions end with a single semicolon, ill stead of providing an implementation_ But as shown in Appendix A, the irnplcmclUation of the member fUllctions can introduce complicated looking syntax, especially for complicated functions like operatop'. Worse, when compiling, the compiler will often complain about missing functions, anel avoiding this problem requires platform-specific solutions.

Consequently, in the online code that accompanies the text, we implement all class templates entjrely in its declaration, in a single header file. Vie do so because it seems to he the only way to avoid compilation problems across platforms. In the text, when illustrating the code, we provide the class interface as if separate compilation was in order since that is easily presentable, but implementations arc shown (\S in the online coeie. In a platformspecific manner, one em mechanically transform our Single-header file implemelltations into separate compilation implementations ir desired. See Appendix A for some of the different scenarios that might apply

1.7 Using Matrices Several algorithms in Chapter J 0 use two-dimensional arrays, which arc popularly known as matrices. The C++ library does not provide a matrix class. However, a reasonable matrix

class can quickly be writLCIl. The basic idea is to usc a vcctor of vectors. Doing this requires additional knowledge of operator overloading. For the matrix, we define operator[J, namely, the array-indexing opnaloL The matrix class is given in Figure 1.24.

1.7.1 The Data Members, Constructor, and Basic Accessors

The matrix is represented hy an array data member that is dcclJred to be a vector of vector<Object>. The constructor first constructs array, as having ro\'/s entries each of type vector<Object> that is constructed with the zero-parameter constructor. Thus we have rOl'l5

zero-length vectors of Object.

The boclyof the constructor is then entered and each row is resized to have col 5 columns. Thus the constructor terminates with what appears to he a two-clirnensional array. The numro\,/s and numcols accessors arc then easily implemented, as shown.

1.7.2 operator[] The idea of operator[] is that if we have a matrix m, then m[i] should return a vector corresponding to row i of matrix m. If this is Jone, then m[ i] [j] will give the entry in position j for vector m [i] , using the normal vector indexing operator. Thus [he matri x operator [] is to return not an Object, but instead a vector<Object>.

vVe now know that operator[] should return an entity of type vector<Object>. Should we usc return by value, by reference, or by constant reference? Immediately we eliminate return by value, hecause the returned entity is large, hut guaranteed to exist after the call.

31



.... Lnapler I ImrUUUUlUlr

lifndef f4ATRIX H 2 Idefine MATRIX H 3 4 #include <vector> .5 using namespace std; 6

7 template <type name Object> 8 class matrix 9

JO public:

11 matrix( int rows, int co1s ) : array( rows)

12 I Ll f4 15

16

fore lnt i '" 0; ; < rOI-/S; i++ )

array[ i ].resize( co1s );

17 const vector<Object> & operator[]{ int row) const 18 { return array[ row]; } 19 vector<Object> & operator[] ( int rOI'/ )

20 { return array[ rol'l ]; } 21

22

23 2'1 25

; nt numrOI'l$ ( ) const

return array.size( ); } int numcols( ) const

return numrows( ) ? array[ 0 ] .size(

26 prj vate:

27 vector< vector<Object> > array; 28 };

29 30 #endif

Figure 1.24 A complete matr; x class

0; }

Thus we arc down 10 return by reference or by constant reference. Consider the following method (ignore the possihility of aliasing or incompalible sizes, neither of which affects the algorithm),

void copy( const matrix<int> & from. matrix<;nt> & to

I for( int i "" 0; ; < to.numro\<ls( ); i++ )

toe i 1 = from( i ];

In the copy function, we attempt 10 copy each row in matrix from into the corresponding row in matrix to. Clearly, if operator[] returns a constant reference, then tori) cannot appear on the left side of the assignment statement. Thus it appears that operator[] should



Exercises

return a reference. Howcver, if we did that, thell an expression such (.l::; from[i] "'to[i] would compile, sillce from[i] would not be J constant \'ector, cven though from was a constant matrix. That cannot be allowed in <l good design.

So what we feally need is for operator[] to return a constant reference for from, but a plain reference for to. In other words, we need two versions of operator[]) which differ only in their return types. That is not allowed, f-lowever, there is a loopholr: Since member function const -Iless (thaL is, whether a function is an accessor or a mutator) is pan of the signature, we em haw' the accessor version of operator[] return a constant reference, and have the lmHator version return the simple reference. Then, all is well. This is shown ill figure 1.24.

1.7.3 Destructor, Copy Assignment, Copy Constructor These arc all taken care of automatically, because the vector has taken care of it. Thl\~ this is all the cude needed for {\ fully functioning matrix chss,

Summary

This chapter sets I he stage for the rest of I he book. The lime taken hy all algoril hm confronted with large amounts of input will be ,.til imporL,mt criterion for deciding if it is a good

algorithm. (Of course, correctness is most important.) Speed b relative. \Vhat is fast for onl' problem on olle machine might be slow for another problem or a different machine. \,Vc wil! begin [0 address these issues in the next chapter and will usc the mathematics discussed here to cSlahlish a formal model.

Exercises

1.1 \Vrite a progralllio solve the selection problem. LeL I? = N /2. Draw a table sbowing the running lime of your program for \,<-lrious values of N.

1.2 \Vrite a program to solve the word puzzle problcnL

1.3 vVritc a function Lo output an arbitrary double llumber (which might 1)(-' negative) using only printDigit for 110,

].4 C++ allows statements of the form

#include filellame

which !'cadsfilcl1mm-: and inserts its contents ill place of the include statement. Include statements may be nested; in other words, the file filename may itself contain an include statement, but, obviously, a file can't include itself in allY chain. \Vritl' a program I hat reads in a file and outputs the file as mod ilicd by t he include 5tat~nlL'!lt5.

1.5 \\ldte a recursive function that returns Ihe Ilumbcruf 1 's inlhc binary rcprescnr<lLion of N. Usc the fact that this is equal to the !lumber ur 1 's i11lhc representation ofN/2, plus l,ifNisodd.

39



4U Lnapter I InlrooUGlOn

1.6 \-Vrile the routines with the following declarations:

void permute( canst string & str ); void permute( canst string & str. int low. int high );

1.7 Prove the following formulas: a. log X < X for all X > 0 b. log(AB) = B log A

1.8 Evaluate the following sums: . ,,00 1 J. L......i=04i

h. L~o~ * c. L:o~

** d. L:o~ 1.9 Estimate

• 1.10 What is 2100 (mod 5)'

1.11 Let Fj be the Fibonacci numbers as defined in Section 1.2. Prove the following: "N~21' F 2 J. Li=l 'j = -<N-

b. FN <lpN. with 1> = (1 + ,/5)/2 ** c. Give a precise closed-form expression for FN .

1.12 Prove the following formulas:

H. L;:1 (2i - 1) = N2

b. L;:1 i 3 = (L;: 1 i)' 1.13 Design a class template, Collection, that stores a collection of Objects (in an array),

along with the current size of the colleclion. Provide public functions isEmpty,

makeEmpty, insert, remove, and contains. contains(x) returns true if and only if an Object that is equal to x is present in the collection.

1.14 Design a class template, Ordered Co 11 ecti on, that stores a collection of Comparab 1 es (in an array), along with the current size of the collection. Provide publ i c functions i sEmpty, makeEmpty, insert, remove, findMi n, and fi ndMax. findMi 0 and fi ndMax return references to the smallest and largest, respectively, Comparable in the collection. Explain what can be done if these operations are performed on an empt y collection.

1.15 Using the findMax routines in figure 1.23, write a maio that finds the "maximum" Emp 1 oyee on the basis of name. The employee should be the one whose name is alphabetically last, ignoring case distinctions.

1.16 For the matr; x class, acId a res; ze member function and zero-parameter constructor.



References

References

There are mall)' good textbooks covering the mathematics reviewed in this chaptcL A small

subset is Ill. 121, Ill. I'll, 1141, ami 1161. Reference lui is specifically geared toward the analysis of algorithms. It is the first volume of a three-volume series that will be cited

throu?,hout this text. More advanced material is covered in [61.

Throughout this book we will assume a knowledge of C++. ror the most pari, 115] describes the final draft standard of C++, and, being written by the original designer of C++,

rnnains the most authoritative. Another standard reference is 110j. Advanced topics in C++ arc discussed in ['51. The two-pan series [.II, 121 gives a great discussion of the many pitfalls

in C++. The Standard TCinplate Library, which we will investigatc throughout this text, is described in 113]. The material in Sect iOlls 1.4,-].7 is mean I to serve as an overview of! he

features that we will usc in this text. \;Ve ;llso assume LlmiliarilY with pointers ,mel recursion

(the recursion summary ill this chapter is meant 10 he a qll.kk review). V"';c will attempt to

provide hints on their usc where appropriate throughout the textbook. Readers not familiar

with these should consult [17] or any good intermediate prognllllming textbook.

General programming style is discussed in ~everal books. Some of the classics arc [41,

[71, and [81

1. Iv1. o. Albertson and .J. P Hutchinsoll, I )iscrctc Mmilc/nalics wich Alp;orithms, John \;Vil('y 6::1 Sons, New York, 1988.

2. z. Havel, Malh Companion/or Compuler SdCllCC, Reston Publishing Co., Reston, Va., 1()82.

1. R. A. Bruakli, Inlmdw_lm}' Com/Jil1a{orics, North-Ilolland, New York, 1977.

4. ! . \V Dijkstra, II Discipline (![ Progrul11/Hing, Prentice Ilall, Englewood Cliffs, Nj., 1976.

'1. H. Eckel, TiIin/?ing, in C++, Prentice Hal!, Ellglewood Cliffs, Nj., 2nd eel. 2002.

6. R. 1.. Crailam, D. E. Knuth, and o. Pat<ls\mik, Concrele i"v/(!t/!emalics, Adclison-\;Vesley, Reading, tvlass., 191)9.

7. D. C;ries, The Science oj Programming, Springer-Verlag, New York, 1981.

8. B. \;V Kernighan and P j. Piauger, The Elemellts of Programming Style, 2d cd., j'v1c(;ra\v-Hill, New York, ] 978.

9. D. E. Knuth, The Art (d Computer Pro,t~ramlJlil!,'!" \lol. l: FlIl1da/nclllal Algorillll1lS, '3d cd., Addison-\\Jcs!cy, Reading, Mnss., 1997.

10. S. B. Lippman andJ. L\i01C, C++ Primc!",:3d ell., l\ddison-\Vl'sky, Reading, ivl"ss., ]998.

II. S. i"vlcycrs, .50 Sp('(Uic \·\l(IYs to Improve HJllr Program5 find Designs, 2cl cd., j\ddison-vVeslcy, Reading, Mass., 1998.

12. S. Meyers, AI/ore l1{eclivc C++: .35 New Ways to Improve )'IJUr Prow·ams and Dcsip,ns, Addison\Vcs]ey, Reading, Mass., 1996.

13. D. R. tvlusser and A. Saini, SIL ']iiloriu! (Jlld l?r:ferel1cc Guide: C++ Pm,t;rmllming with the Standard Temp/ate I..ilmuy, J\ddison-\;Vcsky, Reading, i'vlass., 1996 .

.14. E S. Roberts, Applied ComN,wlorics, Prelltice llall, J;.nglcwood (:liITs, N.J., 1984.

! '5. H. Stroustrop, The C++ Progra/J!l1lillg L(/I1,lS!WS:C, 3d cd., Adclisoll-\Veslcy, Reading, Mass.,

1997.

16. i\. Tucker, Applied Combina/oric.'>, 2d cd., John \,Vilcy & Sons, New York, 1984.

17. tv1. A. ·Weiss, }\!J!.0rilhms, Data Structures, and Problcm So/vil1p, ll'ilil C++, 2nd cd , Addisotl\Vcslcy, Reading, Mass., 2000.

41





APTER 2

Algorithm Analysis

An algorithm is a clearly specified set of simple instructions to he followed to solve a problem. Once an algorithm is given for a prohlem and decided (somehow) to be correct, an imporlant step is to determine how much in the way of resources, such as time or space, the algorithm will require. An algorithm that solves a problem hut requires a year is hardly of any use. Likewise, an algorithm that requires several gigabytes of main memory is not (currently) useful on most machines.

In this chapter, we shall discuss

• How to estimate the time required for a program.

How to reduce the funning time of a program from days or years to fractions of a second .

• The results of careless use of recursion.

Very efficient algorithms to raise a number to a power and to compute the greatest common divisor of l wo numbers.

2.1 Mathematical Background The analysis required to estimate the resource use of an algorithm is generally a theoretical issue, and therefore a formal framework is required. We begin with some mathematical definitions.

Throughout the book we will use the following four definitions:

Definition 2.1. T(N) = O(J(N» if there are positive constants c and "0 such that T(N) :S cI(N) when N 2: no·

Definilion 2.2. T(N) = r/ (g(N» if there are positive constants c and "0 such that T(N) ": cg(N) when N~ no.

Definilion 2.3. T(N) = (0(!J(N» if and only if T(N) = O(!J(N» and T(N) = r/(!J(N».

Definilion 2.4. T(N) = o(p(N» if for all constants c there exists an "0 such that T(N) < cp(N) when N> "0. Less formally, T(N) = o(p(N)) if T(N) = OrreN)) and T(N) t ("l(p(N). 41



.... ,'up" •. ,.L '''bV ' "''''' ''''''''1-''"'

The idea of tlwse definitions is to establish a relative order among functions. Civcn two functions, there arc usually points where one function is smaller thall the other functioll, so it does not make sense to claim, for instancc,J(N) < (~(N). Thus, we compare their relative rates of growth. vVhcn we apply this to the analysis of algOlithms, \",e shall see why this is

the important measure. Although l ,OOON is larger than N L for small values of N, N L grmvs at a faster fmc, and

thus N L will eventually be the larger fUllction. Thc lUrning point is N = LOOO in this case. The first definition says that eventually there is SOUle poinll1,) past which c -feN) is always at least as Jarge as T(N), so that if constml1 factors are ignored, .f(N) is at least .'L<; hig as T(N). In our case, we have T(N) = I ,OOON,f(N) = N 2, "0 = 1,000, and c = I. We could also usc 110 = 10 and c = 100. Thus, we can say that 1,OOON = O(NL) (order N-squared). This Ilotation is knO\vn as Big-Oh notation. Frequently, instead of saying "order ," one says '"Big-Oh .

If we use the tradilion,!.l inequality operators to compare growth rates, then the first definition says that the growth rate of T(N) is less than or equal to (.:::::) that offeN). The second definition, T(N) = Q(g(N») (pronounced "omega"), says that the growth ratl' or T(N) is greatCl" than or equal to (e» that of g(N). The third defInition, T(N) = (H)(h(N»

(pronounced "theta"), says that the growth rate or T(N) eyuals (=) the growth rate of II(N). The last cldinition, T(N) = o(p(N») (pronounccd "liulc-oh"), says that the growt.h rate 01 T(N) is less than «) thc growth rate of peN). This is differcllt from Big-Oh, hecause Big-Oh allows the possibility that the growth rates arc the same.

To prove 1hat. sOllie function T(N) = O(J(N»), we usually do not apply these definitions formally hut instead use a repertoire of knO\vn results. In general, this means thm a proof (or determination that the assumplion is incorrect) is a vcry simpk calculation and should not involve calculus, except in extraordinary circumsl<mCfS (not likely to occur in an algorithm analysis).

\Vhcn wc say that T(N) = O(f(N), Wf" arc guaranteeing thaL the fUIlCtion T(N) grows at a rate no faster thanI(N); Ihusf(N) is an upper bound on T(N). Since this implies Lha! f(N) = Q(T(N», we say that J(N) is a lower bound on[(N).

As an example, NJ grows faster than N2, so \ve can say that N1 = O(N:·") or NJ = Q(N'). f(N) = N2 and g(N) = 2N2 grow at the same rate, so both feN) = ()(,~(N» and f(N) = r2(g(N)) are true. \Vhcl1 two functions grow at the same rate, then the decision of whether or nol to Signify this with ("')0 can depend on the particular COIl!CXl. Intuitively,

if g(N) = 2N 2, then g(N) = O(N"), g(N) = CJ(N'), and g(N) = O(N') arc alltechnicaliy

correct, but the last option is the best answer. Writing (~(N) = ("')(N 1) says not only that g(N) = CJ(N2), hut also thm the result is as good (light) as possible.

The important things t.o kno\-\' arc

Rule I,

Wlt(N) = O([(N» and T2(N) = O(g(N», then (a) T,(N) + T2 (N) = O(J(N) + g(N» (Intuitively and less formally it is

max(O(J(N», ()(g(N»»,

(b) l,(N) * T,(N) = O(J(N) * g(N».

Rule 2, II T(N) is a polynolllial of degree k, then T(N) = H(N").



L.I NlamemaUCal tlacKground

Function Na.mc

c C(Hlst,Ult

logN l_ogarithm ic

Jog2 N IJ)g~squarcd

N /jnear

l\llogN

N2 Quadratic

N 1 Cubic 2N Exponential

Figure 2.1 Typical growth rates

Rule 3. lng/I N = O(N) for any constant I?~ This tells us that logarithms grow velY slowly

This information is sufficient to arrange most of lhe common functions hy growth rate (sec fig. 2.1).

Several points arc in order. First, it is very bad style to include constants or low-order terms inside a Big-Oh. Do not say TeN) = O(2N2) or T(N) = O(N' + N). In both cases, the corrcct form is T(N) = O(N2). This means that in any analysis that will require a Big

Oh answer, all sorts of shortcuts are possible. Lower-order terms can generally be ignored, and conslants can be tbrown away Considerably less precision is required in these cases.

Second, we can ahvays determine the relative growth rates of two functions J (N) and

g(N) by computing lim,\!_ "xl (N)/g(N), using DIOpital's rule if necessary.] The limit can have four possible values:

The limit is 0: This means that/eN) = o(g(N».

The limit is c '" 0 TI1lS means thmf(N) = 0)(g(N)).

The limit is co: TIllS means that g(N) = o(f(N».

The limit oscillates: There is no relation (this will nOI happen in our context).

Using this method almost always amounts to overkill. Usually [hc relation betweenf(N) ~ltld g(l\I) can be derived by simple algebra. For instance, if[(N) = N log Nand g(N) = N1.5, then to decide which ofJ(N) and g(N) grows bster, one really needs to delermine which

or log N and NO.""> grows faster. This is like determining 'which of log2 1\1 or N grows faster. This is a sirnplc problem, because. il is already known that N grows fasler than any power

of a log. Thus, g(N) grows faster than f (N).

lnh)pil<lJ:C; rule statf'S tbat if lim'\'_>,-"oJ(N) = CXJ and limN __ >-cx;,I-i(N) = CXJ, thell lim,\, Hx;}(N)h-:(N) =

lim,\, Hxj'(N)/g'(N), whcrcj'(N) nlldp,'(N) arC' the derivatives o[J(N) antIgeN), respectively.

45



\...ltd!.llel L J-\.I~UllUtllIl"\lIalpl)

One stylistic note: It is bad to sayI(N).:s O(~(N)), because the inequality is implied by the definition. It is wrong to write feN) 2:::. O(g(N», which docs no! make sense.

As an example of the typical kinds of analysis that arc performed, consider the prohlem or downloading a file over the Internet. Suppose there is an initial .1-sec delay (to set up a connection), after which the download proceeds at 1.5 K(bylcs)/sec. Then if the file is N kilohytes, the lime 10 download is described by the formula T(N) = N /1.5 + .1. This is a linear function. Notice that the time to download a 1,.'500K file (1003 sec) is approximately (but not exactly) twice the time to download a 750K file (503 sec). This is typical of a linear function. Notice also that if the speed of the connection doubles, both times decrease, but the 1, SOOK file still takes approximately twice the time to download as a 750K file. This is the typical characteristic of linear-time algorithms, and is why we write T(N) = O(N), ignoring constant factors. (Although using Big~Theta would be rnore precise, Dig-Oh answers arc typically given.) Observe also that this hehavior is not true of all algorithms. For the first selection algcnitnm desCIihcd in Section 1.1, the running time is controlled by the time it takes to perform a sort. For a simple sorting algorithm, such as the suggested bubhlesort, when the amount of input doubles, the running lime increases hy a factor of 4, for large amounts of input. This is because those algorithms afe not linear. Instead, as we will see when we discuss sorting, trivial sorting algorithms arc O(N1

), or quadratic.

2.2 Model In order to analyze algorithms in a formal framework, wc need a model of computation. Our model is basically a normal computer, in which instlllcLions arc executed sequcntiilily Our model has the standard repertoire of simple instructions, such as addition, multiplication, comparison, and assignment, but, unlike the case with real computers, it takes exactly one time unit to do anything (simple). To be reasonable, we will assume that, like a modern computer, our rnode! has fixed-size (say, :32-bit) integers and that there are no fancy operations, such .:IS matrix inversion or sorting, that clearly cannot be done in onc time unit. 'vVc also assume infinite memory.

This model clearly has some weaknesses. Obviously, in rcalhfe, Ilot all operations take exactly the same tin'te. In particular, in our model one disk read counts the same as an addition, even though the addition is typically several orders of magnitude faster. Also, by assllming infinite memory, we never worry about page faulting, which can be a real problem, espcclally for efficient algorithms.

2.3 What to Analyze The most. important resourcc to analyze is generally the running time. Several factors affect Ihe running time of a program. Some, such as the compiler and computer used, arc obviously hcyoncl the scope of any theoretlcal model, so, although they are important, we cannot deal with them here. The other main factors are 1he algorithm Hsed and the input to 1 he algorithm.



L..:J VVlIdL LO MlIdlYLe

·l),pically, the size of the input is the main consideration. \Ve dc/inc two runctions, TavgU'n and TW\)]",,[(N), ,1S the average and worSI-case running time, rl'~pcctivcJy, uscd by an

algorithm on input ofsizc N. Clearly, TJ\.g(N) ~ T\\.prsl(N). If there is more thall onc input, these fUilctions may have more than one argument.

Occasionally, tIl(' lXS1-case performance of an algorithm is (lIwlyzcd. Ilowevcr, Ihis is of len of liuk interest, because it docs not represent typical behavior. 1\\,crage-C<lsc performance oflcn rt'flccts typical behavior, while worst-case performance represents a

guarantee for performance on any possible input. Notice also that, although in this chapter we analyze C++ code, these bounds ,He really bounds for the algorithms, n.llher than programs. Programs are ,\ll implelllCIltCltioll of the algorithm in a particular programming language, and almost always 1he details of the programming language do nOl alTect a BigOh answer. If a program is running much more slowly than {he algorithm analysis sugges1s,

there llli.1.y be an implementation inefficiency. This call occur ill C++ \vhen arrays are inadvertently copied in their ent irel y, instead of passed with references. Another extremely subtle example of this is ill the lastlwo paragraphs of Section 12.7. Thus in future chapters, we will analyze the algorithms rather than the programs.

Generally, the quantity required is the worst-case time, unless otherwise specified. One reason for this is that it provides a hound for all illplll, including particularly bad input, which an average-case analysis docs not provide. The. other reason is that average-case bounds arc usually much more difTiclllt to compute. In some instances, the ddinition of ·'avcr<.lge" can affect the result. (for instance, what is average input for the following problem?)

1\s ail example, in the !lext section, we shall consider the following problem:

Maximum Subsequence Sum Problem.

Givcn (possibly negativr) integers 11 1,;\2) ... ,A,\" find t he maximum value of Z=;/OOC[ A" (For corwcnience) tbe maximum suhsequence sum is 0 if all the integers arc ncgati\T.)

Example: hH" input ~2, 11, ~4, ! 3, ~5, ~2, the answer is 20 (112. through At).

This problem is interesting mainly hecause there arc so mallY algorithms to solve it, and the performance of these algorithms varies dras1.ically. \Ve wHl discuss four algorithms to solve this prohlem. Thc funning time un some COmplllC1" (the exact computer is unimportant) for these algorithms is given ill Figure 2.2.

There are several important things worth noting in this table. h)J" ;\ small amount of input, the algorithms all run in a blink of the eye, so if only a small amount of input is expected, it might he silly to expend a great deal of elTon to design a clever algorithm. On the other hand, there is a large market these days for rcwriting programs that were written fivc years ago based on a llo-longer-valid assumption of small input size. These programs arc nO\v too slow, because they llsed poor algorithms. For large amounts of input, algorithm

4 is clearly the best choice (although algorithm':' is still usable). Second, the linKS given do not include the 1 imc required to read the input. f'"oralgorilbm

4, the time merely to read in the input from a disk is likely to be an order ofmagniludc larger

than the lime required to solve the problem. This is typical of many el1icient algorithms. Reading lhe uata is generally the bOllleneck; once the data arc read, the problem can be

.. , Bismillah hiir Rehman nir Raheem-----------------------------Assalat o Wasalam o Alika Ya RasoolALLAH


..... "ul-', ... , L '"b''' "'"'' '''''-''1.1'-'

~\.!_ W ~~-!_~l.l.~l .. Ti 111~' Input 1 2 1 4 Size CJ(N 3) O(N') O(N log N) CJ(N)

N= 10 0.000009 0.000004 0.000006 O.OOOO())

N= 100 0.002'580 0.000109 0.00004'5 0.000006

N= 1,000 2.28101) 0.010201 0.00048'5 O.OOOrll I N= IO,O()O NA 1.2129 0.005712 0.000317

N = 100,000 NA 135 O.OlA618 0.003206

Figure 2.2 Running times of severa! algorithms for maximum subsequence sum (in seconds)

'---'---~-'IT---'---,------,---,---,-r-, , , , , I I I I I I

I I

/-------

Linear O(N log N)

Quadratic Cubic

OL-~L-~~--~~I'~~L---L-__ ~ __ ~ __ C~ __ ~

10 20 30 40 50 60 70 gO 90 100 Input Size (N)

Figure 2.3 Plot (N vs. time) of various algorithms

solved quickly. ror inefficient algorithms this is not true, and significant computer resources must be used. Thus it is important that, whenever possible, algorithms be efficient enough not to be the bottleneck or a problem.

Figure 2.3 shows the growth rates of the running times of the four algorithms. Even though this graph encompasses only values of N mnging from 10 to 100, the relative growth rates arc still evident. Although the grJph for the O(N Jog N) seems lineur, it is easy to verify that it is not hy using a straight-edge (or piece of paper). rigure 2.4 shows the performance for brgcr values. [t dramatica.lly illustrates how useless ineHicicllt algorithms me for even moderately large amounts of input.



L."I- ~UllllIIlg lillie l.dllUldtlUfI~

2.4 Running Time Calculations There arc sevcral ways to estimate the running tlmc of a prognlm. The previous table was obtained empirically. If two programs me expected to take similar t iIlKS, probably the best

way 10 decide which is faster is to code them both up and run thcm! Generally, thuc arc several algorithmic ideas, and we would like to eliminate the. had

ones carly, so an analysis is usually required. furthermore, the ability to do an analysis

usually provides insighl into designing efficient algorithms. The analYSis also generally pinpoints the botllenecks, which arc worth coding carefully.

]() simplify the analysis, we will adopt the convention that there arc no particular units of timc. Thus, we throwaway leading constants. Vic will also throwaway low-order terms, so what we arc cssentially doing is computing a Big-Oil running time. Since Gig-Oh is an upper bound, we must he carcfulncvel" to ulH.krcslimate the running timl' of the program. In effect, the answer provided is a guarantee lila! the program will trnninatc within a certain

time period. The program may stop earlier than this, but never laler.

2.4.1 A Simple Example

Ilcn.' is a simple program fi-agment to calculate L~c 1 (5:

int sum( int n )

i nt parti a 1 Sum;

partialSum '" 0:

2 for ( i nt ; = 1; i <" n; i ++ )

J partialSullI += ; -;.: i -;.: i;

4 return partialSum;

·["he analysis 01 this fragment is simple. fhe declarations count for no time. J.ines ! and 4

count for one unit cacho Line:3 COUllts fnr four units per lime executed (two multiplications, one addition, alld one aSSignment) and is executed N limes, for a total of 4N units. Line '2 hdS the hidden COSIS of initializing i, testing i :s N, and incrcmenting i. The 10lal cost of all these is 1 to initialize, N + "J for all the lests, and N for ali the increments, which is 2N + 2. vVe ignore the costs of cal!ing the function and rrlurnillg, for a tOlal or 6N + 4. Thus, we

say that this function is O(N), Ifwc had to perform all this work every time we needeclto <malyzc a program, the task

would quickly becornc infeasible. fortunately, since we are giving the answer in terms of Big-Oh, there are lots of shortcuts that call be \;:lken without affecting thc final answer. For instance, line 3 is obviously an O( 1) statemcllt (per execution), so it is silly [0 count precisely

whether it is two, lhree, or four units; it cloes not maHer. Line 1 is ohviously insignificant compared with the for loop, so it is silly to waste time here. This leads to several general

rules.

"" Bismillah hiir Rehman nir Raheem-----------------------------Assalat o Wasalam o Alika Ya RasoolALLAH


\..IIQl-'ll;l;;' MI5UIIlIIIII MIIOly-:>t-:>

I I ~ I

Linear ~

O(N log N) , Quadratic

Cubic

J I I

I """", ~ J' """",

J(/' .

/////

o 10110 2000 3000 4000 SOOO 6000 7000 SOOO 9000 10000 Input Size (N)

Figure 2.4 Plot (N vs. lime) of various algorithms

2.4.2 General Rules Rule I-FOR loops. The running time of a for loop is at most the running time of the statements inside the for loop (including tests) times the number of iterations.

Rule 2-Nesled loops. Analyze these inside out. The total running lime of a statement inside a group of nested loops is the IUnning time of the statement rnultiplied by the product of the sizes of all the loops.

As an example, the following program fragment is O(N2):

for( i = 0; i < n; i++ )

for( j '" 0; j < n; j++

k++;

Rule 3-Consecutive Statements.

These just add (which means that the maximum is the one that counts; see rule 1 on page 44)

As an example, the following program fragment, which has O(N) work follmved by O(N2) work, is also O(N1):

for( i = 0; i < n; i++

a[;]"O;

fore i = 0; i < n; i++

for( j "" 0; j < n; j++

a[ ; ] +" a[ j ] + ; + j;



Rule 4-lfjElse. For I he fragment

if( condition

51

else

52

2.4 I{unmng lime lalculatlOns

the running time of an if lel se statement is never more than the running time of 1 he tesl plus the larger of the running times ofSJ and S2.

Clearly, this can be ,Ill ovcrestim<ue in some cases, but it is ncver an underestimate. Other rules (1.re obvious, but a basic strategy or analyzing from the inside (or deepest

pan) out works. If there arc functiun calls, these must he analyzed first. If there are recursive functions, there arc several options. If the recursion is really just a thinly veiled for loop, the analysis is usually triviaL For instance, the following funcl ion is really just a simple loop

and is O(N):

long factorial ( int n )

if(n<=l

return 1;

else return n * factorial ( n - 1 );

This example is really a poor usc of recursion. \Vherl recursion is properly used, it is difficult to COllven lhe recursion into a simple loop structure. In this case, the analysis will

involve a recurrence relation thaI needs to be solved. To see whM might happen, consider the fol!ov/ing program, which turns on! to be a horrible usc of recursion:

2

long fib( iot 0 )

{

if{n<=l

return 1;

else return fib( n - 1 ) + fib( n - 2 );

At flrst glance, this seems like a vc.ry clever use of recursion. However, if the program is

coded up and run I' Of values ofN around 40, it be-comes apparent thal this program is terribly inefficient. The analysis is fairly simple. Let T(N) be the running time for lhe function call fi b{n). If N = 0 or N = 1, then the running time is some constant value, which is the time to do the test at line I and return. \Ve can say thal T(O) = T(l) = I because constants do not matler. The running time for other vtllucs of N is then measured relative to the running time of the base casco For N > 2, the tilne to execute the function is the constant \\'ork at line J plus the work at line T Line :3 consists of an addition and Iwo function calls. Since the function calls arc no! ~irnpk operations, ! hey must be analyzed by themselves. The first function call is fib(n-l} and hcncc, by the definition of T, requires T(N - J) units of time. ;\ similar argument shows that! he sccc)Jld function ca!1 requilTs T(N - 2) units of time. The

5J



:u Lnapler L AlgUrltHIIl f\lIdly~l~

total time required is then T(N - 1) + T(N - 2) + 2, where the 2 accounts for the work

at line I plus the addition at line "S. Tilus, for N 2:: 2, we have the following formula I'm the running time of fib(n}:

/(N) = T(N - I) + I(N .. ~ 2) + 2

Since fib(n) = fib(n-l) + fib(n-2), it is casy tll show by induction that T(N).:::: fib(o). In Section 1.25, we showed thaLfih(N) < (S(3)'\1. A similar calculation shows (kit (for N > 4)

fibCN) :::: C')j2);\l, and so the running time of this program gn.l\VS cxponcllliuily. This is "bout as bad as possible. By keeping a simple anay <mel using a for loop, the running time can be

reduced suhstantially. This program is slow hecause there is a huge amount of redundant work being per

formed, violating the fourth lllajlH rule of recursion (the compound illterest rule), which

was prescnted in Section 1_3. Notice thM the first call on line :~, fib(n-l), actually cornpuLes

fib(n-2) at some point. This information is thrown away and recomputed hy the second

ca1J on linl' 1. The amoullt of informatioTl tbrown <1W,l)' compounds recursively and results

in the huge running time. This is perhaps the fillest cx,ll1lpk of the maxilll "Don't compute

anything more t km once" and should not scare yuu away rrom using recursion. ThrougllOlIl Ihis book, we shall sec outstanding uses of recursion.

2.4.3 Solutions for the Maximum Subsequence Sum Problem

\Ve wi!! now preselll four algorithms to sul\'l' the maximuiTl subsequence sum prohlem

posed earlier. The first algorit hm, which merely cxh,\l\st ivcly tries all possibilities, is depicted

in Figure 2.'1. The indices in the for loop reflee! the EKt that in C++, arrays begin at 0,

instead of 1. Also, the algorithm docs not compute the actual subsequences; <1dditiona!

code is required to do this.

COllvince yourself that this algurithrn works (this should nOl take much convincing).

The running tirne is O(N}) ancl is entirely due to lines 1_) and l"t) which consist or an O( 1)

statement buried inside three rlCs\cd for loops. The loop at line 8 j~ of size N_

The second loop has size N - i which could he smaH but could also be or size N. \Ve

must assume the worst, with the knuwledge thai this could make 1 he final houlld it hit high.

The third loop has size j -- i + .I, which, again, we lllust assume b of size N. The total is

O( I . N . N . N) = O(N') Line () takes only O( I) Inial, and lines 16 and 17 lake emly O(N/)

total, since they arc easy expressions inside only two loops.

It turns out that a more precise analysis, taking imo accollnt the (Ictua] size of these

loops, shows that the answer is (;')(j\lJ) and that nur estimate ahove was a factor of 6 too

high (which is all right, becausc constants do nut matter). This is generally true in these

kinds of prohlems. The precise analysis is obtained from the sum L;:~)l L_;~~! :L;i=i I,

which \ells how many times line 14 is executed. The sum can be ('valualed inside Ollt, using

formulas from Seclion 1.2.'). Tn particular, we will usc the formulas for the sum of the first

N integers and first j\I squares. hrst we have

I

1.:: I =j ... i + I 11=1



/'* 2 * Cubic maximum contiguous subsequence sum algorithm.

J *j 'I int mdxSubSuml( canst vector<;nt> & d )

i

() int maxSum '" o· I

H fore int i '" O· i < a.size( ); i++ )

o fore int j = i; j < a.size( ); j++

10 (

/1 int thisSum = O·

U

U

I" 15 1(,

17

IH

10

fore int k = i; k <= j; k++ )

thisSum +oO a[ k ];

H( thisSulll > maxSum )

max Sum '" thisSurn;

2(1 return maxSum;

Figure 2.5 Algorithm J

Next we evaluate

N---l L (j - i + I) = c-( 1\_' _-_'-c+_I-'-)(c--I\_' ---,,-0 2

1---'--'1

2.4 Kunmng lime Calculations

Thb sum is computed hy observing that i! is JUS! the sum of the flrst N - i integers. To complete the calculation, we evalml1c

,\'- ] N L (N- i -I I)(N - i) = L (N - i + l)(N --- i + 2)

2 i_c] 2

I ~l ( 1) ~ . 1 2, _ ~ =:; L.., ,- N +:; L..,' + :;(N +,N + 2) L.., 1 1' __ ·1 I, __ ~I 1=1

= 1 N(N + 1)(2N + I) 2 6

N I -t-lN 2 -1- 2N

()

N(N-J-l) N 2 -J-1N+2 --~~-+ N

2 2

'v\it: c<tn "yuid the cubic running time by removing a for loop_ This is not always possihle, but in this case there arc an awful 10\ of unnecessary cllmpuuuions present in the algorithm. The incflicicnc), thm thc irnprovcd algorithm corrects can be seen by noticing



UldPH:~r L I-\lgUfllIIlIIl-\lldly.'>l.'>

/** 2 * Quadratic maximum contiguous subsequence sum algorithm .

.1 */ 4 int maxSubSumZ( canst vector<int> & a ) j

(; int maxSum 0;

7 8 for( int i '" 0; i < a.size( ); i++ ) 9

int thisSum = 0; 10 11

12

for( lnt j -" i; j < a.size( ); j++) {

13 thi sSum += a [ j ];

14 15 if( thisSum > max5um 16 maxSum = thisSum; 17

18

19

20 return maxSum; 21

Figure 2.6 Algorithm 2

that :Lj,=i AI< = Aj + 2:';;1=~ AI<> so the computaiiOll at lines 1J and 14 in algorithm 1 is

unduly expensive. Figure 2.6 shmvs an improved algorithm. Algorithm 2 is clearly O(f\l 2 );

the analysis is even simpler than before. There is a recursive and relatively complicated O(N log N) solution to this problem,

which we now describe. If there didn't happen to he an O(f\I) (linear) solution, this would be an excelknt example of the power of recursion. The algorithm uses a "divide-and-conquer" strategy. The idea is to split the problem into two roughly equal subproblems, which are then solved recursively. This is the "divide" par!. The "conquer" stage consists of patching toget her the two solutions of the subproblems, and possibly doing a small amOUllt ofaddil ional work, to arrive at a solution for the whole problcrn.

In our case, the maximum subsequence sum call he in une of three places. Either it occurs entirely in the left half or the input, or entirely in the right half, or it crosses lhe rniddle and is in both halves. The first two cases can be solved recursively. The last case can be obtained by finding the largest sum in the {irs! haH that includes the last element ilt the first half, and the largest sum in the second half that includes the first clement in the second half. These two sums can then be added together. As an example, consider the following input:

First Iiall Second J Jail

4 -1 3 -2 --I 2 6 -2



2.4 Running Time Calculations

The maximum subsequence sum for the first half is 6 (dements;\] through A -) and for the second half is 8 (elements A6 through Aj).

The maximum sum in the first half that includes the last clement in the first haH is 4 (elements Al through A4 ), and the maximum sum in the second half that includes t he firs! clement in the second half is 7 (clements A'j through 117). Thus, the maximum sum that spans both halves and goes through the middle is 4 + 7 = 11 (ekments A J through At)·

We see, then, that among the three ways to form a large maximum subsequence, for our example, the best way is to include clements from hoth halves. Thus, the 3nS\Ver is I !. figure 2.7 shows an implementation of this strategy.

The code for algorithm 3 deserves some commcnt. The general form of the call for the recursive function is to pass the input array along with the left and right borders, which delimit the ponion of the array that is operated upon. A one-line driver progriU11 sets this up by passing the borders 0 and N - I along with the array.

Lines B to 12 handle the hase case. If left "'''' right, there is one element, and it is the maximum subsequence if the elemcnL is nonnegative. The case 1 eft > ri ght is not possihle unless N is negative (although minor perturhations in the code could mess this up), Lines 15 and 16 perform the two recursive Gl11s. We can sec that the recursive calls arc always on a smaller problem than the original, although minor perturhations in the code could destroy this property. Lines 18 to 24 and 26 to 32 calculate the two maximum sums that touch the center divider. The sum of these two values is the maximum sum that spans both halves. The routine max3 (not shown) returns the largest of the three possibilities.

Algorithm :1 dearly requires more effort to code than either of the two previous algorithms. However, shorter code does not always mean better code. As we have seen in the earlier table showing the running times of the algorithms, this algorithm is considcrJbly faster than the other two for all hut the smallest of input sizes.

The running t.ime is analyzed in much the same way as for the program that computes the Fibonacci numbers. Let T(N) be the time it takes to solve a maximum subsequence surn prohlem of size N. If N = 1, then the program takes some constant amount of time to executc lines 8 to J 2, which we shall call one unit. Thus, TO) = 1. Otherwise, the program must pcxform two recursive calls, the t\\'o for loops between lines I <) and 32, and some small amount of bookkeeping, such as lines 14 and 34. The two for loops combine to

touch cvery clement. in the subarray, and t.here is constant work inside the loops, so the time expended in lines 19 to 32 is O(N). The code in lines 8 to l4, .IS, 26, and·34 is all a constant amount of work and can thus be ignored compared with O(N). The remainder of the work is performed in lincs 15 and J 6. These lines solve two suhsequrncc problems of size NI2 (assuming N is even). Thus, these lines take T(N/2) units of time each, for a total of 2T(N(2). The total time (or the algonthm then is 2T(N(2) + O(N). This gives the equations

1(1)= I

I(N) = 2T(N /2) + O(N)

To sirnplify the calculations, we can replace the O(N) term in the equation above with N; since T(N) will be expressed in Big-Oh notation anyway, this will not affcct the answer. In Chapter 7, \ve shall see how to solve this equation rigorously. For 11m\', if T(N) = 21(N /2) + N, and T( I) = I, then T(2) = 4 = 2 * 2, 1(4) = 12 = 4 * 3, T(H) = 32 = H * 'I,

55



56 Chapter 2 Algorithm AnalysIS

1 r-"/.-2 * Recursive maximum contiguous subsequence sum algorithm. J * Finds maximum sum in subarray spanning a[left .. right]. 'i 'i.- Does not attempt to maintain actual best sequence.

5 */ () int maxSumRec( const vector<int> & a, int left, int fight

7 8 if{ left "'''' right II Base case 'i if(a[left]>O)

10 return a[ left];

II else

12 return Q.

/3

Hint center = ( left + right) /2;

15 i nt maxleftSum maxSumRec( a, left, center ); /6 int maxRightSum '" maxSumRec{ d, center + 1, right);

17 18 int maxLeftBorderSum D, leftSorderSum = 0; It) fore int i = center; i >" left; i-- )

20

21

22

23

24 25

leftBotderSum += a[ i ];

if ( , eftBorderSum > maxLeftBorderSum

maxLeftBorderSum = leftBorderSum;

26 lnt maxRightBorderSum '" 0, rightBorderSum " 0; 27 fore int j " center + 1· j <" right; j++ )

28

29 rightBorderSum += a[ ]; .10 if( rightBorderSum > maxRightBorderSum 31 maxRightBorderSum = rightBorderSum;

32 JJ 34 return max3( maxLeftSum, maxRightSum, 35 maxLeftBorderSum + maxRightBorderSum ); J(i

37 JH /** 39 * Driver for divide-and-conquei' maximum contiguous

40 * subsequence sum algorithm.

4/ */ '12 int maxSubSum3( const vector<int> & a

"3 'H return maxSumRec( a. 0, a.size( ) - 1 );

"5 Figure 2.7 Algorithm j



2.4 Running Time Calculations

I /** 2 * Li near- time maxi mum cant i guous subsequence sum a 1 gorithill.

3 */ 4 int maxSubSulll4( const vector<;nt> & a )

5

6 int lllaxSum '" O. thisSum " 0;

7

8 fore int j '" 0; j < a.size( ); j++ } o {

10 thisSum += a[ j ];

J J

12 if ( thi sSum > maxSull1 13 maxSum '" thisSum; 14 else if( thisSum < 0

15 this$um = 0;

16

J 7 18 return maxSum; 19

Figure 2.8 Algorithm 4

and T(l6) = 80 = ] 6 * 5, The pattern that is evident, and can be derived, is lhal jf N = ii, then T(N) = N * (11 + 1) = N log N + N = O(N log N).

This analysis assumes N is evel1, since otherwise N/2 is not defined. By the recursive nature of the analysis, il is really valid only \vhen N is a power of 2, since otherwise we eventually get a subproblem that is not an even size, and the equation is invalid. \Vhen N is not a POWcf of 2, a somewhat more complicated analysis is required, hut the Big-Oh result remains unchanged.

In future chapters, we will see several clever applications or recursioll. I Iere, \vt' presenL a fourth algorithm to find the maximum suhsequellce Stllll. This algorithm is simpler to implement thaJlthe recursive algorithm and also is more efficiellt. It is shovm in Figure 2.H.

It should be e1e;:\]' why the time hound is correct, lJut it takes a little thought to sec why the algorithm actually \vorks. To skctch the logic, note that like algorithms 1 alld 2, j is representing the cnd of the current sequence, while i is represcnting the start of the current sequence. It happens that the use of i can he optimized oul of the program ir we do not need to know where the actual hest suhsequence is, but in designing the algorithm, let:'? pretend that i is needed, and that we arc trying to improve algorithm 2. Onc observation is that if a[l] is negalivc, thcll it cannot possibly be thc start of the optimal subsequence, since any subsequence thaI begins by including a[l] would be improved by beginning with a [1+1]. Similarly, any negative subscquence canllol possibly be a prefix of the optimal suhsequence (same logic). If, in the inner loop, we detect that the subsequence from a[i] to a [j] is negative, then we can advance i. The crucial observation is thal not only can we advance ito i+1, but we can also actually advance it alllhc way to j+1. To sec this, let p be any index between i+l and j. Any subsequence that starts at index p is not larger than the



58 Chapter 2 Algonthm AnalYSIs

corresponding subsequence thai slarts at index i and includes the subsequence from a[iJ to a[p-lJ, since the latter suhsequence is not llcgative (j is the first index thai causes the suhsequence starting at index i to becol11c ncg,-llivc)_ Thus advancing ito j+l is risk free: \ve cannot rnbs an optimal solution.

This algorithm is typical of many clcYLT algorithms: The running time is obvious, hut the correctness is not. For these algorithms, fOfnwl correctness proofs (more formal than the sketch above) arc almost always rrquirccl; even then, many people still arc not convinced. Also, many of these algorithms require trickier programming, lead ing to longer development. But when these algorithms work, they run quickly, and we can test much of the code logic hy comparing it with an inefficient (but (,(lsily implemented) hrute~force algorithm Llsing small input sizes.

An exira advantage ofthis algorithm is that it makes only one pass through the data, and oncc a[i] is read and processed, il docs 1101 need to he remembered. Thus, irlhe array is on a disk or is being transmitted over the Internet, il can he read sequentially, and there is no need to store any part ofit in main mClllory. runhcrnlOrc, at ,my point in time, the algorithm can corrcctly give all answer to the suhsequence problem for the data it has already rraci (the other algorithms do not share this properly). Algorithms that can do this arc called on-line algorithms. An oil-line algorithm that requires only constant space and runs in linear time is just about as good as possible.

2.4.4 Logarithms in the Running Time The most confusing aspect of analyzing algorithms prnb;;lhly centers around the logarithm. \Ve have already seell !ilal some divide-and-conquer algoril hrns will run in O(N log N) time. Besides dividc-and~conquer algorithms, the Illost frequellt appearance of logarithms centers around the following general rule: ;\/1 algorithm is O(log N) iIUtat<es cons/ani (O( I))

lime [0 (HI {i1e proIJ/cm size by (j fractioll (lvhich is ({Stwi!y 1). On the other hand, if constant

time is required to merely reduce the prohlem by a constant (H110UIl( (such as to make the problem smaller by I), then the algorithm is O(N).

It should be obvious that only special kinds of problems can be O(log N). For instance, if tbe input is a list of N numbers, an algorithm must take Q (N) mcrely to read the input ill. Thus, when we talk about O(1og N) algorithms for these kinds of problems, we usually plTsumc that the illl)Ut is preread. 'vVe pro\'ide three examples of logarithmic behavior.

Binary Search The firs! example is usually referred to as hinary search.

Binary Search. Given an integer X and integers All, AI, ... ,A:-; --I, which arc presorlcd and already in memory, Gnd i such that 1\ = X, or rC\urn i = ~ 1 if X is not in the inpul.

The obvious solution consists of scanning tbrough the list from left to right and runs in linear timc. However, this algorithm docs not take advantage of the fact that the list is

sorted, and is thus not likely to be best. A bettCf strategy is to check if X is the middle clement. If so, the answer b at hal1(L If X is smaller than the middle clement, we can apply the same strategy to the sorted subarray to the left or the middle clement; likeWise, if X is larger than the middle clcment, we look to the right half. (There is also the case of when to



LA Kunmng lime LaiCUla1l0nS

I" 2 * Performs the standard binary search using two comparisons per level.

3 * Returns index where item is found or -1 if not found.

4 *1 5 template <type name Comparable>

6 lnt binarySearch( const vector<Comparable> & a. const Comparable & x }

7 8 int 101'1 '" 0, high'" a.size( ) - 1; o

10 while( low <" high) 11 12 intmid"(low+high)/2; U

14 15

16

17

18

if( a[ mid 1 < x ) lOI'l=mid+l;

else if( a[ mid] > x )

high'" mid - l'

el se

]9 return mid; / / Found

20 21 return NOT_FOUND; II NOT FOUND is defined as -1 22

Figure 2.9 Binary search

stop.) Figure 2.9 shows the code for binary search (the answer is mid). As usual, the code rellects C++5 convention that arrays hegin with index O.

Clearly, all the work done inside the loop takes 0(1) per iteration, so the analysis requires determining the number of times around the loop. The loop starts with hi gh - low = N - 1

and finishes with high - low ~ - L Every time through the loop the value high - 101'1 must be at least halved from its previous value; thus, t.he number of limes around the loop is at most flog(N - 1)1 + 2. (As an example, if high - 101'1 = 12H, then the maximum values of high - 101'1 after each iteration are 64, 32, 16,8,4,2, I, 0, ~ 1.) Thus, the running time is

O(log N). Equivalently, we could write a recursive formula for the running time, but this kind of brute-force approach is usually unnecessary when you understand what is really going on and why.

Binary search em be viewed as our first data structure implementation. It supports the eonta i ns operation in O(log N) ! irne, but all other operations (in part icular insert) require O(N) time. In applications where lhe data are static (that is, insertions and deletions arc not allowed), this could be very useful. The input would then need to be sorted once, hut afterward accesses v·/Ould be fast. An example is a program that needs to maintain information about the periodic table of elements (which arises in chemistry and physics). This table is relatively stahle, as new dements arc added infrequently. The element. names could be kept sorted. Since there are only about 1 10 elements, at most eight accesses would



uu

long gcd( long m. long n )

2 (

.1 while{ n !'" 0

4 (

5 long rem '" m % n;

6 m = n·

7 n = rem; R 9 return m;

10

Figure 2.10 Euclids algorithm

be required to find an demenL. Performing a sequential search would require many more accesses,

Euclid's Algorithm A second example is Euclid's algorithm for computing the greatest common divisor. The greatest common divisor (gal) of two integers is the largest integer that divides both, Thus, gcd('50, ] 5) = 5. The algorithm in Figure 2.10 computes ,~cd(M} N), assuming M ~ N. Of N > M, the first iteration of the loop swaps them.)

The algorithm works by continually computing remainders until 0 is reached. The last nonzero remainder is the answer. Thus, if 1\11 = 1,989 and N = 1,590, then the sequence of remainders is 399, .393, 6, 3, O. Therefore, gcd(19H9, 1590) = 3. As the example shows, this is a fast algorithm.

As before, estimating the enlire running time of the. algorithm depends on determining how long the sequence of remainders is. Although log N seems like a good answer, it is not at all ohvious that the value of the remainder has to decrease by a constanL factor, since we see that the remainder went from 399 to only 393 in the example. Indeed, the remainder does not decrease by a constant factor in one iteration. However, we can prove that after two iterations, the remainder is ilt most half of its original value. This \vnuld show that the number of iterations is at most 2 log N = G(\ng N) ilnd establish the running time. This proof is easy, so we include it here. It follows directly from t.he following theoreJll.

Theorem 2.1. If M > N, then MmodN < MIL

Proof. There are two cases. IfN ::s M/2, then since the remainder is smaller than N, the theorem is true for this case. The other case is N > M/2. But then N goes into M once with a remainder M - N < 1'111/2, proving the thcorclT1.

One might \vonc.kr if this is the best bound possihle, since 21ng N is ahout 20 for our example, and only seven operations \\'er(' performed. 11 turns oul that the constant. can he improved slightly, to roughly 1.44 log N, in the worst case (\\'hich is achic\'ahlc if Iv1 and N arc consecutive, fibonacci numbers). The average-case performance of Euclic.\s algorithm



I

2

J 4 5

long pow( long (

if( n "" 0 return

i f( n ""= 1

x,

I-

) () return X;

int n

7 if( isEven( n ) )

)

8 return pow ( X * x. n / 2 ); 9 else

10

II

return pow( x * x, n / 2 ) , x;

Figure 2.11 Efficienl exponcnl ial ion

L4 Kunnmg lime LalCUlanons

requires pages and pages of highly sophisticated mathematical analysis, and it lUrns out that the average numher of iterations is about (121n 21n N)/n 2 + 1.47.

Exponentiation Our last example in this section deals with raising an integer to a power (v·"hich is also an integer). Numbers that result from exponentiation are generally quite large, so an analysis works only if we can assume that we have a machine that can store :-iuch large integers (or a compiler that can simulate this). We will counl the number of multiplications as the mcasurement of running timC.'.

The obvious algorithm to compute XN uses N ~ I multiplications. A recursive algorithm can do beller. N ~ ! is the base case of the recursion. Otherwise, if N is even, \ve have X N = XN/l. XN/ 2, and if N is odd, XN = X(N-·l)/2. X(N···l)/2. X.

For instance, to compute x02, the algorithm does the following calculations, which involve only nine multiplications:

The number of multiplications required is clearly at most 210g N, because at most two multiplications (if N is odd) arc required to halvc the problem. Again, a recurrence formula can be written and solved. Simple intuition obviates the need for a brute-force approach.

Figure 2.11 implements this idea. Tt is somelirncs intcresting to sec how mllch the code can be tweaked without affecting correctness. In Figure 2.U, lines') to 6 are actually unnecessary, because if N is ], then line 10 docs the right thing. Line 10 can also be rewritten as

10 return pow( x, n - 1 ) * X;

without allccling the correctncss of the program. Indeed, the program will still run in O(log N), because t.he sequence of multiplications is the same as before. IIowever, all of the following alternatives for line K arc bad, even though they look correct:

8(1

8b

return pow{ pow{ x, 2 ). n / 2 ); return pow( pow( x, n I 2 ), 2 ); return POl'J( x, n / 2 ) * pow( x, n / 2 );

." Bismillah hiir Rehman nir Raheem-----------------------------Assalat o Wasalam o Alika Ya RasoolALLAH


'-,." ... 1.,. .... ' L rlll)U"UIIIIT"lllU'r-""

double probRelPrime( int n 2 ( 3 intrel"'O.tot"'O;

4 5 for( int i "'- 1; i <'" n; i++ )

6 for( int j '" i + 1; j <".- n; j++ )

7 (

H

9

10 II

12

tot++; if( gcd( i, j)

rel++;

13 return (double) reI/tot; 14

1 )

Figure 2.12 Estimate the probability that two random numbers are relatively prime

Both lines Sa and 8b are incorrect because when N is 2, one of the recursive calls to pow

has 2 as the second argument. Thus no progress is made, and an infinite loop results (in a.n eventual crash).

Using line He affects the efficiency, because there are now two recursive calls of size N /2 instead of only one. An analysis will show that the running time is no longer O(log N). 'vVe leave it as an exercise to the reader to determine the new running time.

2.4.5 Checking Your Analysis Once an analysis has been performed, it is desirable losee if the answer is correct and as good as possible. One way to do this is to code up the program and see if the empirically observed running time matches the running time predicted by the analysis. When N doubles, the running time goes up by a factor of 2 for linear programs, 4 for quadratic programs, and 8 for cuhic programs. Programs that run in logarithmic time take only an a.dditive constant longer when N doubles, and programs that run in O(N log N) take slightly more than twice as long to run under the same circumstances. These increases can he hard to spot if the lower-order terms have relatively large coefficients and N is not large enough. An example is the jump from N = 10 to N = 100 in the running time for the various implementations oj the maximum subsequence sum prohlem. It also can be very difficult to differenliale linear programs from O(N log N) programs purely on empirical evidence.

Another commonly used trick to verify that some program is O(j(N») is to compute the values T(N)!I(N) for a range of N (usually spaced out by factors of 2), where T(N) is the empirically observed running time. If feN) is a tight answer for the running time, then the computed values converge to a positive constant. Iff(N) is an overestimate, the values converge to zero. Iff(N) is an underestimate and hence wrong, the values J.jverge.

As an example, the program fragment in Figure 2.12 computes the probability that two distinct positive integers, less than or equal to N and chosen randomly, are relatively prime. (As N gets large, the answer approaches 6/rr 2 )



Summary

N CPU lime CI) liN' liN' Tj (t'i! log N)

100 022 ,002200 ,000022000 ,0OOH77

20O 0')6 ,001400 J)00007000 0002M2

300 118 ,00ll11 ,000004170 ,0002299

400 207 J)01294 ,000003234 ,00021 '39

)00 '118 ,001272 J)00002'544 JJ002047

60O 466 00129'1 ,OOOOO2Ij7 JJOO2024

700 644 ,001114 ,000001877 ,Oll0200h

800 846 001322 ,000001652 JJOOI977

90O 1,086 ,001 HI ,000001490 ,0001971

1,000 1,162 001)62 OOOOCll362 ,0001972

1 ,')00 '3,240 ,001440 JIOOOO0960 ,0001969

2,000 5,(H0 JIll 14H2 ,000000740 ,0001947

4,000 2'5,720 ,00 I 60B JIOOOOIH02 ,0001918

Figure 2.13 Empirical running times for the routine in l:igme 2.J2

You should be able to do the analysis for this progral11 instantaneously. figurc 2. L3 shows the actual ohserved running time for this routinc on a real computer. The tahle shows that the last columll is 1110St likely, and thus the analysis that you should h,lVe gotten is prohahly correcL Notice that there is not a great deal of difference between O(N2.) and O(N2 1og N), since logarithms grow so slowly.

2.4.6 A Grain of Salt Sometimes the analysis is shown empirically to be an overestimate. If this is the case, then either the analysis needs to be tightened (usually hy a clever observation), or it may be that the (/vcrc~gc running lime is significantly less than the worsl-case running time and no improvement in the bound is possihle. For 111<1l1y complicated algorithms the worstcase bound is achievahle hy some had input hut is usually all overestimate in practice. Unfortunately, for most of these problems, all average-case analysis is extremely complex (in mallY cases still unsolved), and a worst-case bound, even though overly pessimistic, is the best analytical result known.

Summary

This chapter gives some hints on how to analyze the complexity of programs. Unfortunately, it is not a completl' guide. Simple programs usually have simple analyses, hut this is not always the case. As an example, later in the lext we shall sec a sorting algorithm (Shellsort, Chapler 7) and an algorithm for maintaining disjoint sets (Chapter B), each of which H.'quires

63



I....napler L J-I.lgomnm AnalYSIS

ahout 20 lines of code, The analysis of Shcllsort is still not complete, and the disjoint set algorithm has an analysis that is extremely difficult and requires pages and pages ofintricate calculations. Most of the analyses that wc will encounter here \vill be simple and involve counting through loops.

An interesting kind of analysis, \",hieh we have not touched upon, is lower~bound analysis. 'We will see an example of this in Chapter 7, \vhere it is proved that any algorithm that sorts by using only comparisons requires Q(N log N) comparisons in the worst casco Lower-bound proofs arc generally the most difficult> hecause they apply not to an algorithm but to a class of algorithms that solve a problem.

We close by mentioning that sornc of the algorithrns described here have rcal~hfc

application. The gal algorithm and the exponentiation algorithm arc both used in cryptog~ raphy Specifically, a 400-c1igit numher is raised 10 a large power (usually another 400-digit number), with only the Imv 400 or so digits retained after each multiplication. Since the calculations require dealing with 400~digit numbers, efficiency is obviously important. The straightforward algorithm for exponentiation would require about 10400 multiplications, whereas the algorithm presented requires only about 2,600 in the worst casco

Exercises

2.1 Order the following functions by growth rate: N, -Jfj, Nl.">, N 2, N log N, N log log N, N log2 N, N log(N 2), 21N, 20.',20.'/2, ")7, N2 ]og N, N'I. IndiCalC which functions grow al the same rate.

2.2 Suppo,c TI(N) = O(l(N» and I,(N) = OU(N» Which of the following arc true' a. TI(N) + T2(N) = O(/(N» b. TI(N) - T2(N) = o(f(N»

c. TI(N) =0(1) . I,(N)

cL T,(N)= O(T,(N»

2.3 \>\1h1cl1 function grows faster: N log Nor NJ-h/JlogN, f > O?

2.4 Prove that for any constant, I?, logll N = o(N).

2.5 Find two functions feN) and g(N) ,ueh Ihat neither feN) = O(g(N» nor g(N) = OU(N».

2.6 In a recent court case, ajudge cited a city forconternpt and ordered a fine 0[$2 for the first day. Each subsequent day, until the city !i)llowed the judge's order, the fine was squared (that is, the fine progressed as follow,: $2, $4, $16, $256, $65,536, ... ). a. \>\1hat \vould be the fine on day N? h. How many clays would it take for t.he f"ine to reach D dollars (a Big~Oh answer

will do)?

2. 7 For each of the following six program fragmcnIs: 3. Give an analysis of the funning time (Rlg~Oh \vilJ do). h. Implemem the code in the language of your choice, and give the running lime

for several values of N. c. Compare your analysis with the actual running times.



(1) sum 0 0;

for( i '" 0; i < n; i ++ )

sum++; (2) sum 0 0;

for( i = 0; i < n; i++ )

for( j '" 0; j < n; j++

sum++; (3) sum 0 0;

fore i '" 0; i < n; i++ )

for( j " 0; j < n * n: j++ )

sum++;

(4) sum 0 0;

for( ; co 0; i < n; i ++

fod j z: 0; j < i; j++ sum++;

(5) sum 0 0;

for( ; '" 0; i < n; i++ )

fore j '" 0; j < i * ;; j++

fore k '" 0; k < j; k++ sum++;

(6) sum 0 0;

fore; = 1; i < n; i++ )

fod j '" 1; j < i * i; j++

if ( j % i 00 0 )

fore k '" 0; k < j; k++

sum++;

Exercises

2.8 Suppose you need to generate a random pennutation or the first N integers. for example, {4, 3, 1, 5, 2) and {3, 1,4,2,5) are legal permutations, but {5, 4, 1,2, l} is not, because one number (1) is duplicated and another (3) is missing. This rout inc is oftcn used in simulation of algorithms. \'\le assume the existence of a randol11 number gencrator, r, with method randInt(i ,j), that generates integers between i and j with equal probability. Here arc three algorithms:

1. Fill the array a from a [0] to a [N-l] as follows: To hll a [i], generate random numbers unt iI you get onc that is not already in a [0] ) a [1], . , a [i -1] .

2. Same as algorithm (1), hut keep an exira array called thc used array. When a random number, ran, is first put in the array a, set used[ran] "" true. This means that whcn filling a[i] with a random number, you can test in one step to see whether the random number has been used, instead of the. (possibly) i steps in the first algorithm.

3. fill the array such thai a[l] = i+1. Then

fore i '" 1; i < n; i++ )

swap( a[ i ], a[ randlnt( 0, i ) 1 );

a. Prove that all thrce algorithms generate only legal permutations and that aU permutations are equally likely.

65



00 UldjJlel L MI~UIHIIIII t\lIdIPI~

h. C;i\,(' as accurate (Big-Oh) an analysis as YOll can or 1 he cxpcdcd runlling lime of each algorithm.

c. \,Vrilc (separate) programs to execute each algorithm 10 times, to get a good average. Run program (I) for N = 2'50, ')00, j ,000, 2,000; program (2) for N = 25,000, SO,llOO, 100,000,200,000,400,000, HOO,OOO; and program (3) lor N = 100,000,200,000,400,000, HOO,OOO, 1,600,000,3,200,000,6,400,000,

d. Compare your analysis \vith the actual running limes. c. \Vhat is the worst-case running time of each algorithm?

2.9 Complete the tahle in Figure 2.2 with estimates for the running Limes that 'were 100

long to simulate. Interpolate the: running Limes lor these algorithms and estimate the lime required to compute the maximum subsequence sum of I milliun numbers. Vlhat assumptions have you made?

2.10 Determinc, for the typical algorithms that you llse to perform calculations by hand, lhe funning lime to do the fullowing: ;L Add two N-digit integers. b. lvlultiply t\\'o N-LligiL inu:gcrs. c. Divide t\\'o N-digl[ integers.

2.11 An algorithm takes 0.5 ms for input size 100. lIow long will illakt' for input size '500 if the running time is the following (assume low-order terms are negligible)? a. linear b. O(N log N) c. quadratic Ll. cubic

2.12 An algorithm takes O. ') ms for input size 100. How large a prohlem can be solved in 1 min if the running time is the following (assume low··order terms are negligible)? a. linear b. O(N log N) c. quallratil" d. cubic

2.13 J low much time is required to COmpllll' I (x) = E;:'o (jiXi:

a. Using a simple routine to perform cxponentiation? h. Using the routine in Section 2.4.4?

2.14 Considcr the following algoril hm (known as Horner\ rule) to evaluate I(x) = "N .i L,i~o;O (/iX :

poly" 0;

for( i n' i >= 0; i-- )

poly = x * poly + a[i];

<1. Show how the steps arc performed hy this algorilhm for x =:3, f(x) = 4x4 + HxJ +x + 2.

b. Explain why this algorithm works. c. vVhat is the running time of this algorithm?



Exercises

2.15 (;ive an efficient algorithm to determine if there exists an integer i such that AI = i in an array of integers A I <:: 1\2. < A:I < ... < /\,\'. \Vhat is the running time or your algorithm?

2.16 \-Vrite an alLern<ltivc gcd algorithm based on the following observations (arrange so

that a > lJ): a. gul(a, /J) = 2<~cd(aI2, h12) if (/ allli]; arc both e\'en. h. galea, h) =gccl(aj2, b) if a is cwn and b is odd. c. gcd(a, h) = gcd(a, hj2) if a is odd and h is even. d. gcd«(/, b) = gcd«(/ + b)/2, (a - iJ)12) rf (/ and h arc both odd.

2.17 (;i\lc efficient algorithms (along with running time analyses) to: a. Find the minimum subsequence sum.

* h. Find llle minimum positive subscqucnce slim. * c. Find lhl' maximum subsequence product.

2.18 An important problem in Tlumeric<ll analysis is to find a solution to the equation I(X) = 0 for some arlJitrary f. If the function is continuous and has two points low and hig,h such that f(lmv) and I(high) have opposite signs, thell a root must exist between low and hi<~1! and can be found by a hinary search. Write a function that takes as parameters f, IOH~ and high and solves for a zero. \Vhal HIust you do to ensure termination?

2.19 The maximum contiguous subsequellce SUIH algoritlulls in the lex[ do not give all)'

indication of the actual sequence. "tv1odify them so tbm they relUrn ill a single ohject the value of the maximum subsequence and the indices of the actual sequence.

2.20 <I. Write a progralll to determine if a positive integer, N, is prime. b. In terms ufN, whm is the worst-case running time of your program? (You should

be ahk to do this in O( ffi).) c Let 13 equal the number of bits in the binary reprcsentalion of N. What is the

value of 13? cl. In terms of H, \\'hat is lhe worst-case running time of your program? c. Compare tbe running limes \0 determine jf a 20-bit nUlllberand a IfO-bit number

arc prime. r Is i1 more reasonable \0 give the running timc in terms of Nor m \Vhy?

* 2.21 rhe Sicve of Eratosthenes is a method used to computc all primes less than J\l, \\le hegin hy making a table of integers 2 to f\L \-Ve flnc1thc smallest integer, i, that is

not crossed out, print i, and cross out i, 2i, 3i, . . \"'hen i > V/\f, the algorithm terminates. \-Vhat is the running Lime of this algorithm?

2.22 Show thm X6 ) can be computed with only eight multiplications.

2.23 \-VrilC the fast exponcntiation routine without recursion.

2.24 (~ive a precise count on the number of multiplications used by the bSl exponentiation roulinc_ (I-lint: Consider the binary representation of N.)

2.25 Programs A aud B arc analyzed and found 10 have worst-case running limes no greater thall I SON log) Nand N}, respectively. ;\nS\\Tr the following questions, if p(x.;sible:

67



00 Uld]Jlt:1 L t\lbUlrUltII11.1ldIPI)

a. VI/hieh program h,lS the hetter guaranLee Oil the running lime, for large values of

N (N > 10,0(0)' h vVhich program has the hClIcr guarantee on 1 he runlling lillle) [or 5J11<1I1 values

of N (N < lOO),

c. \Vhich program will nm bSLef on (lVCrclSr for N = 1,OOC)"? d. Is it pO.'::isible lhal program 13 will run faster than prognll1l A on (//1 possihle inputs?

2.26 A m(-~jorjl y cJcmcllt in an array, 1\, of size N is an clement tl1<11 appears more than N/2 times (thus, 1hrr(' is a1 most one). ror example, the array

3,3,4,2,4,4,2,4,4

11<15 a m;\j()fity rlement (4), whereas the (lrray

l,l, 4, 2, 'f, 4, 2, 'f

does not. If there is no l1ujority clement, your program should indicate this. I !crc

is a sketch or an algorithm to solve the prohlem:

First, a canditbtc !1uljorily elemenl is Ji>tmd (lhis is the /ulrdn pcnt). This candidate is

{he ol1/Y clement thaI could possibly be (he l11eUorily clement- Tht' secol!d step dcl.crl1lincs if (his wndidate is w:tually the Ill({jorily This isjust (/ sequential search Ihroup,h lile urray To find (/ wndidel/f in lhe array, i\,jiJnn a secoJJd arwy, B. Then COIll]7(I!"C Al ond A2· ~r thcy (/rc cqual, add one or these to B; otherwise do Ilothin(~. Fhcll compare A'I and AI. Again if they aft' equal, add one of these to B; other wist' do 11()lhill,~. Continue in this fmilioll until the entire w"/"uy is re(u/. T/Jen rccursivetv find (j (Undidale .for H; this is the (dlldiduie .!()/

II (why')'

a. J low docs the recursion terminate?

* h. f Imv is the case where N is odd handled?

c. VVhat is the runlling lime of the algorithm?

d. J low can we (l\'oid using an extra array m * c. \,Vrite a program to compute t he m<.~inrity clcment.

2.27 Thc input is an N by N mal rix of numbers that is already ill memory. Each individual

row is increasing frolll left to right. Each indi\'idual column is increasing from top

to bottom. (~i\'C <.1.n O(N) worst-case algorithm that decides if a number X is ill the

matrix.

2.28 Design efficient algoritllllls that ukc an array of positi\'C numbers a, and dClennilK·

a. the maximum value of a[j]+a [i], with j ?: i.

b. the maximum value' of a[j] -a [i], with j 2: i

c. the m,lximullJ value of a[j] *a [i], with j 2:. i.

d. the maximum value or a[j]/a[i], \vith j 2:~ i.

* 2.29 \'Vhy is it important to aSSlItllt' thal integers in our computer model have a fixed

size?

2.30 Consider the word puzzle probletll descrihed ill Chapter·j. Suppose we fix the size

of the longest word to be 10 characters. <i. [11 terms of Rand (.', which arc the number of rows and columns ill the puzzle, and

\.V, which is the llumber of words, what arc tht' running times of the algorithms

described in Chapler l?



I{eterences

h. SUPIJUSC the word Jist is presorted. Show how tu usc binary search to ohtain an

algorithm with significantly hettcf running time.

2.31 Suppose that linc 1'5 in the hinary scarch routine had the statement low'" mid instcad

of low'" mid + 1. \Vould the routinc still work?

2.32 Implelllcntthc hil1ary search so that only onc two-way comparison is performed in each iteration.

2.33 Suppose thallines IS and 16 in algorithm.1 (rig. 2.7) arc replaced hy

15 int maxLeftSum maxSubSum( a, left, center - 1 ); )() int maxRightSum maxSubSum( a, center, right );

\,Vould the routine still work?

* 2.34 The innCl" loop of t he cubic Illaximum subsequence sum algorithm performs

N(N + l)(N + 2)/h iterations of the innermost code. The quaclnHic version per~

forms N(N + 1)/2 iterations. The linear version perfurms N iterations, \\lhat paltcrn

is c\'idellt? Can you give a combinatoric explanation of this phenomenon?

References

Analysis or the running lime of algorithms was iirst made popular hy Knuth in Ihe thn.'c~

part series ["ij, 161, and 171. Analysis of the p,cd algorithm appears in 16] Another early text

on I he subject is II J

Big~Oh, hig-omega, hig-theta, and lillle-oh notation were advocated by Knuth in 18!. There is stili nut uniforlll agreement Oll the maller, especially when it comes 10 using ('"')0.

Man), people prefer In llSC 00, even though it is less expressl\'e. Additionally, 00 is still

used in some corners to express a lower bound, when Q 0 is called for.

The maximulll suhsequcnce sum problem is from 1:31. The series or hooks [2]) [3], and

[4J show how to op1imizc programs for speed.

A V Aho,J. L I jopcroh, and,J. n. Ullman, The Design w!(l An(11y~h (!f Computer Alg,ofilillm, Adclisoll-\Vcsley, Reading, lvlass., 107,4.

2. L Bentley, \\frilill}!, L({i( ii'llt Pro,'<;foHlS, Prentice IIaU, Englewood (:lifts, NJ., 19i:l2.

). j. L. fktltlcy, Prog,rdlmnillp; Pearls, Acidison~Wesley, Reading, Mass., 1086

4. J. i" Benticy Mo/'(' PmgUllllmill(~ Pcnfb, AddisOlI-\Vcslcy, Reading, [Vh1:;S., 19B8

). I). L KrlLuh. Tile ;\fl of Computer Frop/ammill,'<;, ViJI .i: FW](/mll(ll/a/ AlgorU/nlls, 3d ceL, AddisOll-\i\·blcy, Hcc\ding, hhss., I~N7.

6. D. L Knuth, TIle Arl (!! Computef JlrogrwJlmillg, Val 2: .'-iemillumcriw/ AI,~()rilhms, :3d cd., Adclison-\Vesley, Reading, [vlass., 1098.

7. D. E. Knuth, The Arl of COIllP[I/tr Pro(~lwl1milJg, Vol 3: Sorting and .'-ic(I/"(ilill,g, 2d eel., ]\ddisOJl\Vcslcy, Reading, 1\'1<15S., 109H.

H ]). L. Kl1ll1h, "[)ig Omicron aod Big Omega and Big Thct;c" ACM SI(;;\C.T News, R (1976), IR-21

69





CHAPTER 3

Lists, Stacks, and Queues

This chapter discusses three of the most simplt' and basic data structures. VinuaHy every significant prognun \vill usc al le;JS\ one of these structures explicitly, and a stack is always implicitly used in a program, whether or not you declare one. Among the highlights of this chapter, we will

Introduce the concept of Abstract J);H(l 'Iypcs (,-\tns).

Show hO\:v to efficiently perform operations on lists.

Introduce the stack AJri and its usc in implementing recursion.

Introduce the queue .-\lH and ilS usc in operating systems and algorit 11m design.

In this chapter, \VC provide code lhm implements a significant subset of two library classes". vector and 1 i st.

3.1 Abstract Data Types (ADTs) Anahstract data lype (,urr) is a set of objects logClhcrwith a set of operations. Ahstract data types arc: mathematical abstractions; nowhere in an ADTS definition is there any mention of how the set of operations is implemented. Objects such as lislS, sClS, and graphs, along with their operations, can be viewl'd as abstract data types, just <IS integers, reals, and booleans arc data lypCS. Integers, reals, and boolcans have operations associated with them, alld so do abstract data types. For the sct A/H, we might have stich operations as add, remove, size, and contains. Alternatively, we might only want the two operations union and find, which would define a difkn:nt Aln un the set.

Thc C++ class allows for the implementatioll of A1HS, with ,{ppropriate hiding 01 implementatioll details. Thus any other part of the. program that needs 1.0 perform all

operation on the ADT can do 50 by cailing the appropriale method. If for ~O!llC reason implementation details need 10 be changed, it should he easy 10 do so by merely changing the routines that perform the ADT operalions. This change, in a perfect world, would bc compleLely transparent to the rest of the program.

There is no rule telling us which operations must be supported for each A!n; this is a design decision. Error handling and tic breaking (where appropriate) arc also generally up to the program designer. The three data slruc!UrfS Ihal wt' will study in this chaptCl" are primary examples of Aty! s. 'vVe will see bow each can he implemented in several ways, hut if they arc done cOlTccliy, the prngrallls that usc them will not nccess,uily need to know which implementafion was used. 71



3.2 The List ADT 'vVe will deal with a general list of the for!l1/\o, AI> /\2> .. "' A N __ 1_ vVe say that the size of this list is N. Vvc \\,111 call the special list of size 0 an empty list.

for any list except the empt y list) \\1(' say that Ai follows (or succeeds) Ai -- 1 (i < N) ami that Ai -- 1 precedes 1\ (i > 0). The first clement of the list is Ao, and the last dement is AN_ j. We will not define the predecessor of Ao or the successor of AN-I- The position of element Ai in a list is i. Throughout this discussion, we will assume, to simplify matters, that the elements in the list are integers, hut in general, arbitrarily complex clements arc allowed (and easily handled by a class template).

i\ssocimcd with these "definitions" is a sct of operations that we would like to perform on 1 he list ADT. Some popular operations arc pri ntL; st and make£mpty) which do the obvious things; find, which returns the position of the firs1 occurrence of an item; insert and remove, which generally insert am} remove some clement from some position in the llst; and fi ndKth, which returns the clement in some position (specified as all argument). If the list is 34, 12, 52,16,12, then find(52) might return 2; insert(x.2) might make t.he list into :14, 12, x, 52, L6, 12 (if we insert into the position given); and remove(52) might turn that list into .34,

12, x, 16, 12. or course, t.he interpretation of what is appropriate for a function is entirely up to the

programmer, as is the handling of special cases (for example, what does find(l) return ahove?). V'.,!c could also add operations such as next and previous, which \\.'ould take a

position as argument and return the position of the successor and predecessor, respectively.

3.2.1 Simple Array Implementation of Lists All of these instructions can be implemented just by usjng an array. Although arrays arc created with a fixed capacity, the vector class, which internally stores an array, allows the array to grO\v by doubling its capacity when needed. This solves the most serious problem with using an array, namely that historically, to usc an array, an estimate of the maximum size of the list was required. This estimate is 110 longer needed.

An array implementation allows printList to be carried out in linear time, and the findKth operation takes constant lime, \vhich is as good as can be expected. HowevCl", insertion and deletion are potentially expensivc, depending on where the insertions and deletions occur. In the worst case, inserting into position 0 (in other words, at the front of the list) requires pushing the entire array down one spot to make room, and Lidding the first clement reqUires shifting all the clements in the list up one spot, so the worst case for these operations is O(N). On average, half of the list needs to be moved for either operation, so linear time is still required. On the other hand, il" all the operations occur at the high end of the list, then no clements need to he shifted, <lnd 1 hen adding and delet ing take 0(1) time.

There are many situations \vhc:re the list is built up by insc:nions at the high end, and then only array aCcesses (i.e., fi ndKth operations) occur. In such a case, the array is a suitahle irnplemcrHation. However, if insfrtions and deletions occur throughout the list, and in particular, at the front of the list, then the alTay is not a good option. The next section deals with the alternative: the lillhed list.



.) • .£. lilt: L.l:Il MUI

3.2.2 Simple Linked Lists In order to avoid the linear cost of insertion and deletion, \vc need to ensure that the list is not stored contiguously, since otherwise entire pans of the list will need 10 be moved. Figure 1, I shows the general idea of a linked list.

The linked list consists of a series of nodes, which are not necessarily adjacent in memory. Each node contains the clement and a link to a node containing its successor. We call this the next link. The last cells next link points to NULl.

To execute printList{) or find(x), we merely stan at the first node in the list and then traverse the list by following the next links. This operation is clearly linear-time, as in the array implementation, although the constant is likely to be larger than if an array implementation were used. The fi ndKth operation is no longer quite as eITlcient as an array implementation; findKth(i) takes O(i) time and works by traversing down the list in the obvious manner. In practice, this bound is pessimistic, because frequently the calls to fi ndKth are in sorted order (by 1). As an example, fi ndKth (2), fi ndKth (3), f i ndKth (4), and findKth(6) can all be executed in one scan down the list.

The remove method can IJC executed in one next pointer change. figure 3.2 shows the result of deleting the second element in the original list.

The; nsert method requires ohtaining a new node from the system by using a new call and then executing two next pointer maneuvers. The general idea is shown in Figure 3.-"3. The dashed line represents the old pointer.

Figure 3.1 A linked lisl

Figure 3.2 Deletion from a linked list

Figure 3.3 Insertion into a linked list

... Bismillah hiir Rehman nir Raheem-----------------------------Assalat o Wasalam o Alika Ya RasoolALLAH


first last

Figure 3.4 A doubly linked list

;\s we can sec, in principle, ir wc know where a change is to be made, inserting or removing an item from a linked list docs not require moving lots of items, and instcad involves only a constant number of changes to node links.

The special case of adding to the front or n..'llloving the lirsl item is thus a constant-t.ime operation, presuming of course that a link to the front 01" the linked !Lst is maintained. The special case of adding at the end (i.c., nwking the new item as the last item) can be constanttimc, as long as we maintain a link to the last node. Thus, a typical linked list keeps links to both ends of the list. Removing the last item is trickier, because we have to find t.he nextto-last item, change its /lex! link to NULL, and then update the link that maintains the last node. In the classic linked list, where each node stores a link to its next node, having a link to the last node provides no information ahout the nl'xt-to-bst node.

The ohvious ic\r;I of maintaining a third link to the next-to-last node doesn't work, hecause il too would need 1.0 he updated during a remove. Instead, we have every node maintain a link to its previous node in the lis!. This is shown ill Figure .. ~.4 and is known as a doubly linked list.

3.3 vector and list in the STL The C++ language includes, in ils library, an implementation or common data structures. This pari of the language is popuhlrly known as the Standard Template Library (STL). The List AUT is olle of the data structures implemcnted in the STL. \,Ve will sec some others in Chapter 4. In general, these data structures arc called collections or containers.

There arc two popular implementations of the List Arn. The vector provides a gro\vahle array implementation ofthc l.isl A])I. Tlw advantage of using the vector is tbat it is indexabk ill constant time. The disadvantage is 1.hal. insenion of JlfW items and removal of existing items is l'xpensivc, unless the changes (Ire made at the cnd of I he vector. The 1 i st provides a doubly linked list implementation of the List AJ}T. The advalllage of using the 1 i st is that insertion of new items and removal of existing items is chCi.\p, provided that the posit ion 0/ the changes is known. The disadvantage is that the 1 ist is not easily indexable. Both vector and 1 i st arc inefficient for searches. Throughout this discussion, 1 i st refers to the douhly linked list in the STL, whereas list (typeset without the monos pace ron!.) refers to thc more general List ADT.

BOlh vector and 1 i st are cldss templates that are instantiated with the type of items that they storc. Both have sevcrallllclhods in comlllOll. The !irst three methods shown are actually aVililahlc for all the STt cOlltairwrs:



j.j vector and list In the ~Il

int size( ) canst: returns the number of elements in the container.

void clear( ): removes all elements from the container.

bool empty( ): returns true if the container contains no elements, and false otherwise.

Both vector and 1 i st support adding and removing from the end of the list in constant time. Both vector and 1 i st support accessing the front item in the list in constant time. The operations are:

va; d push_back ( canst Object & x ): adds x to the end of the list.

voi d pop _back ( ): removes the object at the end of the list.

canst Object & back( ) canst: returns the object at the end of the list (a mutator that returns a reference is also provided).

canst Object & front( ) canst: returns the object at the front of the lis! (a mutator thai returns a reference is also provided).

Because a doubly linked list allows efficient changes at the front, but a vector does not, the following two methods are available only for 1 i st:

void push_front( canst Object & x): adds x to the front of the list.

voi d pop_front ( ): removes the o~ject at the front of the 1 i s L

The vector has its own set of methods that are not part of 1 i st. Two methods allow efficient. indexing. The other two methods allow the programmer to view and change tbe internal capacity. These methods are-.

• Object & operator[] ( int idx ): returns the object at index idx in the vector, with no bounds-checking (an accessor that returns a constant reference is also provided).

Object & at ( int idx ): returns the object at index idx in the vector, witb houndschecking (an accessor that returns a constant reference is also provided).

int capacity( ) const: returns the internal capacity of the vector. (See Section 3.4 for more details.)

void reserve( int new Capacity): sets the new capacity. If a good estimate is avai!;\blc, it can be used to avoid expansion of the vector. (See Section 3.4 for more dewils.)

3.3.1 Iterators Some operations on lists, most critically those to insert and remove from the middle of the list, require the notion of a position. In the STL, a position is represented by a nested type, iterator. In particular, for a list<string>, the position is represented by the type 1 i 5 t<s tri ng>: :; terator; for a vector, the position is represented by a class vector<int>: :iterator, <Ind so on. In describing some methods, we'll simply use iterator as a shorthand, but when writing code, we will use the actual nested class name.

lnitially, there are three main issues to address: first, how one gets an ito·atm; second, what. operations the iterators themselves can perform; third, which List ADT methods require ileralOrs as parameters.

75



'0

Getting an !terator For the first issue, the STL lists (and all other ST1. containers) define a pair of methods:

iterator begin{ ): returns an appropriate itcrator representing the IJrst item in the container.

iterator end( ): returns an appropriate itl'ralor representing the end marker in the container (i.e., the position after the last item in the container).

The end method seems a little unusual, because it returns an iterator that is "OUI-or-

hounds." To see the idea, consider the folknving code typically used to prim the items in a vector v:

fore int i '" 0; i too v.size( }; ++i )

cout « v[ i ] « endl;

If we were to rewrite this code using ileralOrs, we would see a natural correspondence with the begi n and end methods:

fore vector<int>: :iterator itr v.begin{); itr != v.end( ); itr.??? ) cout « itr.??? « endl;

In the loop termination test, both i !=v.size( ) and itr!=v.end( ) arc intended to test if the loop counter has becomc "out-of-bounds," The code fragment also brings us to the second issuc, which is that thc itcrator l1lust have methods associated with it (these unknown methods arc represented with ???).

Iterator Methods Based on the code fragment above, it is obvious that iterators can he corn pared with l = and ==, anc1likely have copy constructors and operator= defined. Thus ilcr3tors have mcthods, and mallY or thc mcthods usc operator overloading. Besides copying, the most commonly used operations on iterators include the following:

itr++ and ++itr: advances the iLcrator itr to the next location. Both the prefix and postfix forms arc allowable.

*itr: returns a reference Lo the ohject stored al iterator itr's location. The rderence returned mayor ma), not be modifiable (we discuss these details shortly).

itl'l==itr2: returns true if iterators itrl and itr2 refer to the same location and false otherwise.

itrl!"'itr2: returns true ifiterators itrl and itr2 refer 1.0 a diffnent location and false otherwise.

\~Tith these operators, the code to print would he:

for( vector<int>:;iterator itr = v.begin( ); itr !~ v.end( ); ++itr ) cout « *itr « endl;

The usc of operator overloading allows one to access the current item, and then advance to the next item using *itr++. Thus an alternative to the fragment ahove is:



vector<int>;:iterator itr = v.begin( ); while( itr !~v.end( ) )

cout « *itr++ « endl;

Container Operations That Require Iterators

j.j vedor and lIst In the STl

For the last issue, the three most popubr methods that require iter<llOrS are those that add

or remove from Ihr list (eithcr a vector or 1 i st) at a specified position:

iterator insert( iterator pos, const Object & x): adds x into the list, prior 10 the position givcn by the ilerator pos. This is a constant-time operation for 1 1st, but not for vector. The return value is an ileralOr representing the position of the inserted item.

iterator erase( i terator pos ): removes the object at the position given by the itcr:l!or.

This is (\ constant-time opcmlion for ·1 i st, but not for vector. The return valuc is the

position of the demellt th,ll followed pos prior to the call. This operation invalidates

pos, which is now stale, since the container ilCIll it was viewing lws heen l"C'llloved.

i terator erase ( i terator start. i terator end): removes all items beginning at posi

tion start, up to, but not including end. OhscrYl' that the entire list can be erased by the call c.erase( c.begin( ), c.end( ) ).

3.3.2 Example: Using erase on a List As all example, w(' provide a rOlHilll' that removes every other item in a list, starling \.\'lth

the initial item. Thus if the list contains 6, ), 1, 4, 2, then alter the method is invoked it

will con lain '5, 4. VI,!e do this hy slcpping through thc list and using the erase method on evcry second item. 011 <llist, this will be a linear-time routine hecause each of the calls to erase takes constant time, hut in a vector the entire routine will take quadratic titne because each of the calls to erase is indflcieIlt, using O(N) time. Thus normally, we would write the code for a list only. I Iowc\'er, for cxperimentation purposes, we write a gcneral fUTlction tcmplme that will work with both a 1 ist or a vector, and then provide timing information. The function template is sbown in figure .3.5. The usc of typename at line 4 is required on some compilers to express the bet that Contai ner:: i terator is a type, and not <l data field or Jllethod. If we run the code, passing ali st, it takes 0.062 sec. for a '+OO,OOO-item 1 i st, and 0. [25 sec. for all HOO,OOO-ill'm list, and is clearly a !inl';:H~lirnc routine, because 1111..' running time increases by thl' same' factor as the input size. \,Vhell WC pass a vector<int>, the routine I;Jkcs almost two-and-a-half minutes for a 400,000-1ton vector, and over ten

minutes for ,In BOO,OOO-itClll vector; the four-fold increase in running tilUe when the input

increases hy only a faClor of Iwo is consistent with quadratic behavior.

3.3.3 canst i teratars The result of *itr is not just the value' of the item that the itermor is vinving, but it is the

ilem ilsclf. This distinction ll1<lkcs the iLeJ"ators very powerful, but also introduces some complications. To sec the heneflt, suppose we vvant to change all tbe items in a collccliolllO a specifIed value. The follOWing rout inc works for both vector and 1 ist, and rllns in linear

timc. ft's a wonderful examptc of writing generic, type-independent coele.

77



16 Lnaprer') L1S[S, )li:llKS, dilU \.,!ut!ues

template <typename Container> 2 void removeEveryOtherltem( Container & 1st) 3 4 type name Container::iterator itr ~ lst.begin( ); .5 while( itr !" Ist.end( ) )

6 (

7 8 9

10

11

itr ~ lst.erase( itr ); iff itr !" Ist.end( ) )

++itr;

Figure 3.5 Using iterators to remove every other item in a List (either a vector or 1 i 5t), Efficient for ali s t, but not for a vector.

template <type name Container, type name Object> void change( Container & c. canst Object & newValue (

typename Container::iterator itr ~ c,begin( ); while( itr != c.end( )

*itr++ ~ newValue;

To seE'. the potential problem, suppose the Contai ner c was passed to a routine using callby-constant reference. This means we would expect that no changes would be allowed to c, and the compiler would ensure this by not allowing calls to any of cs mutators. Consider the following code that prints ali st of integers, but also tries to sneak in a change to the 1; st:

void print( canst list<int> & 1st, ostream & out ~ cout

typename Container::iterator itr ~ lst.begin( ); while( itr !" Ist.end( ) ) {

out « *itr « end1;

*itr=O; //Thisisfishy!!! itr++;

If this code were legal, then the const-ness of the 1; st would be completely meaningless, because it would be so easily bypassed. The code is not legal and will not compile. The solution provided by the STL is that every collection contains not only an i terator nested type, but also a const_iterator nestecl type. The main difference between an iterator and a const _ iterator is that operator* for const _ iterator returns a constant reference, and thus *itr for a const_iterator cannot appear on the left~hand side of an assignment statement.



3.4 Implementation of vector

template <type name Container> 2 void printCollection( canst Container & c. ostream & Qut cout)

3 f if( c.empty( ) ) 5 out « "(empty)";

6 else

7 8

9

10

II

12

13

1-1

15

typename Container::const_iterator itr ~ c.begin( ); out « "[ " «*itr++; /1 Print first item

while( itr !o c.end( ) ) out « ". " « *i tr++;

out « II ]" « endl;

Figure 3.6 Printing any container

Further, the compiler will force you to lISC a canst_aerator 10 traverse a constant collection. It docs so hy providing two versions of begin and two versions of end, as follows:

iterator beg; n ( )

canst_iterator begin( ) canst

iterator end( )

canst_aerator end( ) canst

The two versions of begin call he in the same class only because the consl-ncss of a method (i.e., whether il is ,1Il accessor or mutator) is considered to he part orthe sign3LUre. VI/c saw this trick in Section 1.7.2 and we: \vill sec it again in Section 3.4, both in the context of overloading operator[].

If begi n is invoked on a non~constant container, the "mutator" version t hat ret urns an i terator is invoked. However, if beg; n is invoked on a constant container, \vha! is returned is a const_iterator, and the return value may not he assigned to an iterator. If you try to

clo so, a compiler c.rror is generated. Once; tr is a const_; terator, *; tr=O is easily detected as being illegal.

The code in figure 3.6 shows the use of const iterator to print any collection, in brackets, with commas separating the items.

3.4 Implementation of vector ln this section, \VC provide the implementation of a usable vector class template. The vector will be a first-class type, rncaning that unlike the primitive array in C++, the vector can be copied, and the memory it Llsescan be automatically reclaimed (via its destructor). In Section 1.'5.6, we described some important features of C++ primitive arrays:

79



80 Chapter 3 lists, ~ta(KS, ana llueues

The array is simply a poinler variable to a block of mcrnory; tht actual array size must be maintained separately by the programmer.

The block of memory can be allocated via new [] , but then must be freed via de 1 ete [] .

The block of memory cannot he resized (but a new, presumably larger hlock can be obtained and initialized with the old block, and then the old block Gill be freed).

To avoid ambiguities with the library class, we will name our class template Vector. Before examining the (tess than one hundred lines of) Vector code, we outline the main details.

1. The Vector will maintain the primitive array (via a pointer vclriable to the block of allocated memory), the array capacity, anclthe current number of items stored in the Vector.

2. The Vector will implement the Dig-Three to provide deep-copy semantics for the copy constructor and operator"', and will provide a destructor to reclaim the primitive array

3. The Vector will provide a res i ze routine that will change (generally to a larger number) the size of the Vector and a reserve routine that will change (generally to a larger number) the capacity of the Vector. The capacity is changed by ohtaining a new block of memory for the primitive array, copying the old block into the new hlock, and reclaiming the old block.

4. The Vector will provide an implementatiOll of operator[) (as mentioned in Section l .7.2, operator[) is typically implemented with both an aCCessor and mutator version).

5. The Vector will provide basic routines, such as size, empty, clear (which arc typically one-liners), back, pop_back, and also push_back. The push_back routine will call reserve jf the size and capacity are same.

6. The Vector will provide support for the nested types iterator and const_i terator, and associated begin and end methods.

figure 3,7 and figure 3.8 show the Vector class. Like its STLcounterpart, there is limited error checking. Later we will briefly discuss how error checking can be provided.

As shown on lines 90-92, lhe Vector stores the size, capacity, and primitive array as iiS data mernbers. The constructor at lines '5 to 7 allows the user to specify an initial size, which defaults to zero. It then initializes the data members, with the capacity slightly larger than the size, so a few push_backs can be performed without changing the capacity.

The copy constructor, shown at lines 8 lO 9, invokes operator'" to make a copy of an existing Vector. The destructor, shown at lines 10 to 11, reclaims the primitive array. The trickiest routine is operator"', shown at lines 13 to 26. After checking for aliasing at line 15, we free the old array at line 17, and at line 2l create a new array with the same capacity as the Vector being copied. After the sizes are copied, \ve copy the data items one by one. Of course, thiS code occasionally frees and reallocates a new array needlessly, since the original array might have been large enough. The extra logiC would he worth writing in a library Vector class, since the cost of memory allocation is significant.

The resize routine is shovm at lines 28 to :)3. The code simply sets the theSize data member, after possibly expanding the capacity. Expanding capacity is very expensive, so ir the capacity is expanded, it is made twice as large as the size to avoid having to change: the



/ template <type name Object> 2 class Vector .J ( 'I public: 5 explicit Vector{ int initSize - 0 ) 6 : theSi ze{ i nitSi ze ). theCapaci ty { i nitSi ze + SPARE_CAPACITY 7 ( objects - new Object[ theCapacity ); } 8 Vector( canst Vector & rhs ) : objects ( NULL) 9 ( operator-( rhs ); }

10 -Vector( ) II ( delete [ ) objects; 12 1.1 canst Vector & operator::: canst Vector & rhs ) 14 ( .15 /6 /7 JR 19 20 21

22 2:l 24 25 26 27

if( this !- &rhs ) (

delete [ ) objects; theSize - rhs.size( ); theCapacity - rhs.theCapacity;

objects - new Object[ capacity( ) ]; fort int k - 0; k < sizer ); k++ )

objects[ k) - rhs.objects[ k);

return *this;

18 void resize( int newSize ) 29 30 if( newSize > theCapacity 31 reserve( newSize * 2 + 1 ); 32 theSize = newSize; 33 34 35 VOl d reserve ( i nt newCapaci ty 36 37 if( newCapacity < theSize 38 return; 39 40 Object *oldArray ~ objects; 4/ If2 ,/3

44 45 46

'17 4R 49

objects - new Object[ newCapacity]; fort int k - 0; k < theSize; k++ )

objects[ k] - oldArray[ k];

theCapacity - newCapacity;

delete [ ] oldArray;

Figure 3.7 vector class (Pan I of 2)



82 Chapter 3 lists, Stacks, and Queues

50 Object & operator[J ( ; nt index

51 { return objects [ index]; }

52 const Object & operator[] ( lnt index) canst

53 { return objects[ index]; }

54 55 bool empty( ) const

56 { return size{ ) == 0;

57 int size( ) const 58 return theSize; } 59 lnt capacHy( ) const 60 return theCapacity;

61

62 void push~back( const Object & x

(13 {

(l't if( the$ize == theCapacity )

65 reserve( 2 * theCapacity + 1 );

6(; objects[ theSize++ ] = X;

67

68 W void pop_back(

70 { theSize--;

71 72 const Object & back ( ) canst

73 { return objects[ theSize l];}

74 75 typedef Object * iterator; 76 typedef const Object * const iterator;

77

7R iterator begin( ) 79 { return &objects [ 0 ];

80 const_iterator begin( ) const 81 I return &objects[ 0 l; } 82 iterator end ( )

83 I return &objects[ size( ) ]; )

84 const~iterator end( ) const 85 ! return &objects[ size! ) l; ) 86 87 enurn I SPARE CAPACITY = 16 );

88

W) private: ()O int theSize; 91 int theCapacity; 02 Object * objects; 03 );

Figure 3.8 vector class (Part 2 of 2)



3.5 Implementation of list

capacity again unless the size increases dramatically (the +1 is used in case the size is 0). Expanding capacity is done by the reserve romine, shO\vn at lines 35 to 49. It has much of the same logic seen in operator[]: allocation of a new arrayaL line 42, copying of Ihe old contents aL lines 4:3 to 44, and the reclaiming of the old array ;:H line 48. As shown at lines 37 to 38, the reserve routine can also be used to shrink the underlying array, but only if the specified new Gllx1City is at least as large as the size. If it isn't, the reserve request is ignored.

The two versions of operator [J are trivial (and in fact very similar to the implementations of operator[J in the matrix class in Section 1.7.2) and are shown in lines 50 to 53. Error checking is easily added by making sure thaL index is in the range 0 to sizeO-l, inclusive, and throwing an exception if it is nol.

A host of short routines, namely empty, size, capacity, push_back, pop_back, and back, are implemented in lines '5'.5 to 73. At line 66, Wt' see the use of the postfix ++ operator, which uses theSi ze to index the array, and then increases theSi ze. We saw the same idiom when discussing iterators: *i tr++ uses itr to decide which item to view, and then advances i tr. The positioning of the ++ matters: in the prefix ++ operator, *++itr advances itr and then uses the new i tr to c1ecide which item to view, and likewise, object [++theSize] would increment theSize and usc the new value to index the array (which is not vithat we would want). pop_back and back could both benefit from error checks in which an exception is thrown H the size is O.

Finally, at lines 75 to 85 we see the declaration of the iterator and const_iterator nested types and the two beg; n ancltwo end methods. This code makes use of the fact that in C++, a pointer variable has all t he same operators that we expect for an i terator. Pointer variables can be copied and compared, the * operator yields the object being pointed at, and most peculiarly, when ++ is applied to a pointer variable, the pointer variable then points at

the object that would be stored next sequentially: if the pointer is pointing inside an array, incrementing the pointer pOSitions it al 1 he next array element. These semantics for pointers date back to the early 70s with the C programming language, upon which C++ is based. The STL itcrator mechanism was designed in part to mimic pointer operations,

Consequently, at lines 75 and 76, we see typedef statements that state the i terator and const_iterator are simply other names for a pointer variable, and begin and end need to simply return the memory addresses representing the first Jrrayposition and the first invalid array position, respectivrly.

The correspondence hetween iterators and pointers for the vector type means that using a vector instead of the C++ array is likely to carry little overhead. The disadvantage is that, as written, t.he code has no error checks. If the iteralor i tr goes crashing past the end marker, neither ++itl' nor *itr will necessarily signal an error. TI:.1 fix this problem would require that the iterator and const_iterator he actual nested class types rather than simply pointer variables. Using nested class types is much more common and is what 'INC will see in the List class in Section 3.'.5.

3.5 Implementation of list In this section, \\'c provide the implcmcTltation ora usable 1 ist class template. As in the case of the vector class, our lisl class will be named List 10 avoid ambiguities with the lihrary class.

83



64 lnapter J lists, )tacKS, ana l,!ueues

Recall that the li st class will he implemented :\s <l doubly linked list, and that wc will

need to maintain pointers 10 bOlh ends or the list. Doing so allows us to maintaIn constant lime cost per operation, so long as the operation occurs at a known position, The known position can he C'ither end, or <It a position specified hy an ilcralor.

In cOllsiclerillg the design, we will need 10 pnwidc four classes:

1. The List class itseH, which COlll,lins links to both ends, tlK size or the list, and a host of l11et hods.

2. Tbe Node class, which is likely to he a private nested class, A node contains the data

and pointers to the prcvious and next nodes, along with appropriate constructors.

3. The const_iterator cbss, which abstrads the notion of a pnsitinn, and is J public nested class. The const_iterator store; a pointer to ··CUlTClll·' llode, and provides implementation of the hasic itcralOr ope]";)t iOJ]s, all in t he form of overloaded operators

sueh as ", "'"", !'-', (Ind ++.

4. The i terator class, which ahstracts the notion ora position, and is a public ncSlc(1 class. The i terator has the saTlle fUllCliollal it y ~IS const _ i terator, except thal operator* ret urns a reference to the item heing viewed, rather than ,\ cOllstant reference 1O the itcnl. An

imponant technical issue is that an iterator Gin be used in any routine that requires a const_iterator, hut 110t \'il'e-\,ersa. In other words, iterator IS-A const_iterator.

Because the it erator classes store a pointer to I he "'current node,~' and the end marker is a valid position, it !Hakes sense 10 create an extra node at tIl(' end of I hI:' list to !TpreSC!ltlilc end

lllarker.l;urthc\", we Gill create an extra node;1t the front of the list" logically representing the hegillning rllarker. These cxtra lllKks are SOllIctimes known (lS sentinel nodcs~ specifically, the node at the [milt is sometimes known <IS a header node, and the node at the end is sometimes known as a tail node

Thc {lch'<1ntagc of using these extra nodcs is lilat they greatly simplify the coding by rcmovinga host ofspceial c:\Scs. For instance, if wc do not usc a header node, then removing

the first node becomes a special case, because \Vl' must rcsvt thc lists link to the first node during the remove, and also because the rcmoye algorithm in geneml needs 10 access the node prior to the node heing H,tlwwd (;mel without a header node, the firsl node docs not helve a node prior to it). Figure 3.0 sho\\'s a doubly linked list with headn and taiinodt's_ hgurc )_ to shows an cmpty lisl.

Figure ').! I and Figurc '3. -! 2 show lhe outline alld parthll implemclltation of thc list class.

, I~i:l J: I, b" 1~t=rJr~-g~ \ head lail

Figure 3.9 i\ douhly linked list with header ,lnel tail nodes



J.:> Implementation or !lst

Figure 3.10 ;\n empty doubly linkcclli,:>t with header and 1,lilll<xlcs

\Vc can sec at line j the beginning or till' declaration or the private nested Node class.

I~al her 1 han using the c I ass keyword, we usc s t ruct. In (:++, the struct is a relic from the (:

prograrnilling language. j\ struct ill C++ is essentially a class in which the members dcbuh

to puhlic. Recall thaI in (l class, the memhers default!O private. Cleady the struct kcywurcl

is not needed, but you will ol"len see it ami il is (onllllOnly used by programmers to signify

a type thaI (ol1[(lil1s mostly data that arc accessed directly, rather thall through methods. Tn

our case, making the members public in the Node class will not be <0\ prohlem, since the Node class is itself priyate and inaccessible outside of the List class.

At lim' 9 we sec the lx'ginning of the declaration of the puhlic nested const~i terator class, and at line 12 we sec the beginning of the dcchlralion of the public nested iterator class. The unusual synt<\x is inheritance, which is a powerful conslruct not otherwise lIsed

in the book. The inheritallce syntax slates thm iterator has eXi.\ctly the same funnionillny ,IS const ~ i terator, witil possibly some add il iOIlS, and 1 hat i teratar is I ype-compatible with

const_iterator, and Gill be used wherever const._iteratar is neetled. \Vc'll discuss those

details when we sec the aetnal implementations later.

Ijnes 70 to 72 contain [he data Illcmhers for Li st, namely the pointer to the header and

tail nodes. \Ve also keep track or the sizc in a dala lllember, 50 that the size method call be

implclllclllecl in constanl tilllt'. The resl of the List class consists of the constructor, the Big-Three, and a host or

mcthods. IvlallY of the mcthods arc one-liners. begin and end return appropriate itnators;

the caU alline 26 is typical of"the implcnlCnl<ltion, in \vhich Wf return a cunstfuClccl itcrator

(thus the iterator alld const_iterator classes each have a constructor that takes a poillter to a Node as its parameter).

The clear method al lines .1\J to A.1 works by repeatedly reJl1m'ing items Uilliithc List is cmpty Using this str,llcgy allows clear to avoid getling its hands dirty reci,liming \lodes

becausc the node reclamation is now funlleled to pop_front. Tlw methods at lincs ++10 59 all work by cleycrly obtaining and lIsin?, an appropriate ileratoL Rc.callthat the insert methud inserts prior to a position, so push~back inserts prior to the cndmarkcr, as required.

In pop_back, note [hal erase(--endO) creates a tClllporary itcmtor corresponding to the

endmarkcr, retreats the tcmporary ileralor, and lIses that iteratOl" to erase. Similar beha\'ior

occurs ill back. Note also thaI il1t11e case of lhe pop_front and pop_back operations, we again

,woicl dealing with node reclamation.

hgure 3.1 ") shows Ihe Node class, consisting of the stored itcm, pointers to the previous and next Node, ;llld a constructor. All the d<l!;"\ memhers;He public.

Figure J.14 :;huws thc const~ iteratar class, and r~igmc ).1') shows the iterator cl<lss.

As we mcntioned earlier, the syntax <It lille N in Figure ·j.l "5 indicates ;111 advanced fcalulT

II!>



.. 0 Uldpler:J LI~l:'>, )ldlK.~, dllU llueue:,

template <type name Object> 2 class List

3 4 private: 5 struct Node

(, { 1* See Figure 3.13 */ }; 7 8 public:

9 class const_iterator 10 { r See Figure 3.14 */ }; 11 12 class iterator : public const iterator 1.3 { 1* See Figure 3.15 */ }; 14 15 public:

16 List{

17 ( 1* See Figure 3.16 */ 18 List( const List & rhs )

19 ( /* See Figure 3.16 */ 20 -List( )

21 { /* See Figure 3.16 */ 22 const List & operator= ( const List & rhs )

23 { 1* See Figure 3.16 */ } 24 25 iterator beg; n ( )

26 { return iterator( head->next ); }

27 const_iterator begin( ) const 28 { return const_iterator( head->next ); }

29 iterator end( ) 30 { return iterator( tail); } 31 const_iterator end( ) const

.32 return const_iterator( tail ); }

33 34 int size( ) const 35 return theSi ze; 36 bool empty( ) const

37 ( return sizer ) "" 0; }

38 39 void clear( )

40 ( 41 while( !empty( ) )

42 pop_front( );

43

Figure 3.11 List class (Part I or 2)



44 (-1-5

'/()

47

43 49 50 51

52

5J 54 55

56 57 58 59 60

61

62 63 64

65 66 67

68 ()9

70

71 72

73 74 75

76 );

Object & front( ) ( return 'begin( ); )

const Object & front( ) const ( return *begin( ); }

Object & back{ ) ( return *--end( ); }

const Object & back{ ) const ( return '--end( ); )

void push_front{ const Object & x ) ( insert( begin{ ), x ); )

VOl d push_back ( cons t Obj ect & x )

( insert( end{ ), x ); ) void pop_front( )

( erase( begin( ) ); ) void pop_back{ )

( erase( --end( ) ); )

iterator insert( iterator itr, const Object & x ) ( /* See Figure 3.18 */ )

iterator erase( iterator itr ( /* See Figure 3.20 */ )

iterator erase( iterator start, iterator end) ( /* See Figure 3.20 */ )

private: int theSize;

Node *head; Node *tai 1;

void init(

( 1* See Figure 3.16 */ )

Figure 3.12 List class (Part 2 0/2)

..) • ..) "I'P'<;"'<::IIlOllVII VI lDl

known as inheritance and means that iterator 1S-A const iterator. \,yhen the iterator class is written this way, it inherits all the daw and rncthods from const_aerator. It may then add new data, add new methods, and override (i.e., redefine) existing methods. In the most general scenario there is significant syntactical baggage (often resulting in the keyword vi rtual appearing in the code).

HO\vevcr, in our case, we can avoid much of the syntactical baggage because we. arc n01

adding new data, nor are we intending to change the behavior of an existing method. \Ve are, however, adding some new methods in the i terator class (with very similar signalUlTs



'-" .... 1"' ..... J ......... ", ............ " ... , .......... "" ..................

s t ruct Node 2

3 4 5 6

7 8 9 };

Object data; Node *prev; Node *next;

Node( const Object & d = Object( ), Node *p NULL, Node *n : datal d ), prev( p ), next( n ) { }

Figure 3.13 Nested Node class for List class

NULL)

10 the existing methods in the const_iterator class). As a result, we can avoid using vi rtual, Even so, there arc quite a few syntax tricks in const_iterator.

Allincs 28 and 29, const_iterator stores as its single data memher a pointer to the "currcm" node. Normally, this would be private, but if it were private, then iterator would not have access to it. Marking members of const_iterator as protected allows the classes that inherit from const_ iterator 10 have access to these members, but does not allow other classes to have access.

At lines 34 and 3'5 we sec the constructor for const_iterator that was used in the List class implementation of begin and end. 'vVe don't want all classes to sec Ihis constructor (iterators are not supposed to he visibly constructed from pointer variables), so it can't be public, but we also want the iterator class to he ahle to sec it, so logically this constructor is made protecled. liowever, this doesn't give List access to the constructor. The solution is the friend declaration at line 37, which grants the list class access to const_iterators non-public members.

The public methods in const_iterator all use operator overloading. operator"'''', operator!"', and operator* arc straightforward. At lines 10 to 21 we see the implementation of operator++. RecaJl that the prefix and postfix versions of operator++ arc completely different in semantics (and precedence), so we need 10 write separate routines for each form, They have the same name, so they must have different signatures to be distinguished. C++ requires that we give them different signatures by specifying an empty parameter list for the prefix form and a single (anonymous) int parameter for the postfix form, Then ++itr calls the zero-parameter operator++; and itr++ calls the one-parameter operator++. The int parameter is never used; it is present only to give a different signature. The implementation suggests that, in many cases where there is a choice between using the prefix or postfix operator++, the prefix form will he faster than tbe postfix form,

In the iterator class, the protected constructor at line 64 uses an initializer list to

initialize the inherited current node. 'vVe do not have to reimplcment operator"'''' and operator!'" because those are inherited unchanged. We cIo provide a new pair of operator++ implementations (because of the changed return type) thilt hicIe the originals in the const_ iterator, and also an accessor/mutator pair for operator*. The accessor operator*, shown at lines 47 to 48, simply uses the same irnplementation as in const _ i terator. The accessor must be explicitly implemented in i terator because otherwise the original implementation is hidden by the newly added mutalor version.



1 class canst iterator 2

.3 public: 4

6

7

8

9

10

11 12 Ll 14

15

16

17

18

19 20

21

22

canst iterator( current( NULL) { I

const Object & operator* ( ) const { return ret ri eve ( ); }

const_iterator & operator++ ( ) {

current = current->next: return *this;

const lterator operator++ { lnt

const_iterator old ~ *this; ++( *this ); return old;

J.J ""P''''''''''''WUV11 VI !I-:lt

23

24

2.5 26 27

bool operator== ( canst const_iterator & rhs const { return current ~~ rhs.current; }

bool operator!= ( const const_iterator & rhs const { return !( *this ~~ rhs ); }

28 protected: 29 Node *current; 30 31 Object & retrieve( ) const 32 { return current->data; .n 34 35

.16

const_iterator( Node *p) current{ p ) { I

37 friend class List<Object>; .18 I;

Figure 3.14 Nested const_iterator class for List class



39 class iterator public const iterator

'iO

'tl public: 42 iterator( 43 I } 11

Object & operator* ( ) { return retrieve( ); }

canst Object & operator* ( ) const

45

46

17 18 49

{ return const_iterator::operator*( ); }

50 51

52 5J

51 55

56 57 58 59

60 (ii

62

iterator & operator++ ( )

I current ~ current->next; return *thi s;

iterator operator++ ( int

iterator old ~ *this; ++( *this ); return old;

63 protected:

65

66

iterator( Node *p} const_iterator{ p ) I }

67 friend class List<Object>; 68 };

Figure 3.15 Nested iterator class for List class

Figure 3.] 6 sho\vs the construdor Jnd Big-Three. Because the zero~par,Hncter constructor and copy constructor must both allocate the heaeler anellail nodes, we provide a private init routine. init creates an empty List. The destructor reclaims Ihe header and tail nodes; all the other nodes are reclaimed when the destrucLOr invokes clear. Similarly, operator= is implemented hy invoking puhlic methods, rather than attempting low-level pointtT manipulations.

figure 3.17 illustr,Hcs how a new node containing x is spliced in between a node pointed at by p and p.prev. The assignment to the node pOinters carl be described as follows:

Node *newNode = new Node{ x, p->prev, p ); p->prev->next '" newNode; p->prev = newNode;

II Steps I and 2

II Step 3

I I Step 4



2

3 4 .5 6

7 8 9

10

List( )

{ init (

-List( )

{

clear(

delete

delete

) ; )

) ; head;

tai 1;

11 List( const List & rhs )

12 { 13 init( );

14 *this '" rhs;

15

16

17 const Li st & operator'" ( const Li st & rhs )

18

19 if ( thi s == &rhs )

20 return *this;

21 clear( );

).:> Imprememaiion OJ IISI

22 23

for( const_iterator itr rhs.begin(); itr l'" rhs.end( ); ++itr)

push_back( *;tr );

24 return *thi s;

25

26 27 void init( )

28 29 the$ize '" 0;

30 head'" new Node;

31 tai 1 '" new Node;

32 head->next tail;

:33 ta i l->prev '" head;

34

Figure 3.16 Constructor, Big-Three, and private init routine for list class

Steps 3 and 4 can be combined, yielding only two lines:

Node *newNode '" new Node( x, p->prev, p );

p->prev '" p->prev->next '" newNode;

II Steps I and 2

II Steps 3 and 4

But then these two lines can also be combined, yielding:

p->prev '" p->prev->next ~ new Node( x, p->prev. p );

This makes the insert routine in Figure 3.18 short.

,., Bismillah hiir Rehman nir Raheem-----------------------------Assalat o Wasalam o Alika Ya RasoolALLAH


3

J p

--- -~

Figure 3.17 lnsertion in a doubly linkccllisl hy gelling new node and thell changing pointcrs in the order indicated

II Insert x before itr. 2 iteratol' insert( iterator itr, canst Object & x )

3

'f Node *p '" i tr.current;

5 (,

7

theSize++; return ; terator( p->prev p->prev->next

Figure 3.18 insert routine for List class

p

new Node ( x. p->prev. p ) );

Figure 3.19 Removing node spcci{]cd by p from a doubly linked list

Figure 3.19 shows the logic or removing a node. If p points In the node being rCllloved, only l\\lO poinlers change' before the node can be reclaimed:

p->prev->next p->next: p->next->prev p->prev; delete p;

figure 3.20 shows a pair of erase routines. The first version of erase contains the three lines of code shown above, and also code to return an i terator rcprescnting the item after the erased clement. Like insert, erase must update theSize. The second vCl"sion of erase simply lIses an iterator (0 call the first V(TsiOll of erase. Noll' lhal we Gll11101 simply usc itr++ in Ihe for loop al Iiol' 16 and ignore the return vallll' of erase at line 17. Tht' valuc of itr is sUlic immediately after the calllo erase, which is why erase returns an iterator.

ln examining the code, \VC can sec a host of errors that can occur, and for which no checks arc provided. l'"or instance, itcrators passed {o erase and insert can he uninilialized,



II Erase item at itr. 2 iterator erase( Herator itr

:J Node *p '" itr.current;

j iterator retVal ( p->next );

I>

7 8 9

10

II

12 /j

p->prev->next p->next; p->next->prev p->prev; delete p; theSize--;

return retVa 1;

)/1 iterator erase( iterator start, iterato\' end

Lj

1(,

17

18

forC iterator itr = from; itr !~ to; itr = erase( itr );

19 return to;

20

Figure 3.20 erase routines for Li st class

3.5 Implementation of list

or for the wrong lisl' ltcrators can have ++ or * applied to them whcn they arc already at the end marker or <Ire uninitializcd.

An uninilializcd itcrator will have current pointing at NULL, so that condition is easily

tested. The cndmarkcr's next pointer points a1 NULL, so lcsling for ++ or * on an end marker

condition is also easy, However, in order to determine if an ileraLor passed to erase or insert is an itcralor for the correct list, the iterator IHust store an additional data member representing a pointer to the List from which il was constructed.

'vVe will sketch t he basic idea, and leave I he details <is an exercise. "In the cons t _ Hera tor class, we add a pointer to tIlt' List, and modify the protecled construcwr to take the List as a paramel.er. Vve can also add methods tlut throw an exception if certain assertions aren't

mel. The I"t.'viscd protected section looks something like the code in Figure 3.21. Then all calls to iterator and const_Herator constructors that formerly look one parameter now lake two, as in the begin method for list:

const_iterator begin( ) const {

const iterator itr( *this, head ); return HHr;

Then insert C;Ill he revised 10 look somcthing like thc code in Figure :").22. Vic leave Ihe details of these modifications ;,\S an excrcise_

93



" .. Lnapler j LiSts, )IaCKS, ana t..!ueues

protected: 2 canst List<Object> *theList;

J Node *current; 4 5 const_iterator( const List<Object> & 1st, Node *p ) I> : theU5t( &15t ), current( p )

7 H

9

10 void assertIsValid( ) canst 11 12 13 14

if( theU5t ~~ NULL II current ~~ NULL II current throw IteratorOutOfBounds£xception( );

theLi 5 t->head )

Figure 3.21 Revised protected section of canst_iterator that incorporates ability to

perform additional error checks

I / / Insert x before itr.

2 iterator insert( iterator itr, canst Object & x ) 3 { 4 itr.assertIsValid( );

5 if( itr. theList ! .. this

6 throw IteratorMi smatchExcept; on ( );

7 H 9

10

11

12

Node *p ~ itr.current; theSize++; return iterator( *this.

p->prev p->prev->next new Node( x, p->prev. p ) );

Figure 3.22 list insert with additional crror checks

3.6 The Stack ADT 3.6.1 Stack Model A stack is a list vvith the restriction that insertions and deletions can be performed in only one position, namely, the end of the list, called the top. The fundamental operations on a stack are push, which is equivalent to an insert, and pop, which deletes the most recently inserted clement. The most recently inserted element can be examined prior to performing a pop by usc of the top routine. A pop or top on an empty slack is generany considered an error in the stack ADT. On the other hand, running out of space when performing a push is an implementation limit but not an ADT error.



pop

top Slack

push I-EC::'C---'-

j.b Ine::,tacKAUI

Figure 3.21 Slack mode!: input to a stack is by push, outpUl is by pop and top

top 2

4 •

j

3 .

6

Figure 3.24 Stack model: only the top element is accessible

Stacks arc_ sometimes known as uro (bst in, flIst out) lislS. The model depicted in Figure 3.23 signifies only that pushes arc input operations and pops and tops arc output. The usual operations to make empty slacks and test for emptiness arc part of the repertoire, but essentially alltllat you Gill do 10 a stack is push and pop.

Figure 3.24 shmvs an abstracl slack after several operations. The gcncnll model is Ihat there is some clement that is at the top of the stack, and it is I-he only element that is visible.

3.6.2 Implementation of Stacks Since a slack is a list, any list irnplcmentation will do. Clearly 1 i st and vector support slack operations; 99(X, of the time they are the most reasonahle choice. Occasionally it can be

faster to design :-1 spccial~purpose implemenlation. Because stack operations are constanllime operations, this is unlikely to yield any disccrnablc improvement except under very unique circumstances.

For lhese special limes, we will giw' !WO popular stack implementations. One Llses a linked structure and lhe olher Llses an array, and both simplify lbe logic in vector and 1 ist, so we do not provide code.

95



Linked List Implementation of Stacks The fIrst implementation of a stack uses a singly linked list. VYe perform a push by inscrLing at the front of the list. Vic perform a pop by deleting the clement at the fronl of the list. A top operation merely examines the clement at the front of the list, returning its value. Sometimes the pop and top operations arc combined into one.

Array Implementation of Stacks An alternative implementation avoids links and is probahly the more popular solution. It uses the back, push_back, and pop_back implementation from vector, so the implementation is trivial. Associated with each stack is theArray and topOfStack, which is ~ I for an empty stack (this is how an empty stack is initialized). To push some element x onto the stack, we increment topOfStack and then set theArray[topOfStack] "" x. To pop, we set the return value to theArray[topOfStack] and then decrement topOfStack.

Notice that Ihese operations arc performed in nOI only constant time, but very fast constant time. On some machines, pushes and pops (of integers) can be written in one machine instruction, operating on a register with auto-increment and aU1.o-decrement addressing. The fact that most modern machines have stack operations as part of the instruction sct enforces the idea that the stack is probably the most fundamental data SlruClure in computer science, after the array

3.6.3 Applications It should come as no surprise that if we restrict the operations allowed on a list, those operations can he performed very quickly. The hig surprise, however, is Ihat the small number of operations left arc so powerful and important. We give three of the many applications of stacks. The third application gives a deep insight into how programs afC organized.

Balancing Symbols Compiler~ check your programs for syntax errors, hut frequently a lack of one symbol (such as a missing brace or comment staner) will cause the compiler to spill nut a hundred lines of diagnostics without identifying the real error.

A useful tool in this situation is a program that checks 'whether everything is balanced. Thus, every right brace, bracket, and parenthesis must correspondlO its left counterpart. The sequence [()] is legal, hut [(]) is wrong. Obviously, it is not wonhwhile writing a huge program for this, but it turns out that jt is easy to check these things. For simplicity, we will just check for balancing of parent he~es, brackets, and braces and ignore any other character that appears.

The simple algorithm uses a stack and is as follows:

Make an empty stack. Read characters until end of file. If the character is an opening symbol, push it onto the stade If it is ~\ clo~ing symbol, then jf the stack is empty report an error. Otherwise, pop the stade If the symbol popped is not the corresponding opening symbol, then report an error. At end of file, if the stack is not empty report an error.



.).0 lilt: .llaLI\ /,,\UI

You should be able to convince yourself that this algorithm works. It is clearly linear and actually makes only one pass through the input. It is thus on-line and quite fast. Extra work can be done to attempt to decide what to do when an error is reportccl~such as identifying the likely cause.

Postfix Expressions Suppose we have a pocket calculator and would like to compute the cost of a shopping trip. To do so, we add a list of numbers and multiply the result by L06; this computes the purchase price of some items with local sales tax added. If the items arc 4.99, 5.99, and 6,<)9, then a natural way to enter this would be the sequence

4.99 + 5.99 + 6.99 * 1.06 =

Depending on the calculator, this produces either the intended answer, 19.05, or the scientific answer, ltL")9. Ivlost simple four-function calculators will give the hrst answer, but mallY advanced calculators know that multiplication has higher precedence than addition.

On the other hand, some items arc taxable and some arc not, so if only the first and last ilems were actually taxable, then the sequence

4.99 * 1.06 + 5.99 + 6.99 * 1.06 = would give the correcl answer (18.69) on a scientific calculator and the wrong answer (19,37) on a simple calculator. A scientific calculator generally comes with parentheses, so we can always get the right answer by parenthesizing, hut with a simple calculator we need to remember intermediate results.

A typical evaluation sequence for this example might be to multiply 4.99 and 1.06, saving this answer as AI' 'vVe then add 5.99 and A j , saving the result in AI' Vie multiply 6.99 and 1.06, saving the answer in A2 , and finish by adding Al and A2, leaving the final answer in Al . We can write this sequence of operations as follows:

4.99 1.06 * 5.99 + 6.99 1.06 * + This notation is known as postfix or reverse Polish notation and is evaluated exactly as we have described above. The easiest way to do this is to usc a stack. When a number is seen, it is pushed onto the stack; when an operator is seen, the operator is applied to the

two numbers (symbols) that arc popped from the stack, and the result is pushed onto the stack. For instance, the postfix expression

6523+8*+3+*

is evaluated as follows: The first four symbols arc placed on the stacie The resulting stack is

topOfStack -'> 3 2 5 6

'''' Bismillah hiir Rehman nir Raheem-----------------------------Assalat o Wasalam o Alika Ya RasoolALLAH


Next a '+' is read, so J and 2 arc popped from the stack ancltheir sum, 5, is pushed.

Next 8 is pushed.

topOfStack -> 5 5 6

topOfStack -> 8 5 5 6

Now a '*' is seen, so 8 and 5 are popped and') * 8 = 40 is pushed.

topOCStack -> 40 5 6

Nexl a '+' is seen, so 40 and') are popped and 5 + 40 = 45 is pushed.

NO\v, "3 is pushed.

topOfStack -> 45 6

topOfStack -> 3 45 6

Next '+' pops.J and 45 and pushes 4'5 +.) = 48.



3.6 The Stack ADT

lopOfStack -} 48 (,

Finally, a '*' is seen and 48 and 6 arc popped; the result) 6 * 48 = 288, is pushed.

topOfSlack -} 288

The time to evaluate a postfix expression is O(N), because processing each element in the input consists of stack operations and thus takes constant time. The algorithm to do so is very simple. Notice that when all expression is given in postfix notation, there is no need to know any precedence rules; Ihis is an obviolls advantage.

Infix to Postfix Conversion Not only call a Slack he used 10 ('valuate a postfix expression, hut \ve can also usc a Slack

to convert an expression in standard form (otherwise known as infix) into postfix. \Vc will concentrate on a small version of the general prohlem hy allowing only the operators +, *, (, ), and insisting on the usual precedence rules. 'vVe will further assume that lhe expression is legaL Suppose we want to convert the infiX expression

a + b * c + ( d * e + f ) * 9

into postfix. 1\ correct answer is abc * + d e * f + 9 * +.

\rVhcn an operand is read, it is immediately placed onto the output. OperalOfs arc 110t

immediately output, so they mllst be saved somewhere. The correct thing to do is to place operators that have been seell, but not placed on the output, onto the stack. \Ve will also Slack left parentheses when the), arc encountered. 'vVe start with an initially empty stack.

If \VC see a right parenthesis, tben we pop the stack, writing symbols until we encounter a (corresponding) lert parenthesis, which is popped but not output.

Ifwe see any other symbol (+, *,0, then we pop entries from the Slack until we fInd an entry of lower priority. One exception is that we ne\'('r remove a ( from the stack except when processing a). For the purposes of Ihis operation, + has lowest priority and (highest. When the popping is done, we push the operator onto the stack

finally, if we read the end of input, we pop the stack until it is empty, writing symbols onto the output.

The idea of this algorithm is that whcn an npcrator is seen, it is placed on the stacie The Slack represents pcnding operators. /-IUWeY{:T, some of the operators on the stack that have high precedence arc now known [0 he completed, and should be popped, as they will no longer be pending. Thus prior to placing the operator on the Slack, operators that arc on

99



.uu I...IIO!,lCI J Llll:), .. .nOLI'I.), (lJIU '-<UCUC)

the sLack, and which arc to be completed prior to the current Opn;ltOf, arc popped. This is illustrated in the following wble:

Expression

a*b-c+d

ajb+c*d

a-b*c/d

a-b*c+d

Stack vVhcn Third Operator Is Processed

+

*

Action

- is completed; + is pushed

Nothing is completed; * is pushed

* is completed; I is pushed

'i<- and _ afC completed; + is pushed

Parent hcscs simply add an additional complication .. 'vVc can vin\' a left parenthesis as a highprecedence opcrmorwhcn it is an input symbol (so l haL pending operators remain pending), anel a low-precedence operator when it is on the Slack (so that it is not accidentally removed by an operator). Right parentheses arc treated as the special casf.

To see how this algorithm performs, we will convert the long infix expression above iIllO its postfix form, First, the symbol a is read, so it is passed through to the output. Then + is read and pushed onlO t he stack. Next b is read and pilssed through to 1 he out put. The state of affairs at this juncture is as follows:

I a b Stack Output

Next a * is read. The top eIllry on the operator stack has lower precedence than *, so nothing is output and * is put on the stade Next, c is read and output. Thus far, we have

Stack L[a~b~c~~~_ ~

Output

The next symbol is J. +. Checking the stack, we find that we \vill pop a * and place it on the output; pop the other +, which is not of lower but equal priority, on the stack; and then push the +.

Stack

The next symbol read is a Then d 1S read and output.

labc*+ Output

{, which, being of highest prcu:dence, is placed on the stacie



3.6 The Stack ADT

labc*+cI Stack Output

\Vc continue by reading <l *. Since open parentheses do not get removed except when a closed parenthc~js is being processed, there is no output. Next, e is read and outpU!.

labc*+cle Stack Output

The next symbol read is a +. \Vc pop Jnd output * and then push +, Then we read and output f.

~ tij Stack

[il:bc*+de*f Output

Now \VC read a ), so the stack is emptied back to the (. vVc outpU! a +.

labc*+cle*r+ Stack Output

\IVC read a * next; it is pushed onto the Slack. Then 9 is read and output.

Stack

labc*+de*F+g Output

The input is now empty, so we pop and output symbols from the stack until il is empty.

u Stack

la be * + cI e * f + 9 * +1 Output

As beforc, this conversion requires only O(N) lime and \\'orks in one pass through the input. We can add subtraction and division to this repertoire by assigning subtraction and addition equal priority and multiplicatiOn and division equal priority. A subtle point is that the expression a - b - c will be converted to a b - c - and not abc - -. Our algorithm does

101



IU,t

the right thing, bcc<\usc these operators associate fromlch 10 right. This is not necessarily the

case in general, since expol1ellliation associates right 10 left: 22\ = 28 = 2'56, not 4 3 = 64. 'vVc leavc as an exercise the problem or adding Cxpollcl1lial ion to the repertOire of operators.

Function Calls The algorithm to check balanced symbols suggests a \vay to implement function calls in compiled procedural and object-oriented languages. The problc:m here is that when a call is made 10 a new function, all the variables local 10 the calling routine need to be saved hy the system, since otherwise the new fUllction will overwrite the memory used by the calling routine's variables. Furthcnnon:, the current location in the routine lllust be saved so that the new function knows where to go after it is done. The variables have gencr<1.lly been assigncd by the compiler to machine registers, and there arc cenain to be conflicts (usually all functions get some variahles aSSigned to register #.1), especially if recursion is involved. The reason that this problem is similar to lxdancing symbols is that a function cal! and function return afC essentially the same as an open parenthesis and closed parenthcsis, so the same ideas should work.

\Vhen there is a function call, all the important information that needs to be s<lVcd, such as register values (corresponding to variable namcs) and the retLIrn address (\\'hich can be obtained from the program counter, which is typically in a register), is saved "on a piece of paper" in an abstract way and put at the top of a pile. Then the control is transferred to the new function, which is free to replace the registers with its values. If it makes other function calls, it follows the same procedure. \Vhcn the function wants to return, it looks at the "paper" at the top of the pile and restores all the registers. Ii then makes the return jump.

Clearly, all of this work can be done using a stack, and that is exactly what happens in virtually every programming language thai impicmel1ls recursioll. The information saved is called either an activation record or stack frame. TypicaHy, a slight adjustment. is made: The current environment is represented at the top of the stacie Thus, a return gives the previous environment (without copying). The stack in a real computer frequently grows from the high end of your memory panition downward, and on many systems thcre is no checking for overllow. There is always the possibility thaI you will run out of stack space by having too mallY simultaneously active functions. Needless to say, running out of stack space is always a f~lt<1l error.

In languages and systems that do not check for stack overflow, programs crash without an explicit explanation.

1n normal events, you should not run out of stack space; doing so is usually an indication of runaway recursion (forgetting a hase case). Un the other hand, some perfectly legal and seemingly innocuous programs (an cause you to run out of stack space. The routine in l:igurc 3.25, which prints out a container, is perfectly legal and actually correct. It properly handles the base case of an empty container, and the recursion is frne. This program can be proven correct. Unfortunately, if the conuliner contains 20,000 clemel1ls to print, there will he a stack of 20,000 activation records representing the nested calls of line 11. Activat.ion records arc typically large because of all the information they contain, so this program is likely to flln out of stack space. (If 20,000 clements arc not enough to make the program crash, replace the number with a larger one.)

This program is an example of an extremely bad usc of recursion known as tail recursion. Tail recursion refers to a recursive call al the last line. Tail recursion can be



1 /** 2 * Print container from start up to but not including end.

3 */ 4 template <type name Iterator> 5 void print( Iterator start, Iterator end, ostream & out cout) (,

7 i f( start == end)

8 return; 9

10 out « *start++ « endl; / / Pri nt and advance start

11 print( start, end, out ); 12

Figure 3.25 A bad USf:'_ of recursion'. printing a container

1 /** 2 * Print container from start up to but not including end .

. 1 */ 4 template <typename Iterator> 5 void print( Iterator start, Iterator end, ostream & out cout) (,

7 while( true)

8 ( 9 if( start == end)

10 return; 11 12 1.1 14

out « *start++ « endl; II Print and advance start

3.6 The Stack ADT

Figure 3.26 Printing a container without recursion; a compiler might do this (you should not)

mechanically eliminated by enclosing the body in i:1 whi 1 e loop and replacing the fecursive call with one assignment per Junction argument. This simulates the recursive call because nothing needs to be saved; after the recursive call finishes, there is really no need to know the saved values. Because of this, we can just go to the top of the function with the values that \'Vould have been used in a recursive call. The function in Figure 3.26 shows the mechanically improved version generated by this algorithm. Removal of tall recursion is so simple that some compilers do it automatically. Even so, it is best not to find out that YOUfS docs not.

Recnfsion can always be completely removed (compilers doso in converting to assembly language), but doing so can be quite tedious. The genenll stmtegy requires using a stack and is worthwhile only if you can manage to put the bare minimum on the stack. We will not dwell on this further, except 10 point ouL that although non recursive programs arc certainly



104 Chapter 3 Lists, Stacks, and Queues

generally faster than equivalent recursive programs, the speed advantage rarely justifies the lack of darily that results from removing the recursion.

3.7 The Queue ADT Like stacks, queues arc lists. \tVith a queue, however, insenion is done at one end, whereas deletion is performed at the other end.

3.7.1 Queue Model The basic operations on a queue arc enqueue, which inserts an dement at the end of the list (called the rear), and dequeue, which deletes (and returns) the clement at the start of the list (known as the front). Figure 3.27 shows the abstract model of a queue.

3.7.2 Array Implementation of Queues As with stacks, any list implementation is legal for queues. Lil«(' stacks, both the linked list and array implementations give fast 0(1) running times for every operation, The linked list implementation is straightforward and left as an exercise. We will now discuss an array implementation of qucues.

for each queue data structure, we keep an array, theArray, and the positions front and baCk, which represent the ends of the queue. We also keep track of the number of elements that are actually in the queue, currentS; ze. The f()llowing table shows a queue in somc intermediate state.

l' l' front back

The operations should be clear. To enqueue an element x, we increment currentS; ze

and back, then set theArray[back] := x. To dequeue an element, we sct the return value to

dequeue Queue

enqueue

Figure 3.27 Model 01 a queue



3.7 The Queue ADT

theArray[front], decrement currentSize, ancI then increment front. Other strategies arc possible (this is discussed later). \Ve will comment on checking for e!Tors presently.

There is one potential problem with this implementation. After]O enqueues, the queue appears to be full, since back is now at the last array index, and the next enqueue would be in a nonexistent position. However, there might only he a few clements in the queue, because several clements may have already been dcqucuccL Queues, like stacks, frequently stay small even in the presence of a lot of operations.

The simple solution is that whenever front or back gets Lo the end of the array, it

is wrapped around to the beginning. The following tables show the queue during some operations. This is known as a circular array implementation.

J nitial State

I I I I I I I I 2 4

t t front back

After enqueue( 1)

After enqueue (3)

t t back front

After dequeue, Which Returns 2

t t back front

After de que u e, Which Returns 4

105



IUO UldjJlel.) U~L::', .)ldUI.), dllU I..lUeUe)

After dequeue, Which Returns I

r back front

U'J After dequeue, Which Returns 3

and Makes the Queue Empty

r r hack front

The extra code required to implement the wraparound is minimal (although it pml)ahly doubles the running lime). lfinncll1cntinl-', either back or front causes jl10 go past the array, the value is reset to the first posit ion in the array.

Some programmers LIse different ways of representing the front and back of a queue. For instance, some do not usc an en! ry to keep t rack of the size, because I hey rely on the base case that when the queue is empty, back = front-I. The size is cornpuled implicitly by comparing back and front. This is a very Iricky way to go, hccause there arc SOme special cases, so be very careful if you need 10 modify code written this v-my. If the currentSize is nOlmaintaincd as an cxplicit data member, thcn tht' queue is full when there arc theArray.length()-l clenlcnts, since only theArray.length{) different sizcs can be differentiated, and one of t1wse is O. Pick any style you like and make sure that all your routines arc consistent. Since there arc a few options for implementation, it is prohahly worth a commcnt or t \v() in the code, if you don't usc the currentSi ze data member.

in applications where you are sure that the number of enqueues is not larger than the capacity of I"he queuc, the wraparound is not necessary As with slacks, dequeues arc rarely performed unless the calling routines arc certain that the queue is not empty. Thus error checks are frequrllliy skipped for this operation, except in critical cocle. This is generally nOljustifialJIc, hecause the time saYings that you arc likely to achicye arc minimal.

3.7.3 Applications of Queues Therc an .. many algorithms that usc queucs to give cfficient running I illlcs. Sevcral of these arc found in gmph theory, and we will discuss thclll in Chapter 9. for now, we will givc some simple examples of queue usage.

\,Yhen jobs arc submitted to a printer, they arc arranged in order of arrival. Thus, essentially, johs sent 10 a line primer arc placed on a queue.]

I \Ve say cs~ellti(/I/y lwcausc jobs can he killed rhi;; amounts to a deletion from ilw middle of the queue,

which is a violation of the "lricl definitioJl.



.JUlllllldlY

Virtually every real~life line is (supposed to be) a queue. For instance, lines at ticket counters are queues, because service is first~come first-served.

Another example concerns computer networks. There are many network setups of personal computers in which the disk is attached to one machine, known as the file server. Users on other machines are given access to files on a first-come first-served basjs, so the data structure is a queue.

Further examples include the following:

Calls to large companies are generally placed on a queue when all operators are busy.

In large universities, where resources are limited, students must sign a waiting list if all terminals are occupied. The student who has been at a terminal the longest is forced off first, and the student who has been waiting the longest is the next user to be allowed on.

A whole branch of mathematics, known as queuing theory, deals with computing, probabilistically, how long Llsers expect to wait on a line, how long the line gets, and other such questions. The answer depends on how frequently users arrive to the line and how long it takes to process a user once the user is served. Both of these parameters afe given as probability distribution functions. In simple cases, an answer can be computed analytically. An example of an easy case would be a phone line with one operator. If the opermor is busy, callers are placed on a waiting line (up to some maximum limit). This problem is important for businesses, because studies have shown that people are quick to hang up the phone.

If there are k operators, then this problem is much more difficult to solve. Problems I hat are difficult to solve analytically are often solved by a simulation. In our case, we would need to use a queue to perform the simulation. If h is large, we also need other data slructufes to do this efficiently. We shall see how to do this simulation in Chapter 6. We could then run the simulation for several values of h and choose the minimum h that gives a reasonable waiting time.

Additional uses for queues abound, and as with stacks, it is staggering that such a simple data structure can be so important.

Summary

This chapter describes the concept of ADTS and illustrates the concept with three of the most common abstract data types. The primary objective is to separate the implementation of the abstract data types from their function. The program must know what the operations do, but it is actually better of[ not knowing how it is done.

List.s, stacks, and queues are perhaps the three fundamental data struct.ures in all of computer science, and their use is documented through a host of examples. In parLicular, we saw how stacks are used to keep track of function calls anel how recursion is actually implemented. This is important to understand, not just because it makes procedural languages possible, hut because knOWing how recursion is implemented removes a good deal of the mystery that. surrounds its use. Although recursion js very powerful, it is not an entirely free operation; misuse and abuse of recursion can result in programs crashing.

lUI



Exercises

3.1 You arc given a list, L, and another lis!, P, containing integers sorted ill ascending

order. The operation printLots(L.P) will print the clements in L thal are in po.<;i11011:-;

specifled hy P. For inslance, if p = !, ~), 4,6, the clements ill positions I, ),4, and 6

in L ,-He printed. \\1ritc the procedure printLots{L,PJ. YUtl may usc only the public

STL container operations. \A,1hat is the running time uf your procedure?

3.2 Swap two adjacent clements by adjusting only the links (and 1101 tbe data) using: a. Singly linked lists. b. Doubly linked lists.

3.3 Implement the STL find routine thaL rctums the iterator ('ol1l<\ining the first

occurrence of x in the range that begins al start :J.nd extends up to but not including

end. If x is not found, end is returned. This is a non-class (global function) with

signal nre:

template <type name Iterator, typename Object> iterator find( Iterator start. Iterator end, const Object & x )

3.4 Given t\:vo sorted lists, L1 and L-2> write a procedun:- to compute L] n Ll using ollly the basic list operations.

3.5 Civl'n t\\lO sorted lists, LJ and L2 , write a procedure to compute L] U Ll using only

the bask list operations.

3.6 The Josephus problem is the following galJlc: N people, numhered I to N, are silting

ill" circle. Starling at person!, a hot potato is passed. Arter Iv! passes, the person

holding the hot potato is eliminated, the circle closes ranks, and the game continues

with the person who was Sluing aher Ihe eliminated person picking up the hot

potato. The last remaining person wins. Thus, if Iv1 = 0 and N = 5, players are

eliminated in order, and player 5 wins. If M = 1 and N = 5, the order of elimin;CHiol1

IS 2, 4, 1,5. a. \\"rlte a program to solve the Josephus problem for gcnl'l"J.1 values of ivl ,mel N.

Try to make your program as efficient as possible. 1\'1ake sure you dispose of cells. b. vVhal is tbe running timc of your program?

c If lvl = I, what is the running time of your program? Jlow is Ihe actual speed

affected hy the delete routine for large values of N (N > 100,0(0)?

3.7 lvlodd)! tbe Vector class to add hounds checks Ii)!" indcxing.

3.8 Add insert and erase to the Vector cbss.

3.9 According to the C++ standard, for the vector, a calltu push_back, pop_back, insert, or erase invalidates (potenlially makes stale) all ilcrato!"s viewing the vector. vVhy?

3.10 Modify the Vector class to provide stringent iterator checking by making iterators class Iypes, rather than poinler variables. The hardest part is dealing with stale iterators, as described in Exercise .1.9.

3.11 Assllme that asingly linked list is implclllCllted with ,\ header node, but no tail node,

and that il maintains only a pointer 10 the header node. \\lrite a class that includes

methods to



i.l.. relUrn the size of the linked list b. print the linked list c test if a value x is contained in the linked list d. "dd a value x if it is llot already contained inlhe lillke.d list e. remove a value x if it is cOlltained in the linked list

3.12 Repeat Exercise 3.11, maintaining the singly linked jist in sorted order.

3."13 Add support for operator-- 10 the List itcrator classes.

LAI:I\.I.>C')

3.14 Looking ahead in an SIL itcrawr requires an application of operatot'++, which Il1

!Urn advances the iterator. In sonlC cases looking at the next item in the list, withnut advancing to iI, may be preferable. \Vrile the memher fUllet ion with the declaration

const_iterator operator+( int k ) const;

10 facilitate this in a general «lSe. The hi nary operator+ returns an ilerator that corresponds to k positions ahc,ld of current.

3.15 Add the spl i ce operation to tht' list cbss. The method declaration:

void splice( iterator position, list<T>& 1st );

ITmO\'es all the items from 1st, pbcing them prior to position in list *this. 1st and *this must be different lists. Your routine must run in constant time.

3.16 Add reverse ilerators to the STI.l i st cl<lsS implementation. Define reverse _ i terator and cons t _reverse _ i terator. Add the methods rbeg i n and rend to return appropriate reverse i[erators representing the position prior to the endmarker and the position thai is the header node. Reverse itcrators internally reverse the meaning or the ++

and -- operators. You should he ahle to print a lisll in reverse by using the code

List<Object>::reverse_iterator itr ~ L.rbegin( ); while( itr [= L.rend( ) )

cout « * it r++ « endl;

3.17 l"vlodify the List class to provide stringent ilcrator checking by using the ideas suggested at the end of Section .3.5.

3.18 \Vhcn an erase method is applied to a list, it invalidates any iterator that is referencing the removed node. Such an ite!"<llor is called staIr. Describe an efficient algorithm that guarantees that any o]1er<.lt ion on a stale iterator acts as though the itefators current is NULL. Note that there may be many stale itcrators. You must explain which classes need 10 be rcwritten in order to implemenl your algorithm.

3.19 Rewrite the List class without using header and tail nodes and describe the differences betwcen the class and the class provided in Section 3.5 .

.3.20 An alternative to the deletion strategy we 11<1ve givcn is to use lazy deletion. To dekle an clement, we merdy mark it ddeted (using an extra bit fidd). The number of deleted and l10ndclctcd clements in the list is kept as part of the data structurc. If 1 here arc as !ll~H1y deleted dements as nondclet nl clements, we traverse the entire list, performing the: standard delrtion algorithm on alll1l<lrked nodes.

.u'"'



a. List the acivanLagcs and disadvantages of lazy deletion. b. Write routines 10 implement. the standard linked list operations llsing lazy delc~

lion.

3.21 vVritc a program to check for balancing symbols in the J()lluwing languages: a. Pascal (beg; n/end, 0, [l, {}) b. C++ (/* */, 0, [J, {))

* c. Explain how to prinl out an error message thaI is likely to reflect the probable cause.

3.22 \Vrite a program to evaluate a postfix expression.

3.23 a. \Vrite a program to convert an infix expression that includes (, ), +, -, *, and / to postfix.

b. Add the expOncl1!ialion operator to your repertoire. c. Write a program to convert a postfix expression 10 infix.

3.24 Write routines to implement two stacks using only one array. Your stack routines should nol declare an overflow unless every slot in the array is used.

3.25 * <1. Propose a data structure that suppons the Slack push and pop operations and a third operation fi ndM; n, \vhlch returns the smallest element in the dat.a st.ruct.ure, all in O(l) worst case time.

* b. Prove that ihve add the fourth operation deleteMin which finds and removes the smallest clement, then at least one of the operations must take Q (log N) time. (This requires reading Chapter 7.)

'* 3.26 Show how to implement· three stacks in one array.

3.27 If the recursive routine in Section 2.4 used to compute Fibonacci numbers is run for N = 50, is stack space likely to run out? vVhy or wby not?

3.28 A cieque is a data structure consisting of a list of it.ems, on which t.he following operations arc possible:

push(x): Insert item x on the front end orthe clcque. pop{): Remove the front item from the cieque and return it. inject (x): Insert item x on the rcar end of the dequc. eject (): Remove the rear item from the deque and return it.

vVritc routines to suppOrt the ueque that take 0(1) time per operation.

3.29 Write an algorithm for printing a singly linkecllist in reverse, using only constant extra space. This instruction implies that you cannot usc recursion but you may assume that your algorithm is a list mcrnher function. Can such an algorithm be written if the routine is a constant memher function?

3.30 a. \Vrite an anay implementation of self-a(~iusting lists. In a self-adjusting list, all insertions arc performed al the front. A self-adjusting list adds a find operation, and when an clement is accessed by a fi nd, it is moved to the from of the list without changing the relative order of the otber items.

b. vVritc a linked list implementation of self~adjusting lists. '* c. Suppose eacb element has a fixed probability, Pi, of being accessed. Show that the

elements with highest access probability are expected to be close to the front.



txermes

3.31 Efflciently implement a stack class using a singly linked list, with no header or tail nodes,

3.32 Efficiently implement a queue class using a singly linked list, with no header or tail nodcs.

3.33 Efficiently implement a queue class using a circular array. You may usc a vector (rather than a primilive array) as the underlying array strllcl ure.

3.34 A linked list contains a cycle if, starting h'om some node [1, following a sufficient number of next links hrings us hack to node p. I} docs not have to he the first nocle in the list. Assume thaI you are given a linked list that contains N nodes. However, the value of N is unknown. a. Design an O(N) algorithm to determine if the list contains a cycle. You may usc

O(N) extra space. * h. Repeat pan (3), hm usc only O( I) extra space. (Ilini: Usc two iteral0rs that arc

initially at the start of the list, but advance at different speeds.)

3.35 One way to implement a queue is to use a circular linked list. In a circular linked list, the last node's next pointer points at the first node. Assume the list does not contain a header and Lhat we can maintain, at most, one iterator corresponding to a node in the list. For \vhtch of the following representations can all basic queue operations be performed in constant worst-case time? Justify your ~ms\vers. 3. Maintain an itcrator that corresponds to the first item in the list. b. Maintain an itcrator that corresponds to the last item in the list.

3.36 Suppose we have a pointer to a node in a singly hnkedlist that is guaranteed 110110

/Je Ihe last node in the list. \Ve do not have pointers to allY other nodes (except by following links). Describe an O(l) algorithm thaI logically removes the value stored in such a node from the linked lisl, maintaining the integrity of the linkecllist. ([-lint:

Invo1\'e the next node.)

3.37 Suppose that <l Singly linked list is implemented v·/ith both a header and a tail node. Describe constant-time algorithms to a. Insert item x before position p (given lJY an iterator). b. Remove the item stored at position p (given by an iterator).

III





HAPTER 4

Trees

ror large amounts of input, the linear access time of linked lists is prohibitive. In this chapter we look at a simple data Slructure for which the running time of most operations is O(log N) on average. 'vVe also sketch a conceptually simple modification to this data structure that guarantees the above time bound in the worst case and discuss a second modification that essentially gives an O(log N) running time per operation for a long sequence of instructions.

The data strucl ure that we arc referring to is knovvn as a binary search tree. The binary search tree is the basis for the implementation of two library collections classes, set and map, \vhich arc used in many applications. Trees in general are vcry useful abstractions in computer science, so we will discLlss their use in other, morc general applications. In this chapter, we will

See hmv trees are used LO implement the fik system of several popular operating systems.

See how trees can he used to evaluate arithmetic expressions.

ShO\v how to use trees to support searching operations in O(log N) average time, and how to refine these ideas to obtain O(log N) worst -case bounds. We will also sec how to implement thesc operations when the data are stored on a disk.

Discuss and use the set and map classes.

4.1 Preliminaries A tree can be defined in several ways. One natural way to define a trec is recursively. A tree is a collection of nodcs. The collection can he empty; otherwise, a tree consists of a distinguished node r, called the root, and zero or more noncmpty (sub)trees T], T2,. " Til,

each of 'whose roots afe conneLled by a directed edge from r. The root of each subtree is said to be a child of r, and r is the parent of each subtree

rool. Figure 4 . .1 shows a typical tree using the recursive definit.ion. from the recursive definition, we find that a nee is a collection of N nodes, one of

which is the root, and N ~ "1 edges. That there are N ~ !. edges follows from the fact that each edge connects some node to its parent, and every node except the root has olle parent (sec Pigmc 4.2).

In the tree of Figure 4.2, the root is A. Node F has A as a parent and 1<, L, and Mas children. Each node may have an arbitrary number of childrc-n, possibly zero. Nodes with

113



... root

Figure 4.1 Generic 1 rce

B ~.

N

Q

Figure 4.2 A liTe

no children arc knmvn as leaves; the leaves in Ihe tree above are 13, C, 1-I, J, l~ Q, K, L, M, and N. Nodes with the s,trne parent afC siblings; thus K, L, and M arc all siblings. Grandparent and grandchild relations can be ddinecl in a similar manner.

A path ['rom node Il! to 111/ is defined as a sequence of nodes n I, 112,. ., nil such 1 hal Ilj is the parent of Ilj+J for I :s i < k. The length of this path is the number of edges on the path, namely}{ ~ 1. There is a path of length zero from every node to itself. Notice that in <l tret' there is exactly one palh from the root to each node.

For any node Ill> the depth of Ilj is the length of the unique path from the root to I1 j .

Thus, the root is al depth o. The height of Ili is the length or the longest path from Hi to a leaf. Thus all leaves are at height O. The height of a tree is equal to the height of the root. For the tree in Figure 4.2, E is at depth .l and height 2; F is at depth 1 and height 1; the hcight of the tree is 3. The depth of a tree is equal to tbe depth of the deepest leaf; this is always equal [0 the height of the tree.

l[ t.here is a path from III to /12, then III is an ancestor of 112 and 112 is a descendant of 11]. If 11] t 112, then 111 is a proper ancestor of 112 and 112 is a proper descendant of Ill·

4.1.1 Implementation of Trees One way to implement a tree would be to have in each node, besides its data, a link to each child of the node. However, since the number of children per node can vary so greatly and



5 [,

struct TreeNode

);

Object element; TreeNode *firstChild; TreeNode *nextSibling;

Figure 4.3 Node declarations for trees

A

B F----( C )-----( D )----( E -~

H

4.1 yrellmmanes

N

Figure 4.4 rirst child/next sibling representation of the tree shown in figure 4.2

is not known in advance, it might be infeasible to I1wke the children direct links in the data structure, hecause there wuuld be too much wasted space. The solution is simple: Keep the children or each node in a linked list of tree nodes. The dcciar;llion in Figure J.L3 is typical.

Figure 4.4 shO\vs how a tree might be represented in this implementation. Horizontal arrows that point downward are fi rstChi ld links. Arrows that go left to right arc nextS; bl i ng links. Null links arc not drawn, because there arc too many.

In the tree of Figure 4.4, node f. has both a link to a sihling (F) and a link (0 a child (I), while some nodes have neither.

4.1.2 Tree Traversals with an Application There arc many applications for trees. One of the popular usC's is the directory structure in many common operating systems, including UNIX and D{)S. Figure 4.5 is a typical directory in the UNIX file system.

The root of this directory is lu-:;r, (The asterisk next to the name indicates that IusI' is itself a directory.) IusI' has three children, mar!?, alex, and /)ilI, which are themselves directories. Thus, lu-:;r contains three directories and no regular files. The filename lusrlmarldhoo!?!ch 1. r is obtained by following Lhe leftmost child three times. Each I after the first indicates an edge; the result is the full pathmnne. This hierarchical file system is very popular, because it allows users to organize their data logically Furthermore, [\'1'0 files in different directories can share the same name, hecause they rnust have different paths from the root and thus

II:>



.. u

lUST'"

-------- -1------~--------

mark* alex" bill'"

I / --------------

book*' c{)ufse* junk junk work*' course*'

/~T-~ \ I eh!.T cil2.r eh3.f cop3530* cop3212"

T-~ -----------/ --

fa1105* spr06* sum06o!" fa1l05* fallOfi*'

I I 1 ---~~.\--~---~ ~-syt.r syLr syl.r grades progLr prog2,r prog2.r prog 1.1' grades

Figure 4.5 UNIX directory

void FileSystem::listAll( int depth" 0) canst

{

printName( depth}; II Print the name of the object 2 if( isDirectory( ) )

.1

4

for each file c in this directory (for each child) c.listAll( depth + 1 );

Figure 4.6 Pseudocode to list a directory in a hierarchical flIe system

have Jiffercm pathnmnes. A directory in the UNIX file system is just a Cile with a list of all its children, so the directories arc structured almost exactly in accordance with the type declaration above, l Indeed, on some versions or UNIX, if the normal command to print a file is applied to a directory, then the names of the files in the directory can be scen in the output (along with ot her non-ASCII information).

Suppose we would like to list the names of all of the files in the directory. Our output format \vill be that files that are depth d i will have their names indcllted by eli tabs. Our algorithm is given in Figure 4.6, as pseudocode.

The recursive function listAll needs to be staned with a depth of 0, to signify no indenting for the rooL This depth is an internal bookkeeping variahle, and is hardly a

parameter that a calling routine should be expected to know about. Thus the default value of 0 is provided for depth.

The logic of the algorithm is simple to follO\v. The name of the fik object is printed out with the appropriate numher of tabs. If the entry is a directory, then we process all children recursively, one hy one. These children are onc level deeper, and thus need to be indent.ed an extra space. The output is in figure 4.7.

1 Each directory in the \JNIX file system alsu has une clllry that points to itself ,mel another entry that points

10 thl' palTlit 01 til(' directury. Thus, technically, the UNIX lile system is not a tfce, hut is tredike



jusr mark

book chl. r

ch2. r

ch3.r course

junk alex

junk bill

work

cop3530 fall05

syl.1' spr06

sy1.1' sum06

syl.r

course

cop3212 f a II 05

grades

progl. r prog2.r

fall06

prog2.r progl.r grades

Figure 4.7 The (preorder) directory listing

4.1 rrellFllIndne~

This traversal strategy is known as a prcorder traversal. In a prcorder traversal, work at a node is performed before (pre) its children are. processed. vVhen this program is run, it is clem that line 1 is executed exactly once per node, since each name is output once. Since line 1 is executed at most once per node, line 2 must also be executed once per node.. Furthermore, line 4 can be executed at most once for each child of each node. BUI the number of children is exactly one less than the number of nodes. Finally, the for loop iterates once per execution of line 4, plus once each time the loop enels. Thus, the lotal amount of work is constant per node. If there arc N file names to be output, then the running time is O(N).

Another common method of traversing a tree is the postorder traversal. In a postorder traversal, the work at a node is performed alier (post) its children arc evaluated. As an

III



losr*( I) --r~--------mark*( I) alcx*(l) bilP'(I)

T~-~_ I / hook*(l) coorsc*(I) junk (6) junk (X) work"'( I) coursc*'( I)

//~T~-~~ \ I ch1.r(3) cI12.r(2) ch3.1"(4) cop3530*(l) cop3212*( I)

--------~-falI05*(I) spr06*(I)sum06*(I)

I I I fal105*( I)

~//"\-----sy!.r( I) syJ.r(S) syl.r(2) gradcs(3) prog l.r(4) prog2.r( 1)

"'-"'-

fal106*( 1)

~--/-1----~ prog2.r(2) prog l.r(7) grades(9)

Figure 4.8 UNIX directory with file sizes obtained via posLOrdcr traversal

int FileSystem: :size( ) canst

int totalSize = sizeOfThisFile( );

if( isDirectory( ) ) for each file c in this directory (for each child)

totalSize +~ c.size( );

return tatalSize;

Figure 4.9 Pseudocode In calculate the size of a directory

example, Figure 4.8 represents the same c.lirectory structure as before, with the nnmbers in parentheses reprcscllting the Illunbcr of disk blocks taken up by each file.

Since the directories arc themselves files, they have sizes too. Suppose we would like 10

calculate the total number of blocks used by all the files in the tree. The most natural \vay [0 do this would be to find the number of blocks contained in the subdirect.ories 11Isrlnwrl~ (30), Am/alex (9), and lusr//Ji/l (32). The total number of blocks is then the total in the subdirectories (71) plus the one block used by IusI', for a total of 72. The pseudocode method size in figure 4.9 implements this strategy.

If the current object is not a directory, then size merely returns the number of blocks it uses in the current object. Othenvise, the number of blocks Llsed by the directory is added to the Ilumber of blocks (recursively) found in all of the children. To sec the difference between the posLorder traversal strategy and the preorder traversal stralegy, Figure 4.10 shows hm\' the size of each directory or file is produced hy the algori! hm.



't.L DIlIOIY 111::1:::>

chI. r 3

ch2.r 2

ch3.r 4

book 10

syl. r 1

fa 11 05 2

syl.r 5

spr06 6

syl. r 2

sum06 3

cop3530 12

course 13

junk 6

mark 30

junk 8 alex 9

work 1

grades 3

progl. r 4

progZ.r 1

fall 05 9

prog2.r 2

progl. r 7 grades 9

fa1106 19

cop3212 29

course 30

bi II 32

jusr 72

Figure 4.10 Trace of the size function

4.2 Binary Trees ;\ binary tree is a tree in which no node can have more than two children.

figure 4.11 shows that a binary tree consists of a root and two subtrees, IL and TR,

both of which could possibly he empty. A property of a binary tree that is sometimes importanl is that the depth of an average

binary tree is considerably smaller than N. An analysis shows that the average depth is oeJ'iJ), and that for a special type of binary tree, namely the /Jinwy search (ree, the average value of the depth is O(log N). Unfortunately, the depth can be as large as N ~ I, as the example in Figure 4.12 shovvs.



root

Figure 4.11 Ceneric binary tree

A

B

c

D

E

Figure 4.12 vVorsl~case binary tree

struct BinaryNode (

};

Object element; BinaryNode *left; BinaryNode *right;

II The data in the node II left child

II Right child

Figure 4.13 Binary tree node class (pseudocode)

4.2.1 Implementation

•

Because a binary tree node has at most two children, we can keep direct links to them, The declaration of tree nodes is similar in structure to that f(H doubly linked lists, in that a node is a structure consisting of the element information plus two pointers (left and right) to

other nodes (scc rig. 4.13). We could drmv the binary trees using the rectangular boxes that are customary for linked

lists, but trees are generally drawn as circles connected by lines, because they arc actually



'l.L jjlnary trees

+

+ *

a * + g

b c * f

d e

Figure 4.14 Expression trcc for (a + b * c) + ((d * e + f) * g)

graphs. VVf:'. also do not explicitly draw NULL links when referring to trees, because every binary tree with N nodes would require N + 1 NULL links.

Binary trees have many important uses not associated with searching. One of the principal uses of binary trees is in the area of compiler design, which we will now explore.

4.2.2 An Example: Expression Trees Figure 4.14 shows an example of an expression tree. The leaves of an expression trec arc operands, such as constants or variable names, and the othcrnodes contain operators. This particular tree happens to be binary, because all of the operators arc binary, and although this is the simplest case, it is possible for nodes 10 have more than two children. It is also possible for a node to have only one child, as is the case with the unary minus operator. We can evaluate an expression tree, T, by applying the operator at the root to the values obtained hy recursivclyevaluating the left and right subtrees. In our example, thc left sublree evaluates to a + (b * c) atld the right subtree evaluates to «d * e) + f) * g. The entire tree therefore represents (a + (b * e» + (((d * e) + f) * g).

We can produce an (overly parenthesized) infix expression by recursively producing a parenthesized left expression, thell printing out the operator at the root, and finally recursively producing a parenthesized right expression. This general strategy (left, node, right) is known as an inorder traversal; it is easy to remember because of the type of expression it produces.

An alternate traversal strategy is to recursively print out the left subtree, the right subtree, and then the operator. If we apply this strategy to our tree above, the output is abc * +

de * f + 9 * +, which is easily seen to be the postfix representation of Section 3.6.3. This traversal strategy is generally known as a postorder traversal. vVe have seen this traversal strategy earlier in Section 4.1.

A third traversal strategy is to print out the operator first and then recursively print out the left and right subtrees. The resulting expression, + + a * b c * + * d e f g, is the less useful prejix notation and the traversal strategy is a preorder traversal, which we have also seen earlier in Section 4.1. vVe will return to these traversal strategies later in the chapter.

.... , Bismillah hiir Rehman nir Raheem-----------------------------Assalat o Wasalam o Alika Ya RasoolALLAH


, .... Constructing an Expression Tree Vve nmv give an algorithm to convert a postfix expression into an expression tree. Since we already have an algorithm to convert infix to postfix, we em generate expression trees from the two common types of input. The method we describe strongly resembles the postfix evaluation algorithm of Section 3.6.). 'vVc read our expression one symbol at a time. If the symhol is an operand, we create a one-node tree and push a pointer to it onto a stacie If the symbol is an operator, we pop (pointers) 10 two trees TJ and T2 frorn the Slack (T1 is popped first) and form a new tree whose root is the operawr and whose left and right children point to T2 and II> respectively. A pointer to this new tree is then pushed onto the stack.

As an example, suppose the input is

a b + c d e + * *

The first two symbols arc operands, so we create one-node trees and push pointers to them onto a stadel

a b

Next, a + is read, so two pointers to trees are poppeel, a new tree is formed, and a poinler to il is pushed onto the stacie

+

a b

Next, c, d, and e are read, and for each a one-node tree is created and a pointer to the corresponding tree is pushed onto the stack

1 For convenience, we will have the stack grow from left to right in the diagrams.



4.2 t.imary Ifees

+ c d e

a b

Nowa + is read, so two trees are merged.

+ c +

a b d e

Continuing, a * is read, so we pop two tree pointers and form a new tree with a * as root.

+ *

a b c +

d e

Finally, the last symbol is read, t\VO trees are merged, and a pointer to the final tree is left on the stack

123



Lnapler 4 I rees

*

+ *

a b +

d

4.3 The Search Tree ADT-Binary Search Trees An important application of binary trees is t heir use in searching. Let us assume that each node in the tree stores an item, In llurcxamplcs, we will aSSUlll(' for simplicity lhat these arcintegers, although arbitrarily complex items arc casily handled in C++. \Ve will also assume that all the items are distinct, and deal with duplicates bter.

The properly that makes a binary tree into a binary search tree is thm for every node, X, in the Iree, the values of all the items in its left subtree arc smaller than the item ill X, and the values of all the items in its right subtree are larger than the item in X. Notice that this implies that all the clements in the tree can be ordered in some consistent manner. In Figure 4.]5, the tree on the left is a binary search tree, but the tref 011 the right is noL. The tree on the right has a node with item 7 in the left subtree of a node with item 6 (which happens 10 be the root).

\Ve now give brief descriptions of the operations that arc usually performed on binary search trees. Note that because of the recursive delinition of \rces, it is common to \vrite these routines recursively. Because the ~l\'erage depth ofa binary search tree turns nUl to be O(log N), we generally do not need to worry about funning OUI of stack space

Figure 4. 16 shows the interface for the BinarySearchTree class template. There arc several things worth noticing. Searching is based on the < operator that must he defined for the particular Comparab 1 e type. Specifically, item x matches y if hoth x<y and y<x arc fa 1 se. This allows Comparable to he a complex type (such as an employee record), with a comparison function defined on only part of the type (such as the social securilY number data member or salary). Section 1.6.3 illustrates the general technique of designing a class that can he used as a Comparabl e. An alternative, descrihed in Section 4.3.1, is to allow a function ohject.

The data member is a pointer to the mot node; this pointer is NULL for empty trees. The publ i c member functions use the general technique of calling pri vate recursive functions. An example of how lhis is done for contains, insert, and remove is shown in Figure 4.l7.



4.3 The Search Tree ADT -Binary Search Trees

6 6

2 2 8

4 4

} } 7

Figure 4.15 Two hi nary trees (only the left tn:c is it search tree)

Severa! of the prj vate lllelll ber fUllctions usc the technique of passing a poilller variable using call by reference. This allow~ the public member functions to pass a pointer to the root (0 the private recursive member functions. The recursive functions can then change the vahw of"the root so that the root points to anot.her noele. We will describe the technique in more detail when we examine the code for insert.

Vve can now describe some of the private methods.

4.3.1 contains This operation requires rl'lurning true if there is a noele in lree T that has itell1 X, or false if t here is no slIch node. The structure of the tlTe makes this simple. If T is empty, then we can just return false. Otherwise, if the item stored at T is X, we can return true. Othenvisc, we make a recursive call on a subtree of T, eHher left or right, depending on the rela1ionship of X to the item stored in T. The cock in figure 4.18 is an implementation of 1 his st rategy.

Notice Ihe order of the tests. It is cruciall.hat the test for an empty lITe be pcrfonncd first, since otherwise, we would generate a run time error attempting to access a daw member through a NUll pointer. The remaining tests are arranged with (he least likely case last. Also note that both recursive calls arc actually tail recursions and can be easily removed with a whi 1 e loop. The usc of tail recursion is justifiable here because the simplicity of algoril hmic expression compensates for the decrease in speed, and the amounl of Slack space used is expected to be only O(log J\I).

Figure 4.19 shows the trivial changes required to usc a function object rather than requiring that the items be Comparable. This mimics the idioms in Section] .6.

4.3.2 fi ndMi nand fi ndMax These private routines return a pointer to Ihe nock containing the smallest and largest ckmcl1ls inlhe trec, respectively To perform a findMin, start at the mot and go !eft as long as there is a !eft child. The slopping point is the smallest ckmcn!. The findMax routine is thc same, except thai hranching is to the right child.

125



... u

template <typename Comparable> 2 class BinarySearchTree 3 { 4 public:

.5 BinarySearchTree( );

6 BinarySearchTree( canst BinarySearchTree & rhs ); 7 -BinarySearchTree( ): 8 9 canst Comparable & findMin( ) const;

10 canst Comparable & findMax( ) canst; 11 bool contains( canst Comparable & x ) const; 12 bool isEmpty( ) const;

13 void printTree{ ) const; 14

15 void makeEmpty( );

16 void insert( canst Comparable & x ); 17 void remove{ canst Comparable & x); 18

19 canst BinarySearchTree & operator~( canst BinarySearchTree & rhs ); 20 21 private: 22 struct BinaryNode

23

24 Comparable element; 25 BinaryNode *left;

26 BinaryNode *right;

27

28 29 30 };

31

BinaryNode( canst Comparable & theElement, BinaryNode *1t, BinaryNode *rt )

: e1ement( theElement ). 1eft( 1t ). right( rt ) ( }

.'32 BinaryNode *root;

.)3

34 void insert( const Comparable & x, BinaryNode * & t canst; 35 void remove( const Comparable & x, BinaryNode * & t const;

36 BinaryNode * findMin( BinaryNode *t ) const; 37 BinaryNode * findMax( BinaryNode *t ) const; .38 bool contains{ const Comparable & x, SinaryNode *t ) canst; 39 void makeEmpty( BinaryNode * & t );

40 void printTree( BinaryNode *t ) const;

41 BinaryNode * clone( BinaryNode *t ) const; 42 };

Figure 4.16 Binary search tree class skeleton



4,3 The Search Tree ADT -Binary Search Trees

I r* 2 * Returns true if x is found in the tree.

3 *1 4 bool contains( const Comparable & x ) const .5

6

7 8

9 /**

return contains( x, root);

10 * Insert x into the tree; duplicates are ignored. 11 *1 12 void 'insert( const Comparable & x ) 13 {

1"1 insert( x, root);

15 1(,

17 r* 18 * Remove x from the tree. Nothing is done if x is not found. 19 *1 20 void remove( const Comparable & x ) 2J {

22 remove( x, root }; 2.1

Figure 4.17 Illustration of public member (unction caUing privatC' rCCLlrsiVl' memher function

I 1** 2 * Internal method to test if an item is in a subtree. 3 * xis item to search for.

'1 * tis the node that roots the subtree.

S *j 6 bool contains( const Comparable & x, BinaryNode *t ) const

7 8 <)

10

II

12

13 H 15

16

it( t == NULL)

return fal s€;

else if( x < t->element return contains( x, t->1eft };

else if{ t->element < x

return contains( x, t->right ); else

return true; II Match

Figure 4.18 conta ins opcrat ion for binary search trees

127



template <type name Object, typename Comparator~less<Object> > 2 class BinarySearchTree 3 4 public: 5 6 II Same methods, with Object replacing Comparable 7

8 pri vate: 9

10 BinaryNode *root; 11 Comparator; sLessThan; 12

13 II Same methods, with Object replacing Comparable 14

15 /** 16 * Internal method to test if an item is in a subtree. 17 * x is item to search for. 18 * t ;s the node that roots the subtree.

19 'I 20 bool contains( canst Object & x, BinaryNode *t ) const 21

22 23 24 25 26 27

28 29 30 31 };

if( t == NULL)

return false; else if( isLessThan(

return contains( else if( isLessThan(

x, t->element x, t->left );

) )

t->element, return contains( x, t->right

else return true: II Match

x ) )

) ;

Figure 4.19 Illustrates usc of a funclion object to implement binary search tree

This is so easy that many programmCfs do not bother using recursion. We will code the routines both ways by doing fi ndMi n recursively and fi ndMax nonrecursively (see figs. 4.20 and 4.21).

Notice how we carefully handle the degenerate case of an emply tree. Although this is always importanl 10 do, il is especially crucial in recursive programs. Also notice that i\

is safe 10 change t in findMax, since we arc only working with a copy of a poimer. Always he extremely cardul, however, because a statement such as t->ri ght '" t->ri ght->ri ght will make: changes.



4.5 I ne ~earcn Iree AUI-lSmary ~earcn Irees

/** 2 * Internal method to find the smallest item in a subtree t.

3 * Return node containing the smallest item.

4 */ .'5 BinaryNode * findMin{ BinaryNode *t ) const

6

7

8 ')

10

11 12

if ( t 00 NULL )

return NULL:

if( t->left 00 NULL

return t;

return findMin( t->left );

Figure 4.20 Recursive implementation of f;nd~1in for binary search trees

1 /** 2 * Internal method to find the largest item in a subtree t. 3 * Return node containing the largest item.

'I * / 5 BinaryNode * findMax( BinaryNode *t ) const

(, (

7 if(t!oNULL)

8 I'-/hile( t->right !'" NULL)

9 t ~ t->right:

10 return t:

11

Figure 4.21 Nonrecursive implementation of fi ndMax for binary search trees

4.3.3 insert The insertion routine is conceptually simple. To insert X into tree T, proceed down the tree as you would with a contai ns. If X is found, do nothing (or "update" something). Otherwise, insert X at the last spot on the path traversed. Figure 4.22 shows what happens. Tn insert 5, we traverse the tree as though a contains were occurring.;\t the node with item 4, we need to go right, but there is no subtree, so 5 is not in the tree, and this is the correct spot.

Duplicates can be handled by keeping an extra fidd in the node record indicating the frequency of occurrence. This adds some extra space to the entire tree, hut. is hetter than putting duplicates in the tree (v-/hich tends to make the tree very deep). or course this strategy does not work if the key that guides the < operator is only part of a larger structure. If that is the case, then we can keep all of the structures that have the same key in an auxiliary data structure, such as a list or another search tree

Figure 4.23 shows the code for the insertion routine. Lines 12 and 14 recursively insert and attach x into the appropriate subtree. Notice that in the recursive routine, the only time

IZ!I



6 6

2 8 2 8

4 4

3 3

Figure 4.22 Binary search trees hefore and "ftcr insening 5

/** 2 * Internal method to insert into a subtree.

j * x ; 5 the item to ; nsert.

4- * tis the node that roots the subtree . .'5 * Set the new root of the subtree.

6 *j 7 void insert( const Comparable & x, BinaryNode * & t )

8

o if ( t "" NULL )

JO t '" new BinaryNode( x, NULL, NULL );

I J else if( x < t->element

12 insert( x, t->left ); 13 else if( t->element < x )

14 insert( x. t->right );

15 else

16 II Duplicate; do nothing 17

Figure 4.23 Insertion into a binary search tree

that t changes is when a new leaf is created. \i\lhen this happens, it means that the recursive routine has been called from some other node, p, which is to be: the leaf's parent. The call will be insert(x.p->left) or insert(x,p->right). Either way, t is now a reference to either p->left or p->right, meaning that p->left or p->right will be changeelto point at the new noele. All in all, a slick maneuver.

4.3.4 remove As is common with many data structures, lhe hardest operation is Jelction. Once we have found the node to be deleted, we need to consider several possibilities.



4.3 The Search Tree ADT-Binary Search Trees

6 6

2 2 8

4

3

Figure 4.24 Deletion of a node (4) with one child, hefore and after

6

3 8

5

3

4 4

Figure 4.25 Deletion of a node (2) with two children, hefore and after

If the node is a leaf, it can be dckted immediately. If the node has one child, the node can be deleted after its parent adjusts a link \0 bypass the node (w(' \vill draw the link directions explicitly for clarity). Se.c figure 4.24.

The complicated case deals 'with a node with 1\\'0 children. The general strategy is to replace the data of this node with the smallest data of the right subtree (which is easily found) and recursively delete that node (which is now empty). Because the smallest node in the right subtree cannot have a left child, the second remove is an easy olle. Figure 4.25 shows all initiaitrLc and thL result of a deletion. The nodL to he deleted is the left child of lhe root; the key value is 2. It is replaced with the smallest data in its right subtree (3), and then that node is deleted as before.

The code in Figure 4.26 performs deletion. It is inefflcient, hecause it makes two passes clown the tree to find and delete the smallest node in the right subtree when this

III



/** 2 * Internal method to remove from a subtree. 3 * xis the ; tern to remove. 4- * tis the node that roots the subtree . ."5 * Set the new root of the subtree.

(, *j 7 void remove( canst Comparable & x, BinaryNode * & t )

8 9

10

11

12

13 14 15

16 17

18

19

20 21

22 23

2'1

25 26

H( t "" NULL)

return; II Item not found; do nothing if{ x < t->element )

remove( x, t->left );

else if( t->element < x ) remove( x, t->right ):

else if( t->left !~ NULL && t->right !~ NULL) /1 Two children (

else

t->element ~ findMin( t->right )->element; remove( t->element, t->right );

BinaryNode *oldNode = t· t = ( t->left != NULL) ? t->left t->right; delete oldNode;

Figure 4.26 Deletion routine for binary search trees

is appropriate. It is easy to remove this inefficiency, by wriling a special removeMin method, and wc have left it in only for simplicity.

If the number of dele.tions is expected to be small, then a popular strategy to use is lazy deletion: \Vhen an element is to be deleted, it is left in the tree and merely mm-hed as being deleted, This is espccially popular if duplicate items are present, because then the data member Ihal keeps count of the frequency of appearance can bc decremenlecL If the number of real nodes in the tree is the same as Ihe number of "deleted" nodes, then the depth of the tree is only expected to go up by a small constant (why?), so there is a very small time penalty associated with lazy deletion. Also, if a deleted item is reinserted, the overhead of allocating a new cell is avoided.

4.3.5 Destructor and Copy Assignment Operator As usual, the destructor calls makeEmpty. The public makeEmpty (not shown) simply calls the private recursive version. As shown in Figure 4.27, after recursively processing t's children, a call to delete is made for t. Thus all nodes are recursively reclaimed. Notice that at the end, t, and thus root, is changed to point at NULL. The copy assignment operator, shown



4.3 I he ~earch Tree ADT -Binary Search Trees

I*' 2 * Destructor for the tree

J */ 4 -BinarySearchTree{ 5 (

() makeEmpty ( );

7

8 /** 9 * Internal method to make subtree empty.

10 */ 11 voi d makeEmpty{ B1 naryNode * & t ) 12 (

13 if( t !" NULL)

14 (

makeEmpty( t->left ); makeEmpty( t->right );

delete t:

L5

16

17

18

19

20 t NULL;

Figure 4.27 Destructor and recursive make£mpty member function

in figure 4.28, follows the usual procedure, first calling makeEmpty to reclaim any memory, and then making a copy of rhs. Vve use a very slick recursive function named clone to do all the dirty work.

4.3.6 Average-Case Analysis Intuitively, we expect that all of tl1l' operatlons of the previous section, except makeEmpty and operator"', should take O(log N) lime, because in constant time we descend a level in the tree, thus operating on a tree that is now roughly half as large. Indeed, the running time of all the operations (except makeEmpty and operator=) is Oed), where d is the depth of the node containing the accessed item.

We prove in this section that the average depth over all nodes in a tree is O(log N) on the assumption that all insertion sequences are equally likely.

The sum of the depths of all nodes in a tree is known as the internal path length. VIe will now calculate the average internal path length of a binary search nee, where the average is taken over all possible insertion sequences into binary search trees.

Let D(N) he the internal path length for some tree T of N nodes. D( \) = O. An N-node tree consists of an i~node left subtree and an (N - i - i)-node righl subtree, plus a rool at depth zero for O:s i < N. O(i) is the internal path length of lhe left subtree with respect to its root. In the main tree, all these nodes are one level deeper. The same holds for the right subtree. Thus, we get the recurrence

D(N) = D(i) + D(N - i-I) + N - ]

133



..... UIQjJlt::! 't Ilt:t:)

/*' 2 * Deep copy •

.l *1 4 canst BinarySearchTree & operator""( canst B;narySearchTree & rhs )

5 { 6 if{ this !oo &rhs }

I {

8 makeEmpty( );

1) root = clone ( rhs. root );

10 11

12

J3

1// j**

return *thi s;

1.'5 * Internal method to clone subtree.

16 *1 17 BinaryNode * clone( BinaryNode *t ) const 18

19

20 21

if( t "" NULL)

return NULL;

22 return new BinaryNode( t->element, clone{ t->left ), clone( t->right ) );

2.3

Figure 4.28 operator'" and recursive clone member function

If all subtree sizes afC equally likely, which is true for binary search trees (since the subtree size depends only on the relative rank of the first clement inserted into the trt'c), hut not

binary trees, then the average value of both DO) and D(N - i-I) is (lIN) I.:;-:~I LJ(j). This yields

[

N-I J D(N) = ~ L. D(i) + N - 1

1\ j=O

Tbis recurrence will he encountered and solved in Chapter 7, obtaining an average value of D(N) = O(N log N) ThllS, the expected depth 0\ an)' node is O(lng N). As an example, the randomly generated 500-node tree shown in figure 4.29 has nodes at expected depth 9.98.

11 is tempting 10 say immediately that this result implies that the ;werage running time 01 all the operations discussed ill the previous section is O(log N), hUlthis is not entirely true. The reason for this is that because of deletions, it is not clear that all binary search trees are equally likely In particular, the deletion algOlithm dCSClibcd above 1;1\'01"5 making the kft subtrees deeper than the right, hecause wc arc always replacing a deleted node wit h a node



4.3 The Search Tree ADT-Binary Search Trees

Figure 4.29 A randomly generated binary search tree

Figure 4.30 Binary search tree after (~)(N2) insert/remove pairs

from the right subtree. The exact effect of this strategy is still unknown, but it seems only to be a theoretical novelty It has been shown that if \ve alternate: insertions and deletions (o.;)(j\J2) times, then the trees will have an expected depth of (':;)( IN). After a quarter-million random insertlremove pairs, the tree that was somewhat right-heavy in Figmf 4.29100ks decidedly unbalanced (average depth = 12 .. '51). Sec Figure 4.10.

135



'vVc could try to eliminate the prohlem by randomly choosing between the smallest element in the right subtree and the largest in the left when replacing the deleted element. This apparently eliminates the bias and should keep the trees balanced, hut nobody has actually proved this. In any event, this phenomenon appears to be mostly a theoretical novelty, because the effect does not show up at all for small trees, and stranger still, if o(N2)

insert/remove pairs arc used, then the tree seems to gain balance! The main point of this discussion is that deciding what "average" means is generally

extremely difflcuh and can require assumptions that mayor may not he valid. In the absence of deletions, or when lazy deletion is used, we can conclude that the average running times of the operations above are O(log N). Except for strange cases like the one discussed above, this result is very consistent with observed behavior.

If the input comes into a tree presorted, then a series of inserts will take quadratic lime and give a very expensive implementation of a linked list, since the tree will consist only 01 nodes with no k,lt children. One solution to the problem is to insist on an extra structural condition called balance: no node is allowed to get too deep.

There are qUite a few general algorithms to implement balanced trees. Most are quite a bit more complicated than a standard hinary search tree, and all take longer on average for updates. They do, however, provide prolection against the embarrassingly simple cases, Below, we will sketch one of the oldest forms ofhalancecl search trees, the AVL tree.

A second, newer method is to forego the balance condition and allow the tree to be arhitrarily deep, but after every operation, a restructuring rule is applied that tends to

make future operations efficient. These types of data Slructures are generally classified as self-adjusting. In the case of a binary search tree, we can no longer guarantee an O(1og N) bound on any single operation, but can show that any sequence of M operations takes total time O(M log N) in the worst case. This is generally sufficient protection against a bad \vorsl case. The data structure we will discuss is known as a splay Iree; its analysis is fairly intricate and is discussed in Chapter 11.

4.4 AVL Trees An AVL (Addson-Velskii and Landis) tree is a binary search tree with a balance condition. The balance condition lnUS\ be easy to maintain, and it ensures that the depth of the tret is O(log N). The simplest idea is to require that the left and right subtrees have the same height. As figure 4.:11 shows, this idea docs not force the lree to be shallow.

Another balance condition would insist that every node must have left and right subtrees of the same height. If the height of an empty subtree is defined to be -1 (as is usual), then only perfectly balanced trees of 211 - 1 nodes would Salisl)! this criterion. Thus, although this guarantees trees of small depth, the balance condition is too rigid to be useful and needs to be relaxed.

An AVI. tree is identical to a binary search tree, except that for every node in t he tree, the height of the left and right subtrees can differ by at most l. (The height of an empty tree is defined to be - 1.) (n Figure 4.32 the tree on the left is all AVL tree, hut the tree on the right is not. Height Information is kept for each node (in the node structure). It can be shown that the height of an AVL tree is at most roughly 1.441og(N + 2) - 1.32H, but in



4.4 AVl Trees

Figure 4.31 A bad binary tree Requiring balance al the mot is nol enough.

5 7

8 2

7 4

3 3 5

Figure 4.32 Two binary search trees. Only the left tree is AVL.

practice it is only slightly morc than log N. As an exmnplc, the AVL tree of height 9 with the fewest nodes ([43) is shown in Figure 4.3.'1 This tree has as a left subtree an AVL tree of height 7 of minimum size. The right suhtree is an AVL tree of height H of minimum size. This tells us that the minimum number of nodes, S(I1), in an AVL tree of height h is given by 5(11) = S(h - I) + 5(11 - 2) + l. For 11 = 0, 5(11) = 1. For 11 = I, S(h) = 2. The function S(h) is closely related to the Fibonacci numbers, from which the bound claimed above on the height of an i\VL trce [0110\"15.

Thus, all the tree operations can be performed in O(log N) time, exec pi possibly insertion (we will assume lazy deletion). When we do an insertion, we need to update all the balancing information for the nodes on the path back to the rool, but the reason Ihat insertion is potentially difficult is that inserting a node could violate the AVL tree property. (for instance, inserting 6 into the AVL tree in Figure 4.32 would destroy the balance condition at the node \vilh key 8.) If this is the case, then the property has to be restored before the insertion step is considered over. It turns out that Ihis can always be done with a simple modification to the tree, known as a rotation.

After an insertion, only nodes that arc on the path from the insertion point to the mOl

might have their balance altered because only those nodes have their subtrees altered. As

137



.... u

Figure 4.33 Smallest AVL tn:c of height 9

we follow the path up to the roOL and update the balancing informatloll, we may find a node whose ncw balance violates the AVL condition, 'We will show how to rebalance the tree at the first (i.c., deepest) such node, and we will prove that this rebalancing guarantees that the entire tree satisfies the Avt properly.

Let us call the node that must be rebalanced Q'. Since any node has at most two children, and a height imbalance requires that a's 1 \YO subt fees' height differ by 1\\10, it is casy 10 sec that a violation migbt occur in four cases:

1. An insertion into the left subtree of the left child of (x.

2. An insenion into the right subtree of the left child uf Q'.

3. An insertion into the left subtree oflhc right child of a.

4. An insertion inLo the right suhtree of the right child of Q'.

Cases I and 4 arc mirror image symmetries with respect to a, as are cases 2 anti ). Consequently, as a matter of theory, there arc two h"lSic cases. from a programming perspective, of course, there are sUll four cases.

The first cast, in which the inserlion occurs on the "outside" (i.c., left-left or rightright), is fixed by a single rotation of the tree. The second case, in which the insenion occurs OJ) the "inside" (i.e., left-right or right-left) is handled by the slightly more complex double rotation. These arc fundamcntal operations 011 the tree that we'll sec used several limes in balanced-tree algorithms. The remainder of this section describes these rotations,



'1".'1" nVL IICt:.)

k2 k,

k, k2 Z

Y Z

Figure 4.34 Single rotation to fix case 1

proves that they suffice to maintain balance, and gives a casual implementation of the AVL

tree. Chapter 12 describes other lJalanced-tree methods with an eye toward a more careful implementation.

4.4.1 Single Rotation Figure 4.34 shows the single rotation that fixes case 1. The before picture is on the left, and the after is on the right. Let us analyze carefully what is going on. Node h2 violates the AVL. balance property because its left subtree is two levels deeper than its right subtree (the dashed lines in the middle of the diagram mark the levels)' The situation depicted is the only possible case 1 scenario that allows hI to satisfy the AVI. property before an insertion hut violate it afterwards. Suht.ree X has grown to an extra level, causing it to be exactly two levels deeper than Z. Y cannot be at the same level as the new X because then h2 would have been out of balance before the insertion, and Y cannot he at the same level as Z because then h} would be the first node on the path toward the root that was in violation of the AVI.. balancing condition.

To ideally rebalance the tree, we would like to move X up a level and Z down a level. Note that this is actually more than the AVL property would require. To do thiS, we rearrange nodes into an equivalent tree as shown in the second part of Figure 4.34. Here is an abstract scenario: visualize the tree as being flexible, grab the child node hi, close your eyes, and shake it, letting gravity take hold. The result is that hI will be the new root. The binary search tree property tells us that in the original tree h2 > hI' so h2 becomes the right child of h] in the new tree. X and Z remain as the left child of hI and right child of h2, respectively. Subt.ree Y, which holds items that are between hI and h2 in the original tree, can be placed as h25 left child in the new tree and satisfy all the ordering requirements.

As a result of this work, which requires only a few pointer changes, we have another binary search tree that. is an AVL tree. This happens because X moves up one level, Y stays at the same level, and Z moves down one level. h2 and hI not only salisfy the AVL requirements, but they also have subtrees that are exactly the same height. Furthermore, the new height of the entire subtree is exactly the same as the height of the original subtree prior to the insertion that caused X to grow. Thus no further updating of heights on the path to the roO\

is needed, and consequently no further rotations are needed. Figure 4.35 shows that after the insertion of 6 into the original AVL tree on the left, node 8 becomes unbalanced. Thus, we do a single rotation between 7 and 8, obtaining the t.ree on the right.

... " Bismillah hiir Rehman nir Raheem-----------------------------Assalat o Wasalam o Alika Ya RasoolALLAH


5 5

7

00 3 3

Figure 4.35 AVI. property deslfoycd by insertion of 6, then fixed by a single rolaiiOIl

k,

k, x

y x

Figure 4.36 Single rotation fixes case 4

As \VC mentioned earlier, case 4 represents a symmetric GISt. Figure 4 .. 16 shows how a single rotation is applied. Let us work throllgh a rather long example. Suppose we start \V11h an initially empty AVL tree- and insert the items 3, 2, !) and then 4 through 7 in sequential order. The first problem occurs when it is time to insert item I hecause the AVt properly is violated at the root. \Vc perform a single rotation between the root and its left child to fix the problem. Hnc arc the before and afler trees:

(D 2

(l~ore 3

after

A dashed line joins the two nodes thai afC the suhject of the rotation, Next we insert 4, which causes no problems, hut the insertion of') creates a violation at node :.) that is fixed



't.'t f\Vl Ifees

by a single rotation. Besides the local change caused by the rotation, the programmer must femember that the rest of the tree has to 1x informed of this change. Here this means that 2~ right child must he reset to link to 4 instead of 3. Forgetting to do so is easy and would destroy the tree (4 would be inaccessible).

2 2

3 4

('~9 3 5

before after

Next we insen 6. This causes a balance problern at the root, since its left subtree is of height 0 and its right suhtree would be height 2. Therefore, we perform a single rotation at the root hetween 2 and 4.

/ 4

4 2 5

3 5 3 6

before 6 after

The rot,llion is performed by making 2 a child of 4 and 45 original left subtree the new right subtree of 2. Every item in this subtree must lie bet\veen 2 and 4, so this transformation makes sense. The next item we inser1 is 7, \vhich causes another rot at ion:

4 4

2 5 2 6

3

0~j; 3 5 7

before after

I'll



, .... LlrajJll::l "t 111;:<::;)

k2 kJ

kJ k,

~ Z x

Ii z

Figure 4.37 Single rotation fails to fix case 2

k2

kJ

c A D

c

Figure 4.38 Left-right double rotation to fix case 2

4.4.2 Double Rotation The algorithm described above has one problem: as Figure 4.37 shows, it does not work for cases 2 or 3. The problem is that subtree Y is too deep, and a single rotation docs not make it any less deep. The double rotation that solves the problem is shown in Figure 4.38.

The fact that subtree Y in Figure 4.37 has had an item inserted into it guarantees that it is nonemply. Thus, \ve may assume that it has a roOI and lwo subtrees. Consequently, the tree may be viewed as four subtrees connened by three nodes. As the diagram suggests, exaclly onc of tree H or C is two levels deeper than D (unless all arc empty), but we cannot be sure which one It turns out not to matter; in figure 4.38, both Band C arc drawn at 11 levels below D.

To rebalance, we see that we cannot leave h.." as the mot, and a rotation between h.-:, and h] was shown in Figure 4.37 to not work, so the only alternative is to place h2 as the new root. This forces hI to he ~l2's left child and k1 to be its right child, and it also completely determines the resulting locations of the four subtrees. 11 is easy to see that the resulting tree satisfies the AVL tree property, and as \vas the case with the single rotation, it restores the height 10 what it was bdore the insertion, thus guaranteeing that all rebalancing and height updating is complete. figure 4.39 shows that the symmetric case:) can also be fixed by a double rotation. In both cases the effect is the same as rotating between a:s child and grandchild, and then between a and its new child.

vVc will continue our previous example by inserting 10 through 16 in reverse order, i"ollowed by 8 and 1 hen 9. Inserting 16 is cas): 5i nee it does not destroy the balance propcrt}~



4.4 AVL Trees

k, A

A J)

Figure 4.39 Right~lcft double rotation to fix case 3

but illserting 15 causes a height imbalance at node 7. This is case 3, which is solved by a right~ldt double rotation. In our example, the right~kft double rotation will involve 7) '16, and 15. In this case, I?J is the node with item 7, hJ is the node with item 1o, and hl is the node "vith item 1.'5. Subtrees A, B, C, and D arc empty.

4 4

2 6 2 6

k,

05 3 5 7 3

\, k} before

Cf3> after

: k2 @

Next we. insert 14, which also requires a double rotation. Here the double rotation that will restore the tree is again a right-left double rotation that will involve 6, 15, and 7. In this case, 1?1 is the node with item 6, h2 is the node with item 7, and h3 is the node with item 1 '5. Subtree A is the tree rooted at the node with item 5; subtree B is the empty subtree that was originally the left child of the node with item 7, subtree C is the tree rooted <It the node \Vith item 14, and finally, subtree D is the tree rooted at the node with item 16,

4 4

k,

2 6 2

\h 3 5

~ &~'" before 14

3

after



144 Chapter 4 Trees

If 13 is now inserted, there is an imbalance at the root. Since 13 is not he tween 4 and 7, we know that the single rotation will work.

4 7 "

"

2 7 4 15

3 14 16

13

before after

Insertion of 12 will also require a single rotation:

7 7

15 4 15

14 16 16 :

before : 12 after

To insert 11, a single rotation needs to be performed, and the same is true for the subsequent insertion of 10. We insert 8 without a rotation, creating an almost perfectly balanced tree:



4.4 AVl Trees

7

4 before 13

6

5

Finally, we will insert 9 to show the symmetric case of the double rotation. Notice that 9 causes the node containing 10 to become unbalanced. Since 9 is between 10 and 8 (which is lOs child on the path to 9), a double rotation needs to be perltJrmeJ, yielding the following tree:

7

4 after 13

2 6

3 5

Let us summarize what happens. The programming details afC fairly straightforward except that there are several cases. To insert a new node with item X into an AVL tree T, we recursively insert X into the appropriate subtree of T (let us call this Trn). If the height



146 Chapter 4 Trees

1 struct Avl Node

2 :-; Comparable element;

'/ AvlNode *left;

.'5 AvlNode *right;

() int height;

7

8 AvlNode( canst Comparable & theElement. AvlNode *1t, o AvlNode *rt, int h ~ 0 )

10 : element( theElement ), left( It ), right( rt ), height( h )

II };

Figure 4.40 Node declaration for ;WL trees

1 /** 2 * Return the height of node t or -1 if NULL .

.3 */ 4 int height( AvlNode *t ) canst 5

return t == NULL? -1 : t->height; 7

Figure 4.41 Function to compute height of an AVL node

of TIR docs not change, then we arc done. Otherwise, if a height imbalance appears in J~

we do the appropriate single or double rotation depending on X and the items in T and Tu?, update the heights (making the connection from the rest of the tree above), and arc done. Since one rotation always surfices, a carefully coded non recursive version generally turns oUl to be significantly faster than the recursive version. However, nonrccufsive versions are quite difficult to code correctly, so many programmers implement AVL trees recursively.

Another effiCiency issue concerns storage of the height information. Since all that is reany required is the difference in height, which is guaranteed to he small, we could gel by with two bilS (to represent +1, 0, -1) if \!ifC really try. Doing so will avoid repetitive calculation of balance factors hut results in some loss of clarity The resulting code is somewhat more complicated than if the height were stored at each node. If a recursive routine is written, then speed is probably not the main consideration. In this case, the slight speed advantage obtained by storing balance factors hal"dly seems worth the loss or clarity and relative simplicity. Furthermore, since most machines will align this to at

least an 8~hit boundary anyway, there is not likely to he any difference in the amount of space used. An eight~bit (signal) char will allow us to store ahsolute heights of up to 127. Since the tree is balanced, it is inconceivable that this would be insufficient (sec lhe exercises).



/** 2 * Internal method to insert into a subtree.

3 * xis the item to insert. 4 * t is the node that roots the subtree.

:5 * Set the new root of the subtree.

6 * I 7 void insert{ canst Comparable & x. AvlNode * & t )

8 9

10

11

12

13 14

15

16

17

18

19

20 21

22

23 24 25 26 27

if( t "" NULL) t " new AvlNode( x, NULL, NULL );

else if( x < t->element )

insert( x, t->left ); if( height( t->left ) - height( t->right

if( x < t->left->element ) rotateWithLeftChild( t );

else doubleWithLeftChild( t );

else if( t->element < x

else

insert( x. t->right ); if( height( t->right ) - height( t->left

if( t->right->element < x ) rotaleWithRighlChild( t );

else doublel~ithRightChild( t );

1/ Duplicate; do nothing

2 )

2 )

28 2Y .10

.11

32 t->height " max( height( t->left ), height( t->right ) ) + 1;

Figure 4.42 Insertion into an AVL tree

4.4 AVL Trees

'vVith all this, we <Ire ready to write the AVL routines. We show some of the code here; the rest is online. First, we need the Avl Node class. This is given in Figure 4.40. We also need a quick function to return the height of a node. This function is necessary to handle the annoying case: of a NUll pointer. This is shown in figure 4.41. The hasic insertion routine is easy to write, since it consists mostly of function calls (see fig. 4.42).

for the trees in Figure 4.43, rotate\4i thleftChi 1 d converts the tree on the left to the tree on the right, returning a pointer to the new root. rotateWithRightChild is symmetric. The code is shown in Figure 4.44.



148 Chapter 4 Trees

k,

k, z x

x y y z

Figure 4.43 Single rotation

I /** 2 * Rotate binary tree node with left child.

,) * For AVl trees, this is a single rotation for case 1.

"1 * Update heights, then set new root.

S *j 6 void rotate~jithLeftChild( AvlNocte * & k2 )

7 { R AvlNocte *kl '" k2->left;

9 k2->left "" kl->right;

10 kl->ri ght - k2;

II k2->height - max( height( k2->left ), height( k2->right ) ) + I;

12 kl->height - max( height{ kl->left ), k2->height ) + I;

13 k2-kl;

1'1

Figure 4.44 Routine to perform single rotation

k,

A Ii c [)

Figure 4.45 Double rotation

The last function we will 'vvrite will perform the double mlalion pictured in Figure 4.40.;, for which the code is shown in Figure 4.46.

Deletion in AVL trees is somewhat lllore complicated than insertion, and is left as an exercise. Lazy deletion is probably the best strategy if deletions arc relatively infrequent.



/'* 2 " Double rotate binary tree node: first left child

J * 11ith its right child; then node k3 with new left child. 4 ->.- For AVL trees, this is a double rotation for case 2.

5 * Update he; ghts, then set new root.

6 */ 7 void doubleWithLeftChild{ AvlNode * & k3 ) 8 9

10 II

rotateWithRightChild( k3->left );

rotatelolithLeftChild( k3 );

Figure 4.46 Routine to perform doubk rotation

4.5 Splay Trees

4.5 Splay Trees

\;Ve now describe a relatively simple data structure, known as a splay tree, that guarantees that any M consecutive tree operations starting from an empty trce take at must OeM log N) time. Although this guarantee docs 110t preclude the possibility thm any single operation might take O(J\f) lime, and thus the hound is not as strong as an O(log N) worst-case bound per operation, the net effect is the same: There arc no bad input sequences. C;cncrally, when a sequence of 7v1 operations has tOlal worst-case running lime' of ()(J\~f(N», \ve say that the amortized running time is O(j"(N». Thus, a splay tree 11<15 all O(log N) amortized cost per operation. Over a long sequence of opeJ"<ltions, some may takc more, sorn(' less.

Splay I rees arc hased on the fact that the O(N) worst -case time per opera! ion for binary search trees is nol bad, as long as it occurs relatively infrequently. Anyone access, evcn if it takes O(N), is sti1llikely to he extremely fast. The problem with binary search trees is that it is possible, and not uncommon, for a whole sequence of bad accesses 10 take place. The cumulative running time then becomes noticeable. A search tree data structure with O(N)

worst-case time, hut a ,~w.nalltce of at most O(M log N) for any M consecutive operations, is certainly satisfactory, because there arc no had sequences.

If any panicular operation is allowed to have an O(N) \vorSL-c<1se timc hound, and wc still want an O(log N) amortized time bound, then it is clear that whenever a node is accessed, it mllst he moved. Otherwise, once we find a decp node, we could keep performing accesses on it. If the node docs not change location, and each access costs O(N), then a sequence of M accesses will cost O(Al ,N).

The basic idea of the splay tree is that after a node is accessed, it is pushed to the rool by a series of ;\v] tree rotations. Notice Ihal if a node is deep, there arc many nodes on the path that arc also relatively deep, and hy restructuring we can make future accesses cheaper on all these nodes. Thus, if the node is unduly deep, then we want this restructuring [0 have the side dfrCl of balancing the tree (to some extern). Resides giving a good time hound in theory, this method is likely 10 have practical utility, because in many applications, when a node is accessed, it is likely 10 be acccssul again in the ncar futurc. Studies have shown that this happens much more often lhan one \vould cxpec!. Splay Lrees also do not require the

149



)!jO enapter 4 I rees

maintenance of height or balance information, thus saving space and simplifying the code to some extent (especially when careful implementJtions are writtcn).

4.5.1 A Simple Idea (That Does Not Work) One way of performing the restructuring described above is to perform single rotations, bottom up. This means that we rotate every node on the access path with its parenl. As an example, consider what happens after an access (a find) on hi in the following tree

The access path is dashed. First, we would perform a single rotation hetween hi and its parent, obtaining the following trec.

c

A B

Then, we rotate between I?l and k'1, obtaining the next tref.



4.5 Splay Trees

.................... 0'5~ ~~ ......

. 4

F

/, k 3

A Ii c 1)

Then two more rotations arc performed until we reach the root.

k3

lJ E

c 1J

k,

k, ~- k 5

k,j

A F

k3 Ii

C f)

These rotations have the dIce! of pushing I~l all the way to the roOL, so that future accesses on h] arc casy (for a while). Unfortunately, it has pushed another node (fl::,) almost as deep as I?] used to be. An access on that node wilt then push another node decp, and so on. Although this strategy makes future accesses of f?1 cheaper, it has not significantly improved the situation for the other nodes on the (original) access path. It turns out that it is possible to prove that using this strategy, there is a sequence of l'v1 operations requiring QUvl· N) time, so this idea is not quile good enough. The simplest way 10 show this is 10 consider the tree formed by insening keys 1,2, J, ... ,N into an initially empty tree (work

151



I:iZ lhapter 4 Irees

G x p p G

D

A c D

B c

Figure 4.47 Zig-zag

x p

A G

B

A c D

Figure 4.48 Zig-zig

this example out). This gives a tree consisting of only left children. This is not necessarily bad, though, since the time to build this tree is O(N) total. The bad part is that accessing the node with key 1 takes N - I units of time. After the rotations are complete, an access of the node with key 2 takes N - 2 units of time. The total for accessing all the keys in order

is L;~~l i = Q(N2 ), After they arc accessed, the tree reverts to its original state, and we can rcpeat the sequence.

4.5.2 Splaying The splaying strategy is similar to the rotation idea above, except that we afC a little more selective about how rotations arc performed. We will still rotate bottom up along the access path. Let X be a (nan root) node on the access path at which we are rotating. If the parent of X is the root of the tree, we merely rot ale X anclthe root. This is the last rotation along the access path. Otherwise, X has both a parent (P) and a grandparent (G), and there are two cases, plus symmetries, to consider. The first case is the zig-zag case (sec fig. 4.47). Here X is a right child and P is a left child (or vice versa). If this is the case, we perform a double rotalion, exactly like an AVI. double rotation. Otherwise, we have a zig-zig case: X and P arc both left children (or, in the symmetric case, both right. children). In that case, we transform the tree on the left of Figure 4.4S to the tree on the right.

As an example, consider the tree from the last example, with a contains on k(



4.5 Splay Trees

The first splay step is all~l' and is clearly a zi,g-zag, so we perform a standard AVt double rotation using I~J' Rl, and /zJ. The resulting tree follows.

k,

A B c D

The next splay step at h] is a zikzig, so we do the zig-zig rotation With hJ' 1<4, and by) obtaining the hnal tree.

A Ii

Although it is hard to see from small examples, splaying not only moves the accessed node to the root, bUl also has the effect of roughly halving the depth of most nodes on the access path (some shallow nodes are pushed down al most two levels),

To sec the difference that splaying makes over simple rotation, consider again the effect of inserting items 1,2,3, ... , N into an initially empty tree. This takes a total of O(N), as before, ancl yieleJs the sarne tree as simple rotations. Figure 4.49 shows the result of splaying at the node with item 1. The difference is that after an access of the node with item J, which takes N - 1 units, the access on the node with item 2 will only take about N /2 units instead of N - 2 units; there afe no nodes quite as deep as before.

153



154 Chapter 4 Trees

7

6 6

5 4 7

4

3

Figure 4.49 Result of splaying at node I

(1)- ---- --- __ _

14

Figure 4.50 Result of splaying at node I a tree of alllcit children



45 Splay Trees

Figure 4.51 Result of splaying the previous rrec at node 2

Figure 4.52 Result of splaying the previous tree at node 3

An access on the node with item 2 wm bring nodes to within N /4 of the roOl, and this is repeated until the depth becomes roughly log N Can example with N = 7 is too small to see the effect well). Figures 450 to 4.58 show the result of accessing items .1 through 9 in a 32-nodc tree that originally contains only left children. Thus we do not get the same had behavior from splay trees that is prevalent in the simple rotation strategy. (Acwally, this turns out to be a very good casco A rather complicated proof shows that for this example, the N accesses take a total of O(N) time.)

These ligures highlight the fundamental and crucial propert y of splay trees. When access paths arc long, thus leading to a longer-than-normal search timc, the rotations tend to be good for future operations. \Alhen accesses are cheap, the rotations are not as good and can be bad. The extreme case is the initial tree formed by the insertions. All the insertions were constant-time operations leading to a bad initial tree. At that point in time, we had a very had tree, but we were running ahead of schedule and had the compensation of less total running timc. Then a couple of really horrible accesses left a nearly balanced tree, hut the cost was that \ve had to give back some of the time that had been saved. The main theore111,

155



Hil> Lnapter 4 I rees

Figure 4.53 Result of splaying Ihe previous tree at node 4

Figure 4.54 Result of splaying the previous tree at node ')

~ ~~~ ~-~ ~-~~~ ~'4~ __


which we will prove in Chapter 11, is that we never fall behind a pace of O(log N) per operation: We are ahl.,lays on schedule, even though there afC occasionally bad operations.

We can perform deletion by accessing the node to be deleted. This puts thc node <111he rooL If it is deleted, we get two subtrees It and TJ(. (left and right). If we find the largest element in T{ (which is easy), then this element is rotalccllo the root of II., and It will now have a root with no right child. We can finish the deletion by making Tf(: the right child.

The analysis of splay trees is diffIcult> hecause it must take into account the everchanging structure of the tree, On the other hand, splay trees arc much simpler to program than AVL trees, since there are fewer cases to consider and no halance informat ion to



4.5 Splay Trees 157

Figure 4.56 Result of splaying the previous tree- at node 7


.......... @.

Figure 4.58 Result or splaying the previous trce at node 9



1:>0

maintain. Some empirical evidence suggests that this translates into faster code in practice, although the case for this is far from complete. Finally, we point out that there are several variations of splay trees that can perform even better in practice. One variation is completely coded in Chapter 12.

4.6 Tree Traversals (Revisited) Because of the ordering information in a binary search tree, it is simple to list all the items in sorted order. The recursive function in Figure 4.59 does the real work

Convince yourself that this funel ion works. As we have seen before, this kind of routine when applied to trees is known as an inorder traversal (which makes sense, since it lists the items in order), The general strategy of an inorder traversal is to process the left subtree first, then perform processing at the current node, and finally process the right subtree. The interesting part about this algorithm, aside from its simplicity, is that the total running time is O(N). This is because there is constant work being performed at every node in the tree, Each node is visited once, and the work performed at each node is testing against NULL,

I*' 2 * Print the tree contents in sorted order.

J 'I 4 void printTree( ostream & out = cout ) const 5 (

[, iff isEmpty( ) ) 7 out « "Empty tree" « endl; 8 else 9 printTree( root, out );

10

II

12 /** 13 * Interna 1 method to prj nt a subtree rooted at tin sorted order.

14 *1 15 void printTree( BinaryNode *t. ostream & out) const 16 (

17 iff t !" NUll) 18 (

19 printTree( t->left. out); 20 out « t->element « endl; 21 pri ntTree ( t->ri ght. out ); 22 23

Figure 4.59 Routine to print a binary search tree in order



J /"

2 * Internal method to compute the height of a subtree rooted at t.

j '/

4 int height( BinaryNode *t )

-'5

(,

7

8 <)

10

iI( t 00 NULL)

return -1;

else return 1 + max( height{ t->left ), height( t->right ) );

4.7 B-Trees

Figure 4.60 Routine to compute the height of a tree using a postorder traversal

setting up two function calls, and doing an output statement. Since there is constant work per node and N nodes, the running time is O(N).

Sometimes we need 10 process hoth suhtrees firsl he fore we can process a noele. For instance, 10 compute the height of a node, we need to know the: height of the suhtrees first- The code in Figure 4.60 computes this. Sillce it is always a good idea to check the spcci<l\ cases-and crucial when recursion is involvcd--noticc that the routine will declare the height of a leaf to be zero, \vhich is correct. This general order of traversal, which we have also sc:en before, is known as a postorder traversal. Again, thc lotal running time is O(N), hecause constant work is performed at each node.

The third popular traversal scheme that we have seen is pre order traversal. Here, the node is processed before the chlldrcn. This could he ltsdtd, for example, if you wanted to label each node with its depth.

The common idea in all of these routines is that you handle the NULL case first, and then the rest. Notice the laek of extraneous variables. These routines pass only the pointer to

the node that roots the subtree, and do not declare or pass any exira variables. The more compact the code, the less likely that a silly hug will turn up. i\ fourth, less often used, traversal (which we have not seen yet) is level~order traversal. I n a level-order traversal, all nodes at depth d are processed hefore any node at depth d + '!. Level-order traversal differs from thc other traversals in tbat it is not donc recursively; " queue is used, instead of the implied stack of recursion.

4.7 B-Trees So hu, we have assumed Ihal we can slore;\I1 ('mire data struclure in the main memory of a computer. Suppose, however, that we have more data than c<tn fit in main memory, and, as a result, must have the data structure reside on disk. \A/hen this happcns, the rules of the game change, because the Big~()h model is no longer meaningful.

The prohlem is that a Hig-()h analysis assumes thaI <1]] operations arc equal. However, this is not true, especially when disk 110 is involved. For example, i1 500-MIPS machine allegedly execlItes ~)OO million instructions per second. That is prelly fast, mainly because



IUU

the speed depends largely on clt'Llrlcal propcnics. 011 the other hanel, a disk is mechanical. Its speed depends largely on the time it takes to spin the disk and to move a disk head. Many disks spin at 7,200 RPM. Thus in I min, it makes 7,200 revolutions; hence, one revolution occurs in 1/120 of a second, Of 8.lms. On average, we might expect that we have to spin a disk halfway to find what we arc looking Cnr, but this is compensated by the time to move the disk head, so we gel an access time of 8.3 ms. (This is a vcry charitable estimate; 9-11 ms access times arc more common.) Consequently, we can do approxima1ely -120 disk accesses per second. This sounds pretty good, until we compare il with the processor speed. "Vha! we have is 500 million instructions equal to 120 disk accesses. Put another way, one disk access is worth about 4,000,000 instructions. Of course, everything here is a rough calculation, but the relativc speeds arc pretty clear: Disk accesses are incredibly expensive. Furthermore, processor speeds are increasing at a much faster rate than disk speeds (it is disk sizes that are increasing quite quickly). So, we arc willing to do lots of calculations just to save a disk access. In almost all cases, it is thc number of disk accesses that will dominate lhe running limc. Thus, if we halve the numher of disk accesses, the running time will halve.

lIere is how the typical scarch tree performs on disk: Suppose we want to access the driving records for citizens in the State of Florida. \Ve assume that we have 10,000,000

items, that each key is :12 bytes (representing a name), and that a record is 256 bytes. We assume this docs not fit in main memory and that we arc 1 of 20 users on a system (so we have 1120 of the resources). Thus, in 1 sec, we can execute 25 million instructions or perform six disk accesses.

The unhalanced binary search tree is <l disaster- In the worst case, it has linear depth and thus could require 10,000,000 disk accesses. On average, a successful search would require 1..:18 log N disk accesses, and since log 10000000;::-;::: 24, an average search ·would require )2 disk accesses, or '5 sec. In a typical randomly constructed trce, we would expect that a few nodrs arc three tinKs deeper; these would require about 100 disk accesses, or 16 sec. An AV\. tree is somC\vhaL helter. The worst case 0(" 1.44 log N is unlikely to occur, and the typical case is very close to log N. Thus an ;\\'1. trec would use about 25 disk accesses on average, requiring 4 sec.

\Ve want to reduce the number of disk accesses to a very small constant, such as three or four. We are willing to write cornplicatcd code to do this, because machine instructions are essentially free, as long as we arc nol ridiculously l..lllrcasonahle. it should probably be clear lhat a binary search tree will not work, since the typical AVI. tree is close to optimal height. We cannot go below log N using a binary search trec. The solution is intuitively simple: If we have more branching, we have less height. Thus, while a perfect binary tree of 31 nodes has five levels, a 5-ary ireI.' of:") I nodes has only lhree levels, as shown in figure 4.61. An M-ary search tree allows M-way hranching. As branching increases, the depth decreases. vVhereas a complete binary tree has height that is roughly log2 N, a complete M-ary tree has height that is roughly Iog,\i N.

We can create an M-ary search tree in much the same way as a binary search tree. In a binary search tree, we need one key In decide which of 1.\\10 branches to take. In an lvl-ary search !ree, wc need lvl - I keys to decide which branch to take. To make this scheme efficient in the worst case, we neeo. to ensure that the A1-ary search tree is balanced in some way. Otherwise, like a binary search tree, it could degencrate illlO a linked list. Actually, we want an even more restrictive halancing condition. That is, we do not ·want an M-ary search



4.7 Hrees

Figure 4.61 S-ary tree of 11 nodes has only three levels

tree to degenerate to even a binary search trcl', because then we \vould be stuck with log N accesses.

One way to implement this is to lISC a B-trce. The basic D-trcc"l is described hcrc.lvfany variations and improvcmcllls arc known, and an imrifmcnlatioll is somewhat complex because there arc quite a few cases. However, it is casy to sec thaI, in principle, a B-lrcc guarantees only a few disk accesses.

A B-trcc of order M is an M-ary tree with the following propcrtics:'+

1. The data items arc stored at leaves,

2. The nonlcaf nodes store up to M - 1 keys to guide the searching; key i represents the smallest key in subtn::c i + 1.

3. The root is either a leaf or has between two and M children.

4. All nonleaf nodes (except the root) have between pvlj21 and M children.

5, All leaves arc at the sallle depth and have hetween rL/21 and L data items, for some L (the determination of L is described shortly).

An example of a B-trce of order') is shown in figure 4.62. Notice that all nonlcaf nodes have between three and five children (and thus het\vcen two and four keys); the root could possibly have only t\\70 children. Here, \\le have L = 5. It happens that Land M arc the same in this example, but this is not necessary. Since L is 5, each leafllJs hetween three and flve data items. Requiring nodes to be half full guarantecs that the B-tree does not degenerate into a simple binary tree. Although there arc various definitions of TI-trecs that change this structure, mostly in minor ways, this definition is one of the popular forms.

Each node represents a disk block, so we choose M and L on the basis of the size of the items that afC being stored. As an exarnpk, suppose one hlock holds 8,.192 bytes. In our Florida example, each key uses 32 bytes. In a B-tree of ordcx 1\1, we would have !VI - I keys, for a total of 32M - 32 bytes, plus M branches. Sitlce each branch is essentially a number of another disk hlock, \ve can assume that a branch is 4 bytes. Thus the branches use 4M bytes. The total memory requirement for a nonIcaf node is thus .-36M - 32. The largest value: of M for which this is no more than 8, '192 is 228. Thus we would choose M = 228. Since each data record is 256 bytes, we would be able to fit 32 records in a hlod:. Thus we would choose L = ·32. 'vVe arc guaranteed that each leaf has bct\vecn 16 and 32

\ What is described IS popularly known <1S a B+ trec.

<I Rules 3 and "j must he relaxed lor the first I, insertions.

161



IU'"

41 66 ~

r

8 18 26 35 48 51 54 I

2 8 18 26 3S 41

~" 54

4 10 20 28 36 42 49 52 S6 6 12 22 30 37 44 50 53 58

14 24 31 38 46 59 - 12 ,- Jl2lJ -~ ~ -

Figure 4.62 B-lITC of order '5

41 66111 87 111 II 8 18 26 35 48 51 54 II 7278 83 ]]

2 8 18 26 35

"~~" 66

~'"~ 4 10 20 28 36 42 49 52 56 68 73 79 84 6 12 22 30 37 44 50 )3 57 69 74 81 85

14 24 31 38 46 58 70 76 L- j(i - cl£ E 22. - ~

Figure 4.63 B-tree after insertion or 57 into the tree in Figure 4J-:i2

97 98 99

92 97

87 92 97 89 93 98 YO 95 99

_L- L-

II II

data records and thal each illlcrnai node (except the mot) hranches in at least I 14 ways. Since there are 10,000,000 records, there arc, al most, 62"5,000 leaves. Conscqucmiy, in the \vorsL case, leaves would be on level 4. In more concrele lerm~, the worst-case number of accesses is given by approximately logAI/2 N, give or take I> (For example, the root and the next level could he c<lehec! in main memory, so that over the long run, disk accesses would be lJeeded only for level ~_) and (\repel'.)

The remaining issue is how to add and removc items rroHlthc B-trec. The ideas involved arc sketched next. Note that many of the themes seen before reoccur.

V/c begin by examining insertion. Suppose we \VcU1t to insert 57 inlO the B~trce ill Figure 4.61. ;\ search clown the trec reveals lhat il is not already in the tree. V·le. can add it to thc leaf as a fifth child. Note that Wf may have to reorganize all the data in the leaf to do this. Ilowcvcr, thc cost of doing this is negligible when compared to thai of the dlsk access, which in this case also includes a disk write.

Of course, that was relatively painless, hecause the leaf was not already full. Suppose we now wallt to insen '5'5. rigurc 4.(1.) shows a prohlem: The leaf where '5') wallIS 10 go is already full. The solution is simple: Since we now have L + 1 items, we split them inl0 two leaves, both guaranteed to have the minimum nurnbcr of data records neecied. vVe form two



8 18 26 35

2 8 4 10 6 12

14 "---ltL

18 26 20 28 22 30 24 31 ~J2

48 51 54 57

4.7 B-Trees

87 92 97 89 93 98 90 95 99

Figure 4.64 lnscrlion of 5:"5 il1lo the B-tree ill Figure 4.6:) causes a split into two leavcs

II II

Figure 4.65 Insertion or 40 into the B-trce in Figure 4.64 causes a split inlO two leaves and then a split of the parent l10de

leaves with thlTe items each. 'I\vn disk accesses arc rcqui rcd to write the~c leaves, and a third disk access is required to update the parent. Note that in the pmellt, hoth keys and br:mehes change, but they do so in a com rolled way that is casily calculated. The resulting B-IH'C is shown ill Figure 4.64. Although splitting nodes is time-consuming because it requires at least two additional disk writes, it is a relatively rare occurrence. If Lis 32, for cA<lmplc,

then when a node is splil, l \\'0 leaves \vit h 16 and I 7 itellls, respect 1\,C\)', arc created. hlr the leaf with 17 items, \VC can perform 15 more insertions without another split. Put another way, for evcry split, there arc roughly Lj2 nOllsplits.

The node splitting in Lhe previous example \vorkcd because the parent did nol have its full complement of children. But what would bappen if it did? Suppose, for example, that we insert 40 into the B-tree in figure 4.64. We must split the leaf containing the keys 35 through 39, and now 40, into two leaves. But doing this would give the parent six children, and it is allowed only five. The solution is to split the parent. The result of this is shown in Figure 4.6'5. \rVhcn the parent is split, w(' must update the values of the keys and also the parent's parent, thus incurring an adclitioll<lltwo disk wriLes (so this inserti(ln costs flve disk writes). However, once again, the keys change ill a very controlled manner, although the code is certainly not simple hecause of a host of cases.



11,4 Chapter 4 Irees

\Vhen a non leaf node is split, as is the case here, ils parent gains a child. What if the parcnL already has rC<-lched ils limit of children? Then we continue splitting nodes up the tree Unlil either we find a parent thai docs not need to be split or we reach the rooL If we Splillhc root, then we have Iwo roots. Obviously, this is unacceptable, but we can create a new root that has the split roots as its two children. This is why the root is granted the special two-child minilllurn exemption. It also is the only way that a l1-trcc gains heighl. Needless to say, splitling all the way up to the root is an exceptionally rare event. This is because a trce with four levels indicates thaL the root has been split three Limes throughout the entire sequence of insertions (assuming no deletions havc occurred). In fact, the splitting of any nonleaf node is also quite rarc.

There arc other \vays to handle the overflowing of children. Ow: technique is to put a child up for adoption should a neighbor havc room. To insert 29 into the B-tree in Figure '+.6'5, forcxalllplc, we could make room by moving 32 to the next leaf. This technique requires a modification of the parellt, because the keys arc affected. However, it tends [0

keep nodes fulln and saves space in the long run. V/C can perform ddetion by finding the item that nccds to be removed and then

removing it. The problem is that jf the leaf it was ill had the minimum number of data items, then it is now below the minimum. 'vVe can rectify this situation by adopting a neighboring item, if the neighbor is not itself at its minimum. If it is, then we can combine \vith the neighbor to form a full Iraf. Unfortunately, this means that the parent has lost a child. If this causcs the parent to hill below its minimum, then it rollows the same strategy. This process could percolate all the way up to the root. The roOl cannot have just one child (and even if this were allowed, it vmuld be silly). If a mot is left with one child <lS a result of the adoption process, then we remove the rool and nuke its child the new root of the tree. This is the only way for a B-trec to lose height. ror example, suppose we \val1t 10 remove 99 from the B-tree in Figure 4.65. Since the !cal' has only two items and its neighbor is already at its minimum of three, we comhine the items into a new leaf of five items. As a result, the parent has only two children. However, it can adopt from a neighbor, because the neighhor has four children. As a result, both have three children. The result is shown in Figure 4.66.

I X IX II II 35 38

2 8 18 26 35 38 4 10 20 28 36 39

" 12 22 30 37 40 14 24 31

~ J.2 L~ tl1L.. '-

--~

r

II II 48 51

41 48 42 49 44 50 46 '-- L-

~

54 57 72 78

51 54 57 66 72 78 52 55 58 68 73 79 S3 .'16 S9 69 74 81

70 76 L- '-- '-- -- '--

II II 87 92

1S3 87 84 89 93 85 90 95

97 98

Figure 4.66 B-tree after the delction of 99 from the B-trcc in Figure 4.65



4.8 Sets and Maps in the Standard library

4.8 Sets and Maps in the Standard Library The sri containers discussed io Chapterl, namely vector and 1 i st arc inef[,cielll for search· ing. Consequently, STL provides two addilion,ll containers, set and map, that guarantee logarithmic cosl for basic operations such as inscrlion, deletion, and sC:lrching.

4.8.1 Sets The set is an ordered container that docs not allow duplicates. JvIany of the idioms used to access items in vector a]l(llist also work for a set. Specifically, nested in the set are iterator and const~;terator types that allow traversal of the set, and several mClhods from vector and list arc identically named in set, including begin, end, size, and empty. The prj nt function tcmpLuc descrihed in Figure 3.6 will work if passed a set.

The unique operations required by the set arc the abilities to insert, remove, and perform a basic search (etricicnliy).

The inserl routine is apily named insert. However, beC<l.Use a set cloes not allow duplicates, it is possible for the insert to fail. As a resulL, we want the return type to be able to indicate this with a Doo]c,m variable. Hmvever, insert has a more complicated return type than a bool. This is because insert also returns an iterator that represents where x is when insert returns. This i terator represents either the nc\vly inserted item or the existing item that caused the insert to fail, and is useful, bccausf knowing the position of the item can make removing it more efficient by avoiding the search and getting directly to the node containing the item.

The STL defines a class lClnplate cailed pai r that is little more than a struct v'lith members first "md second 10 access the two itel11s in the pair. There arc two different insert routines:

pair<iterator,bool> insert( const Object & x ): pair<iterator,bool> insert( iterator hint, const Object & x );

The one-parameter insert behaves as described above. The two-parameter insert allows the specification of a hint, which represents the position where x should go. If the hint is accuratc, the insertion is fast, often 0(1). If not, the insertion is donc using the normal insertion algorithm, and perfonns cornpmably wilh the one-paramcter insert. For instancc, the following coele might he fastcr using the t\vo-panuncler insert rather than the 011('

parameter insert·

set<int> s; for( int i " 0: i < 1000000; i++ )

s.insert{ s.end{ ), i );

There arc severa! versions of erase:

int erase( const Object & x ); iterator erase( iterator itr ); iterator erase( iterator start, iterator end );



166 Chapter 4 Trees

The firsl OIlC-P<H,-lJllctcr erase removes x (if found) and returns the number of items actually removed, which is obviously cit her 0 or 1, The second OllC- parameter erase hehaves the same <IS in vector and 1 i st. it iTI110VCS I he object at the position given by the iterator, returns an iterator representing the Clcllll'llllhal followed i tr immediately prior to the caU 10 erase, and invalidates itr, which becumes stale. The two-jJClr,llllC[(T erase behaves the 's<I!11C as in a vector or 1 1st, removing all the items swning at start, up 10 butllOl including the item <:It end.

For searching, rather than a contains rontine that rcLUrns a Boolean variable, the set provides a find routine that returns an iterator representing the iocation of the item (or the end marker if the search fails). This pmvidcs considerahly more information, at no cost in running time. The signature or find is:

iterator find( canst Object & x ) const;

[)y (]eElul\., ordering uses 1 he 1 ess<Object> funCl ion object, which it sci r is implemented by invoking operator< for the Object. An alternative ordering can be specified hy instantiating the set template with a function ohject type. For insIancc, we GHl create a set thm stores s t ri ng objects, ignoring case distinct ions hy using the Case I nsens it i veCompare function ohject coded in Figure 1.22. In the following code, the set s has size 1.

set<string.CaseInsensitiveCompare> s;

s.insert( "Hello" ); s.insert( "Hello R );

cout « "The size is: " « s.size( ) « end';

4.8.2 Maps J\ map is lIsed to store a collection of ordered entries that consists of keys and their values. Keys must he unique, but several keys can map to the same values. Thus values need not be unique. The keys in the map arc maintained in logically sorted order.

The map hehaves like a set instantiated with a pai r, whose comparison function refers only to Ihe key.'") Thus it supports begin, end, size, [inc] empty, but the undcr~ lying itcraLor is a key-value pair. In other words, for an iteratol'itr, *itl' is of type pai r<KeyType, Val ueType>. The map also supports insert, fi nd, and erase. for insert, one must provide a pai r<KeyType, Val ueType> ohject. All hough fi nd requires only a key, the iterator it returns references a pail'. Using only these oprratlons is often not worth\vhill' because the syntactic baggage call be expensive.

rortLlnatcly, the map has an important exIra operation thai yields simple syntax. The illTay-indexing opc-rmor is overloaded [or maps as follows:

ValueType & operator[] ( const KeyType & key);

The semantics of operator[] arc as follows. If key is present in the map, a reference to thc corresponding value is returned. !f key is nol. present in the map, il is inserted with a default value inlO the map and then a reference In the inserted default value is returned. The default value is obtained hy applying a zew-parameter cnnsl ructor or is zero for I he primi! ive types .

. , like ,1 seC ,Ill op1ional tcmplate par:lllWllT c;m he used tn specifY;1 compansun fUllction lh<11 difkrs from

less<KeyType>.




map<string,double> salaries; 2

3 salaries[ "Pat" ] = 75000.00; L/ cout « salaries[ I'Pat" ] « endl;

.5 cout « salaries[ "Jan" ] « end1;

6

7 map<strlng,double>::const iterator itr; 8 itr =' salaries.find( "Chris" ); 9 if( itr == salaries.end( ) )

10 cout « "Not an employee of this company!" <<; end]; I J e1 se 12 cout « itr->second « endl;

Figure 4.67 Acccssin~ values in <l map

These semantics do not allow an accessor version of operator[), so operator[] cannot be used on a map thaI is constant. For instance, if a map is passed by constant reference, inside the routine, operator[] is unusahle.

The code snippet in Figure 4.67 illustrates [\\/0 techniques to access items ill a map. rirst observe Ihat at tinc ), the lefl-hand side invokc5- operator[], thus inserting "Pat" and a double of value 0 inl0 the map, and returning;,\ reference to that double. Then the assignment changes that double inside the map to 7')000. Line 4 outputs 7')000. Unforlunatcly, line ') inserts "Jan" and a salary of 0.0 into the map and then prints il. This Hlay or may not be the proper thing to do, depending on the appliGllion. If it is imponant to distinguish hClWeell items lhat arc in the map ancllbosc not in the map, or if it is important not 10 insert into I he map (because it is immutable), then an altcrnate approach shown at lines 7 10 12 can be used. There we scc a call to fi nd. If thc key is 110t found, the i terator is the endmmker and can he teslf.d, If' the key is found, we caIl access the second item in the pair referenced by 1 he i terator, which is the value associated with the key \Vc could also assign to itr->second

if, instead of a const_iterator, itr is an iterator.

4.8.3 Implementation of set and map C++ requires that set and map support the basic insert, erase, and find uperations in logarithmic worst-case timc. COllsequently, the underlying implementation is a balanced binary search tree. 'I)-'pically, an AVL tree is nol used; instead, top-down red-black trees, which are discussed in Scction 12.2, arc often used.

An important issue in impklllcll!ing set and map is providing support for the ileralor

classes. Of course, internally, the itcrator maintains a puinter to the "current" node in the iteration. The hard part is effiCiently acl\'ancing 10 the next node. There ,He sC\Tral possible solutions, some of which are listed here:

1. \\then the iterator is conslruclcd, have each itcraLm store ,15 its data an array containing the set items. This doesn't work: it makes it impossible to cflicicntly implcmelll any of the routines thaI return an itcrator aftCl" !llOdif)ling the set, such as some of the versions of erase and insert.



168 Chapter 4 Trees

2, J-Ja\'(: the itcrator mainLain a slack storing nodes on the path to the current node. With this information, onc can deduce the next node in the iteration, which is either the node: in the current node's right subtree that contains the minimum item, or the nearest ancestor that contains the currcnt node in ils left subtree. This makes the i\cral"or somewhat large, and makes the itcraLOr code clumsy.

3. Have each node ill the search trec store its parent in addition to the children. The itcrator is not as large, hm there is now extra memory required in each node, and the code to iterate is still clumsy.

4, I·-lave each node maintain extra links: one to the next smaller, and one to the next larger node. This takes space, but the iteration is very simple to do, and it is casy to maintain these links.

5. Maintain the extra links only for nodes that have NULL left or right links, by using extra Boolean variables to allow the routines to tell if a left link is being used as a standard hinary search tree left link or a link to the next smaller node, and similarly for the right link (Exercise 4.49). This idea is called a threaded tree, and is Llsed in many of the STL implementations.

4.8.4 An Example That Uses Several Maps Many words arc similar to other words. For instance, by changing the first letter, the \vonl vline can become dine, fine, line, mine, nine, pine, or vine. By changing the third letter, wi ne can become \'Ii de, wife, l'Ii pe, or \'Ii re, among others. By changing the fourth letter, w; ne can hecome \'Ii nd, wi ng, wi nk, or wi ns, among others. This gives IS different words th8t can he obtained by changing only one letter in \'I; ne. In fact, there arc over 20 different words, some more obscure. We would like to write a program to find all words that can 1)(' changed inlo at least IS other words by a single one-character substitution. We assume that we have a dictionary consisting of approximately 89,000 different words of varying lengths. Most words are between 6 and 11 chclractcrs. The distribution includes 8,205 six~letter words, 1.L ,989 seven-IcLter words, 1.1,672 eight ~letter words, 1:"3,014 nine-letter words, 11,297 ten-letter words, and 8,617 eleven-letter words. (J n reality, the most changeable words are thIT!.>, four- and hve-kuer words, hut the longer words are the time-consuming nnrs [0

check) The most straightfonvard strategy is to usc a map in which the keys arc words and the

values are vectors containing the \\lords I hat can be changed from the key v'lith a onecharacter substitution. The routine in figure 4.68 shmvs how the map that is eventually produced (we have ycllo \vrite cocle for that part) can be used to print the required answers. The code uses a const_iterator to step through the map and views entries that are pairs consisting of a word and a vector or words. The constant references at lines 8 and 9 arc used to replace complicated expressions and avoid making unneeded copies.

The main issue is how to construct the map from an array that contains the 89,000 worcls. Tbe routine in figure 4.69 is a straightforward function to test if two characters are identical except for a one-character substitution. Vie can llse the routine to provide the simplest algorithm for the map constrLlClion, which is a brute-force test of all pairs of words. This algorithm is shown in figure 4.70.




void printHighChangeables( canst map<string,vector<string> > & adjWords, 2 i nt mi n\·/ords = 15 )

:3 "t map<string,vector<string> >: :const_iterator itr;

5 6 for( itr '" adjWords.begin( ); itr !" adj\~ords.end( ); Hitr

7 ( 8 canst pair<string,vector<string> > & entry'" *itr;

const vector<string>

if( \'Iords.size( ) >= min\~ords

{

& words'" entry. second; '!

10 11

12

U

14

15

16 17

18

19

20

cout« entry.first«" (" «\'Iords.size( ) «"}:";

fore int i '" 0; i < words.size( ); ;++ ) cout « " " « words[ i ];

cout « endl;

Figure 4.68 Given a map containing words as keys and a vector of words that diffcr in only one character as values, output words that have min\~ords or more words obtainable by a one-character substitution.

/ / Returns true if wordl and word2 are the same 1 ength

2 II and differ in only one character.

3 bool oneCharOff( canst string & wordl, canst string & word2

4 I 5 if{ wordl.length( ) !~ word2.1ength{ ) )

6 return false;

7

8 int diffs '" 0;

9 10 for( int i '" 0; i < wordl.length( ); i++ )

11 if{ wordl[ i 1 !~ word2[ i 1 ) 12 if{ ++diffs > I )

13 return false;

14

15 return diffs == 1;

16

Figure 4.69 Routine to check if two words differ in only on(' character



IJU lnapter 4 I rees

II Computes a map in which the keys are \'lords and values are vectors of words 2 /1 that differ in only one character from the corresponding key. 3 1/ Uses a quadratic algorithm.

Lf map<string,vector<string> > computeAdjacenUJords( canst vector<string> & words)

'j {

6 map<stri ng. vector<stri ng> > adjHords;

7 8 for( inti '" 0; i < words,size( ); i++ )

\) for( int j ;0 i + 1; j < words.size( ); j++

10 if( oneCharOff( words[ i ], l'Iords[ j 1 ) ) II {

11

J3 1,1

J5

adjNords [ words [ ;

adjWords [ words [ j

lh return adj\·jords;

17

] .push_back( \'Iords[

] .push_back( words[ i

1 ); 1 );

Figure 4.70 f'unctinn to compute a map containing words as keys and a vector of words thal eli/fer in only one character ,15 values. This vCI'sion runs in 6.5 minutes on an 89,000-word dictionary.

Jf we find a pairofworcis thal dilfcr in only one dWraC[(T, we can update the map allitKS

12 and 13. The idiom we arc using at line 12 is that adj [strJ represents the vector of words that arc identical to str, except for one character. Jfwc have previously seen str, then it [s in the map, and \VC fll'cd only add the ne\\' word to the vector in the map, and we do this by calling push_back. If we have never seen str before, tilen the act of using operator£] places it in the map, with a vector of size 0, and returns this vector, so the push_back updates the vector to be size 1. All in all, a super-slick idiom for maintaining a map in which the value is a collection.

The problem with t·his <.Ilgorithm is thai it is slow, and takfs 6.'5 minutes on our cornpulfr. An obviolls irnpnwcmcnt is to avoid comparing words of different lengths. We can do this by ,grouping words by their length, and thul rullning the previous algorithm on each of lhe separate groups.

To do this, we can usc a second map! llcrc the key is an integer representing a \\'ord length, and the value is a collection of alllhf words of that length. V.fc can usc a vector to store each collection, and the same idiom applies. The cock is shown in Figure 4.71. Line 8 shows the declaration for the second map, lines] I and 12 populate the map, and then an extra loop is useclto iterate over each group of words. Compared to the first algorithm, the ::iccond algorithm is only marginally more difficult to code and rllns in 77 seconds, or about six times as fast.

Our third <1lgorithm is more COIllI)\cx, and uses additional maps! As before, we group the words by word length, and then work on each group separately. .ji:) sec how thi5 algorithm works, suppose we arc working on words or length 4. First wC' wall I to lind word pairs




II Computes a map in which the keys are words and values are vectors of words 2 II that differ in only one character from the corresponding key. 3 II Uses a quadratic algorithm, but speeds things up a little by

4 II maintaining an additional map that groups words by their length. 5 map<string,vector<string>:> computeAdjacentWords( canst vector<string> & words) 6 { 7 map<string,vector<string> > adjWords; 8 map<; nt. vector<s tf; n9> :> wordsByLength;

9

10 /1 Group the words by the; r 1 ength

lJ fore int i '" 0; i < words.size( ); i++ )

12 wordsByLength [ words [ i ].1 ength ( ) ] . push_back ( words [ i ] ); 1.3

14 / / Work on each group separately 15 map<int,vector<string> >::const_iterator itr; 16 fore itr '" wordsByLength.begin( ); itr 1= wordsByLength.end{ ); ++itr )

17 {

18

19 20

21 22 23

24 25 26 27 28

canst vector<string> & groupsWords '" itr·>second;

for( int i ~ 0; ; < groupsWords.size{ ); i++ }

for( int j '" i + 1; j < groupsWords,size( ); j++ )

if( oneCharOff( groupsWords[ i ], groupsWords[ j ] ) ) {

adjWords[ groupsWords[ i ] .push_back( groupsWords[ j ] ); adjWords[ groupsWords[ j ].push_back( groupsWords[ i ] ),

29 return adjWords;

30

Figure 4.71 Function to compute a map containing words as keys and a vector of words that differ in only one character as values. It splits words into groups by word length. This version runs in 77 seconds on an 89,OOO-word dictionary.

such as wine and nine that are identical except for the first letter. One way to do this is as follows: for each word of length 4, remove the first character, leaving a three-character word representative. Form a map in which the key is the representative, and the value is a vector

of all words that have that representative. For instance, in considering the first character of the four-letter word group, representative "i nell corresponds to "di ne", "fine", "wi ne",

"nine", "mine", "vine", "pine", "line". Representative "oot" corresponds to "boot", "foot",

"hoot", "loot", "soot", "zoot". Each individual vector that is a value in this latest map forms a clique of words in which any word can be changed to any other word by a one-character suhstitution, so after this latest map is constructed, it is easy to traverse it and add entries

171



".I. Ll1d!.llel 't lIee~

to the original map that is being computed. Vic would then proceed to lh~ second character of the four~lcltcr word group, with a new map, and then the third character, (md finally the fourth character.

The general outline is:

for each group g, containing \'lords of length len for each position p (ranging from 0 to len-I)

Make an empty map<string,vector<string> > repsTo~Jords

for each word w

Obtain w's representative by removing position p Update repsToHords

Use cliques in repsTo~Jords to update adjWords map

Figure 4.72 contains an implementation of this algorithm. The running lime improves to flve seconds. It is interesting to note lhat although the use of the addiLional maps makes the algorithm bster, and the sylllax is relatively clean, the code makes no lise of the fact that the keys of the map arc maintained ill sorted order.

As such, it is possible that a data struclure thai supports the map operations but docs not guarantee sorted order can perforrn better, since il is being asked to do less. Charter '5 explores this possibility. Some C++ lihraries include such a class, known as a hash _map, but it is not part of Standard C++.

1 I I Computes a map in whi ch the keys are words and values are vectors of words 2 II that differ in only one character from the corresponding key .

.3 II Uses an efficient algorithm that is O(N log N) with a map. 4 map<string,vector<string> > computeAdjacenUlords{ const vector<string> & words)

5 ( () map<string,vector<string> > adj\,jords;

7 map<int,vector<string> > wordsByLength;

R 9 I I Group the \'lords by thei r 1 ength

10 for{ int i = 0; i < \'lords.size( ); i++ )

11 wordsByLength [ words [ i 1.1 ength ( ) 1. push_back ( words [ i 1 );

Figure 4.72 Function to compute a map containing words as keys and a vector of words that differ in only one character as values. This version rllns in '5 seconds on an [19,000 word dictionary.



/2

Ll

14

/5 / (,

17

IH

10

20 21

22 2.1

24 25 26 27

28

29 .10

31

32 33

31 35

36 37

38

39 '10 11

12 13 4'1

'15

46 17

It.ll )eI~ ana [vlaps In me )lanOara Llorary

1/ 110rk on each group separately

map<int,vector<string> >::const_iterator itr; fore itr "" l'lordsByLength.begin{ ): itr !o= l'lordsByLength.end( }; ++itr)

{

canst vector<string> & groupsl1ords '" itr->second:

int groupNum = itr->first;

/1 Hark on each pos it i on in each group fore inti '" 0; ; < groupNum; ;++ ) {

II Remove a character in given position, computing representative. /1 I-lords with same representatives are adjacent; so populate a map

map<stri ng. vector<s t r; ng> > rep T oHord;

fore int j '" 0; j < groupsHords.size( ); j++ )

{

str; ng rep'" groups\>iords [ j ];

rep.erase( i, 1 );

repToHord[ rep J.push_back( groupsHords[ j J );

/1 and then look for map values with more than one string

map<stl'ing,vector<string> >: :const_iterator itr2:

for( itr2 '" repToHord,begin( ); itr2 != repTol1ord.end( ); Hitr2

{

const vector<string> & clique ~ itr2->second; if( clique.size( ) >0 2 )

fore lnt p := 0: P < clique.size( ); p++ )

fore jnt q := p + 1; q < clique,size( ); q++ ) {

adjHords[ clique[ p J.push_back( clique[ q J );

adjflords[ clique[ q J.push_back{ clique[ p J );

48 return adj\·jords; efl)

Figure 4.72 (continued)

II.)



II ..

Summary

We have seen uses of trees in operating systems, compiler design, and searching. Expression trees arc a small example of a more general structure known as a parse tree, which is a central data structure in compiler design. Parse trees arc not binary, but arc relatively simple extensions of expression trees (although the algorithms to build them are not quill' so simple).

Search Lrees arc of great importance in algorithm design. They support almost all the useful operations, and the logarithmic average cost is vcry smalL Nonrecursivc implementations of search trees are somewhat faster, but the recursive versions arc sleeker, more elegant, and casier to understand and debug. The problem 'with search trees is that their performance depends heavily on the: input being random. If this is not the case, the running time increases Significantly, to the point whnc search trees become expensive linked lists.

We sawsc\,cral ways to deal with this problem. AVL trees work by insisting that all nodes' left and right subtrees differ in heights by at most one. This ensures that the tree cannot get too deep. The operations that do not change the tree, as insertion does, can all use the standard binary search tree code. Operalions that change the tree must restore the tree. This can he somewhat complicated, espeCially in the case of deletion. We showed how to restore the tree after insertions in O(\og N) time.

We also examined the splay tree. Nocles in splay trees can get arbitrmily deep, but after every access the tree is adjusted in a somewhat mysterious manner. The net effect is that any sequence of M operations takes OeM log N) time, which is the same as a balanced tree would take.

B-trees arc balanced M~way (as opposed to 2~way or binary) trees, which are well suited for disks; a special case is the 2~3 treC" (M = 3), which is another way to implement balancecl search trees.

In practice, the running time of all the balanced tree schemes, \vhile slightly faster for searching, is worse (by a constant factor) for insertions and deletions than t.he simple binary search tree, hut this is generally acceptable in view of the protection being given against easily obtained worst~case input. Chapter 12 discusses some additional search tree data structures and provides detailed implementations.

A final note: By inserting elements into a search tree and then performing an inorder traversal, we obtain the clements in sorted order. This gives an O(N log N) algorithm to son, which is a worst-case bound irany sophisticated search tree is used. 'vVc shall sec better ways in Chapter 7, but none that have a lower time bound.

Exercises

Questions 4.1 to 4.3 reIer 10 tlJe I fee ill Figure 4. 7.3.

4.1 for the tree in Figure 4.73: a. 'vVhich node is the root? b. vVhich nodes arc leaves?

4.2 For each node in the tree of Figure 4.73: a. Name the parent node.



Exercises

B

]) E

G II

Figure 4.73 Tr(:'e for rXl'rcises 4.1 to ~t.1

b. I,ist the children.

c. List the siblings. d. Compute the depth. c. Computc the height.

J

L

4.3 \IVhm is the depth of the tree in hgure 4.7:.)]

c

F

K

M

4.4 Show that in a hinary tree of N nodes, thert' arc N + I NULL links representing children.

4.5 Show that the maximum number of nodes in a hinary [ree of height h is 2h+ 1 - 1.

4.6 AIull node is a node with two childrcn. Prove thai the number of fut! nodes plus one is equal to the numher of leaves in a nOl1empty binary tree.

4.7 SUPI)oSe a binary trec has leaves i l , 12) ... ,1,\[ at depths ell, d), , _ . ,d,\/, fcsprcti\'ely.

Prove thaL L;~cl 2 A::s .1 and determine \vhcn lhe equality is true.

4.8 Cive the prefix, infix, and postfix expressions corresponding to [ill' nee ill Figure 4.74.

4.9 a. Show the result of inserting 3, I) 4, 6, 9, 2, ), 7 into an initially empty binary search I ree.

h. Show the result of deleting the root.

4.10 LClI(N) he the average numher of full nudes in an N-nodl' binary search tree. a. Determine the values 011(0) and(O) h. Show 1hat for N > 1

,\/-1

N-2 I L fiN) = -- + - (f(1) +(N - i-I» - N N

jc--=O

175



176 Chapter 4 Trees

e

" +

a b c d

Figure 4.74 Tree for Exercise 4,B

c. Show (by induction) that feN) = (N ~ 2)/3 is J solmioll to the equation in part (h), with the initial conditions in pan (a).

d. Usc the results of Exercise 4.6 to determine the average number of leaves in a

binary search tree.

4.11 \Vrile an implementation of the set class, with associated iLcrators using a binary search tree. Add to each node a link to the parent node.

4.12 Write an implementation of the map class by storing a data member of type set<Pair<KeyType.ValueType> >.

4.]3 vVritc an implementation of the set class, with associated itcrators using a binary search tree. Add to each node a link to the next smallest and next largest node. To make your code simpler, add a header and tail noele which arc not part of the binary search tree, but help make the linked list pan of the code simpler.

4.14 Suppose you want to perform an experiment to veril)! the prohlems thaI can be caused by random i nsert/remove pairs. Here is a strategy that is not perfectly random,

but close enough. You huild a tree with N elements hy inserting N dements chosen at random from the range "1 to M = aN. You then perform 1\1 2 pairs of insertions followed by deletions. Assume the existence of a routine, randomlnteger( a. b), which returns a uniform random integer between a and b inclusive. <:l. Explain how to generate a random integer between .1 and IVJ that. is not already

in the tree (so a random insertion can be performed). In terms or N and a, what is the running time of this operation?

h. Explain how to generate a random integer between 1 and Jvl that is already in the trce (so a random deletion can be performed). vVhat is the running time of this

operation? c \VhaL is a good choice of a? vVhy?

4.15 vVrite a program to evaluate empirically the following strategies for removing nodes

with two children: <t. Replace with the largest node) X, in 1't and recursivcJy remove X.



Exercises

b. Alternately replace with the largest node in T/. and the smallest node in Tn, and recursively remove the appropriate node.

c. Replace with either the largest node in Tl. or the smallest node in TN (recursively removing the appropriate node), making the choice randomly.

vVhich strategy seems to give the most balance? vVhich takes the least CPU time to process the entire sequence?

4.16 Redo the binary search tree class to implement lazy deletion. Note carefully that this affects all of the routines. Especially challenging arc fi ndMi nand fi ndMax, which must now be done recursively.

** 4.17 Prove that the depth of a random binary search tree (depth of the deepest nocle) is O(log N), on average.

4.18 * a. Give a precise expression for the minimum number o/" nodes in an AVL tree of height h.

b. What is the minimum number o/" nodes in an AVL tree of height 1 '5?

4.19 Show the result of inserting 2, 1,4,5,9,3,6,7 into an initially empty AVI- trec.

* 4.20 Keys 1,2,. " 211 ~ I are inserted in order into an initially empty AVL tree. Prove that the resulting tree is perrectly balanced.

4.21 Write the remaining procedures to implement AV! single and double rotations.

4.22 Design a linear-lime algorithm that verifies that the height information in an AVL tree is correctly maintained and that the balance property is in order.

4.23 vVrite a non recursive function to insert into an AVI tree.

* 4.24 How can you implement (nonhlzy) deletion in AV! trees?

4.25 3. I low lTlany bits arc required per node to store the height of a node in an N-node AVL tree?

h. What is the smallest AVL tree that overflO\vs an 8-biL height counter?

4.26 V\Trite the functions to perform the double rotation without the inefficiency of doing two single rotations.

4.27 Show the result of accessing the keys 3, 9, !, 5 in order in the splay tree in Figure 4.75.

4.28 Show the result of deleting the clement with key 6 in the resulting splay tree for the previous exercise.

4.29 a. Show Ihat ifall nodes in a splay tree are accessed in sequential order, the resulting tree consists of a chain of left children.

** b. Sho\\' that if all nodes in a splay tree arc accessed in sequential order, then the total access time is O(N), regardless of the inilialtrcc.

4.30 vVrite a program to perform random operations on splay trees. CounL the lOtal number of rotations performed over the sequence. How docs the running time compare to AVL trees and unbalanced binary search trees?

4.31 Virile efficient func! ions t hat take only a pointer to the root of a binary tree, T, and compute: a. The number of nodes in 1. b. The number of leaves in T.



178 Chapter 4 Trees

10

4 II

2 6 12

3 5 8 13

7 9

Figure 4.75 Tree for Exercise 4,27

c. The number or full nodes in 1. \\lha1 is the running lime of your routines?

4.32 Design a recursive linear-lime algorithm that tests whClhcr a billary tree satisfies lhe

search tfec order properly at evcry node.

4.33 \Nritc a rt.'cursivc function that takes a poil11crlo the root node ofa tree T and returns a pointer to the root node of the ncc that results from removing all leaves from T.

4 .34 \~lritc a function to generate an 1\1 -llode random binary search tree with distinct keys 1 through N. \Vhat is the running lime of your routine?

4,35 \VriLl' a [unction \0 generate the i\VL lrce of height II with fewest nodes. \\111<11 is the

running time of your function?

4.36 \,\frill' a function to generate (\ per[(~ctly halanced binary search tree of height 11 with keys I through 2h-i-1 - L \A/hat is the running time of your function?

4.37 \\fritc a function that takes as input a binary search Iree, T, and two keys hI and h2' which arc ordered so that h1 :S 1:1 2 , and prints all clements X in the \tTl' such that III ::s f«(V(X) :S h2. Do nol assume any information ahout the type of keys except that they can be ordered (consistcntly). Your program should run in 0(1< + log N) average time, v\'hcrc I< is the number of keys printed. Bound the running time or your algorithm.

438 The larger hi nary trees in this chapter \vere generated automatically hy a program. This was done hy assigning an (x) y) coordinate to each trec node, drawing a circle around each coordinatc (this is hard to sec in some pictures), and connecting each node to its parent. Assumc you have a binary search liTe stored in memory (perhaps generated hy onc of lhe routines above) and that ('arh node has two eXIra fields to

Slore the coordinates. a. The x coordinate can be computed hy assigning the inorder traversal numher.

\,\1rit(' a routinc_ to do this for ('etch node ill the tree.



Exercises

h. They coordinme can be computed by using the negative of the depth of the node. \"'rite a routine to do this for each node in the tree.

c. In terms of some imaginary unit, what will the dimensions of the picture he? How can you adjust the units so that the tree is always roughly two-thirds as high as it is wide?

d. Prove that using this system no lines cross, and that for any node, X, all clements in X's left subtree appear to the left of X and all clements in X's right subtree appear to the right of X.

4.39 \Vrite a general-purpose tree-drawing program that will convert a tree into the following graph-assemhler instructions: a. Cirde(X, y) b. Dra-l,vLine(i,j) The first instruction draws a circle a1 (X, Y), and the second instruction connects the ilh circle to the jth circle (circles are numbered in the order drawn). You should either make this a program ano define some sort of input language or make this a function that can \.)e called from any program. vVhat is the running time of your routine?

4.40 Write a routine to list out the nodes of a binary tree in Ic\'e1~ordCf: List the root, then nodes at depth 1, followed by nodes at depth 2, and so on. You must do this in linear time. Prove your time bound.

4.41 * J. Write a routine to perform insertion into a B-tree. * h. Write a routine to perform deletion from a 8-tree. \Vhen an item is deleted, is it

necessary to update information in the internal nodes? * c. Modify your insertion routine so thm if an altempl is made to adel into a node

that already has M entries, a search is performed for a sibling with less than M children before the node is split.

4.42 A R*-iree of order M is a 8-tree in which each interior node has between 2M/3 and M children. Describe a method to perform insertion into a B*-trcc.

4.43 Show how the tree in figure 4.76 is represented using a child/sibling link implementation.

4.44 Write a procedure to travcrse a tree stored with child/sibling links.

B

E

H J

o R

Figure 4.76 Tree for Exercise 4.43

179



IOU Lnapler 4 Iree~

A A

B c c D E G G

F H H

Figure 4.77 l\vo isomorphic trees

4.45 T"vo binary trees arc similar if they arc both empty or both l10ncmpty and have similar left and right subnees. Vv'rite a function to decide whether \\\'o hinary trces arc similar. vVilat is the running lime ofyoLlr fUllction?

4.46 Two trees, 1'] and 12> arc isomorphic if T j can be transformed into T2 by swapping left and right children of (some of the) nodes in II' For instance, the two trees in figure 4.77 arc isomorphic hecause they arc the same if the children of A, H, and G, but not the other nodes, afC swapped. a. Give a polynomial time algorithm to decide if [wo trees arc isomorphic.

* b. vVhat is the running time of your program (there is a linear solution)?

4.47 * 3. Show that via AVL single roLations, any binary search tree II can be transfonlKd into another search tree T2 (with the same itcms)_

* b. Give an algorithm to perform this transformation using O(N log N) rotations on average.

**. c. Show that this transformation can be done with O(N) rOiations, worst-casco

4.48 Suppose we want to add the operation findKth to our repertoire. The operation findKth(k) returns the kth smallest item in the tree Assume all items arc distincl. Explain how to modify the binary search tree to support this operation in O(log N) average time, without sacrificing the time bounds of any other operation.

4.49 Since a binary search tree with N nodes has N -+ I NULL pointers, half the space allocated in a binary search tree for pointer informaLion is wasted. Suppose that if a node has a NULL left child, we make its left child link 10 its inorder predecessor, ;md if a node has a NULL right child, we make ils right child link to its inordcr succeSSOL This is known as a threaded tree and the extra links arc called thrcads. a. l--low can we distinguish threads from real children pointers? h. vVrite routlnes to perform inserlion and deletion into a tree threaded in the

manner described abovc. e. What is the advantage of using threaded trees?

4.50 Write a program that reads a C++ source code file and out pUIS a list of all identifiers (that is, variable names, but no\ keywords, that are not founci in comments or string constants) in alphabetical order. Each identifier should be output with a list of line numbers on which it occurs,

4.51 Generate an index for a hool<. The input file consists of a set of index entries. Each line consists of the string IX:, followed by an index entry name enclosed in braces,



IX, (Series I () (2)

IX, {Series!geometricl (} (4)

IX, {Euler's constant} (4)

IX, {Series!geometricl)} (4)

IX, {Series!arithmeticl () (4)

I X, {Series!arithmeticl)} (5)

IX, {Series!harmonicl (} ( 5)

IX, {Euler's constant} (5)

IX, {Series!harmonicl)} (5)

IX, (Seri es I ) } (5)

Figure 4.78 Sample input for Exercise 't.51

Euler's constant: 4, 5 Series: 2-5

arithmetic: 4-5 geometric: 4 harmonic: 5

Figure 4.79 Sample OUi put for Exercise 4.51

References

followed by a page numher that is enclosed in hraces. Each! in an index entry nJme represents a sub-leveL A I ( represents tlK starl of a range, and a I) represents the end of the range. Occasionally, this range will be the same page. In that case, output only a single page number. Otherwise, do not collapse or expand ranges on your own. As an example, figure 4.78 shows sample input and Figure 4.79 shows the corresponding output.

References

More information Oil binary search trees, and in parliclliar the mathematical properties of trees, can be found in the two books by Knuth, !22j and 1211.

Several papers deal with the lack of balance caused hy hiased deletion algorithms in hinary search trees. I-libhard's paper [191 proposed the original deletion algorithm and cs!"ablishedthat one deletion preserves the randomness of the trees. A complete analysis has been performed only for trees with three nodes 120) and four nodes 15). Eppinger's paper r 14J provided early empirical evidence of nonrandomncss, and the papers by Culberson and Munro I] OJ, ! 11] provided some analytical evidence (hut not a complete proof for the general case of intermixed insertions and deletions).

AVI. trees were proposed hy Adclson-Velskii and Landis [II. Simulation results for AV[

! rees, and variants in which the height irnbalance is allowed to be at most I? for various values of h, afC presented in [211. A deletion algoritlull for AVI, trees can be found in [23].



182 Chapter 4 Trees

Analysis of the average search cost in AV! trees is incomplete, but S(Hne results arc contained

in 1241 [3] and IS] considered self-adjusting trees like the type in Section 4.5.1, Splay trees arc

described in 1281 R-trces first appeared in [6], The implemenlalion described in the original paper allows

data to be stored in internal nodes as well as leaves, The data structure we have described is sometimes known as a n+ -tree. A survey of the different types of B-trees is presented in !9]. Empirical results of the various schemes are reponed in [J 7]. Analysis of 2-3 trees and B-lrees can he fonnel in 14 J, 1 Ll J, and [121.

Exercise 4.17 is deceptively difficult. A solution can be found in [15]. Exercise 4.29 is from [32], Information on 8*'-lrcc5, described in Exercise 4.42, can be found in [121. Exercise 4.46 is from [2]. A solution to Exercise 4.47 using 2N - (1 rotations is given in [29]. Using threads, a la Exercise 4.49, was first proposed in [271. f?-d trees, \vhich handle mulliclimcnsional data, were first proposed in [7] and are discussed in Chapter 12.

Other popular balanced search trees are red-black trees [IS] and weight-balanced trees 126J. Ivlme halanced tree schemes can be found in Chapter 12, as well as in lhe books [161,

[2~1, ancll301.

1. G. M. Adelson-Velskii and E. M. LandiS, "An Algorithm for the Organization of Information,"

Soviel. Mal. Dol1l<"ly" (1962), 125L)~ 1263.

2. A. V Aho, J E. Hopcroft, andJ D. Ullman, The Desi(~11 (llld Analysis (?{ Computer AIg,orii/um, Acldisol1~\i\lcslcy, Reading, Mass., 1974,

3. B. Allen and J L tv\unro, "Self Organizing Search Trel's," .Journal of the ACM, 2'5 (1978), '526~'51'5.

4. R. A Baeza-Yates, ''Expected Behaviour of nl'-trees under Random Inscr1ions," Ada ft~[orl1lali((l, 26 (J 989), 4J9~471.

5. R. A Baeza-Yates, "A Trivial Algorithm \Vhosc Analysis bn't: A Continuation," nu; 29 (1989), 88~113.

6. R Bayer and E. M. McCreight, "Organization and tvlailltcnance of Large Ordered indices," Ac/a h~j(JrnJ(jtiw, 1 (1972), IT3~189.

7. J. L Benile)" "Multidimensional Binary Search Trees Used for Associative Searching," Commul1ica/ions q[ lhe AClvt, 1 H (197.'5), '509~.'517.

8. J. R. Bitner, "Heuristics 1h,11 Dynamically Organize Data Structures," SIAJv! journal on C0Il1Fulillf!,,8 (1979),82-1 10.

0. D. Comer, "The Ubiquitous BArel'," CompulingSurveys, 11 (1979), 121-117.

10. J. Culberson and j. I. Munro, "Fxp!aining the Behavior of Him1ry Search Trees uneler Prolonged Updates: A l\llodel and Simulations," COHlJ)[{tcrJo[Jr/wl, .12 (1989), 68~7,).

II. j. Culberson and J 1. Munro, "Analysis of the Standard Delction Algorithms in Exact Fii Domain Binary Search Trees," AlgoritiJmiC(!,.'5 (1990) 29'5~J11.

12. K Culik, T Ottman, and D. vVond, "Dense Multiway Trees," ACM Jl'aI1SClClioIlS 0/1 Data/JOse Syslems,6 (1981), 486~512.

1). B. Eist:nbath, N. Zivian<1, C;. H. Connet, K. ivlelhorn, and D. vVood, 'The Theory of Fringe Ana!ysis and Its Application to 2~ 3 ]rces and B-trees," [n{ormalion (llld Control, ."55 (.I 9H2), 125~174.

l4. J L Eppinger, "An Empirical SLUdy of Insertion and Deletion in Binary Search Trees," Communications oj the ACi\r1, 26 (19H3), 663-669.



References

J '5. P Fl(~jolcl and A. Odlyzko, 'The ;\\Tragc Ilcighr of Binary Trees and Other Simple Trees:' .lou rna! o[ Computer and System Scicnces, 2') (] (82), t 7] -21 ~3.

16. C;. II. C;onnCl and R. Baeza-Yales, llalldboo/? o[Alp,oritlnm diU/ now SlnK/urc:;, 2d ccL, Addisoll-\Ves!cy, Heading, lvbss., ! 99! .

17. E. C;ucks and S. TSlIf, "Expnlmcnts with I)-liTe Reorganization," Prou:cdill,O;;s ufil(,'M S]GMOn S)'ll1posiu1l1 0/1 Ivlal1agcIllCl11 (~r Della (1980), 200-206.

18. L J. Cuibas and J( Scdgnvick, "}\ Dichromatic Framework for Balanced Trccs," Procc't'(lill,(C:s of the Nil1etccnl/1 AIllluallEEE Symposium on FOlJndations (~f ComFliter Science (1978),8-21.

19. T H. Hibbard, "Some C:ornbinaLOrial Properties of Certain Trees with l\pplicalions 10 Searching '-lnd Sorting," JOUr/wi oj Ihe ACM, 9 (1962), L1-28.

20. A. T Jonassen alld D. E. Knuth, "A Trivi("d Algorithm \Vhosc Analysis Isn't," Journal of Computer (/lui System Scicnces, 16 (1978), :30 !~_)22.

21. P L Karllon, S. H. Fuller, R_ [~_ Scroggs, and E. B. Kaehler, "Performance of Ilcight H'l.lanced Trees," CO/11l1wniw/ions of the ACM, Il) (! 976),23--2.8.

22. D. E. Knuth, The Arl cd Computer Progrmnming,: Vol. J: hmdwnentul A!gori!lllns, -_:)d c(L, i\ddison-\,Vesley, Reading, Ivlass., ] 997.

2-3. D. F. Knuth, TJ1C Art of Computer Prof.ramminf.: Vol. 3: Sorlill{~ (llld Sc(/rchill,~, 2cl ed_, Addisotl\Nesley, Reading, Ivlass., 1998.

24. K. lvlc1horn, "1\ Partial Analysis of Height-Balanced Trees under Random Insertions and Dclnions," SIAM Journal (~r Computing, 11 (1982), 7415-760.

25. K Melhorn, Da/(J S/mctl1 res and Algmithms l: Sorting (111£1 SC(JrciJillg, Springcr-Vcr!,,{g, Berlin, 19134.

2(i J. Nievcrgclt and E. lv1. Reingold, "Hillary Search Trees of Bounded Balallce," STAiv1jOllnW[ on COl1lpulinp" 2 (! 973), _)3-4:3.

27. A . .I. Perlis and (" Thorntoll, "Symbol i'vianipulalion in Threaded IjSIS," COIl1HwniwliollS of the ACM, ) (J960), 195-204.

2e. D. D. Slcator and R. L Tmjan, hSelf~adjl1S1ing Binar}1 Search Trees," JOUr/wi o/Ihe ACM, 32

(1985), 652-686.

29. D. D. Skalor, R. E. Tlljan, and \V P Thurslon, "Rotation Disrance, Triangulations, and Hyperbolic C;C0111Cl ry," }Ollnwl of the _i\MS (J ge8), 647-682.

)0. 1-1. E Smith, Dato Slnj(:lurn-form and fune/ion, llmrourt Brace .Jovanovich, Orlando, ria., 1987.

] I. R. E. T-11:ian, "Sequential Access in Splay Trees -hlkcs Linear Til1le:~ COlllbill(!IOrica, ') (1915'5), 367-l711

32- A. c:. Yao, "On Random 2--} Trees," Acta 11l/()fI)WliUl, 9 (1978), 1'59-170.

181





CHAPTER 5

Hashing

In Chapter 4, we discussed the search tree ADT, which allowed various operations on a set of clements. In this chapter, we discuss the lIash tahle ADT, which supports only a suhsct 01 the operations allowed by binary search trees.

The implementation of hash tahles is frequently called hashing. I-lashing is a technique used for performing inserLions, deletions, and finds in constant average time. Tree operations that require any ordering information among the clements are not supported efficiently. Thus, operations slIch as findMin, findMax, and the printing or the entire table in sorted orcler in linear time are nol supporlcd.

The central data structure in this chapter is the hash table. \Vc \\.Ii11

Sec several methods or implcrnenting the hash table.

Compare these methods analytically.

Show numerous "pplications of hashing.

Compare hash tables with binary search trccs.

5.1 General Idea The idcal hash table data structure is merely an array of some fixed size, containing the items. As discussed in Chapter 4, generally a search is performed on some part (that is, data member) of the item. This is called the key. For instance, an item could consist of (\ string (that serves as the key) and additional data members (for instance, a name that is part of a large employel' structure). We will refer to the table size as -f(!/.J/cSizf, \vith the understanding that this is pan ora hash data structure and not merely somcvariablc floating around globally. The common convention is to have thc tahle run from 0 to TahleSize - 1; we will sec why shortly.

Each key is mapped into some numher in the range 0 to TahlcSizc - I and placed in the appropriate cell. The mapping is called a hash function, which ideally should he simpk to

computc and should ensure that any two distinct keys get different cells. Since there arc a finite numbcr of cells and a virtually inexhaustihle supply orkeys, this is dearly impossible, and thus we seeka hash function that distributes the keys evenly among the cells. Figure 5.1 is typical ora perfect situation. In this c:xample,joiln hashes to 3, phil hashes to 4, dave hashes to 6, and I11(//Y hashes to 7. 185



, .. ., Lnapler :J Hasmng

o

2

3 john 25000 --~----~~-

4 phil 31250

5 .. ~

6 dave 27500 .~

7 mary 2X200 ----.,---~-----

X

9

Figure 5.1 An ideal hash table

This is the basic idea or hashing. The only remaining problems deal with choosing a function, deciding what to do when L \..\'0 keys hash to the same value (this is known as a collision), and deciding on the tahle size.

5.2 Hash Function If the input keys arc integers, then simply returning }<cy mod TahleSizc is generally a reasonable strategy, unless Key happens to have some undesirable properties. In this case, the choice of hash function needs (0 1x carefully considered. for instance, ir the tahle size is 10 and the keys all end in zero, then the standard hash function is a bael choice. [or reasons we shall sec later, and to avoid situations like the one ahove, it is often a good idea 10 ensure th"l the table size is prime \Vhcn the input keys are random integers, then Ihis fUllClion is not only vcry simple \0 compute but also distributes Ihe keys evenly.

Usually, the keys arc strings; in this case, the hash function needs to be chosen carefully. One option is to add up the ASCll values of the characters in the string. The routine in

Figure 5.2 implements this strategy The bash function depicted in Figure '5.2 is simple to implcmclll and computes an

answcr quickly. However, if the tahle size: is large, the fUllction cloes not distribute the keys vvTl1. For instance, suppose that Ti:I1J/cSizc = "] 0,0070.0,007 is a prime number), Suppose all the keys arc eight or fewer characters long. Since an ASCI! character has an inlt:ger value that is aJways at most 127, the hash function typically can only assume v<tlues between 0 <tncll,OJ6, which is 127 * 8. This is clearly not an equitable distribution!

Another hash runet ion is shown in Figure .5 .3. This hash function assumes that Key h<IS at least three characters. The value 27 represents the numher of letters in Ihe English alphabet, plus the hlank, and 729 is 272. This function examines only the first three characters,



int hash( canst string & key, int tableSize ) 2

3 int hashVal '" 0;

4

5 fore int i '" 0; ; < key.length( ); i++)

6 hashVal +" key[ i J; 7

8 return hashVal % tableSize;

9

Figure 5.2 A simple hash function

lint hash( const string & key, int tableSize 2

5.2 Hash FUnction

3 return ( key[ 0 ) + 27 + key[ 1 ) + 729 * key[ 2 ) ) % tableSi,e;

4

Figure 5.3 Another possible hash function~nOl too good

but if these are random and the table size is 10,007, as bdore, then \ve would expect a reasonably equitable distribution. Unfortunately, English is nOl random. Although there afC 26 3 = 17,576 possible combinations of three characters (ignoring blanks), a check of a reasonably large on-line dictionary reveals that the number of different combinations is actually only 2,851. Even if none of these combinations collide, only 2B percent of the table can actually be hashed to. Thus this function, although easily computable, is also not appropriate if the hash table is reasonably large.

figure 5.4 shows a third attempt at a hash function. This hash function involves all characters in the key and can generally be expected to distribute well (it computes

L:~~SiZf-J Key{KeySize ~ i - 1]· 3i, and brings the result into proper range). The code computes a polynomial function (of 37) by usc of Horner's rule. For instance, anOlher way 01 computing h" = ko + 3711, + 37211, is by the formula h" = ((h,) * 37 + 11,) * 37 + ho. l1orner's rule extends this to an nth degree polynomial.

The hash function takes adval1lage of the fact that overflow is allowed. This may introduce a negative number; thus the extra lesf at the end.

The hash function described in Figure 5.4 is not necessarily the best with respect to table distribution, hut does have the merit. of extreme simplicity and is reasonably fast. If the keys are very long, the hash function will take too long to compute. A common practice in this case is not to use all the characters. The length and properties of the keys would then influence the choice. For instance, the keys could be a complete street address. The hash function might. include a couple of characters from the street address and perhaps a couple of characters from the city name and II\' code. Some programmers implement their hash function by using only the characters in the odd spaces, with the idea that the time saved computing the hash function will make up for a slightly less evenly distributed function.



188 Chapter 5 Hashing

/*' 2 * A hash routine for string objects .

.J '/ 4 lnt hash( canst string & key, int tableSize 5 6 int hashVal '" 0;

7

8 for( ; nt ; := 0; ; < key. 1 ength ( ); ; ++

9 hashVal =: 37 * hashVal + key[ i ];

10

1 J hashVal %'" tableSize;

12 if( hashVal < 0 ) 13 hashVal +"" tableSize;

14 15 return hashVa 1 ;

16

Figure 5.4 A good hash function

The main programming dctail1eft is collision resolution, If, when an ckmenl is inserted, it hashes to the same value as an already inserted element, then we have a collision and need to resolve it. There are several llWlhods for dealing with this. We will discuss two 01 the simplest: separate chaining and open addressing.

5.3 Separate Chaining The first strategy, commonly known as separate chaining, is to keep a list of all eleillents that hash to the same value \Ve can usc the standard lihrary list implementation. If space is tight, it might be preferahle to avoid their lIse (since these lists arc doubly linked and waste space). \}.,Ie assume for this section Ihal the keys <-Irc the first 10 perfect squares and that the hashing runction is simply Iwsh(x) = x mod .10. (The tahle size is not prime hUI is used here for simplicity.) rigure 5.5 should make this clear.

To perform a search, \VC use the hash function to determine which list to traverse. \Vc then search the appropriate list. To perform an insert, we cbrek the appropriate list to sec whether the clement is already in place (if duplicates arc expected, an exIra clata memher is usually kept, and this data member v>'Ould be incremented in the event or a match). If the element turns out to be new, it can be inserted at the front of the list, since it is convenient and also because frequently it happens that recently inserted elements arc the most likely 10 be accessed in the near future.

The class interface for a separate chaining implementation is shown in Figure '5.6. The hash tahle 51 ores an array of linked lisls, which "lrc allocated in the constructor.

The chlss interface illustralcs a syntax point: in the declaration of theLists, a space is required between the two >s; since» is a C++ token, and hecause it is longer than >,

» would he recognized as the token.



o

2

3

4 f--~-+-

5

6

9

Figure 5.5 A separate chaining iBSh wblc

template <typename HashedObj> 2 class HashTable .3

'4 pub] ic;

.'5 explicit HashTable{ int size'" 101 ); (,

7 bool contains( const HashedObj & x ) canst;

8 9 void makeEmpty{ );

10 void insert( const HashedObj & x ); 11 void remove( canst HashedObj & x );

12 1.3 private:

14 vector<list<HashedObj> > theLists; II The array of Lists

15 int currentSize; 16

17 void rehash( ); 18 int myhash( const HashedObj & x ) canst; 19 };

20 21 int hash( const string & key);

22 int hash{ int key);

Figure 5.6 'Iypc declaration for separate chaining hash tahle

5.3 Separate Chaining 189 Bismillah hiir Rehman nir Raheem-----------------------------Assalat o Wasalam o Alika Ya RasoolALLAH



int myhash( canst HashedObj & x } canst 2 ( 3 int hashVal " hash( x ); 4 5 hashVal %0: thelists.size( }; 6 iff hashVal < 0 ) 7 hashVal +" theLists.size( ); 8

9 10

return hashVal;

Figure 5.7 myHash member function for hash tables

II Example of an Employee class 2 class Employee 3 ( 4 public:

5 const string & getName{ ) const 6 { return name; } 7 8 bool operator~:( canst Employee & rhs ) canst 9 ( return get Name ( ) "" rhs. getName ( ); }

10 bool operator!~( canst Employee & rhs ) canst 1/ { return ! ( *thi 5 "'" rhs; }

12

13 /1 Additional public members not shown 14 15 private:

16 string name; 17 double salary; IR

19

; nt seniority;

20 II Additional private members not shown 21 };

22 23 lnt hash( const Employee & item)

24 25 return hash( item.getName( ) );

26

Figure 5.8 Example of a class that can be used as a HashedObj



5.3 Separate Chaining

void makefmpty( ) 2 {

.3 for( inti '" 0; i < theLists.size( }; i++ }

4 theLists[ i j.elear( );

5 (,

"j boo 1 ccnta; ns ( canst HashedObj & x ) canst

8 <)

10

11

canst list<HashedObj> & whichList = theLists[ myhash( x) ];

return find{ whichList.begin( }, whichList.end( ), x) != \~hichList.end( );

12

IJ bool remove( const HashedObj & x )

14

15

11>

17

18

1 'J 20 21

22

2J

n

]ist<HashedObj> & whichList = theLists[ myhash{ x } ];

list<HashedObj>::iterator itr '" find( whichLisLbegin( ). whichLisLend( }. x );

if( itr "" whiehList.end( ) )

return false;

whichLisLerase( itr );

--currentSize; return true;

Figure 5.9 makeEmpty, contains, and remove routines for separate chaining hash table

Just as the binary search tre(' works only for objects that arc Comparable, the hash tables in this chapter \vork only f~w ohjects tilaL provide a hash function and equality operators (operator== or operator! =, or possihly hoth). ·rhis automatically will include i nt and stri ng, because Ihese hash funCiions arc provided as glohal, non-class functions in the HashTable

class. Instead of requiring hash functions that take both the object and the table size as

parameters, we have our hash functions lake only the object as the parametCI", and return an i nt, The hash table class has a priV<1I(' member function myhash that scales the result into a suitable array index. myHash is shown in rigurc 5.7.

Figure 5.8 illustrates an Employee class thaL can be stored in the generic hash table, using the name mcrnber as the key. The Employee class implements the HashedObj requirements by providing equality operators and a hash function.

The code 10 implement makeEmpty, contains, and remove is shown in Figure '5.9. Next comes the insertion rowinc. If lhe item to hc inserted is already present, then

we do nothing; otherwise, we place it in the list (sec Fig. '5.10). The clel1lent can br placed anywhere in the list; using push_back is most convenient in our case. I'Ihi chLi s t is a reference variahle; src Section 1.'5.4 I"or a discussion of Ihis usc of rcfcrenc~ variables.



",'" I...lld!Jl~1 j nd!)lIl11g

bool insert( canst HashedObj & x ) 2 {

3 list<HashedObj> & whichList " thelists[ myhash( x ) ];

4 if( find( whichList.begin( J. whichList.end( ), x) !" whichList.end( ) )

5 return false;

6 whichList.push_back{ x );

7 8 9

10

11

12

IJ

II Rehash; see Section 5.5 if( ++currentSize > thelists.size( ) )

rehash ( );

return true;

Figure 5.10 insert routine for separate chaining hash tahle

Any scheme could he used hesides linked lists to resolve the collisions; a binary search tree or even another hash table would work, but we expect that if the table is large and the hash function is good, all the lists should be short, so it is not worthwhile to try anything complicated.

We define the load factor, A, of a hash table to be lhe ratio of the number of clements in the hash table to the table size. In the exarnple above, A = 1.0. The average length of a list is A. The effort required to perform a search is the constant time required to evaluate the hash function plus the time to traverse the list. In an unsuccessful search, the number of nodes to examine is A on average. A successful search requires that ahout 1 + (A/2) links be traversed. To see this notice that the lisl that is being searched contains the one node that stores the match plus zero or more other nodes. The expected number of "other nodes" in a table of N clements and Mlists is (N - l)/M = A - 11M, which is essentially A, since M is presumecllarge. On average, half the "other nodes" are searched, so combined with the matching node, we obtain an average search cost of 1 + A/2 nodes. This analysis shows that the table size is not really imponant, but the load factor is. The general rule for separate chaining hashing is to make the table size about as large as the number of clements expected (in other words, let A;::::: 1). In the code in Figure 5.10, if the load bctor exceeds 1, we expand the table size hy calling rehash at line 10. rehash is discussed in Section '5.5. f! is also a good idea, as mentioned beforc, to kecp the table size prime to ensure a good distribution.

S.4 Hash Tables Without Linked Lists SeparalC cbaining hashing has the clis<ldvantage of using linked lists. This could slow the algorithm down a bit because of the timc required to allocate nc\v cells (especially in other languagcs), and also essentially requires the implementation of a second data structure. An alternative to resolving collisions with linked lists is to try alternative cells until an



5.4 Hash Tables Without Linked Lists

Empty Table After S9 After IS After 49 After 58 After 69

0 49 49 '19 I 58 58

2 69 , 4

5

6

7

8 18 18 18 18

9 89 89 89 89 89

Figure 5.11 Hash table with linear probing, after each insertion

empty cell is found. More formally, cells lJo(x), IlJex), h2 (x), ... are tried in succession, where h,(x) = (hash(x) + f(i)) mod '{(lbleSize, with /(0) = 0, The function, (, is the collision

, ,

resolution strategy. Because all the data go inside the !able, a bigger table is needed in such a scheme than for separate chaining hashing. Generally, the load factor should be below A = 0.5 for a hash table that doesn't use separate chaining. We call such tables probing hash tahles. We now look at three common collision resolution strategies.

5.4.1 Linear Probing In linear probing,] is a linear fUllction ofi, typicallyf(i) = i. This amounts to trying cells sequentially (with wraparound) in search of an empty cell. Figure S.ll shmvs the result 01 inserting keys {89, JR, 49, 58, 69} into a hash table using the same hash function as before and the collision resolution strategy,] (i) = i.

The first collision occurs when 49 is inserted; it is put in the next available spot, namely, spot 0, which is open. The key '58 collides with 18,89, and then 49 before an empty cell is found three away. The collision for 69 is handled in a similar manneL As long as the table is hig enough, a free cell can always be found, but the time to do so can get quitl' large. Worse, even if the table is relatively empty, blocks of occupied cells start forming. This elfect, known as primary clustering, means that any key that hashes into the duster will require several attempts to resolve the collision, and then it will add to the cluster.

Although we will not perform the calculations here, it can he shown that the expected number of probes using linear probing is roughly ~(l + 1/(1 ~ A)2) for insertions and

unsuccessful searches, and i(1 + l/(I - A» for successful searches. The calculations afe somewhat involved. It is easy to sec from the code that insertions and unsuccessful searches require the same number of probes< A l11oment~ thought suggests that, on average, successful searches should take less time than unsuccessful searches.

The corresponding formulas, if clustering is not a problem, are fairly easy [0 derive. We will assume a very large table and that each probe is independent of the previous probes.

193



\"IIQPlt:1 J IIQ)IIIII~

Thesc assumptions arc satisfied by a random collision resolution strategy and are reasonable unless "A is very close to I I:irst, we derive the expected number of probes in an unsuccessful search. This is just the expected number of prohes ulltil we find an empty cell. Since the fraction of empty cells is 1 - A, the numher of cells we expect to prohe is l/( 1 - "A). The llumher of probes for a successful search is equal to the numher of probes required when the particular clement was inserted. \~rhen an clement is inserted, it is done as a result of an unsuccessful search. Thus, we can use the cost of an unsuccessful search to compute the average cost of a successful search.

The caveat is thai ),. changes from 0 to its currcn\ value, so that earlier insertions arc cheaper and should bring the average down. for instancc, in the table in I\gure '5.11, A = 0.'5, hut the cost of accessing IH is determined when IB is inscrtcd. At that point. A = 0.2. Since IB was inserted into a relatively empty table, accessing it should bc easier than accessing a recently inserted clemen 1 such as 69. Vic call estimate the average by llsing an inl(');ralto calculate the mean value of the insertion lime, oblaining

11" I I I I(A) = - ~-dx= -In~-), 0 I-x ), I-A

These formulas arc clearly better than the corrcsponding formulas for linear prohing. Clustering is not only a theoretical prohlem hut actually occurs in real implementations.

Figure '5.12 compares the performance of linear prohing (clashed curves) with \vhat \·voulcl he expecLed from more random collision resolution. Successful seardws arc indicated by an 5, and unslKcess[ul searches and insertions are rnarked with U and 1, respectively.

If )c = 0.75, then the formula ahove indicates that 8.5 prohes are expected for an insertion in linear prohing. If)" = 0.9, then 50 probes arc expected, which is unreasonable. This compares with 4 and 10 probes for the respective IOHd factors if clustering were not

15.0

12.0

9.0

6.0

3.0

0.0

, /

/ /

/

/ /

I U,I

/

/

/ /

U,J

s I

___ /~s -~=~, ~,-,-::::::::"'~~-

.10 .15 .20 .25.30.35.40.45.50.55.60.65 .70.75.80.85 .90.95

Figure 5.12 Numher of probes plotlcd against load bctor for linear prohing (dashed) and random strategy (5 is successful search, U is unsuccessful search, and I is insertion)



o

2

3

4 'j

6

7

8

9

Empty '[~\bk After 89 Aller 18

18

89

Aller 49

49

18

89

5A Hash Tables Without linked lists

After '58

49

58

18

89

After 69

49

58 69

18

89

Figure 5.13 I-lash table with quadratic probing, after each insenion

a problem. vVe see from these formulas that linear probing can be a bad idea if the tahle is expected to be more than half full. If A = 0,'5, hO\vever, only 2.5 prolJCS are required on average for insertion, and only 1.5 prohes arc required, on average, for a successful search.

5.4.2 Quadratic Probing Quadratic probing is a collision rcsolution mel hod that eliminates the primary clustering problem of linear probing. Quadratic probing is what you \vould expect-the collision function is quadratic. The popular choice isJ(i) = i2

. Figure 5. J 3 shmvs the resulting hash table with this collision function on the same input used in the linear probing example.

When 49 collides with 89, the next position attempted is one cell away. This cell is empty, so 49 is placed there. Next 58 collides at position 8. Then the cdl one away is tried, but another collision occurs. A vacant cdl is found at the next cell tried, which is 22 = 4 away. 58 is thus placed ill cell 2. The same thing happens ror 69.

For linear probing it is a bad idea to let the hash table- get nGl.rly full, because pcr~ fnrmancc degrades. For quadratic probing, the situation is even more drastic: There is no guarantee of finding an empty cell once the table gets more than half full, or even before the table gets half full if the Lable size is not prime. This is hecause at most half of the lahk can he used as alternative locations to resolve collisions.

Indeed, we prove now that if the table is half empty and the table size is prirne, then we arc always guaranteed to bc ahle 10 insert a new clemenl.

Theorem 5.1. 1f quadralic prohing is used, and the table size is primc, then a new clement can always be insertcd if the tahle is at least half empty.

Proof. Let the tahle size, TablcSizc, be an (odd) prime greater than 3. \"/c show that the first f]cthleSize/21 alternalive locations (including the initial location ho(x)) arc all distinct. "1\vo or these locations are hex) + j2 (mod Tc.//J/cSizc) and h(x) +/ (mod TahlcSize) ,




where 0.:: i,j :s. LJeib/cSize/2J. Suppose, for the sake of contradiction, that these locations arc the same, but i :fj. Then

hex) + i2 = hex) +/ j2=/

i' -}' = 0

(i-j}(i+j)=O

(mod JahlcSize)

(mod JahleSize)

(mod lableSize)

(mod JilbleSize)

Since lablcSizc is prime, it follows that either (i ~ j) or (i + j) is equal to 0 (mod TahlcSize). Since i and j arc distinct, 1he first option is not possible. Since o ~ i,j:2: lTa1J/eSizej2J, the second option is also impossible. Thus, the lirsl rT(/blcSizej21 alternative locations are distinct. If at most LJelbicSizcj2J positions arc taken, then an empty spot can always be found.

If the table is even one more than half full, the insertion could fail (although this is extremely unlikely). Therefore, it is important to keep this in mind. It is also crucial that the table size be prime.} If the table size is not prime, the number of alternative locations can be severely reduced. As an example, if the table size werc 16, then the only alternative locations would he at distances 1, 4, or 9 away.

Standard deletion cannot he performed in a probing hash table, because the cell might have caused a collision to go pas\. il. For instance, if we remove 89, then virtually all of the remaining find operations will fail. Thus, probing hash tables require lazy deletion, although in this case there really is no laziness impHcd.

The class interface required to implement probing hash tables is shown in figure 5.14. Instead of an array of lists, we havc an array of hash tablc cntry cells. The nested class HashEntry stores the state of an entry in the info member; this state is either ACTIVE, EMPTY,

or DELETED. In C++, these constants can be declared as static const data members with initial values. Thus we could have, in the HashTable class,

static const int ACTIVE 0;

static canst int EMPTY l'

static canst int DELETED 2;

Unfortunately, though mandated by C++ Standard, this is nol supported hy all compilers. Thus we use the enumerated type instead:

en urn EntryType { ACTIVE, EMPTY, DELETED );

which achieves the same effect. The lype EntryType is no more than an int, and lhe values for ACTIVE, EMPTY, and DELETED arc assigned by the compiler in sequential order. This trick has long been used by C++ programmers to declare integer class constants. for instance,

enum { MAX VALUE' ID );

I If I he table size is a prime of the form 411 + 3. and the quadratic collision resolution strategy I(i) = ±i2. is

used, lhen the entire tilblc can he probed. The cost is a sligbtly more complicatcd routine.



5.4 Hash Tables Without linked lists

template <typename HashedObj>

2 class HashTable .3 4 public:

.'5 explicit HashTable( int size'" 101 );

6

7 bool contains( const HashedObj & x ) const; 8

void makeEmpty( );

bool ; nsert ( const HashedObj & x ) ;

bool remove( const HashedObj & x );

9

10

11

12

13 14

enum Entt'yType ( ACTIVE, E~lPTY, DELETED

15 private: 16 struct HashEntry

17

18 HashedObj element;

19 EntryType info;

20

);

21

22 HashEntry( const HashedObj & e = HashedObj( ). EntryType ;

: element( e ), info( i ) ( )

2l };

2'J

25 vector<HashEntry> array:

26 int currentSize; 27

28 bool isActive( int currentPos ) const; 29 int findPos( const HashedObj & x ) const; 30 void rehash( );

31 int myhash{ const HashedObj & x } const;

32 };

EMPTY )

Figure 5.14 Class intcr{~Kc for hash tables using probing strategies, including the nested HashEntry class

Constructing the tahle (Fig. 5.15) consists of setting the; nfo member to EMPTY for each cell. contains(x}, shown in figure 5.16, invokes private member functions isActive and fi ndPos. The private member function fi ndPos performs the collision resolution. We ensure in the; nsert routine that the hash table is at least twice as large as the number of elements in the tal.Jle, so ql\~ldratic resolution will always work. In the implemcntation in Figure 5.16, eicmenls that are marked as deleted count as being in the table. This can cause problems, because the table can get too full prematurely. Vie shall discuss this item present ly.



1911 Chapter 5 Hashmg

explicit HashTable( iot size 101} array( nextPrime( size) ) 2 { makeEmpty{ ); }

J

4 void makeEmpty( 5 { (,

7

8 9

currentSize fore int i

array[

0; 0; i < array.size( ); i++ }

i 1 .info " EMPTY;

Figure 5.15 Routines to initialize quadratic probing hash tahle

bool contains( canst HashedObj & x ) canst 2 { return isActive( findPos( x ) ); }

3 'J int findPos( canst HashedObj & x ) canst

5 (,

7

8 ')

10

11

12

13

14

15

16

17

18 19

20

int offset'" 1;

int currentPos ~ myhash{ x );

\'Ihile( array[ currentPos ]. info !'" EMPTY &&

array[ currentPos ].element != x }

currentPos += offset; II Compute ith probe offset +-'" 2; if( currentPos >= array.size( ) )

currentPos array.size();

return currentPos;

21 bool isActive( illt currentPos ) canst 22 return array[ currentPos ] .info ~~ ACTIVE;

Figure 5.16 contains routine (and private helpers) for hashing with quadratic probing

Lines 12 through 1'5 represent the fast way of doing quadratic resolution. from the definition of the quadratic resolution function, I (i) =ICi - l) + 21 - 1, so the next cell to

Iry is a distance from the previous celltrkcl and this distance increases by 2 on successive probes. If the new location is past the array, it can be put back in range by subtracting TabieSizc, This is bstcr than the obvious method, because it avoids the multiplication and division that seun to he required. An important warning: The order of testing at lines 9 and lOis important. Don't switch it!



J

(,

7

bool insert( const HashedObj & x

1/ Insert x as active lnt currentPos = findPos( x ); if{ isActive( currentPos ) )

return false;

5.4 Hash Tables Without linked lists

8

'J

array[ (urrentPos ] = HashEntry( x, ACTIVE );

10

II

12 13

14 15 /(i

/1 Rehash; see Section 5.5 if( +-t-currentSize > array.size( ) / 2 )

rehash( );

return true;

17 bool remove( const HashedObj & x )

18

19

20

21 22 23

2'1

25

i nt currentPos '" fi ndPos ( x ); if( !isActive( currentPos ) )

return false;

array[ currentpos ] .info DELETED; return true;

Figure 5.17 insert and remove rOlltincs for hash tables with quadratic probing

The final routine is insertion. As with separate chaining hashing, we do nothing jf x is already present. II is a sirnplc modification to do something else Otherwise, we place it at the spot suggested by the findPo$ mutine. The code is shown in figure S.l7. If the load facior exceeds 0.'5, the table is full and we enlarge the hash table. This is called rehashing, and is discussed in Section '5.5. Figure 5. J 7 also shows remove.

Although quadratic probing eliminates primary clustering, clements that hash to the same position will probe the same alternative cells. This is knowll as secondary clustering. Secondary clustering is a slight thcorelical hlemish. Simulation results suggest that it generally causes less than an extra half probe per search. The following technique eliminates this, but does so at the cost of computing <l.n extra hash function.

5.4.3 Double Hashing The last collision resolution method we will cX<lminc is double hashing. For double hashing, one popular choice is} 0) = i . /wsh2(x). Tbi5 formula says that we apply a second hash function tox (md probe at a distance hash2(x), 2hash2(x), ' .. ,and so on. r\ poor choice of Iwsll;1(X) would be disastrous. For instance:, lht ohvious choice /1(/.,11 2 (x) = x mod 9 would




Empty -bble Af,er 89 After 18 After 49 Afler 58 After 69

0 69

2

1 58 58

4 5

6 49 49 49 7

8 18 18 18 18

9 89 89 89 89 89

Figure 5.18 Hash tahle with double hashing, after each insertion

no! help if 99 we[e inserlecl into the input in the previous examples. Thus, the function must never evaluate to zero. It is also important 10 make sure all cells can be probed (this is not possihle in the example below, because the table size is not prime). A function such as Iwsh 2(x) = a - (x mod R), with n a prime smaller than ]ableSize, will work well. If we choose R = 7, then Figure 5.18 shows the results of inserting the same keys as before.

The first collision occurs when 49 is inserted. l1(/s/12(49) = 7 - 0 = 7, so 49 is inserted in position 6. Iwsh2(58) = 7 - 2 =), so 58 is inserted at location 1. Finally, 69 collides and is inserted at a Jistance 11(1s11 2(69) = 7 ~ 6 = ! away. If we tried to insert 60 in position 0, we would have a collision. Since hash 2(60) = 7 ~ 4 = 3, we would then try positions 3, 6, 9, and then 2 until an empty spot is found. It is generally possible to find some bad case, hut there are not too many here.

As we have said before, the size of our sample hash table is not plinK. \Vc have done this ror convenience in computing the hash function, but it is worth seeing why it is important to rnake sure the table size is prime when double hashing is used. If we attempt to insert 23 into the table, it would collide with 58. Since 11(Is11 2(23) = 7 - 2 = 5, and the table size is 10, we essentially have only one alternative location, and it is already taken. Thus, if the tahle size is not prime, it is possible to run out of alternative locations prematurely. However, if double hashing is correctly implemented, simulations imply Ihal the expected number of probes is almost the same as for a random collision resolution slrategy. This makes double hashing theoretically interesting. Quadratic probing, however, docs not require the use of a second hash function and is thus likely to be simpler and faster in practice, especially for keys like strings whose hash functions are expensive to compute.

s.s Rehashing If the table gelS too full, the running lime for the operations will start taking too long and insertions might l~lil for open addressing hashing \vith quadratic resolution. This can happen if there are too many removals intennixed with insertions. A solution, then, is to



o

2

3

4

5

6

6

15

24

13

Figure 5.19 Hash table "with linear probing with inrut 13, 15,6,24

o

2

3

4

5

6

6

15 --

23

24

13

Figure 5.20 Hash table with linear probing after 23 is inserted

5.5 Rehashing

build another table. that is about twice as big (with an associated new hash function) and scan down the entire original hash table, computing the new hash value for each (nonde1etecl) element and inserting it in the new tahle.

As an example, suppose the clements 13, 15,24, and 6 ayc inserted into a linear probing hash table or size 7, The hash [unction is hex) = x mocJ 7. The resulting hash table appears in figure 5.19.

If 23 is inserted into the table, the resulting table in hgure 5.20 will he over 70 percent full. Because the table is so full, a llC\V table is creat-eeL The size of this table is 17, because this is the first prime that is twice as large as the olcltable size. The new hash function is then l1(x) = x mod! 7. The old table is scanned, and dements 6, 1 '5,23, 24, and 13 are inserted into the new table. The resulting table appears in Figure 5.2J.

This entire operation is called rehashing. This is ohviously a very expensive operation; the running time is O(N), since there afe N clements to rehash and the table size is roughly 2N, lJut it iS~KLua\ly not all that bad, because it happens very infrequently. In parLicular, there must have been N /2 insertions prior to the last rehash, so it essentially adds a constant cost



.tu.t Lnapler) Masmng

o

2

3

4

5

6

7

9

10

II

12

13

14

15

16

6

23

24

13 ~-

15

Figure 5.21 I--lash tahle after rehashing

to each inscniorL2 If this data structure is part of the program, the effect is nUl nOlicc;'lble. On the other hand, jf the hashing is performed as parI of an interactive system, thell the unfortunate LIseI' \vhose insertion caused a rehash could Sl'C a slowdowll.

Rehashing can be implemented in several ways with quadratic prohing. One alternative is LO rehash as soon as the table is haH fulL The other extreme is to rehash only when an insertion fails. A third, middle-of-the-road strategy is to rebash \vhcn the table reaches a certain load bctor. Since performance docs degrade as the load factor increases, the third strategy, implemented with a good cwolf, could be best.

Rehashing for separate chaining hash tables is similar. Figure 5.22 shows that rehashing is simple to implement, and provides an implementation for separate chaining rehashing also.

2 This is why lhc ncw lable is made [wice as large a" the old [,Iblc



/** 2 * Rehashing for quadratic probing hash table.

3 *1 4 void rehash( 5 { 6 vector<HashEntry> oldArray = array; 7 8 II Create new double-sized, empty table 9 array.resize( nextPrime( 2 * oldArray.size( ) ) );

10 fore int j '" 0; j < array.size( ); j++ )

11 array[ j J. info' EMPTY; 12

13

14 15 16

17 18

19

20 /**

II Copy table over currentSize '" 0; fore int i '" 0; i < oldArray.size( ); i++

if( oldArray[ i J. info" ACTIVE) insert( oldArray[ i J.element );

21 * Rehashing for separate chaining hash table. 22 *1 23 void rehash( 24 { 25 vector<list<HashedObj> > oldLists '" theLists; 26

27 28

29 30 31

32 33 34 .35 36 37 38 39 40

II Create new double-sized, empty table thelists.resize( nextPrime( 2 * theLists.size( ) ) ,; fore int j '" 0; j < theLists.size( ); j++ )

theLists[ j J.elear( );

II Copy table over currentSize '" 0; fore tnt i = 0; i < oldlists.size( ); i++ )

{

list<HashedObj>::iterator itr::o oldlists[ i ].begin( ); while( itr !, oldLists[ i J .end( ) )

insert( *itr++ );

5.5 Rehashing

Figure 5.22 Rehashing for both separate chaining hash tables and probing hash tables

203



..... ,,'-'1"' .... , -' """""""6

5.6 Hash Tables in the Standard Library The Standard Ijbrary docs not contain hash tahle implementations of set or map. I ]OWCVC1",

many compilers provide hash_set and hash_map, with the same memher functions as the set

and map class. To use hash_set or hash_map, you must provide the appropriate include directive, and

perhaps use the appropriate namcspace, both of which (lrc compiler dependent. You must then provide the appropriate type paramcters to instantiate the hash_set or hash_map. Tn

the case of the hash_map, these type parameters include the type of thc key, the type of the value, the hash function (which returns an unsigned integer), and an equality opera.tor. Unfortunately, how the type parameters beyond the key and value type are expressed is ag<lin compiler dependent.

Inevitably, the next m;:~jor revision of C++ will choose one of the competing implcHlental ions of hash~ set and hash_map.

5.7 Extendible Hashing Our last topic in this chapter deals with the case where the amount of data is too large to

fit in main Illemory. As we saw in Chapter 4, the main consideration then is the number 01 disk accesses required to retrieve data.

As before, we assume that at any point we have N records to store; the value of N changes over time. Furthermore, at most M records fit in one disk hlclCk. 'vVc will usc M = ,+ in this section.

If either pmhing hashing or separatc chaining hashing is used, the major problem is that collisions could ca.use scveral hlocks to he examined during a search, even for a wdldistrihuted hash tahle. furthermore, when the tahle gets lOu full, an extremely expensive rehashing step must be performed, which requires O(N) disk accesses.

A clever alternative, known as extendible hashing, allows a search 10 be perilxmcd in two disk accesses. Insertions also require few disk accesses.

"l0le recall from Chapter 4 that a B-tree has depth O(logM/2. N). As I'vI increases, the depth of a B-trcc decreases. 'vVc could in theory choose A'lto he so large lhat the depth of the fHrcc would he 1. Then any search after the first would take one disk access, since, presumably, the root node could be stored in main memory. The problem with this strategy is that the branching l~lClOr is so high that il would take considerable processing to determine which leaf the data was in. If t.he time to perform this step could be reduced, then we would have a practical scheme. This is exactly the strategy used hy extendible hashing.

Let us suppose, for the moment, thai our data consists of sl'vcral six-bit integers. figure S.23 shows an extendible hashing scheme for these data. The rool. of the "tree" contains fOllr pointers determined by the leading two bits of the data. Each leaf has up to M = 4 elements. It happens that in each leaf the first two bits are identical; thls is indicated by the number in parenthests To be more formal,]) will represent the number orbits used by the root, which is sometimes known as the directory. The numher of entries in the directory is thus 2J). cil. is the numher of leading bits that all the clements of some leaf L hmre in common. £If will depend on the particular teaf, and dr. .::: v.



5.7 txtendtble Hashmg

I 00 I 01 10 I I 1 I I j \ \

(2) (2) (2) (2)

000100 010100 100000 111000

001000 011000 101000 111001

001010 101100

001011 101110

Figure 5.23 l:xlcndihle hashing: original data

(2) (2) (3) (3) (2)

000100 010100 100000 101000 111000

001000 011000 100100 101100 111001

001010 101110

001011

Figure 5.24 Fxtcndiblc hashing: after insertion of !OO.lOO and directory split

Suppose that we want to illsert the key 100100. This would go into the third leaf, but as the third leaf is already full, there is no mom. Vve thus split this leaf into two leaves, which arc now determined hy the first three bits. This requires increasing the directory size to 3. These changes arc rdlcctcd in Figure 5.24.

NoliC(~ that all of the leaves not involved in the split arc now pointed to by two adjacent directory entries. Thus, although an entire directory is rewritten, nOlle of the other leaves is actually accessed.

205



Lnaprer ::. Mosmng

l 000 I 001 I 010 011 I 100 I 101 I 110 I III I ~ ~ ~l \ \ \/

(3) (3) (2) (3) (3) (2)

000000 001000 010100 100000 101000 111000

000100 OOIOIO 01 1000 100100 WI 100 I I 1001

001011 101 I 10

Figure 5.25 Extendible hashing: after insertion of 000000 and leaf split

If the key 000000 is now inserted, then the first leaf is split, generating two leaves with dL = 3. Since D = J, the only change required in the directory is the updating of the 000 and 001 pointers. See figure 5.25.

This very simple strategy provides quick access times for insert and search operations on large databases. There arc a few important details we have not considered.

First, it is possihle that several directory splits will be required if the elements in a leaf agree in more than D + 1 leading hits. for instance, starting at the original example, with D = 2, if 111010, 111011, and finally 111100 are inserted, the directory size must he increased to 4 to distinguish between the five keys. This is an easy detail to take care of, hut must not be forgotten. Second, there is the possibility of duplicate keys; if there arc more than [vI duplicates, then this algorithm does not work at all. In this case, some other arrangements need to be made.

These possibilities suggest that it is important for the bits to he fairly random. This can be accomplished by hashing the keys into a reasonably long integer ........... hence the nmne.

vVe close by mentioning some of the performance properties or extendihle hashing, which are derived after a very difficult analysis. These results arc hased on the reasonable assumption that lhe bit patterns arc uniformly distrihuted.

The expected number of leaves is (N/M) 10g2 c. Thus the average !caf is In 2 = 0.69 full. This is the same as for B-trecs, which is not entirely surprising, since for both data structures new nodes arc created when the (M + l)th entry is added.

The morc surprising result is that. the expected size of the directory (in other words, 2/) is O(NH-ljl\.l/M). If 1\1 is vcry small, then the directory can get unduly large. Tn this case, we can have the leaves contain pointers to the records instead of the actual records, thus increasing the value of M. This adds a second disk access to each search operation in order to maintain a smaller directory. If the directory is too large to lit in main mcrnory, the second disk access would be needed anyway.



Summary

Summary

1 lash wbles call be used to illl p!cmclll the insert JJ ld conta ins operations in constant ,we rage

time. It is especially impurLant 10 pay attention to details such as load factor when using

hash tables, since otherwise the lime bounds arc not valid. It is also imponant to choose the hash funnion carefully when the key is not a short string ur integer.

for separate chaining hashing, the load hlclor should be close to J, although PCr!CHmanec does not signillcantly degrade unless the load factor becomes vcry large. For probi ng hashing, the load factor should not exceed 0.'3, unless this is completely unavoidable. If linear probing is used, performance degenerates rapidly as ilK hJ<ld bctm approaches I. Rehashing call be illlpkmcnLcd to allow the ",hIe to grow (and shrink), thus maintaining a reasonable load factor. This is imponam if space is tight and it is no! possihle jllst 10 declare a huge hash tahle.

Billary search trees call also he used to implemcnt insert and contains operations. Ahhough the resulting average tillle hounds are O(log N), hinary sl'arch trees also support routines that require order and ,UT thus more powerful. Using a hash table, i[ is not possihh· to find the minimum element. It is not possible to search efficiently for a SIring unless the

exact string is known, A binary S(,(lfCh tree could quickly find all items in a certain range; this is not supported hy hash tables. furthermore, the O(lng N) bound is not necessarily that much mOlT than O( I), ('specially since no Illultiplications or divisions arc required by sUHch trees.

On the other hand, the worst case for h;bhing generally resulis from an implemen1ation error, where,)s sorted input can make hinary trees perform poorly Balanced search trees arc quite expensive In implement, so if ilO ordering information is required and there is any suspicion that the input lYlight be sorteci, then h"tshing is the data structure of choice.

I lashing applicmiolls Jrc ahunciant. Compilers usc hash tables 1O keep track of declared variables in source code. The data structure is l<nown as a symhol table. Hash tables arc the ideal application ft)]" this problem. ldentifiers afC typically short, so the hash function can he compUlcd quickly, and alphahetizing the 'variables is often unnecessary.

i\ hash !ahle is useful for any graph theory prohkm where lhe nodes have realnamcs inslead of numbers. I lere, <1S the input is rcad, venices arc assigned integers from 1 onward

hy order of appearance, Again, the input is likely to have large groups of alphahetized entries. For example, the vertices cOllld be computers. Then if one particular installation lists its computers as i/)I11/, ihm2, ibm3, . , there could be a dramatic effecI on efficiency if a search tree is used.

A third common usc uf hash tables is in pmgrarns that play games. As the program searches thmugh differellt lines or play, it keeps track of positions it has seen hy computing a hash function based 011 the position (and storing its move Cor t};al position). If the same positioll reoccurs, usually by a simple transposition of llJO\'('S, the program can avoid expensivc recomputation. This general feature of all gal11(:>playing programs is known as the transposition tahle.

Yet another usc oC hashing is in un-line spelling checkers. If misspelling detection (as opposed to correction) is imporlant, an entire dictionary can he pre-hashed and words call be checked in constant timC'. Hash tahles ,IlT weI! suited for this, hecause it is not importalll to alphabetize words; printing out misspellings in the order they occurred in tIll' doctuncilt is certainly acceptable.




We close this chaptcr by returning 10 the word puzzle problem or Chapter 1. If the second algorithm described in Chapler 1 is used, and we assume that the maximum word size is some small constant, then the time to read in the dictionary containing \\1 words and put it in a hash table is O(W). This time is likely to be dominated hy the disk I/O and not the hashing routines. The rest of the algorithm would test for the presence of a word [\x each ordered quadruple: (nm: CO/UllIn, orientLltioll, l1umlJCF (?F c!lare/cters)_ As each lookup would be O(l), and there arc only a constant number of orientations (8) and characters per word, the running time of this phase would br OCR· C). The 10lal running Lime would be OCR· C + W), which is a distinct improvement over rhe original O(R· C· \V). 'vVe could make funher optimizations, which would decrease the running time in practice; these are described in the exercises.

Exercises

5.1 Givcn input 14371, 1323,6173,4199,4344,9679, 1(89) and a hash Junction hex) = x (mod (10), show the resulting: a. Separate chaining hash table. h, Hash tahle Llsing linear probing. c. Hash tahle using quadratic probing. d. I-lash tahle with second hash function h2(x) = 7 - (x mod 7).

5.2 Show the result of rehashing the hash tahles in Fxercise 5.1.

5.3 Write a program to compute the number of collisions required in a long random sequence of insertions using linear probing, quadratic probing, and double hashillK

5.4 A large number of deletions in a separate chaining hash \ahk can cause the table to

be fairly empty, which wastes space. In this casc, we can rehash to a tahle half as large. Assume that we rehash to a larger table when there arc tv·/ice as many clements as the table size. How empty should the table be hefore we re-hash to a smaller table?

5.5 The i sEmpty routine for quadratic probing has not been written. Can you implement it by returning the expression currentSi ze==O?

5.6 In the quadratic prohing hash table, suppose that in~tcad of inserting a new item into the location suggested by findPos, we inscrt it into the first inactive cell on the search path (thus, it is possible 10 reclaim a cell that is marked deleted, potentially saving space). a. Rewrite the insertion algorithm 10 usc this observation. Do this by having fi ndPos

maintain, with an additional variahle, the location of the first inactive cell it

encounters. h. Explain the circumstances under which the revised algorithm is faster than the

original algorithm. Can i1 he slowcr?

5.7 Tbe hash function in Figure '5.4 makes repealed calls to key.length{) in the for loop. Is it worth computing Ihis once prior to entering the loop?

5.8 \Vhat are the advantages and disadvantages of the various collision re-solution stralegit's?



Exercises

S.9 Suppose that tn mit igate the crfecls of secondary clll~tcrillg we usc as t he collision resolution runctionf(i) = i· r(Jwsh(x», whac Iwsh(x) is the .12-bil hash value (not yet scaled 10 a suitahle array index), and rev) = 148271y(mod(2ll -- 1»)1 mod 7(Ih!C')'izf,

(Sec lion 10.4. I describes a method of perrorming this Gilcubtion without overflows, but it is unlikely that over/low matters in this case.) Explain why this st ratcgy tends to avoid sccondary clustering, and compare this S!f<llt'gy with both double hashing

and quadnllic probing.

5.10 Rehashing requires recomputing the hash fUllction for all items in the bash tahle. Since computing the hash fUTlction is expcnsive, supposc objects provide a hash member fUllction of their OW]1, and each ohject stores the result in an additional data member the first lime the hash function is computed for it. Show how such a

schemc would apply for the Employee class in Figure '5.8, and explain under what circumstances the rememhered hash value rcmains valid in each Employee.

S.II vVrite a program to implement the following strategy for multiplying two sparse polynOlnials PI> P,!. of size M and N, n::spectively. Each polynomial is rcpresented as a list of objects consisting of a coefficient and an exponent. \Ve multiply each term in tJ l hy a term in P2 for a total of MN operations. One mel hod is 10 sort these lerms and comhine like terms, hUllhis requires sorting MN records, which could be expensive, especially in small-mcnlory environments. Alternatively, we could merge terms as they arc computet! and thell sort the result.

a. v\Trite a program to implement the alternative strategy. b. If the output polynomial has about OeM + N) terms, what is the running time

of hoth methods?

"I< 5.] 2 Describe a procedure t h,ll avoids i nit bib ng a hash table (at t he expense of mCHlory).

5.13 Suppose we want to find the first occurrence of a string PIP2 ... Ph in a long input string I\ j l\2 ... AN' \Ve can solve this problem hy hashing the pattern string, obtaining a hash value IIp> and comparing lhis value with the hash value rormed from

AIA2 ... AII' 1\.1;;\.3' .. AIH·1 , ;\)At .,. ;\11+2, and so on until;\N_h+1 AN_h+2 ., ';\1\"

If we have a match or Iwsh values, we compare the strings character by characttT to

Verd)' the 111Jtch. vVc return the position (in A) if the strings actually do match, <lnd \VC conlinue in the unlikely event that thc match is blse.

"I< <.l. Show thai if the hash value of /\/\'1 I ... AI 1.11 1 is known, thcn the h'l.sh value

of ''\-I-JAj.u ' . ';\H II can be computed ill constant time. b. Show thai the rUllning time is O(I? + N) plus the lime spent refuting false

matches. -I< c. Show t.hat the expected number of false matches is negligible.

d. \\frile a program to implement this algorithm. * c. Oescribe all algorithm thai runs in O(/z + N) worst-case time.

** f. Describe an algorithll1that runs in ()(Njh) average time.

5.14 An (0Id-5t yle) I.t\<)]C progn.1l1l consists of a series of statements numbered in ascending order. Control is passed hy usc of a ,1',010 or gO\uiJ and a s!atclIlcfll numher. \\lritc

a program that reads in (1 legal BI\\I<. progr;ml and n'I1L1111IKrS 111(' statements so that the first starts m number F and each statement has a numher J) higher than

Ihe previous statement. You may assume an upper limit of N statements, but the




statement numbers in the input might be as large ;lS a .)2-.hil integer, Your prograrn

must run ill linear time.

5.15 :-to Illlpkmcnt the word puzzle program usjng the algorithm described aL the end of

the chaplt:r. h. \Ve can gel a big speed increase by storing, in addition to each word W, aU 01

Ws prefixes. Of one of W~S prefixes is another word in the dictionary, it is stored ;b a real word.) Although this may seem to increase the size of the hash table drastically, it docs not, because lllallY words have the same prefixes. \Vhcn a scan is performed in a particular dircCliol1, if the word thai is looked up is not even in the hash tahle asa prefix, then the scan in that direction can he terminated early- Usc this idea to write an improved progralll [0 solve the word puzzle.

c. If we are willing to sacrifice the sanctity of the hash tahle ,un, we can speed up

the program in part (h) by nOling Ihat if, for example, we have just computed the hash fUllction for "excel," we do nol need 10 compute the hash function for

"excels" from scr;Hcil, Adjust your hash fUllction so that it can take advantage of

ils previous caicu\cHion. d. In Chapter 2, we suggested using binary search. Incorporate the idea of using

prdixcs into your binary search algorithm. The modification should be simple.

\\lhich algorithm is faster?

5.16 Under certain assumptions, thc expected (OSloI' all insertion into a hash table with secondary clustering is givcn by 1/(.1 - A) - A - ln(l. - A). Unfortunately,

this formula is not accurate for quadratic probing. lIo'Ncver, assuming that it is,

determine the following: a. The expected cost of an ullsuccessful search. b. The expected cost or a successful search.

5.17 Implement a generic Map t hat supports the insert and lookup operations. The imp1crnentatioll will store a hash table of pairs (key, definition). You will lookup a definition by providing a key figure 5.26 provides the r~ap specifIcation (minus

some details).

5.18 Implement a spelling checker hy using a hClSh table. f\ssumc that the dictionary comes from two sources: an existing large dictionary and a second file cuntaining a personal dictionary. Output all misspdlcd words and the line numhers on which

they occur. Also, for each misspelled word, list any words in the dictionary that arc ohtainable hy applying any of the following rules:

<t. Add one character. b. Remove one character. c. Lxchangc adjacent characters.

5.19 Show the result of inserting the keys I Olllllll , 000000 I 0, 100110 II, 101 I 1110, 011111] I, 01010001,10010110,00001011, ]j001111, 10011110,11011011, 00J010I], 01100001,11110000,01101111 into an initially empty extendible

hashing data structure with [vI = 4.

5.20 vVritc a program to implemcnt extendihle hashing. If the table is small enough to [it in main memory, how docs its pcrfOr!l1<l1lCe compare with separate chaining and

open addrcssing hashing?



template <type name HashedObj, typename Object> 2 class Pair .3

4 HashedObj key; 5 Object def;

6 II Appropriate Constructors, etc. 7 }; 8

9 template <typename HashedObj, typename Object> 10 class Dictionary II

12 public: 13 Dictionary( ); 14

15 void insert( const HashedObj & key, canst Object & definition ); /6 canst Object & lookup( canst HashedObj & key) canst; 17 bool isEmpty( ) canst; 18 void makeEmpty{ ); 19

20 private: 21 HashTable<Pair<HashedObj,Object> > items; 22 };

Figure 5.26 Dictionary skeleton for Exercise_ 5.17

References

References

Despite the apparent simplicity of hashing, much of the analysis is quite difficult and there arc still many unresolved questions. There arc also many inlCrCsling theoretical issues, which generally attempt to make it unlikely that the worst~case possibilities of hashing arise,

An early paper on hashing is [161. A wealth of information on the sulJject, including an analysis of hashing \vith liucar prohing, can he found in [Ill. An excellent survey on the subject is [14]; [J 5] contains suggestions, and pitfalls, for choosing hash functions. Precise analytic and simulation results for all of the methods described in this chapler can be found in [81

An analysis of double hashing can be found in !9] and [111. Yct another collision resolution scheme is coalesceu hashing, described in [17J. Yao 119] has shown that uniform hashing, in which no clusLering exists, is optimal with respect to cost of a successful search.

If the input keys arc known in advance, then perfect hash functions, which do not allow collisions, exist 12l, 17]. Some more complicated hashing schemes, for which the V-lOrst case depends not on the particular input but on random numbers chosen by the algorithm, appear in [:11 and [4].

Extendible hashing appears in [')1, \vith analysis in !o] and [I H]. Fxercisc S. 13 (a-d) is from [10 [ Pan (e) is from [[21, and pan (I) is from [II

211



Lnapler J Md~lllflg

1. R. 5, Boyer and J S. lv1ourc, "A hlSl Snillg Searching Algorithm," Communicatiolls or t.he ACM, 20 (1977), 7Cl2-772.

2. J. I .. Carter and 1'1.'1. N. \Vegman, "Universal (:hLSSCS of l1<ls11 hlllCtiollS," ]ournal (?F COn/pliler

(mel Sy.$cm SciI'IJ( CS, IB (1979), 14:1-1 '54.

'1. Iv1. Dictzfdhingcr, A. R. Karlin, K. l'vtclhorn, F i\-lcycr auf cln Heide, I L Rohnert, and R. E.

"Eujan, "Dynamic Perfect I lashing: Upper and Lower Bounds," SIAM Jotmwl 011 Computillg,

23 (1994), 738-761

4. R. J Enbody and 11. C ])u, "Dynamic I· lashing Schemes," Computing Surveys, 20 (1988),

85-ILl.

5. R. Fagin,J Nicwrgclt, N. Pippcllger, and H. R. Strong, "Extcndible llashillg~~A hISI Access

Met hod for Dynamic nics," ;\CM 71'(//ls(/(/i0I1:-> 0/1 J)(j(ulJasc .\vsleI1IS, 4 (1979), 31 '5-'")44.

6. P Flajolcl, "()lllhc Performance Fvalualion ofTxtcndibk J-lashingand Tric Searching," )\clu

Injfm1Ul1icu, 20 (1983), 34'5-'.)69.

7. M. L Frcdman,,J. K011110s, and F. Szc-mercdi, "Storing a Sparse Table with O( $ \Vorst Case

Access Time," 1our/wl of tlu: /\CM,.5 1 0(84), '5')B~.')44.

B. C. H. Connel and R. Baeza-Yates, flalJdhooh of Alg,oritllln5 U/lci /Jato StrlH:!urcs, 2nd eeL

Addison-Wesley, Heading, 1v1a55., 1991.

9. L J- Cllihas and E. Szemercdi, "The ,<\nalysis of Double I hlshing," .Journal (!f Compuler and System Sciences, 16 (1971:\), 226-·27"1-

10. R. M. Karp and M. n. Rabin, "Efficient Randomized Pallcrn-lvIatching Algorithms," Ai/tell Computer J (1Imralor.y I<cpoH TJ<-J I-R1, llar\'ard Uniw'rsiLy, Cambridge, Mass., I qSI.

] 1. D. F. Knuth, The Art (!{Computcr Prognmllnin(\~, Vol. 3: Sorling and S('(/r(hin(~, 2d cd., Addison

Wesley, Reading, tvlass., 1998.

J2. D. E. Knuth,J 11. Morris, and V R. Prall, "Fast Pattern tvlalching ill Slrings," SlAlvl}ounwl 011 Compulil1~, () (1977), 32.J,~350.

13. C. Lucker and ?v1. iVlo!oduwiLch, "lVlore Analysis of Double I lashing," Proceedings (?f tile TWfnliclh ACM Symposium (1) Themy of Computing (1988), ]5L~--359.

14. \;V D. Ivlaurer and T C. 1.cwis, "1 lash '[~lble Iv1cLhods," Computil1g, Surveys, 7 (197,)), '5-20,

1 '5. B. J 1\kKenzie, R. Harries, and l. Bell, "Sclcuitlg a Hasbing Algorithm," S(dtware-Practice and Experience, 20 (1990),209-224.

16. \V. \\1. Peterson, "Addressing for Random Access Storage," fBi"J JOllrnal of Research and Development,.1 (19'57), !30~-146.

17. J S. Vincr, "Implemenlalions for Coalesced 1 lashing," COll1l1HmiUlliolls (!f the ACId, 2'5

(1982),911--926.

18. A. C Yao, "1\ NoLe on lhe I\nalysis of Extendible llashing," Information Proccssin,g Leiters, 11 (1980),84--86.

19, A. C Yao, "Uniform I lashing Is Oplimal,"jounwl 0/1111' ;\CM, 32 ([ (85),687-693



CHAPTER 6

Priority Queues (Heaps)

Although jobs sent to a printer arc generally placed 011 a queue, this might not always be the best thing to do. For instance, olle job might be panicularly important, so it might he desirahle 10 allow that job to be run as soon as the printer is available. Conversely, if, when the printer becomes available, there arc several 1 ~pagc johs and onc I OO-page job, it might be reasonahle to make the long job go last, even if it is not the 1as\ job subminctL (Unforl unalcly, most sys\(:ms do not do this, which can he particularly annoying at times.)

Similarly, in a multiuser environment, the operating system scheduler must decide which of several processes to run. Generally a process is allowed to run only for a fixed period of lirne. Une algorithm uses a qucue. Jobs arc initially placed at the end of the queue. The schcdukr will repeatully take tIll' first job on the qucue, run it until either it finishes or its time limit is up, and place it at the end of the queue if it docs not finish. This strategy is generally not appropri<lle, bccause vcry short jobs will seem to lake along lime hecause of the wail iJlvolvcd to run. Cellcrally, il is important th,n short johs finish ,-lS fast as possible, so thesc jobs should have prccedence over jobs that havc already been running. runhul1lorc, somcjobs tilal arc nut short arc still very important and should abo have precedence.

This particular application seems to requi re a special kind of qucue, kllO\vn as a priority queue. [11 this chaptcr, we will discuss

EfFicient implementation of thc priority queuc ADT.

Uses of priority queues.

Advanced implemcntations or primity queues.

The data structurcs we will sec arc among tlK most eJcgant in computer science.

6.1 Model ;\ priority queue is a data structure that allmvs at least the rollowing 1\\10 operations: insert, which docs the obviolls thing; and del etel'li n, which fll1ds, returns, and removes the minimum clement in Ihe priorit)' queue. I '[,he insert operation is the equivalent of enqueue,

and del etel'1in is Ihe priority queue equivalcnt of the queue's dequeue operation.

I rhe C++ code proVides two versiOllS or deletet,ljn. One rcmO\'t'o, the minil1lllm; the other l"t'nlO\TS the

minimum and stores the rellloved value in an object p;)Ssnl hy rderl'nn:: 213



lnapter b Prlonty llueues {HeapS)

deleteMin Priority Queue

insert

Figure 6.1 Basic model of a priority queue

As with most data structures, it is somctirncs possible to add other operations, hut these arc cxtensions and not part of the hasic model depiCled in Figure 6.].

Priority queues have many applications besides oprrating systems. In Chapter 7, we will sec how priority queues are used for external sorting. Priority queues arc also important in the implementation of greedy algorithms, which operate by repeatedly finding a minimum; we will sec specific examples in Chapters 9 and 10. In this chapter we will sec a usc of priority queues in discrete event simulation.

6.2 Simple Implementations There are several obvious ways to implement a priority queue. Vve could usc a simple linked list, performing insertions at the front in 0(1) and traversing the list, which requires O(N) time, to delete the minimum. Alternatively, we could illSis\ tbat the list he kept always sorted; this makes insertions expensive (0(1\1)) and de 1 ete~li ns cheap (O( I)). The former is probably the better idea of the two, hasec.l on the bct that there are never more del eteMi ns than insertions.

Another way of implementing priority queues would be to usc a binary search tree. Tbis gives an O(log N) average running lime for both operations. This is true in spite of tbe fact that although the insertions arc random, I he deletions arc not. Recall that the only clement \ve ever delete is the minimum. RCpealC(lly removing a node thaI is in the len subtree would seem to hurt the balance of the tree by making the right subtree heavy. However, the right subtree is random. In the worst case, where the deleteMins have depleted the left subtrec, the right subtree would have at most twice as many elements as it should. This adds only a small constant to its expected depth. Notice that the bound can be made into a worst~case bound by using a balanced tree; this protects one against bad insenion sequences.

Using a search tree could be overkill because it supports a host of operations that arc not required. The basic data structure we will use will not require links and will support both operations in O(log N) worst-case time. Insertion will actu({Uy take constant time on average, and our implementation will allow building a priority qucue of N items in linear time, if no deletions intervenc. We will then discuss how to implement priority queues to

support efficient merging. This additional operation seClllS to complicate matt.crs a hit and apparently requires thc usc of a linked structure.



6.3 Binary Heap The implementation Wt will usc is known as a binary heap. Its usc is so COl1l1non for priorit)' queue ill1pkmcnl<llions thaI, in the context of priority queues, when the word Ileal' is used without a qualifier, it is generally assumed to be refcrring to this implementation of the data structure. In this section, we will rder to binary heap::; merely as hCdpS. Like binary search trees, heaps have two properties, namely, a structure properly and" heap-order property. As with AVL trees, an operation on a heap call destroy one or the properties, so (\ heap operation must nOllerminatc until all heap properties arc in order. This turns out to be simple to do.

6.3.1 Structure Property A heap is a binary tree that is completely filled, with the possihle exception of the bO!lnm level, which is lil!cd from left to right. Such a tree is know11 as a complete binary tree. hgure 6.2 shows an example.

II is easy 10 show that a complete binaIY tree of height 11 has between 2h and 21t j-I ~ 1

nodes. This implies that the height of J. complete billJ.ry tree is LJog N J) which is clearly O(log N),

An important observation is that because a compirlc binary tree is so regular, it can be represented in an array and no links arc necessary. The anay in Figure 6.3 corresponds to the heap in Figure 6.2.

A

B c

E F G

II

Figure 6.2 A complete binary tree

A B c D E F G H J

o 2 3 4 5 6 7 8 10 11 12 13

Figure 6.3 Array implementation of complete binary [ree

&.:>



'-"UP"'" u ,,'U"\Y '-<u\..u\.." \',\..UP"!

template <type name Comparable>

2 class BinaryHeap

4 public:

5 explicit BinaryHeap( int capacity'" 100);

() explicit BinaryHeap( canst vector<Comparable> & items );

7 8 boo I isEmpty( ) canst;

9 canst Comparable & findMin( ) canst;

10

] 1 void insert( canst Comparable & x );

12 void deleteMin( );

13 void deleteMin( Comparable & minItem );

l'f void makeEmpty( );

15

16 private:

17 int currentSize;

18 vector<Comparable> array;

19

20 void buildHeap( );

21 void percolateOol'in( int hole);

22 };

II Number of elements in heap

II The heap array

Figure 6.4 Class intcrface for priority queue

For ,my clement in array positiol1 i, the lert child is in position 2[, the right child is in the cell after the left child (2i + 1), and the parent is in position li/2J. Thus not only arc links not required, but the operations required to traverse the tree are extremely simple and likely to be vcry {~lst Oil most computers. The only problem with this implementation is that an estimate or the maximum heap size is required in aciv;:mcc, but typically this is not a prohlem (and wc can resize' if n('{:,(kd). In figure 6.3, the limit on the heap size is 1·3 clements. The array has a position 0; more on this later.

;\ heap dala structure will, then, consist of an array (of Comparabl e objects) and an integer representing the curren\. heap size. Figure 6.4 shows a priority queue interface.

Throughout this chapler, we shall draw the heaps as trees, with the impliGu ion that an

actual implcmnHatioJ) will usc sirnpk arrays.

6.3.2 Heap-Order Property Thc property that allows operations to he performed quickly is the heap-order property. Since we want to be ahle to find the minimum quickly, it makes SCIlSC thm the smallest clement should he at the root. If we consider (hat any subtree should also be a heap, thell <lllY node should be smaller than all of its descendants.

Applying this logic, we arrive ell the heap-order property. In a heap, for every node X, the key in the parent of X is srnaller than (or equal to) the key in X, with the exception of



b,j tunary Heap

13 13

21 16 21 16

19 68 31 19 68

Figure 6.5 Two complete trees (only the left tree is a heap)

the root (which h~lS no parcnt)/- In Figure 6.5 the tree on the left is a heap, but the tree 011

the right is not (the dashed line shows the violation of heap order). fly the heap-order property, the minimum dement can always be found at the root.

Thus, we get the extra operation, findMin, in constant time.

6.3.3 Basic Heap Operations It is easy (both conceptually and practically) to pcrf()]"Jl1 the two required operations. All the \vork involves ensuring that the heap-order properly is maintained.

insert To insert an clement X into the heap, we create a hole in the next available location, since

otherwise the tree will not be complete. If X can be placed in the hole \vithout violating heap order, then we do so and are clone. Otherwise we slide the element that is in the hole's parent node into the hok, thus bubbling the hole up toward the root. \Ve continue this process until X can be placed in the hole. f'igurc Ci6 shows thal to insert 14, we create a hole in the next available heap location. Inserting 14 in the hole would violate the heaporder property, so )J is slid down into the hole. This stralegy is continued in FIgure 6.7 until the correct location for -14 is found.

This general strategy is known as a percolate up; the new clement is percolated up the heap until the correct location is found. InserLion is easily implemented with the code

shown in Figure 6.8. We could have irnpkmenlccl fhe percolation in the insert routinc by performing

repeated s\vaps until the corrccl order was estahlished, but a swap requires three assignment statements. If an clement is percolated up d 1cvc-\s, the number of assignments performed by the swaps would be 3d. Our method uses d + 1 assignments.

l An;l\ngously, we can dr'clare a (wax) heap. which enahles ns \0 crflcient\y find and remove the maximum

clemen!, by changing the heap-urder properly. Thus, <1 priority queut' can be us<:d \0 find either a minimum

or a maximum, bUlthis needs to he decided ahead of time.

-I."



\"IIOl-'l1:;:1 V rilVIllY I..lU1:;:U1:;:.) ~11t;:0I-'::')

13 13

21 16 21 16

19 68 Q 19

@ (~i) 68

o Figure 6.6 Attempt to insert 14: creating the hole, and bubbling the hole up

13 13

,

16 14 16

(l~ 19 68

J'~ 19 68

32 '@ 2(, 32 @ Figure 6.7 The remaining two steps to insert 14 in previous heap

If the ckment to be inserted is the ncw minimum, it will be pushed all the way to the lOp. At somc point, hole will he 1 and we will wltrllto break out of the loop. We could do this \vith an explicillcst, or \\'f call put a copy of the inserted item in position 0 in order to

make the loop terminate. We ckct to usc the explicit IcSt.

The time to elo the insertion could be as much as O(log N), if the clement to be inserted

is the new minimum and is percoiated "lllhe way to the root. On (l\'Crage, the percolation terminates early; it has been shown that 2.607 comparisons are required UIl average to

perform an insert, so the average insert moves an clement up ! .607 levels.

deleteMin del eteMi ns arc handled in a similar manner as insertions. hnding the minimum is easy; the

hard p<1rt is removing it. \Vhen the minimum is removed, a hole is created at the rooL Since the heap now hecomes one smaller, it follows that the lasl denKlll X in the heap must move somewhere in the heap. If X can be placed in thl' hole, then we arc done. This is unlikely, so we slide the smaller of the hole's children illtn the hole, thus pushing the hole down one

kvcl. \Vc H'pCJt this step until X can be placed in the hole. Thus, our action is tn place X in its correct spot along a path from the root containing minimum children.



/*' 1 * Insert item x. allol1;ng duplicates .

.1 */ 4 void insert( canst Comparable & x } 5 {

6 if( currentSize "'~ array.size( ) - 1 ) 7 array.resize( array.size( ) * 2 ); 8

/ / Percolate up

int hole ~ ++currentSize; 9

10

11

12

1.1

14

fori ; hole> I && x < array[ hole / 2 ]; hole /0 2 )

array[ hole] 0 array[ hole / 2 1; array[ hole] 0 X;

Figure 6.8 Procedure to iI1scn into a hinary heap

13

14 16 14

19 68

31

Figure 6.9 Creation of the hole at the root

6.3 Binary Heap

16

19 68

In figure 6.9 the left figure shows a heap prior to the de leteMi n. After L3 is removed, we must nmv try to place 31 in the heap. The value 31 cannot be placed in the hole, because this would vioial<: heap oreier. Thus, we place the smaller child (14) in the hole, sliding the hole down one level (sec Fig. 6.10). We repeat this again, and since 31 is larger than 19, we place 19 into the hole and create a new hole one level deeper. \Ve then place 26 in the hole and create a new hole on the bottom level since, once again, 31 is too large. Finally, we arc able to place 31 in the hole (Fig. 6.11). This general sl rategy is knmvn as a percolate down. We use the same technique as in the insert routine to avoid the use of s\vaps in this routine.

A frequent implementation error in heaps occurs when there are an even number of clements in tbe heap, and the Olle node that has only one child is encountered. You must make sure nol to assume that there are always two children, so this usually involves ~m extra test. In the code depicted in Figure 6.12, we've dOlle this test at line 40. One extremely tricky solution is always to ensure that your algorithm thinb every node has two children.

219



.uu Lnapter b t'flomy llueues tHeapS)

14

16 \1:9

19 68

cit%) 68

31

Figure 6.10 Next two steps in deleteMin

14 14

19 16 19 16

th 21 19 68

ci) @ 32 31 65 @

19 68

Figure 6.11 Last two steps in de 1 eteMi n

Do Ihis by placing a sentinci, of value higher than any in the heap, at the spot after the heap ends, at the start of each percolate dOlvn when the heap size is even. You should think very carefully before attempting this, and you must put in a prominent comment if YOll do lLSC

this technique. Although this eliminates the need to lest for the presence of a right child, you cannot eliminate the requirement that you lest when you reach the botlom, because Ihis would require a sentinel for every leaL

The worst-case running time for this operation is OOog N). On average, the element that is placed at the rooL is percolated almost to the bottom of the heap (\\'hich is the level it came from), so the average running time is O(!og N).

6.3.4 Other Heap Operations Notice that although finding the minimum can he performed in constant tillle, a heap designed to find the minimum clement (also known as a (mil1)hcap) is of no help whatsoever in finding the maximum elemenl. In {~lCt, a heap has very little ordering information, so there is no way to find any particular clement without a linear scan through the entire heap. To sec this, consider the Jarge heap structure (the clements arc not shown) in Figure 6.! 3, where we see that the only information known ahout the maximum element is that it is at



/*' 2 * Remove the minimum item .

.3 * Throws Underfl owExcept i on if empty.

4 *j .5 void deleteMin{ )

(, (

7 if( isEmpty{ ) )

R throw UnderflowException{ ); 'J

10

11

12

l.l

14 /**

array[ 1 ] ~ array[ currentSize-- ]; percolateDown( 1 ):

15 * Remove the minimum item and place it in minItem.

16 * Throws UnderflowException if empty. 17' j 18 void delete~lin( Comparable & minItem )

19 ( 20 i f( i sEmpty{ ) )

21 throw UnderflowException( );

22

n 24

25 2(1

27

28 /**

minltem = array[ 1 ]; array[ 1 ] = array[ currentSize-- ]; percolateDown( 1 );

29 'I< Internal method to percolate down ;n the heap.

30 * hole is the index at which the percolate begins.

31 *j 32 void percolateDown( int hole}

33 ( :H int child;

35 Comparable tmp co array[ hole];

3()

.37 for( ; hole * 2 <~ currentSize; hole child) lB (

.39 child'" hole * 2; 40 if( child loo currentSize && array[ child + 1 ] < array[ child] )

41 child++;

'12 if( array[ child 1 < tmp

,13 array[ hole] '" array[ child];

44 else

4.'5 break;

46 '17 array[ hole 1 ' tmp;

'18

Figure 6.12 Iv1ethocl to perkwm deleteMin in a binary heap



Figure 6.13 A very large complete binary tree

one of the leaves. Half the elements, though, arc contained in leaves, so this is practically useless information. For this reason, if it is important to know where elements are, some other data structure, such as a hash table, must be used in addition to the heap. (Recall that the model does not allow looking inside the heap)

If we assume that the position of every element is known by some other method, then several other operations become cheap. The first three operations below all run in logarithmic worst-case time.

decreaseKey The decreaseKey{p • .6.) operation lowers the value of the item at position p by a positive amount J3.. Since this might violate the heap orcier, it must be fixed by a percolate up. This operation could he useful to system administrators: They can make their programs run with highest priority.

increaseKey The increaseKey{p,~) operation increases the value of the item at position p by a positive amount ~. This is done with a percolate down. Many schedulers automatically drop the priority of a process that is consuming excessive CPU time.

remove The remove(p) operation removes the node at position p from the heap. This is done by first performing decreaseKey{p,oo) and then performing deleteMinO. When a process is terminated by a Llser (instead of fmishing normally), it must be removed from the priority queue.



explicit BinaryHeap( canst vector<Comparable> & items) 1 array( items.size( ) + 10 ), currentSize( items.size{ ) ) 3

'J

(,

7 R

9 /**

for( lnt i '" 0; i < items,size( ); i++ )

array[ i + 1 ] = items[ i ];

buildHeap( );

10 * Establish heap order property from an arbitrary II * arrangement of items. Runs in linear time. !2 *j JJ void buildHeap( )

H ( 15 for( int i = currentSize / 2; i > 0; i-- )

16 percolateDol'ln( i ).

17

Figure 6.14 buildHeap and construcLor

buil dHeap

6.3 Binary Heap

The binary heap is $olnelimcs constructed from an initial collection of items. This consl ruclor Lakes as input N items and place-i Illl'fll into a heap. ObvioLlsly, lhis call he done with N successive inserts. Since each insert will take O( I) average and O(log N) worst-case time, the total running time or this algorithm would be O(N) average hut O(N Jog N) worst-case. Since this is a special instruction and there arC' no other operati0l15 intervening, and we already know that the instruction can he performed ill lincar average time, it is reasonable to expect that with reasonable care a linear lime bound can be guaranteed,

The general algorithm is to place the N items inio the tree in allY order, maintaining the structure property. Then, if perea 1 ateOown(i) percolates dO\vn from node i, the bui 1 dHeap

routine in figure 6.14 can he used hy the constructor 10 create a heap-ordered tree. The first In'e in Figure 6.15 is the unordered tree. The seven re.maining trees in

Figures 6.1 ') through 6. 111 show 1 he result of Celeh of the seven pereal ateOo\'lns. Each dashed line corresponds to [\\,o comparisons: one to find the smaller child and one to compare the smallcrchikl wit h t he node. Notice that there arc on Iy 1 o dashed lines in the entire algorithm (there eould have been an t lth~whcre?) corresponding t.o 20 comparisons.

To hound the running time of bull dHeap, \ve must bound the number of dashecllines. This can be done by compnting t he sum of t.he heights or <1111 he nodes in the heap, which is the maximulll numher of dashed lines \Vilat we would like to show is that this sum is (l(N).

Theorem 6.1. For the perfect hinary tree of height II containing i 1

j.] - I nodes, the sum Or! he heights of the !lodes is 211 + I - I - (11 + 1).

221



\..11Clf-Ht;1 U rllUlllY \..tUt;Ut;.") \11t;QI-0j

ISO 150

Figure 6.15 Left: initial heap; right: after percolateDOI,n(7)

150 ISO

80 40 80 40

Figure 6.16 Left: after percolateDown(6); right: after percolateDown(5)

150 150

Figure 6.17 Left: after percolateDown(4); right: after percolateDown(3)

Proof. It is easy to see that lhis Lree consists of ! node at height 11, 2 nodes at height h ~ 1,22

nodes at height h - 2, and in general 21 nodes al height IJ - i. The sum of the heights of all the nodes is then



6.4 ApplicatIOns ot Pnonty Queues

ISO 10

10 40 20 40

Figure 6.18 Left: ufter percolateDo"n(2); right: after percolateDo"n(l)

" 5 = L 2'(h - i) i=O

= h + 2(11 - I) + 4(h - 2) + 8(11 - 3) + 16(11 - 4) + ... + 2"-1(1) (61)

Multiplying by 2 gives the equation

25 = 2h + 4(11 - I) + 8(h - 2) + 16(h - 3) + ... + 2"(1) (62)

We subtract these two equations and obtain Equation (6.3). \;Ve find that certain terms almost ({l!1eei. For instance, we have 211 - 2(h - 1) = 2, 4(11 - I) - 4(h - 2) = 4, and so 011. The last term in Equation (6.2), 211 , cloes not appear in Equation (6.1); thus, it appears in Equation (6.3). The first term in Equation (6.1), 11, does not appear H1

Equation (6.2); thus, -It appears in Equation (6.3). We obtain

5=-h+2+4+8+·· .+2',1 +2"=(2"+1 -1) - (h+ I) (63)

which proves the theorem.

/\. complete tree is not a perfect hinary tree, but the result we have obtained is an upper bound on 1 he sum of the heights of (he nodes in a complete t rce. Since a complete tree has between 2i! and 2h+! nodes, this theorcm implies that this slim is O(N), where N is the numher of nodes.

Although the result we have ohtained is sufficient 10 show that buildHeap is linear, the hound on the sum of the heights is not as strong as possible, For a complete tlTe with N = 211 nodes, the bound we have obtained is roughly 2N. The sum of the heights can be shown by induction to be N -h(N), where beN) is the numher of Is in the binary representation ofN.

6.4 Applications of Priority Queues We have already mentioned how priority queues arc used in operating systems design. In Chapter 9, we will sec how priority queues arc used to implement several graph algorithms efficiently. Here we will show how to usc priority queues to ohtainsolutions to two problems.

ZZ5



'-""Y''-' V ",v'''Y ,<U'-U'-J \"'-UY"',

6.4.1 The Selection Problem Thc first prohlem wc will examine is the selectio/l problem from Chapter I, Recall that the input is a list of N clements, which call be totally ordered, and an integer It. The selection prohlern is to lind the IHh largest clement.

1\vo algorithms were given in Chapter I, but neither is vcry efficient. The first algorithm, whicb wc shall call algorithm .lA, is to read the ciemenls inLo all array and sort them, ret urning t he appropriate clcment. Assuming a simple sort ing algorithm, the running! imc is O(N2). The a!tcrnati\'e algorithm, 1 B, is to read h elements into an array and sort them. The smallest of Li1C"se is in the hth position. \;Vc process the remaining ckments olle by one. As an dement arrives, it is compm-ed with the l?th clement in the array If it is larger, then tbe hlh dement is removed, and the new clement is placed ill the correct place among lhe remaining!? - I elemcnt::., \Vhcn the algorithm ends, the clement in the Inh posilion is the answer. The running lime is O(N· It) (why?). If h = rN/21, thell bOlh algorithms arc O(N2). Notice that for <m)' h, we call solve the symmetric problem of finding the (N - !? + 1)th smallest clement, so h = rN /21 is really tbc hardest case for these algorithms. This also happens to be the must interesting else, since this value of I? is known as the mediall.

\Vc givf two algorithms herc, both of which run in 0(1'\1 log N) in the extreme case 01 h = rN/21 , \vhich is a distinct improvement.

Algorithm 6A For simplicity, we assurnc that \Ne arc interested ill finding the hth smallest clemClll. The algorithm is simple. \Ve read the N clements into an array. \Ve then apply lhe bui 1 dHeap algorithm to t his array. l;inally, we perform h de 1 ete~li n operations. The last clement ext rae led from the heap is our answer. It should he clear that by changing the heap-order properly, we could solvc the original prohlem of finding the l?lh ImJ!,fs/ elemene

The correctness or the algorithm should be clear. The worst~case timing is O(J\!) to construct the heap, if bui ldHeap is used, and O(log N) for each de 1 eteMi 11, Since there are h deleteMins, we obtain a total running tillle of O(N + h log N). If I? = O(N/log N), then the running time is dominated hy the bui ldHeap operation and is O(N). For larger values of h, the mnning time is O(h log N). If h = rN/21, then the running lime is ("')(N log N).

Notice thai if we run this program for h = N (\nd record the \'nlues as they leave the heap, we will have essentially sorted the input file in O(N log N) time. In Chapter 7, 'loVe will refine this idea to obtain a fast sorting algorithm known as hcapsorl.

Algorithm 68 For the second algorithm, we return to the original problem and find the J?th largfst clement. \Vc Llse the idea from algorithm lB. At any point in tim(, we will lD<.lintain a set S of the I? i::n-gest clements. Arter the first h clements are read, when a new dement is read it is compared with the I(th largest clemcllt, which we denote by 51<' Notice Ihal Sh is the sma!lesl clemen! in S. 11' the new c1emcnt is larger, then it replaces SII in S. S will then have it new smallest c1emcnt, which mayor may not be the newty added clement. At the end of the inpUl, \ve find the smallest clement in S and return it as the anSWCL



6.4 Applications of Priority Queues

This is l'SSC'l1ti,llly the same algorithm described in Chapter 1 Here, however, we will

lise a hcap to implement S. The first /( elements arc placed into the heap in total time O(/?) with a call to bUi ldHeap. The lime 10 process each of the remailling clemenls is O( I), to tcst if' the element goes into S, plus O(log h), lO delete SII and insert the nC\v element if this is necessary Thus, the tolal lime is O(I? + (N - i?) log h) = O(N log (,,). This algorithm also gives;\ bound of (><)(N log N) for finding the median.

In Chapter 7, we will sec how to solve 1 his problem in O(N) average time. In Chapter 10,

we will sec an elegant, alheit impractical, algorithm to solve this problem in O(N) worst-case time.

6.4.2 Event Simulation In Section 3.7.-"3, we descrihed an important queuing problem. Recall that wc have it systelll, such as a b<lnk, wlwre customcrs arrive and w,lit in a line until onc of h tellers is avclitahlc. Cuslomer i.uriva! is governed by a probahility distrihutiun function, as is the service time (the amount of time to be served once ,\ teller is ilv'lilah!c). Vic arc interesled in statistics such as how long on average ,I customer has to wait or how long the line might he.

\Vith certain probability dislriblllions and values of!?, these answcrs call hc computed

exactly. liowever, as k gets larger, the analysis becollles considerably more difficult, so it is appealing to usc a computer to simulate the operation of the bank. In this way, the hank officers can determine how many tellers arc needed 10 ensure reasonably slllooth service.

A simulation consists of processing cvents. The two C\Tnts here arc (a) a customer arriving and (11) a customer departing, thus freeing up a teller.

\Vc can lise the probability fUllctions to generatc all input stream consisting of ordercd pairs of arrivaltirnc and service tillle for each customer, sorted hy arrival time. 'vVc do not

lleed to usc the_ exact lime of day. Rather, we can usc a quantum unit, which we will refer to as a lid.

One way to do this simulation is to start a simulation clock at zero ticks. 'vVe then advancc the clock one lick al a time, checking to see if there is an event. If Ihere is, then wc process the c\fem(s) and compile statistics. 'vVhcll there arc no custoillers left ill the input stream and all the tellers arc frec, thell the simulation is oveL

The problem wilh this simulation strategy is that its running time docs not depcnd on L he numher of customers or eYents (there are I wo events per customer), but instead depends on the numher of ticks, which is nul really pan of the input. To sec why this is impOn;-1J11, suppose we changed the clock units [0 milliticks and multiplied all the timcs in the input

by ! ,000. The result would he that the simulation would take 1,000 times kmger! The key to avoiding this prohlem is to advance the clock to the next cvent time at each

stage. This is conceptually rasy \0 do. At any point, the ncxt e\'Cllt thal can occur is either (a) the next customer in the inplll fik arrives or (b) olle or the customers at a tdler leaves.

Since all the times when the ('vents will happen arc availahle, we just need to find rhe c\'ent Lhal h<lppcns nearest in lhe future and process that event.

If the evellt is a dcpttrturc, processing includes gathering statistics for the dep;ming custDmer and checking the linC" (queue) to see whether there is another customer wailing.



...... Lnapler b t'flurny I...!ueue~ ~nt.·dP~)

If so, we add that customer, process wh31('ver statistics are required, compute the time \vhcn that customer willlcavc, and add that departure to the set of events wailing to happen.

If the event is an arrival, we check for an available teller. If there is none, \ve place the arrival on the line (queue); otherwise \ve give the customer a telkr, compute the customer's departure timC', and acid the clcparLurc to the set of ('v('nts waiting 10 happen.

The waiting lint' for customers call be implemented as a qucue. Since we need to find the evcnlneares/ in the future, it is appropriate that the set of departures wailing to happen he organized in a priority Cjucuc. The next event is thus the next arrival or next departure (whichever is sooner); both are easily available.

It is then straightforward, although possibly lime-consuming, to write the simulation routines. If there arc C customers (and thus 2C events) and h tellers, then the running time o{ the sil1wialiol1 would he O(e loge/? + I)) because computing and processing each event t~ke.s O(lug 11), \vhere 11 =}{ + I is the siz<: or the heap.3

6.5 d-Heaps Dinary heaps are so simple that they are almost always used when priority queues are needed. A simple generalization is a d-heap, which is exactly like a hinary heap except that all nodes have d children (thus, a binary heap is a 2-hcap).

Figure. 6.19 shows a 3~heap. Notice thelt a d-h<:ap is much shallower than a hinary heap, improving the running time or inserts to O(logd N). I [owevl'r, for large d, the deleteMin operation Is Illore expensive, hecause cven though the tree is shallower, the minimum of d childtTn must be found, which takes d - 1 comparisons using a standard algorithm, This raises the time for this operation to O(d lugd 1\1). if d is a constant, both running times are, of course, OOog N). Although an array can still he used, the multiplications ,mel divisions to find children and parents arc now by d, which, unless d is a power of 2, seriously illcreases the running time, hecause we can no longer implement division by a bit shift. dheaps <Ire inLeresting in theory, hecause there are many <llgorithms where the number of insertions is much greater than the number of deleteMins (and thus a theoretical speedup is possible). They afe also of interest when the. priorily queue is too large to fit entirely in main memory. 1n this casc 1 a d-hcap can be advantageous in much the same way as B-trees. Finally, there is evidence suggesting tbat 4-heaps may outperform binary heaps in practice.

The most glaring weakness of the heap implementation, aside from the inability to

perform fi nds, is lI1tlt combining two heaps into one is a hard operation. This extra operation is kl10wn as a merge. There arc quite a few \\lays of implementing heaps so that the running time of a merge is O(log N), 'vVe will now discuss tbree data structures, of various complexity, that support the merge operation efficiently. We will dder any complicated analysis until Chapter II.

1 We lISC' O(C log (11 + 1) inslead of O(C log h) lO avoid confusion for the It =~ 1 case



6.6 Leftist Heaps

10 13 8 9

Figure 6.19 II d-hcap

6.6 Leftist Heaps IL seems ditTicult to design a data structure that efficiently supports merging (that is, processes a merge in o(N) time) and uses only an array, as in a binary heap. The reason for this is that merging would seem to require copying one array into another, which would take (;<)(N) time for equal-sized heaps. For this rcason, all the advanced data structures that support ci1icicnL merging require the usc of a linked data structure. In practice, \VC can expect I hat this will make all the other operations slower.

Like <l binary heap, a leftist heap has both a structural property and an ordering property Incieed, a leftist heap, like virtually all heaps used, has the same heap-order properly we have already seen. Furthermore, a leftist heap is also a binary tree. The only cWlcrence bet ween a Iclhst heap and a hi nary heap is that leftist heaps are not perfectly

balanced, hut actually attempt to be very unbalanced.

6.6.1 Leftist Heap Property vVc define the null path length, npl(X), of any node X tn be the length of the shortest path

from X to a node without two children. Thus, the I1pl of a node with zero or one child is 0, while npl(NULL) = ~ L In the tree in Figure 6.20, the null path lengths are indicated inside

t he tree nodes. Notice that the null path length of any node is I more than the minimum of the null

path lengths of its children. This applies to nodes \\'ith less than two children hecause the null path length of NULL is ~ l.

The leflist heap property is that for every node X in the heap, the null path lrngth of the left child is at least as large as that of thc right child. This property is satisfled by only one of the trees in Figure 6.20, namely, the tree on the left. This property actually goes OLlt

of' its way to ensure that the treC' is unbalanced, because il clearly biases the tree to get deep

toward the left. Indeed, a tree consisting of a long path of left noell's is possible (and actually preferable to facilitate merging)-hencc thc name l~ftisl heap.

Because leftist heaps tend 10 have deer ldl paths, it follmvs that the right path ought to

be short. indeed, the right path down a leftist heap is as short as any ill the heap. Otherwise,

229



cnapler b I-'flomy l/UeUeS ~HeapS)

o I'" o

o o

o o o

Figure 6.20 Null path lengths for two trees; only the left tree is leftist

there would he a path that gues through some node X and takes the left child. Then X would violate the leftist property.

Theorem 6.2. A lcftisltre(: with r nodes on the right path must have at least 2; - I nodes.

Proof.

The proof is hy induction. If r = I, liwrc must be at Ica~t one tree node. Otherwise, suppose that the theorem is true for \,2, .. , , r. Consider a leftist tree with r + \ nodes on the right path. Then the root has a right suht fee with r nodes on the right path, and a left subtree \vith at least r nodes on the right path (otherwise it would not be leftist)' Applying the inducti\'(' hypothesis to these subtrees yields a minimum of 21" - 1 nodes in each subtree. This plus the root gin's at least 2

,+ J

- ! nodes in the tree, proving the theorem.

From this theorem, it follows immediately that a leftist tree of N nodes has a right path containing at most Llog(N + l)J nodes. The general idea for the leftist heap operations is to perform all the work on the right path, \vhich is guarantced to he short. The only Iricky part is that performing inserts and merges on the right path could destroy the leftist heap property. It turns out to he extremely easy to restore the property.

6.6.2 Leftist Heap Operations The fundamental operation on ldtist beaps is merging. Notice that inserlion is merely a special case of merging, since we may view an insertion as a merge of a one-node heap with a larger heap. V'll' will fIrst give a simple recursive solLuioll and then 5hmv how this might be done llonrecursivdy. Our input is the Iwo leftist heaps, 1-1 J and fh, in Figure h.21. You should check thaI these heaps really art: leftist. Notice that the smallest clements arc at the roots. III addition to space for the data and left and right pointers, each node will have an entry that indicates the llull path lellgth.



6.6 Leftist Heaps

6

10

21 IX

23 26 H, 33

Figure 6.21 T\\'o leftist hCilPS 111 and Il2

12 7

37

33

26

Figure 6.22 Result of merging H2 with HI's right subhc<\p

Jr either of the two heaps is empty, then we carl return the other heap_ Otherwise, to

merge the two heaps, we compare their roots. hrsl, we recursively merge the heap with the larger rool with the right suhhcap or the heap with the smaller moL In our example, this means we recursively merge 1-1} with the sublKap of H1 rooted at' 8, obtaining the heap in Figure 6.22.

Since this tree is formed recursively, and we have not yet finished the description of the algorithm, we cannot at this point show how this heap was ohlainul (--1ow('vcr, it is reasonable to assume that the resulting tree is a leftist heap, because it was obtained via a recursive step. This is much like the inductive hypothesiS in a proofhy inductioll. Since we can handle the base case (which occurs whcn onc free is cmpty), we can aSSUllle that the recursive step works as lOllg i.lS we em finish the merge; this is rule ·3 of recursion, which



In Chapter 6 Pnonty Queues (HeapS)

3

10 6

21 14 12 7

23 ]8 37

26

Figure 6.23 Result of attaching leftist heap of previous figure as Hj's right child

we discussed in Chapter 1. Vve now make this new heap the right child of the root of I-I] (sec Figure 6.23)

Although the resulting heap satis[Jes the heap-order property> it is not leftist hecause the left subtree of the root has a null path length of 1 whereas the right subtree has a null path length of 2. Thus, the leftist property is violated at the roOL "However, it is casy 10 see that the remainder of the tree must he leftist. The right subtree of the root is leftist, because of the recursive step. The left subtree of the root has not been changed, so it too must still be leftist. Thus, we need only to fix the root. \Ve can make the entire tree leftist by merely swapping the root's left and right children (Figure 6.24) and updating the null path lcngth~ the new null path length is 1 plus the null path length of the new right child-completing the merge. Notice that if the null path length is not updated, then all null path lengths \vill

be 0, and the heap will not be leftist hut merely random. In Ihis case, the algorlthrn \vill work, but the time hound we will claim will no longer be valid.

The description of the algorithm translates directly into code. The node class (figure 6.2.'5) is the same as the hinary tree, except that it is augmented with the npl (nul1 path length) data member. The leftist heap stores a pointer to the roOL as its data member. We have seen in Chapter 4 that when an clement is inserted into an empty binary tree, the node referenced by the root will need to change. \Ve use the usual technique of implementing private recursive methods to do the merging. The class skeleton is also shown in Figure 625.

The two merge routines (Figure 6.26) are drivers designed to remove special cases and ensure that HI has the smaller root. The actual merging is performed in mergel (Figure 6.27). The public merge method merges rhs into the controlling hcap. rhs becomes empty. The alias Lest in the public method disallows h,merge(h).



6.6 leftist Heaps

3

6 10

21 14

18 37 23

26

Figure 6.24 Result of swapping children of l-IJ's root

The time to perform the rncrgc is proportional to the sum of the length of the right paths, because constant work is performed at each node visited during the recursive calls. Thus we obtain an O(log N) lime hound 10 merge two leftist heaps. We can also perforrn this operation nonrecursivdy by esscmially performing two passes. In the first pass, we create a new tree hy merging the right paths of both heaps. To do this, we arrange the nodes on the right paths of HI and 112 in sorted order, keeping their respective left children. In our example, the new right path is _), 6, 7, B, 18 and the resulting tree is shown in Figure 6.28. A second pass is made up rhe heap, and cbild swaps are performed at nodes that violate the leftist heap properly. In Figure 6.28, there is a swap at nodes 7 and 3, and the sanle tree as before is obtained. The l10nrecursive version is simpler to visualize but harder to code. Vve leave it to the reader to show that the recursive and non recursive procedures do the same thing.

As mentioned above, we can carry out insertions by making the item 10 be inserted a one-node heap and performing a merge. To perform a del eteMi n, we merely destroy the root, creating two heaps, which can then be merged. Thus, the time to perform a deleteMin is O(log N). These two routines are coded in Egure 6.29 and Figure 6.30.

finally, we can huild a leftist heap in O(N) time by building a binary heap (obviously using a linked implementation). Although a binary heap is clcarly leftist, this is not necessarily the best solution, because the heap we obtain is the worst possible leftist heap. f'urtherrnore, traversing the tree in reverse-level order is not as easy with links. The bui ldHeap dIed can be obtained by recursively builcling the left and right subtrees and then percolating the root down. The exercises contain an alternative solution.



Lnaprer b PrlOrny I...!UeUe~ ~nedP~)

template <typename Comparable>

2 class LeftistHeap

3

4 public:

5 LeftistHeap( );

() LeftistHeap( const LeftistHeap & rhs );

7 ~LeftistHeap( );

R 9 bool isEmpty{ ) const;

10 canst Comparable & findMin{ const;

II 12 void insert( const Comparable & x );

n void deleteMin( );

14 void deleteMin( Comparable & minItem );

/5 void makeEmpty( );

l() void mel'ge( LeftistHeap & rhs );

17 18 canst LeftistHeap & aperator~{ const LeftistHeap & rlls );

19 20 private:

21 struct LeftistNode

22

23

2'1

25 2(i

27

Comparable element;

LeftistNode *left;

LeftistNode *right;

i nt npl;

28 Left is tNode ( const Comparab 1 e & theEl ement, Left i stNode *It NULL,

29 LeftistNode *rt " NULL, int np '" 0 )

30 element( theElement), left( It), right( rt), npl( np) ( )

31 );

32 .13 Left is tNode *root;

34 35 LeftistNode * merge{ LeftistNode *h1, LeftistNode *h2 );

3() Left is tNode * merge 1 ( Left is tNode *hl, Left is tNode *h2 );

37 3R void s\,lapChildren{ LeftistNode *t );

39 void reclaimMemory( LeftistNode *t );

40 LeftistNode * clone{ LeftistNode *t ) const;

4 J );

Figure 6.25 Leftist heap type declarations



/*' 2 * Merge rhs into the priority queue . .3 * rhs becomes empty. rhs must be different from thi s.

4 '/ 5 void merge( leftistHeap & rhs ) 6 {

7 8 9

10

11 12

13

14 /*'

if( this "" &rhs ) return;

II Avoid aliasing problems

root = merge( root. rhs.root ); rhs.root = NUll;

15 * Internal method to merge two roots. 16 * Deals with deviant cases and calls recursive mergel. 17 '/ 18 LeftistNode * merge( leftistNode *hl. leftistNode *h2 ) 19

20 21

22

2.1 24

25 26 27

28

if( hI "" NULL) return h2;

if( h2 "" NULL) return hI;

if( hl->element <

return mergel( else

return mergel(

h2->element hI, h2 );

h2, hI ) ;

Figure 6.26 Driving routines for merging leftist heaps

6.7 Skew Heaps

b.t ::'KeW MeapS

A skew heap is a self~adjusting version of a leftist heap that is incredibly simple 10

implement. The relationship of skew heaps lO leftist heaps is analogous to the relation between splay trees and AVL trees. Skew heaps are binary trees with heap order, hut. there is no structural constraint. on these t.rees. Unlike leftist heaps, no information is maintained ahout the null path length of any node. The right path of a skew heap can be arbitrarily long at any time, so the worst-case running time of all operations is O(N). However, as with splay trees, it can be shown (see Chapter 11) that for any M consecutive operations, the total worst-case running time is O(M log N). Thus, skew heaps have O(log N) amortized cost per operation.

As with leftist. heaps, the fundamental operation on skew heaps is merging. The merge routine is once again recursive, and we perform the exact same operations as before, with one exception. The difference is that for leftist heaps, we check 10 see whether the left and



I /** 2 * Internal method to merge two roots.

:3 * Assumes trees are not empty. and hI's root contains smallest item.

4 *1 5 LeftistNode * mergel( LeftistNode *hl. LeftistNode *h2

6 I II Single node 7

8

o 10

11 12

13 14 15

Iv 17

if( hl->left "" NULL hl->left '" h2: // Other fields in hI already accurate

else

hl->right '" merge( hl->right. h2 );

if( hl->left->npl < hl->right->npl }

sl'lapChildren( hI );

hl->npl ~ h1->right->npl + 1;

return hI;

Figure 6.27 Actual routine to merge leftist heaps

3

23 18

33

Figure 6.28 Result 01 merging right paths 01 Il, and H,

6

17

26

I~

right children satisfy the leftist heap structure property and swap thel11 if they do not. hH skew heaps, the swap is unconditional; we a/ways elo it, with the one exception that the largest of all the nodes on the right paths docs mH have its children s\vapped. This one exccption is what happens in the natural recursive inljJiemct1!atioll, so it is not really a



/*' 2 * Inserts x; duplicates allowed .

.l '/ '1 void insert( canst Comparable & x

5 {

() root '" merge ( nel" Left is tNode ( x ). root ); 7

Figure 6.29 Insertion routine for left ist heaps

j** '2 * Remove the minimum item.

3 * Throws Underfl ol'lExcept i on if empty.

'I */ .5 void deleteMin( )

h {

8 'J

10

II

12

Il

1'1

15 /**

if{ isEmpty{ ) ) throw Underflol'lExceptiQn( );

LeftistNode *oldRoot ~ root; root ~ merge{ root->left. root->right ); delete oldRoot;

j(j * Remove the minimum item and place it in minItem. 17 * Throws Underfl ow Except i on if empty.

18 '/ Ii) void deleteMin( Comparable & minItem )

20 { 21 minItem = findMin( );

22 deleteMin( ); 2.3

Figure 6.30 del eteMi n routine for leftist heaps

6.7 Skew Heaps

special case at all. Furthermore, it is nol necessary to prove the bounds, but since this node is guaranteed not to have a right child, it would be silly to perform the swap and give it one. (In our example, there arc no children of this node, so wc do not \-vorry about it.) Again, suppose our input is the same two heaps as before, Figure 6 . ."3.l.

If we recursively merge I [2 with the subhcap of H I rooted at 8, we will get the heap in hgurc 6.32.

Again, this is done l"Ccursively, so by the third rule of recursion (Section .1.3) wc need not worry ahout how if was obtained. This heap happens to be leftist, but there is no guarantee



238 Chapter 6 Pnonty Queues (HeapS)

3 6

7

21 18 37 18

23 26 H j 33 H,

Figure 6.31 Two skew heaps }-1 I and 1-[2

6

7 12

37 18 24

33

26

Figure 6.32 Result of merging 1-1 2 with HI's right suhheap

that this is always the casco Vve make this heap the new left child of HI, and the old left child of H1 hecomes the new right child (see figure 6.33).

The entire tree is leftist, but it is casy to see that that is not always true: lnserting 15 into this new heap would destroy the leftist property.

We can perform all operations nonrecursivcly, as with leftist heaps, by merging the right paths and swapping left and right children for every node on the right path, with the exception of the last. Alter a few examples, it becomes clear that since all but the last node on the right path have their children swapped, the net effect is that this be-comes the new



6.8 Binomial Queues

3

6 10

7 12 21 14

37 18 24 23

33

26

Figure 6.33 Result of merging skC\v heaps] 1] and }-1 2

left path (see the preceding example to convince yourselll This makes it vcry easy to merge two skew heaps visually."t

The implementation of sknv heaps is left as a (trivial) exercise. Note that because a right path could be long, a recursive implementation could fail because of lack of stack space, evell though performance would otherwise be acceptable. SkC\v heaps have the advantage that no exIra space is required to maintain path lengths and no tests are required to determine when to swap children. It is an open problem to determine precisely the expected right path length of both leftist and skew heaps (the latter is undoubtedly more difficult) Such a comparison would make it easier to determine whether the slight loss of balance information is compensated by the lack of testing.

6.8 Binomial Queues Although both leftist and skew heaps suppon merging, insertion, and deleteMin all effectively in O(Jog N) time per operation, there is room for improvement because we know that hinary heaps support insertion in constant average time per operalion. Binomial queues support all three operations in O(log N) worst~case time per operation, but insertions take constant time on averagc.

·t This is no! exactly tbe same as the recursive implementation (bu! yields the same time bounds). If we

OIl!Y swap children for nodes on the right path that <He above [he point where the merging of right paths terminated due [0 exhaustion of one heap~ right path, we get the same result as the recursive version.



240 Chapter 6 Priority Queues (Heaps)

Bo

o

Figure 6.34 Binomial trees fio, HI, B2 , lL" and B4

6.8.1 Binomial Queue Structure Binomial queues differ from allthc priority queue implementations that we have seen in that a binomial queue is not a heap-ordered tree but rather a collection of heap-ordered trees, known as a forest. Each of the heap-ordered trees is of a constrained form known as a binomial tree (the reason for the name will be obvious later), There is at most one binomial tree of every height. A binomial tree of height 0 is a one-node tree; a binomial tree, 13'1> of height h is formed by attaching a binomial tree, BII _1> to the root of another binomial tree, BII _ I . figure 6.34 shows binomial trees 80, Bll B2, B-,), and B4 .

From the diagram we see that a binomial tree, BIP consists of a root \vith children Bo, B 1,. ., B/I~ 1. Binomial trees of height h have exactly 2/1 nodes, and the number of

nodes at depth d is the binomial coefficient CD. If we impose heap order on the binomial trees and allow at most one binomial tree of any height, we can represent a priority queue of any size by a collection of binomiallrees. For instance, a priority queue of size 13 could be represcnt.ed by the forest B3, B2, Bo. We might write this representation as 11.01, \vhich not only represents 13 in binary but also represents the fact that B1 , E2 , and Bo are present. in the representation and B 1 is not.

As an example, a priority queue of six elements could he represented as in Figure 6.35.



® ®

12

65

Figure 6.35 Binomial queue HJ with six elements

H 2 : @

® ®

® @)

12

65

23

Figure 6.36 Two binomial queues 111 and H2

6.8.2 Binomial Queue Operations

6.8 Binomial Queues

The minimum element can then he found by scanning the roots of all the trees, Since there arc at most log N different trees, the minirnul11 can be found in O(log N) time. AlLernatively, we can maintain knowledge of the minimum and perform the operation in O(l) time, if we remember to update the minimum when it changes during other operations.

Merging two binomial queues is a conceptually easy operation, which we will describe by example. Consider the two binomial queues, Ii} and H2 , with six and seven elements, respectively, pictured in Figure 6.36.

The merge is performed hy essentially adding the two queues together. Lel H-l be the new binomial queue. Since HI has no binomial tree of height 0 and Ih does, we can jusl use the binomia.l tree of height 0 in Ih as part of H"). Next, \ve add binomial t.rees of height. 1. Since hoth H I and H2 have binomial trees of height 1, we merge them by making the larger root a subtree of the smaller, creating a binomial tree of height 2, shown in Figure 6.37. Thus, lh wilt not have a binomial tree of height 1. There arc nov .. ' three binomial t.rees of height. 2, namely, the original trees of H! and 112 plus the tree formed by the previous step. We keep one binomial tree of height. 2 in HJ and merge the other two, creating a binomial tree of height 3. Since Hl and H2 have no trees of height 3, this tree becomes part of H3 and we are {inished. The resulting binomial queue is shown in Figure 6.38.

Since merging two binomial trees takes constant time with almost any reasonable implementation, and there are O(log N) hinomial trees, the merge takes O(log N) time in the worst case. To make this operation efficient, we need to keep the trees in the binomial queue sorted by height, which is certainly a simple thing to do.




14

18

Figure 6.37 Merge of the two HI trees in III and J-i2

23

51 24 24 14

65 65

18

Figure 6.38 Binomial queue J I): the result of merging 111 and 112

Figure 6.39 After t is inserted

Figure 6.40 Alter 2 is inserted

lnsertion is just a special case of merging, since we merely create a one-node tree and perform a merge. The worst-case lime of this operation is likewise O(log N). More precisely, if the priority queue into which the clement is being inserted has the property that the smallest nonexistent hinomial tree is Bi , the running time is proportional to i + !. for example, 1-1.3 (figure 6,38) is missing a binomial tree of height 1, so the insertion will terminate in two steps. Since each tree in a binomial queue is present with probability 1, it follows that we expect an insertion to terminate in two steps, so the average lime is constant. Furthermore, an analysis will show that performingN inserts on an initially empty binomial queue will take O(N) worst-case time. Indeed, it is possible to do this operation using only N - 1 comparisons; we leave this as an exercise.

As an example, we show in Figures 6.39 through 6.45 the binomial queues that arcformed hy inserting 1 through 7 in order. Inserting 4 shows off a bad case. We merge 4 with Ho, obtaining a nt::\v tree of height 1. vVe then merge this tree with HI, obtaining a tree of height 2, which is the new priority queue. We count this as three steps (two tree merges



6.8 Binomial Queues

Figure 6.41 Afler 3 is inserted

2 3

4

Figure 6.42 After 4 is inserted

4


4


plus the stopping case). The next insertion after 7 is inserted is another bad case anel would require three tree merges,

A de 1 eteMi n can be performed by first finding the hinomial tree with the slnallest root. Let this tree be BII , and let the original priority queue be H. We remove the binomial tree HII from the forest of trees in I-i, forming the new binomial queue HI. We also remove the root of BII> crealing binomial trees Bo, El , ... , BII _ I , which collectively form priority queue HI/. We finish the operation by merging HI and HI/.

As an example, suppose we perform a deleteMin on H1, which is shown again in Figure 6.46. The minimum root is 12, so we obtain the two priority queues HI and HI! in Figure 6.47 and Figure 6.48. The binomial queue that results from merging HI and HI!

is the final answer and is shown in Figure 6.49.




4

Figure 6.45 Afler 7 is inserted

23

14

65 65

Figure 6.46 Bi nomial queue 1-1)

@ JI':v

23

65

Figure 6.47 Binomial queue HI) containing all the binomial trees in H-l except lh,

fI"@ ® ®

14

18

Figure 6.48 Binomial queuc fi": B-) with 12 removed

18

f70r the analysis, note first that the deleteMin operation breaks the original binomial queue into two. It takes O(log N) time to find the tfee containing the minimum clement and to create the queues H' and II", iVkrging these two queues takes OOog N) time, so the enlire del eteMi n operation takes OOog N) time.

6.8.3 Implementation of Binomial Queues The deleteMin operation requires the ability to {Ind all the subtrees of the root quickly, so the standard representation of general trees is required: The children of each node are kept



23

65

Figure 6.49 Resull of applying de 1 eteMi n lo H'j

23 12

51 24 21

65

Figure 6.50 Binomial queue lh drawn as a forcst

18

Figure 6.51 Representation of binomial queue H1

65

24

65

65

14

14

b./j nmOmJat I....!ueues

18

~ ~ ~®

in a linked list, and each nodI.' has a pointer to its firsl child (if any). This operation also requires t hat I he children be ordered by the size of th6r subt rccs. Vic also need to make sure that it is easy to merge two trees. When lwo trees arc merged, one of the trees is added as a child to the other. Since this new tree will be the largest subtree, it makes sense to maintain the subtrees in decreasing sizes. Only then will we be able to merge two binomial trees, and thus two binomial queues, efficiently The binomial queue will be an array of binomial trees.

To summarize, then, each node in a binomial tree will con Lain the data, first child, and right sibling. The children in a binomial tree are arranged in decreasing rank.

Figure 6.') I. shows how the binomial queue in Figure 6.50 is represented. Figure 6.52 shows the type declarations for a node in the binomial trec, and the binomial queue class intfrfacc.



..... ,1<11-'''';, u "'U'''Y \,(u<-u<-~ \" ... u .... ~1

template <typename Comparable> 2 class BinomialQueue 3

4 public: 5 BinomialQueue( ); 6 BinomialQueue( const Comparable & item ); 7 BinomialQueue( const BinomialQueue & rhs ); 8 -BinomialQueue( ); 9

10 bool isEmpty( ) canst; 11 const Comparable & findMin( ) const; 12

13 void insert( canst Comparable & x ); 14 void deleteMin( ), 15 void deleteMin( Comparable & minItem ); 16 17 void makeEmpty( ), 18 void merge( BinomialQueue & rhs ); 19

20 const BinomialQueue & operator~ ( const BinomialQueue & rhs ); 21 22 private: 23 struct BinomialNode 24

25 26 27

28

Comparable element; BinomialNode *leftChild; BinomialNode *nextSibling;

29 30

31

BinomialNode( canst Comparable & theElement. BinomialNode *1t. BinomialNode *rt )

: element( theElement 1, leftChild( It ), nextSibling( rt ) { } .12 } , 33 .14 enum { DEFAULT_TREES I}, 35 36

37

38

lnt currentSize; vector<BinomialNode *> theTrees;

39 lnt findMinlndex( ) const; 40 int capacity( ) canst;

II Number of items in priority queue II An array of tree roots

41 BinomialNode * combineTrees( BinomialNode *tl, BinomialNode *t2 ); 42 void makeEmpty( BinomialNode * & t ); 43 BinomialNode * clone{ BinomialNode *t ) canst; 44 },

Figure 6.52 Binomial queue class interface and node definition



Q.O DII IUllliai l!UI:W::)

12 12

14 + 65

18 18

Figure 6.53 lvlerging two binomial trees

/** 2 * Return the result of merging equal-sized t1 and t2.

J *j 4 BinomialNode * combineTrees( BinomialNode *t1, BinomialNode *t2 )

5 { 6 if( t2->element < t1->element )

7 return combineTrees( t2~ tl );

8 t2->nextSibling "" tl->leftChild;

9 tl->leftChild " t2;

10 return tl;

11

Figure 6.54 Routine to merge two equal-sized binomial trees

In order to merge two binomial queues, we need a routine to merge two binomial trees of the same size. Figure 6.53 shows how the links change when two binornial trees arc merged. The code to do this is simple and is shown in Figure 6.54.

We provide a simple irnplcmentatioll of the merge routine. III is represented by the current object and H2 is represented by rhs. The routine comhines HI and Hz, placing the result in HI and making lT2 empty. At any point we are dealing with t.rees or rank i. t1 and t2 arc the trees in H] and Hz> respectively, and earry is the tree carried from a previous step (it might be NULL). Depending on each of the eight possible cases, the tree Ihal results for rank i a.nd the carry tree of rank i + 1 is formed. This process proceeds from rank 0 to the last. rank in the resulting binomial queue. The code is shown in Figure 6. '55. Improvements to the code are suggested in Exercise 6.3'5.

The del eteMi n routine for binomial queues is given in Figure 6.56 (on pages 249-2 '50). We can extend binomial queues to support. some of the nonstandard operations that

binary heaps allow, such as decreaseKey and remove, \vhen the position of the affected clement is known. A decreaseKey is a perea' ateUp, which can be perfonnecl in O(log N) time if we acId a data member to each node that stores a parent link. An arbitrary remove

can be performed hy a comhination of decreaseKey and deleteMin in O(log N) time.

" .. , Bismillah hiir Rehman nir Raheem-----------------------------Assalat o Wasalam o Alika Ya RasoolALLAH


""'''1-'''''' v """"1 '-t ............... \', .............. /

/** 2 * ~1erge rhs into the pri or; ty queue . .3 * rhs becomes empty. rhs must be different from this. 4 */ 5 void merge( BinomialQueue & rhs ) [, (

7

8

9

10

II

12 13 14 15

16

17 18

19

20

21

22 23

24 25 26 27

28 29 30

.11

.12

33 34 35 36 37 38 39 40

41 42

43

if( this "" &rhs ) return;

II Avoid aliasing problems

cunentSize += rhs.currentSize;

if( currentSize > capacity( ) ) (

lnt oldNumTrees '" theTrees.size( ); int newNumTrees ~ max( theTrees.size( ), rhs.theTrees.size{ ) ) + 1; theTrees.resize( newNumTrees ); fore int i ~ oldNumTrees; i < newNumTrees; i++ )

theTrees[ i 1 " NULL;

BinomialNode *carry = NULL; fore int i = 0, j ~ 1; j <= currentSize; i++. j *= 2 )

BinomialNode *tI theTrees[ i ]; BinomialNode *t2 i < rhs.theTrees.size( ? rhs.theTrees[ i ]

NULL; int whichCase '" tl == NUll? 0 : 1; whichCase += t2 NULL? 0 : 2; whichCase += carry"'''' NULL? 0 : 4;

switch( whichCase ) {

case 0: 1* No trees *1 case 1: 1* Only this *1

break; case 2: 1* Only rhs *1

theTrees[ i 1 t2; rhs.theTrees[ i 1 " NULL; break;

case 4: 1* Only carry *1 theTrees[ i ] '" carry; carry = NULL; break;

Figure 6.55 Routine to merge: two priority queues



case 3: /* this and rhs *j carry = combineTrees( tl, t2 ); theTrees[ i ] '" rhs.theTrees[ i

break; case 5: J* this and carry *j

carry ~ combineTrees( tI, carry); theTrees[ i 1 0 NULL;

break; case 6: j* rhs and carry */

carry ~ combineTrees{ t2, carry); rhs.theTrees[ i 1 0 NULL;

break;

case 7: j* All three */ theTrees[ i ] '" carry; carry = combineTrees( tl, t2 ); rhs.theTrees[ i 1 0 NULL;

break;

NULL;

4-1

45 46

47 48 49

50 51

52 53

54 55 56 .57

58 59 60

61

62 63 64

65 66 67

fore int k '" 0; k < rhs. theTrees.size( ); k++ ) rhs.theTrees[ k 1 0 NULL;

rhs,currentSize '" 0;


/** 2 * Remove the minimum item and place it in minItem . .3 * Throws Underfl owExcepti on if empty.

4 *j 5 void deleteMin( Comparable & minJtem )

6 (

7 if( isEmpty( ) )

8 throw UnderflowException( ); 9

10

11

t2

int ffiinlndex = findMinlndex( ); minItem = theTrees[ minIndex ]->element;

Figure 6.56 deleteMin for binomial queues

.£'."



I.l 14 15

If>

17

IR

19

20 21 22 2J 24

25 26 27 28 29 30

31

32 Tl

34 35 /**

8inomialNode *oldRoot '" theTrees[ minlndex ];

BinomialNode *deletedTree '" oldRoot->leftChild;

delete oldRoot;

II Construct H' 1

BinomialQueue deletedQueue; deletedQueue,theTrees.resize( minIndex + 1 ); deletedQueue.currentSize = ( 1 « minlndex ) - 1; for( int j '" minlndex - 1; j >= 0; j-- }

{

deletedQueue,theTrees[ j ] '" deletedTree; deletedTree ~ deletedTree->nextSibling; deletedQueue.theTrees[ j ]->nextSibling = NULL;

II Construct H' theTrees[ minIndex ] '" NULL; currentSize -= deletedQueue.currentSize + I:

merge( deletedQueue ):

36 * Find index of tree containing the smallest item in the priority queue.

37 * The priority queue must not be empty.

38 * Return the index of tree containing the smallest item.

39 *j 40 int findMinIndex( const 41

42 43 .1'1

If.')

46 47 48 49 50

51

.52 53 54

i nt i; int minlndex;

for( i " 0; theTrees[ i 1 NULL; i++ )

for( minIndex oo;; ; < theTrees.size( ); i++)

if( theTrees[ i 1 !" NULL && theTrees[ i ]->element < theTrees[ minIndex )->element minIndex'" i;

return ffiinIndex:




6.9 Priority Queues in the Standard Library The binary heap is implemented in the STL by the class template named priority_queue found in the standard header file queue. The STL implements a max~heap> rather than a min~ heap, so the largest rather than smallest item is the one thal is accessed. The key member functions afC:

void push{ canst Object & x ); canst Object & tope ) const; void pop( );

bool empty( );

void clear( );

push adds x to the priority queue, top returns the largest clement in the priority queue, and pop removes the largest element from the priority queue. Duplicates are allowed; if there arc several largest elements, only one of them is removed.

The priority queue template is instantiated with an item type, the container type (almost always you would want to usc a vector that stores the items), and the comparator; defaults are allowed for the last two parameters, and the defaults yield a max-heap. Using a greater

function object as the comparator yields a min-heap. Figure 6.57 shows a test program that illustrates how the pri or; ty _queue class template

can be used as both the default max-heap and a min-heap.

Summary

In this chapter we have seen various implementations and uses of the priority queue ADT.

The standard binary heap implementation is elegant because of its simplicity and speed. It requires no links and only a constant amount of extra space, yet SUpp0rlS the priority queue operations efficiently.

We considered the additional merge operation and developed three implementations, each of which is unique in its own way The left.ist heap is a wonderful example of the power of recursion. The skew heap represents a remarkable data structure because of the lack of balance criteria. Its analysis, which we will perform in Chapter 1.1, is interesting in its own right. The binomial queue shows how a simpJr idea can be used to achieve a good time hound.

We have also seen several uses of priority queues, ranging from operating systems scheduling to simulation. We will sec their use again in Chapters 7, 9, and 10.

Exercises

6.1 Can both insert and fi ndMi n be implemented in constant time?

6.2 a. Show the result of inserting 10, 12, I, 14,6, '5, 8,1'5,3,9,7,4, II, 13, and 2, ont at a time, into an initially empty binary heap.

.. "'. Bismillah hiir Rehman nir Raheem-----------------------------Assalat o Wasalam o Alika Ya RasoolALLAH


'-'IUI-'"'-' U "'V"'Y l..{U'-U'-" \"<'UI-''')

I #include <;ostream> 2 #include <vector> J #include <queue> 4 #include <functional> 5 #include <string> 6 using namespace std; 7

8 II Empty the priority queue and print its contents. 9 template <type name PriorityQueue>

10 void dumpContents( canst string & msg. PriorityQueue & pq } II ( IL cout « msg « ":" « endl;

Ll while( !pq.empty( )

14 (

15 cout«pq.top( «endl; 16 pq.pop( );

17 18 19 20 II Do some inserts and removes (done in durnpContents). 21 tnt main( ) 22

pr;or;ty~queue<int> rnaxPQ; 2'/ pri ori ty _ queue, greater > mi nPQ;

25 26 minPQ.push( 4); minPQ.push( 3); minPQ.push( 5);

27 maxPQ.push( 4 ); maxPQ.push( 3 ); maxPQ.push( 5 );

28 29 30

31

dumpContents ( "m; nPQ". m; nPQ ); dumpContents ( "maxPQ". maxPQ );

/ / 3 4 5

/ / 5 4 3

32 return 0; J.l

Figure 6.57 Routine that demonstrates the STL priority_queue; the comment shmvs the expected order of output

h. Show the result of llsing the linear-time algorithm to huild a binary heap u~ing the same input.

6.3 Shmv the result of performing three del eteMi n operations in the hC<lp of the previous exerCIse.

6.4 A complete binary tree of N eknwllts uses array positions! to N. Suppose \VC try 10 use an <lrray representation of a binary tree that is not complete. Determine how large the array must be for the following:



I:xeroses

<l. a hirwry tn'e then has two extra levcls (tha! is, it b vcry slightly unhalanctd) b. a binary trce 111<\t has a deepest node at depth 2 log N c. a binary tree that has a deepest node at depth 4.1 Jog N d. the \vorst-case binary tree

6.5 Rewrite the BinaryHeap inse)'t routine hy placing <.1, copy 0/ the inserted item in po.siLion O.

6.6 llow many nodes arc in thc large heap in figure 6. 13?

6.7 a. Prove that for hillary heaps, bui 1 dHeap docs at most 2N - 2 comparisons bet\'\'cen elements.

h. Show thal <l heap of eight clullents can he constructed in eight comparisons betweell heap clements.

** c. Civc an algorithm to build a binary heap in ljN + O(log N) element compariSOTls.

6.8 Show the following regarding the Illaximum item in the heap: <1. 11 must be at one of the leaves. b. There arc exactly rN/21Ieavcs. c. Every leaf must be examined to find it.

** 6.9 Show that the expectcd depth of the 1tt.h smallest clement in a large complete heap (you may assume N = i' -- I) is hounded hy log h.

0.10 <I. C;ivc an algorithm to find all nodes less than some value, X, ill a hinary heap_ Your algorithm should fUll in O(K), where K is the number of nodes outpul.

h. Docs your ,,\Igorithnll'xtend to any of the other heap structures discussed inlhis chaptcr?

* c. Give an algorithm that finds an arbitrary item X in a binary heap using at most roughly '30,1/4 comparisons.

** 6.11 Propose all algorithm to illsrn lvl nodes into a binary heap on N elements in nUd + log N log log N) time. Prove your lime bOllncL

6.]2 \Vrite a program to take N clements and do the following: i.1. Insert them into a heap one hy one, h. Build a heap in Iincar time. Compare the running tillle of hoth algorithms for sOr\rc\, reverse-ordered, and rand()lll inl)ll[s.

6.13 Each deleteMin operation uses 2 Jog N comparisons in the worst case . .. a. Propose a schellK so that the del eteMi n operation uses only log N + log log N +

O( I) comparisons hetwcen clements. This need not imply less data movement. ** b. Extencl your scheme in part (a) so lilat only log N + log log log N + O( 1) com

parisons arc performed. H c. How far can YOll take this idea?

d. Do the savings in comparisons compensate for the increased eomplc-xity of your algorithm?

6.14 If a £I-heap is stored as an ,lYra}" for an ('Ill ry located in position i, where arc the parents and children?

25)



.... "ul ....... ' v ",U, .. , "-( .. " ......... .J " .......... -'1

6

81 87

42

Figure 6.58 Min-max heap

6.15 Suppose we need to perform M percolateUps and N deleteMins on a d-heap that initially has N elements. a. What is the total running time of all operations in terms of M, N, and d? b. If d = 2, what is the running time of all heap operations? c. If d = 8(N), what is the total running time'

* d. vVhat choice of d minimizes the total running time?

6.16 Suppose that binary heaps are represented. using explicit links. Give a simple algorithm to fmd the tree node that is at implicit position i.

6 .. 17 Suppose that hinary heaps are represented using explicit links. Consider the problem of merging binary heap 1 hs with rhs. Assume both heaps are full complete trees, containing 2! ~ 1 and 21" - 1 nodes, respectively. a. Give an O(log N) algorithm to merge the two heaps if 1= r. b. Give an O(log N) algorithm to merge the two heaps if 11- rl = 1.

* c. Give an O(log2 N) algorithm to merge the two heaps regardless of I and r.

6.18 A min-max heap is a data structure that supports both deleteMin and deleteMax in O(log N) per operation. The structure is identical to a binary heap, but the heaporder property is that for any node, X, at even depth, the element stored at X is smaller than the parent but larger than the grandparent (where this makes sense), and for any node X at odd depth, the dement stored at X is larger than the parent but smaller than the grandparent. See Figure 6.58. a. How do we find the minimum and maximum elements?

* h. Cive an algorithm to insert a new node into the min~max heap. * c. Give an algorithm to perform deleteMin and deleteMax.

* d. Can you build a min-max heap in linear time?



txerCIses

2 4

II 5 9

12 17 8 18 10

18 15 31

Figure 6.59 lnput for Exercises 6 J 9 and 6.26

H C. Suppose we would like to support deleteMin, deleteMax, and merge. Propose a

data structure (0 support all operations in O(log N) lirne.

6.19 lv1crge the two leftist heaps in Figure 6.'59.

6.20 Show t he result of inserting keys .I to .I "j in order into an initially empty leftist heap.

6.21 Prove or disprove: J\ perfectly hal anced tree forms if keys .I to i l - 1 are inscrted in

order into an initially cmpty leftist heap.

6.22 C;ive an example of input that generates the best leftist heap.

6.2.3 CI. Can leftist heaps ef(icicntiy support decreaseKey?

h. \~/hat changes, if any (ir possible), arc required 10 do this?

6.24 One way to dcleLC nodes from a known position in a leftist heap is 10 usc a lazy 5t ratcgy. To delete a node, merely mark it delctcd. \,-Vhen a fi nd~li n or de 1 eteMi n is performed, there is a potential problem if the root is marked deleted, since then the node has to be act.ually deleted and the real minimum needs to be found, which may involve deleting other marked nodes. In this strategy, removes cost one unit, but the cost of a de 1 ete~li nor fi ndMi n depends on the numher of nodes that are marked deleted. Suppose that after a deleteMin or find~1in there are h fewer marked nodes than before the operation.

* CI. Show how to perform the del eteMi n in O(h log N) time. ** b. Propose an implementation, with an analysis to show that the timc 10 perform

the deleteMin is O(i1log(2N/II».

6.25 We can perform bui 1 dHeap in linear lime for leftist heaps by considering each element as a one-node leftist heap, placing all these heaps on a queue, and performing the following step: Until only one heap is on the queue, dequeuc two heaps, merge them, and enqueue the result. CI. Prove thaI this algorithm is O(N) in the worst casco b. \-\lhy might this algorithm be preferable to the algorithm descrihed in the text?

6.26 Merge the two skew heaps in figure 6.')9.

6.27 Show the result ofinscrLing keys 1 to 1'5 in order into a skew heap.

6.28 Prove or disprovc: A perfect Iy balanced tree forms if the ke,ys 1 to 211 - 1 aJ"c inserted in order inlo an initially empty skew heap.



......... 1" ..... v • " .... , ,., '<U ... U ... " \' ''''''''1''''1

~ 23

@ ®

2

Figure 6.60 Input [or Exercise 6.12

14

18

55

6.29 A skew heap of N elements can be built using the standard binary heap algorithm. Can we use the sanK merging strategy described in Exercise 6.25 for skew heaps to get an O(N) running time?

6.30 Prove that a binomial tree H/I has binomial trees Bo, B1, ... , BII -1 as children of the rool.

6.3] Prove that a binomial tree of height k has CD nodes at depth d.

6.32 Merge fhe two hinomial queues in Figure 6.60.

6.33 a. Show that N inserts into an init.ially empty binomial queue take O(N) time in the worst casco

h. Give an algorithm to build a binomial queue of N elements, using at most N - .I comparisons between elements.

* C. Propose an algorit.hm to insert M nodes into a binomial queue of N elements in OeM + log N) worst-case time. Prove your bound.

6.34 Writ.e an efficient routine to perform insert using binomial queues. Do not call merge.

6.35 for tbe binornial queue: a. Modify the merge routine to terminate merging if there are no trees left in H 2 and

the carry tree is NULL.

h. Modify the merge so that the smaller trce is always merged imo the larger.

** 6.36 Suppose we extend binomial queues to allow at most two trees of the same height per structure. Can we ohtain 0(1) worst-case time for insertion while retaining O(log N) for the other operations?

6.37 Suppose you have a number of boxes, each of which can hold total weight C and items i1, i2 , iJ , ... , iN, which weigh H'I, H'2, Hl3, ... , wN, respectively. The object is 10 pack all the items without placing more weight in any box than ils capacity and using as few boxes as possible. for instance, if C = 5, and the items have weights 2,2,3,3, then we can solve the problem with t.wo hoxes.



Kererences

In general, this problem is very hard, and no dflciCnl solution is known. Write programs to implement efficiently the following approximation strategies:

* a. Place the weight in the first box for which it filS (creating a new box if there is

no box with enough room). (This strategy and all that follow would give three boxes, which is suboptimal.)

b. Place the weigh! in the box with I he most room for it. * c. Place the weight in the most lilled box that can accept it without overflowing.

** d. Are any of these strategies enhanced by presoning the items by weight?

6.38 Suppose we \vant to add the decreaseAll Keys(6.) operation to the heap repenoire. The result of this operation is that all keys in the heap have their valLIe decreased by an amount 6.. For the heap implementation of your choice, explain the necessary modifications so that all other operations retain their running times and decreaseA 11 Keys runs in O( I).

6.39 vVhicb of the two seleclion algorithms has the better time bound?

6.40 The standard operator'"' and makeEmpty for leftist heaps can fail because of Loo lllallY recursive calls. Although this was true for hinary search trees, it is more problematic for leftist heaps, because a leftist heap can be vcr)' deep, evell while it has good worst-case performance for basic operations. Thus operator= and makeEmpty need to

be reimplcmented to avoid deep recursion in leftist heaps. Do this as follows: a. Reorder the recursive routines so that the recursive call to t->left follows the

recursive call to t->right, b. Rewrite the routines so that the last statement is a recursive call on the left suhtree. c. Eliminate the tail recursion. d. These functions arc still recursive. Give a precise bound on the depth of the

rernaining recursion. * e. Explain how to rewrite operator= and makeEmpty for skew heaps.

References

The binary heap was first described in 12BI. The linear-time algorithm for its construction is from [141.

The Ilrst description of d-heaps was in /19]. Recent results suggest that 4-heaps may improve binary heaps in some circumstances 122]. Leftist heaps were invented by Crane ! I I] and described in Knuth 12!]. SkC\v heaps were developed by SIcator and 'L\~jan 124]. Binomial queues were invented by Vuillemin [27]; Brown provided a detailed analysis and empirical study showing thm they perform \vell in practice [41, if carefully implemented.

Exercise 6.7(b-c) is taken from 117J. Exercise 61O(c) is from [6J. A method for constructing binary heaps thai lISC'S about L'52N comparisons on average is described ill 123J. I.azy deletion in leftist heaps (Exercise 6.24) is from I] 0]. A solution to Exercise 6.36

can be found in 1B]. Min-max heaps (Excrcise 6.18) wcre originally described in [IJ. A rnorl'. efficient

implementation of the opcratic)l1s is given in 118[ anel [25]. Alternative representations for double-ended priority queues are the deap and diamond deque. Details can be founel in [51, [71, and [91. Solutions LO 6.18(e) are given in [12J and [201.

Z51



.... "ul-"'--, V ",v"'Y ,-<U'-U'--' \"'--UI-'-'I

A theoretically interesting prioril y queue representation is the fibonacci Iwop [ 16], \vhich

we will describe in Chapter I I.. The Fibonacci heap allows all operations to he performed in

O( I) amortized time, except for deletions, which arc O(log N). Relaxed heaps! 131 achieve

identical bounds in the worst case (with the exception of merge). The procedure of [3]

achieves optimal worst-case houncls for all operations. Another interesting implementation

is the pairing heap [IS], which is described ill Chapter 12. Finally, priority queues that work

when the data consist of small illtegers afC descrihed in 121 and !26].

1. lv1. D. Atkinson, J. R. Sack, N. Santoro, and T Strothottc, "lvlin-i'vtax Heaps and Ceneralized Priorit y Quelles," GlnllmmiwtiollS (:f the ACM, 29 (1986), 996-1000.

2. J. D. Bright, "Range Restricted lvtcrgcahk Priority Queues," InJormatiol11-'roccssin,l!, Leiters, "t7 (1993),IW-IM

3. C. S. BrodaL "\Vorst-Case Efficiclll Priority Quelles," Procct'dill,~s or the Sevcnrl] Annual ACM-SIAM Symposium on IJiscfrlf A/g,orill!1ils (1996), ')2-')8.

4. Ivt. R. Brown, "Implemcntation and Analysis of Binomial Queue Algorithlns," SJAMJoti.nwl

Oil Compuling, 7 (197H), 298-,"319.

5. S. Carlssoll, "The Dcap~;\ DOllbk-[:nded Heap to lrnplcnKnt Double-Ended Priority Queues," Information Pmcfssill{~ I_ctters, 26 (ltJ87), 33--)6.

6. S. Carlsson anel J Chen, "The Complexity of Heaps," Proceedings of the Third Symposilllll 011

Discrete Algorithms (1992), :N.3~402.

7. S. Cadsson,J Chen, and T. StrotilOllc,"1\ Note on the Construction orthe Data Structure 'Deap'," h!{on1wliol1 Processing, LeiLers, 31 (19H9), 315~317.

8. S. Carlsson, J 1. tvtUlll"O, and P V Poblctc, "An Implicit Binomial Queue with Constant illsertion Time," })locccdillgS (~r /'"irsl Sumdil1(1l'i(lll \Vori?sllOp on ;\/,,<priti1m T]1fIny (198(;), 1- 1:\.

9. S. C Chang and \4. \V Due, "Diamond Dcque: A Simple Data Structure for Priority Deqlles,"

JnfiJr/llalion Processing, Lellers, 46 (19~n), 2 i 1-~23 7.

10. D. Cheriton and R. L "Lujan, "fil1ding l'Vlinimum Spanning Trees," SJAlvj Journal Oil

Computing,5 (976), 724-742.

11. C A Cranc, "Linear Lists and Priority Queues as Balanced Binary Trees," Tedmiral Report STi\N-CS-72-259, Compllln Science Department, Stanford University, Stanford, CaliL, 1972.

J 2. Y. Ding and i",,1. A \Vejss, "The Rrlaxed lvlin-lvlax Heap: i\ Iv1ergcahlc Double-Ended Priorit),

Queue," Acln l!~f(mn(jtic(/, .10 (1993), 21 '5-231.

13. 1. R. Driscoll, II. N. (~abo\V, R. Shrairillall, and R. E. "Lujan, "Rchlxed \-leaps: An Alternative to FihonaccI I leaps with Applicatiolls to Parallel Computation," ComHluniwlions of/he ACId, II (J9BB), n4l-Jl54.

14. R. 'vV noyd, "Algorithm 245: Treesorl 3," Communications of the i\Uvf, 7 (1964), 701.

1 '5. lvL L. Fredman, R. Scdgewick, D. D. Skator, and R. E Tal:j<ln, "The Pairing Heap: i\ New Form orSc!f-adjusling Heap," Alwnithrnica,.1 (1986), 111-129.

16. 1'1,11. L Fredman (lnd R. E. 1:1rJan, "Fibonacci 1 leaps and Thcir Uses ill Improved Network Optimization Algorithms," )our/wl of the I\CA{ :34 (1987), '596~61 '5.

17. G. II. COline! and J. L Munro, "\ leaps on I leaps," SlAAJ jour/Jal 011 Computing, t 5 (1986), l)(Yt-l) 7 1 .

18. A. Ilasham andJ. R_ Sack, '"Bounds ii)!- Min-max lleaps," Bn: 27 (1l)87), ") I 1-'n3



t{ereren(e~

19. D. n, Johnson, "Priority Qucucs with Update and Finding Minimum Spanning Trees," lI~formaU()l1 Processing. Lellers, 4 (197'5). '5:3-'57.

20. eM. Khoong and H. Vv. Leong, "Double-Ended Binomial Queues," Prou:edinf!,s (!f the hnlrti1 Anmwl International Symposium on Algorithms (JIu1 Computation (l99~»), 128-1 37.

21. D. L Knuth, The ArU!f Comput.er Programmill,t;, Vol. 3: Sorting (lnd Sean:hillg, 2d cd., AddisonWesley, Reading, Mass., 1998,

22. A LaMarca and R. E Ladner, "The InHucnce of Caches on the Performance of Soning," Proceedings (?f the Eighth Annual i\CM~Sl!\M Symposium on DiscrCle Alg,orithms (1997), 370-379.

2-3. C]. I-I. McDiarmid and B. A. Reed, "Building Heaps Fast," Journal (if Algorithms, 10(1989), 352-365.

24. D. D. Sleator and R. L Tmjan, "Sclf-;:ldjusting I leaps," SIA;\;1}ournal on Computing, 1'5 (1986), 52-(19

25. T. Strothotte, P hilzsson, and S. Vallner, "A Note 011 Construcling Min-max Heaps," Tin; 29 (1989),251-256.

26. I~ van Emde Boas, R. Kaas, and E. Zljlstra, "Design and Implementation of an Efficient Priorily Queue," Jvlathematical Systems Themj, 10 (1977), 99-127.

27. ]. Vuillemin, "A Data Structure for Manipulating Priority Queues," Comnml1iwlions (~r lhe ACM, 21 (1978),,09-314

28. J. \V J Vvilliams, "Algorithm 212: I--lcapsort," Communications (?f [he ACM, 7 (t 9(4), 347--

348.

":,,, Bismillah hiir Rehman nir Raheem-----------------------------Assalat o Wasalam o Alika Ya RasoolALLAH

Documents

Data structures and algorithm analysis (3rd ed) by mark allen weiss (p1)