Algorithms and Data Structures
Objectives: To review the fundamental algorithms and data structures that are commonly used in programs. To see how to use and implement these algorithms and data structures in different languages and to see what language and library support exists for them.
Topics
Arrays and VectorsListsLinear/Binary SearchQuicksortHash Tables - DictionariesC, C++, Java, Perl
Arrays
Sequence of itemsIndexable (can refer to any element immediately, by its index number)(Conceptually) contiguous chunks of memory
0 51 2 3 4 6
Arrays
Access: constant-time (θ(1))Searching: Sorted array - θ(lg(n)) Unsorted - θ(n)
Inserting/removing Unordered - θ(1)
(Add to end, or swap last guy w/deleted guy) Ordered - θ(n)
Need to make (or fill in) a hole, move n/2 elements (on average) to maintain relative order
Resizing Arrays
If the number of elements in an array is not known ahead of time it may be necessary to resize the array.Involves dynamic memory allocation and copyingTo minimize the cost it is best to resize in chunks (some factor of its current size)
Libraries - Arrays
Arrays (lists) in Perl, Bash, Python, Awk, resize automatically, as do C++ vectors, and the Array in JavaThis does not mean that it is free. Depending on the task, what goes on underneath the hood may be importantWe shall create our own machinery in C
Some C memory things…void* malloc(int n) – allocates n contiguous bytes from heap, returns address of first byte (NULL if it failed)free( void* p ) – returns memory addressed by p to the heap (does nothing to p itself)void* memmove( void* d, void* s, size_t n ) – moves n bytes from s to (possibly overlaping region at) dvoid* memcpy( void* d, void* s, size_t n ) – copies n bytes from s to (non-overlaping region at) dsizeof() – actually an operator, returns size, in bytes, of given object or typevoid* realloc( void* src, size_t n ) – attempts to resize array for you, and copy the stuff over. Returns ptr to new location (might be the same), NULL if it fails.
Growing Arrays in Cenum { INIT_SIZE=1, GROW_FACTOR=2 };
int curr_size = INIT_SIZE;
int nr_elems = 0; /* # of useful elements */
int *a = (int*)malloc( INIT_SIZE * sizeof( int ));
… /* some stuff here */
/* attempt to insert 24 */
if( nr_elems >= curr_size ) { /* need to grow */
int *t = realloc( a, curr_size*GROW_FACTOR*sizeof( int ))
if( t != NULL ) { /* success! */
curr_size *= GROW_FACTOR;
a = t;
a[nr_elems++] = 24;
}
else
/* FAILURE! */
}
Lists
A sequence of elementsNot indexable (immediately)Space is allocate for each new element and consecutive elements are linked together with a pointer.Note, though, that the middle can be modified in constant time.
data 1 data 2 data 3
NULL
data 4
head
Lists
Access (same as searching) – linear, θ(n)Modifying anywhere – constant time, θ(1)θ(1) time to insert at front, θ(n) to append unless pointer to last element kept
Lists in Python
The list-type in Python is really an array (I think. Somebody experiment? How would you?).We can make a linked-list in a very LISP way; build it out of simple pairs 1st element – the data at that node 2nd element – the rest of the list
Lists in PythonL = []
# add 24 to front
L = [ 24, L ]
print L
# add 3 to the front
L = [ 3, L ]
print L
Would output:[ 24, [] ]
[ 3, [ 24, [] ]]
Lists in Python – append to end
append the number 86:t = L # start at beginning
# get hands on last cell
while t != [] :
t = t[1] # move to next "cell", or node
t.extend( [ 86, [] ] )
Lists in Python – searching
def search( L, t ) :
"""Return cell of L that contains t,
None if not found"""
while L != [] :
if L[0] == t :
return L
L = L[1] # move to next "cell"
return None # didn't find it
Apply function to elements
def apply( l, fn ) :
while l != [] :
fn( l )
l = l[1]
fn is any function that takes a single cell, does something
Lists in Python – apply
Print list:def printCell( cell ) :
print cell[0],
apply( L, printCell )
Add 2 to each element in list:def add2( cell ) :
cell[0] += 2
apply( L, add2 )
Lists in Ctypedef struct sNode sNode;struct sNode { /* a node (cell) in a singly-link list */
int data; /* the payload */sNode* next; /* next node in list */
};
/* newitem: create new item from name and value */sNode* newNode( int d ) {
sNode *newp;newp = (sNode*) malloc( sizeof( sNode ));if( newp != NULL ) {
newp->data = d;newp->next = NULL;
}return newp;
}
Lists in C/* addfront: add newp to front of listp *
* return ptr to new list */
sNode* addfront( sNode *listp, sNode *newp )
{
newp->next=listp;
return newp;
}
list = addfront( list, newitem( 13 ));
Append Element to Back of List/* append: add newp to end of listp * * return ptr to new list */sNode* addend( sNode *listp, sNode *newp ){ sNode *p; if( listp == NULL ) return newp; for( p=listp; p->next!=NULL; p=p->next ) ; p->next = newp; return listp;}
Lookup Element in List/* lookup: linear search for t in listp *
* return ptr to node containing t, or NULL */
sNode* lookup( sNode *listp, int t )
{
for( ; listp != NULL; listp = listp->next )
if( listp->data = t )
return listp;
return NULL; /* no match */
}
Apply Function to Elements in List/* apply: execute fn for each element of listp */void apply( sNode *listp, void (*fn)(sNode ),
void* arg ){
for ( ; listp != NULL; listp = listp->next)(*fn)( listp, arg ); /* call the function */
}
void (*fn)( sNode* ) is a pointer to a void function of one argument – the first argument is a pointer to a sNode and the second is a generic pointer.
Print Elements of a List/* printnv: print name and value using format in
arg */
void printnv( sNode *p, void *arg )
{
char *fmt;
fmt = (char*) arg;
printf( fmt, p->data );
}
apply( list, printnv, “%d ” );
Count Elements of a List/* inccounter: increment counter *arg */void incCounter( sNode *p, void *arg ){ int *ip; /* p is unused – we were called,
there's a node */ ip = (int *) arg; (*ip)++;}
int n = 0;/* n is an int, &n is its address (a ptr) */
apply( list, inccounter, &n );printf( “%d elements in list\n”, n );
Free Elements in List/* freeall: free all elements of listp */void freeall( sNode *listp ){
sNode *next;for ( ; listp != NULL; listp = next) {
next = listp->next;free(listp)
}}
? for ( ; listp != NULL; listp = listp->next)? free( listp );
Delete Element in List/* delitem: delete first t from listp */sNode *delitem( sNode *listp, int t ) {
sNode *p, *prev = NULL;for( p=listp; p!=NULL; p=p->next ) {
if( p->data == t ) {if( prev == NULL ) /* front of list */
listp = p->next;else
prev->next = p->next;free(p);break;
}prev = p;
}return listp;}
Linear Search
Just exhaustively examine each elementWorks on any list (sorted or unsorted)Only search for a linked-listExamine each element in the collection, until you find what you seek, or you've examined every element Note that order of examination doesn't
matter
Need θ(n), worst and average
Linear Search - exampleL = range( 1, 11 ) # fill w/numbers 1-10random.shuffle( L ) # mix it uptarget = 7 # we're looking for 7i = 0while i < len( L ) and not bFound :if L[i] == target :
breaki += 1
if i < len( L ) : # we found it!print str(target) + \
" was found at index " + str(i)
Binary Search
Only works on sorted collectionsOnly works on indexed collections (arrays, vectors)Start in the middle: Find it?
Less than? Look in lower ½ Greater than? Look in upper ½
Cut search space in ½Need θ(lg(n)) time, worst (and avg.)
Binary Search# from previous exampleL.sort() # put into ordert = 7 # our targeti = -1l = 0 ; h = len(L)-1
while l<=h and i==-1 :m = (l+h)/2 # find middle elementif( L[m] == t ) : # found
i = melif t < L[m] : # look in left ½
h = m – 1else : # look in right ½
l = m + 1
if i != -1 :print "Found " + str(t) + " at " + str(i)
Quicksort
pick one element of the array (pivot)partition the other elements into two groups those less than the pivot those that are greater than or equal to
the pivot
Pivot is now in the right placerecursively sort each (strictly smaller) group
Quicksort – the recursive algorithmdef qsort( L, b, e ) :
# base caseif b >= e : # nothing to do - 0 or 1 element
return;# partition returns index of pivot element
piv = partition( L, b, e )# Note that the pivot is right where it
belongs# It does not get sent out in either call
qsort( L, b, piv-1 ) # elements in [ b, piv )qsort( L, piv+1, e ) # elements in ( piv, e ]
Quicksort – the Partitiondef partition( L, b, e ) :
"Return: index of pivot"
# move pivot element to L[0]swap( L, b, random.randint( b, e ))last = b
# Move <p to left side, marked by `last'for i in range( b+1, e+1 ) :
if L[i] < L[b] :last += 1swap( L, last, i )
swap( L, b, last ) # restore pivot
return last
Quicksort - Swap
def swap( l, i, j ) :
"swaps elements at indices i, j in list l"
tmp = l[i]
l[i] = l[j]
l[j] = tmp
Libraries - sort
C: qsort (in stdlib.h)C++: sort (algorithm library from STL)Java: java.util.collections.sort
Perl: sortPython: list.sort
qsort – standard C libraryqsort( void *a, int n, int s,int(*cmp)(void *a, void *b) );Sorts array a, which has n elementsEach element is s bytescmp is a function that you must provide compares 2 single elements, *a & *b
qsort must pass void pointers (addresses), since it doesn't know the type
cmp does, since you provide it for a given sort returns integer -1 if a<b, 0 if a==b, and 1 if a>b For sorting in descending order, return 1 if a < b
qsort (int)int arr[N];
qsort(arr, N, sizeof(arr[0]), icmp);
/* icmp: integer compare of *p1 and *p2 */int icmp( const void *p1, const void *p2 ) { int v1, v2; v1 = *((int*) p1); v2 = *((int*) p2); if( v1 < v2 ) return –1; else if( v1 == v2 ) return 0; else return 1;}
qsort (strings)char *str[N];
qsort(str, N, sizeof(str[0]), scmp);
/* scmp: string compare of *p1 and *p2 *//* p1 is a ptr to a string, or char*, so is a *//* ptr to a ptr, or a char** */int scmp(const void *p1, const void *p2) { char *v1, *v2;
v1 = *((char**) p1); v2 = *((char**) p2); return strcmp(v1, v2);}
Dictionary (Map)
Allows us to associate satellite data w/a key e.g., phone book, student record (given
a student ID as key), error string (given an error # as key), etc.
Operations: insert find remove
Dictionary
The key-value pairs may be stored in any aggregate type (list, struct, etc)Using an unordered list: constant-time insertion linear lookup
Ordered list linear insertion lg(n) lookup
Other dictionaries
Binary Search Tree lg(n) insertion, lookup (if balanced)
Hash table constant-time insertion, lookup (if done
well)
Trees
A binary tree is either NULL or contains a node with the key/value pair, and a left and right child which are themselves trees. In a binary search tree (BST) the keys in the left child are smaller than the keys at the node and the values in the right child are greater than the value at the node. (Recursively true through all subtrees.)O(log n) expected search and insertion timein-order traversal provides elements in sorted order.
Examples
In the following examples each node stores the key/value pair: name – a string value – a hex #, that represents the
character (MBCS)
As well as a reference (ptr) to each of the 2 subtrees
Trees in Python
We can use a list of 3 elements:0. The key/value pair1. The left subtree2. The right subtree
The following is a tree w/one node (empty subtrees)T = [ ["Smiley", 0x263A], [], [] ]
BST Lookup - Pythondef lookup( T, name ) :
"""lookup: look up name in tree T, return the cell, None if not found"""
if T == [] : # T is the empty treereturn None
if T[0][0] == name :return T
elif name < T[0][0] : # look in left subtreereturn lookup( T[1], name );
else : # look in right subtreereturn lookup( T[2], name );
}
Binary Search Tree
typedef struct bNode bNode;
struct Node {
char *name;
int value;
sNode *left;
sNode *right;
};
“smiley”0x263A“zeta”0x03b6
NULL NULL
“smiley”0x263A“smiley”0x263A
“smiley”0x263A
“Aacute”0x00c1
“smiley”0x263A“Acirc”0x00c2
NULL NULL
“smiley”0x263A“AElig”0x00c6
NULL NULL
Binary Search Tree Lookup/* lookup: look up name in tree treep */sNode *lookup( sNode *treep, char *name ){
int cmp;
if( treep == NULL )return NULL;
cmp = strcmp(name, tree->name);if( cmp == 0 )
return treep;else if( cmp < 0 )
return lookup( treep->left, name );else
return lookup( treep->right, name );}
Hash Tables
Provides key lookup and insertion with constant expected costUsed to create lookup table (with m slots), where it is not usually possible to reserve a slot for each possible elementhash function maps key to index (should evenly distribute keys) H(k) -> [ 0, m-1 ]
duplicates stored in a chain (list) – other mechanisms are possible.
Dictionaries in Python
Python (as well as Perl and Awk) has a built-in dictionary (associative array)Type is dict
d = {} # empty dictionary
# initialise
nvList = { 'Smiley': 0x263A, 'Aacute': 0x00c1 }
print nvList[ 'Smiley' ]
Dictionaries - Python
Add an entry:nvList[ 'Zeta', 0x03b6 ]
Test for existence:if nvList.has_key( 'AElig' ) :
Remove an entry:nvList.pop( 'Smiley', None )
Iterate over keys:for i in nvList.keys() :
Hash Table - C
typedef struct sNode sNode;struct sNode { char *name; int value; sNode *next; /* in chain */};
sNode *symtab[NHASH]; /* a symbol table */
NULLNULLNULLNULL
NULL
NULLname 1
data 1
name 2
data 2
NULLname 3
data 3
Hash Table Lookup/* lookup: find name in symtab, with optional create */sNode *lookup(char *name, int create, int value){ int h; sNode *sym;
h = hash(name); for (sym = symtab[h]; sym != NULL; sym = sym->next) if (strcmp(name, sym->name) == 0) return sym; if (create) { sym = (sNode *) emalloc(sizeof(sNode)); sym->name = name; /* assumed allocated elsewhere */ sym->value = value; sym->next = symtab[h]; symtab[h] = sym; } return sym;}