View
223
Download
0
Category
Tags:
Preview:
Citation preview
1
Data Structures
A program solves a problem.A program solves a problem. A solution consists of:A solution consists of:
a way to organize the dataa way to organize the data sequence of steps to solve the problem. sequence of steps to solve the problem.
In other words:In other words: programs = algorithms + data structures programs = algorithms + data structures
2
Data Structures
From A to ZFrom A to Z Problem specificationProblem specification Analysis and DesignAnalysis and Design
Break up the solution in well-defined Break up the solution in well-defined modulesmodules
Specify pre/post-conditionsSpecify pre/post-conditions Specify how modules interactSpecify how modules interactIDEA: separate the purpose of a module IDEA: separate the purpose of a module
from its implementationfrom its implementation
3
Data Structures
Decide how the data will be organizedDecide how the data will be organizedDecide what operations can be performed on Decide what operations can be performed on
datadataIDEA: separate the data organization and IDEA: separate the data organization and
operations from the implementationoperations from the implementation ImplementationImplementation Testing Testing MaintenanceMaintenance
Program must be easy to understand/modifyProgram must be easy to understand/modify
4
Data Structures Key ideas:Key ideas:
Modularity Modularity break problem into subproblemsbreak problem into subproblems
AbstractionAbstractionhide non-relevant information from the userhide non-relevant information from the user
Information hidingInformation hidinghide data not needed by a modulehide data not needed by a module
Software reuseSoftware reusedo not reinvent the wheeldo not reinvent the wheel
5
Data Structures
Abstract Data Type (ADT) = Abstract Data Type (ADT) = a collection of data, and a collection of data, and a set of operations on the data. (typical ops?)a set of operations on the data. (typical ops?)
The user does not know The user does not know the internal representationthe internal representation the implementations of the operationsthe implementations of the operations
The user does knowThe user does know a conceptual picture of the organization of the dataa conceptual picture of the organization of the data how to perform operations on ithow to perform operations on it
6
Data Structures
Goals:Goals: Program must be easy to understand and Program must be easy to understand and
modify.modify. Design a data structure that can be reused in Design a data structure that can be reused in
other programs.other programs. Design (or select) the Design (or select) the rightright data structure. data structure. The program must be correct and efficient The program must be correct and efficient
(memory/time).(memory/time).
7
Mathematics Review Basic formulas for derive and reviews basic proof Basic formulas for derive and reviews basic proof
techniquestechniques ExponentsExponents
1
2
222
2
)(
NNN
NNNN
ABBA
BAB
A
BABA
XXXX
XX
XX
X
XXX
8
Mathematics Review LogarithmsLogarithms All logarithms are to the base 2 unless specified otherwise.All logarithms are to the base 2 unless specified otherwise. DefinitionDefinition: :
0,0,log BXABifonlyandifBX XA
1665536log,101024log,12log,01log
0,log
logloglog
logloglog
0;log
loglog
xxx
BABA
BAAB
CA
BB
equalitiesUseful
c
cA
10
Mathematics Review Series: Geometric Series: Geometric
seriesseries
DerivationDerivation
Let Let S = 1+A+AS = 1+A+A22+…… +…… (1)(1)
where, 0<A<1where, 0<A<1
then then AS = A+AAS = A+A22+A+A33+…+…(2)(2)
Subtracting (1) and (2), Subtracting (1) and (2), we get S-AS = 1, i.e.we get S-AS = 1, i.e.A
A
thenAIf
A
AA
N
i
i
NN
i
i
1
1
,10
1
1
0
1
0
AS
1
1
11
Mathematics Review Series: Arithmetic Series: Arithmetic
seriesseriesExample: To find the Example: To find the
sum sum
2+5+8+….+ (3k-1)2+5+8+….+ (3k-1)
= 3(1+2+3+…+k) = 3(1+2+3+…+k)
- (1+1+1+….+1)- (1+1+1+….+1)
2
)1(
0
NNi
N
i
kkk
2
)1(3
12
Mathematics Review
The P wordThe P word
- - to proof a false to proof a false statementstatement::
proof by counter proof by counter exampleexample
- - to proof a correct to proof a correct statementstatement
- proof by induction- proof by induction(1) proving a base case(1) proving a base case(2) inductive hypothesis(2) inductive hypothesis
- proof by contradiction- proof by contradiction(1) assume it is false(1) assume it is false(2) show that this assumption (2) show that this assumption
is falseis false
13
Algorithms
A problem is a specification of an input-A problem is a specification of an input-output relationship.output relationship.
Algorithm = Algorithm = a a well-definedwell-defined computational computational
procedure that takes a set of values as procedure that takes a set of values as input and produces a set of values as input and produces a set of values as output. output.
a tool for solving a well-specified a tool for solving a well-specified computational problem.computational problem.
14
Algorithm characteristics
CorrectnessCorrectness algorithm halts with correct output for algorithm halts with correct output for
every legal instance of the problem.every legal instance of the problem. proof of correctnessproof of correctness
GeneralityGenerality must work for all legal data (eg. a must work for all legal data (eg. a
sorting algorithm should work on both sorting algorithm should work on both numbers and names)numbers and names)
15
Algorithm characteristics
FinitenessFiniteness won’t take forever to runwon’t take forever to run
EfficiencyEfficiency making efficient use of resources (time making efficient use of resources (time
and space)and space)
16
Algorithm characteristics
PseudocodePseudocode = an English-like language with limited = an English-like language with limited vocabulary that we use to describe algorithms in an easy vocabulary that we use to describe algorithms in an easy to understand manner.to understand manner.
Example: Linear SearchExample: Linear Search
Input : array A, key xOutput: index of array element equal to x, -1 if x is not in Afor i = 1 to length[A] if A[i]==x then return i endifendforreturn -1
17
Algorithm characteristics
Is the Linear Search algorithm...Is the Linear Search algorithm... correct?correct? efficient?efficient? general?general? finite?finite?
18
Algorithm analysis
Analyzing an algorithm = estimating the Analyzing an algorithm = estimating the resources it requiresresources it requires
TimeTime Number of steps/operations executedNumber of steps/operations executed Each operation has a costEach operation has a cost A function of problem (input) sizeA function of problem (input) size
SpaceSpace Amount of temporary storage required.Amount of temporary storage required. We usually don’t count the input.We usually don’t count the input.
19
Algorithm analysis
Best case analysisBest case analysis Given the algorithm and input of size n that Given the algorithm and input of size n that
makes it run fastest (compared to all other makes it run fastest (compared to all other possible inputs of size n), what is the running possible inputs of size n), what is the running time?time?
Worst case analysisWorst case analysis Given the algorithm and input of size n that Given the algorithm and input of size n that
makes it run slowest (compared to all other makes it run slowest (compared to all other possible inputs of size n), what is the running possible inputs of size n), what is the running time?time?
20
Algorithm analysis
Average case analysisAverage case analysis Given the algorithm and a typical, Given the algorithm and a typical,
average input of size n, what is the average input of size n, what is the running time?running time?
21
Insertion Sort One of the simplest methods to sort an array is an insertion sort. An One of the simplest methods to sort an array is an insertion sort. An
example of an insertion sort occurs in everyday life while playing cards. example of an insertion sort occurs in everyday life while playing cards. To sort the cards in your hand you extract a card, shift the remaining To sort the cards in your hand you extract a card, shift the remaining cards, and then insert the extracted card in the correct place. This cards, and then insert the extracted card in the correct place. This process is repeated until all the cards are in the correct sequence. Both process is repeated until all the cards are in the correct sequence. Both average and worst-case time is average and worst-case time is OO((nn22). ).
Assuming there are Assuming there are nn elements in the array, we must index through elements in the array, we must index through nn - 1 - 1 entries. For each entry, we may need to examine and shift up to entries. For each entry, we may need to examine and shift up to nn - 1 - 1 other entries, resulting in a other entries, resulting in a OO((nn22) algorithm. The insertion sort is an ) algorithm. The insertion sort is an in-in-placeplace sort. That is, we sort the array in-place. No extra memory is sort. That is, we sort the array in-place. No extra memory is required. The insertion sort is also a required. The insertion sort is also a stablestable sort. Stable sorts retain the sort. Stable sorts retain the original ordering of keys when identical keys are present in the input original ordering of keys when identical keys are present in the input data. data.
22
Starting near the top of the array in Figure (a), we Starting near the top of the array in Figure (a), we extract the 3. Then the above elements are shifted extract the 3. Then the above elements are shifted down until we find the correct place to insert the 3. down until we find the correct place to insert the 3. This process repeats in Figure (b) with the next This process repeats in Figure (b) with the next number. Finally, in Figure (c), we complete the sort number. Finally, in Figure (c), we complete the sort by inserting 2 in the correct place.by inserting 2 in the correct place.
23
Algorithm Analysis
InsertionSort(A)
for i=2 to length(A)
item = A[i]
j = i - 1
while (j > 0 and A[j] > item)
A[j+1] = A[j]
j = j - 1
A[j+1] = item
cost times executedc1 n
c2 n-1
c3 n-1
c4 ti
c5 (ti-1)
c6 (ti-1)
c7 n-1
24
Algorithm Analysis
Observations:Observations: the constants are “unimportant” detailsthe constants are “unimportant” details some terms “grow” faster than others as some terms “grow” faster than others as
the size of the input increases.the size of the input increases. Let’s simplify this...Let’s simplify this...
25
Algorithm Analysis
Consider:Consider: nn22 + 100n + 1000 + 100n + 1000 How does this change as n becomes larger? How does this change as n becomes larger? Is this a “good” running time?Is this a “good” running time?
Compare:Compare: nn22/3 to 3n/3 to 3n Do the constants really make a difference?Do the constants really make a difference?
26
Algorithm Analysis
Idea:Idea: comparing algorithms:comparing algorithms:
upper boundsupper boundslower boundslower boundstight bounds tight bounds
we are interested in the growth rate of an we are interested in the growth rate of an algorithm: how its running time changes algorithm: how its running time changes proportional to the size of the input.proportional to the size of the input.
27
Algorithm Analysis: Big Oh
A function A function f(n)f(n) is O( is O(g(n)g(n)) if there exist constants ) if there exist constants cc, , nn00>0>0 such that such that f(n)f(n) c c··g(n)g(n) for all for all n n n n00
What does this mean in English? What does this mean in English? cc··g(n)g(n) is an upper bound of is an upper bound of f(n)f(n) for large for large nn
Examples:Examples: f(n) = 3nf(n) = 3n22+2n+1 is O(n+2n+1 is O(n22)) f(n) = 2n is O(nf(n) = 2n is O(n22) (it is also O(n) )) (it is also O(n) ) f(n) = 1000nf(n) = 1000n33 is O(n is O(n1010))
28
Algorithm Analysis: Omega
A function A function f(n)f(n) is is ((g(n)g(n)) if there exist ) if there exist constants constants c,c, nn00>0>0 such that such that
f(n) f(n) c c··g(n)g(n) for allfor all nnnn00
What does this mean in English? What does this mean in English? cc··g(n)g(n) is a lower bound of is a lower bound of f(n)f(n) for large for large nn
Example:Example: f(n) = nf(n) = n22 is is (n)(n)
29
Algorithm Analysis: Theta
A function A function f(n)f(n) is is ((g(n)g(n)) if it is both O) if it is both O((g(n)) and g(n)) and (g(n)) (g(n)) , in other words:, in other words: There exist constants There exist constants c1,c2,n>0c1,c2,n>0 s.t. s.t.
0 0 c c1 1 g(n) g(n) f(n) f(n) c c22g(n) g(n) for allfor all nnnn00
What does this mean in English? What does this mean in English? f(n) f(n) andand g(n) g(n) have the same order of growth have the same order of growth indicates a tight bound.indicates a tight bound.
Example:Example: f(n) = 2n+1 is f(n) = 2n+1 is (n)(n)
30
Algorithm Analysis
O, O, , , are transitive and reflexive are transitive and reflexive is symmetricis symmetric f(n)f(n) = O( = O(g(n)g(n)) iff ) iff g(n)g(n) = = ((f(n)f(n))) analogy:analogy:
f(n)f(n) = O( = O(g(n)g(n)) ) a a b b f(n)f(n) = = ((g(n)g(n)) ) a a b b f(n)f(n) = = ((g(n)g(n)) ) a = b a = b Note: this doesn’t mean that all functions are Note: this doesn’t mean that all functions are
asymptotically comparableasymptotically comparable
31
Growth of functions
• A way to describe behavior of functions in the limit -- asymptotic efficiency.
• Growth of functions.
• Focus on what’s important by abstracting away low-order terms and constant factors.
• How to indicate running times of algorithms?
• A way to compare “sizes” of functions:
O ≤ ≥
=
o < ω >
33
Standard notations and common functions
Monotonicity:Monotonicity: f (n) f (n) is is monotonically increasingmonotonically increasing if if m m ≤ ≤ n n ⇒ ⇒ f (m) f (m) ≤ ≤ f (n)f (n).. f f ((nn) ) is is monotonically decreasingmonotonically decreasing if if m m n n ⇒ ⇒ f f ((mm) )
≥ ≥ f f ((nn)).. f f ((nn) ) is is strictly increasingstrictly increasing if if m m < < n n ⇒ ⇒ f f ((mm) < ) < f f ((nn)).. f f ((nn) ) is is strictly decreasingstrictly decreasing if if m m n n ⇒ ⇒ f f ((mm) > ) > f f ((nn))..
Floor and Ceilings: Floor and Ceilings: x – 1x – 1 < < xx xx xx < < x+1x+1 Modular arithmetic: Modular arithmetic: a mod n = a - a mod n = a - a/na/n nn
-- refer to the textbook (p36-38).-- refer to the textbook (p36-38). Polynomials:Polynomials:
Exponentials:Exponentials: Logarithms:Logarithms: Factorials:Factorials:
34
Algorithm Analysis
What if a constant is too large?What if a constant is too large? Consider Consider 10101010nn as compared to as compared to nn22
35
Algorithm analysis
Input: Collection of size nOutput: The smallest difference between any two different numbers in the collection Algorithm: List all pairs of different numbers in the collectionestimate=difference between values in first pair.for each remaining pair compute the difference between the two values if it is less than estimate set estimate to that differencereturn estimate
Recommended