Upload
others
View
0
Download
0
Embed Size (px)
Citation preview
Last Updated: 24 March 2015 EECS 2011 Prof. J. Elder - 1 -
Loop Invariants and Binary Search
Last Updated: 24 March 2015 EECS 2011 Prof. J. Elder - 2 -
Learning Outcomes
Ø From this lecture, you should be able to: q Use the loop invariant method to think about iterative algorithms.
q Prove that the loop invariant is established.
q Prove that the loop invariant is maintained in the ‘typical’ case.
q Prove that the loop invariant is maintained at all boundary conditions.
q Prove that progress is made in the ‘typical’ case
q Prove that progress is guaranteed even near termination, so that the exit condition is always reached.
q Prove that the loop invariant, when combined with the exit condition, produces the post-condition.
q Trade off efficiency for clear, correct code.
Last Updated: 24 March 2015 EECS 2011 Prof. J. Elder - 3 -
Outline
Ø Iterative Algorithms, Assertions and Proofs of Correctness
Ø Binary Search: A Case Study
Last Updated: 24 March 2015 EECS 2011 Prof. J. Elder - 4 -
Outline
Ø Iterative Algorithms, Assertions and Proofs of Correctness
Ø Binary Search: A Case Study
Last Updated: 24 March 2015 EECS 2011 Prof. J. Elder - 5 -
Assertions
Ø An assertion is a statement about the state of the data at a specified point in your algorithm.
Ø An assertion is not a task for the algorithm to perform.
Ø You may think of it as a comment that is added for the benefit of the reader.
Last Updated: 24 March 2015 EECS 2011 Prof. J. Elder - 6 -
Loop Invariants
Ø Binary search can be implemented as an iterative algorithm (it could also be done recursively).
Ø Loop Invariant: An assertion about the current state useful for designing, analyzing and proving the correctness of iterative algorithms.
Last Updated: 24 March 2015 EECS 2011 Prof. J. Elder - 7 -
Other Examples of Assertions
Ø Preconditions: Any assumptions that must be true about the input instance.
Ø Postconditions: The statement of what must be true when the algorithm/program returns.
Ø Exit condition: The statement of what must be true to exit a loop.
Last Updated: 24 March 2015 EECS 2011 Prof. J. Elder - 8 -
Iterative Algorithms
Take one step at a time
towards the final destination
loop
take step
end loop
Last Updated: 24 March 2015 EECS 2011 Prof. J. Elder - 9 -
From the Pre-Conditions on the input instance we must establish the loop invariant.
Establishing Loop Invariant
Last Updated: 24 March 2015 EECS 2011 Prof. J. Elder - 10 -
Maintain Loop Invariant
Ø Suppose that q We start in a safe location (pre-condition)
q If we are in a safe location, we always step to another safe location (loop invariant)
Ø Can we be assured that the computation will always be in a safe location?
Ø By what principle?
Last Updated: 24 March 2015 EECS 2011 Prof. J. Elder - 11 -
Maintain Loop Invariant • By Induction the computation will always be in a safe location.
(0)
, ( )
, ( ) ( 1)
S
i S i
i S i S i
⇒ ⎫⎪⎪⇒∀ ⇒⎬⎪⎪⇒∀ ⇒ + ⎭
Last Updated: 24 March 2015 EECS 2011 Prof. J. Elder - 12 -
Ending The Algorithm Ø Define Exit Condition
Ø Termination: With sufficient progress,
the exit condition will be met.
Ø When we exit, we know q exit condition is true
q loop invariant is true
from these we must establish
the post conditions.
Exit
Exit
0 km Exit
Last Updated: 24 March 2015 EECS 2011 Prof. J. Elder - 13 -
Definition of Correctness <PreCond> & <code> è<PostCond>
If the input meets the preconditions,
then the output must meet the postconditions.
If the input does not meet the preconditions, then nothing is required.
Last Updated: 24 March 2015 EECS 2011 Prof. J. Elder - 14 -
MAR 12, 2015 End of Lecture
Last Updated: 24 March 2015 EECS 2011 Prof. J. Elder - 15 -
Outline
Ø Iterative Algorithms, Assertions and Proofs of Correctness
Ø Binary Search: A Case Study
Last Updated: 24 March 2015 EECS 2011 Prof. J. Elder - 16 -
Define Problem: Binary Search
Ø PreConditions q Key 25
q Sorted List
Ø PostConditions q Find key in list (if there).
3 5 6 13 18 21 21 25 36 43 49 51 53 60 72 74 83 88 91 95
3 5 6 13 18 21 21 25 36 43 49 51 53 60 72 74 83 88 91 95
Last Updated: 24 March 2015 EECS 2011 Prof. J. Elder - 17 -
Define Loop Invariant
Ø Maintain a sublist.
Ø If the key is contained in the original list, then the key is contained in the sublist.
key 25
3 5 6 13 18 21 21 25 36 43 49 51 53 60 72 74 83 88 91 95
Last Updated: 24 March 2015 EECS 2011 Prof. J. Elder - 18 -
Define Step
Ø Cut sublist in half.
Ø Determine which half the key would be in.
Ø Keep that half.
key 25
3 5 6 13 18 21 21 25 36 43 49 51 53 60 72 74 83 88 91 95
If key ≤ mid, then key is in left half.
If key > mid, then key is in right half.
mid
Last Updated: 24 March 2015 EECS 2011 Prof. J. Elder - 19 -
Define Step
Ø It is faster not to check if the middle element is the key.
Ø Simply continue.
key 43
3 5 6 13 18 21 21 25 36 43 49 51 53 60 72 74 83 88 91 95
If key ≤ mid, then key is in left half.
If key > mid, then key is in right half.
Last Updated: 24 March 2015 EECS 2011 Prof. J. Elder - 20 -
Make Progress
Ø The size of the list becomes smaller.
3 5 6 13 18 21 21 25 36 43 49 51 53 60 72 74 83 88 91 95
3 5 6 13 18 21 21 25 36 43 49 51 53 60 72 74 83 88 91 95
79 km
75 km
Last Updated: 24 March 2015 EECS 2011 Prof. J. Elder - 21 -
Exit Condition
Ø If the key is contained in the original list,
then the key is contained in the sublist.
Ø Sublist contains one element.
Exit
3 5 6 13 18 21 21 25 36 43 49 51 53 60 72 74 83 88 91 95
0 km
• If element = key, return associated entry.
• Otherwise return false.
key 25
Last Updated: 24 March 2015 EECS 2011 Prof. J. Elder - 22 -
Running Time
The sublist is of size n, n/2, n/4, n/8,…,1 Each step O(1) time.
Total = O(log n)
key 25
3 5 6 13 18 21 21 25 36 43 49 51 53 60 72 74 83 88 91 95
If key ≤ mid, then key is in left half.
If key > mid, then key is in right half.
Last Updated: 24 March 2015 EECS 2011 Prof. J. Elder - 23 -
Running Time
Ø Binary search can interact poorly with the memory hierarchy (i.e. caching), because of its random-access nature.
Ø It is common to abandon binary searching for linear searching as soon as the size of the remaining span falls below a small value such as 8 or 16 or even more in recent computers.
Last Updated: 24 March 2015 EECS 2011 Prof. J. Elder - 24 -
<precondition>: A[1..n] is sorted in non-decreasing order<postcondition>: If is in A[1..n], algorithm returns
1, its location
loop-invariant>: If is
BinarySea
in
rch(A[1..n],
whil
)
e p q
key
keyq p
e
n
k y
<>
= =
if [ ]
els
2
1
return( )
return("Key n
A[1..n], then
e
endendif [ ]
end
is in A[p..
ot in list")
q]p qmid
q mid
p mi
key A m
key
id
key A p
e
d
plse
≤
+⎢ ⎥= ⎢ ⎥⎣ ⎦
=
= +
=
Last Updated: 24 March 2015 EECS 2011 Prof. J. Elder - 25 -
Simple, right?
Ø Although the concept is simple, binary search is notoriously easy to get wrong.
Ø Why is this?
Last Updated: 24 March 2015 EECS 2011 Prof. J. Elder - 26 -
Boundary Conditions
Ø The basic idea behind binary search is easy to grasp.
Ø It is then easy to write pseudocode that works for a ‘typical’ case.
Ø Unfortunately, it is equally easy to write pseudocode that fails on the boundary conditions.
Last Updated: 24 March 2015 EECS 2011 Prof. J. Elder - 27 -
1
if [ ]
else
end
q mid
p
key A mid
mid
=
=
≤
+
Boundary Conditions
1
if [ ]
else
end
q mid
p
key A mid
mid
=
=
<
+or
What condition will break the loop invariant?
Last Updated: 24 March 2015 EECS 2011 Prof. J. Elder - 28 -
Boundary Conditions
key 36
3 5 6 13 18 21 21 25 36 43 49 51 53 60 72 74 83 88 91 95
mid
sC eod lek cey t A[m rige hid] t lf: ha≥ →
Bug!!
Last Updated: 24 March 2015 EECS 2011 Prof. J. Elder - 29 -
1
if [ ]
else
end
q mid
p
key A mid
mid
=
=
≤
+
Boundary Conditions
1
if [ ]
else
end
q mid
p
key A mid
mid
=
=
<
+
if < [ ]
else
end
1q mid
p
key A mid
mid
= −
=
OK OK Not OK!!
Last Updated: 24 March 2015 EECS 2011 Prof. J. Elder - 30 -
key 25
3 5 6 13 18 21 21 25 36 43 49 51 53 60 72 74 83 88 91 95
Boundary Conditions
mid 2+⎢ ⎥= ⎢ ⎥⎣ ⎦p q mid
2+⎡ ⎤= ⎢ ⎥⎢ ⎥p q
or
Shouldn’t matter, right? Select mid 2
p q+⎡ ⎤= ⎢ ⎥⎢ ⎥
Last Updated: 24 March 2015 EECS 2011 Prof. J. Elder - 31 -
6 74
Boundary Conditions
key 25
95 91 88 83 72 60 53 51 49 43 36 25 21 21 18 13 5 3
If key ≤ mid, then key is in left half.
If key > mid, then key is in right half.
mid Select mid
2p q+⎡ ⎤= ⎢ ⎥⎢ ⎥
1
if [ ]
else
end
q mid
p
key A mid
mid
=
=
≤
+
Last Updated: 24 March 2015 EECS 2011 Prof. J. Elder - 32 -
25 18 74
Boundary Conditions
key 25
95 91 88 83 72 60 53 51 49 43 36 21 21 13 6 5 3
If key ≤ mid, then key is in left half.
If key > mid, then key is in right half.
mid Select mid
2p q+⎡ ⎤= ⎢ ⎥⎢ ⎥
1
if [ ]
else
end
q mid
p
key A mid
mid
=
=
≤
+
Last Updated: 24 March 2015 EECS 2011 Prof. J. Elder - 33 -
25 13 74
Boundary Conditions
key 25
95 91 88 83 72 60 53 51 49 43 36 21 21 18 6 5 3
If key ≤ mid, then key is in left half.
If key > mid, then key is in right half.
• Another bug!
No progress toward goal:
Loops Forever!
mid Select mid
2p q+⎡ ⎤= ⎢ ⎥⎢ ⎥
1
if [ ]
else
end
q mid
p
key A mid
mid
=
=
≤
+
Last Updated: 24 March 2015 EECS 2011 Prof. J. Elder - 34 -
if [
mid
]2
1else
end
key A mi
p q
q mid
p mid
d
+⎢ ⎥= ⎢ ⎥⎣ ⎦
= +
≤=
Boundary Conditions
if [
mid
]2
1else
end
key A mi
p q
q mid
p mid
d
+⎡ ⎤= ⎢ ⎥⎢ ⎥
= +
≤=
if < [
mid2
1]
else
end
key A mid
p q
q mid
p mid
+⎡ ⎤= ⎢ ⎥⎢ ⎥
= −
=
OK OK Not OK!!
Last Updated: 24 March 2015 EECS 2011 Prof. J. Elder - 35 -
if [
mid
]2
1else
end
key A mi
p q
q mid
p mid
d
+⎢ ⎥= ⎢ ⎥⎣ ⎦
= +
≤=
Getting it Right
Ø How many possible algorithms?
Ø How many correct algorithms?
Ø Probability of guessing correctly?
midr 2
o ? p q+⎡ ⎤= ⎢ ⎥⎢ ⎥
if < [ or ?]key A mid
elseo
end
1r q mid
p mid
= −
=
Last Updated: 24 March 2015 EECS 2011 Prof. J. Elder - 36 -
BinarySearch(A[1..n],key)<precondition>: A[1..n] is sorted in non-decreasing order<postcondition>: If key is in A[1..n], algorithm returns its locationp = 1, q = nwhile q ≥ p
< loop-invariant>: If key is in A[1..n], then key is in A[p..q]
mid =p + q
2
⎢
⎣⎢⎢
⎥
⎦⎥⎥
if key <A[mid]q = mid − 1
else if key > A[mid]p = mid + 1
elsereturn(mid)
endendreturn("Key not in list")
Alternative Algorithm: Less Efficient but More Clear
ΘStill (log ), but with slightly larger constant.n
Last Updated: 24 March 2015 EECS 2011 Prof. J. Elder - 37 -
Assignment 3 Q2: kth Smallest of Union
Ø e = kthSmallestOfUnion(k) q e.g., kthSmallestOfUnion(6) = 7
Ø Observation: e must be in first k positions of A1 or A2, i.e.,
Ø à Step 1: Truncate A1 and A2 to length k.
1 2 5 7 10 16 18
3 4 8 9 11 12 14 15 17
A1
A2
e ∈A1[0…k −1]∪ A2[0…k −1]
Last Updated: 24 March 2015 EECS 2011 Prof. J. Elder - 38 -
Assignment 3 Q2: kth Smallest of Union
Ø e = kthSmallestOfUnion(k) q e.g., kthSmallestOfUnion(6) = 7
Ø Step 2: Divide and Conquer! q Case 1: A1[2] > A2[2]. In what intervals must the kth smallest lie?
q Case 2: A1[2] < A2[2]. In what intervals must the kth smallest lie?
1 2 5 7 10 16 18
3 4 8 9 11 12 14 15 17
A1
A2
Compare (A1[2], A2[2])
Last Updated: 24 March 2015 EECS 2011 Prof. J. Elder - 39 -
Assignment 3 Q2: kth Smallest of Union
Ø More generally: maintain the loop invariant that the kth smallest key is stored in
Compare (A1[2], A2[2])
A1 k1l …k1u⎡⎣ ⎤⎦∪ A2 k2l …k2u⎡⎣ ⎤⎦
A1
A2
k1l k1u
k2l k2u
Last Updated: 24 March 2015 EECS 2011 Prof. J. Elder - 40 -
Assignment 3 Q2: kth Smallest of Union
Ø Now bisect A1: and define .
Ø Note that
Ø Now compare
Ø What sub-intervals can you safely rule out?
Ø Now update accordingly, and iterate!
Compare (A1[2], A2[2])
A1
A2
k1l k1u
k2l k2u k2
k1 = k1l + k1u( ) / 2⎢⎣ ⎥⎦ k2 = k − k1 −1
k1
k1 + k2 = k −1
A1 k1⎡⎣ ⎤⎦ and A2 k2⎡⎣ ⎤⎦.
k1l ,k1u ,k2l ,k2u
Last Updated: 24 March 2015 EECS 2011 Prof. J. Elder - 41 -
Assignment 3 Q2: kth Smallest of Union
Ø To simplify the problem, assume that original input arrays are of the same length.
Ø Note that k < 2n, or a RankOutOfRangeException is thrown.
Compare (A1[2], A2[2])
A1
A2
0 n−1
0 n−1
Last Updated: 24 March 2015 EECS 2011 Prof. J. Elder - 42 -
Assignment 3 Q2: kth Smallest of Union
Ø What if k < n?
Ø Then we first truncate both arrays to be of length k.
A1
A2
0 n−1
0 n−1
A1
A2
0 k −1
0 k −1
Last Updated: 24 March 2015 EECS 2011 Prof. J. Elder - 43 -
Assignment 3 Q2: kth Smallest of Union
Ø What if k > n?
Ø Then we first trim the tails of the arrays so they are of length k.
A1
A2
0 n−1
0 n−1
A1
A2
0 k −1
0 k −1
Last Updated: 24 March 2015 EECS 2011 Prof. J. Elder - 44 -
Assignment 3 Q2: kth Smallest of Union
Ø What if k > n + 1?
Ø Then we first trim the beginning of both arrays so they are of length n – (k – n -1) + 1 = 2n – k + 1.
A1
A2
0 n−1
0 n−1
A1
A2
k − n−1 n−1
n−1 k − n−1
Last Updated: 24 March 2015 EECS 2011 Prof. J. Elder - 45 -
Assignment 3 Q2: kth Smallest of Union
Ø Thus at the beginning of the loop, we have the kth smallest element in
A1 k1l …k1u⎡⎣ ⎤⎦∪ A2 k2l …k2u⎡⎣ ⎤⎦
A1
A2
k1l k1u
k2l k2u
Last Updated: 24 March 2015 EECS 2011 Prof. J. Elder - 46 -
Assignment 3 Q2: kth Smallest of Union
Ø Now let and
Ø Then we have that
Ø In the loop we will compare and , and update, while preserving 3 loop invariants:
Ø //LI1: kth smallest is in A1[k1l...k1u] or A2[k2l...k2u]
Ø //LI2: k1 + k2 = k-1
Ø //LI3: |n1-n2| < 2
A1
A2
k1l k1u
k2l k2u
k1 = k1l + k1u( ) / 2⎡⎢ ⎤⎥
k2 = k2l + k2u( ) / 2⎢⎣ ⎥⎦k1 + k2 = k −1.
k1
k2
A1 k1[ ] A2 k2[ ]
Last Updated: 24 March 2015 EECS 2011 Prof. J. Elder - 47 -
Ø A volunteer, please.
Card Trick
Last Updated: 24 March 2015 EECS 2011 Prof. J. Elder - 48 -
Pick a Card
Done
Thanks to J. Edmonds for this example.
Last Updated: 24 March 2015 EECS 2011 Prof. J. Elder - 49 -
Loop Invariant: The selected card is one
of these.
Last Updated: 24 March 2015 EECS 2011 Prof. J. Elder - 50 -
Which
column?
left
Last Updated: 24 March 2015 EECS 2011 Prof. J. Elder - 51 -
Loop Invariant: The selected card is one
of these.
Last Updated: 24 March 2015 EECS 2011 Prof. J. Elder - 52 -
Selected column is placed in the middle
Last Updated: 24 March 2015 EECS 2011 Prof. J. Elder - 53 -
I will rearrange the cards
Last Updated: 24 March 2015 EECS 2011 Prof. J. Elder - 54 -
Relax Loop Invariant: I will remember the same
about each column.
Last Updated: 24 March 2015 EECS 2011 Prof. J. Elder - 55 -
Which
column?
right
Last Updated: 24 March 2015 EECS 2011 Prof. J. Elder - 56 -
Loop Invariant: The selected card is one
of these.
Last Updated: 24 March 2015 EECS 2011 Prof. J. Elder - 57 -
Selected column is placed in the middle
Last Updated: 24 March 2015 EECS 2011 Prof. J. Elder - 58 -
I will rearrange the cards
Last Updated: 24 March 2015 EECS 2011 Prof. J. Elder - 59 -
Which
column?
left
Last Updated: 24 March 2015 EECS 2011 Prof. J. Elder - 60 -
Loop Invariant: The selected card is one
of these.
Last Updated: 24 March 2015 EECS 2011 Prof. J. Elder - 61 -
Selected column is placed in the middle
Last Updated: 24 March 2015 EECS 2011 Prof. J. Elder - 62 -
Here is your card.
Wow!
Last Updated: 24 March 2015 EECS 2011 Prof. J. Elder - 63 -
Ternary Search
Ø Loop Invariant: selected card in central subset of cards
Ø How many iterations are required to guarantee success?
1Size of subset = / 3
wheretotal number of cards
iteration index
in
ni
−⎡ ⎤⎢ ⎥
==
Last Updated: 24 March 2015 EECS 2011 Prof. J. Elder - 64 -
Outline
Ø Iterative Algorithms, Assertions and Proofs of Correctness
Ø Binary Search: A Case Study
Last Updated: 24 March 2015 EECS 2011 Prof. J. Elder - 65 -
Learning Outcomes
Ø From this lecture, you should be able to: q Use the loop invariant method to think about iterative algorithms.
q Prove that the loop invariant is established.
q Prove that the loop invariant is maintained in the ‘typical’ case.
q Prove that the loop invariant is maintained at all boundary conditions.
q Prove that progress is made in the ‘typical’ case
q Prove that progress is guaranteed even near termination, so that the exit condition is always reached.
q Prove that the loop invariant, when combined with the exit condition, produces the post-condition.
q Trade off efficiency for clear, correct code.