Upload
dongoc
View
219
Download
0
Embed Size (px)
Citation preview
Divide and Conquer (continued) Example: Integer Multiplication (Karatsuba algorithm) The problem: Let π₯ and π¦ be two π-digit numbers. Compute π₯ β π¦.
A straightforward algorithm (like what a human does) would require π(π2) time to multiply each digit of
π₯ with each digit of π¦. Letβs try to do divide-and-conquer.
Let π =π
2. Let π₯ = 2π β π₯1 + π₯2 and π¦ = 2π β π¦1 + π¦2. Then
π₯ β π¦ = 22π β π₯1π¦1 + 2π β (π₯1π¦2 + π₯2π¦1) + π₯2π¦2
So, we have π(π) = 4 β π (π
2) + π(π). By Masterβs theorem, π = 4, π = 2, and π = 1. So, π(π) = π2.
Disappointingβ¦
The trick is to reduce π. Notice that (π₯1 + π₯2)(π¦1 + π¦2) = π₯1π¦1 + (π₯1π¦2 + π₯2π¦1) + π₯2π¦2. Therefore,
after computing π₯1π¦1 and π₯2π¦2, the value π₯1π¦2 + π₯2π¦1 can be computed in the following way,
π₯1π¦2 + π₯2π¦1 = (π₯1 + π₯2)(π¦1 + π¦2) β π₯1π¦1 β π₯2π¦2
Thus, only three multiplications are needed. Therefore,
π(π) = 3 β π (π
2) + π(π)
Thus, by Masterβs theorem, π = 3, π = 2, and π = 1. So, π(π) = πlog2 3 < π1.59.
Practical concerns:
1. π is odd?
2. What base to use?
3. For small π overhead is too high. Better use π(π2) method. For π > 1000, this method is
better.
4. There is an π(π log π 2Ξ(logβ π)) method that is asymptotically better but requires very large π
to be beneficial. Here logβ π is the number of times the logarithm function must be iteratively
applied before the result is less than or equal to 1.
Example: Matrix Multiplication (Strassen Algorithm) Let π΄ and π΅ be two matrices, πΆ = π΄ Γ π΅, then ππ,π = β ππ,πππ,π
ππ=1 . For brevity we assume each matrix is
π Γ π matrix. A straightforward algorithm would take π(π3) time.
Remark: Note that in this case the input size is π2 but not π. The time complexity is a parameter related
to the input size.
We can divide each matrix into four blocks as follows:
Each block is a π
2Γ
π
2 matrix. Then
But using these straightforwardly will not make things run faster. Instead, Strassen computed seven
other matrices:
Then it can be verified that
Matrix additions take only π(π2) time. So, π(π) = 7π (π
2) + π(π2). By using Masterβs theorem, the
time complexity is π(π) = π(πlog2 7) β π(π2.807).
Nobody knows how Strassen figured out the seven intermediate matrices π1 to π7. With the
knowledge of Karatsuba algorithm for integer multiplication, a clever person would think one may be
able to apply the same trick to matrices. The remaining is a strong belief in it and the persistence of
trying out different ways. Be prepared to fail many times before you find the right answer.
Example: Polynomial Multiplication
Given two polynomials π(π₯) = π0 + π1π₯ + π2π₯2 + β― + πππ₯π andπ(π₯) = π0 + π1π₯ + π2π₯2 + β― +
πππ₯π, compute π(π₯) β π(π₯). The divide-and-conquer idea is very similar to the integer multiplication.
Work it out as an exercise.
Example: Counting inversions If you rank π movies with order 1, 2, β¦ , π, and your friend ranks the same movies with order π1, β¦ , ππ.
The number of pairs of movies that you and your friend rank with different orders is called the
inversions in the permutation π1, β¦ , ππ. This can be used as a measure of how similar (dissimilar) you and
your friend are. More formally, the number of inversions is |{(ππ, ππ): π < π and ππ > ππ}|.
Algorithm 1: For every pair of π < π, check if ππ > ππ and count. π(π2).
Algorithm 2: Divide and Conquer.
Divide the array π΄[1. . π] into two halves. Then the number of inversions is equal to the sum of the
number of inversion in the first and second halves, as well as the inversions π΄[π] > π΄[π] for π and π in the
first and second halves, respectively. One can check that a straightforward counting of the inversion
across the two halves take π2
4 comparisons. By Master theorem, this divide-and-conquer ends up with an
π(π2) algorithm. To improve, we have to reduce the time complexity to count the inversions across the
two halves.
An often used technique when dealing with arrays is to check if things become easier if the arrays are
sorted. Consider the situation when π΄[1. . π/2] and π΄[π/2 + 1. . π] are sorted, respectively. If π΄[π] >
π΄[π] for 1 β€ π β€ π/2 and π
2< π β€ π, then we know that π΄[πβ²] > π΄[π] for every πβ² β₯ π. If we count in a
proper way, all these inversions may be counted together in one operation. This saves time.
Let π΄ and π΅ be two sorted array with sizes π and π, respectively. We put π΄ before π΅ and count the
inversions between them. The idea is that we loop through every π΅[π] in the second array, and find the
first π such that π΄[π] > π΅[π]. Then we know that π΅[π] is involved in precisely π β π + 1 inversions. We
sum up all the inversions of every π΅[π].
CountPairSorted
Input: Two sorted arrays π΄ and B with length π and π, respectively
Output: the number of inversions when π΄ is put before π΅.
1. π β 1; π β 0;
2. For π from 1 to π
3. while π β€ π and π΄[π] β€ π΅[π]
4. π++
5. π β π + π β π + 1 6. Return π
The correctness follows the discussion above the pseudocode.
Time complexity: Note that the condition in line 3 can only be true for at most π times during the whole
algorithm execution. Also, for each π, the condition in line 3 can be false only once. Therefore, lines 3
and 4 take π(π + π) times in total. Line 5 takes π(1) time for each π. Therefore, the total time
complexity is π(π + π).
Now we can use CountPairSorted as a subroutine for the divide and conquer algorithm.
SortAndCount:
Input: A is a list of π elements.
Output: (S, k), where S is sorted, and k is the number of inversions.
1. If π β€ 3 sort and count with a trivial algorithm and return.
2. (π1, π1) β SortAndCount (A[1..n/2])
3. (π2, π2) β SortAndCount (A[n/2+1..n])
4. π3 β CountPairSorted(π1,π2).
5. π β Merge(π1, π2)
6. Return (π, π1 + π2 + π3).
Proof of correctness. By induction. Base case is straightforward and omitted. Inversions in π΄ includes
inversions in the first half, second half, and across the two halves. This is indeed what line 4 of
SortAndCount computes.
Time complexity: π(π) = 2π (π
2) + π(π). Therefore, π(π) = π(π log π).
Remark: It is possible to combine lines 4 and 5 together into a single MergeAndCount subroutine. This is
perhaps what you will do when you write the program. But since it does not affect the big O complexity,
separating them is easier for our proofs.