Introduction to Algorithms: Brute-Force Algorithmscs.boisestate.edu/~cs521/slides/NewSlides/BruteForce.pdfCS 421 - Introduction to Algorithms 28 When increment up to n, change is at

Introduction to Algorithms:Brute-Force Algorithms

Introduction to Algorithms

Brute Force• Powering a Number• Selection Sort• Exhaustive Search

• 0/1 Knapsack Problem• Assignment Problem

CS 421 - Analysis of Algorithms2

Brute-Force Algorithm Design


• Straightforward, usually based on problem definition.

• Rarely the most efficient but can be applied to wide range of problems.

• For some elementary problems, almost as good as most efficient. May not be worth cost.

• May work just as well on smaller data sets.• Used to measure other algorithms against.

Calculating Powers of a Number

CS 421 - Analysis of Algorithms

Problem: Compute a n, where n ∈N.

Naive algorithm: Θ(n).

a n = n * n * n * … * n

4

a times

Selection Sort


Given a list of n orderable items, rearrange them in non-decreasing order.1. Scan entire list to find smallest item.2. Exchange it with first item. First element now in its

final, sorted position.3. Scan remaining n – 1 items, starting with second

element, and find smallest item.4. Exchange it with second item. Second element now in

its final, sorted position. 5. Repeat for a total of n – 1 times.

5

Selection Sort -Example


Selection sort on the list: 89, 45, 68, 90, 29, 34, 17. Each line corresponds to an iteration of the algorithm. The values in bold are the smallest item for that iterations. Elements to the left of the vertical bar are in their final positions.

6

| 89 45 68 90 29 34 1717 |45 68 90 29 34 8917 29 | 68 90 45 34 8917 29 34 | 90 45 68 8917 29 34 45 | 90 68 8917 29 34 45 68 | 90 8917 29 34 45 68 89 | 90

Selection Sort -Analysis


Selection sort is implemented using nested forloops:

Outer loop: iterates from 0 to n – 2 – don’t have to visit element because already in final sorted position

Inner loop: finds smallest value remaining the list. Even though gets smaller each iteration, still on order of n.Therefore:

T(n) = Θ(n2)7

Sequential search

In a list No better than linear timeWe can do better if not a listWorst case O(n)

CS 421 - Analysis of Algorithms 8

Exhaustive Search


Brute-force approach to combinatorial problems (i.e. permutations, combinations, subsets of a given set).1. Generate all elements of problem domain (all

possible solutions).2. Select those that satisfy the constraints of the

problem.3. Chose one or more that are most desirable (i.e.

optimize the objective function).

0/1 Knapsack ProblemAnalysis


The most costly operation is generating all of the subsets of n items. Since there are 2n subsets,

Brute-force Approach to Knapsack problem: Ω(2n).

Not feasible for any but the smallest values of n.

0-1 knapsack Algorithmpublic boolean knapsack (double [] v, double V, double [] w, double W,

double counterV, double counterW, int i){if(i<v.length){

boolean b1=knapsack(v,V,w,W, counterV,counterW, i+1);boolean b2=knapsack(v,V,w,W,counterV+v[i],counterW+w[i], i+1);return b1||b2;

}else{return (counterV>= V) && (counterW<= W)

}}

T(n)=2T(n-1) +1T(n-1)<=C*2^{n-1}T(n)<=C*2*2^(n-1)+1T(n)<=C*2*2^(n-1)+1<=C*2^n

0-1 knapsack Algorithmpublic boolean knapsack (double [] v, double V, double [] w, double W,

double counterV, double counterW, int i){if(i<v.length){

boolean b1=knapsack(v,V,w,W, counterV,counterW, i+1);boolean b2=knapsack(v,V,w,W,counterV+v[i],counterW+w[i], i+1);return b1||b2;

}else{return (counterV>= V) && (counterW<= W)

}}

T(n)=2T(n-1) +1T(n-1)<=C*2^{n-1}T(n)<=C*2*2^(n-1)+1T(n)<=C*2*2^(n-1)+1<=C*2^n

T(n)=2T(n-1) +1T(n-1)<=C*(n-1)2^{n-1}T(n)<=C*2*2^(n-1)*(n-1)+1T(n)<=C*2*2^(n-1)*(n-1)+1<=C*n*2^n

0/1 Knapsack Problem


Given n items of known weights w1, w2, …, wn and values v1, v2, …, vn , and a knapsack of capacity W, find most valuable subset of items that will fit.

1. Generate all subsets of n items.2. Calculate the weights of each and eliminate all

the infeasible solutions.3. Find the subset with the maximum value.

Assignment Problem


There are n jobs that need to be completed, and n people that need to be assigned to a job, one person per job. The cost for the i th person to perform job j is known C[i, j]. Find the assignment with the lowest cost.

1. Generate all permutations of n people assigned to n jobs.

2. Calculate the cost of each permutation/solution.3. Find the solution with the minimum value.

Assignment ProblemAnalysis


The most costly operation is generating all of the permutations. Since there are n! permutations,

Brute-force Approach to Assignment problem: Θ(n!).

Not feasible for any but the smallest values of n.

String Matching


String Matching


String Matching


Complexity

m(n-m+1) O(nm)


Travelling salesman problem




Path={(A,B),(B,C),(C,D),(D,A)}



Path={(A,B),(B,C),(C,D),(D,A)}Path={(A,B),(B,D),(D,A),(A,B), (B,A)}

Hamiltonian Path



Hamiltonian Path



A Hamiltonian path or traceable path is a path that visits each vertex exactly once

Hamiltonian Path




Hamiltonian Path


Path*={(A,B),(B,C),(C,D),(D,A)}Path={(A,B),(B,D),(D,A),(A,B), (B,A)}


Optimal

Graph Coloring


Binary counter

CS 421 - Introduction to Algorithms 28

When increment up to n, change is at most 1+floor(log n) bits -number of bits in the binary representation of n.

Thus, the cost of counting up to n, a sequence of nincrements, is O(n log n).

Counter value

A[k] A[k-1] … A[2] A[1] A[0] Cost

0 0 0 … 0 0 0 0

1 0 0 … 0 0 1 1

2 0 0 … 0 1 0 2

3 0 0 … 0 1 1 1

4 0 0 … 1 0 0 3

5 0 0 … 1 0 1 1

6 0 0 … 1 1 0 2

7 0 0 … 1 1 1 1

8 0 0 … 0 0 0 4

… … … … … … …

2k - 1 1 1 … 1 1 1 k

Binary counter


454 Chapter 17 Amortized Analysis

analysis, we assign the amortized cost of each operation to be the average cost. Inthis example, therefore, all three stack operations have an amortized cost of O.1/.

We emphasize again that although we have just shown that the average cost, andhence the running time, of a stack operation is O.1/, we did not use probabilisticreasoning. We actually showed a worst-case bound of O.n/ on a sequence of noperations. Dividing this total cost by n yielded the average cost per operation, orthe amortized cost.

Incrementing a binary counterAs another example of aggregate analysis, consider the problem of implementinga k-bit binary counter that counts upward from 0. We use an array AŒ0 : : k ! 1! ofbits, where A: length D k, as the counter. A binary number x that is stored in thecounter has its lowest-order bit in AŒ0! and its highest-order bit in AŒk ! 1!, so thatx D

Pk!1iD0 AŒi ! " 2i . Initially, x D 0, and thus AŒi ! D 0 for i D 0; 1; : : : ; k ! 1. To

add 1 (modulo 2k) to the value in the counter, we use the following procedure.

INCREMENT.A/

1 i D 02 while i < A: length and AŒi ! == 13 AŒi ! D 04 i D i C 15 if i < A: length6 AŒi ! D 1

Figure 17.2 shows what happens to a binary counter as we increment it 16 times,starting with the initial value 0 and ending with the value 16. At the start ofeach iteration of the while loop in lines 2–4, we wish to add a 1 into position i .If AŒi ! D 1, then adding 1 flips the bit to 0 in position i and yields a carry of 1,to be added into position i C 1 on the next iteration of the loop. Otherwise, theloop ends, and then, if i < k, we know that AŒi ! D 0, so that line 6 adds a 1 intoposition i , flipping the 0 to a 1. The cost of each INCREMENT operation is linearin the number of bits flipped.

As with the stack example, a cursory analysis yields a bound that is correct butnot tight. A single execution of INCREMENT takes time ‚.k/ in the worst case, inwhich array A contains all 1s. Thus, a sequence of n INCREMENT operations onan initially zero counter takes time O.nk/ in the worst case.

We can tighten our analysis to yield a worst-case cost of O.n/ for a sequence of nINCREMENT operations by observing that not all bits flip each time INCREMENTis called. As Figure 17.2 shows, AŒ0! does flip each time INCREMENT is called.The next bit up, AŒ1!, flips only every other time: a sequence of n INCREMENT

NP-Hard

NP-HardOptimized approach:n Cut n Branch and Bound


Conclusion


The Brute Force approach:• Straightforward approach to solving

problem, usually based directly on problem description.

• Wide applicability and simple.• But in general, has subpar performance.

31

Documents

Introduction to Algorithms: Brute-Force Algorithmscs.boisestate.edu/~cs521/slides/NewSlides/BruteForce.pdfCS 421 - Introduction to Algorithms 28 When increment up to n, change is at