View
4
Download
0
Category
Preview:
Citation preview
NP-Completeness, Approximations, & Meta-Heuristics
Teófilo González
Computer Science Department
U of California, Santa Barbara
Soccer
• There are n Soccer teams participating in a soccer tournament. Every team will play against every other team two games (one in each home field).
• Can-Win-Tournament (CWT) Problem:
At mid-season we want to know whether or not a given team X can win the tournament
• Design an efficient algorithm to solve this problem. … Is it possible? …
Old FIFA Point Assignment
• Winning team: 2 points … Loosing team: 0. • Each team in a tie game receives 1 point.
• Under the old FIFA rule, the problem can be
solved efficiently using maximum flow techniques…
• The Point Assignment rule was changed to encourage teams to reduce ties and make games more interesting…
New FIFA Point Assignment
• Winning team: 3 points … Loosing team: 0.
• Each team in a tie game receives 1 point.
• Under the new FIFA rule, the CWT problem is computationally intractable. i.e., it is unlikely that there is an efficient algorithm to solve this problem.
• All known algorithms take exponential time.
Why?
• During the last 50 years we have developed methodologies to solve problems efficiently and techniques to identify computationally intractable problems.
• All of these cannot be fully understood in ones hour, or a few days, or a few months. It takes years to “fully” understand them.
NP-Completeness
• Computationally intractable problems. I.e. problem can be solve computationally, but as far as we know the resources (time) needed for its solution grow exponentially with the size of the problem.
Intractable (Polynomial Time?):
Tractable (Polynomial Time or Efficient)
Intractable vs. Tractable
• Intractable:
• Traveling Salesperson
• Knapsack
• Scheduling
• CWT (3,1,0)
• Rectangular Partitions
• Minesweeper
• Tractable
• Min Cost Spanning Tree
• Sorting
• Shortest Paths
• CWT (2,1,0)
• Guillotine Partitions
Decision Problem
• Problem whose answer is Yes or No (e.g. CWT)
• Shortest Path Problem: Given an edge weighted graph G (representing cities and roads between cities, the weights represent miles), is there a path from vertex s to vertex t with weight at most W (Is the distance between cities s and t at most W?)
• W=5 =>Yes.
• W=4 => No.
NP-Complete Problems
• A class (or group) of decision problems that have the following properties:
• Either all of them can be solved efficiently
or none of them can be solved efficiently.
Conjecture: None of them can be solved efficiently (polynomial time).
Class P
• P: The set of all decision problem that can be solved in polynomial time (O(n), O(n^2), … O(n^k) for some constant k independent of the problem size).
• NP: Set of all decision problems whose YES-answer can be verified (checked) in polynomial time. I.e., we can check correct answers in polynomial time.
• NP-Complete Problems are the “hardest” problems in NP.
• Is P = NP? Central question in Computer Science.
Solvability vs. Checking Yes-Answers
• Solving the CWT(3,1,0) in poly-time means providing an efficient algorithm that given any instance of the problem it decides if its answer is YES or NO.
• Checking a Yes-Answer for the CWT(3,1,0) in poly-time means that given any instance of the problem and an outcome for each of the remaining games, we can check in poly-time if the outcome of the games provided proves that the instance has a YES answer.
• Checking Yes-answers is “easier” than solvability.
Examples
• Solvability is like a student solving an exam.
• Checking (Verifying) is like grading an exam.
• Grading an exam is much simpler than solving an exam.
P = NP?
• Conjecture: P is different than NP. I.e., Solvability is “harder” than Verification.
• If P=NP then programming is really easy. An efficient algorithm to solve the CWT(3,1,0) can be easily derived from an algorithm that given the outcome of the remaining games checks to see if the outcomes proves that the instance is a YES-Instance (has Yes Answer).
Consequences
• If you settle the P=NP question you will be awarded $1,000,000 by the Clay Institute of Mathematics and become world famous for many centuries to come.
• If you come up with a really fast algorithm to solve any NP-Complete problem and have efficient code, then you can start your own company for solving NP-complete instances and become a billionaire quickly and …
you will be able to hire
Bill Gates to clean your windows…
How can we Prove that P = NP?
• Just design an efficient (Poly Time) algorithm to solve the CWT(3,1,0) … Very Difficult Problem.
• The NP-Completeness Theory can then be used to solve all the other NP-Complete problems.
Coping with NP-Complete Problems
• Design efficient algorithm to solve restricted version of NPC problems that arise in practice and are not NPC.
• Approximation Algorithms that provide provable good solutions
• Meta-Heuristics that solve a large number of problem instances.
• …
Approximation Algorithms
• Algorithms that guarantee near-optimal solutions to optimization problems.
Why?
Takes too long to find an optimal solution
e.g.
Input might be noisy (not exact).
GOAL: Find Near-Optimal solutions quickly.
Notation
• P: Problem (Shortest Path)
• I: Problem Instance
• : : Value of optimal solution to I.
• A: Algorithm for P (generates feasible solutions)
• : Value of solution to I generated by A.
Min Problem:
Max Problem:
Example
Bounding the Approximation
Possible Solutions
• K = 3 Three solutions for I
• K = 2 Two solutions for instance I
• K = 1 Only one solution for instance I
• For any K < 20 one solution for instance 10 I
Consequence
• For a large number of problems finding a solution with absolute error at most K, when K is a constant, is as hard as finding an optimal solution!
• Poly Time absolute approximations might not exist for all problems.
Percentage Bounding
• Epsilon is the “approximation or error”
Fully Polynomial time Approx. Scheme
• For some problems and any constant epsilon there are well known efficient approximation algorithms.
• Time Complexity Poly Time in (1/epsilon, n).
• E.g., in O((1/epsilon) n) time
• For example: Knapsack problems, some scheduling problems, …
Poly. Time Approximation Scheme
• For some problems and any constant epsilon there are well known efficient approximation algorithms.
• Time Complexity Poly Time with respect to n.
• E.g., in O(n^(1/epsilon)) time.
• Some scheduling and packing problems.
• Only of theoretical value as they cannot be used in practice.
Single-Point Approximation
• For some value of epsilon it can be approximated efficiently.
• E.g., Metric Travelling Salesperson Problem for epsilon = ½ or 1.
• Many other problems too.
Problems that cannot be Approximated unless P=NP
• Approximating the problem for any constant epsilon is as hard as finding an optimal solution.
• Travelling Salesperson Problem, Clique Problem, …
Approximations
FPTAS – PTAS – Single-Point - Inapproximability
Methodologies for Approximations
• Restriction (Algorithmic or Structural)
• Relaxation (Values or Structure)
• Rounding
• Problem Transformation
• MetaHeuristics
• Combinations of the above techniques
ICS 2016 - Chiayi, Taiwan 2
Restriction • Restrict the set of solutions for the problem and then
find an optimal or suboptimal solution to this
restricted problem.
• Restrict the solution space so that it provides some
structure which can be exploited by an algorithm.
• The restricted problem has an optimal solution value
that is not too far from the true optimal solution
value of the original problem.
• Structural or Algorithmic
ICS 2016 - Chiayi, Taiwan 3
Restriction • Solution Space --- Restricted Solution Space
ICS 2016 - Chiayi, Taiwan 4
Rectangular & Guillotine Partitions
• NPC: Optimal Rectangular Partition.
• Poly-Time: Optimal Guillotine Partition.
• Approximate RP via GP, i.e., Via Restriction.
ICS 2016 - Chiayi, Taiwan 5
0.5-Approximation
ICS 2016 - Chiayi, Taiwan 6
Relaxation • Add a new set of solutions that are not feasible for
the original problem instance in such a way that the
resulting problem can be solved in polynomial time.
• If the solution found is not feasible for the original
instance, then transform it into one.
• (Values) For example if X can only be 0 or 1, allow
0 <= X <= 1. Then round the result …
• (Structure) Instead of visiting each city exactly
once, you may visit it more than once …
ICS 2016 - Chiayi, Taiwan 7
Relaxation • Solution Space ---- Relaxed Solution Space
ICS 2016 - Chiayi, Taiwan 8
Rounding • Round the parameters of the problem in such a
way that the resulting problem can be solved
efficiently. Then use that solution to approximate
the original instance.
• For example, if the profit of an investment is
$1,000,293.45, the only use $1,000,000 instead.
ICS 2016 - Chiayi, Taiwan 9
Rounding • Solution Space --- Rounded Solution Space
ICS 2016 - Chiayi, Taiwan 10
Problem Transformation • Given an instance of problem P, construct an
instance of problem Q for which we can find easily
an optimal or suboptimal solution. Then use that
solution to generate a solution to the original
problem.
• Local Ratio: Make a “local decision” and solve the
“remaining” problem. Local decision subject to
certain constraints (good approximation to local
decision and to resulting problem).
ICS 2016 - Chiayi, Taiwan 11
Transformation • Original Problem --- Transformed Problem
ICS 2016 - Chiayi, Taiwan 12
Primal - Dual • Solution Space --- Primal-Dual Space
ICS 2016 - Chiayi, Taiwan 13
Scheduling, Bin Packing, TSP &
Steiner Trees
ICS 2016 - Chiayi, Taiwan 14
MetaHeuristics • Tabu Search
• Simulated Annealing
• Ant Colony Optimization
• Memetic Algorithms
• Evolutionary Computation
• (Sort of…) Very Large Neighborhood Search,
Reactive Search, Local Search, Stochastic Local
Search, Neural Networks, …
ICS 2016 - Chiayi, Taiwan 15
MetaHeuristics • Single Point vs. Population Based
• Fixed vs. Variable neighborhood
• Memory vs. Memoryless
• Nature-Inspired vs. Algorithmic Based
• Used in practice to solve important problem
instances.
ICS 2016 - Chiayi, Taiwan 16
Basic Approach • Exploration (Diversification) and Exploitation
(Intensification).
ICS 2016 - Chiayi, Taiwan 17
HAAM • A lot of work to be done in this area.
• It is in the boundary of theory and practice.
• 2nd Edition on its way …
ICS 2016 - Chiayi, Taiwan 18
Recommended