View
32
Download
0
Category
Tags:
Preview:
DESCRIPTION
CS 484. Discrete Optimization Problems. A discrete optimization problem can be expressed as (S, f) S is the set of all feasible solutions f is the cost function Goal: Find a feasible solution x opt such that f(x opt )
Citation preview
CS 484
Discrete Optimization Problems
A discrete optimization problem can be expressed as (S, f) S is the set of all feasible solutions f is the cost function
Goal: Find a feasible solution xopt such that f(xopt) <= f(x) for all x in S
Discrete Optimization Problems
Examples VLSI layout Robot motion planning Test pattern generation
In most problems, S is very largeS can be converted to a state-space graph and then can be reformulated to a search problem.
Discrete Optimization Problems
NP-hard Why parallelize?
Consider real-time problems robot motion planning speech understanding task scheduling
Faster search through bigger search spaces.
Search Algorithms
Depth First SearchBreadth First SearchBest First SearchBranch and Bound Use cost to determine expansion
Iterative Deepening A* Use cost + heuristic value to
determine expansion
Parallel Depth First Search
Critical issue is distribution of search space.
Static partitioning of unstructured trees leads to poor load balancing.
Dynamic Load BalancingConsider sequential DFS
Parallel DFS
Each processor performs DFS on a disjoint section of the tree. (Static load assignment)After the processor finishes, it requests unsearched portions of the tree from other processors. Unexplored sections are stored in the stack Pop off a section from the stack and give it
to somebody else.
Parallel DFS Problems
Splitting up the work How much work should you give to
another processor?
Determining a donor processor Who do you request more work from?
Work Splitting Strategies
When splitting up a stack, consider Sending too little or too much increases
work requests
Ideally, rather than splitting the stack, you would split the search space. HARD Nodes high in tree --> big subtrees, &
vice-versa
Work Splitting Strategies
To avoid sending small amounts of work, nodes beyond a specified stack depth are not sent. Cut-off depth
Strategies Send only nodes near bottom of stack Send nodes near cut-off depth Send 1/2 of nodes between bottom and cut-
off
Load Balancing Schemes(Who do I request work from?)
Asynchronous Round Robin each processor maintains target Ask from target then increment target
Global Round Robin target is maintained by master node
Random Polling randomly select a donor each processor has equal probability
Speedups of DFS
Best-First Search
Heuristic is used to direct the searchMaintains 2 lists Open
Nodes unsearched Sorted by heuristic value
Closed Expanded nodes
Memory requirement is linear in the size of the search space explored.
Parallel Best-First Search
Concurrent processors pick the most promising node from the open list Newly generated nodes are placed
back on the open list
Centralized Strategy
Expand the node togenerate successors
Expand the node togenerate successors
Expand the node togenerate successors
at designated processorGlobal list maintained
best node
nodesPut expanded
Getcurrent
Pick the best nodefrom the list
Place generated
nodes in the list
Pick the best nodefrom the list
Place generated
nodes in the list
Unlock the list
Pick the best nodefrom the list
Place generated
nodes in the list
Unlock the listUnlock the list
Lock the list
Lock the list
Lock the list
Centralized Best-First Search
Termination condition A processor may find a solution but
not the best solution. Modify the termination criteria (how?)
Centralization leads to congestion Open list must be locked when
accessed
Extra work
Decentralizing Best-First Search
Let each processor maintain its own open list
Issues: Load balancing Termination (make sure it is the best)
Communication Strategies
Random Periodically send some of the best nodes to a
random processor
Ring Periodically exchange best nodes with neighbors
Blackboard Select best node from open list If l-value is OK then expand If l-value is BAD then get some from blackboard If l-value is GREAT then give some to blackboard
Ring Communication
Blackboard
What about searching a graph?
Problem: node replicationPossible solution: Assign each node to a processor Use hash function Whenever a node is generated, check
to see if it already has been searched
Costly
Speedup Anomalies
Due to nature of the problem, speedup can vary greatly from one execution to the next.Two anomaly types: Acceleration Deceleration
Termination Detection
Dijkstra's Token Termination Detection When idle, send idle token to next processor When idle token is received again, all done
Tree-Based Termination Detection Associate a weight of 1 with initial work load Assign portions of the weight When finished give the weight portion back When processor 0 has weight of 1 --> all
done.
Dijkstra’s Token Termination
All processes are either active or inactive. inactive processes may not send messages other
than the token active processes may turn inactive inactive processes may turn active if they receive
a work message
Termination can only occur if all processes are inactive We must determine if all processes are inactive
and if there are no more messages in the system.
Dijkstra’s Token Termination
Arrange the processes logically in a ringSince all processes must be inactive to terminate, designate process P0 as the process that can start termination detectionWhen inactive, P0 sends a token traveling from process i to i + 1The token only leaves a process if the process is inactive
problem: an inactive process may receive a message and turn active after the token has already left.
solution: introduce colors
Dijkstra’s Token Termination
All processes are initially colored white.Any process i that sends a message to a process j such that j < i is suspect for reactivating a process: change that processes color to blackIf a black process receives a token, it colors the token black.If process 0 receives a white token, send poison pill
Dijkstra’s Token Termination
1
2 3
4
0
1
2 3
4
0
1
2 3
4
0
1
2 3
4
0
1
2 3
4
0
Active
Inactive
Token
work
Dijkstra’s Token Termination
Problem: Fast token and slow work Suppose process j sends work to process i < j Suppose the work message takes a long time to get there In the mean time, process j gets the token and changes it
to black and sends it on. It then changes its state to white. P0 gets the black token and starts the process again Process i now receives the new white token before
receiving the work message that is still in transit Process i passes on the white token Process j will also pass on a white token since it changed
its state to white after sending on the black token The new white token will now arrive at P0 signaling
termination poison pill sent out
Dijkstra’s Token Termination
Problem: Fast token and slow work Suppose process i sends work to process i + 4 Suppose the work message takes a long time to get there In the mean time, process i becomes idle. Process i now receives a white token Process i passes on the white token Process i + 4 receives the white token before the work
message Process i + 4 will also pass on a white token The white token will now arrive at P0 signaling termination
poison pill sent out
Dijkstra’s Token Termination
Solution: Message Counts Send message counts along with the token Initially, all processes are white and have a
message count of 0 Whenever a process receives a message, it
decrements its count and increments its count if it sends a message sum of message counts will be zero iff all
messages have been delivered Token sums message counts as it is passed.
Dijkstra’s Token Termination
If P0 becomes inactive, it turns white and sends a white token with its message count to process 1
If a process sends or receives a message, it turns black
Process i keeps the token as long as it is active. If it turns inactive:
if process i is black, change token to black. Otherwise token color is unchanged
add message count to the token forward the token change state to white
Dijkstra’s Token Termination
If P0 receives a black token, try again.If P0 receives a white token Token has passed through only white
processes However, a message may be in flight
token’s message count will be non-zero If message count is zero, send poison pill Otherwise try again
Recommended