Transcript Document

Chapter 4: Basic Graph
Algorithms and
Computational Complexity
Massoud Pedram
Dept. of EE
University of Southern California
Outline








M. Pedram
Analysis of Algorithms
Graph Algorithms
Dynamic Programming
Mathematical Programming
Stochastic Search
Greedy Algorithms
Divide and Conquer
Floorplanning
Analysis of Algorithms

Big-O notation


Small-o notation


O(f(n)) means that as n  , the execution time t is
at most cf(n) for some constant c
o(f(n)) means that as n  , the execution time t is
at least cf(n) for some constant c
Theta notation

(f(n)) means that as n
 , the execution time t is
at most c1f(n) and at least c2f(n) for some constants c1
and c2 where n = input size
M. Pedram
Big-O Notation

Express the execution time as a function
of the input size n. Since only the
growth rate matters, we can ignore the
multiplicative constants and the lower
order terms, e.g.,
3n2+6n+2.7 is O(n2)
n1.1 + 10000000000n is O(n1.1)
n1.1 is O(n1.1)
M. Pedram
Effect of Multiplicative Constant
Run time
800
f(n)=n2
600
400
f(n)=10n
200
0
M. Pedram
10
20
25
n
Growth Rate of Some Functions
M. Pedram
Exponential
Functions
O(nlog n), O(2n), O(3n), O(4n), O(nn),
O(n!)
Polynomial
Functions
O(log n), O(log2n), O(n0.5), O(n),
O(n log n), O(n1.5), O(n log2n), O(n2),
O(n3), O(n4)
Exponential Functions

Exponential functions increase rapidly, e.g., 2n
will double whenever n is increased by 1
n
10
20
30
40
50
60
M. Pedram
2n
103
106
109
1012
1015
1018
1ms x 2n
0.001 s
1s
16.7 mins
11.6 days
31.7 years
31710 years
NP-Complete



M. Pedram
The class NP-Complete is a set of
problems that, we believe, has no
polynomial time algorithms
Therefore, they are hard problems
If a problem is NP-complete, there is no
hope to solve it efficiently
Solution Type for NP-Complete
Problems




M. Pedram
Exponential time algorithms
Special case algorithms
Approximation algorithms
Heuristic algorithms
Graph Algorithms
Basic Graph Definitions


A graph G(V,E) is a collection of vertices in V
connected by a set of edges E
Dual of a graph G is obtained by replacing each
face of G with one vertex and connecting two
such vertices by an edge if the two
corresponding faces share a common edge in G
A
B
A
D
C
M. Pedram
Graph G
B
D
C
Dual Graph G
Types of Graph Commonly Used in
VLSI CAD





M. Pedram
Interval graphs
Permutation graphs
Comparability graphs
Vertical and horizontal constraint
graphs
Neighborhood graphs and
rectangular dualization
Interval Graphs
A
C
B
E
D
G
F
A
B
C
E
F
G
D
V = Set of intervals
E = {(vi,vj) | vi and vj intersect}
M. Pedram
Permutation Graphs
A
B
C
B
E
D
A
D
E
A
C
E
B
D
C
V = Set of lines
E = {(vi,vj) | vi and vj intersect}
M. Pedram
Comparability Graphs

Orientable Property: Each edge can be
assigned a one-way direction in such a way
that the resulting graph G(V,F) satisfies the
following property:


(a, b)  F and (b, c)  F  (a, c)  F
An undirected graph which is transitively
orientable is called a comparability graph
?
Orientable graph
M. Pedram
Non-orientable graph
Horizontal Constraint Graphs
A
C
D
A
B
wA
B
wB
wA
D
C
wC
Weighted directed graph:
V = Set of modules
E = {(vi,vj) | vj on the right of vi}
weight(vi, vj) = width of module vi
M. Pedram
Vertical Constraint Graphs
A
C
D
A
hA
B
B
hB
D
C
Weighted directed graph:
V = Set of modules
E = {(vi,vj) | vj above vi}
weight(vi, vj) = height of module vi
M. Pedram
Neighborhood Graphs
A
B
C
D
A
B
D
C
V = Set of modules
E = {(vi,vj) | vj and vi share an edge}
M. Pedram
Rectangular Dualization
Rectangular dualization does the inverse
operation of the neighborhood graph
construction
A
B
A
D
C
B
D
C
Each rectangle represents a vertex
vi and vj share an edge exactly if e(vi,vj) exists
Some graphs have no rectangular duals
M. Pedram
Basic Graph Algorithms Commonly
Used in VLSI CAD





M. Pedram
Minimum Spanning Tree (MST)
Problems
Steiner Minimum Tree (SMT)
Problems
Rectilinear SMT Problems
Partitioning Problems
Network Flow Problems
Minimum Spanning Tree (MST)
Problem: Given a connected
undirected weighted graph G(V,E),
construct a minimum weighted tree
T connecting all vertices in V
Note: A graph is connected exactly if there
is at least one path for any pair of vertices
M. Pedram
Minimum Spanning Tree (MST)

Kruskal’s algorithm:


Prim’s algorithm :




M. Pedram
O(m log n)
O(m log n)
Can be improved to O(n log n)
A greedy algorithm
Note: m = no. of edges; n = no. of
vertices
Kruskal’s Algorithm



M. Pedram
Sort the edges
T=
Consider the edges in ascending order
of their weights until T contains n-1
edges:
 Add an edge e to T exactly if adding e
to T does not create any cycle in T
Kruskal’s Algorithm Example
e
e
M. Pedram
Kruskal’s Algorithm Example (Cont’d)
e
e
M. Pedram
Kruskal’s Algorithm Example (Cont’d)
M. Pedram
Prim’s Algorithm



M. Pedram
T=
S = {v} for an arbitrary vertex v
Repeat until S contains all the
vertices:
 Add the lightest edge e(v1,v2) to T
where v1S and v2(V-S). Add v2
to S
Prim’s algorithm Example
M. Pedram
Prim’s algorithm Example (Cont’d)
M. Pedram
Steiner Minimum Tree (SMT)
Problem: Given an undirected weighted
graph G(V,E) and a demand set D  V,
find a minimum cost tree T which spans
a set of vertices V’V such that DV’
Elements of V’-D, which have a degree larger
than 2 in T, are called the Steiner points
M. Pedram
Steiner Minimum Tree (SMT)



M. Pedram
When D = V, SMT = MST
When |D| = 2, SMT = Single Pair
Shortest Path Problem
The SMT problem is NP-Complete
Rectilinear Steiner Tree (RST)

A Steiner tree whose edges are
constrained to be vertical or horizontal
Steiner point
Demand point
M. Pedram
Rectilinear Steiner Minimum Tree
(RSMT)


M. Pedram
Similar to the SMT problem, except that
the tree T is a minimum cost rectilinear
Steiner tree
This problem is NP-Complete but we can
get an approximate solution to this
problem by making use of the MST
Approximate Solution to RSMT


Construct a MST T
Obtain a RST T’ from T by connecting
the vertices in T using vertical and
horizontal edges
There can be many solutions
or
MST
M. Pedram
a better one
Approximate Solution to RSMT

M. Pedram
It is proved that:
WMST  1.5 WRSMT
Let W be the weight of the Steiner tree
obtained by this method, i.e., by
“rectilinearizing” the MST
What is the smallest b such that:
W  bWMST
Other Graph Problems





M. Pedram
Minimum Clique Covering Problem
Maximum Independent Set Problem
Minimum Coloring Problem
Maximum Clique Problem
Network Flow Problem
Minimum Clique Covering



M. Pedram
A clique of a graph G(V,E) is a set of vertices
V’ V such that every pair of vertices in V’
are joined by an edge
The clique cover number, k, of a graph G(V,E)
is the minimum no. of cliques needed to cover
all of the vertices of G
The problem of finding the clique cover
number is NP-complete for general graphs
Maximum Independent Set (MIS)




M. Pedram
An independent set of a graph G(V,E) is a set
of vertices V’ V such that no pair of vertices
in V’ are joined by an edge
The MIS problem is to find the independence
(stability) number, a, of a graph, i.e., the size
of the largest independent set in the graph
This problem is NP-complete for general
graphs
What is the relationship between the clique
cover number and the independence number?
Answer: a (G)  k (G)
Graph Problems in Interval Graphs




What is the meaning of a clique cover in an interval
graph?
What is the meaning of a MIS in an interval graph?
The clique covering and MIS in an interval graph can
be found in O(n log n) time where n is the no. of
intervals. How?
Answer for MIS:




M. Pedram
Sort the 2n points in ascending order
Scan list from left to right until encounter a right endpoint
Output the interval having this right endpoint as a member of
MIS and delete all intervals containing this point
Repeat until done
Minimum Chromatic Number


M. Pedram
The chromatic number, c, of a graph G(V,E) is
the minimum no. of colors needed to color the
vertices such that every pair of vertices joined
by an edge are colored differently
The problem of finding the chromatic number
is NP-complete for general graphs
Maximum Clique



M. Pedram
The maximum clique problem is to find the
clique number, w, of a graph, i.e., the size
of the largest clique in the graph
This problem is NP-complete for general
graphs
What is the relationship between the
chromatic number and the clique number?
Answer: w (G)  c (G)
Graph Problems in Interval Graphs (Cont’d)



What is the meaning of the chromatic number and the
clique number in an interval graph?
The chromatic number and the clique number of an
interval graph can be found in O(n log n) time. How?
Answer for maximum clique {simple O(n2) algorithm}



A = Sort intervals (I);
cliq=0; max-cliq=0;
for i=1 to 2n do

if A[i]=Left then cliq++;



M. Pedram
if cliq > max-cliq then max-cliq=cliq;
else cliq--;
Return max-cliq;
Perfect Graphs

A graph G(V,E) is called perfect of all of
its induced sub-graphs satisfy the
following two properties:




M. Pedram
P1: a (GI )  k (GI )
P2: w(GI )  c (GI )
Interval graphs, permutation graphs and
orientable graphs are examples of the
perfect graphs; Cycle of odd length > 3 is
not a perfect graph
The four key graph problems can be
solved in polynomial time for the class of
perfect graphs
Partitioning Problem
Problem: Given an undirected graph
G(V,E), partition V into two sets V1 and
V2 of equal sizes such that the number
of edges |E1| between V1 and V2 is
minimized
E1 is called the cut
M. Pedram
Partitioning Problem

More general versions of this problem:



M. Pedram
Specify the sizes of V1 and V2
Partition V into k sets where k  2
All these problems are NP-complete
Network Flow Problem
Problem: Given a weighted directed
graph G(V,E). One vertex sV is
designated as the source and one vertex
tV as the sink. Find the maximum flow
from s to t such that the flow along each
edge does not exceed the capacity
(weight) of the edge
M. Pedram
Network Flow Problem

For example:
a
10
s


M. Pedram
4
5
7
3
c
b
5
2
7
6
d
t
10
What is the maximum flow in this
network? (see next slide)
Network flow problem can be solved in
polynomial time
Network Flow Problem

The maximum flow is 9
9/10
s
2/5
7

4/4
c 2/2
b
5
3/7
6
d
t
6/10
A well known theorem:

M. Pedram
a
3/3
Maximum Flow = Minimum Cut
Network Flow Problem

The minimum cut is 9
3
a
10
s
M. Pedram
5
5
7

4
c
b
2
7
6
d
t
10
Notice that we only count the edges
going from s to t
Network Flow Problem

M. Pedram
Unfortunately, we cannot use this
network flow method to solve the
partitioning problem. Why?
Dynamic Programming
Dynamic Programming

M. Pedram
In dynamic programming, an array
A[1..n] (can be of higher dimension) of
data is defined and stored. The
computation of A[k] is dependent on the
values of A[j] where j < k, so we can
compute their values one by one, i.e.,
A[1], then A[2], then A[3],
Dynamic Programming
For example:
s
t
How many shortest multi-bend routes
can there be from s to t?
M. Pedram
Dynamic Programming
We define an array A[1..8, 1..6] where A[x,y]
is the number of routes from s to (x,y):
x
s
1
1
1
2
3
1
y
1
1
1
1
1
A[1,k] = 1
A[j,1] = 1
A[j,k] = A[j-1,k] + A[j,k-1]
3
1
1
1
t
How about if we only allow 1-bend?
M. Pedram
Dynamic Programming in Maze Routing
Criteria: Follow Shortest Manhattan Distance
Always route towards t with minimum distance
s
A[1,k] = 1
A[j,1] = 1
A[j,k] = A[j-1,k] + A[j,k-1]
t
M. Pedram
Dynamic Programming in Maze Routing
Allow detours, i.e., allow routing away from t (routes do
not follow the shortest Manhattan distance
detour
s
If at most 2 detours are allowed,
what is the DP for this problem?
t
detour
M. Pedram
A[1,k] =?
A[j,1] = ?
A[j,k] =?
Mathematical Programming

In mathematical programming, the
problem is expressed as an objective
function and a set of constraints. The
followings are common problems that
are solvable in polynomial time:



M. Pedram
Linear Programming
Quadratic Programming
Geometric Programming
Mathematical Programming

For Example:
2
1
3
M. Pedram
Let A1, A2 and A3 be the area
of the three modules
What should be their dimensions
(w1, w2, and w3) in order to
minimize the size of the bounding
rectangle?
Mathematical Programming
Objective:
Minimize WH
2
1
3
M. Pedram
Constraints:
W  w1 + w2
W  w3
H  A3/w3 + A1/w1
H  A3/w3 + A2/w2
Linear Programming (LP)



Both the objective and constraint
functions are linear
LP can be solved optimally
Many solvers are available that can
efficiently solve problems with a large
number of variables

M. Pedram
http://www-fp.mcs.anl.gov/otc/Guide/SoftwareGuide/Categories/linearprog.html
Integer Linear Programming (ILP)



M. Pedram
ILP is a variant of LP in which all of the
variables are integer
ILP is NP-complete
LP that contains both integer and real
variables is called mixed integer linear
programming
Stochastic Search




M. Pedram
Local Search
Tabu Search
Genetic Algorithm
Simulated Annealing
Local Search


M. Pedram
Local search is a simple searching
algorithm that always move “downhill” in
the solution space
The notion of neighborhood N(f) of a
feasible solution f is used. Given f, N(f) is
a set of feasible solutions that are
“close” to f based on some criteria
Local Search
Algorithm:
Initialize a feasible solution f
Do
G  {g | g  N(f) and cost(g) < cost(f)}
If (G  ), assign f as one element in G
while (G  )
Return f
M. Pedram
Local Search



M. Pedram
It is called first improvement if we
always assign f with the first element
found with a lower cost in the do-whileloop
It is called steepest descent if we always
assign f with the element of the lowest
cost in the do-while-loop
The biggest problem of local search is
getting stuck in a local minimum
Tabu Search



M. Pedram
Tabu search allows “uphill” moves
Given a neighborhood set N(f) of a
feasible solution f, the principle of tabu
search is to move to the cheapest
element gN(f) even when cost(g) >
cost(f)
A Tabu list containing the k last visited
solutions is maintained to avoid circular
search pattern of length  k
Tabu Search
Algorithm:
Initialize a feasible solution f
best  f, Q  an empty queue
Do
G  {g | g  N(f) and g  Q}
If (G  )
Assign f as the cheapest element in G
If |Q| < k, enqueue(Q, f); else dequeue(Q),
enqueue(Q, f)
If cost(best) > cost(f), best  f
while (G  ) and not stop()
Return best
M. Pedram
Genetic Algorithms (GA)



M. Pedram
Instead of keeping just one current
solution, GA keeps track of a set P of
feasible solutions, called population
In each iteration, the current population
Pi is replaced by the next one Pi+1
To generate a feasible solution hPi+1,
two solutions f and g are selected from
Pi and h is generated such that it
inherits properties from both parents f
and g . This is called crossover
Genetic Algorithms (GA)


M. Pedram
A lower cost solution in a population will
have a higher chance of being selected
as the parent
Mutation may occur to avoid getting
stuck in a local minimum
Genetic Algorithms (GA)
Algorithm:
Initialize P with k feasible solutions
Do
newP  
For (i = 1; i  k; i++}
Select two low cost solutions from P as f and g
h  crossover(f, g)
Add h to newP
P  newP
while not stop()
Return the best solution in P
M. Pedram
Markov Chain




Let G(V,E) be a directed graph with V={S1,S2,…..Sn}
Let C: VZ be a vertex-weighting function
We write Ci for C(Si)
Let P: E[0,1] be an edge-weighting function
(notation: Pij=P(Si,Sj) such that:

Pij  1
for all SiV
S j N ( Si )




M. Pedram
N(Si) is the set of next next states of Si
We call (G,P)  (finite) time-homogenous Markov
chain
Pij is called the conditional probability of edge(Si,Sj)
Assume Pij>0, if Pij=0, simply delete edge(Si,Sj) from E
Irredundant Markov Chain

Irredundant Markov Chain



A Markov chain (G,P)is irreducible if G is strongly
connected
P=[pij] is the matrix whose <i,j> entry is pij
Stationary distribution


i is the probability of being in state i
A state distribution  of a Markov Chain (G,P) is
stationary if
 P 


i  1



M. Pedram
Chapman  Kolmogorov equations
Markov Chain Example
1
1/2
2/3
2
1/3
1
1/2
3
0 2/3 1/ 3
1  2  3  0 1/ 2 1/ 2  1  2  3 
1 0
0 
1  2  3 1

 1  3/10
 2  4/10

 3  3/10
M. Pedram
Simulated Annealing (SA)


M. Pedram
Simulated annealing is a powerful
technique to provide high quality
solutions to some difficult combinatorial
problems
It keeps a variable Temperature (T)
which determines the behavior of the
annealing process. This variable T is
initialized to a very large value at the
beginning, and will be gradually
decreased (cooled down).
Simulated Annealing
Algorithm:
Initialize T and a feasible solution f
While (T  a threshold)
• Make a slight modification to f to get g
• Check if g is better than f, i.e., cost(g) 
cost(f)?
• If yes, accept g, i.e., f  g; else, compute p as
e-k(cost(g)-cost(f))/T where k is a positive constant,
and then, accept g with probability p
• Update T
M. Pedram
Basic Ingredients of Simulated
Annealing




Solution Space
Neighboring Structure
Cost Function
Annealing Schedule

M. Pedram
Moves are selected randomly, and the
probability that a move is accepted is
proportional to system’s current temp
Simulated Annealing
500 modules
This is a floorplanning result obtained
by simulated annealing
M. Pedram
Floorplanning
Hierarchical Design

Several blocks after partitioning:

Need to:


Put the blocks together.
Design each block.
Which step to go first?
M. Pedram
Hierarchical Design


M. Pedram
How to put the blocks together without
knowing their shapes and the positions
of the I/O pins?
If we design the blocks first, those
blocks may not be able to form a tight
packing
Floorplanning
The floorplanning problem is to plan the
positions and shapes of the modules at
the beginning of the design cycle to
optimize the circuit performance:





M. Pedram
chip area
total wire length
delay of critical path
routability
others, e.g., noise, heat dissipation, etc.
Floorplanning v.s. Placement


Both determines block positions to
optimize the circuit performance.
Floorplanning:


Placement:

M. Pedram
Details like shapes of blocks, I/O pin
positions, etc. are not yet fixed (blocks with
flexible shape are called soft blocks)
Details like module shapes and I/O pin
positions are fixed (blocks with no flexibility
in shape are called hard blocks)
Floorplanning Problem

Input:



Output:


Coordinates (xi, yi), width wi and height hi
for each block such that hi wi = Ai and ri 
hi/wi  si
Objective:

M. Pedram
n Blocks with areas A1, ... , An
Bounds ri and si on the aspect ratio of
block Bi
To optimize the circuit performance
Bounds on Aspect Ratios
If there is no bound on the aspect
ratios, we can surely pack very tightly:
However we don’t want to layout blocks
as long strips, so we require ri  hi/wi 
si for each i
M. Pedram
Bounds on Aspect Ratios


M. Pedram
We can also allow several shapes for
each block:
For hard blocks, the orientations can be
changed:
Objective Function
A commonly used objective function is a
weighted sum of area and wirelength:
cost = aA + bL
where A is the total area of the packing,
L is the total wire length, and a and b
are constants
M. Pedram
Wire length Estimation



Exact wire length of each net is not
known until routing is done
In floorplanning, even pin positions are
not known yet
Some possible wire length estimations:


M. Pedram
Center-to-center estimation
Half-perimeter estimation
Dead space

Dead space is the space that is wasted:
Dead space


M. Pedram
Minimizing area is the same as
minimizing dead space
Dead space percentage is computed as
(A - iAi) / A  100%
Mixed Integer Linear Program

A mathematical program such that:




M. Pedram
The objective is a linear function
All constraints are linear functions
Some variables are real numbers and some
are integers, i.e., “mixed integer”
It is almost like a linear program, except
that some variables are integers
Problem Formulation

Minimize the packing area:




Y
Need to have constraints
so that blocks do not overlap
Associate each block Bi with 4 variables:


M. Pedram
Assume that one dimension W is fixed
Minimize the other dimension Y
xi and yi: coordinates of its lower left corner
wi and hi: width and height
W
Non-overlapping Constraints

For two non-overlapping blocks Bi and
Bj, at least one of the following four
linear constraints must be satisfied:
hi
(xi, yi)
M. Pedram
Bi
wi
hj
(xj, yj)
Bj
wj
Integer Variables


M. Pedram
Use integer (0 or 1) variables xij and yij:
xij=0 and yij =0 if (1) is true
xij=0 and yij =1 if (2) is true
xij=1 and yij =0 if (3) is true
xij=1 and yij =1 if (4) is true
Let W and H be upper bounds on the total
width and height. Non-overlapping
constraints:
(1' ) xi  wi  x j  W ( xij  yij )
(2' ) xi  w j  x j  W (1  xij  yij )
(3' ) yi  hi  y j  H (1  xij  yij )
(4' ) yi  h j  y j  H (2  xij  yij )
Formulation
M. Pedram
Formulation with Hard Blocks

M. Pedram
If the blocks can be rotated, use a 0-1 integer variable
zi for each block Bi s.t. zi = 0 if Bi is in the original
orientation and zi = 1 if Bi is rotated 90o
Formulation with Soft Blocks


M. Pedram
If Bi is a soft block, wihi=Ai. But this
constraint is quadratic!
Linearized by taking the first two terms
of the Taylor expression of hi=Ai/wi at
wimax (max. width of block Bi)
hi =himin+li(wimax-wi)
where himin =Ai/wimax and li=Ai/wimax2
Formulation with Soft Blocks
M. Pedram

If Bi is soft and Bj is hard:

If both Bi and Bj are soft:
Solving Linear Program


Linear Programming (LP) can be solved
by classical optimization techniques in
polynomial time.
Mixed Integer LP (MILP) is NP-Complete.

M. Pedram
The run time of the best known algorithm is
exponential to the no. of variables and
equations
Complexity

For a problem with n blocks, and for the
simplest case, i.e., all blocks are hard:




M. Pedram
4n continuous variables (xi, yi, wi, hi)
n(n-1) integer variables (xij, yij)
2n2 linear constraints
Practically, this method can only solve
small size problems.
Successive Augmentation

A classical greedy approach to keep the
problem size small: repeatedly pick a small
subset of blocks to formulate a MILP, solve it
together with the previously picked blocks
with fixed locations and shapes:
Next subset
Y
M. Pedram
Partial Solution