Transcript Analysis & Design of Algorithms (CSCI 321)

Analysis & Design of
Algorithms
(CSCE 321)
Prof. Amr Goneid
Department of Computer Science, AUC
Part 11. Backtracking
Algorithms
Prof. Amr Goneid, AUC
1
Backtracking Algorithms
Prof. Amr Goneid, AUC
2
Backtracking Algorithms
 Problem Statement
 Brute Force & Backtracking
 Permutation Tree
 The 4-Queens Problem
 Sum of Subsets
 Unbound DFG: Permutation Tree
 Bound DFG: Backtracking
 General Backtracking Algorithm
Prof. Amr Goneid, AUC
3
Backtracking Algorithms
 Example: n-Binary Variables Problem
 The n-Queens Problem
 Another Backtracking Schema
 The Hamiltonian Circuit Problem by
Backtracking
Prof. Amr Goneid, AUC
4
1. Problem Statement
 Given sets S1, S2, …, Sn of values x
with m1, m2, …, mn values in the sets.
 We seek a solution vector (n-tuple)
X = (x1, x2, …, xn)
chosen from the sets out of
m = m1 m2 ….mn possible candidates
satisfying a criterion function P(X)
Prof. Amr Goneid, AUC
5
2. Brute Force & Backtracking
 Sometimes the best algorithm for a problem is to try
all possibilities.
 This is always slow, but straightforward.
 Brute Force (Exhaustive Search) Methods:
Will try all possible m trials and save those satisfying
P(X)
Prof. Amr Goneid, AUC
6
Brute Force & Backtracking
 Backtracking:
 Yields same answer with fewer than m trials.
 Builds solution vector one component at a time.
 Examines if a partial vector Xi = (x1,x2,..,xi) has a
chance of success.
 If it does not satisfy constraints, we drop out the k
remaining trials, k = mi+1 mi+2 …mn
Prof. Amr Goneid, AUC
7
The 8-Queens Problem
 Place 8 Queens on an 8 x 8 chessboard such that no




two can attack each other. There are about 4.4 billion
trials.
Obvious: Each queen must be on a different row. All
solutions are 8-tuples
X = (x1,x2,..,x8) , where xi is the column number of
the ith queen.
Brute Force Trials: m = 8*8*….*8 = 88 = 224 = 16 M
Constraint(1) : Each queen must be on a different
column. This reduces number of trials to 8! = 40,320
Constraint(2): No two queens can be on the same
diagonal.
Prof. Amr Goneid, AUC
8
The 8-Queens Problem (continued)
 Backtracking:
Q
will use only 1625 trials
to achieve solution.
 Possible Solution:
Q
Q
Q
X = (4 , 6 , 8 , 2 , 7 , 1 , 3 , 5)
Q
Q
Q
Q
Prof. Amr Goneid, AUC
9
3. Permutation Tree
 Represents the problem to be solved.
 Problem State: A node in the tree.
 Layer: between two consecutive levels and




represents one variable in the tuple. Edges from level
(i) to level (i+1) are labeled with values of variable xi.
State Space: All paths from root to other nodes.
Solution States: Nodes from the root defining a
tuple.
Answer States: Solution states satisfying implicit
conditions.
Node Generation: Depth First
Prof. Amr Goneid, AUC
10
Example
 Problem:




Find all sequences of three binary variables
such that no two consecutive values are the
same.
S1 = S2 = S3 = {0,1} , m1 = m2 = m3 = 2
n = 3 , xi = 0 or 1 , i = 1 , 2 , 3
Total number of brute force trials = m1m2m3 =
2*2*2=8
Implicit Conditions:
x 1  x 2 , x2  x 3
Prof. Amr Goneid, AUC
11
Brute Force Permutation Tree
1
x1
0
1
2
x2
9
0
1
3
x3 0
4
6
0
1
5
7
1
0
10
1
8
13
0
1
11
12
{0,1,0)
0
14
1
15
{1,0,1)
Prof. Amr Goneid, AUC
12
Backtracking Method
 Generate nodes in Depth First order.
 Kill nodes (and their children) not
satisfying constraints.
 Backtrack to higher level to seek a
different path to a leaf node.
 If all leaves are killed, the problem has
no solution.
Prof. Amr Goneid, AUC
13
Bactracking Permutation Tree
1
x1
0
1
2
x2
9
0
1
3
x3 0
4
6
0
1
5
7
1
0
10
13
1
0
1
0
1
8
11
12
14
15
{0,1,0)
{1,0,1)
Prof. Amr Goneid, AUC
14
The Solution
 # of solution states = # of leaves = 8
 Total # of nodes in tree = measure of brute




force cost = 15
# of nodes generated = measure of
Backtracking cost = 11
# of surviving nodes = 7
# of killed parent nodes = 4
# of answer states (3-tuples) = 2
X1 = {0,1,0} , X2 = {1,0,1}
Prof. Amr Goneid, AUC
15
Conclusion
 Backtracking is a Divide & Conquer Brute Force




(exhaustive) search with pruning.
Saves time by killing internal nodes that have no
useful leaves.
Let Ne and Nb be the number of nodes generated by
the exhaustive search and backtracking methods,
respectively.
A measure of the Gain obtained by backtracking is
G = (1 – Nb / Ne)*100 %
For the previous example, G = (1 – 11/15)*100 =
26.7%
Prof. Amr Goneid, AUC
16
4. The 4-Queens Problem
(a) The Problem
 Place 4 Queens on an 4 x 4 chessboard such
that no two can attack each other.
 With no constraints, there are
16 * 15 * 14 * 13 = 43,680 possible
placements.
 Constraint (1): Each Queen must be on a
different row. Now we seek the 4-tuples
X = {x1 , x2 , x3 , x4} representing column
numbers. There are 4*4*4*4 = 256 such
tuples.
Prof. Amr Goneid, AUC
17
The Problem (cont.)
 Constraint(2): Each Queen must be on a
different column.
The number of possible tuples reduces to
4*3*2*1 = 4! = 24
 Backtracking will be used to impose the final
No Attack constraint so that no two queens
can be placed on the same diagonal
Prof. Amr Goneid, AUC
18
(b) The Permutation Tree
 The root will have 4 children
 In general, each node in level L will have
(4-L+1) children. The total # of nodes in the
tree will be
1 + 4 + 4*3 + 4*3*2 + 4*3*2*1 = 65 nodes
 The # of leaves will be 4! = 24 tuples. Which
of these will be answers ?
Prof. Amr Goneid, AUC
19
(c) Portion of the Permutation Tree
1
1
2
18
2
2
4
3
3
1
8
2
9
19
13
4
11
2
3
14
16
4
3
24
29
1
30
3
3
15
X = {2,4,1,3} 31
Prof. Amr Goneid, AUC
20
(d) Solutions
Leaf 31
Leaf 39
Q
Q
Q
Q
Q
Q
Q
Q
{2,4,1,3}
{3,1,4,2}
Prof. Amr Goneid, AUC
21
(e) Performance
 Brute Force cost = 65
 Backtracking:
# nodes generated = 32
# nodes surviving = 18
# parent nodes killed = 14
Gain G = (1-32/65)*100 = 50.8%
Prof. Amr Goneid, AUC
22
5. Sum of Subsets
(a) The Problem
 Given a set W of positive integers wi ,
i = 1,2,…,n and m, find all subsets of W
whose sums are equal to m.
 Example: n = 4 , W = {11,13,24,7},
m = 31
There are 2 possible subsets:
S1 = {11,13,7} , S2 = {24,7}
Prof. Amr Goneid, AUC
23
The Problem (cont.)
 Constraints:
1. A member appears only once in the
subset.
2. The sum of a subset is exactly m.
3. No multiple instances of the same subset.
For example:
{1,4,2} and {1,2,4}
This is satisfied by requiring wi < w i+1
Prof. Amr Goneid, AUC
24
(b) Backtracking Solution
 Consider the previous example with
n=4,
W = {11,13,24,7},
m = 31
 Let X = {x1,x2,..,xn} be a solution such
that xi  {0,1} ,
xi = 1 if wi is selected and xi = 0
otherwise. Hence, we obtain fixed-size
tuples.
Prof. Amr Goneid, AUC
25
Brute Force Permutation Tree
 The permutation tree will be a complete
binary tree with a height of n+1. The #
of leaves (possible subsets) will be
2n = 16 and the total # of nodes will be
31.
Prof. Amr Goneid, AUC
26
(c) Portion of Permutation Tree
1
0
1
2
1
3
1
1
5
10
0
7
4
0
6
1
8
17
0
9
1
0
11
14
12
Prof. Amr Goneid, AUC
13
15
16
27
Performance
 # nodes generated = 25
# nodes survived = 14 # killed = 11
 G = 19.4%
 Two possible solutions X1 = {1,1,0,1}
and
X2 = {0,0,1,1}
Prof. Amr Goneid, AUC
28
6. Unbound DFG: Permutation Tree
 The full (Brute Force) permutation tree
is generated by an unbound Depth-First
Generation (DFG) algorithm
 Example:
Binary strings of length (n) bits.
The DFG algorithm generates a full
permutation tree of n+1 levels
Prof. Amr Goneid, AUC
29
Unbound DFG: Algorithm
// Assume there is a root node
void Unbound_DFG(int k, int n)
{
for (i = 0; i <= 1; i++) {
Generate edge (i) from current parent;
Generate child node;
if (k == n) then leaf node has been reached;
else Unbound_DFG(k+1, n);
}
}
Prof. Amr Goneid, AUC
30
Example: All 2-bit strings
1
x1
0
1
2
x2
0
3
5
1
0
4
6
{00}
{01}
{10}
n = 2; Unbound_DFG(1,n)
Prof. Amr Goneid, AUC
1
7
{11}
31
7. Bound DFG: Backtracking
 A partial permutation tree is generated
by a bound Depth-First Generation
(DFG) algorithm
 Example:
n-bit binary strings, with a bounding
function B(k,i) == true
The DFG algorithm generates a partial
permutation tree of n+1 levels
Prof. Amr Goneid, AUC
32
Bound DFG: Algorithm
// Assume there is a root node
void Bound_DFG(int k, int n)
{
for (i = 0; i <= 1; i++) {
if ( B( k , i ) ){
Generate edge (i) from current parent;
Generate child node;
if (k == n) then leaf node has been reached;
else Bound_DFG(k+1, n);
}
}
}
Prof. Amr Goneid, AUC
33
Example: n-bit binary strings
1
x1
0
1
2
x2
5
0
0
3
6
{00}
{10}
n = 2; Bound: 2nd bit should not be 1
B’(k,i) = (k==2) && (i==1)
Hence B(k,i) = (k == 1) || (i == 0)
Invoke as Bound_DFG(1,n)
Prof. Amr Goneid, AUC
34
8. General Backtracking Algorithm
 Consider:
n = no. of variables
k = index of a variable
x1 , x2, .. ,xk, .. xn is a path to a solution
state.
m = no. of possible values for x
vi = a value for a variable, i = 1, 2, .. m
B(k,i) = Bounding Function
= true if xk can take the value vi
Prof. Amr Goneid, AUC
35
Brute Force Algorithm
void Brute_Force (int k, int n)
{
for all possible values vi of a variable (i = 1..m)
{
Let variable xk take the value vi ;
if (xk is the last variable )
output vector x[1:n];
else Brute_Force (k+1, n);
}
}
Prof. Amr Goneid, AUC
36
General Brute Force Code
void Brute_Force (int k, int n)
{
for (i = 1; i <= m; i++)
{
x[k] = v[i] ;
if (k == n) output vector x[1:n];
else Brute_Force (k+1, n);
}
}
Prof. Amr Goneid, AUC
37
Backtracking Algorithm
void Backtrack(int k, int n)
{
for (i = 1; i <= m; i++) {
if ( B( k , i )) {
// Pruning
x[k] = v[i] ;
if (k == n) output vector x[1:n];
else Backtrack(k+1, n);
}
}
}
Prof. Amr Goneid, AUC
38
9. Example:
n-Binary bits Problem
 Problem:
Find all strings of n-binary bits such that no
two consecutive bits are the same.
 n given , m = 2 , xk = 0 or 1 , k = 1 , 2 , .. , n
 Bounding Function:
Assume x0 = -1 then x1  x2 , x2  x3 , ……
or if (xk-1  i) then xk can take the value (i)
Prof. Amr Goneid, AUC
39
Backtracking Algorithm
void Backtrack(int k, int n)
{
for (i = 0; i <= 1; i++) {
if ( x[k-1] != i) { // assume x[0] = -1
x[k] = i ;
if (k == n) output vector x[1:n];
else Backtrack(k+1, n);
}
}
}
Prof. Amr Goneid, AUC
40
Permutation Tree for n = 3
1
x1
0
1
2
x2
9
0
0
1
3
6
0
10
1
1
13
0
1
11
12
x3
7
8
{0,1,0)
{1,0,1)
Prof. Amr Goneid, AUC
41
10. The n-Queens Problem
(a) The Problem
 Find all possible arrangements of n Queens
on an n x n chessboard such that no two can
attack each other.
 Example:
8 queens on an 8x8 board
The problem has 92 solutions.
Only 12 are unique, others are
reflections or rotations.
Prof. Amr Goneid, AUC
42
The n-Queens Problem
 Pre-Condition: Each Queen is on a
different row.
 We seek the n-tuples
X = {x1 , x2 , … , xn} representing column
numbers satisfying the problem.
 Hence, we seek all permutations of X
Prof. Amr Goneid, AUC
43
(b) The Bounding Function
 Assume that Queens 1, 2, .. K-1 have already been
properly placed.
 The bounding function for Queen (k) is that it can be
placed in column (i) iff:
 It does not share the same column (i) with any of
the previous queens (j), j = 1,2,.., k-1, i.e Xj  i
 It is not on the same diagonal with any of the
Q
Row j
previous queens:
│k-j│  │Xj - i│ , j = 1,2,.., k-1
j
Row k
Qk
Col X[ j]
Prof. Amr Goneid, AUC
Col (i)
44
The Bounding Function
bool can_place (int k, int i)
{
for (int j = 1; j < k; j++)
if ((x[ j ] == i) || abs(x[ j ] - i) == abs(k-j))
return false;
return true;
}
Prof. Amr Goneid, AUC
45
(c)Backtracking Algorithm
void NQueens(int k, int n)
{
for (i = 1; i <= n; i++) {
if ( can_place( k , i ) ){
x[k] = i ;
if (k == n) output vector x[1:n];
else NQueens(k+1, n);
}
}
}
Prof. Amr Goneid, AUC
46
The 8-Queens Problem Animation
Prof. Amr Goneid, AUC
47
11. Another Backtracking Schema
 Another schema exists when we replace
the bounding function by a function that
assigns to x[k] a legal value after x[1: k-1]
have already been assigned legal values
out of m possible values. If no such value
is available, the algorithm backtracks.
 Assume the function to be
NextValue (k , n) and the vector x[1:n] is
initially set to zero.
Prof. Amr Goneid, AUC
48
The Algorithm
void Backtrack2(int k, int n)
{
do {
NextValue(k,n); // Assign to x[k]
// a legal value
if( !x[k] ) break ; // No possible value
if (k == n) output vector x[1:n];
else Backtrack2(k+1, n);
} while(1);
}
Prof. Amr Goneid, AUC
49
12. Hamiltonian Circuit Problem
by Backtracking
0
a
c
d
a
b
1
f
b
e
2
9
c
f
3
6
d
e
4
e
e
7
8
d
f
dead end
c
10
11
dead end
5
d
f
12
dead end
The Knight’s Tour problem is
a Hamiltonian circuit problem
Prof. Amr Goneid, AUC
a
solution
50