Transcript Data Structures and Other Objects Using C++

1
Chapter 6. Branch and Bound

CHAPTER 6
Foundations of Algorithms
Chapter 6 introduces an algorithm
design technique called “Branch
and Bound”.
2
Branch and Bound

Similar to “Backtracking”
- a state-space tree is used to solve a problem

Different from “Backtracking”
- does not limit us to any particular way of
traversing a tree
- is used only for optimization problems
3
Branch and Bound

Step 1:
- computes a number (bound) at a node to determine
whether the node is promising
( the number is a bound on the value of the solution
that could be obtained by expanding beyond the
node )

Step 2:
- if the bound is no better than the value of the best
solution found so far, the node is non-promising.
4
6.1 The 0-1 Knapsack Problem
 Breadth-First Search with Branch and Bound
Pruning
$0
0
Example:
item 1 :
item 2 :
item 3 :
item 4 :
W = 16
pi
40
30
50
10
wi pi/wi
2
20
5
6
10
5
5
2
$0
0
$40
2
$70
7
$120
17
…
$80
12
$30
5
$40
2
$70
7
$90
12
$70
7
$100
17
$40
2
$90
12
$80
15
…
$30
5
$0
0
$50
10
$0
0
…
5
6.1 The 0-1 Knapsack Problem
 Breadth-First Search with Branch and Bound
Pruning
Nodeo
$0
0
$115
Example:
 Bound on Maximum Possible Profit:
Node1 : 40 + 30 + (50 * 9/10) = 115
 Queue: { Nodeo }
 Current best solution = 0
item 1 :
item 2 :
item 3 :
item 4 :
W = 16
pi
40
30
50
10
wi
2
5
10
5
pi/wi
20
6
5
2
6
6.1 The 0-1 Knapsack Problem
 Breadth-First Search with Branch and Bound
Pruning
Example:
$0
0
$115
40
2
115
Node1
0
0
82
Node2
 Bound on Maximum Possible Profit:
Node1 : 40 + 30 + (50 * 9/10) = 115
Node2 : 0 + 30 + 50 + (10*1/5) = 82
 Queue: { Node1 , Node2 }
 Current best solution = 40
item 1 :
item 2 :
item 3 :
item 4 :
W = 16
pi
40
30
50
10
wi
2
5
10
5
pi/wi
20
6
5
2
7
6.1 The 0-1 Knapsack Problem
 Breadth-First Search with Branch and Bound
Pruning
Example:
item 1 :
item 2 :
item 3 :
item 4 :
W = 16
$0
0
$115
0
0
82
40
2
115
70
7
115
Node3
40
2
98
30
5
82
Node4 Node5
0
0
60
Node6
pi
40
30
50
10
wi
2
5
10
5
pi/wi
20
6
5
2
 Bound on Maximum Possible Profit:
Node3 : 40 + 30 + (50 * 9/10) = 115
Node4 : 40 + 0 + 50 + (10*4/5) = 98
Node5 : 0 + 30 + 50+(10 * 1/5) = 82
Node6 : 0 + 0 + 50 + 10 = 60
 Queue: { Node3 , Node4 , Node5 }
 Current best solution = 70
8
6.1 The 0-1 Knapsack Problem
 Breadth-First Search with Branch and Bound
Pruning
Example:
$0
0
$115
0
0
82
40
2
115
40
2
98
70
7
115
120
17
0
Node7
70
7
80
90
12
98
item 1 :
item 2 :
item 3 :
item 4 :
W = 16
30
5
82
40
2
50
0
0
60
80
15
82
30
5
40
Node8 Node9 Node10 Node11 Node12
 Queue: { Node8 , Node9 }
 Current best solution = 90
pi
40
30
50
10
wi
2
5
10
5
pi/wi
20
6
5
2
 Bound on Maximum Possible Profit:
Node7 : 0 (overweight)
Node8 : 40 + 30 + 0 + 10 = 80
Node9 : 40 + 0 + 50+(10 * 4/5) = 98
Node10 : 40 + 0 + 0 + 10 = 50
Node11 : 0 + 30 + 50+(10 * 1/5) = 82
Node12 : 0 + 30 + 0 + 10 = 40
9
6.1 The 0-1 Knapsack Problem
 Breadth-First Search with Branch and Bound
Pruning
Example:
$0
0
$115
0
0
82
40
2
115
40
2
98
70
7
115
120
17
0
80
12
80
70
7
80
70
7
70
90
12
98
100
17
0
item 1 :
item 2 :
item 3 :
item 4 :
W = 16
30
5
82
40
2
50
0
0
60
80
15
82
90
12
90
Node13 Node14 Node15 Node16
30
5
40
pi
40
30
50
10
wi
2
5
10
5
pi/wi
20
6
5
2
 Bound on Maximum Possible Profit:
Node13 : 40 + 30 + 0 + 10 = 80
Node14 : 40 + 30 + 0 + 0 = 70
Node15 : 0 (overweight)
Node16 : 40 + 0 + 50 + 0 = 90
 Queue: { }
 Current best solution = 90
10
6.1 The 0-1 Knapsack Problem
 Breadth-First Search with Branch and Bound
public static int knapsack2(int n, int[ ] p, int[ ] w, int W)
{
queue_of_node Q ; node u, v ; int maxProfit ;
initialize(Q) ;
v.level = 0 ; v.profit = 0; v.weight=0 ;
maxProfit = 0 ;
enqueue(Q,v) ;
while(! Empty(Q) ){
dequeue(Q,v) ;
u.level = v.level + 1 ;
take care of the left child ;
take care of the right child ;
}
return maxProfit ;
}
public class node
{
int level ;
int profit ;
int weight ;
}
11
6.1 The 0-1 Knapsack Problem
 Breadth-First Search with Branch and Bound
Left
Child
Right
Child
u.weight = v.weight + w[u.level] ;
u.profit = v.profit + p[u.level] ;
if (u.weight<=W && u.profit > maxProfit)
maxProfit = u.profit ;
if (bound(u) > maxProfit)
enqueue(Q,u) ;
u.weight = v.weight ;
u.profit = v.profit ;
if (bound(u) > maxProfit)
enqueue(Q,u) ;
public class node
{
int level ;
int profit ;
int weight ;
}
12
6.1 The 0-1 Knapsack Problem
 Best-First Search with Branch and Bound Pruning

Basic Idea
- uses bound to select a node to expand next, rather than
just determine whether a node is promising
- uses a priority queue of nodes where the priority is
determined by the bound value of a node
13
6.1 The 0-1 Knapsack Problem
 Best-First Search with Branch and Bound Pruning
Nodeo
$0
0
$115
Example:
 Bound on Maximum Possible Profit:
Node0 : 40 + 30 + (50 * 9/10) = 115
 Queue: { Nodeo }
 Current best solution = 0
item 1 :
item 2 :
item 3 :
item 4 :
W = 16
pi
40
30
50
10
wi
2
5
10
5
pi/wi
20
6
5
2
14
6.1 The 0-1 Knapsack Problem
 Best-First Search with Branch and Bound Pruning
Example:
$0
0
$115
40
2
Node1 115
0
0
82
Node2
 Bound on Maximum Possible Profit:
Node1 : 40 + 30 + (50 * 9/10) = 115
Node2 : 0 + 30 + 50 + (10*1/5) = 82
 Queue: { Node1 , Node2 }
 Current best solution = 40
item 1 :
item 2 :
item 3 :
item 4 :
W = 16
pi
40
30
50
10
wi
2
5
10
5
pi/wi
20
6
5
2
15
6.1 The 0-1 Knapsack Problem
 Best-First Search with Branch and Bound Pruning
Example:
item 1 :
item 2 :
item 3 :
item 4 :
W = 16
$0
0
$115
40
2
115
70
7
115
Node3
40
2
98
Node4
0
0
82
Node2
pi
40
30
50
10
wi
2
5
10
5
pi/wi
20
6
5
2
 Bound on Maximum Possible Profit:
Node3 : 40 + 30 + (50 * 9/10) = 115
Node4 : 40 + 0 + 50 + (10*4/5) = 98
 Queue: { Node3 , Node4 , Node2 }
 Current best solution = 70
16
6.1 The 0-1 Knapsack Problem
 Best-First Search with Branch and Bound Pruning
Example:
$0
0
$115
0
0
82 Node
2
40
2
115
70
7
115
120
17
0
Node5
70
7
80
Node6
40
2
98
Node4
item 1 :
item 2 :
item 3 :
item 4 :
W = 16
pi
40
30
50
10
wi
2
5
10
5
pi/wi
20
6
5
2
 Bound on Maximum Possible Profit:
Node5 : 0 (overweight)
Node6 : 40 + 30 + 0 + 10 = 80
 Queue: { Node4 , Node6 , Node2 }
 Current best solution = 70
17
6.1 The 0-1 Knapsack Problem
 Best-First Search with Branch and Bound Pruning
Example:
$0
0
$115
0
0
82 Node
2
40
2
115
40
2
98
70
7
115
120
17
0
Node5
70
7
80
Node6
90
12
98
40
2
50
Node7 Node8
item 1 :
item 2 :
item 3 :
item 4 :
W = 16
pi
40
30
50
10
wi
2
5
10
5
pi/wi
20
6
5
2
 Bound on Maximum Possible Profit:
Node7 : 40 + 0 + 50 + (10 * 4/5) = 98
Node8 : 40 + 0 + 0 + 10 = 50
 Queue: { Node7 , Node2 , Node6 }
 Current best solution = 90
18
6.1 The 0-1 Knapsack Problem
 Best-First Search with Branch and Bound Pruning
Example:
$0
0
$115
0
0
82 Node
2
40
2
115
40
2
98
70
7
115
120
17
0
70
7
80
Node6
90
12
98
40
2
50
100
17
0
90
12
90
Node9
Node10
item 1 :
item 2 :
item 3 :
item 4 :
W = 16
pi
40
30
50
10
wi
2
5
10
5
pi/wi
20
6
5
2
 Bound on Maximum Possible Profit:
Node9 : 0 (overweight)
Node10 : 40 + 0 + 50 + 0 = 90
 Queue: { Node2 , Node6 }
 Current best solution = 90
19
6.1 The 0-1 Knapsack Problem
 Best-First Search with Branch and Bound Pruning
Example:
$0
0
$115
0
0
82 Node
2
40
2
115
40
2
98
70
7
115
120
17
0
70
7
80
Node6
90
12
98
100
17
0
40
2
50
90
12
90
item 1 :
item 2 :
item 3 :
item 4 :
W = 16
pi
40
30
50
10
wi
2
5
10
5
pi/wi
20
6
5
2
 Since both of Node2 and Node4 have
bound values less than 90, they will not be
expanded further.
 Queue: { }
 Final best solution = 90
20
6.1 The 0-1 Knapsack Problem
 Best-First Search with Branch and Bound
public static int knapsack3(int n, int[ ] p, int[ ] w, int W)
{
priority_queue_of_node PQ ; node u, v ;
int maxProfit ;
v.level = 0 ; v.profit = 0; v.weight=0 ; maxProfit = 0 ;
v.bound = bound(v) ;
PQ.enqueue(v) ;
while( ! PQ.Empty() ){
v = PQ.dequeue() ;
if (v.bound > maxProfit) {
u.level = v.level + 1 ;
take care of the left child ;
take care of the right child ;
}
}
}
public class node
{
int level ;
int profit ;
int weight ;
int bound ;
}
21
6.1 The 0-1 Knapsack Problem
 Best-First Search with Branch and Bound
Left
Child
Right
Child
u.weight = v.weight + w[u.level] ;
u.profit = v.profit + p[u.level] ;
if (u.weight<=W && u.profit > maxProfit)
maxProfit = u.profit ;
u.bound = bound(u) ;
if (u.bound > maxProfit)
PQ.enqueue(u) ;
u.weight = v.weight ;
u.bound = bound(u) ;
u.profit = v.profit ;
if ( u.bound > maxProfit)
PQ.enqueue(u) ;
public class node
{
int level ;
int profit ;
int weight ;
float bound ;
}
22
6.1 The 0-1 Knapsack Problem
 Best-First Search with Branch and Bound
public static float bound(node u)
{
index j,k ; int totWeight ; float result ;
if (u.weight >= W) return 0 ;
else {
result = u.profit ;
j = u.level + 1 ;
totWeight = u.weight ;
while (j<=n && totWeight+w[j] <= W){
totWeight = totWeight + w[j] ;
result = result + p[j] ;
j++ ;
}
k=j;
if (k <= n)
result=result+(W-totWeight)*p[k]/w[k];
return result ;
}
}
public class node
{
int level ;
int profit ;
int weight ;
float bound ;
}
6.2 The Traveling SalesPerson
Problem
 The Branch and Bound Approach to T.S.P.
Given a directed graph with n nodes, let [i1, i2, … , ik] be a path
from i1 to ik passing through i2, i3, …, and ik-1
[1]
[1,2]
[1,3]
[1,2,3] [1,2,4] [1,2,5]
…
[1,2,3,…,n-2]
[1,2,3,…,n-2,n-1]
…
[1,n]
…
…
[1,2,3,…,n-2,n]
…
23
6.2 The Traveling SalesPerson
Problem

24
The Branch and Bound Approach to T.S.P.
How to define how to compute the bound on each node?
- At the level k of the state space tree, each node corresponds
to a state where (k+1) vertices have been visited.


lower bound on the root node
=  (lowest weight of edge leaving vm)
vmV
V1
Vi2
Vi3
…
Vik
…
Vin
lowest weight edge
6.2 The Traveling SalesPerson
Problem

The Branch and Bound Approach to T.S.P.

lower bound on node [1, i2, ... , ik] ( 1 < k < n )
= sum of actual weight from V1 to Vik
+  (lowest weight of edge leaving Vm
vmA excluding those to vertices i2, ..,ik and
the edge from Vik to V1 )
where A = V – {V1,Vi2, ..,Vik-1}
V1
Vi2
…
Vik-1
A
Vik
…
Vik+1
Vin
25
6.2 The Traveling SalesPerson
Problem

The Best-First Search with Branch and Bound

Example:
W=
0
14
4
10
20
14
0
7
8
7
4
5
0
7
16
11
7
9
0
2
18
7
17
4
0
The start node is V1.
 the lower bound on the root node
=  (lowest weight of edge leaving vm) = 4 + 7 + 4 + 2 + 4 = 21
vmV
26
6.2 The Traveling SalesPerson
Problem

27
The Best-First Search with Branch and Bound
Nodeo
[1]
21
Example:
0
14
4
10
20
14
0
7
8
7
4
5
0
7
16
11
7
9
0
2
18
7
17
4
0
 Lower Bound on Minimum Cost Tour
Node0 : 4 + 7 + 4 + 2 + 4 = 21
 Queue: { Nodeo }
6.2 The Traveling SalesPerson
Problem

28
The Best-First Search with Branch and Bound
[1]
21
[1,2]
31
[1,3]
22
Example:
[1,4]
30
[1,5]
42
0
14
4
10
20
14
0
7
8
7
4
5
0
7
16
11
7
9
0
2
18
7
17
4
0
Node1 Node2 Node3 Node4
 Lower Bound on Minimum Cost Tour
Node1 : 14 + (7 + 4 + 2 + 4) = 31
Node2 : 4 + (7 + 5 + 2 + 4) = 22
Node3 : 10 + (7 + 4 + 2 + 7) = 30
Node4 : 20 + (7 + 4 + 7 + 4) = 42
 Queue: { Node2, Node3 , Node1 , Node4}
6.2 The Traveling SalesPerson
Problem

29
The Best-First Search with Branch and Bound
[1]
21
[1,2]
31
Node1
[1,3]
22
Example:
[1,4]
30
[1,5]
42
0
14
4
10
20
14
0
7
8
7
4
5
0
7
16
11
7
9
0
2
18
7
17
4
0
Node3 Node4
[1,3,2] [1,3,4] [1,3,5]
27
22
39
Node5 Node6 Node7
 Lower Bound on Minimum Cost Tour
Node5 : 4+5 + (7 + 2 + 4) = 22
Node6 : 4+7 + (7 + 5 + 4) = 27
Node7 : 4+16 + (8 +7 + 4) = 39
 Queue: { Node5, Node6 , Node3 , Node1, Node7 , Node4}
6.2 The Traveling SalesPerson
Problem

The Best-First Search with Branch and Bound
[1]
21
[1,2]
31
Node1
[1,3]
22
Example:
[1,4]
30
Node6 Node7
[1,3,2,4][1,3,2,5]
L = 31
Node8 Node9
[1,5]
42
0
14
4
10
20
14
0
7
8
7
4
5
0
7
16
11
7
9
0
2
18
7
17
4
0
Node3 Node4
[1,3,2] [1,3,4] [1,3,5]
27
22
39
L = 37
30
 Lower Bound on Minimum Cost Tour
Node8 : 4+5+8 + (2+18) = 37
Node9 : 4+5+7 + (4+11) = 31
 Queue: { Node6 , Node3 , Node1, Node7 , Node4}
 Current best solution = 31
6.2 The Traveling SalesPerson
Problem

The Best-First Search with Branch and Bound
[1]
21
[1,2]
31
[1,3]
22
Node1
Example:
[1,4]
30
Node7
[1,3,2,4][1,3,2,5] [1,3,4,2][1,3,4,5]
L = 31
[1,5]
42
0
14
4
10
20
14
0
7
8
7
4
5
0
7
16
11
7
9
0
2
18
7
17
4
0
Node3 Node4
[1,3,2] [1,3,4] [1,3,5]
27
22
39
L = 37
31
L = 43
L = 34
Node10 Node11
 Lower Bound on Minimum Cost Tour
Node10 : 4+7+7 + (7 + 18) = 43
Node11: 4+7+2 + (7 + 14) = 34
 Queue: { Node3 , Node1, Node7 , Node4}
 Current best solution = 31
6.2 The Traveling SalesPerson
Problem

32
The Best-First Search with Branch and Bound
[1]
21
[1,2]
31
Node1
…
[1,3]
22
Example:
[1,4]
30
[1,5]
42
0
14
4
10
20
14
0
7
8
7
4
5
0
7
16
11
7
9
0
2
18
7
17
4
0
Node4
[1,4,2] [1,4,3] [1,4,5]
38
45
30
Node12 Node13 Node14
 Lower Bound on Minimum Cost Tour
Node12 : 10+7+ (7 +4 + 17) = 45
Node13 : 10+9+ (7 +5 + 7) = 38
Node14 : 10+2+ (7 +4 + 7) = 30
 Queue: { Node14, Node1,Node7 , Node4}
 Current best solution = 31
6.2 The Traveling SalesPerson
Problem

33
The Best-First Search with Branch and Bound
[1]
21
[1,2]
31
[1,3]
22
Example:
[1,4]
30
Node1
…
[1,5]
42
0
14
4
10
20
14
0
7
8
7
4
5
0
7
16
11
7
9
0
2
18
7
17
4
0
Node4
[1,4,2] [1,4,3] [1,4,5]
38
45
30
[1,4,5,1][1,4,5,3]
L = 30
L = 48
Node15 Node16
 Lower Bound on Minimum Cost Tour
Node15 : 10+2+7 + (7 + 4) = 30
Node16 : 10+2+17 + (5 + 14) = 48
 Queue: { Node1, Node7 , Node4}
 Current best solution = 30
6.2 The Traveling SalesPerson
Problem

The Best-First Search with Branch and Bound
public static number travel2(int n, number[ ] W,
node optTour)
{
priority_queue_of_node PQ; node u, v ;
number minLength ;
PQ.initialize() ;
v.level = 0; v.path =[1]; minLength= ;
v.bound=bound(v);
PQ.enqueue(v) ;
while(! PQ.Empty() ) {
v = PQ.dequeue() ;
if v is promising // the bound of v < minLength
take_care_of_children ;
}
}
public class node
{
int level ;
ordered_set path;
number bound ;
}
34
6.2 The Traveling SalesPerson
Problem

The Best-First Search with Branch and Bound
u.level = v.level +1;
take_care_of_children for (all i such that 2 i n && i not in v.path) {
u.path = v.path ; put i at the end of u.path;
if ( u.level == n-2 ) {
put index of only vertex not in u.path at the end of u.path ;
put 1 at the end of u.path;
if (length(u) < minLength) {
minLength = length(u) ; optTour = u.path ;
}
}
else {
u.bound = bound(u) ;
if (u.bound < minLength )
PQ.enqueue(u) ;
}
35