Transcript TABU Search (TS)
Nuno Abreu, Zafeiris Kokkinogenis, Behdad Bozorg, Muhammad Ajmal 1
OUTLINE
Introduction to TS Parameters of TS Basic concepts of TS TS vs. other Meta-heuristics Memory in TS Use of Memory in TS Tabus Aspiration criteria Stop condition TS basic algorithm Search Process – 1 Search Process – 2 Flowchart of a Standard TS Algorithm Example Pros and Cons 2
INTRODUCTION TO TS
TS is an iterative procedure designed for the solution of optimization problems Invented by Glover (1986) Uses a neighborhood search procedure to iteratively move from a solution x to a solution x* in the neighborhood of x Uses memory structures so that the algorithm does not visit a given solution repeatedly Tabu Search Benefits Cycle avoidance which also saves time Guide search to promising regions of the search space 3
PARAMETERS OF TABU SEARCH
Space search procedure Neighborhood structure Short-term memory: Tabu list Types of moves Addition of a Tabu move Maximum size of Tabu list Aspiration conditions Stopping rule 4
BASIC CONCEPTS OF TABU SEARCH
Saves information
according to the exploration process It will be used to
limit the moves
through the neighborhood Structure of the neighborhood of the solutions varies from iteration to iteration Infeasible solutions can be accepted and evaluated to
escape local minimum
.
To prevent from cycling,
recent moves are forbidden
.
A
tabu list
records forbidden moves, which are referred to as
tabu moves
A tabu move can be accepted using
aspiration criteria
.
Allows
exploitation
of good solution and
exploration
unvisited region of the search space of 5
TS VS OTHER META-HEURISTICS
Traditional descent methods cannot allow
non-improving moves
, TS can.
SA and GA does not have
memory
, TS has.
SA uses
randomness forbidden moves
.
to escape local minimum, TS uses TS claims that a bad strategic choice can yield more information than a random choice.
6
MEMORY IN TS
TS uses mainly two types of memory:
Short-term
memory Recent solutions Structure were tabu moves are stored Avoids cycling
Long-term
memory (frequency-based) Number of iterations that “solution
components
” have been present in the current solution 7
USE OF MEMORY IN TS
Use of memory leads to
learning
Prevent the search from
repeating
moves
Explore
the unvisited area of the solution space By using memory to avoid certain moves, TS can be seen as
global optimizer
rather than local.
8
TABUS
Tabus are one of the distinctive elements of TS when compared to LS Prevent cycling when moving away from local minimum through non-improving moves Stored in the
short-term memory –
Tabu list
Tabu tenure
is the number of iteration the move is tabu Tabu list can be of fixed-length or dynamically varying 9
ASPIRATION
If a move which is tabu
can lead to better solution
, the tabu status should be
overruled
This will be performed using
aspiration
level conditions
Aspiration criteria
: accepting an improving solution even if generated by a tabu move A tabu move becomes admissible if it yields a solution that is better than an aspiration value A tabu move becomes admissible if the direction of the search (improving or non-improving) does not change 10
BASIC TABU SEARCH ALGORITHM
Step 1: Choose an
initial
solution i in S. Set i*=i and k=0.
Step 2: Set k=k+1 and generate a subset V of solutions in N(i,k) such that the Tabu conditions are not violated or the aspiration conditions hold.
Step 3: Choose a best j in V and set i=j.
Step 4: If f(i) < f(i*) then set i* = i.
Step 5: Update Tabu and aspiration conditions.
Step 6: If a stopping condition is met then stop. Else go to Step 2.
11
STOPPING CONDITIONS
N(i, K+1) = 0 (
no feasible solution
in the neighborhood of solution i) K is larger than the
maximum
number of iterations allowed The number of iterations since the last improvement of i* is larger than a specified number Evidence can be given than an
optimum solution
has been obtained 12
SEARCH PROCESS - 1
Intensification
: searches solutions
similar
wants to space
intensively
to the current solution explore known promising areas of the search creates solutions using the more attractive components solutions in memory (recency memory) of the best Another technique consists in changing the neighborhood’s structure by allowing different moves
Diversification
: Examines
unvisited
regions of the search space Generates different solutions Using rarely components present in the current solution Biasing the evaluation of a move by modifying the objective function adding a term related to component frequencies Intensification and diversification phases
alternate
search during the 13
SEARCH PROCESS - 2
Allowing infeasible solution
Constraints defining the search space can lead the process to mediocre solutions Induces diversification By dropping some constraints (
relaxation
) a larger space can be explored Penalize objective for the violation A well known technique:
Strategic oscillation Surrogate objective
Evaluates neighbors using a simpler function than the objective in order to spot promising candidates. (Intensification)
Auxiliary objective
Objective function can’t drive the search to more interesting areas Orient the search by measuring desirable attributes of solutions 14
FLOWCHART OF A STANDARD TABU SEARCH ALGORITHM Heuristic procedure Tabu restrictions Candidate lists Aspiration criteria Elite solutions Modified choice rules for diversification or intensification Generate initial solution and initialize memory structures Construct modified neighborhood Yes Select best neighbor Stop No More iteration?
Update memory structures Short and long term memory Restarting Strategic oscillation Execute specializes procedures Update best solution 15
EXAMPLE
• • Minimum spanning tree problem with constraints.
Objective: Connects all nodes with minimum costs
Costs A B 20 15 10 25 C D 5 40 30 E A B 20 15 10 C 25 D 5 40 30 E An optimal solution without considering constraints
Constraints 1: Link AD can be included only if link DE also is included. (penalty:100) Constraints 2: At most one of the three links – AD, CD, and AB – can be included.
(Penalty of 100 if selected two of the three, 200 if all three are selected.)
16
EXAMPLE
Iteration 1 Cost=50+200 (constraint penalties) New cost = 75 (iteration 2) ( local optimum) B 20 30 A Delete 15 10 C 25 D 5 E 40 Add Add BE BE BE CD CD DE DE DE Delete CE AC AB AD AC CE AC AD Cost 75+200=275 70+200=270 60+100=160 60+100=160 65+300=365 85+100=185 80+100=180
75+0=75
Constraints 1: Link AD can be included only if link DE also is included. (penalty:100) Constraints 2: At most one of the three links – AD, CD, and AB – can be included.
(Penalty of 100 if selected two of the three, 200 if all three are selected.)
17
EXAMPLE
Tabu list: DE Iteration 2 Cost= 75
* A tabu move will be considered only if it would result in a better solution than the best trial solution found previously (Aspiration Condition)
Iteration 3 new cost = 85 Delete B 20 30 Escape local optimum Add A 10 C 5 E Add AD AD AD Delete DE* CE AC Cost Tabu move 85+100=185 80+100=180 15 25 D 40 Tabu BE BE BE CE AC AB 100+0=100 95+0=95
85+0=85
CD CD DE* CE 60+100=160 95+100=195
Constraints 1: Link AD can be included only if link DE also is included. (penalty:100) Constraints 2: At most one of the three links – AD, CD, and AB – can be included.
(Penalty of 100 if selected two of the three, 200 if all three are selected.)
18
EXAMPLE
Tabu list: DE & BE Iteration 3 Cost= 85
* A tabu move will be considered only if it would result in a better solution than the best trial solution found previously (Aspiration Condition)
Iteration 4 new cost =
70
Override tabu status Add Delete Cost 20 B 30 Tabu AB AB AB BE* CE AC Tabu move 100+0=100 95+0=95 10 A C 5 E 15 25 D Add 40 Tabu Delete AD AD AD DE* CE AC 60+100=160 95+0=95 90+0=90 CD CD DE* CE
70+0=70
105+0=105
Constraints 1: Link AD can be included only if link DE also is included. (penalty:100) Constraints 2: At most one of the three links – AD, CD, and AB – can be included.
(Penalty of 100 if selected two of the three, 200 if all three are selected.)
19
EXAMPLE
Optimal Solution Cost = 70 Additional iterations only find inferior solutions A B 20 15 10 25 C D 5 40 30 E 20
PROS AND CONS
Pros: Allows non-improving solution to be accepted in order to
escape from a local optimum
The use of
Tabu list
For larger and more difficult problems tabu search can beat other approaches Cons:
Too many parameters
to be determined Number of iterations could be very large Global optimum may not be found, depends on parameter settings
Too complex
21
REFERENCES
Glover, F., Kelly, J. P., and Laguna, M. 1995. Genetic Algorithms and Tabu Search: Hybrids for Optimization. Computers and Operations Research. Vol. 22, No. 1, pp. 111 – 134.
Glover, F. and Laguna, M. 1997. Tabu Search. Norwell, MA: Kluwer Academic Publishers.
Hanafi, S. 2001. On the Convergence of Tabu Search. Journal of Heuristics. Vol. 7, pp. 47 – 58.
Hertz, A., Taillard, E. and Werra, D. A Tutorial on Tabu Search. Accessed on April 14, 2005: http://www.cs.colostate.edu/~whitley/CS640/hertz92tutorial.pdf
Gendreau, M. 2002. An Introduction To Tabu Search, Centre de Recherche sur les Transports et Département d´informatique et de Recherche opérationnelle, Université de Montréal. Hillier, F.S. and Lieberman, G.J. 2005. Introduction to Operations Research. New York, NY: McGraw-Hill. 8th Ed.
22
23
OUTLINE
Introduction Tabu Search Basic Approach – 1 Basic Approach – 2 The TS framework for multi-dimensional bin packing Search methods Computational test - Dataset Computational test Sample of results Conclusions 24
INTRODUCTION
The paper deals with
multi-dimensional
algorithms cutting and packing Assumes that the item will be packed with
fixed orientation
(no rotation) Lodi, Martello and Vigo present an effective
BP Tabu Search framework
The main characteristic is the adoption of a search scheme and a neighborhood independent of the packing problem to be solved 25
TABU SEARCH
The goal is
empty
a specified target bin Given a current solution, the moves modify it by changing the packing of a set S of items The target bin is the one
minimizing
a filling function φ(.) φ(.) measures the
easiness of emptying the bin
The idea is to favor target bins packing a
small area
and a relatively
large
number of items 26
BASIC APPROACH - 1
Select
target
bin Include
one item
, j, from the
target
other bins in a set S bin, and contents of
k
Execute an greedy-type heuristic A on S The size of the neighborhood
k
is automatically updated If the items of S are packed into k (or less) bins, item j is
removed
selecting a different set of k bins, or a different item from the target bin from the target bin. Otherwise, S is changed by Then, a new item is selected, a new set S is defined and a new move is performed The execution is
halted
as soon as a proven optimal solution is found, or a time limit is reached 27
BASIC APPROACH - 2
If the algorithm gets stuck The target bin is not emptied, the neighborhood is enlarged by increasing the value of up to a prefixed upper limit Tabu list stores,for each forbidden move, the sum of the filling function values of the k+1 involved bins Variable Neighborhood Search strategy Small values of k = Small neighborhoods, fast to explore Accepting moves dealing with increased k = Enlarged neighborhood, more chances to improve solution By changing the size of the neighborhood, the algorithm plays “Intensification” and “Diversification” The execution is
halted
as soon as a proven optimal solution is found, or a time limit is reached 28
THE TS FRAMEWORK FOR MULTI-DIMENSIONAL BIN PACKING 29
SEARCH METHODS
Intensification
procedure explores the neighborhood of the current solution (Inner loop) Two types of
diversification
are defined: “
soft
” diversification - select as target bin the one having the second smallest filling function value “
hard
” diversification - re-pack into separate bins the items currently packed in the z/2 bins ( z being the number of bins in the current solution) with smallest filling function value The
tabu list
stores, for each forbidden move: the sum of the
filling function values
the
last moves
30
COMPUTATIONAL TESTS - DATASET
The benchmark consists of 500 random instances with n = {20,40,60,80,100} Ten different classes of instances were used wi (width), hi (height) generated from uniform distribution of varying intervals 31
COMPUTATIONAL TESTS
Measures the
improvement
to three
greedy-type
heuristics: brought by TS when comparing Hybrid Best-Fit algorithm (HBP) Knapsack Packing (KP) Alternate Directions (AD) Results show that the TS allows a
good improvement
in the quality of the solution obtained by the greedy-type heuristics 32
SAMPLE
OF RESULTS
33
CONCLUSIONS
The main idea of the framework:
Isolate the information
concerning the problem Let a
greedy-type feasible solutions
heuristic take care of the structure and construct Tabu Search is then used to
drive the search
through the solution space by:
re-combining
the packed items,
exploring a neighborhood
by alternating between “
intensification
” and “
diversification
” This results in a very general algorithm for
multidimensional bin packing
34