The Traveling Salesman Problem in Theory & Practice

Lecture 5: Tour Construction Heuristics 18 February 2014 David S. Johnson [email protected]

http://davidsjohnson.net

Seeley Mudd 523, Tuesdays and Fridays

Outline Tour Construction Heuristics

• Definitions and Worst-Case Results • • NN, Greedy, Savings, Insertion Algorithms, etc.

Performance in Practice (Tour quality)

Nearest Neighbor (NN):

1. Start with some city.

2. Repeatedly go next to the nearest unvisited neighbor of the last city added.

3. When all cities have been added, go from the last back to the first.

Theorem

[

Rosenkrantz, Stearns, & Lewis, “An analysis of several heuristics for the traveling salesman problem,” SIAM J. Comput. 6 (1977), 563-581 ]: 1. There is a constant c , such that, for any instance I the ∆-inequality, NN(I) ≤ clog(N)  Opt(I) . obeying [Proved last week] 2. There exists a constant c’ , such that for all N > 3 , there are N -city instances I obeying the triangle inequality for which we have NN(I) > c’log(N)  Opt(I) .

[To be proved today] Definition: For any algorithm A case ratio for A is and instance size N , the worst Corollary: R N (A) = max{A(I)/OPT(I): I is an N -city instance } R N (NN) = Θ(log(N)) .

Lower Bound Examples

(Unspecified distances determined by shortest paths) F 1 : 1+ ε 1 1 NN path starts at left, ends in middle Number N 1 of vertices = 3 OPT(F 1 ) = 3+ε NN-path(F 1 ) = 2 2 2 F 2 : 1+ ε 1 1 1+ ε 1 1+ ε 1+ ε 1 1+ ε 1 1 N 2 = 2N 1 + 3 = 9 OPT(F 1 ) = N 2 +5ε = 9+5ε NN-path(F 2 ) = 10

F k-1 without shortcut between left and right endpoints 2 k-1 2 k-1 F k : 1+ ε 1 1+ε 1+ ε 1 Number N k OPT(F k ) = N k of vertices = 2*N k-1 + ½(N k + 3 + 1)ε (for k ≥ 2) NN-path(F k ) = 2NN-path(F k-1 )+2 k +2

General Formula for N

k

N

k

= 3



(2

k

-1)

Proof by Induction: Initialization: N 2 = 9 = 3  (2 2 Assume true for k’ < k . Then -1) N k = 2N k-1 + 3 = 2(3  (2 k-1 -1)) + 3 = 3  2 k – 6 +3 = 3  (2 k -1) Note for future reference: N k is always odd.

General Formula for Opt(F

k

) when k ≥ 2 Opt(F

k

) = N

k

+ ½(N

k

+ 1)ε

Proof: All edge lengths must be 1 or 1+ε , which means they must lie along the base of F k , where the edges start with 1+ ε and then alternate, and since N we will get one more 1+ε than 1 .

k is odd, this means that means Note that this means that, as ε → 0 = 3  (2 k -1) , and hence log(OPT(F k )) → for sufficiently large k we have and k → ∞ , OPT(F k ) → N k k+1 + log(3/2) , and hence k+1 > ½log(Opt(F k ))

General Formula for NN-path(F

k

) NN-path(F

k

) = (k+1)



2

k

– 2

Proof by Induction: Initialization: NN-path(F 2 ) = 10 = 3  2 2 Assume true for k’ < k . Then - 2 NN-path(F k ) = 2  NN-path(F k-1 ) + 2 k + 2 = 2(k  2 k-1 – 2) + 2 k + 2 = (k+1)  2 k – 2 .

Corollary: as ε → 0 and k → ∞ , NN-path(F k )/OPT(F k ) → (k+1)  2 k /(3  2 k ) > log(OPT(F k ))/6

F k-1 without shortcut between left and right endpoints 2 k-1 2 k-1 F k : 1+ ε 1 1+ ε 1+ ε 1 Number N k OPT(F k ) = N k of vertices = 2*N k-1 + ½(N k + 3 + 1)ε (for k ≥ 2) NN-path(F k ) = 2NN-path(F k-1 )+2 k +2 But are these two vertices really nearest neighbors?

F F k F k-1 2 k-1 2 k-1 1+ ε 1 1+ ε 1 k-1 1+ε 2 k-2 2 k-2 1 B 1+ ε 1 Is d(B,C) + 1+ ε > 2 k-1 C

Results for General k

2 k-1 2 k-1 F k A 1+ ε 1 B 1+ ε 1 Define: D 1 (F k ) = d(A,C), D 2 (F k ) = d(B,C) = d(A,B) C We shall prove by induction that D 1 (F k ) > 2 k+1 - 3 and D 2 (F k ) > 2 k - 1 Note: A shortest path from A to B need never go leftwards (by the triangle inequality)

Assuming Induction Hypothesis (for k’ < k): D 1 (F k’ ) > 2 k ’ +1 - 3 D 2 (F k’ ) > 2 k ’ – 1 There are two choices for the shortest A-C path in F k 2 k-1 2 k-1 A 1+ ε 2 k-1 1 B 1+ ε 2 k-1 1 C A 1+ ε 1+ ε 1 C Consequently, D 1 (F k ) = D 1 (F k-1 ) + 1+ε + 2 k-1 + D 2 (F k-1 ) Similarly, D 2 (F k ) = 2+ε + D 1 (F k-1 )

Basis for the induction: F

2 2 2 1 1 1 +ε 1 1+ ε 1 1+ ε 1+ ε D 1 (F 2 ) = 5 + ε D 2 (F 2 ) = 3 + ε 1 1+ D 1 (F k ) = D 1 (F k-1 ) + 1+ε + 2 k-1 + D 2 (F k-1 ) D 2 (F k ) = 2+  ε ε 1

k

D 1 (F k ) D 2 (F k )

2

5 + ε 3 + ε

3

13 + 3 ε 7 + ε

4

29 + 6 ε 15 + 4 ε

5

61 + 11 ε 31 + 7 ε

Let us ignore the precise values of the  terms

k

D 2 (F k ) D 2 (F k-1 ) + 1+  2 k-1

2

3+

3

7+ 4+ 4

4

15+ 8+ 8

5

31+ 16+ 16

k

(2 k -1)+ (2 k-1 )+ 2 k-1 So the correct NN choice is the 2 k-1 edge, as claimed.

NN Running Time

• •

To find the k

th

vertex, k > 1 , find the shortest distance among N-k-1 candidates.

Total time =

Θ

( ) =

Θ

(N

2

)

Greedy (Multi-Fragment) (GR):

1. Sort the edges, shortest first, and treat them in that order. Start with an empty graph.

2. While the current graph is not a TSP tour, attempt to add the next shortest edge. If it yields a vertex degree exceeding 2 or a tour of length less than N, delete.

• Theorem [ Ong & Moore, “Worst-case analysis of two travelling salesman heuristics,” Inform. Proc. Letters 2 (1984), 273-277 ]: – For all instances I obeying the  -Inequality, GR(I)/OPT(I) = O(logN) .

• Theorem [A. M. Frieze, Worst-case analysis of algorithms for travelling salesman problems,” Methods of Operations Research 32 (1979), 93-112.] : – There are N -city instances that obey the I N for arbitrarily large  -Inequality and have N GR(I N )/OPT(I N ) = Ω(logN/loglogN) .

Greedy Running Time

• • • Initial sort takes Θ (N 2 log(N)) .

Processing an edge takes constant time to check for degree 3, and two union-find operations to check for short cycles, for a total of at most Θ (N 2 α (N)) .* Total time = Θ (N 2 log(N)) .

*Note: The union-find operations can be avoided by storing with each vertex degree in the current forest (initially 0) and, if the degree is 1, the identity of the vertex v at the other end of the path containing it, which rules out edge {u,v} u its as a potential choice of next edge. This information is easily updated in constant time whenever we add an edge.

1.

2.

3.

Clarke-Wright “Savings” Heuristic

Start with a pseudo-tour in which an arbitrarily chosen city is the “hub” and the salesman returns to the hub after visiting each city (a multigraph in which every hub city is connected by two edges to the hub).

For each pair of non-hub cities, let the “savings” be the amount by which the pseudo-tour would be shortened if we added an edge between the two and deleted one edge to the hub from each.

As long as we do not yet have a tour, find a pair of non-hub cities that have not yet undergone two such shortcuts, and yields the most savings, and perform the shortcut for them. ∂ ∂ ∂ ∂ ∂ ∂ ∂ ∂ ∂ ∂ ∂ ∂ ∂ ∂ ∂ ∂ ∂ ∂ ∂ ∂ ∂

Clarke-Wright “Savings” Heuristic

• Theorem [A. M. Frieze, Worst-case analysis of algorithms for travelling salesman problems,” Methods of Operations Research 32 (1979), 93-112]: There are N -city instances I N and have for arbitrarily large N that obey the  -Inequality Savings(I N )/OPT(I N ) = Ω(logN/loglogN) .

No upper bounds are known under the triangle inequality.

We do have R N (A) = Θ(log(N)) which we for a sequential variant on the Savings heuristic, in 1.

2.

Pick an initial non-hub vertex v as the “current” vertex. Perform the best legal shortcut involving v has not yet been involved in a shortcut.

and some other non-hub vertex that 3.

4.

Declare the other non-hub vertex involved in the shortcut to be the new “current” vertex. Repeat until all non-hub vertices have been involved in a shortcut.

[Ong & Moore, 1984] [B. Golden, “Evaluating a sequential vehicle routing algorithm,” AIIE Transactions 9 (1977), 204-208]

Savings Heuristic Running Time

• • Mirrors Greedy: – Sort all (N-1)(N-2)/2 savings: Θ (N 2 log(N)) .

potential shortcuts by decreasing – Processing a shortcut takes constant time for involvement in more than two shortcuts, and two union-find operations to check for short cycles, for a total of at most Θ (N 2 α (N)) .* – Total time = Θ (N 2 log(N)) .

Sequential version mirrors NN analogously.

– Total time = Θ (N 2 ) .

*See Footnote for Greedy running time – same observations apply.

Nearest Addition (NA): 1. Start with 2-city tour consisting of some city and its nearest neighbor.

2. Repeatedly insert the non-tour city tour city v that yields the shorter tour).

u that is closest to a into one of the tour edges involving v (the one

Insertion Variants

Nearest Insertion (NI): 1. Start with 2-city tour consisting of some city and its nearest neighbor.

2. Repeatedly insert the non-tour city city v necessarily one involving v ).

u that is closest to a tour into the tour edge that yields the shortest new tour (not Cheapest Insertion (CI): 1. Start with 2-city tour consisting of some city and its nearest neighbor.

2. Repeatedly perform the insertion of a non-tour vertex w tour edge {u,v} that results in the shortest new tour. into a

More Insertion Variants

Farthest Insertion (FI): 1. Start with 2-city tour consisting of the two cities at maximum distance from each other. 2. Repeatedly insert the non-tour city new tour.

u , whose minimum distance to a tour city is maximum, into the tour edge that yields the shortest Arbitrary Insertion (AI): 1. Start with some 2-city tour.

2. Repeatedly pick some non-tour vertex u and insert it into the tour edge {u,v} that yields the shortest new tour.

Farthest Insertion

Theorem

[Rosenkrantz, Stearns, & Lewis, 1977] : If instance I obeys the triangle inequality, then 1.

2.

R N (AI) = O(log(N)) .

R N (NA) = R N (NI) = R N (CI) = 2.

The best lower bound known for R N (AI) is Ω(log(N)/loglog(N)) [Y. Azar, “Lower bounds for insertion methods for TSP,” Combinatorics, Probability & Computing 3 (1994) 285-292].

The best lower bound known for Proc. (1992), 346-352].

R N (FI) is R N (FI) ≥ 6.5

. [C. A. J. Hurkens, “Nasty TSP instances for farthest insertion,” IPCO

Upper Bound Proofs

• • The proof that R N (AI) = O(log(N)) mirrors that for NN.

For the R N (A) ≤ 2 straightforward: bounds, the results for NA and NI are – – Note that the vertices are added to the tour in the same order that they are added to the tree in Prim’s algorithm for computing an MST, and the optimal tour is at least as long as the MST.

Thus we need only show that the increase in tour length when a vertex u is added under NA is no more than twice the increase in tree length in Prim’s algorithm, or exceeds NA’s cost.

2d(u,v) , where v is u ’s nearest neighbor in the tour. The result for NI will follow since its cost at each step never w u d(u,w) ≤ d(u,v) + d(v,w) by  -inequality, so d(u,w) + d(u,v) – d(v,w) ≤ 2d(u,v) v

Upper Bound for CI

• • • • Basic plan: We will associate each insertion with a unique edge of the MST, with the cost of the insertion being no more than twice the cost of the edge.

Label the vertices v 1 , v 2 , …, v N in the order by which they are added to the tree under CI. Let T be the MST and let T i be the tree just before v i is added.

Say v j the unique path in T v k is compatible with v i is not in T i ).

if j < i between v j and if v k is an internal vertex on and v i , then k > i . (In other words, For each for v i i > 1 , the “critical vertex” for with largest index, and the ”critical edge” for v i edge on the path to v i v i is the compatible vertex from its critical vertex.

is the first

Critical Vertices and Edges

1 7 8 3 4 2 9 10 6 5 Note: In this example, every edge is critical for precisely one vertex.

We can prove that this will always be the case.

Unique Criticality

Suppose an edge {v i ,v j } , i < j , is critical for two distinct vertices, v h and v k , with h < k .

v k v h v j v i Since v i is the critical vertex for v h , we have that h > i internal vertices in the path from v i since that would imply j < k and so v i to v h must exceed and that v j h and all the . (We cannot have would not be compatible with v k .) h = j Since {v i ,v j } is critical for v k , all the internal vertices of the path from must exceed k > h . (In the case when k = j v i to v k there are no internal vertices.) Hence, all the internal vertices on the (blue) path from indices exceeding h , and so v h to v k must have is compatible with v k , with index h > i , implying that v i is NOT the critical vertex for v k contradiction.

and {v i ,v j } v h is not the critical edge, a

Completing the CI proof

• • • We first show that the cost of inserting vertex i j } , i < j , for v k , v k .

is no more than twice the Let the critical edge be {v i ,v j } , i < j . By definition of critical edge we must have j ≥ k .

Thus, at the time v k is added, v i was in the tour but v j was not.

Hence d(v i, v j ) must be at least as large as the length of the shortest edge joining a tour vertex u to a non-tour vertex v .

• By our NA argument, the insertion of v into a tour edge incident on u will cost at most 2d(u,v) ≤ 2d(v i, v j ) . The cheapest insertion can cost no more.

Summing over all inserted vertices we get that the total tour length for CI is at most two times the sum of the lengths of all the critical edges, which by the uniqueness lemma is simply the length of the MST.

QED

Lower Bound Examples

2 ε 2 ε 2 ε 2 ε Adjacent points have distance 1, all other distances are the distance along the line minus ε.

2 ε 2 ε 2 ε NA(I) = NI(I) = CI(I) = 2N-2-(N-2) ε 2 ε OPT(I) = N+1 ε

More Lower Bounds: Double MST and Christofides

Recall: Start by computing a minimum spanning tree [O(N 2 ) time] Then add edges so that every vertex has even degree.

Double MST Double the edges of the spanning tree [O(N) time] Christofides Add minimum-weight matching on odd-degree vertices [O(N 3 ) time] Find an Euler tour for the resulting graph [O(N) time] Traverse the Euler tour, taking shortcuts to avoid revisiting vertices [O(N) time]

More Lower Bounds: Double MST Algorithm

MST Length = n + (n+1)(1-ε) + 2ε = 2n + 1 – (n-1)ε DoubleMST Tour Length ∼ 2n + (2n)(1-ε) + 2ε = 4n – 2(n-1)ε OptimalTour Length ∼ 2n + 2

More Lower Bounds: Christofides Algorithm

1 N cities on bottom, N+1 on top.

Distance to nearest city in same row = 1 .

Distance to nearest city on other row = 1 – ε’ .

OPT: Length = 2N + 1 - 2ε’ MST: Length = 2N(1 – ε’) Christofides: Length = 2N(1 – ε’) + N = 3N – 2Nε’

Subquadratic Algorithms for Euclidean Instances:

Strip

1. Let R be a minimum rectangle containing all the cities.

2. Partition R into floor(sqrt(N)/3) vertical strips.

3. Sort the cities within each strip by y -coordinate.

4. Starting with the bottom-most point in the leftmost strip, traverse the cities up one strip and down the next until all have been visited, and then return to the starting point.

Total Running time O(Nlog(N)) .

∂ ∂ ∂ ∂ ∂ ∂ ∂ ∂ ∂ ∂ ∂ ∂ ∂ ∂ ∂ ∂ ∂ ∂ ∂ ∂ ∂ ∂ ∂ ∂ ∂ ∂ ∂ ∂ ∂ ∂ ∂ ∂ ∂ ∂ ∂ ∂ ∂ ∂ ∂ ∂ ∂

∂ ∂ ∂ ∂ ∂ ∂ ∂ ∂ ∂ ∂ ∂ ∂ ∂ ∂ ∂ ∂ ∂ ∂ ∂ ∂ ∂ ∂ ∂ ∂ ∂ ∂ ∂ ∂ ∂ ∂ ∂ ∂ ∂ ∂ ∂ ∂ ∂ ∂ ∂ ∂ ∂ ∂ ∂ ∂ ∂ ∂ ∂ ∂ ∂ ∂ ∂ ∂ ∂ ∂ ∂ ∂ ∂ ∂ ∂ ∂ ∂ ∂ ∂ ∂ ∂ ∂ Assuming neighboring points are one unit apart, OPT = N Strip(I N ) > sqrt(N)/3)(N/4) = Ω (sqrt(N)  OPT) ∂ ∂ ∂ ∂ ∂ ∂ ∂ ∂ ∂ ∂ ∂ ∂ ∂ ∂ ∂ ∂ ∂ ∂ ∂ ∂ ∂ ∂ ∂ ∂ ∂ ∂ ∂ ∂ ∂ ∂ ∂ ∂ ∂ ∂ ∂ ∂ ∂ ∂ ∂ ∂ ∂ ∂ ∂ ∂ ∂ ∂ ∂ ∂ ∂ ∂ ∂ ∂ ∂ ∂ ∂ ∂ ∂ ∂ ∂ ∂ ∂ ∂ ∂ ∂ ∂ ∂ ∂ ∂ ∂

Subquadratic Algorithms for Euclidean Instances:

Spacefilling Curve

Visit the cities in the order that they occur along a spacefilling curve for a square that contains them all.

Wikipedia Entry for Spacefilling Curve Running time O(Nlog(N)) .

For details, see [Platzmann & Bartholdi, “Spacefilling curves and the planar travelling salesman problem,” J. ACM 36 (1989), 719-737].

More Geometric Tour Construction Heuristics

• • • • • Insertion variants for geometric instances – – – Cheapest Insertion into the convex hull (CHCI) “Greatest Angle” insertion into the convex hull (CHGA) Convex Hull, Cheapest Insertion with greatest angle (CCA) Double Strip (best of both horizontal and vertical strip tours) (DST) Karp’s Partitioning Algorithm (KP) Litke’s Clustering Algorithm Bentley’s Fast Recursive Partitioning Heuristic (FRP)

• • • •

Performance

“

In Practice

”

Data Sources

Johnson, Bentley, McGeoch, & Rothberg, “ Near-Optimal Solutions to Very Large Traveling Salesman Problems ,” unpublished (and as-yet uncompleted) monograph (1994).

Johnson & McGeoch, “ The traveling salesman problem: A case study in local optimization ,” chapter in Local Search in Combinatorial Optimization, Aarts & Lenstra (editors), Princeton University Press, Princeton, NJ, 2003, 215-310 [Also available on DSJ’s website].

Johnson & McGeoch, “ Experimental analysis of heuristics for the STSP ,” chapter in The Traveling Salesman Problem and its Variations, Gutin & Punnen (editors), Kluwer Academic Publishers, Dordrecht, 2002, 369-443 [Also available on DSJ’s website].

Website for the “ since 2002].

8 th DIMACS Implementation Challenge: The Traveling Salesman Problem ” [http://dimacs.rutgers.edu/Challenges/TSP – only minor updates

Performance

“

In Practice

” •

Testbed 1: Random Euclidean Instances

N = 10,000 (Results appear to be reasonably well-correlated with those for our real-world instances.)

Performance

“

In Practice

” •

Testbed 2: Random Clustered Instances

Choose N/100 centers uniformly, then generate 100 normally-distributed points around each.

Performance

“

In Practice

” •

Testbed 3: Instances from TSPLIB

Printed Circuit Boards Geography Laser Logic

Performance

“

In Practice

” •

Testbed 4:

Random Symmetric Distance Matrices (Unlikely to obey Triangle Inequality.) Let’s start with Random Euclidean and just a few algorithms…

Nearest Neighbor

Greedy

Percent Excess over Held-Karp bound Smart-Shortcut Christofides

Standard Shortcuts

Smart Shortcuts

Random Euclidean Performance for N = 1,000,000

(% Excess over HK Bound)

Algorithm

Smart-Shortcut Christofides Savings Farthest Insertion Greedy Classic-Shortcut Christofides Random Insertion Convex Hull, Cheapest Insertion Cheapest Insertion Nearest Neighbor Nearest Insertion Strip Nearest Addition Spacefilling Curve Double MST w. Smart Shortcuts

%

9.8

12.2

13.5

14.2

14.5

15.2

22.0

22.1

23.3

27.0

30.2

32.6

35.1

39.9

Solution Quality Distributions

Random Euclidean N = 10,000

Smart-Shortcut Christofides

Savings

Farthest Insertion

Greedy

Savings versus Classic Christofides

Nearest Neighbor versus Nearest Insertion

Greedy versus Nearest Insertion

Savings versus Nearest Insertion

Conclusion: Many of the heuristics that are famous theoretically are totally dominated in practice.

Question: Why care about these dominated heuristics?

Tour Quality versus Running Time Tradeoffs

Random Euclidean N = 1,000