Ant Colony Optimization: the Metaheuristic. Application Examples. Connection between ACO and other Metaheuristics. Encodings. Capacited Vehicle Routing.
Outline DM63 HEURISTICS FOR COMBINATORIAL OPTIMIZATION Ant Colony Optimization: the Metaheuristic Application Examples Connection between ACO and other Metaheuristics Encodings Capacited Vehicle Routing
Lecture 11
Ant Colony Optimization Exercises Marco Chiarandini
DM63 – Heuristics for Combinatorial Optimization Problems
Ant Colony Optimization Metaheuristic I
I
I
Note:
Population-based method in which artificial ants iteratively construct candidate solutions. Solution construction is probabilistically biased by pheromone trail information, heuristic information and partial candidate solution of each ant (memory). Pheromone trails are modified during the search process to reflect collective experience.
Ant Colony Optimization (ACO): initialize pheromone trails While termination criterion is not satisfied: | generate population sp of candidate solutions using subsidiary randomized constructive search | | perform subsidiary perturbative search on sp | |b update pheromone trails based on sp DM63 – Heuristics for Combinatorial Optimization Problems
3
I
In each cycle, each ant creates one candidate solution using a constructive search procedure.
I
Ants build solutions by performing randomized walks on a construction graph G = (V, E) where V are solution components and G is fully connected.
I
All pheromone trails are initialized to the same value, τ0 .
I
Pheromone update typically comprises uniform decrease of all trail levels (evaporation) and increase of some trail levels based on candidate solutions obtained from construction + perturbative search.
I
Subsidiary perturbative search is (often) applied to individual candidate solutions.
I
Termination criterion can include conditions on make-up of current population, e.g., variation in solution quality or distance between individual candidate solutions.
2
Example: A simple ACO algorithm for the TSP (2)
Example: A simple ACO algorithm for the TSP (1) I
Search space and solution set as usual (all Hamiltonian cycles in given graph G).
I
Associate pheromone trails τij with each edge (i, j) in G.
I
Use heuristic values ηij :=
I
Initialize all weights to a small value τ0 (τ0 = 1).
I
Constructive search: Each ant starts with randomly chosen vertex and iteratively extends partial round trip πk by selecting vertex not contained in πk with probability
1 cij
I
Subsidiary perturbative search: Perform iterative improvement based on standard 2-exchange neighborhood on each candidate solution in population (until local minimum is reached).
I
Update pheromone trail levels according to X τij := (1 − ρ) · τij + ∆ij (s) s∈sp 0
where ∆ij (s) := 1/Cs if edge (i, j) is contained in the cycle represented by s 0 , and 0 otherwise. Motivation: Edges belonging to highest-quality candidate solutions and/or that have been used by many ants should be preferably used in subsequent constructions.
[τij ]α · [ηij ]β pij = P [τil ]α · [ηil ]β l∈Nik I
α and β are parameters. DM63 – Heuristics for Combinatorial Optimization Problems
Termination: After fixed number of cycles (= construction + perturbative search phases).
5
ACO Variants
Ant System on TSP I
I I
Ant System AS (Dorigo et al., 1991) Elitist AS (EAS)(Dorigo et al., 1991; 1996) I
I
I
The iteration best solution adds more pheromone I
Only best ranked ants can add pheromone Pheromone added is proportional to rank
I
Max-Min AS (MMAS)(St¨ utzle & Hoos, 1997)
I
Ant Colony System (ACS) (Gambardella & Dorigo, 1996; Dorigo & Gambardella, 1997)
I
Approximate Nondeterministic Tree Search ANTS (Maniezzo 1999) Hypercube AS (Blum, Roli and Dorigo, 2001)
I
m CNN Motivation: sligthly more than what evaporates Construction: m ants in m randomly chosen cities τij = τo =
Rank-Based AS (ASrank)(Bullnheimer et al., 1997; 1999) I
Initialization:
[τij ]α · [ηij ]β pij = P , [τil ]α · [ηil ]β
α and β parameters
l∈Nik I
Update τij ← (1 − ρ) · τij m X τij ← τij + ∆kij
to all the edges to the edges visited by the ants, ∆kij =
1 Ck
k=1
DM63 – Heuristics for Combinatorial Optimization Problems
7
DM63 – Heuristics for Combinatorial Optimization Problems
8
Elitist Ant System on TSP
Rank-based Ant System on TSP
I I
Update τij ← (1 − ρ) · τij m X ∆kij + e · ∆bs τij ← τij + ij
Update: only w − 1 best ranked ants + the best-so-far solution deposit pheromone: τij ← (1 − ρ) · τij
to all the edges τij ← τij +
to the edges visited by the ants
k=1
w−1 X
∆kij + w · ∆bs ij
to all the edges to the edges visited by the ants
k=1
∆bs ij
=
1 Cbs
0
(ij) in tour k, bs best-so-far otherwise
∆bs ij
DM63 – Heuristics for Combinatorial Optimization Problems
9
DM63 – Heuristics for Combinatorial Optimization Problems
MAX − MIN Ant System (MMAS)
Three main ideas: I Different state transition rule β arg maxl∈Nki {τil ηil } α β ij ] j = pij = P[τij ] ·[η α [τil ] ·[ηil ]β k
Update τij ← (1 − ρ) · τij τij ← τij + ∆bs ij
10
Ant Colony System (ACS) on the TSP
Peculiarities in pheromone management: I
to all the edges only to the edges visited by the best ant
l∈N
Meaning of best alternates during the search between: I I
I I I
I
best-so-far iteration best
I
Reinitialization of τ if: I
stagnation occurs idle iterations
otherwise
i
Global pheromone update
to only (ij) in best-so-far tour (O(n) complexity) Local pheromone update: happens during tour construction to avoid other ants to make the same choices: τij ← (1 − ) · τij + τ0
Results obtained are better than AS, EAS, and ASrank, and of similar quality to ACS’s DM63 – Heuristics for Combinatorial Optimization Problems
if q ≤ q0
τij ← (1 − ρ) · τij + ρ∆bs ij (s)
bounded values τmin and τmax τmax 1 τmax = ρC ∗ and τmin = a I
1 Ck 1 = bs C
∆kij =
11
= 0.1, τ0 =
1 nCNN
Parallel construction preferred to sequential construction DM63 – Heuristics for Combinatorial Optimization Problems
12
Approximate Nondeterministic Tree Search on TSP
Strongly Invariant ACO on TSP Considers instances which are equivalent up to a linear transformation of units.
I
Use of lower bound to compute heuristic value I
I
The siACO is an algorithm that enjoy the property of that its internal state at each iteration is the same on equivalent instances.
Add an arc to the current partial solution and estimate LB of complete solution
For AS:
Different solution construction rule ατij + (1 − α)ηij pkij = P ατil + (1 − α)ηil
I
Use heuristic values ηij :=
I
Update according to
l∈Nk i I
Different pheromone trail update rule k X θ(1 − τij ← τij + ∆kij ∆kij = i=1 0
CNN n·cij
τij := (1 − ρ) · τij +
X
∆ij (s)
s∈sp 0 Ck −LB Lavg −LB
if (ij) in belongs to T k otherwise
NN
C where ∆ij (s) := m·C s if edge (i, j) is contained in the cycle represented by s 0 , and 0 otherwise.
Can be extended to other ACO versions and to other problems: QAP and Scheduling DM63 – Heuristics for Combinatorial Optimization Problems
13
Analysis
DM63 – Heuristics for Combinatorial Optimization Problems
14
Pheromone development
Other things to check I
Synergy
I
Pheromone Development
I
Strength of local search (exploitation vs exploration)
I
Heuristic Information (linked to parameter β) Results show that with β = 0 local search can still be enough
I
Lamarkian vs Darwinian Pheromone Updates
I
Run Time impact
DM63 – Heuristics for Combinatorial Optimization Problems
16
Analysis
Parameter tuning
Number of tours generated to find the optimal solution as a function of the number m of ants used DM63 – Heuristics for Combinatorial Optimization Problems
17
DM63 – Heuristics for Combinatorial Optimization Problems
18
ACO: Theoretical results
Ant Colony Optimization . . . I
A framework and several problem applications are available from: http://www.aco-metaheuristic.org/
I
good performance also in dynamic optimization problems, such as routing in telecommunications networks
I
Gutjahr (Future Generation Computer Systems, 2000; Information Processing Letters, 2002) and St¨ utzle and Dorigo (IEEE Trans. on Evolutionary Computation, 2002) have proved convergence with prob 1 to the optimal solution of different versions of ACO
I
Meuleau and Dorigo (Artificial Life Journal, 2002) have shown that there are strong relations between ACO and stochastic gradient descent in the space of pheromone trails, which converges to a local optima with prob 1
I
Birattari et al. (TR, 2000; ANTS 2002) have shown the tight relationship between ACO and dynamic programming
I
Zlochin et al. (TR, 2001) have shown the tight relationship between ACO and estimation of distribution algorithms
For further details on Ant Colony Optimization, see the book by Dorigo and St¨ utzle [2004].
DM63 – Heuristics for Combinatorial Optimization Problems
19
DM63 – Heuristics for Combinatorial Optimization Problems
20
Application Examples
Generalized Assignment Problem (GAP)
Input:
Linear permutations problems: SMTWTP Construction graph: Fully connected and the set of vertices consists of the n jobs and the n positions to which the jobs are assigned.
Pheromone Trails: τij expresses the desirability of assigning job i in position j (cumulative rule) 1 hi
where hi is a dispatching rule.
DM63 – Heuristics for Combinatorial Optimization Problems
I
the amount bi of resource available to agent i
Task: Find an assignment of jobs to agents σ : J → I such that: P min f(σ) = cσ(j)j P j∈J aij ≤ bi ∀i ∈ I s.t.
Constraints: all jobs have to be scheduled.
Heuristic information: ηij =
I
a set of jobs J = {1, . . . , n} and a set of agents I = {1, . . . , m}. the cost cij and the resource requirement aij of a job j assigned to agent i
I
j∈J,σ(j)=i
21
DM63 – Heuristics for Combinatorial Optimization Problems
22
Application Examples (2)
Pheromone:
Assignment problems: Generalized Assignment Problem
Two choices: I which job to consider next
Construction Graph: a complete graph with vertices I ∪ J and costs on edges. An ant walk must then consist of n couplings (i, j).
I
Pheromone and heuristic on:
(alternatively the graph is given by I × J and ants walk through the list of jobs choosing agents. An order must be decided for the jobs.) Constraints: I
I
which agent to assign to the job
I
desirability of considering job i2 after job i1
I
desirability of assigning job i on agent j
if only feasible: the capacity constraint can be enforced by restricting the neighborhood, ie, Nki for a ant k at job i contains only those agents where job i can be assigned. if also infeasible: then no restriction
DM63 – Heuristics for Combinatorial Optimization Problems
23
DM63 – Heuristics for Combinatorial Optimization Problems
24
Connection between ACO and other Metaheuristics
Application Examples (3)
Greedy Randomized “Adaptive” Search Procedure (GRASP): While termination criterion is not satisfied: | generate candidate solution s using subsidiary greedy randomized constructive search | b perform subsidiary perturbative search on s
Subset problems: Set Covering Construction graph: Fully connected with set of vertices that corresponds to the set of columns plus a dummy vertex from where all the ants depart.
Adaptive Iterated Construction Search: initialise weights While termination criterion is not satisfied: | generate candidate solution s using subsidiary randomized constructive search | | perform subsidiary perturbative search on s b adapt weights based on s
Constraints: each vertex can be visited at most once and all rows must be covered. Pheromone Trails: associated with components (vertices); τj measures the desirability of including column j in solution. Heuristic information: on the components as function of the ant’s partial solution. e ηj = cjj where ej is the # of additional rows covered by j.
DM63 – Heuristics for Combinatorial Optimization Problems
Squeaky Wheel: Construct, Analyze, Prioritize
25
Sudoku
DM63 – Heuristics for Combinatorial Optimization Problems
Iterated Greedy (IG): destruct, reconstruct, acceptance criterion
GCP into SAT
27
DM63 – Heuristics for Combinatorial Optimization Problems
28
Set Covering into MAX-SAT
Sudoku into Exact Hitting Set Exact Covering: Set partitioning with ~c = ~1 I A = 1, 4, 7; I B = 1, 4; I C = 4, 5, 7;
min
n P j=1 n P
I D = 3, 5, 6;
yj aij yj = 1
∀i
j=1
I E = 2, 3, 6, 7;
yj ∈ {0, 1}
and I F = 2, 7.
A 2 1 1 2 60 6 3 60 6 4 61 6 5 60 6 40 7 1
B
C
1 0 0 1 0 0 0
0 0 0 1 1 0 1
D 0 0 1 0 1 1 0
E 0 1 1 0 0 1 1
F 3
0 17 7 07 7 07 7 07 05 1
The dual of Exact Covering is the Exact Hitting Set I A = 1, 2 I B = 5, 6 I C = 4, 5
max
n P
I D = 1, 2, 3
j=1 n P
I E = 3, 4
j=1
I F = 4, 5
xj aij xj = 1
xj ∈ {0, 1}
∀i
A 2 1 1 2 61 6 3 60 6 4 60 5 40 6 0
B
C
D
0 0 0 0 1 1
0 0 0 1 1 0
1 1 1 0 0 0
E 0 0 1 1 0 0
F 0 0 0 1 1 0
G 3
1 07 7 17 7 07 15 1
I G = 1, 3, 5, 6 DM63 – Heuristics for Combinatorial Optimization Problems
29
Asymmetric TSP into Symmetric TSP
DM63 – Heuristics for Combinatorial Optimization Problems
30
Capacited Vehicle Routing (CVRP) Input: I complete graph G(V, A), where V = {0, . . . , n} I vertices i = 1, . . . , n are customers that must be visited I vertex i = 0 is the single depot I arc/edges have associated a cost cij (cik + ckj ≥ cij , ∀ i, j ∈ V) I costumers have associated a non-negative demand di I a set of K identical vehicles with capacity C (di ≤ C) Task: Find collection of K circuits with minimum cost, defined as the sum of the costs of the arcs of the circuits and such that: I each circuit visit the depot vertex I each customer vertex is visited by exactly one circuit; and I the sum of the demands of the vertices visited by a circuit does not exceed the vehicle capacity C. Lower bound to K: K ≥ Kmin where Kmin is the number of bins in the associated Bin Packing Problem)
DM63 – Heuristics for Combinatorial Optimization Problems
31
DM63 – Heuristics for Combinatorial Optimization Problems
32
Construction Heuristics
CVRP Construction Heuristics
Construction heuristics specific for TSP I Heuristics that Grow Fragments I I I
I
I
Nearest Addition Farthest Addition Random Addition
I
Clarke-Wright savings heuristic
I
I I
I
Savings heuristics (Clarke and Wright)
I
Insertion heuristics Route-first cluster-second Cluster-first route-second
I I I I
Nearest Insertion Farthest Insertion Random Insertion
I I I I
Minimum span tree heuristic Christofides’ heuristics Fast recursive partitioning heuristic
I I
Solution representation: sets of integer sequences, one per route Neighborhoods structures: I I
DM63 – Heuristics for Combinatorial Optimization Problems
Sweep algorithm Generalized assignment Location based heuristic Petal algorithm
Perturbative Search
Heuristics based on Trees I
Nearest neighbors
I
Heuristics that Grow Tours I
I
Nearest neighborhood heuristics Double-Ended Nearest Neighbor heuristic Multiple Fragment heuristic (aka, greedy heuristic)
I
33
intra-route: 2-opt, 3-opt inter-routes: λ-interchange, relocate, exchange, CROSS, ejection chains, GENI
DM63 – Heuristics for Combinatorial Optimization Problems
34