Ant Colony Optimization Exercises

149 downloads 0 Views 392KB Size Report
Ant Colony Optimization: the Metaheuristic. Application Examples. Connection between ACO and other Metaheuristics. Encodings. Capacited Vehicle Routing.
Outline DM63 HEURISTICS FOR COMBINATORIAL OPTIMIZATION Ant Colony Optimization: the Metaheuristic Application Examples Connection between ACO and other Metaheuristics Encodings Capacited Vehicle Routing

Lecture 11

Ant Colony Optimization Exercises Marco Chiarandini

DM63 – Heuristics for Combinatorial Optimization Problems

Ant Colony Optimization Metaheuristic I

I

I

Note:

Population-based method in which artificial ants iteratively construct candidate solutions. Solution construction is probabilistically biased by pheromone trail information, heuristic information and partial candidate solution of each ant (memory). Pheromone trails are modified during the search process to reflect collective experience.

Ant Colony Optimization (ACO): initialize pheromone trails While termination criterion is not satisfied: | generate population sp of candidate solutions using subsidiary randomized constructive search | | perform subsidiary perturbative search on sp | |b update pheromone trails based on sp DM63 – Heuristics for Combinatorial Optimization Problems

3

I

In each cycle, each ant creates one candidate solution using a constructive search procedure.

I

Ants build solutions by performing randomized walks on a construction graph G = (V, E) where V are solution components and G is fully connected.

I

All pheromone trails are initialized to the same value, τ0 .

I

Pheromone update typically comprises uniform decrease of all trail levels (evaporation) and increase of some trail levels based on candidate solutions obtained from construction + perturbative search.

I

Subsidiary perturbative search is (often) applied to individual candidate solutions.

I

Termination criterion can include conditions on make-up of current population, e.g., variation in solution quality or distance between individual candidate solutions.

2

Example: A simple ACO algorithm for the TSP (2)

Example: A simple ACO algorithm for the TSP (1) I

Search space and solution set as usual (all Hamiltonian cycles in given graph G).

I

Associate pheromone trails τij with each edge (i, j) in G.

I

Use heuristic values ηij :=

I

Initialize all weights to a small value τ0 (τ0 = 1).

I

Constructive search: Each ant starts with randomly chosen vertex and iteratively extends partial round trip πk by selecting vertex not contained in πk with probability

1 cij

I

Subsidiary perturbative search: Perform iterative improvement based on standard 2-exchange neighborhood on each candidate solution in population (until local minimum is reached).

I

Update pheromone trail levels according to X τij := (1 − ρ) · τij + ∆ij (s) s∈sp 0

where ∆ij (s) := 1/Cs if edge (i, j) is contained in the cycle represented by s 0 , and 0 otherwise. Motivation: Edges belonging to highest-quality candidate solutions and/or that have been used by many ants should be preferably used in subsequent constructions.

[τij ]α · [ηij ]β pij = P [τil ]α · [ηil ]β l∈Nik I

α and β are parameters. DM63 – Heuristics for Combinatorial Optimization Problems

Termination: After fixed number of cycles (= construction + perturbative search phases).

5

ACO Variants

Ant System on TSP I

I I

Ant System AS (Dorigo et al., 1991) Elitist AS (EAS)(Dorigo et al., 1991; 1996) I

I

I

The iteration best solution adds more pheromone I

Only best ranked ants can add pheromone Pheromone added is proportional to rank

I

Max-Min AS (MMAS)(St¨ utzle & Hoos, 1997)

I

Ant Colony System (ACS) (Gambardella & Dorigo, 1996; Dorigo & Gambardella, 1997)

I

Approximate Nondeterministic Tree Search ANTS (Maniezzo 1999) Hypercube AS (Blum, Roli and Dorigo, 2001)

I

m CNN Motivation: sligthly more than what evaporates Construction: m ants in m randomly chosen cities τij = τo =

Rank-Based AS (ASrank)(Bullnheimer et al., 1997; 1999) I

Initialization:

[τij ]α · [ηij ]β pij = P , [τil ]α · [ηil ]β

α and β parameters

l∈Nik I

Update τij ← (1 − ρ) · τij m X τij ← τij + ∆kij

to all the edges to the edges visited by the ants, ∆kij =

1 Ck

k=1

DM63 – Heuristics for Combinatorial Optimization Problems

7

DM63 – Heuristics for Combinatorial Optimization Problems

8

Elitist Ant System on TSP

Rank-based Ant System on TSP

I I

Update τij ← (1 − ρ) · τij m X ∆kij + e · ∆bs τij ← τij + ij

Update: only w − 1 best ranked ants + the best-so-far solution deposit pheromone: τij ← (1 − ρ) · τij

to all the edges τij ← τij +

to the edges visited by the ants

k=1

w−1 X

∆kij + w · ∆bs ij

to all the edges to the edges visited by the ants

k=1

 ∆bs ij

=

1 Cbs

0

(ij) in tour k, bs best-so-far otherwise

∆bs ij

DM63 – Heuristics for Combinatorial Optimization Problems

9

DM63 – Heuristics for Combinatorial Optimization Problems

MAX − MIN Ant System (MMAS)

Three main ideas: I Different state transition rule  β  arg maxl∈Nki {τil ηil } α β ij ] j = pij = P[τij ] ·[η α  [τil ] ·[ηil ]β  k

Update τij ← (1 − ρ) · τij τij ← τij + ∆bs ij

10

Ant Colony System (ACS) on the TSP

Peculiarities in pheromone management: I

to all the edges only to the edges visited by the best ant

l∈N

Meaning of best alternates during the search between: I I

I I I

I

best-so-far iteration best

I

Reinitialization of τ if: I

stagnation occurs idle iterations

otherwise

i

Global pheromone update

to only (ij) in best-so-far tour (O(n) complexity) Local pheromone update: happens during tour construction to avoid other ants to make the same choices: τij ← (1 − ) · τij + τ0

Results obtained are better than AS, EAS, and ASrank, and of similar quality to ACS’s DM63 – Heuristics for Combinatorial Optimization Problems

if q ≤ q0

τij ← (1 − ρ) · τij + ρ∆bs ij (s)

bounded values τmin and τmax τmax 1 τmax = ρC ∗ and τmin = a I

1 Ck 1 = bs C

∆kij =

11

 = 0.1, τ0 =

1 nCNN

Parallel construction preferred to sequential construction DM63 – Heuristics for Combinatorial Optimization Problems

12

Approximate Nondeterministic Tree Search on TSP

Strongly Invariant ACO on TSP Considers instances which are equivalent up to a linear transformation of units.

I

Use of lower bound to compute heuristic value I

I

The siACO is an algorithm that enjoy the property of that its internal state at each iteration is the same on equivalent instances.

Add an arc to the current partial solution and estimate LB of complete solution

For AS:

Different solution construction rule ατij + (1 − α)ηij pkij = P ατil + (1 − α)ηil

I

Use heuristic values ηij :=

I

Update according to

l∈Nk i I

Different pheromone trail update rule k  X θ(1 − τij ← τij + ∆kij ∆kij = i=1 0

CNN n·cij

τij := (1 − ρ) · τij +

X

∆ij (s)

s∈sp 0 Ck −LB Lavg −LB

if (ij) in belongs to T k otherwise

NN

C where ∆ij (s) := m·C s if edge (i, j) is contained in the cycle represented by s 0 , and 0 otherwise.

Can be extended to other ACO versions and to other problems: QAP and Scheduling DM63 – Heuristics for Combinatorial Optimization Problems

13

Analysis

DM63 – Heuristics for Combinatorial Optimization Problems

14

Pheromone development

Other things to check I

Synergy

I

Pheromone Development

I

Strength of local search (exploitation vs exploration)

I

Heuristic Information (linked to parameter β) Results show that with β = 0 local search can still be enough

I

Lamarkian vs Darwinian Pheromone Updates

I

Run Time impact

DM63 – Heuristics for Combinatorial Optimization Problems

16

Analysis

Parameter tuning

Number of tours generated to find the optimal solution as a function of the number m of ants used DM63 – Heuristics for Combinatorial Optimization Problems

17

DM63 – Heuristics for Combinatorial Optimization Problems

18

ACO: Theoretical results

Ant Colony Optimization . . . I

A framework and several problem applications are available from: http://www.aco-metaheuristic.org/

I

good performance also in dynamic optimization problems, such as routing in telecommunications networks

I

Gutjahr (Future Generation Computer Systems, 2000; Information Processing Letters, 2002) and St¨ utzle and Dorigo (IEEE Trans. on Evolutionary Computation, 2002) have proved convergence with prob 1 to the optimal solution of different versions of ACO

I

Meuleau and Dorigo (Artificial Life Journal, 2002) have shown that there are strong relations between ACO and stochastic gradient descent in the space of pheromone trails, which converges to a local optima with prob 1

I

Birattari et al. (TR, 2000; ANTS 2002) have shown the tight relationship between ACO and dynamic programming

I

Zlochin et al. (TR, 2001) have shown the tight relationship between ACO and estimation of distribution algorithms

For further details on Ant Colony Optimization, see the book by Dorigo and St¨ utzle [2004].

DM63 – Heuristics for Combinatorial Optimization Problems

19

DM63 – Heuristics for Combinatorial Optimization Problems

20

Application Examples

Generalized Assignment Problem (GAP)

Input:

Linear permutations problems: SMTWTP Construction graph: Fully connected and the set of vertices consists of the n jobs and the n positions to which the jobs are assigned.

Pheromone Trails: τij expresses the desirability of assigning job i in position j (cumulative rule) 1 hi

where hi is a dispatching rule.

DM63 – Heuristics for Combinatorial Optimization Problems

I

the amount bi of resource available to agent i

Task: Find an assignment of jobs to agents σ : J → I such that: P min f(σ) = cσ(j)j P j∈J aij ≤ bi ∀i ∈ I s.t.

Constraints: all jobs have to be scheduled.

Heuristic information: ηij =

I

a set of jobs J = {1, . . . , n} and a set of agents I = {1, . . . , m}. the cost cij and the resource requirement aij of a job j assigned to agent i

I

j∈J,σ(j)=i

21

DM63 – Heuristics for Combinatorial Optimization Problems

22

Application Examples (2)

Pheromone:

Assignment problems: Generalized Assignment Problem

Two choices: I which job to consider next

Construction Graph: a complete graph with vertices I ∪ J and costs on edges. An ant walk must then consist of n couplings (i, j).

I

Pheromone and heuristic on:

(alternatively the graph is given by I × J and ants walk through the list of jobs choosing agents. An order must be decided for the jobs.) Constraints: I

I

which agent to assign to the job

I

desirability of considering job i2 after job i1

I

desirability of assigning job i on agent j

if only feasible: the capacity constraint can be enforced by restricting the neighborhood, ie, Nki for a ant k at job i contains only those agents where job i can be assigned. if also infeasible: then no restriction

DM63 – Heuristics for Combinatorial Optimization Problems

23

DM63 – Heuristics for Combinatorial Optimization Problems

24

Connection between ACO and other Metaheuristics

Application Examples (3)

Greedy Randomized “Adaptive” Search Procedure (GRASP): While termination criterion is not satisfied: | generate candidate solution s using subsidiary greedy randomized constructive search | b perform subsidiary perturbative search on s

Subset problems: Set Covering Construction graph: Fully connected with set of vertices that corresponds to the set of columns plus a dummy vertex from where all the ants depart.

Adaptive Iterated Construction Search: initialise weights While termination criterion is not satisfied: | generate candidate solution s using subsidiary randomized constructive search | | perform subsidiary perturbative search on s b adapt weights based on s

Constraints: each vertex can be visited at most once and all rows must be covered. Pheromone Trails: associated with components (vertices); τj measures the desirability of including column j in solution. Heuristic information: on the components as function of the ant’s partial solution. e ηj = cjj where ej is the # of additional rows covered by j.

DM63 – Heuristics for Combinatorial Optimization Problems

Squeaky Wheel: Construct, Analyze, Prioritize

25

Sudoku

DM63 – Heuristics for Combinatorial Optimization Problems

Iterated Greedy (IG): destruct, reconstruct, acceptance criterion

GCP into SAT

27

DM63 – Heuristics for Combinatorial Optimization Problems

28

Set Covering into MAX-SAT

Sudoku into Exact Hitting Set Exact Covering: Set partitioning with ~c = ~1 I A = 1, 4, 7; I B = 1, 4; I C = 4, 5, 7;

min

n P j=1 n P

I D = 3, 5, 6;

yj aij yj = 1

∀i

j=1

I E = 2, 3, 6, 7;

yj ∈ {0, 1}

and I F = 2, 7.

A 2 1 1 2 60 6 3 60 6 4 61 6 5 60 6 40 7 1

B

C

1 0 0 1 0 0 0

0 0 0 1 1 0 1

D 0 0 1 0 1 1 0

E 0 1 1 0 0 1 1

F 3

0 17 7 07 7 07 7 07 05 1

The dual of Exact Covering is the Exact Hitting Set I A = 1, 2 I B = 5, 6 I C = 4, 5

max

n P

I D = 1, 2, 3

j=1 n P

I E = 3, 4

j=1

I F = 4, 5

xj aij xj = 1

xj ∈ {0, 1}

∀i

A 2 1 1 2 61 6 3 60 6 4 60 5 40 6 0

B

C

D

0 0 0 0 1 1

0 0 0 1 1 0

1 1 1 0 0 0

E 0 0 1 1 0 0

F 0 0 0 1 1 0

G 3

1 07 7 17 7 07 15 1

I G = 1, 3, 5, 6 DM63 – Heuristics for Combinatorial Optimization Problems

29

Asymmetric TSP into Symmetric TSP

DM63 – Heuristics for Combinatorial Optimization Problems

30

Capacited Vehicle Routing (CVRP) Input: I complete graph G(V, A), where V = {0, . . . , n} I vertices i = 1, . . . , n are customers that must be visited I vertex i = 0 is the single depot I arc/edges have associated a cost cij (cik + ckj ≥ cij , ∀ i, j ∈ V) I costumers have associated a non-negative demand di I a set of K identical vehicles with capacity C (di ≤ C) Task: Find collection of K circuits with minimum cost, defined as the sum of the costs of the arcs of the circuits and such that: I each circuit visit the depot vertex I each customer vertex is visited by exactly one circuit; and I the sum of the demands of the vertices visited by a circuit does not exceed the vehicle capacity C. Lower bound to K: K ≥ Kmin where Kmin is the number of bins in the associated Bin Packing Problem)

DM63 – Heuristics for Combinatorial Optimization Problems

31

DM63 – Heuristics for Combinatorial Optimization Problems

32

Construction Heuristics

CVRP Construction Heuristics

Construction heuristics specific for TSP I Heuristics that Grow Fragments I I I

I

I

Nearest Addition Farthest Addition Random Addition

I

Clarke-Wright savings heuristic

I

I I

I

Savings heuristics (Clarke and Wright)

I

Insertion heuristics Route-first cluster-second Cluster-first route-second

I I I I

Nearest Insertion Farthest Insertion Random Insertion

I I I I

Minimum span tree heuristic Christofides’ heuristics Fast recursive partitioning heuristic

I I

Solution representation: sets of integer sequences, one per route Neighborhoods structures: I I

DM63 – Heuristics for Combinatorial Optimization Problems

Sweep algorithm Generalized assignment Location based heuristic Petal algorithm

Perturbative Search

Heuristics based on Trees I

Nearest neighbors

I

Heuristics that Grow Tours I

I

Nearest neighborhood heuristics Double-Ended Nearest Neighbor heuristic Multiple Fragment heuristic (aka, greedy heuristic)

I

33

intra-route: 2-opt, 3-opt inter-routes: λ-interchange, relocate, exchange, CROSS, ejection chains, GENI

DM63 – Heuristics for Combinatorial Optimization Problems

34