Centre for Computational Intelligence (CCI)
Evolutionary Computation for Dynamic Optimization Problems
CCI (www.cci.dmu.ac.uk):
Mission: Developing fundamental theoretical and practical solutions to real-world problems using a variety of CI paradigms Members: 16 staff, 4 research fellows, 30+ PhDs, visiting researchers Three labs: Robotics, Eye Gaze, Game Programming
Shengxiang Yang
Funding:
Research Councils/Charities: EPSRC, ESRC, EU FP7 & Horizon 2020, Royal Academy of Engineering, Royal Society, Innovate UK, KTP, Innovation Fellowships, Nuffield Trust, etc. Government: Leicester City Council, DTI Industries: Lachesis, EMDA, RSSB, Network Rail, etc.
Centre for Computational Intelligence School of Computer Science and Informatics De Montfort University, United Kingdom Email:
[email protected] http://www.tech.dmu.ac.uk/∼syang
Collaborations:
http://www.sigevo.org/gecco-2015/
Universities: UK, USA, Spain, and China Industries and local governments
Permission to make digital or hard copies of part or all of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage, and that copies bear this notice and the full citation on the first page. Copyrights for third-party components of this work must be honored. For all other uses, contact the owner/author(s). Copyright is held by the author/owner(s).
Teaching/Training:
DTP-IS: University Doctor Training Programme in Intelligent Systems MSc Intelligent Systems, MSc Intelligent Systems & Robotics BSc Artificial Intelligence with Robotics
GECCO’15 Companion, July 11–15, 2015, Madrid, Spain. ACM 978-1-4503-3488-4/15/07. http://dx.doi.org/10.1145/2739482.2756589
Shengxiang Yang (De Montfort University)
Tutorial: EC for DOPs
YouTube page: http://www.youtube.com/thecci GECCO’15, 11/7/2015
1 / 84
Shengxiang Yang (De Montfort University)
Instructor/Presenter — Shengxiang Yang
Tutorial: EC for DOPs
GECCO’15, 11/7/2015
3 / 84
Outline of the Tutorial
Education and career history: PhD, Northeastern University, China, 1999 Worked at King’s College London, University of Leicester, and Brunel University, 1999-2012 Joined De Montfort University (DMU) as Professor in Computational Intelligence (CI) in July 2012 Director of Centre for Computational Intelligence (CCI)
Part I: Fundamentals Introduction to evolutionary computation (EC) EC for dynamic optimization problems (DOPs): Concept and motivation Benchmark and test problems Performance measures
Research interests:
Part II: Approaches and Case studies
Evolutionary Computation (EC) and nature-inspired computation Dynamic optimisation and multi-objective optimisation Relevant real-world applications
EC enhancement approaches for DOPs Case studies
Over 190 publications and over £1.2M funding as the PI
Part III: Issues, future work, and summary
AE/Editorial Board Member for 7 journals, including IEEE Trans. Cybern., Evol. Comput., Inform. Sci., and Soft Comput.
Relevant issues Future work
Chair of two IEEE CIS Task Forces
Summary and references
EC in Dynamic and Uncertain Environments Intelligent Network Systems Shengxiang Yang (De Montfort University)
Tutorial: EC for DOPs
GECCO’15, 11/7/2015
2 / 84
Shengxiang Yang (De Montfort University)
629
Tutorial: EC for DOPs
GECCO’15, 11/7/2015
4 / 84
What Is Evolutionary Computation (EC)?
EC Applications
EC encapsulates a class of stochastic optimization algorithms, dubbed Evolutionary Algorithms (EAs) An EA is an optimisation algorithm that is
EAs are easy-to-use: No strict requirements to problems Widely used for optimisation and search problems
Generic: a black-box tool for many problems Population-based: evolves a population of candidate solutions Stochastic: uses probabilistic rules Bio-inspired: uses principles inspired from biological evolution
Shengxiang Yang (De Montfort University)
Tutorial: EC for DOPs
GECCO’15, 11/7/2015
Financial and economical systems Transportation and logistics systems Industry engineering Automatic programming, art and music design ......
5 / 84
Shengxiang Yang (De Montfort University)
Design and Framework of an EA
Tutorial: EC for DOPs
GECCO’15, 11/7/2015
7 / 84
EC for Optimisation Problems
Given a problem to solve, first consider two key things:
Traditionally, research on EAs has focused on static problems
Representation of solution into individual Evaluation or fitness function
Aim to find the optimum quickly and precisely
Then, design the framework of an EA: Initialization of population Evolve the population Selection of parents Variation operators (recombination & mutation) Selection of offspring into next generation
But, many real-world problems are dynamic optimization problems (DOPs), where changes occur over time In transport networks, travel time between nodes may change In logistics, customer demands may change
Termination condition: a given number of generations
Shengxiang Yang (De Montfort University)
Tutorial: EC for DOPs
GECCO’15, 11/7/2015
6 / 84
Shengxiang Yang (De Montfort University)
630
Tutorial: EC for DOPs
GECCO’15, 11/7/2015
8 / 84
What Are DOPs?
Why EC for DOPs?
In general terms, “optimization problems that change over time” are called dynamic problems/time-dependent problems
Many real-world problems are DOPs EAs, once properly enhanced, are good choice
~ t) F = f (~x , φ,
Inspired by natural/biological evolution, always in dynamic environments Intrinsically, should be fine to deal with DOPs
~ parameter(s); t: time – ~x : decision variable(s); φ:
Many events on EC for DOPs recently
DOPs: special class of dynamic problems that are solved online by an algorithm as time goes by
Shengxiang Yang (De Montfort University)
Tutorial: EC for DOPs
GECCO’15, 11/7/2015
9 / 84
Shengxiang Yang (De Montfort University)
Why DOPs Challenge EC?
Tutorial: EC for DOPs
GECCO’15, 11/7/2015
11 / 84
Relevant Events Books (Monograph or Edited): Yang & Yao, 2013; Yang et al., 2007; Morrison, 2004; Weicker, 2003; Branke, 2002
For DOPs, optima may move over time in the search space Challenge: need to track the moving optima over time
PhD Theses: Mavrovouniotis, 2013; Helbig, 2012; du Plessis, 2012; Li, 2011; Nguyen, 2011; Simoes, 2010
Journal special issues: Neri & Yang, 2010; Yang et al., 2006; Jin & Branke, 2006; Branke, 2005
Workshops and conference special sessions: DOPs challenge traditional EAs
EvoSTOC (2004–2015): part of Evo* ECiDUE (2004–2015): part of IEEE CEC EvoDOP (’99, ’01, ’03, ’05, ’07, ’09): part of GECCO
Once converged, hard to escape from an old optimum
IEEE Symposium on CIDUE (2011, 2013, 2014, 2015) IEEE Competitions: within IEEE CEC 2009 & CEC 2012
Shengxiang Yang (De Montfort University)
Tutorial: EC for DOPs
GECCO’15, 11/7/2015
10 / 84
Shengxiang Yang (De Montfort University)
631
Tutorial: EC for DOPs
GECCO’15, 11/7/2015
12 / 84
Benchmark and Test DOPs
Moving Peaks Benchmark (MPB) Problem Proposed by Branke (1999) The MPB problem in the D-dimensional space:
Basic idea: change base static problem(s) to create DOPs
F (~x , t) = max
Real space:
i=1,...,p
Switch between different functions Move/reshape peaks in the fitness landscape
1 + Wi (t)
Hi (t) PD 2 j=1 (xj (t) − Xij (t))
– Wi (t), Hi (t), Xi (t) = {Xi1 · · · XiD }: height, width, location of peak i at t The dynamics:
Binary space: Switch between ≥ 2 states of a problem: knapsack Use binary masks: XOR DOP generator (Yang & Yao’05)
Hi (t) = Hi (t − 1) + height_severity ∗ σ Wi (t) = Wi (t − 1) + width_severity ∗ σ s ~vi (t) = ~ ~ ~r + ~vi (t − 1) ((1 − λ)r + λvi (t − 1))
Combinatorial space: Change decision variables: item weights/profits in knapsack problems Add/delete decision variables: new jobs in scheduling, nodes added/deleted in network routing problems
~ i (t) = X ~ i (t)(t − 1) + ~vi (t) X
– σ ∼ N(0, 1); λ: correlated parameter – ~vi (t): shift vector, which combines random vector ~r and ~vi (t − 1) and is normalized to the shift length s Shengxiang Yang (De Montfort University)
Tutorial: EC for DOPs
GECCO’15, 11/7/2015
13 / 84
Shengxiang Yang (De Montfort University)
The DF1 Generator
GECCO’15, 11/7/2015
15 / 84
Dynamic Knapsack Problems (DKPs) Static knapsack problem: Given n items, each with a weight and a profit, and a knapsack with a fixed capacity, select items to fill up the knapsack to maximize the profit while satisfying the knapsack capacity constraint The DKP:
Proposed by Morrison & De Jong (1999) The base landscape in the D-dimensional real space: v u D uX f (~x ) = max Hi − Ri × t (xj − Xij )2 i=1,...,p
Constructed by changing weights and profits of items, and/or knapsack capacity over time as:
j=1
– ~x = (x1 , · · · , xD ): a point in the landscape; p: number of peaks – Hi , Ri , Xi = (Xi1 , · · · , XiD ): height, slope, center of peak i
Max f (~x (t), t) =
The dynamics is controlled by a logistics function:
GECCO’15, 11/7/2015
pi (t) · xi (t), s. t. :
n X i=1
wi (t) · xi (t) ≤ C(t)
– ~x (t) ∈ {0, 1}n : a solution at time t – xi (t) ∈ {0, 1}: indicates whether item i is included or not – pi (t) and wi (t): profit and weight of item i at t – C(t): knapsack capacity at t
– A ∈ [1.0, 4.0]: a constant; ∆t : step size of changing a parameter
Tutorial: EC for DOPs
n X i=1
∆t = A · ∆t−1 · (1 − ∆t−1 )
Shengxiang Yang (De Montfort University)
Tutorial: EC for DOPs
14 / 84
Shengxiang Yang (De Montfort University)
632
Tutorial: EC for DOPs
GECCO’15, 11/7/2015
16 / 84
The XOR DOP Generator
Constructing Cyclic Environments with Noise We can also construct cyclic environments with noise:
The XOR DOP generator can create DOPs from any binary f (~x ) by an XOR operator “⊕” (Yang, 2003; Yang & Yao, 2005)
Each time before a base state is entered, it is bitwise changed with a small probability
Suppose the environment changes every τ generations For each environmental period k = ⌊t/τ ⌋, do: 1
State 1
M(1)=1001011010 M(0) + T(0)
State 4 . . .
2
M(1) + T(1)
M(0)=0000000000
M(3)=1101000101
State 0 (Initial State)
State 3 M(2) + T(2)
3
M(2)=0111010011 State 2 T(0)=1001011010 T(1)=1110001001
Base State 1 M(1)=1001011011
Create a template Tk with ρ ∗ l ones
~ ) incrementally Create a mask M(k ~ M(0) = ~0 (the initial state)
Base State 0 (Initial State) M(0)=0000000000
~ ~ )⊕T ~ (k ) M(k + 1) = M(k
Evaluate an individual: ~ )) f (~x , t) = f (~x ⊕ M(k
T(2)=1010010110
Tutorial: EC for DOPs
GECCO’15, 11/7/2015
17 / 84
Shengxiang Yang (De Montfort University)
M(1)=1001011010 Base State 0 (Initial State)
M(0) T(0)
M(0)=0000000000
M(1) T(1)
2
Base State 2
M(3)=0110100101 Base State 3
3
May involve dynamic cost (distance) matrix
~ Create 2K masks M(i) as base states
D(t) = {dij (t)}n∗n – dij (t): cost from city i to j; n: the number of cities The aim is to find a minimum-cost route containing all cities at time t DTSP can be defined as f (x, t):
~ Cycle among M(i)’s every τ generations
f (x, t) = Min(
~ t )) = f (~x ⊕ M(k ~ %(2K ))) f (~x , t) = f (~x ⊕ M(I
Tutorial: EC for DOPs
GECCO’15, 11/7/2015
n X
dxi ,xi+1 (t))
i=1
– k = ⌊t/τ ⌋: environmental index – It = k %(2K ): mask index
Shengxiang Yang (De Montfort University)
19 / 84
Dynamic TSP (DTSP):
~ + 1) = M(i) ~ ~ (i%K ), i = 0,· · · , 2K −1 M(i ⊕T
M(2) T(0)
GECCO’15, 11/7/2015
Given a set of cities, find the shortest route that visits each city once and only once
~ (0),· · · ,T ~ (K −1) Construct K templates T
~ M(0) = ~0 (the initial state)
M(2)=1111111111 M(3) T(1)
Tutorial: EC for DOPs
Stationary traveling salesman problem (TSP):
Form a partition of the search space Each contains ρ × l = l/K ones
Base State 1
Bit 1 changed by noise
Dynamic Traveling Salesman Problems
Can extend the XOR DOP generator to create cyclic environments: Partition Templates: T(0)=1001011010 v T(1)=0110100101
M(2)=0111111111
Bit 6 changed by noise
Base State 3
Constructing Cyclic Dynamic Environments
1
Base State 2
M(3)=0110110101
τ and ρ controls the speed and severity of change respectively
Shengxiang Yang (De Montfort University)
Bit 10 changed by noise
where xi ∈ 1, · · · , n. If i 6= j, xi 6= xj , and xn+1 = x1
18 / 84
Shengxiang Yang (De Montfort University)
633
Tutorial: EC for DOPs
GECCO’15, 11/7/2015
20 / 84
Dynamic Permutation Benchmark Generator
Generalized DOP Benchmark Generator (GDBG) Proposed by Li & Yang (2008), GDBG uses the model below:
The dynamic benchmark generator for permutation-encoded problems (DBGP) can create a DOP from any stationary TSP/VRP by swapping objects:
Generalized DBG
Real Space
Binary Space
Combinatory Space
.....
Instance k
....
Instance 2
Instance 1
.....
.....
Instance k
– φ: system control parameter Dynamism results from tuning φ of the current environment
....
Modify the encoding of the problem instance with m × n pairwise swaps
Instance 2
3
Instance 1
In GDBG, DOPs are defined as:
Instance k
Generate another randomly re-order vector r~′ (T ) that contains only the first m × n objects of ~r (T )
....
2
Instance 2
Generate a random vector ~r (T ) that contains all objects every f iterations
Instance 1
1
F = f (x , φ, t),
φ(t + 1) = φ(t) ⊕ ∆φ
– ∆φ: deviation from the current control parameter(s) The new environment at t + 1 is as follows:
More details: M. Mavrovouniotis, S. Yang, & X. Yao (2012). PPSN XII, Part II, LNCS 7492, pp. 508–517
f (x , φ, t + 1) = f (x , φ(t) ⊕ ∆φ, t) Shengxiang Yang (De Montfort University)
Tutorial: EC for DOPs
GECCO’15, 11/7/2015
21 / 84
Shengxiang Yang (De Montfort University)
Effect on Algorithms
Tutorial: EC for DOPs
GECCO’15, 11/7/2015
23 / 84
GDBG: Dynamic Change Types
Similar with the XOR DOP generator, DBGP shifts the population of an alg. to new location in the fitness landscape The individual with the same encoding as before a change will have a different cost after the change
Change types: 1 Small step: ∆φ = α · kφk · rand() 2 Large step: ∆φ = kφk · (α + (1 − α)rand()) 3 Random: ∆φ = kφk · rand() 4 Chaotic: φ(t + 1) = A · φ(t) · (1 − φ(t)/kφk) 5 Recurrent: φ(t + 1) = φ(t%P) 6 Recurrent with nosy: φ(t + 1) = φ(t%P) + α · kφk · rand() 7 ...... More details: C. Li & S. Yang (2008). SEAL’08, LNCS 5361, pp. 391–400
Can extend for cyclic and cyclic with noise environments Shengxiang Yang (De Montfort University)
Tutorial: EC for DOPs
GECCO’15, 11/7/2015
22 / 84
Shengxiang Yang (De Montfort University)
634
Tutorial: EC for DOPs
GECCO’15, 11/7/2015
24 / 84
DOPs: Classification
Performance Measures For EC for stationary problems, 2 key performance measures Convergence speed Success rate of reaching optimality
Classification criteria:
For EC for DOPs, over 20 measures (Nguyen et al., 2012)
Time-linkage: Does the future behaviour of the problem depend on the current solution?
Optimality-based performance measures Collective mean fitness or mean best-of-generation Accuracy Adaptation Offline error and offline performance Mean distance to optimum at each generation ......
Predictability: Are changes predictable? Visibility: Are changes visible or detectable Cyclicity: Are changes cyclic/recurrent in the search space? Factors that change: objective, domain/number of variables, constraints, and/or other parameters
Shengxiang Yang (De Montfort University)
Tutorial: EC for DOPs
GECCO’15, 11/7/2015
Behaviour-based performance measures Reactivity Stability Robustness Satisficability Diversity measures ......
25 / 84
Shengxiang Yang (De Montfort University)
DOPs: Common Characteristics
Tutorial: EC for DOPs
GECCO’15, 11/7/2015
27 / 84
Performance Measures: Examples Collective mean fitness (mean best-of-generation): F BOG =
Common characteristics of DOPs in the literature:
Xi=G 1 Xj=N 1 × ( × FBOGij ) i=1 j=1 G N
– G and N: number of generations and runs, resp.
Most DOPs are non time-linkage problems
– FBOGij : best-of-generation fitness of generation i of run j
For most DOPs, changes are assumed to be detectable In most cases, the objective function is changed
Adaptation performance (Mori et al., 1997) Ada =
Many DOPs have unpredictable changes Most DOPs have cyclic/recurrent changes
1 X (fbest (t)/fopt (t)) T t=1..T
Accuracy (Trojanowski and Michalewicz, 1999) Acc =
1 X (fbest (i) − fopt (i)) K i=1..K
– fbest (i): best fitness for environment i (best before change)
Shengxiang Yang (De Montfort University)
Tutorial: EC for DOPs
GECCO’15, 11/7/2015
26 / 84
Shengxiang Yang (De Montfort University)
635
Tutorial: EC for DOPs
GECCO’15, 11/7/2015
28 / 84
Part II: Approaches and Case studies
EC for DOPs: General Approaches
Many approaches developed to enhance EAs for DOPs Typical approaches: Memory: store and reuse useful information Diversity: handle convergence directly Multi-population: co-operate sub-populations Adaptive: adapt generators and parameters Prediction: predict changes and take actions in advance
EC enhancement approaches for DOPs Case studies
They have been applied to different EAs for DOPs
Shengxiang Yang (De Montfort University)
Tutorial: EC for DOPs
GECCO’15, 11/7/2015
29 / 84
Shengxiang Yang (De Montfort University)
EC for DOPs: First Thinking
Tutorial: EC for DOPs
GECCO’15, 11/7/2015
31 / 84
Memory Approaches Cyclic DOPs: change cyclically among a fixed set of states
Recap: traditional EAs are not good for DOPs Goal: to track the changing optimum How about restarting an EA after a change? Natural and easy choice But, not good choice because: 1 2
It may be inefficient, wasting computational resources It may lead to very different solutions before and after a change. For real-world problems, we may expect solutions to remain similar
Memory works by storing and reusing useful information Two classes regarding how to store information
Extra approaches are needed to enhance EAs for DOPs
Implicit memory: uses redundant representations Multiploidy and dominance (Ng & Wong, 1995; Lewis et al., 1998) Dualism mechanisms (Yang, 2003; Yang & Yao, 2005)
Explicit memory: uses extra space to store information
Shengxiang Yang (De Montfort University)
Tutorial: EC for DOPs
GECCO’15, 11/7/2015
30 / 84
Shengxiang Yang (De Montfort University)
636
Tutorial: EC for DOPs
GECCO’15, 11/7/2015
32 / 84
Explicit Memory: Direct vs Associative
Implicit Memory: Diploid Genetic Algorithm Encoding
Dominance Scheme
Genotype
Genotype−to−Phenotype Mapping
0
o
1
i
Chromosome 1
0
0
0
0/1
0
Chromosome 2
o
0
0
1
0/1
1
0/1
1
1
1
i
0
0/1
1
1
Dominance Scheme
Associative memory: store environmental information + good solutions (Yang & Yao, 2008) Main Population
Ng & Wong (1995) Phenotype
Same Phenotypic Alleles Evaluating
Direct memory: store good solutions (Branke, 1999)
External Environment
Fitness Genotypic Alleles: Phenotypic Alleles:
A
B
C
D
A
0
0
0
1
B
0
0
0
1
C
0
0
1
1
D
1
1
1
1
Update memory
Update memory 3. Replace
1. Extract env info 2. Associate
+ 1. Merge
2. Replace one memory solution
Lewis et al. (1998)
2. Create
3. Replace 1. Associate Retrieve memory
Retrieve memory
Memory
Memory
Dominance scheme may change or be adaptive (Uyar & Harmanci, 2005) Tutorial: EC for DOPs
2. Select
1. Select best pop member
Each individual has a pair of chromosomes Dominance scheme maps genotype to phenotype
Shengxiang Yang (De Montfort University)
Main Population
GECCO’15, 11/7/2015
Environment information
Direct Memory
33 / 84
Shengxiang Yang (De Montfort University)
Solution
Associative Memory
Tutorial: EC for DOPs
GECCO’15, 11/7/2015
Explicit Memory Approaches
Associative Memory Based Genetic Algorithm
Basic idea: use extra memory
~ to represent environmental info. Idea: Use allele distribution (AD) D
With time, store useful information of the pop into memory When a change occurs, use memory to track new optimum
~ S> pairs Use memory to store