Evolutionary Optimization in Spatioâtemporal ... - Semantic Scholar
Evolutionary Optimization in Spatio–temporal Fitness Landscapes Hendrik Richter HTWK Leipzig, Fachbereich Elektrotechnik und Informationstechnik, Institut Mess–, Steuerungs– und Regelungstechnik, Postfach 30 11 66, D–04125 Leipzig, Germany [email protected]
Abstract. Spatio–temporal fitness landscapes that are constructed from Coupled Map Lattices (CML) are introduced. These landscapes are analyzed in terms of modality and ruggedness. Based on this analysis, we study the relationship between landscape measures and the performance of an evolutionary algorithm used to solve the dynamic optimization problem.
1
Introduction
In theoretical studies of evolutionary algorithms, concepts of fitness landscapes have shown to be a useful tool [4,7,14,16]. A fitness landscape assigns fitness values to the points of the search space through which the evolution moves. This search space can be constructed by a genotype–to–fitness mapping or more generally by encoding the set of possible solutions of an optimization problem to form a representation space for which additionally a neighborhood structure needs to be defined. Typically, the topology of the search space is considered to be constant over the run–time of the evolutionary algorithm and hence such a fitness landscape is a static concept. As dynamic optimization turned more and more into an important topic in evolutionary computation [17,2,9,6,19], it became desirable to have concepts of fitness landscapes in dynamic environments as well. Whereas recent results have shown that such an approach is useful to characterize and classify types of dynamic landscapes [5], there is a certain lack of environments that show sufficiently complex structure in both spatial topology and temporal dynamics. In this paper, such spatio–temporal fitness landscapes are introduced and it is shown how these landscapes can be constructed from spatio–temporal dynamical systems, namely from Coupled Map Lattices (CML). CML have been the subject of intensive research, which revealed a broad variety of spatio–temporal behavior, including different types of pattern formation, spatio–temporal chaos, quasi– periodicity and emergence [8,3]. In the next section, a methodology to construct spatio–temporal fitness landscapes from CML is given. In Sec. 3, these landscapes are analyzed in terms of modality and ruggedness. Further, quantifying measures such as the number of T.P. Runarsson et al. (Eds.): PPSN IX, LNCS 4193, pp. 1–10, 2006. c Springer-Verlag Berlin Heidelberg 2006
2
H. Richter
optima and the correlation structure are studied. In Sec. 4, factors contributing to problem hardness in dynamic optimization like change frequency and dynamic severity are reviewed. Numerical experiments using an evolutionary algorithm to optimize in the spatio–temporal fitness landscape are reported in Sec. 5. In the final section, the findings are summarized and conclusions are drawn.
2
Constructing Spatio–temporal Fitness Landscapes
Constructing spatio–temporal fitness landscapes should begin with defining a spatio–temporal dynamical system. Therefore, we lay out a lattice grid with I × J equally sized cells, which form a 2D–structure. For every discrete time step k, k = 0, 1, 2, . . ., each cell is characterized by its height h(i, j, k),
i = 1, 2, . . . , I,
j = 1, 2, . . . , J,
(1)
where i, j denote the spatial indices in vertical and horizontal direction, respectively, see Fig. 1. This height h(i, j, k), which we will interpret as fitness according to the geometrical metaphor of a fitness landscape, is subject to changes over time, which are described by the two–dimensional CML with nearest–neighbor coupled interaction [8,3] h(i, j, k + 1) = (1 − )g(h(i, j, k)) +
g (h(i − 1, j, k)) 4
+ g (h(i + 1, j, k)) + g (h(i, j − 1, k)) +g (h(i, j + 1, k))
,
(2)
where g(h(i, j, k)) is a local mapping function and is the diffusion coupling strength. In other words, as h(i, j, k) denotes the height of the h(i, j)–th unit bar situated in the (i, j)–lattice cell at time step k, Eq. (2) describes how this height changes over time depending on its own height and the heights of the surrounding bars, see Fig. 1a. The CML is initialized by the heights h(i, j, 0) being realizations of a random variable uniformly distributed on [0, 1]. To complete the definition of the CML (2), we employ the logistic map g(h(i, j, k)) = αh(i, j, k)(1 − h(i, j, k))
(3)
as mapping function and render the period boundary conditions h(I + 1, j, k) = h(1, j, k),
h(i, J + 1, k) = h(i, 1, k).
(4)
To summarize, the CML is a spatio–temporal dynamical system with discrete space (lattice) and time (map). The system’s states, which we interpret as heights, are continuous and situated on the lattice. They dynamically interact with surrounding states via the nonlinear map (3). The spatio–temporal behavior of the CML depends on two parameters, the coupling strength and the nonlinear parameter α. These parameters span a parameter space in which different regions represent different types of spatio–temporal behavior. In this paper, we focus on the parameter . Based on this description of a spatio–temporal dynamical system, we formulate the spatio–temporal fitness landscape. Therefore, we project the lattice
Evolutionary Optimization in Spatio–temporal Fitness Landscapes
3
1
h(i,j,k)
h(x,k)
0.8 0.6 0.4 0.2 0 24
J
16
j+1
24
i−1 i 1
16
8
I
j j−1
i+1
8
0
0
x2
1
a)
x1
b)
Fig. 1. Coupled map lattice: a) Generic structure. b) Spatio–temporal fitness landscape for I = J = 9 and s1 = s2 = 0.5.
cells on a zero plane and introduce scaling factors s1 , s2 ∈ R for the vertical and horizontal extension. By these scaling factors, we convert the integer search space to a real value search space with step function characteristics, see Fig. 1b. The si can additionally be used to adjust severity of the dynamic optimization T problem to be solved. Next, we define the search space variable x = (x1 , x2 ) , T T impose a rounding condition, so that s1 x1 , s2 x2 = i, j and find the spatio–temporal fitness function for the two–dimensional CML (2): ⎧ ⎨
f (x, k) =
⎩
h(s1 x1 , s2 x2 , k)
f or
0
⎫ 1 ≤ s1 x1 ≤ I ⎬ 1 ≤ s2 x2 ≤ J , k ≥ 0. ⎭ otherwise
(5)
The dynamic optimization problem is max f (x, k) =
x∈R2
max
1≤s1 x1 ≤I 1≤s2 x2 ≤J
h(s1 x1 , s2 x2 , k), k ≥ 0
(6)
and solving it yields the solution trajectory xS (k) = arg max f (x, k) = arg x∈R2
max
1≤s1 x1 ≤I 1≤s2 x2 ≤J
h(s1 x1 , s2 x2 , k), k ≥ 0,
(7)
which we intend to find using an evolutionary algorithm. The given construction of spatio–temporal fitness landscapes has the advantage that spatial topology and temporal dynamics are generated by the same system. No external driving system for inducing dynamics is required. The given formulation of a spatio–temporal fitness landscape is valid for a two–dimensional search space. An extension to n–dimensional search spaces is possible using the given framework and employing n–dimensional CML, see e.g. [13]. In addition, we only considered flat heights on the lattice cells. A generalization can be achieved by introducing morphological structures on the cell tops.
3
Properties of Spatio–temporal Fitness Landscapes
In order to evaluate the performance of an evolutionary algorithm in a spatio– temporal fitness landscape, we need a notion of how difficult a certain landscape
4
H. Richter
is to optimize in. This question is addressed by concepts and quantifiers for measuring fitness landscapes. For static landscapes, this topic has been studied intensively, e.g. [18,4,16]. In the following, we adopt and modify these results to measure spatio–temporal fitness landscapes. First, we look at the modality of the landscape by measuring the number of local maxima. Despite the continuous search space of the spatio–temporal fitness landscape (5), the local maxima need to be described within the framework of local optima in discrete search spaces. A local optimum is a solution optimal within a neighboring set of solutions, which is the same as in combinatorial optimization problems [10], p. 7. The neighborhood structure we consider here are the surrounding heights. That means the neighborhood structure N (i, j) of the (i, j)–th lattice cell is N (i, j) = (i + β, j + δ), (β, δ) = (−1, −1) ∧ (−1, 0) ∧ (−1, 1) ∧ (0, −1) ∧ (0, 1) ∧ (1, −1) ∧ (1, 0) ∧ (1, 1). T
Here, (i, j)T = (s1 x1 , s2 x2 ) . Hence, the fitness function possesses a local maximum at the time k if h(i, j, k) ≥ h(N (i, j), k).
(8)
Due to the countable number of lattice cells, the number of local maxima can be determined by enumeration. We denote #LM (k) the number of local maxima at time k. As a spatio–temporal fitness landscape changes over time, the number of local maxima needs not to be constant. Therefore, we consider its time average: K−1 1 #LM (k). K→∞ K
#LM (k) = lim
(9)
k=0
To get an approximate value of the time average number of local maxima, the K−1
1 #LM (k) is replaced by #LM = K #LM (k) with K sufficiently large. In k=0
Figs. 2a,c,e the average number of maxima is given for different and different lattice sizes. Here, as well as in the following experiments, we fix α = 3.999 and consider varying . We see that the number of maxima increases steadily with increasing lattice size and that this occurs symmetrically in the vertical as well as in the horizontal direction identified by I and J. For smaller , this increase is steeper than for larger values. On the other hand, for varying there is no clear trend as to how the number of maxima scales with it, but there are also sections in the –parameter space (for instance 0.3 ≤ ≤ 0.9) for which a proportional relation can be observed. Next, we determine the ruggedness of the spatio–temporal fitness landscape modelled by the CML. A standard procedure to assert ruggedness of fitness landscapes is to analyze its correlation structure. This method works by performing a random walk on the landscape and calculating its random walk correlation function. For the spatio–temporal fitness landscape (5), this starts with generating a time series h(τ, k) = h(i(τ ), j(τ ), k), τ = 1, 2, . . . , T , of T the heights h(s1 x1 , s2 x2 , k) with (i, j)T = (s1 x1 , s2 x2 ) . For performing the random walk, we create two times T independent realizations of an integer random variable uniformly distributed on [−1, 1]. Starting from an initial cell, the next cell on the walk is obtained by adding the two independent realizations
Evolutionary Optimization in Spatio–temporal Fitness Landscapes 50
4
45
3.5
40
3
I = J = 20 I = J = 16 I = J = 12
35 30
2.5
λ
#LM
5
2
25 1.5 20
I = J = 20 I = J = 16 I = J = 12
1
15
0.5
10 5 0
0.2
0.4
ε
0.6
0.8
0 0
1
0.2
0.4
ε
1
b)
100
1
80
0.8
60
0.6
40
0.4
20
0.2
0 30
0 30 30
25
10
20
15
15 10
J
25
20
20
15
30
25
25
20
15 10
J
I
c)
10
= 0.05
100
1.8
80
1.6
60
1.4
I
d)
λ
#LM
0.8
λ
#LM
a)
0.6
40
1.2
20
1
0 30
0.8 30 30
25
J
10
20
15
15 10
25
20
20
15
30
25
25
20
15 10
J
I
e)
= 0.95
10
I
f)
Fig. 2. Average number of local maxima #LM and average correlation length λ with α = 3.999 for K = 1500 and T = 100000
of the random variable to the current cell index (i, j). In addition, the boundary condition (4) is observed. The random walk in the two spatial dimensions as specified by i(τ ), j(τ ) yields the needed time series on the spatio–temporal fitness landscape by recording the heights h(τ, k) = h(i(τ ), j(τ ), k) at time k. For this time series the spatial correlation can be calculated. The spatial correlation is widely used in determining ruggedness of static landscapes [18,4,15]. It is an estimate r(tL , k) of the autocorrelation function of the time series with time lag tL , also called random walk correlation function: T −tL
r(tL , k) =
¯ h(τ, k) − h(k)
¯ h(τ + tL , k) − h(k)
τ =1 T
τ =1
¯ h(τ, k) − h(k)
2
,
(10)
6
H. Richter 50
12
45
11
40
9
σ
30
#LM
10
I = J = 20 I = J = 16 I = J = 12
35
25 20
8 7
15 6 10
0 0
I = J = 20 I = J = 16 I = J = 12
5
5 0.5
1
1.5
2
λ
2.5
3
3.5
4
4 0
0.2
a)
0.4
ε
0.6
0.8
1
b)
Fig. 3. a) Correlation between number of maxima and autocorrelation. b) Severity σ for (i, j)T = (x1 , x2 )T .
¯ where h(k) =
1 T
T
h(τ, k) and T tL > 0. The spatial random walk correlation
τ =1
function measures the correlation between different regions of the fitness landscape for a fixed k. As r(tL , k) changes over time, we consider its time average r(tL , k), for which we calculate numerically an approximated value r(tL ), similarly as for the average number of maxima. It has been shown that ruggedness is best reflected by the correlation length [15] λ = −1/ ln (|r(1)|).
(11)
This quantity is given for the spatio–temporal fitness landscape, see Figs. 2b,d,f. The lower the values of λ are, the more rugged the landscape is. For varying , the correlation length scales in a similar manner as the number of maxima, compare Fig. 2a. Increasing lattice sizes lead to a small increase of λ only, see Figs. 2d,f. Also, for smaller values of , the correlation length is distributed smoother on the I − J-lattice size plane. Fig. 3a shows ruggedness measured by the correlation length versus modality measured by the number of maxima. This also serves as a test on the relationship between modality and ruggedness. The results show that there is an exponential dependency between correlation length λ and the number of maxima #LM , which is also clearly distinct for different lattice sizes.
4
Problem Hardness in Dynamic Optimization
Optimization in spatio–temporal fitness landscapes is in essence dynamic optimization. In dynamic optimization, it is generally understood that problem hardness depends not only on the problem difficulty of the landscape but also on features of the involved dynamics [2,9]. The most prominent features are the (relative) speed of the landscape changes, which is expressed by change frequency and the (relative) strength of the landscape changes, which can be attributed by dynamic severity. Both quantities are briefly recalled. The optimization problem (6) can be solved by an evolutionary algorithm with real number representation and μ individuals a ∈ R2 , which build the population P ∈ R2×μ . The dynamics
Evolutionary Optimization in Spatio–temporal Fitness Landscapes
7
Table 1. Settings of the evolutionary algorithm and the numerical experiments Design parameter Population size Initial population width Base–mutation rate Hyper–mutation rate
Symbol Value μ 50 5 ω2 bm 0.1 hm 30
Setting Symbol Number of considered runs R Generations in a run T Eq. (3) α Severity scaling s = s1 = s2
Value 150 1500 3.999 1
of such an evolutionary algorithm can be described by the generation transition function ψ : R2×μ → R2×μ , see e.g. [1], pp. 64–65. The generation transition function includes the genetic operators selection, recombination and mutation. It generates the movement of the population within the fitness landscape by transforming a population at generation t ∈ N0 into a population at generation t + 1, P (t + 1) = ψ (P (t)) ,
t ≥ 0.
(12)
Here, t represents a discrete time variable, as k does for the spatio–temporal fitness function (5). In dynamic optimization, both time scales can be related to each other by the environmental change period γ ∈ N (cf. [11]). Between the frequency of the changes in the population and that of the dynamic fitness landscape, we find t = γk.
(13)
A second prominent factor in problem hardness of dynamic optimization is dynamic severity [17,12], which measures the (relative) magnitude of the changes in the landscape by comparing k to k + 1. In terms of the spatio–temporal fitness landscape considered here, severity means to evaluate the distance between the largest height before and after a change. Hence, severity can be calculated σ(k + 1) = xS (k + 1) − xS (k) ,
(14)
where xS (k) is the solution of the dynamic optimization problem (7). As this quantity may vary with time k, we focus on the time average severity σ(k) = K−1
1 σ(k). For the solution trajectory (7), we obtain σ(k) = σ· s21 + s22 , lim K K→∞
k=0
where σ is severity for (i, j)T = (x1 , x2 )T . The quantity σ is shown in Fig. 3b and can be regarded as almost constant for a given lattice size and a large majority of values of . Hence, severity of the optimization problem can be adjusted by s1 and s2 .
5
Evolutionary Optimization
We now give experimental results on evolutionary optimization in spatio–temporal fitness landscapes modelled by CML. The performance of the evolutionary algorithm is examined depending on problem difficulty expressed by change frequency, severity and landscape measures as modality and ruggedness. The numerical experiments have been conducted with an evolutionary algorithm that
8
H. Richter 0.1
0.1
0.09
γ = 10
0.09
γ = 10
0.08
γ = 20
0.08
γ = 20
0.07
γ = 30
0.07
γ = 30
0.06
MFE
MFE
0.06 0.05
0.05
0.04
0.04
0.03
0.03
0.02
0.02
0.01 0 20
0.01 40
60
μ
80
a)
100
120
0 0
0.2
0.4
0.6
0.8
1
1.2
s
b)
Fig. 4. Performance specified by the MFE with = 0.5 and I = J = 16 for different γ versus: a) Population size μ for severity scaling s = 1. b) Severity scaling s for μ = 50
uses real number representation, fixed population size, tournament selection with tournament size 2, fitness–based recombination and base– as well as hyper– mutation. The initial population is created by realizations of a random variable normally distributed on [0, ω 2 ]. Tab. 1 summarizes the design parameters of the algorithm together with settings for the numerical experiments. The performance of the algorithm is measured by the Mean Fitness Error (MFE)
MF E =
R T 1 1 , f (xS , t/γ) − max f (a, t/γ) a∈P R r=1 T t=1
(15)
where f (xs , t/γ) is the maximum fitness value at generation t, max f (a, t/γ) is a∈P
the fitness value of the best individual a ∈ P at generation t, T is the number of generations used in the run, and R is the number of consecutive runs. In a first experiment, we look at the design parameter population size μ and its scaling with the MFE, see Fig. 4a. Here, as well as in the following figures the MFE together with its 95% confidence interval is given. We see an exponential decrease of the MFE with increasing μ and a clear distinction between different environmental change periods γ, which is a typical result for dynamic optimization [9,11]. Next, the MFE depending on dynamic severity for s = s1 = s2 is given, see Fig. 4b. We observe a steep increase of the MFE for values of s getting smaller, which means for larger dynamic severity. Again, this is in agreement with previous findings [17,12]. In a next set of experiments, we study how the landscape measures modality and ruggedness relate to the MFE, see Fig. 5. These results stem from calculating the MFE for different values of , given in Fig. 5a. The results are the MFE for 150 runs and again the 95% confidence intervals are shown. Note that the intervals are small so that this MFE represents a good approximation of the long–term behavior. In the Figs. 5b,c the MFE is shown depending on the number of maxima #LM and the correlation length λ. Within certain bounds of #LM and λ, we observe that the MFE gets gradually smaller with an increasing number of maxima, while it gets larger with increasing correlation length. This trend is
Evolutionary Optimization in Spatio–temporal Fitness Landscapes 0.08
0.08
γ = 10
0.07
γ = 10
0.07
γ = 20
γ = 20 γ = 30
0.05
0.05
0.04
0.04
0.03
0.03
0.02
0.02
0.01 0
0.2
γ = 30
0.06
MFE
MFE
0.06
0.4
0.6
ε
0.8
0.01 5
1
10
15
20
25
30
#LM
a)
b)
0.08
1
γ = 10
0.07
γ = 20
0.9
ρ(MFE,#LM)
0.8
ρ(MFE,λ)
γ = 30
0.06
0.05
ρ
MFE
9
0.7
0.04 0.6 0.03 0.5
0.02
0.01 0
0.5
1
1.5
λ
c)
2
2.5
3
3.5
0.4 5
10
15
20
γ
25
30
35
40
d)
Fig. 5. The MFE as a function of: a) Diffusion coupling strength . b) Number of maxima #LM . c) Correlation length λ. d) Correlation between the MFE and #LM , λ.
particularly visible for larger environmental change periods γ. Also, within these bounds, γ is the leading factor in performance. For very large values of #LM and very small values of λ, this relation vanishes. Finally, we consider the correlation between the MFE and #LM , denoted by ρ(M F E, #LM ), and between the MFE and λ, denoted by ρ(M F E, λ), respectively, depending on γ. We notice that both correlations decrease with increasing γ. This indicates that the correlation between the landscape measures and the performance gets weaker for increasing γ. For change frequency being in general the leading factor in the algorithm’s performance, this can be interpreted as follows: a large number of maxima and a high ruggedness does not influence the performance strongly for the algorithm having enough time to search for the maxima. On the other hand, for small γ, these landscape measures clearly determine the performance.
6
Conclusions
In this paper, spatio–temporal fitness landscapes constructed from Coupled Map Lattices were introduced. These dynamic landscapes were analyzed in terms of modality (number of optima) and ruggedness (correlation structure). Based on these results, the relationship between landscape measures and the performance of the evolutionary algorithm used to solve the dynamic optimization problem was studied.
10
H. Richter
The results show that the landscape measures modality and ruggedness scale with the algorithm’s performance and hence allow its prediction. In particular, for small change frequencies, we find a strong correlation between the performance and landscape measures, that is, modality and ruggedness. For larger change frequencies, this correlation ceases as the algorithm is more likely to find the optimum despite the problem hardness induced by the landscape.
References 1. B¨ ack, T.: Evolutionary Algorithms in Theory and Practice: Evolution Strategies, Evolutionary Programming, Genetic Algorithms, Oxford Univ. Press, NY (1996). 2. Branke, J.: Evolutionary Optimization in Dynamic Environments, Kluwer Academic Publishers, Dordrecht (2001). 3. Chazottes, J.R., Fernandez, B.: Dynamics of Coupled Map Lattices and of Related Spatially Extended Systems, Springer–Verlag, Berlin (2005). 4. Hordijk, W.: A Measure of Landscapes. Evolut. Comput. 4 (1996) 335–360. 5. Hordijk, W., Kauffman S.A.: Correlation Analysis of Coupled Fitness Landscapes. Complexity 10 (2005) 42–49. 6. Jin, Y., Branke, J.: Evolutionary Optimization in Uncertain Environments–A Survey. IEEE Trans. Evolut. Comput. 9 (2005) 303–317. 7. Kallel, L., Naudts, B., Reeves, C.R.: Properties of Fitness Functions and Search Landscapes. In: Kallel, L. et al. (eds.): Theoretical Aspects of Evolutionary Computing, Springer–Verlag, Berlin (2001) 177–208. 8. Kaneko, K.: The Coupled Map Lattice. In: Kaneko, K. (ed.): Theory and Application of Coupled Map Lattices, John Wiley, Chichester (1993) 1–49. 9. Morrison, R.W.: Designing Evolutionary Algorithms for Dynamic Environments, Springer–Verlag, Berlin (2004). 10. Papadimitriou, C.H., Steiglitz, K.: Combinatorial Optimization: Algorithms and Complexity, Dover, Mineola, NY (1998). 11. Richter, H.: Behavior of Evolutionary Algorithms in Chaotically Changing Fitness Landscapes. In: Yao, X. et al. (eds.): Parallel Problem Solving from Nature–PPSN VIII, Springer–Verlag, Berlin (2004) 111–120. 12. Richter, H.: A Study of Dynamic Severity in Chaotic Fitness Landscapes. In: Corne, D. (ed.): Proc. Congress on Evolutionary Computation, IEEE CEC 2005, IEEE Press, Piscataway, NJ (2005) 2824–2831. 13. Shibata, T., Kaneko, K.: Coupled Map Gas: Structure Formation and Dynamics of Interacting Mobile Elements with Internal Dynamics. Phys. D181 (2003) 197–214. 14. Smith, T., Husbands, P., Layzell, P., O’Shea, M.: Fitness Landscapes and Evolvability. Evolut. Comput. 10 (2002) 1–34. 15. Stadler, P.F.: Landscapes and Their Correlation Functions. J. Math. Chem. 20 (1996) 1–45. 16. Stadler, P.F., Stephens, C.R.: Landscapes and Effective Fitness. Comm. Theor. Biol. 8 (2003) 389–431. 17. Weicker, K.: An Analysis of Dynamic Severity and Population Size. In: Schoenauer, M. et al. (eds.): Parallel Problem Solving from Nature–PPSN VI, Springer–Verlag, Berlin (2000) 159–168. 18. Weinberger, E.D.: Correlated and Uncorrelated Fitness Landscapes and How to Tell the Difference. Biol. Cybern. 63 (1990) 325–336. 19. Yang, S., Yao, X.: Experimental Study on Population-based Incremental Learning Algorithms for Dynamic Optimization Problems. Soft Comput. 9 (2005) 815–834.
Abstract. Spatio–temporal fitness landscapes that are constructed from Coupled Map Lattices (CML) are introduced. These landscapes are analyzed in terms of modality and ruggedness. Based on this analysis, we study the relationship between landscape measures and the performance of an evolutionary algorithm used to solve the dynamic optimization problem.
1
Introduction
In theoretical studies of evolutionary algorithms, concepts of fitness landscapes have shown to be a useful tool [4,7,14,16]. A fitness landscape assigns fitness values to the points of the search space through which the evolution moves. This search space can be constructed by a genotype–to–fitness mapping or more generally by encoding the set of possible solutions of an optimization problem to form a representation space for which additionally a neighborhood structure needs to be defined. Typically, the topology of the search space is considered to be constant over the run–time of the evolutionary algorithm and hence such a fitness landscape is a static concept. As dynamic optimization turned more and more into an important topic in evolutionary computation [17,2,9,6,19], it became desirable to have concepts of fitness landscapes in dynamic environments as well. Whereas recent results have shown that such an approach is useful to characterize and classify types of dynamic landscapes [5], there is a certain lack of environments that show sufficiently complex structure in both spatial topology and temporal dynamics. In this paper, such spatio–temporal fitness landscapes are introduced and it is shown how these landscapes can be constructed from spatio–temporal dynamical systems, namely from Coupled Map Lattices (CML). CML have been the subject of intensive research, which revealed a broad variety of spatio–temporal behavior, including different types of pattern formation, spatio–temporal chaos, quasi– periodicity and emergence [8,3]. In the next section, a methodology to construct spatio–temporal fitness landscapes from CML is given. In Sec. 3, these landscapes are analyzed in terms of modality and ruggedness. Further, quantifying measures such as the number of T.P. Runarsson et al. (Eds.): PPSN IX, LNCS 4193, pp. 1–10, 2006. c Springer-Verlag Berlin Heidelberg 2006
2
H. Richter
optima and the correlation structure are studied. In Sec. 4, factors contributing to problem hardness in dynamic optimization like change frequency and dynamic severity are reviewed. Numerical experiments using an evolutionary algorithm to optimize in the spatio–temporal fitness landscape are reported in Sec. 5. In the final section, the findings are summarized and conclusions are drawn.
2
Constructing Spatio–temporal Fitness Landscapes
Constructing spatio–temporal fitness landscapes should begin with defining a spatio–temporal dynamical system. Therefore, we lay out a lattice grid with I × J equally sized cells, which form a 2D–structure. For every discrete time step k, k = 0, 1, 2, . . ., each cell is characterized by its height h(i, j, k),
i = 1, 2, . . . , I,
j = 1, 2, . . . , J,
(1)
where i, j denote the spatial indices in vertical and horizontal direction, respectively, see Fig. 1. This height h(i, j, k), which we will interpret as fitness according to the geometrical metaphor of a fitness landscape, is subject to changes over time, which are described by the two–dimensional CML with nearest–neighbor coupled interaction [8,3] h(i, j, k + 1) = (1 − )g(h(i, j, k)) +
g (h(i − 1, j, k)) 4
+ g (h(i + 1, j, k)) + g (h(i, j − 1, k)) +g (h(i, j + 1, k))
,
(2)
where g(h(i, j, k)) is a local mapping function and is the diffusion coupling strength. In other words, as h(i, j, k) denotes the height of the h(i, j)–th unit bar situated in the (i, j)–lattice cell at time step k, Eq. (2) describes how this height changes over time depending on its own height and the heights of the surrounding bars, see Fig. 1a. The CML is initialized by the heights h(i, j, 0) being realizations of a random variable uniformly distributed on [0, 1]. To complete the definition of the CML (2), we employ the logistic map g(h(i, j, k)) = αh(i, j, k)(1 − h(i, j, k))
(3)
as mapping function and render the period boundary conditions h(I + 1, j, k) = h(1, j, k),
h(i, J + 1, k) = h(i, 1, k).
(4)
To summarize, the CML is a spatio–temporal dynamical system with discrete space (lattice) and time (map). The system’s states, which we interpret as heights, are continuous and situated on the lattice. They dynamically interact with surrounding states via the nonlinear map (3). The spatio–temporal behavior of the CML depends on two parameters, the coupling strength and the nonlinear parameter α. These parameters span a parameter space in which different regions represent different types of spatio–temporal behavior. In this paper, we focus on the parameter . Based on this description of a spatio–temporal dynamical system, we formulate the spatio–temporal fitness landscape. Therefore, we project the lattice
Evolutionary Optimization in Spatio–temporal Fitness Landscapes
3
1
h(i,j,k)
h(x,k)
0.8 0.6 0.4 0.2 0 24
J
16
j+1
24
i−1 i 1
16
8
I
j j−1
i+1
8
0
0
x2
1
a)
x1
b)
Fig. 1. Coupled map lattice: a) Generic structure. b) Spatio–temporal fitness landscape for I = J = 9 and s1 = s2 = 0.5.
cells on a zero plane and introduce scaling factors s1 , s2 ∈ R for the vertical and horizontal extension. By these scaling factors, we convert the integer search space to a real value search space with step function characteristics, see Fig. 1b. The si can additionally be used to adjust severity of the dynamic optimization T problem to be solved. Next, we define the search space variable x = (x1 , x2 ) , T T impose a rounding condition, so that s1 x1 , s2 x2 = i, j and find the spatio–temporal fitness function for the two–dimensional CML (2): ⎧ ⎨
f (x, k) =
⎩
h(s1 x1 , s2 x2 , k)
f or
0
⎫ 1 ≤ s1 x1 ≤ I ⎬ 1 ≤ s2 x2 ≤ J , k ≥ 0. ⎭ otherwise
(5)
The dynamic optimization problem is max f (x, k) =
x∈R2
max
1≤s1 x1 ≤I 1≤s2 x2 ≤J
h(s1 x1 , s2 x2 , k), k ≥ 0
(6)
and solving it yields the solution trajectory xS (k) = arg max f (x, k) = arg x∈R2
max
1≤s1 x1 ≤I 1≤s2 x2 ≤J
h(s1 x1 , s2 x2 , k), k ≥ 0,
(7)
which we intend to find using an evolutionary algorithm. The given construction of spatio–temporal fitness landscapes has the advantage that spatial topology and temporal dynamics are generated by the same system. No external driving system for inducing dynamics is required. The given formulation of a spatio–temporal fitness landscape is valid for a two–dimensional search space. An extension to n–dimensional search spaces is possible using the given framework and employing n–dimensional CML, see e.g. [13]. In addition, we only considered flat heights on the lattice cells. A generalization can be achieved by introducing morphological structures on the cell tops.
3
Properties of Spatio–temporal Fitness Landscapes
In order to evaluate the performance of an evolutionary algorithm in a spatio– temporal fitness landscape, we need a notion of how difficult a certain landscape
4
H. Richter
is to optimize in. This question is addressed by concepts and quantifiers for measuring fitness landscapes. For static landscapes, this topic has been studied intensively, e.g. [18,4,16]. In the following, we adopt and modify these results to measure spatio–temporal fitness landscapes. First, we look at the modality of the landscape by measuring the number of local maxima. Despite the continuous search space of the spatio–temporal fitness landscape (5), the local maxima need to be described within the framework of local optima in discrete search spaces. A local optimum is a solution optimal within a neighboring set of solutions, which is the same as in combinatorial optimization problems [10], p. 7. The neighborhood structure we consider here are the surrounding heights. That means the neighborhood structure N (i, j) of the (i, j)–th lattice cell is N (i, j) = (i + β, j + δ), (β, δ) = (−1, −1) ∧ (−1, 0) ∧ (−1, 1) ∧ (0, −1) ∧ (0, 1) ∧ (1, −1) ∧ (1, 0) ∧ (1, 1). T
Here, (i, j)T = (s1 x1 , s2 x2 ) . Hence, the fitness function possesses a local maximum at the time k if h(i, j, k) ≥ h(N (i, j), k).
(8)
Due to the countable number of lattice cells, the number of local maxima can be determined by enumeration. We denote #LM (k) the number of local maxima at time k. As a spatio–temporal fitness landscape changes over time, the number of local maxima needs not to be constant. Therefore, we consider its time average: K−1 1 #LM (k). K→∞ K
#LM (k) = lim
(9)
k=0
To get an approximate value of the time average number of local maxima, the K−1
1 #LM (k) is replaced by #LM = K #LM (k) with K sufficiently large. In k=0
Figs. 2a,c,e the average number of maxima is given for different and different lattice sizes. Here, as well as in the following experiments, we fix α = 3.999 and consider varying . We see that the number of maxima increases steadily with increasing lattice size and that this occurs symmetrically in the vertical as well as in the horizontal direction identified by I and J. For smaller , this increase is steeper than for larger values. On the other hand, for varying there is no clear trend as to how the number of maxima scales with it, but there are also sections in the –parameter space (for instance 0.3 ≤ ≤ 0.9) for which a proportional relation can be observed. Next, we determine the ruggedness of the spatio–temporal fitness landscape modelled by the CML. A standard procedure to assert ruggedness of fitness landscapes is to analyze its correlation structure. This method works by performing a random walk on the landscape and calculating its random walk correlation function. For the spatio–temporal fitness landscape (5), this starts with generating a time series h(τ, k) = h(i(τ ), j(τ ), k), τ = 1, 2, . . . , T , of T the heights h(s1 x1 , s2 x2 , k) with (i, j)T = (s1 x1 , s2 x2 ) . For performing the random walk, we create two times T independent realizations of an integer random variable uniformly distributed on [−1, 1]. Starting from an initial cell, the next cell on the walk is obtained by adding the two independent realizations
Evolutionary Optimization in Spatio–temporal Fitness Landscapes 50
4
45
3.5
40
3
I = J = 20 I = J = 16 I = J = 12
35 30
2.5
λ
#LM
5
2
25 1.5 20
I = J = 20 I = J = 16 I = J = 12
1
15
0.5
10 5 0
0.2
0.4
ε
0.6
0.8
0 0
1
0.2
0.4
ε
1
b)
100
1
80
0.8
60
0.6
40
0.4
20
0.2
0 30
0 30 30
25
10
20
15
15 10
J
25
20
20
15
30
25
25
20
15 10
J
I
c)
10
= 0.05
100
1.8
80
1.6
60
1.4
I
d)
λ
#LM
0.8
λ
#LM
a)
0.6
40
1.2
20
1
0 30
0.8 30 30
25
J
10
20
15
15 10
25
20
20
15
30
25
25
20
15 10
J
I
e)
= 0.95
10
I
f)
Fig. 2. Average number of local maxima #LM and average correlation length λ with α = 3.999 for K = 1500 and T = 100000
of the random variable to the current cell index (i, j). In addition, the boundary condition (4) is observed. The random walk in the two spatial dimensions as specified by i(τ ), j(τ ) yields the needed time series on the spatio–temporal fitness landscape by recording the heights h(τ, k) = h(i(τ ), j(τ ), k) at time k. For this time series the spatial correlation can be calculated. The spatial correlation is widely used in determining ruggedness of static landscapes [18,4,15]. It is an estimate r(tL , k) of the autocorrelation function of the time series with time lag tL , also called random walk correlation function: T −tL
r(tL , k) =
¯ h(τ, k) − h(k)
¯ h(τ + tL , k) − h(k)
τ =1 T
τ =1
¯ h(τ, k) − h(k)
2
,
(10)
6
H. Richter 50
12
45
11
40
9
σ
30
#LM
10
I = J = 20 I = J = 16 I = J = 12
35
25 20
8 7
15 6 10
0 0
I = J = 20 I = J = 16 I = J = 12
5
5 0.5
1
1.5
2
λ
2.5
3
3.5
4
4 0
0.2
a)
0.4
ε
0.6
0.8
1
b)
Fig. 3. a) Correlation between number of maxima and autocorrelation. b) Severity σ for (i, j)T = (x1 , x2 )T .
¯ where h(k) =
1 T
T
h(τ, k) and T tL > 0. The spatial random walk correlation
τ =1
function measures the correlation between different regions of the fitness landscape for a fixed k. As r(tL , k) changes over time, we consider its time average r(tL , k), for which we calculate numerically an approximated value r(tL ), similarly as for the average number of maxima. It has been shown that ruggedness is best reflected by the correlation length [15] λ = −1/ ln (|r(1)|).
(11)
This quantity is given for the spatio–temporal fitness landscape, see Figs. 2b,d,f. The lower the values of λ are, the more rugged the landscape is. For varying , the correlation length scales in a similar manner as the number of maxima, compare Fig. 2a. Increasing lattice sizes lead to a small increase of λ only, see Figs. 2d,f. Also, for smaller values of , the correlation length is distributed smoother on the I − J-lattice size plane. Fig. 3a shows ruggedness measured by the correlation length versus modality measured by the number of maxima. This also serves as a test on the relationship between modality and ruggedness. The results show that there is an exponential dependency between correlation length λ and the number of maxima #LM , which is also clearly distinct for different lattice sizes.
4
Problem Hardness in Dynamic Optimization
Optimization in spatio–temporal fitness landscapes is in essence dynamic optimization. In dynamic optimization, it is generally understood that problem hardness depends not only on the problem difficulty of the landscape but also on features of the involved dynamics [2,9]. The most prominent features are the (relative) speed of the landscape changes, which is expressed by change frequency and the (relative) strength of the landscape changes, which can be attributed by dynamic severity. Both quantities are briefly recalled. The optimization problem (6) can be solved by an evolutionary algorithm with real number representation and μ individuals a ∈ R2 , which build the population P ∈ R2×μ . The dynamics
Evolutionary Optimization in Spatio–temporal Fitness Landscapes
7
Table 1. Settings of the evolutionary algorithm and the numerical experiments Design parameter Population size Initial population width Base–mutation rate Hyper–mutation rate
Symbol Value μ 50 5 ω2 bm 0.1 hm 30
Setting Symbol Number of considered runs R Generations in a run T Eq. (3) α Severity scaling s = s1 = s2
Value 150 1500 3.999 1
of such an evolutionary algorithm can be described by the generation transition function ψ : R2×μ → R2×μ , see e.g. [1], pp. 64–65. The generation transition function includes the genetic operators selection, recombination and mutation. It generates the movement of the population within the fitness landscape by transforming a population at generation t ∈ N0 into a population at generation t + 1, P (t + 1) = ψ (P (t)) ,
t ≥ 0.
(12)
Here, t represents a discrete time variable, as k does for the spatio–temporal fitness function (5). In dynamic optimization, both time scales can be related to each other by the environmental change period γ ∈ N (cf. [11]). Between the frequency of the changes in the population and that of the dynamic fitness landscape, we find t = γk.
(13)
A second prominent factor in problem hardness of dynamic optimization is dynamic severity [17,12], which measures the (relative) magnitude of the changes in the landscape by comparing k to k + 1. In terms of the spatio–temporal fitness landscape considered here, severity means to evaluate the distance between the largest height before and after a change. Hence, severity can be calculated σ(k + 1) = xS (k + 1) − xS (k) ,
(14)
where xS (k) is the solution of the dynamic optimization problem (7). As this quantity may vary with time k, we focus on the time average severity σ(k) = K−1
1 σ(k). For the solution trajectory (7), we obtain σ(k) = σ· s21 + s22 , lim K K→∞
k=0
where σ is severity for (i, j)T = (x1 , x2 )T . The quantity σ is shown in Fig. 3b and can be regarded as almost constant for a given lattice size and a large majority of values of . Hence, severity of the optimization problem can be adjusted by s1 and s2 .
5
Evolutionary Optimization
We now give experimental results on evolutionary optimization in spatio–temporal fitness landscapes modelled by CML. The performance of the evolutionary algorithm is examined depending on problem difficulty expressed by change frequency, severity and landscape measures as modality and ruggedness. The numerical experiments have been conducted with an evolutionary algorithm that
8
H. Richter 0.1
0.1
0.09
γ = 10
0.09
γ = 10
0.08
γ = 20
0.08
γ = 20
0.07
γ = 30
0.07
γ = 30
0.06
MFE
MFE
0.06 0.05
0.05
0.04
0.04
0.03
0.03
0.02
0.02
0.01 0 20
0.01 40
60
μ
80
a)
100
120
0 0
0.2
0.4
0.6
0.8
1
1.2
s
b)
Fig. 4. Performance specified by the MFE with = 0.5 and I = J = 16 for different γ versus: a) Population size μ for severity scaling s = 1. b) Severity scaling s for μ = 50
uses real number representation, fixed population size, tournament selection with tournament size 2, fitness–based recombination and base– as well as hyper– mutation. The initial population is created by realizations of a random variable normally distributed on [0, ω 2 ]. Tab. 1 summarizes the design parameters of the algorithm together with settings for the numerical experiments. The performance of the algorithm is measured by the Mean Fitness Error (MFE)
MF E =
R T 1 1 , f (xS , t/γ) − max f (a, t/γ) a∈P R r=1 T t=1
(15)
where f (xs , t/γ) is the maximum fitness value at generation t, max f (a, t/γ) is a∈P
the fitness value of the best individual a ∈ P at generation t, T is the number of generations used in the run, and R is the number of consecutive runs. In a first experiment, we look at the design parameter population size μ and its scaling with the MFE, see Fig. 4a. Here, as well as in the following figures the MFE together with its 95% confidence interval is given. We see an exponential decrease of the MFE with increasing μ and a clear distinction between different environmental change periods γ, which is a typical result for dynamic optimization [9,11]. Next, the MFE depending on dynamic severity for s = s1 = s2 is given, see Fig. 4b. We observe a steep increase of the MFE for values of s getting smaller, which means for larger dynamic severity. Again, this is in agreement with previous findings [17,12]. In a next set of experiments, we study how the landscape measures modality and ruggedness relate to the MFE, see Fig. 5. These results stem from calculating the MFE for different values of , given in Fig. 5a. The results are the MFE for 150 runs and again the 95% confidence intervals are shown. Note that the intervals are small so that this MFE represents a good approximation of the long–term behavior. In the Figs. 5b,c the MFE is shown depending on the number of maxima #LM and the correlation length λ. Within certain bounds of #LM and λ, we observe that the MFE gets gradually smaller with an increasing number of maxima, while it gets larger with increasing correlation length. This trend is
Evolutionary Optimization in Spatio–temporal Fitness Landscapes 0.08
0.08
γ = 10
0.07
γ = 10
0.07
γ = 20
γ = 20 γ = 30
0.05
0.05
0.04
0.04
0.03
0.03
0.02
0.02
0.01 0
0.2
γ = 30
0.06
MFE
MFE
0.06
0.4
0.6
ε
0.8
0.01 5
1
10
15
20
25
30
#LM
a)
b)
0.08
1
γ = 10
0.07
γ = 20
0.9
ρ(MFE,#LM)
0.8
ρ(MFE,λ)
γ = 30
0.06
0.05
ρ
MFE
9
0.7
0.04 0.6 0.03 0.5
0.02
0.01 0
0.5
1
1.5
λ
c)
2
2.5
3
3.5
0.4 5
10
15
20
γ
25
30
35
40
d)
Fig. 5. The MFE as a function of: a) Diffusion coupling strength . b) Number of maxima #LM . c) Correlation length λ. d) Correlation between the MFE and #LM , λ.
particularly visible for larger environmental change periods γ. Also, within these bounds, γ is the leading factor in performance. For very large values of #LM and very small values of λ, this relation vanishes. Finally, we consider the correlation between the MFE and #LM , denoted by ρ(M F E, #LM ), and between the MFE and λ, denoted by ρ(M F E, λ), respectively, depending on γ. We notice that both correlations decrease with increasing γ. This indicates that the correlation between the landscape measures and the performance gets weaker for increasing γ. For change frequency being in general the leading factor in the algorithm’s performance, this can be interpreted as follows: a large number of maxima and a high ruggedness does not influence the performance strongly for the algorithm having enough time to search for the maxima. On the other hand, for small γ, these landscape measures clearly determine the performance.
6
Conclusions
In this paper, spatio–temporal fitness landscapes constructed from Coupled Map Lattices were introduced. These dynamic landscapes were analyzed in terms of modality (number of optima) and ruggedness (correlation structure). Based on these results, the relationship between landscape measures and the performance of the evolutionary algorithm used to solve the dynamic optimization problem was studied.
10
H. Richter
The results show that the landscape measures modality and ruggedness scale with the algorithm’s performance and hence allow its prediction. In particular, for small change frequencies, we find a strong correlation between the performance and landscape measures, that is, modality and ruggedness. For larger change frequencies, this correlation ceases as the algorithm is more likely to find the optimum despite the problem hardness induced by the landscape.
References 1. B¨ ack, T.: Evolutionary Algorithms in Theory and Practice: Evolution Strategies, Evolutionary Programming, Genetic Algorithms, Oxford Univ. Press, NY (1996). 2. Branke, J.: Evolutionary Optimization in Dynamic Environments, Kluwer Academic Publishers, Dordrecht (2001). 3. Chazottes, J.R., Fernandez, B.: Dynamics of Coupled Map Lattices and of Related Spatially Extended Systems, Springer–Verlag, Berlin (2005). 4. Hordijk, W.: A Measure of Landscapes. Evolut. Comput. 4 (1996) 335–360. 5. Hordijk, W., Kauffman S.A.: Correlation Analysis of Coupled Fitness Landscapes. Complexity 10 (2005) 42–49. 6. Jin, Y., Branke, J.: Evolutionary Optimization in Uncertain Environments–A Survey. IEEE Trans. Evolut. Comput. 9 (2005) 303–317. 7. Kallel, L., Naudts, B., Reeves, C.R.: Properties of Fitness Functions and Search Landscapes. In: Kallel, L. et al. (eds.): Theoretical Aspects of Evolutionary Computing, Springer–Verlag, Berlin (2001) 177–208. 8. Kaneko, K.: The Coupled Map Lattice. In: Kaneko, K. (ed.): Theory and Application of Coupled Map Lattices, John Wiley, Chichester (1993) 1–49. 9. Morrison, R.W.: Designing Evolutionary Algorithms for Dynamic Environments, Springer–Verlag, Berlin (2004). 10. Papadimitriou, C.H., Steiglitz, K.: Combinatorial Optimization: Algorithms and Complexity, Dover, Mineola, NY (1998). 11. Richter, H.: Behavior of Evolutionary Algorithms in Chaotically Changing Fitness Landscapes. In: Yao, X. et al. (eds.): Parallel Problem Solving from Nature–PPSN VIII, Springer–Verlag, Berlin (2004) 111–120. 12. Richter, H.: A Study of Dynamic Severity in Chaotic Fitness Landscapes. In: Corne, D. (ed.): Proc. Congress on Evolutionary Computation, IEEE CEC 2005, IEEE Press, Piscataway, NJ (2005) 2824–2831. 13. Shibata, T., Kaneko, K.: Coupled Map Gas: Structure Formation and Dynamics of Interacting Mobile Elements with Internal Dynamics. Phys. D181 (2003) 197–214. 14. Smith, T., Husbands, P., Layzell, P., O’Shea, M.: Fitness Landscapes and Evolvability. Evolut. Comput. 10 (2002) 1–34. 15. Stadler, P.F.: Landscapes and Their Correlation Functions. J. Math. Chem. 20 (1996) 1–45. 16. Stadler, P.F., Stephens, C.R.: Landscapes and Effective Fitness. Comm. Theor. Biol. 8 (2003) 389–431. 17. Weicker, K.: An Analysis of Dynamic Severity and Population Size. In: Schoenauer, M. et al. (eds.): Parallel Problem Solving from Nature–PPSN VI, Springer–Verlag, Berlin (2000) 159–168. 18. Weinberger, E.D.: Correlated and Uncorrelated Fitness Landscapes and How to Tell the Difference. Biol. Cybern. 63 (1990) 325–336. 19. Yang, S., Yao, X.: Experimental Study on Population-based Incremental Learning Algorithms for Dynamic Optimization Problems. Soft Comput. 9 (2005) 815–834.
