A fuzzy-based lifetime extension of genetic ... - Semantic Scholar
Fuzzy Sets and Systems 149 (2005) 131 – 147 www.elsevier.com/locate/fss
A fuzzy-based lifetime extension of genetic algorithms Mark Last∗ , Shay Eyal Department of Information Systems Engineering, Ben-Gurion University of the Negev, Beer-Sheva 84105, Israel Available online 19 August 2004
Abstract In knowledge discovery, Genetic Algorithms have been used for classification, model selection, and other optimization tasks. However, behavior and performance of genetic algorithms are directly affected by the values of their input parameters, while poor parameter settings usually lead to several problems such as the premature convergence. Adaptive techniques have been suggested for adjusting the parameters in the process of running the genetic algorithm. None of these techniques have yet shown a significant overall improvement, since most of them remain domain-specific. In this paper, we attempt to improve the performance of genetic algorithms by providing a new, fuzzy-based extension of the LifeTime feature. We use a Fuzzy Logic Controller (FLC) to adapt the crossover probability as a function of the chromosomes’ age. The general principle is that for both young and old individuals the crossover probability is naturally low, while there is a certain age interval, where this probability is high. The concepts of “young”, “old”, and “middle-aged” are modeled as linguistic variables. This approach should enhance the exploration and exploitation capabilities of the algorithm, while reducing its rate of premature convergence. We have evaluated the proposed Lifetime methodology on several benchmark problems by comparing its performance to the basic genetic algorithm and to several adaptive genetic algorithms. The results of our initial experiments demonstrate a clear advantage of the fuzzy-based Lifetime extension over the “crisp” techniques. © 2004 Elsevier B.V. All rights reserved. Keywords: Genetic algorithms; Fuzzy logic; Fuzzy genetic algorithms; Adaptive fuzzy control; Optimization; Fuzzy rule-based systems
1. Introduction In knowledge discovery, Genetic Algorithms have been used for classification, model selection, and other optimization tasks. Practically, GA provides a general-purpose search methodology, which uses ∗ Corresponding author.
E-mail addresses: [email protected] (M. Last), [email protected] (S. Eyal). 0165-0114/$ - see front matter © 2004 Elsevier B.V. All rights reserved. doi:10.1016/j.fss.2004.07.011
132
M. Last, S. Eyal / Fuzzy Sets and Systems 149 (2005) 131 – 147
principles of the natural evolution [13,19]. However, behavior and performance of genetic algorithms are directly affected by the interaction between their parameters [7], which have fixed values in the Simple Genetic Algorithm [18]. Poor parameter settings usually lead to several problems such as premature convergence. Adaptive techniques have been suggested to adjust parameters in the process of running the genetic algorithm. These include adapting mutation probability (Pm), crossover probability (Pc), both Pm and Pc, population size (N) [1,8,28], and even the crossover operator [27]. Lately, several features have been added to the basic genetic algorithm in order to emulate the natural evolution process as precisely as possible. Particularly, the approach called GAVaPS - GA with varying population size [2], which utilizes the concept of LifeTime and age, has shown promising results. Further studies enhancing the basic GAVaPS took place in [10,11] combining known features like incest prohibition and mating between individuals with phenotype similarities. Recently, there has been an increasing interest in the integration of Fuzzy Logic (FL) and Genetic Algorithms (GAs). The motivation is to control GA parameters based on fuzzy logic techniques [14,15,17]. Currently, experience and knowledge on GA have become available as a result of empirical studies, which may be useful for avoiding premature convergence and improving GA behavior. However, too much of this information is vague, incomplete or ill structured which causes it to be rarely applied. Therefore, the use of fuzzy logic would be suitable for dealing with this type of information. The good performance of the FGA (Fuzzy GA) approaches leads to an important conclusion: GAs may be improved through the use of FL. The purpose of this work is to test the effect of fuzzy-based lifetime extension of genetic algorithms through the adaptation of crossover probability. Like in the nature, crossover is practically a method for sharing information between two chromosomes. In the simple genetic algorithm, this action takes place with a certain probability, called crossover probability. The crossover probability has been found crucial to the genetic algorithm performance [24], and since its value varies a lot across different domains, it would be most contributing to focus on it. Therefore, we enhanced the GAVaPS algorithm with a fuzzy logic controller, which adapts the crossover probability according to a fuzzy rule base. We expect to maintain the genetic diversity of a GA and create conditions for reducing the percentage of runs in which the algorithm gets trapped in local optima. The paper is organized as follows: Section 2 gives an overview of related work on GA with varying population size and fuzzy logic control. Section 3 describes the proposed Lifetime extension of GA, while Section 4 presents the results of applying the proposed method to three benchmark problems: Royal Road, Traveling Salesman, and OneMax. Section 5 concludes the paper with the summary of results and directions for future research.
2. Previous work 2.1. Genetic algorithms GA provides a general-purpose search methodology, which uses principles inspired by natural genetics to evolve solutions to problems [13,19]. The simple genetic algorithm starts off with a population of randomly generated chromosomes, each representing a candidate solution to the concrete problem being solved, and advances towards better chromosomes by applying genetic operators, which correspond
M. Last, S. Eyal / Fuzzy Sets and Systems 149 (2005) 131 – 147
Initialize population
133
Selection Crossover
Moving to the next generation…
Mutation Insert into population
Answer Stop criteria satisfied? Fig. 1. Simple genetic algorithm—fundamental mechanism.
to those occurring in nature. This population evolves over time through a successive iteration process of competition and controlled variation. Each state of population is called generation. Associated with each chromosome at every generation is a fitness value, which indicates the quality of the solution the chromosome values lead to. Based upon these fitness values, the selection of the chromosomes, which form the new generation, takes place. Like in nature, the new chromosomes are created using genetic operators such as crossover and mutation. The fundamental mechanism consists of the following stages (as described in Fig. 1): 1. Generate randomly the initial population. 2. Select the chromosomes with the best fitness values (for each chromosome, a fitness value is calculated). 3. Recombine selected chromosomes through crossover and mutation operators. 4. Insert offsprings into the population. 5. If a stop criterion is satisfied, return the chromosome with the best fitness. Otherwise, go to 2. So far, GA’s have demonstrated impressive results in a wide range of domains. However, it has also been found that the GA behavior is strongly determined by the balance between exploiting what already works best (i.e., depth search in previously examined areas) and exploring possibilities that might eventually evolve into something better (i.e. breadth search in new areas). Therefore, poor parameter settings may make exploitation/exploration relationship (EER) disproportionate and produce the lack of diversity in the population. In such cases, where EER is disproportionate, a common problem appears: the premature convergence (see Fig. 2). In this case, a local optimum may prevail and survive throughout the algorithm run and finally would become the algorithm solution. A secondary effect of the EER disproportion is a time consuming evolutionary procedure, which in extreme cases may become too wasteful to use. Finding robust methods for determining the universally best GA parameter settings is probably impossible, since the optimal values are problem-dependent and the GA parameters interact with each other in a complex way [7]. Furthermore, to induce an optimal exploitation/exploration balance, the parameter values may need adjustment in the course of running the algorithm. Among many others, one promising adaptive approach is called GAVaPS.
134
M. Last, S. Eyal / Fuzzy Sets and Systems 149 (2005) 131 – 147
Fig. 2. A local optimum during the GA run.
Procedure GAVaPS { t=0 Random_initialize P(t) Evaluate P(t) While (not termination condition) { t= t+1 Increase the age of each individual by 1 Recombine P(t) Evaluate P(t) Remove from P(t) all individual with age greater than their lifetime } } where P(t) stands for the population on generation at time t.
Fig. 3. GAVaPS general structure.
2.2. GAs with varying population size (GAVaPS) The GAVaPS is a GA with Varying Population Size where each individual has a lifetime, which is measured in generations [2,23]. An individual remains in the population during its lifetime. Parents are selected randomly because the selective pressure is assured by the fact that genomes representing better solutions have longer lifetimes. The general structure of GAVaPS is shown in Fig. 3. Comparisons of GA’s with fixed and varying population sizes have found that in most cases GA’s with varying population size performed at least equivalent to the Simple GA, even if the size of the population is changed randomly during the run [6]. Methods enhancing the GAVaPS by combining incest prohibition and mating between individuals with phenotype similarities have also shown, in most cases, their superiority over the Simple GA [10,11]. 2.3. Fuzzy genetic algorithms The integration of FL and GA’s has shown some promising results. Generally, these two methodologies may be combined for two different purposes [15–17,31]: a. The use of fuzzy-based techniques for either improving GA behavior or modeling GA components. The resulting methods are called fuzzy genetic algorithms (FGAs).
M. Last, S. Eyal / Fuzzy Sets and Systems 149 (2005) 131 – 147
135
b. The application of GAs in various optimization and search problems involving fuzzy systems. Within FGAs, different combinations of GA components based on FL techniques exist, and the most common are: adaptation of GA control parameters, fuzzy operators, fuzzy representation, fuzzy expert control [26] and fuzzy stopping criteria. Regarding the adaptation of GA control parameters, previous studies using Fuzzy Logic Controllers have controlled population size, crossover and mutation probability, and even selection pressure based on fitness and diversity measurements [15,17,29]. Such methods try to enhance the GA behavior using guiding principles, like maintaining diversity in the population. In contrast, our method tries to enhance the GA behavior using a biological property, namely the age, as a state variable to control the crossover probability. Fuzzy Logic Controller (FLC) enables the dynamical calculation of appropriate GA control parameters, based on experience and knowledge of the GA [17,22]. The fuzzy knowledge base (or fuzzy government [3]) is used for detecting the emergence of solutions, dynamically tuning algorithm parameters, as in [5,16,31], and monitoring the evolutionary process to avoid undesired behaviors such as premature convergence. Generally, an FLC allows one to qualitatively express the control strategies based on experience as well as intuition. For detailed description of FLC see appendix A.
3. Proposed method The proposed method of fuzzy-based lifetime extension of genetic algorithm enhances GAVaPS with a fuzzy logic controller, which adapts the crossover probability according to a fuzzy rule base, thus creating a fuzzy-based lifetime extension of genetic algorithms. The general structure of the proposed algorithm is shown in Fig. 4.
Procedure LifetimeExtendedGA { t=0 Random_initialize P(t) Evaluate P(t) While (not termination condition) { t= t+1 Increase the age of each individual by 1 Recombine P(t) { Select parents from P(t) For each pair of parents { Assign crossover probability (Pc) { Call Fuzzy Logic Controller (parameters) } Perform crossover with probability Pc Perform mutation with probability Pm Insert offsprings into P(t) } } Evaluate P(t) Remove from P(t) all individuals with age greater than their lifetime } }
Fig. 4. Fuzzy-based lifetime extended GA—general structure.
M. Last, S. Eyal / Fuzzy Sets and Systems 149 (2005) 131 – 147
Membership Function
136
Middle1 Young
0.2
age
0.3
Old
0.5
0.7
1
Age distribution
Fig. 5. The age linguistic variable.
3.1. Fuzzy logic controller (FLC) The FLC components were designed as follows: • State variables: age and lifetime of the chromosomes to be crossovered (parents) and the population average lifetime. All are crisp values. • Controlled parameter: crossover probability (= P c), a crisp value. 3.1.1. Fuzzification interface The fuzzification interface defines for each parent the possibilities of being {Young, Middle-age, Old}, which are linguistic labels of both inputs. These values determine the degree of truth for each rule premise. This computation takes into account only one parent at a time, and relies on the triangular membership functions shown in Fig. 5. In our case studies (see next section), we have experimented with three different settings of these functions. The relative age shown on the x-axis in Fig. 5 refers to the ratio between chromosome’s age and the average lifetime in the population. 3.1.2. Knowledge base Knowledge base enables the representation of the vague information mentioned earlier or any other intuitive information, which helps to emulate the natural process. Our general principle is that crossover probability should vary across age intervals as a function of lifetime: for both young and old individuals crossover probability is naturally low, while there is a certain age interval where this probability goes up. In this way, Exploration–Exploration relationship would maintain appropriateness: very young offsprings will be less crossovered enabling exploration capability, an old offspring being less crossovered and eventually dying out helps avoiding local optimum (premature convergence), and middle-age individuals that will be mostly crossovered, would enhance the exploitation aspect. The linguistic terms representing the range of each fuzzy variable are as follows: Age ∈ [Young, Middle-age, Old] P c ∈ [Low, Medium, High] The best configuration of the fuzzy rule base found by an extensive set of experiments [9] is described at Table 1.
M. Last, S. Eyal / Fuzzy Sets and Systems 149 (2005) 131 – 147
137
Table 1 Fuzzy rule base Parent I Young
Middle-age
Old
Young Middle-age Old
Low Medium Low
Medium High Medium
Low Medium Low
Membership Function
Parent II
1
Low
Medium High
0.2 0.5 0.8 Pc
1
Fig. 6. The crossover probability linguistic variable.
Each rule is assigned a relevance degree equal to the minimum value of its conditions. Afterwards, the inference engine assigns each linguistic output (Low, Medium, and High) a single value, which is the maximum value of its respective fired rules. This inference method is known as MAX-MIN method [22]. 3.1.3. DeFuzzification interface The Defuzzification interface computes a crisp value for our controlled parameter crossover probability according to the values of the linguistic labels {Low, Medium, High}, which are the outputs of the rule base, and the triangular membership functions shown in Fig. 6. The settings of the linguistic labels were chosen according to previous studies that have determined (experimentally) the recommended parameters’ values for the single point crossover that are applicable to a wide range of optimization problems [8]. The triangular shape of the membership function was chosen due to its common use as well as simplicity. The defuzzification method used is center of gravity, also called CENTROID: the crisp value of the output variable is computed by finding the center of gravity under the membership function for the fuzzy variable [22].
3.2. Lifetime calculation strategy The GAVaPS algorithm assigns a lifetime to each individual at its birth. Three lifetime calculation strategies are proposed, but none is determined as superior [23]. In our fuzzy-based algorithm, we chose
138
M. Last, S. Eyal / Fuzzy Sets and Systems 149 (2005) 131 – 147
the Bi-Linear allocation strategy, which is computed as follows: If AvgF it fitness[i] If AvgF it < fitness[i]
lif etime [i] = MinLT + · lif etime [i] =
f itness [i] − MinF it , AvgF it − MinF it
1 f itness [i] − AvgF it , (MinLT + MaxLT ) + · 2 MaxF it − AvgF it
where: =
1 · (maxLT − minLT ) 2
MaxLT , MinLT —allowed maximal and minimal lifetime, respectively. MaxF it, AvgF it, MinF it—the best, average, and worst fitness of the current population, respectively. Hence, better solutions get a longer lifetime and therefore remain in the population for more generations and will have higher chance of producing an offspring.
4. Experiments and discussion 4.1. Experimental design The following algorithms were examined and compared: 1. Simple GA [13], served as a benchmark in [1,2,10,11,29]. 2. GA with varying population size [2], which has been under comparison in [10,11]. 3. Adaptive Genetic Algorithm (AGA), where crossover and mutation probabilities are adapted as a function of solution fitness [28]. 4. Fuzzy-based lifetime extended GA. We experimented with three different combinations of the two parameters, as described in Table 2. The test cases used in this study are as follows: 1. Royal Road problem [20,24]. 2. Traveling Salesman Problem (TSP) [25]. 3. 100-bit OneMax problem [21,30].
Table 2 Fuzzy-based lifetime extension of GA—tested configurations Parameter
LTexGA#1
LTexGA#2
LTexGA#3
FLC—Young upper limit (%) FLC—Old lower limit (%)
30 70
25 75
20 80
M. Last, S. Eyal / Fuzzy Sets and Systems 149 (2005) 131 – 147
139
The following performance measurements were taken: 1. Performance ratio—number of runs yielding the optimal solution to the total number of runs. 2. Number of generations to achieve the optimal solution. 3. Genotypical Diversity (GD)—diversity measure that is calculated by comparing the inner structure of the chromosomes. For a binary chromosome (Royal Road and OneMax), the diversity was measured as follows: N N
D (p) =
Xi (j ) i=1 j =i , N (N −1) 2
Xi (j ) =
Xi (j ) =
L
Yij (k) ,
k=1
1, If the Kth gene of genome i equals to the Kth gene of genome j, 0, Otherwise,
where N is the population size, and L is the chromosome length. For a list of integer-valued chromosomes (TSP), the diversity was measured as follows: N N
D (p) =
Xi (j ) i=1 j =i , N (N −1) 2
Yij (m, n) =
1, 0,
Xi (j ) = L −
L L
Yij (m, n),
m=1 n=m
If cities m and n are neighbors in genomes i and j, Otherwise,
where N is the population size and L is the chromosome length. For instance, consider a 5-cities TSP and these two chromosomes: (3,5,1,2,4), (4,1,2,5,3). In this case, Xi (j ) = 3. But for (3,5,1,2,4) and (1,2,4,3,5), Xi (j ) = 0 because it is practically the same route. 4. Phenotypical Diversity (PD)—diversity measure that compares only the final solutions quality in each generation. PD =
fbest f
, where f and fbest are the average and best fitness, respectively [15].
Results were averaged over 30 runs in each experiment. 4.2. Summary of results 4.2.1. Royal road The Royal Road functions were designed to study building blocks’ interactions during the GA process. These functions are made up of a structure of building blocks that allows the monitoring of the GA process. The GA must follow a path formed by this structure—the “Royal Road”—to achieve the global optimum [20,24]. For this experiment, we used a genome of 240 bits and the highest level of 4. Each level represents an interval of fitness values, where level 1 means low fitness values and level 4 stands for the optimal fitness.
M. Last, S. Eyal / Fuzzy Sets and Systems 149 (2005) 131 – 147 RoyalRoad - Level Achived 5%
13%
55%
57%
85%
60%
60%
83%
35%
30%
2)
)
A
A
ex G A1
(#
G FL
FL
AG
A
AV aP S G
Si m pl eG
20%
15%
10%
0%
Level2 Level1 Level0
3)
20%
Level4 Level3
ex G A
40%
(#
Runs
60%
10%
20%
25%
80%
FL
17%
(#
100%
ex
140
Algorithm
Fig. 7. Level reached by each algorithm.
Complete function definition can be found in [20]. The settings of the Simple GA are: • Population size—defines the amount of individuals in the population in each generation. For this test case we used population size of 512 chromosomes. • Number of generations—number of iterations in a single run, usually used as a stopping criterion. We used 400 generations. • Roulette wheel selection—a method to select chromosomes based on the magnitude of the fitness value relative to the rest of the population: chromosome with higher fitness has a higher probability to be chosen. Practically, the probability for a chromosome to be chosen is equal to its fitness divided by the sum of fitness values of each chromosome in the population. • 1-point crossover method: A single crossover position is chosen at random and the genes (bits) of two parents before and after the crossover position are exchanged to form two offsprings. • Crossover probability: the probability of two chosen chromosomes to be crossovered. We used crossover probability of 0.9. • Flip mutation: given chromosome, every bit value changes with a mutation probability. • Mutation probability: the probability mutation will happen. We used 0.01. • 1-elitism: transfer of the best chromosome (one or more) from the current population to the next one “as is”. This feature is a part of the simple genetic algorithm [13]. GAVaPS used the same values except for crossover probability (0.65) and mutation probability (0.005), because they were found more suitable in [11]. Both GAVaPS and Fuzzy-based lifetime extended GA used: • Minimum lifetime: minimal number of generations a chromosome may remain in the population. We chose a value of 1. • Maximum lifetime: maximal number of generations a chromosome may remain in the population. We chose a value of 7. • Reproduction ratio: the maximum population growth allowed within a single generation. We chose a value of 0.4. • Lifetime calculation strategy: Bi-Linear (as described earlier). AGA parameters (k1, k2, k3, k4) were set to the values recommended in [28]. Fig. 7 shows the levels reached by each algorithm, and Table 3 presents the average number of generations needed to achieve each of the four possible levels. For GAVaPS and Simple GA, similar results were found in [10]. The obtained results clearly show that the Fuzzy-based lifetime extension of GA
M. Last, S. Eyal / Fuzzy Sets and Systems 149 (2005) 131 – 147
141
Table 3 Number of generations to reach levels for each algorithm Level
Simple GA
GAVaPS
AGA
LTexGA#1
LTexGA#2
LTexGA#3
Level 1 Level 2 Level 3 Level 4
5.9 65.7 310.4 —
12.909 48.545 159.44 368.67
8.388 60 — —
10.85 40.35 150.11 207
11.1 40.56 261.3 336.5
12.1 44.4 214.3 339.5
Royal Road - Diversity 0.8 0.7 SimpleGA GAVaPS FLexGA(#1) FLexGA(#2) FLexGA(#3) AGA
Diversity
0.6 0.5 0.4 0.3 0.2 0.1 0 1
Generations
400
Fig. 8. Diversity of each algorithm along the run.
outperformed significantly (according to t-test with = 0.05) the Simple GA, the AGA and even the GAVaPS: it reached a higher level in most runs and in fewer generations (except level 1). Among the three, the first configuration of the fuzzy membership functions performed best in terms of both performance measures. Fig. 8 shows the genotypical diversity of each algorithm. The best performance was achieved with the second configuration, with minor but statistically significant differences according to t-test with = 0.05, still much better than the Simple GA and better than GAVaPS: The diversity has converged to a reasonable value and stayed stable along the run.AGA has converged prematurely and maintained a very low diversity, which correlates to its poor performance (see Fig. 7). 4.2.2. Traveling salesman problem (TSP) The TSP involves finding a cyclic path in a graph such that all vertices (e.g. cities) are visited exactly once. The objective is to find a route of minimum total distance (weight). The TSP is an NP-complete problem. Three problems were chosen from the TSPLIB [25], all three are symmetric TSPs with Euclidean distance: Eil51, Eil76, Rat99 (51, 76 and 99 cities). The complete definitions can be found in [25]. The settings of the Simple GA for this problem are: • Population size: 100. • Number of generations: 200. • Roulette wheel selection. • Partial match crossover [23]—crossover used for the TSP because it always creates feasible solution (unlike 1-point crossover). • Crossover probability: 0.7.
142
M. Last, S. Eyal / Fuzzy Sets and Systems 149 (2005) 131 – 147
Table 4 Shortest and average path length achieved by each algorithm Problem
SimpleGA
GAVaPS
AGA
LTexGA#1
LTexGA#2
LTexGA#3
Best known
Eil51
1,094.86 (1,225.21) 1,792.1 (1,917.3) 5,729.7 (6,122.35)
729.5 (812.23) 1,252.31 (1,301.2) 3,742.42 (4003.8)
1,053.91 (1,160.35) 1,674.49 (1,856.16) 5,624.53 (5,954.41)
525.46 (584.2) 925.846 (1,011.53) 2,664.5 (2,922.35)
611.72 (665.25) 1,067.5 (1,113.31) 2,983.71 (3,183.28)
692.66 (748.45) 1,107.38 (1,173.4) 3,199.8 (3,443.64)
426
Eil76 Rat99
538 1,211
• Swap mutation—given a chromosome, it inverts the order of bits between two selected positions chosen at random. • Mutation probability: 0.01. • 1-elitism. Both GAVaPS, AGA and Fuzzy-based lifetime extended GA used the same additional parameters as described in the previous case study. Table 4 presents the shortest and the average paths achieved within 30 runs of 200 generations each. All three configurations of the fuzzy-based lifetime extension of GA performed better than the Simple GA, the AGA, and the GAVaPS, while the first configuration is preferable: In Eil51 it averaged a total route distance which is 5 times closer to the best known result vs. the one achieved by the Simple GA and 2 times closer than GAVaPS. Its average distance is also better than the second and the third configurations (1.5 and 2 times closer). AGA performed better, though not significantly, than Simple GA, but fell behind GAVaPS and furthermore behind the proposed method. In Eil76 and Rat99 the first configuration still produced a shorter distance than the other two, though with smaller differences than in Eil51: In Eil76, the first configuration reached an average distance about 3 times closer to the best known result than the Simple GA and 1.6 times closer than the GAVaPS. In Rat99 the differences were reduced by 2.8 times and 1.65 respectively. As expected, when the number of cities grows, given a fixed number of generations, the improvement decreases, but it still exists. Comparison to the best results that can be achieved with each configuration has not taken place because of the extremely large number of required generations. In our experiment, we run only 200 generations while in [1] the best solution for Eil51 was achieved after as many as 383,139 generations and the best solution for Eil76 after 329,516 generations. Figs. 9, 10, 11, describe the different genotypical diversities for each problem. In this case study, the best performance was achieved with the third configuration, with significant differences. The fuzzy based lifetime extended GA reached better results that the Simple GA, AGA and GAVaPS most of the time, but longer runs are needed to validate this assumption. Still, one can infer from the overall performance that the fuzzy-based method is preferable for this case study. 4.2.3. Onemax problem In this case study, also known as counting ones problem, the fitness of a given string is simply the number of ones the string contains [21,30]. The aim is to obtain a string containing all ones. Detailed studies concerning this problem can be found in [4,12,21,30]. A string length of 100 was used for the
M. Last, S. Eyal / Fuzzy Sets and Systems 149 (2005) 131 – 147 Eil51 Diversity 55
Diversity
50
SimpleGA GAVaPS
45
LTexGA#1 LTexGA#2
40
LTexGA#3 AGA
35 30 0
100 Generations
200
Fig. 9. Eil51 problem—diversity of each algorithm.
Eil76 Diversity 80 75
Diversity
SimpleGA 70
GAVaPS
65
LTexGA#1 LTexGA#2
60
LTexGA#3 AGA
55 50 0
100 Generations
200
Fig. 10. Eil76 problem—diversity of each algorithm.
Rat99 Diversity 98
Diversity
96 94
SimpleGA GAVaPS
92
LTexGA#1 LTexGA#2
90
LTexGA#3 AGA
88 86 84 0
Generations
200
Fig. 11. Rat99 problem—diversity of each algorithm.
purposes of this study. The Simple GA settings are: • Population size: 100. • Number of generations: 200. • Roulette wheel selection. • 1-point crossover with probability of 0.9. • Flip mutation with probability of 0.01. • 1-elitism.
143
144
M. Last, S. Eyal / Fuzzy Sets and Systems 149 (2005) 131 – 147
Table 5 Statistics of the fitness reached by each algorithm Fitness statistic
SimpleGA
GAVaPS
AGA
LTexGA#1
LTexGA#2
LTexGA#3
Worst Average Best Std. Dev
86 90.90 95 2.006
97 98.07 99 0.785
66 70.60 80 3.050
94 95.80 99 1.064
93 95.30 98 1.119
94 95.33 97 0.994
Table 6 Number of generations each algorithm needed to reach optimal solution Statistic
SimpleGA
GAVaPS
AGA
LTexGA#1
LTexGA#2
LTexGA#3
Worst Average Best Std. Dev
747 3,502 1,437 793.054
280 999 529 247.828
— — — —
357 1,011 620 213.806
236 971 629 262.242
473 1,009 739 184.963
GAVaPS and Fuzzy-based lifetime extended GA used the same additional parameters as described in the previous case study except for the reproduction ratio (0.3). Table 5 presents the statistics of the fitness reached by each algorithm after 200 generations: GAVaPS and LTexGA algorithms reached significantly higher fitness (t-test with = 0.05) than the Simple GA, by about 5–8%, though there are some differences vs. the first two experiments: • The difference between the GAVaPS and the lifetime extended GA is statisitically significant (t-test with = 0.05) though the GAVaPS average performance surpasses the best configuration of our algorithm by 2.3% only. • The first configuration of the fuzzy-based lifetime extension of GA is better than the other two by nearly 0.5%. However, there is no significant difference between the average fitness values of the three different configurations. • AGA performed significantly worse than the Simple GA. Table 6 presents the statistics of the number of generations needed to reach the optimal solution (= 100). The second configuration of the fuzzy-based lifetime extended GA has reached the optimal solution faster than GAVaPS by nearly 2.8%. AGA statistics are missing because it did not reach the optimal solution even after 15,000 generations. Figs. 12 and 13 describe the phenotypical and genotypical diversities respectively. Both figures show that except AGA, all four algorithms have reached similar results, which are preferable over the Simple GA. According to the t-test with = 0.05, there is no significant difference between GAVaPS and the fuzzy-based lifetime extended GA, while across the three configurations for the fuzzybased lifetime extended GA there is no significant difference (t-test with = 0.05) with respect to the diversity measure.
M. Last, S. Eyal / Fuzzy Sets and Systems 149 (2005) 131 – 147
145
Phenotype Diversity 0.3 0.25 SimpleGA GAVaPS LTexGA#1
Diversity
0.2 0.15
LTexGA#2 LTexGA#3
0.1
AGA
0.05 0 1
51
101 Generations
151
200
Fig. 12. OneMax problem—phenotype diversity of each algorithm.
Genotype Diversity 0.6
Diversity
0.5 0.4
SimpleGA GAVaPS
0.3
LTexGA#1 LTexGA#2
0.2
LTexGA#3 AGA
0.1 0 0
100 Generations
200
Fig. 13. OneMax problem—genotype diversity of each algorithm.
5. Conclusions and future work This paper presented the Fuzzy-based lifetime extension of GA, which enables the use of ill-structured experience and vague information, as well as the human intuition. We used this capability to improve the emulation of the biological process by dynamically adapting the crossover probability. However, other parameters, preferable to those derived from nature, may be modeled as well. The Fuzzy-based lifetime extension of GA outperformed the Simple GA, the AGA, and the GAVaPS in all case studies: at least one fuzzy-based configuration reached better results in fewer generations. With a fixed number of generations, it reached higher fitness than GAVaPS in two cases out of three, while in the OneMax problem it was only slightly worse than GAVaPS. One explanation is that in some problems, there exists a relationship between the membership function and the objective function. Logically, it is possible that with a suitable membership function the lifetime extended GA would perform better than GAVaPS in the third case study as well, but it is an open issue to be explored. Within the fuzzy-based lifetime extension of GA, the first configuration, which defines up to 30% as young and 70% as old, was found to be preferable most of the times, which seems reasonable when we consider the biological process. Still, the study of these parameters, as well as the settings of the “expert knowledge”, requires further research and experimentation.
146
M. Last, S. Eyal / Fuzzy Sets and Systems 149 (2005) 131 – 147
Knowledge base
Fuzzification Interface
State Variables
Inference System
Defuzzification Interface
Controlled System
Control parameters
Fig. 14. Fuzzy logic controller generic structure.
Appendix A. The generic structure of an FLC is shown in Fig. 14. The FLC components are: • Knowledge base—Database, containing the definition of the fuzzy control rules linguistics labels (i.e. the membership functions of the fuzzy sets specifying the meaning of the linguistic terms) and a rule base created by collection of fuzzy control rules representing the expert knowledge. • Fuzzification interface—transformation of crisp data into fuzzy sets based on pre-defined membership functions. • Inference system—reasoning about the fuzzy sets based on the knowledge base. Inference methods consist of forward chaining inference. • Defuzzification interface—translation of the fuzzy control action into a real control action using defuzzification method. • Controlled system—the Genetic algorithm in our case. In addition, it is most important to define the FLC state variables, which include, in the case of a GA, measures describing the GA behavior, such as population diversity, maximum, average, and minimum fitness, as well as the controlled parameters, such as population size, crossover and mutation probability, etc. References [1] M. Annuziato, S. Pizzuti, Adaptive parameterization of evolutionary algorithms driven by reproduction and competition, atti di ESIT2000 European Symp. on Intelligent Techniques, Aachen (Germania), September 2000. [2] J. Arabas, Z. Michalewicz, J. Mulawka, GAVaPS—a Genetic Algorithm with varying population size, in: Proc. of the first IEEE Conf. on Evolutionary Computation,1994, pp. 73–78. [3] S. Arnone, M. Dell’Orto, A. Tettamanzi, Toward a fuzzy government of genetic populations, in: Proc. of the 6th IEEE Conf. on Tools with Artificial Intelligence,1994, pp. 581–585. [4] T. B”ack, Optimal mutation rates in genetic search, in: S. Forrest (Ed.), Proc. of the Fifth Internat. Conf. on Genetic Algorithms, San Mateo, CA, Morgan Kaufmann, Los Altos, CA, 1993, pp. 2–9. [5] A. Bergman, W. Burgard, S. Hemker, Adjusting parameters of genetic algorithms by fuzzy control rules, in: K.-H. Becks, D. Perret-Gallix (Eds.), New Computing Techniques in Physics Research III, World Scientific, Singapore, 1993. [6] J. Costa, R. Tavares, A.C. Rosa, An experimental study on dynamic random variation of population size, in: Proc. of the 1999 IEEE Internat. Conf. on Systems, Man, and Cybernetics,1999, pp. 607–612.
M. Last, S. Eyal / Fuzzy Sets and Systems 149 (2005) 131 – 147
147
[7] K. Deb, S. Agrawal, Understanding interactions among genetic algorithm parameters, in: W. Banzhaf, C. Reeves (Eds.), Foundations of Genetic Algorithm 5, Morgan Kaufmann, San Francisco, CA, 1998, pp. 265–286. [8] A.E. Eibon, R. Hinterding, Z. Michalewicz, Parameter control in evolutionary algorithm, IEEE Trans. Evolution. Comput. (1999) 124-141. [9] S. Eyal, A fuzzy-based age extension of genetic algorithm, Master’s Thesis, Department of Information Systems Engineering, Faculty of Engineering, Ben-Gurion University, 2003. [10] C. Fernandes, A. Rosa, A study on non-random mating and varying population size in genetic algorithms using royal road function, in: Proc. of the 2001 Congress on Evolutionary Computation CEC2001,2001, pp. 60–66. [11] C. Fernandes, R. Tavares, A. Rosa, niGAVaPS—Outbreeding in genetic algorithms, in: Proc. of the 2000 ACM Symp. on Applied Computing, Villa Olmo, Como, Italy, 2000. [12] P. Giguere, D.E. Goldberg, Population sizing for optimum sampling with genetic algorithms: a case study of the onemax problem, in: J.R. Koza et al. (Eds.), Proc. Genetic Programming, Morgan Kaufmann, Los Altos, CA, 1998, pp. 496–503. [13] D.E. Goldberg, Algorithm in Search Optimization and Machine Learning, Addison-Wesley, Reading, MA, 1989. [14] F. Herrera, M. Lozano, E. Herrera-Viedma, J.L. Verdegay, Fuzzy tools to improve genetic algorithms, in: Proc. Second European Conf. on Intelligent Techniques and Soft Computing (EUFIT’94), Vol. 3, September 1994, pp. 1532–1539. [15] F. Herrera, M. Lozano, Adaptation of genetic algorithm parameters based on fuzzy logic controllers, in: F. Herrera, J. Verdegay (Eds.), Genetic Algorithms and Soft Computing, 1996, pp. 95–125. [16] F. Herrera, M. Lozano, Adaptive genetic operators based on coevolution with fuzzy behaviors, IEEE Trans. Evolution. Comput. 5 (2) (2001) 1–18. [17] F. Herrera, M. Lozano, Fuzzy genetic algorithms: issues and models, Technical Report DECSAI-98116, Department of Computer Science and A.I., University of Granada, June 1998. [18] J.H. Holland, Adaptation in Natural and Artificial Systems, MIT Press, Cambridge, MA, 1975. [19] J.H. Holland, Genetic algorithms, Sci. Am. 267 (1) (1992) 44–150. [20] J.H. Holland, Royal Road Functions. Genetic algorithms, GAList Internet mailing list, Digest, Vol. 7(22), Article dated August 7, 1993. [21] J. Horn, C.R. Reeves, The genetic crossover landscape for the onemax problem, in: J. Alander (Ed.), Proc. of the 2nd Nordic Workshop on Genetic Algorithms and their Applications, University of Vaasa Press, Vaasa, Finland, 1996, pp. 27–43. [22] G.J. Klir, B. Yuan, Fuzzy Sets and Fuzzy Logic: Theory and Applications, Prentice-Hall Inc., Englewood Cliffs, NJ, 1995. [23] Z. Michalewicz, Genetic algorithms + Data structures = Evolution programs, Verlag, Heidelberg, Berlin, Third Revised and Extended Edition, 1999. [24] M. Mitchell, An Introduction to Genetic Algorithms, MIT Press, Cambridge, MA, 1996. [25] G. Reinelt, ‘TheTSPLIB’, http://www.iwr.uni-heidelberg.de/iwr/comopt/soft/TSPLIB95/TSPLIB.html. [26] Y. Shi, R. Eberhart, Y. Chen, Implementation of evolutionary fuzzy systems, IEEE Trans. Fuzzy Syst. 7 (2) (1999) 109– 119. [27] W.M. Spears, Adapting crossover in evolutionary algorithms, in: Proc. of 4th Annu. Conf. on Evolutionary Computing, Ed Fogel,MIT Press, Cambridge, MA, 1995. [28] M. Srinivas, L.M. Patnaik, Adaptive probabilities of crossover and mutation in genetic algorithms, IEEE Trans. Systems, Man Cybernet. 24 (4) (1994) 656–667. [29] R.J. Streifel, R.J. Marks II, R. Reed, J.J. Choi, M. Healy, Dynamic fuzzy control of genetic algorithm parameter coding, IEEE Trans. Systems, Man Cybernet. Part B: Cybernetics 29 (3) (1999) 426–433. [30] G. Syswerda, Uniform crossover in genetic algorithms, in: J.D. Schaffer (Ed.), Proc. of the Third Internat. Conf. on Genetic Algorithms and their Applications, Morgan Kaufmann, San Mateo, 1989, pp. 2–9. [31] L. Takagi, Dynamic control of genetic algorithms using fuzzy logic techniques, in: S. Forrest (Ed.), Proc. of the 5th Internat. Conf. On Genetic Algorithms (ICGA), 1993, pp. 76–83.
A fuzzy-based lifetime extension of genetic algorithms Mark Last∗ , Shay Eyal Department of Information Systems Engineering, Ben-Gurion University of the Negev, Beer-Sheva 84105, Israel Available online 19 August 2004
Abstract In knowledge discovery, Genetic Algorithms have been used for classification, model selection, and other optimization tasks. However, behavior and performance of genetic algorithms are directly affected by the values of their input parameters, while poor parameter settings usually lead to several problems such as the premature convergence. Adaptive techniques have been suggested for adjusting the parameters in the process of running the genetic algorithm. None of these techniques have yet shown a significant overall improvement, since most of them remain domain-specific. In this paper, we attempt to improve the performance of genetic algorithms by providing a new, fuzzy-based extension of the LifeTime feature. We use a Fuzzy Logic Controller (FLC) to adapt the crossover probability as a function of the chromosomes’ age. The general principle is that for both young and old individuals the crossover probability is naturally low, while there is a certain age interval, where this probability is high. The concepts of “young”, “old”, and “middle-aged” are modeled as linguistic variables. This approach should enhance the exploration and exploitation capabilities of the algorithm, while reducing its rate of premature convergence. We have evaluated the proposed Lifetime methodology on several benchmark problems by comparing its performance to the basic genetic algorithm and to several adaptive genetic algorithms. The results of our initial experiments demonstrate a clear advantage of the fuzzy-based Lifetime extension over the “crisp” techniques. © 2004 Elsevier B.V. All rights reserved. Keywords: Genetic algorithms; Fuzzy logic; Fuzzy genetic algorithms; Adaptive fuzzy control; Optimization; Fuzzy rule-based systems
1. Introduction In knowledge discovery, Genetic Algorithms have been used for classification, model selection, and other optimization tasks. Practically, GA provides a general-purpose search methodology, which uses ∗ Corresponding author.
E-mail addresses: [email protected] (M. Last), [email protected] (S. Eyal). 0165-0114/$ - see front matter © 2004 Elsevier B.V. All rights reserved. doi:10.1016/j.fss.2004.07.011
132
M. Last, S. Eyal / Fuzzy Sets and Systems 149 (2005) 131 – 147
principles of the natural evolution [13,19]. However, behavior and performance of genetic algorithms are directly affected by the interaction between their parameters [7], which have fixed values in the Simple Genetic Algorithm [18]. Poor parameter settings usually lead to several problems such as premature convergence. Adaptive techniques have been suggested to adjust parameters in the process of running the genetic algorithm. These include adapting mutation probability (Pm), crossover probability (Pc), both Pm and Pc, population size (N) [1,8,28], and even the crossover operator [27]. Lately, several features have been added to the basic genetic algorithm in order to emulate the natural evolution process as precisely as possible. Particularly, the approach called GAVaPS - GA with varying population size [2], which utilizes the concept of LifeTime and age, has shown promising results. Further studies enhancing the basic GAVaPS took place in [10,11] combining known features like incest prohibition and mating between individuals with phenotype similarities. Recently, there has been an increasing interest in the integration of Fuzzy Logic (FL) and Genetic Algorithms (GAs). The motivation is to control GA parameters based on fuzzy logic techniques [14,15,17]. Currently, experience and knowledge on GA have become available as a result of empirical studies, which may be useful for avoiding premature convergence and improving GA behavior. However, too much of this information is vague, incomplete or ill structured which causes it to be rarely applied. Therefore, the use of fuzzy logic would be suitable for dealing with this type of information. The good performance of the FGA (Fuzzy GA) approaches leads to an important conclusion: GAs may be improved through the use of FL. The purpose of this work is to test the effect of fuzzy-based lifetime extension of genetic algorithms through the adaptation of crossover probability. Like in the nature, crossover is practically a method for sharing information between two chromosomes. In the simple genetic algorithm, this action takes place with a certain probability, called crossover probability. The crossover probability has been found crucial to the genetic algorithm performance [24], and since its value varies a lot across different domains, it would be most contributing to focus on it. Therefore, we enhanced the GAVaPS algorithm with a fuzzy logic controller, which adapts the crossover probability according to a fuzzy rule base. We expect to maintain the genetic diversity of a GA and create conditions for reducing the percentage of runs in which the algorithm gets trapped in local optima. The paper is organized as follows: Section 2 gives an overview of related work on GA with varying population size and fuzzy logic control. Section 3 describes the proposed Lifetime extension of GA, while Section 4 presents the results of applying the proposed method to three benchmark problems: Royal Road, Traveling Salesman, and OneMax. Section 5 concludes the paper with the summary of results and directions for future research.
2. Previous work 2.1. Genetic algorithms GA provides a general-purpose search methodology, which uses principles inspired by natural genetics to evolve solutions to problems [13,19]. The simple genetic algorithm starts off with a population of randomly generated chromosomes, each representing a candidate solution to the concrete problem being solved, and advances towards better chromosomes by applying genetic operators, which correspond
M. Last, S. Eyal / Fuzzy Sets and Systems 149 (2005) 131 – 147
Initialize population
133
Selection Crossover
Moving to the next generation…
Mutation Insert into population
Answer Stop criteria satisfied? Fig. 1. Simple genetic algorithm—fundamental mechanism.
to those occurring in nature. This population evolves over time through a successive iteration process of competition and controlled variation. Each state of population is called generation. Associated with each chromosome at every generation is a fitness value, which indicates the quality of the solution the chromosome values lead to. Based upon these fitness values, the selection of the chromosomes, which form the new generation, takes place. Like in nature, the new chromosomes are created using genetic operators such as crossover and mutation. The fundamental mechanism consists of the following stages (as described in Fig. 1): 1. Generate randomly the initial population. 2. Select the chromosomes with the best fitness values (for each chromosome, a fitness value is calculated). 3. Recombine selected chromosomes through crossover and mutation operators. 4. Insert offsprings into the population. 5. If a stop criterion is satisfied, return the chromosome with the best fitness. Otherwise, go to 2. So far, GA’s have demonstrated impressive results in a wide range of domains. However, it has also been found that the GA behavior is strongly determined by the balance between exploiting what already works best (i.e., depth search in previously examined areas) and exploring possibilities that might eventually evolve into something better (i.e. breadth search in new areas). Therefore, poor parameter settings may make exploitation/exploration relationship (EER) disproportionate and produce the lack of diversity in the population. In such cases, where EER is disproportionate, a common problem appears: the premature convergence (see Fig. 2). In this case, a local optimum may prevail and survive throughout the algorithm run and finally would become the algorithm solution. A secondary effect of the EER disproportion is a time consuming evolutionary procedure, which in extreme cases may become too wasteful to use. Finding robust methods for determining the universally best GA parameter settings is probably impossible, since the optimal values are problem-dependent and the GA parameters interact with each other in a complex way [7]. Furthermore, to induce an optimal exploitation/exploration balance, the parameter values may need adjustment in the course of running the algorithm. Among many others, one promising adaptive approach is called GAVaPS.
134
M. Last, S. Eyal / Fuzzy Sets and Systems 149 (2005) 131 – 147
Fig. 2. A local optimum during the GA run.
Procedure GAVaPS { t=0 Random_initialize P(t) Evaluate P(t) While (not termination condition) { t= t+1 Increase the age of each individual by 1 Recombine P(t) Evaluate P(t) Remove from P(t) all individual with age greater than their lifetime } } where P(t) stands for the population on generation at time t.
Fig. 3. GAVaPS general structure.
2.2. GAs with varying population size (GAVaPS) The GAVaPS is a GA with Varying Population Size where each individual has a lifetime, which is measured in generations [2,23]. An individual remains in the population during its lifetime. Parents are selected randomly because the selective pressure is assured by the fact that genomes representing better solutions have longer lifetimes. The general structure of GAVaPS is shown in Fig. 3. Comparisons of GA’s with fixed and varying population sizes have found that in most cases GA’s with varying population size performed at least equivalent to the Simple GA, even if the size of the population is changed randomly during the run [6]. Methods enhancing the GAVaPS by combining incest prohibition and mating between individuals with phenotype similarities have also shown, in most cases, their superiority over the Simple GA [10,11]. 2.3. Fuzzy genetic algorithms The integration of FL and GA’s has shown some promising results. Generally, these two methodologies may be combined for two different purposes [15–17,31]: a. The use of fuzzy-based techniques for either improving GA behavior or modeling GA components. The resulting methods are called fuzzy genetic algorithms (FGAs).
M. Last, S. Eyal / Fuzzy Sets and Systems 149 (2005) 131 – 147
135
b. The application of GAs in various optimization and search problems involving fuzzy systems. Within FGAs, different combinations of GA components based on FL techniques exist, and the most common are: adaptation of GA control parameters, fuzzy operators, fuzzy representation, fuzzy expert control [26] and fuzzy stopping criteria. Regarding the adaptation of GA control parameters, previous studies using Fuzzy Logic Controllers have controlled population size, crossover and mutation probability, and even selection pressure based on fitness and diversity measurements [15,17,29]. Such methods try to enhance the GA behavior using guiding principles, like maintaining diversity in the population. In contrast, our method tries to enhance the GA behavior using a biological property, namely the age, as a state variable to control the crossover probability. Fuzzy Logic Controller (FLC) enables the dynamical calculation of appropriate GA control parameters, based on experience and knowledge of the GA [17,22]. The fuzzy knowledge base (or fuzzy government [3]) is used for detecting the emergence of solutions, dynamically tuning algorithm parameters, as in [5,16,31], and monitoring the evolutionary process to avoid undesired behaviors such as premature convergence. Generally, an FLC allows one to qualitatively express the control strategies based on experience as well as intuition. For detailed description of FLC see appendix A.
3. Proposed method The proposed method of fuzzy-based lifetime extension of genetic algorithm enhances GAVaPS with a fuzzy logic controller, which adapts the crossover probability according to a fuzzy rule base, thus creating a fuzzy-based lifetime extension of genetic algorithms. The general structure of the proposed algorithm is shown in Fig. 4.
Procedure LifetimeExtendedGA { t=0 Random_initialize P(t) Evaluate P(t) While (not termination condition) { t= t+1 Increase the age of each individual by 1 Recombine P(t) { Select parents from P(t) For each pair of parents { Assign crossover probability (Pc) { Call Fuzzy Logic Controller (parameters) } Perform crossover with probability Pc Perform mutation with probability Pm Insert offsprings into P(t) } } Evaluate P(t) Remove from P(t) all individuals with age greater than their lifetime } }
Fig. 4. Fuzzy-based lifetime extended GA—general structure.
M. Last, S. Eyal / Fuzzy Sets and Systems 149 (2005) 131 – 147
Membership Function
136
Middle1 Young
0.2
age
0.3
Old
0.5
0.7
1
Age distribution
Fig. 5. The age linguistic variable.
3.1. Fuzzy logic controller (FLC) The FLC components were designed as follows: • State variables: age and lifetime of the chromosomes to be crossovered (parents) and the population average lifetime. All are crisp values. • Controlled parameter: crossover probability (= P c), a crisp value. 3.1.1. Fuzzification interface The fuzzification interface defines for each parent the possibilities of being {Young, Middle-age, Old}, which are linguistic labels of both inputs. These values determine the degree of truth for each rule premise. This computation takes into account only one parent at a time, and relies on the triangular membership functions shown in Fig. 5. In our case studies (see next section), we have experimented with three different settings of these functions. The relative age shown on the x-axis in Fig. 5 refers to the ratio between chromosome’s age and the average lifetime in the population. 3.1.2. Knowledge base Knowledge base enables the representation of the vague information mentioned earlier or any other intuitive information, which helps to emulate the natural process. Our general principle is that crossover probability should vary across age intervals as a function of lifetime: for both young and old individuals crossover probability is naturally low, while there is a certain age interval where this probability goes up. In this way, Exploration–Exploration relationship would maintain appropriateness: very young offsprings will be less crossovered enabling exploration capability, an old offspring being less crossovered and eventually dying out helps avoiding local optimum (premature convergence), and middle-age individuals that will be mostly crossovered, would enhance the exploitation aspect. The linguistic terms representing the range of each fuzzy variable are as follows: Age ∈ [Young, Middle-age, Old] P c ∈ [Low, Medium, High] The best configuration of the fuzzy rule base found by an extensive set of experiments [9] is described at Table 1.
M. Last, S. Eyal / Fuzzy Sets and Systems 149 (2005) 131 – 147
137
Table 1 Fuzzy rule base Parent I Young
Middle-age
Old
Young Middle-age Old
Low Medium Low
Medium High Medium
Low Medium Low
Membership Function
Parent II
1
Low
Medium High
0.2 0.5 0.8 Pc
1
Fig. 6. The crossover probability linguistic variable.
Each rule is assigned a relevance degree equal to the minimum value of its conditions. Afterwards, the inference engine assigns each linguistic output (Low, Medium, and High) a single value, which is the maximum value of its respective fired rules. This inference method is known as MAX-MIN method [22]. 3.1.3. DeFuzzification interface The Defuzzification interface computes a crisp value for our controlled parameter crossover probability according to the values of the linguistic labels {Low, Medium, High}, which are the outputs of the rule base, and the triangular membership functions shown in Fig. 6. The settings of the linguistic labels were chosen according to previous studies that have determined (experimentally) the recommended parameters’ values for the single point crossover that are applicable to a wide range of optimization problems [8]. The triangular shape of the membership function was chosen due to its common use as well as simplicity. The defuzzification method used is center of gravity, also called CENTROID: the crisp value of the output variable is computed by finding the center of gravity under the membership function for the fuzzy variable [22].
3.2. Lifetime calculation strategy The GAVaPS algorithm assigns a lifetime to each individual at its birth. Three lifetime calculation strategies are proposed, but none is determined as superior [23]. In our fuzzy-based algorithm, we chose
138
M. Last, S. Eyal / Fuzzy Sets and Systems 149 (2005) 131 – 147
the Bi-Linear allocation strategy, which is computed as follows: If AvgF it fitness[i] If AvgF it < fitness[i]
lif etime [i] = MinLT + · lif etime [i] =
f itness [i] − MinF it , AvgF it − MinF it
1 f itness [i] − AvgF it , (MinLT + MaxLT ) + · 2 MaxF it − AvgF it
where: =
1 · (maxLT − minLT ) 2
MaxLT , MinLT —allowed maximal and minimal lifetime, respectively. MaxF it, AvgF it, MinF it—the best, average, and worst fitness of the current population, respectively. Hence, better solutions get a longer lifetime and therefore remain in the population for more generations and will have higher chance of producing an offspring.
4. Experiments and discussion 4.1. Experimental design The following algorithms were examined and compared: 1. Simple GA [13], served as a benchmark in [1,2,10,11,29]. 2. GA with varying population size [2], which has been under comparison in [10,11]. 3. Adaptive Genetic Algorithm (AGA), where crossover and mutation probabilities are adapted as a function of solution fitness [28]. 4. Fuzzy-based lifetime extended GA. We experimented with three different combinations of the two parameters, as described in Table 2. The test cases used in this study are as follows: 1. Royal Road problem [20,24]. 2. Traveling Salesman Problem (TSP) [25]. 3. 100-bit OneMax problem [21,30].
Table 2 Fuzzy-based lifetime extension of GA—tested configurations Parameter
LTexGA#1
LTexGA#2
LTexGA#3
FLC—Young upper limit (%) FLC—Old lower limit (%)
30 70
25 75
20 80
M. Last, S. Eyal / Fuzzy Sets and Systems 149 (2005) 131 – 147
139
The following performance measurements were taken: 1. Performance ratio—number of runs yielding the optimal solution to the total number of runs. 2. Number of generations to achieve the optimal solution. 3. Genotypical Diversity (GD)—diversity measure that is calculated by comparing the inner structure of the chromosomes. For a binary chromosome (Royal Road and OneMax), the diversity was measured as follows: N N
D (p) =
Xi (j ) i=1 j =i , N (N −1) 2
Xi (j ) =
Xi (j ) =
L
Yij (k) ,
k=1
1, If the Kth gene of genome i equals to the Kth gene of genome j, 0, Otherwise,
where N is the population size, and L is the chromosome length. For a list of integer-valued chromosomes (TSP), the diversity was measured as follows: N N
D (p) =
Xi (j ) i=1 j =i , N (N −1) 2
Yij (m, n) =
1, 0,
Xi (j ) = L −
L L
Yij (m, n),
m=1 n=m
If cities m and n are neighbors in genomes i and j, Otherwise,
where N is the population size and L is the chromosome length. For instance, consider a 5-cities TSP and these two chromosomes: (3,5,1,2,4), (4,1,2,5,3). In this case, Xi (j ) = 3. But for (3,5,1,2,4) and (1,2,4,3,5), Xi (j ) = 0 because it is practically the same route. 4. Phenotypical Diversity (PD)—diversity measure that compares only the final solutions quality in each generation. PD =
fbest f
, where f and fbest are the average and best fitness, respectively [15].
Results were averaged over 30 runs in each experiment. 4.2. Summary of results 4.2.1. Royal road The Royal Road functions were designed to study building blocks’ interactions during the GA process. These functions are made up of a structure of building blocks that allows the monitoring of the GA process. The GA must follow a path formed by this structure—the “Royal Road”—to achieve the global optimum [20,24]. For this experiment, we used a genome of 240 bits and the highest level of 4. Each level represents an interval of fitness values, where level 1 means low fitness values and level 4 stands for the optimal fitness.
M. Last, S. Eyal / Fuzzy Sets and Systems 149 (2005) 131 – 147 RoyalRoad - Level Achived 5%
13%
55%
57%
85%
60%
60%
83%
35%
30%
2)
)
A
A
ex G A1
(#
G FL
FL
AG
A
AV aP S G
Si m pl eG
20%
15%
10%
0%
Level2 Level1 Level0
3)
20%
Level4 Level3
ex G A
40%
(#
Runs
60%
10%
20%
25%
80%
FL
17%
(#
100%
ex
140
Algorithm
Fig. 7. Level reached by each algorithm.
Complete function definition can be found in [20]. The settings of the Simple GA are: • Population size—defines the amount of individuals in the population in each generation. For this test case we used population size of 512 chromosomes. • Number of generations—number of iterations in a single run, usually used as a stopping criterion. We used 400 generations. • Roulette wheel selection—a method to select chromosomes based on the magnitude of the fitness value relative to the rest of the population: chromosome with higher fitness has a higher probability to be chosen. Practically, the probability for a chromosome to be chosen is equal to its fitness divided by the sum of fitness values of each chromosome in the population. • 1-point crossover method: A single crossover position is chosen at random and the genes (bits) of two parents before and after the crossover position are exchanged to form two offsprings. • Crossover probability: the probability of two chosen chromosomes to be crossovered. We used crossover probability of 0.9. • Flip mutation: given chromosome, every bit value changes with a mutation probability. • Mutation probability: the probability mutation will happen. We used 0.01. • 1-elitism: transfer of the best chromosome (one or more) from the current population to the next one “as is”. This feature is a part of the simple genetic algorithm [13]. GAVaPS used the same values except for crossover probability (0.65) and mutation probability (0.005), because they were found more suitable in [11]. Both GAVaPS and Fuzzy-based lifetime extended GA used: • Minimum lifetime: minimal number of generations a chromosome may remain in the population. We chose a value of 1. • Maximum lifetime: maximal number of generations a chromosome may remain in the population. We chose a value of 7. • Reproduction ratio: the maximum population growth allowed within a single generation. We chose a value of 0.4. • Lifetime calculation strategy: Bi-Linear (as described earlier). AGA parameters (k1, k2, k3, k4) were set to the values recommended in [28]. Fig. 7 shows the levels reached by each algorithm, and Table 3 presents the average number of generations needed to achieve each of the four possible levels. For GAVaPS and Simple GA, similar results were found in [10]. The obtained results clearly show that the Fuzzy-based lifetime extension of GA
M. Last, S. Eyal / Fuzzy Sets and Systems 149 (2005) 131 – 147
141
Table 3 Number of generations to reach levels for each algorithm Level
Simple GA
GAVaPS
AGA
LTexGA#1
LTexGA#2
LTexGA#3
Level 1 Level 2 Level 3 Level 4
5.9 65.7 310.4 —
12.909 48.545 159.44 368.67
8.388 60 — —
10.85 40.35 150.11 207
11.1 40.56 261.3 336.5
12.1 44.4 214.3 339.5
Royal Road - Diversity 0.8 0.7 SimpleGA GAVaPS FLexGA(#1) FLexGA(#2) FLexGA(#3) AGA
Diversity
0.6 0.5 0.4 0.3 0.2 0.1 0 1
Generations
400
Fig. 8. Diversity of each algorithm along the run.
outperformed significantly (according to t-test with = 0.05) the Simple GA, the AGA and even the GAVaPS: it reached a higher level in most runs and in fewer generations (except level 1). Among the three, the first configuration of the fuzzy membership functions performed best in terms of both performance measures. Fig. 8 shows the genotypical diversity of each algorithm. The best performance was achieved with the second configuration, with minor but statistically significant differences according to t-test with = 0.05, still much better than the Simple GA and better than GAVaPS: The diversity has converged to a reasonable value and stayed stable along the run.AGA has converged prematurely and maintained a very low diversity, which correlates to its poor performance (see Fig. 7). 4.2.2. Traveling salesman problem (TSP) The TSP involves finding a cyclic path in a graph such that all vertices (e.g. cities) are visited exactly once. The objective is to find a route of minimum total distance (weight). The TSP is an NP-complete problem. Three problems were chosen from the TSPLIB [25], all three are symmetric TSPs with Euclidean distance: Eil51, Eil76, Rat99 (51, 76 and 99 cities). The complete definitions can be found in [25]. The settings of the Simple GA for this problem are: • Population size: 100. • Number of generations: 200. • Roulette wheel selection. • Partial match crossover [23]—crossover used for the TSP because it always creates feasible solution (unlike 1-point crossover). • Crossover probability: 0.7.
142
M. Last, S. Eyal / Fuzzy Sets and Systems 149 (2005) 131 – 147
Table 4 Shortest and average path length achieved by each algorithm Problem
SimpleGA
GAVaPS
AGA
LTexGA#1
LTexGA#2
LTexGA#3
Best known
Eil51
1,094.86 (1,225.21) 1,792.1 (1,917.3) 5,729.7 (6,122.35)
729.5 (812.23) 1,252.31 (1,301.2) 3,742.42 (4003.8)
1,053.91 (1,160.35) 1,674.49 (1,856.16) 5,624.53 (5,954.41)
525.46 (584.2) 925.846 (1,011.53) 2,664.5 (2,922.35)
611.72 (665.25) 1,067.5 (1,113.31) 2,983.71 (3,183.28)
692.66 (748.45) 1,107.38 (1,173.4) 3,199.8 (3,443.64)
426
Eil76 Rat99
538 1,211
• Swap mutation—given a chromosome, it inverts the order of bits between two selected positions chosen at random. • Mutation probability: 0.01. • 1-elitism. Both GAVaPS, AGA and Fuzzy-based lifetime extended GA used the same additional parameters as described in the previous case study. Table 4 presents the shortest and the average paths achieved within 30 runs of 200 generations each. All three configurations of the fuzzy-based lifetime extension of GA performed better than the Simple GA, the AGA, and the GAVaPS, while the first configuration is preferable: In Eil51 it averaged a total route distance which is 5 times closer to the best known result vs. the one achieved by the Simple GA and 2 times closer than GAVaPS. Its average distance is also better than the second and the third configurations (1.5 and 2 times closer). AGA performed better, though not significantly, than Simple GA, but fell behind GAVaPS and furthermore behind the proposed method. In Eil76 and Rat99 the first configuration still produced a shorter distance than the other two, though with smaller differences than in Eil51: In Eil76, the first configuration reached an average distance about 3 times closer to the best known result than the Simple GA and 1.6 times closer than the GAVaPS. In Rat99 the differences were reduced by 2.8 times and 1.65 respectively. As expected, when the number of cities grows, given a fixed number of generations, the improvement decreases, but it still exists. Comparison to the best results that can be achieved with each configuration has not taken place because of the extremely large number of required generations. In our experiment, we run only 200 generations while in [1] the best solution for Eil51 was achieved after as many as 383,139 generations and the best solution for Eil76 after 329,516 generations. Figs. 9, 10, 11, describe the different genotypical diversities for each problem. In this case study, the best performance was achieved with the third configuration, with significant differences. The fuzzy based lifetime extended GA reached better results that the Simple GA, AGA and GAVaPS most of the time, but longer runs are needed to validate this assumption. Still, one can infer from the overall performance that the fuzzy-based method is preferable for this case study. 4.2.3. Onemax problem In this case study, also known as counting ones problem, the fitness of a given string is simply the number of ones the string contains [21,30]. The aim is to obtain a string containing all ones. Detailed studies concerning this problem can be found in [4,12,21,30]. A string length of 100 was used for the
M. Last, S. Eyal / Fuzzy Sets and Systems 149 (2005) 131 – 147 Eil51 Diversity 55
Diversity
50
SimpleGA GAVaPS
45
LTexGA#1 LTexGA#2
40
LTexGA#3 AGA
35 30 0
100 Generations
200
Fig. 9. Eil51 problem—diversity of each algorithm.
Eil76 Diversity 80 75
Diversity
SimpleGA 70
GAVaPS
65
LTexGA#1 LTexGA#2
60
LTexGA#3 AGA
55 50 0
100 Generations
200
Fig. 10. Eil76 problem—diversity of each algorithm.
Rat99 Diversity 98
Diversity
96 94
SimpleGA GAVaPS
92
LTexGA#1 LTexGA#2
90
LTexGA#3 AGA
88 86 84 0
Generations
200
Fig. 11. Rat99 problem—diversity of each algorithm.
purposes of this study. The Simple GA settings are: • Population size: 100. • Number of generations: 200. • Roulette wheel selection. • 1-point crossover with probability of 0.9. • Flip mutation with probability of 0.01. • 1-elitism.
143
144
M. Last, S. Eyal / Fuzzy Sets and Systems 149 (2005) 131 – 147
Table 5 Statistics of the fitness reached by each algorithm Fitness statistic
SimpleGA
GAVaPS
AGA
LTexGA#1
LTexGA#2
LTexGA#3
Worst Average Best Std. Dev
86 90.90 95 2.006
97 98.07 99 0.785
66 70.60 80 3.050
94 95.80 99 1.064
93 95.30 98 1.119
94 95.33 97 0.994
Table 6 Number of generations each algorithm needed to reach optimal solution Statistic
SimpleGA
GAVaPS
AGA
LTexGA#1
LTexGA#2
LTexGA#3
Worst Average Best Std. Dev
747 3,502 1,437 793.054
280 999 529 247.828
— — — —
357 1,011 620 213.806
236 971 629 262.242
473 1,009 739 184.963
GAVaPS and Fuzzy-based lifetime extended GA used the same additional parameters as described in the previous case study except for the reproduction ratio (0.3). Table 5 presents the statistics of the fitness reached by each algorithm after 200 generations: GAVaPS and LTexGA algorithms reached significantly higher fitness (t-test with = 0.05) than the Simple GA, by about 5–8%, though there are some differences vs. the first two experiments: • The difference between the GAVaPS and the lifetime extended GA is statisitically significant (t-test with = 0.05) though the GAVaPS average performance surpasses the best configuration of our algorithm by 2.3% only. • The first configuration of the fuzzy-based lifetime extension of GA is better than the other two by nearly 0.5%. However, there is no significant difference between the average fitness values of the three different configurations. • AGA performed significantly worse than the Simple GA. Table 6 presents the statistics of the number of generations needed to reach the optimal solution (= 100). The second configuration of the fuzzy-based lifetime extended GA has reached the optimal solution faster than GAVaPS by nearly 2.8%. AGA statistics are missing because it did not reach the optimal solution even after 15,000 generations. Figs. 12 and 13 describe the phenotypical and genotypical diversities respectively. Both figures show that except AGA, all four algorithms have reached similar results, which are preferable over the Simple GA. According to the t-test with = 0.05, there is no significant difference between GAVaPS and the fuzzy-based lifetime extended GA, while across the three configurations for the fuzzybased lifetime extended GA there is no significant difference (t-test with = 0.05) with respect to the diversity measure.
M. Last, S. Eyal / Fuzzy Sets and Systems 149 (2005) 131 – 147
145
Phenotype Diversity 0.3 0.25 SimpleGA GAVaPS LTexGA#1
Diversity
0.2 0.15
LTexGA#2 LTexGA#3
0.1
AGA
0.05 0 1
51
101 Generations
151
200
Fig. 12. OneMax problem—phenotype diversity of each algorithm.
Genotype Diversity 0.6
Diversity
0.5 0.4
SimpleGA GAVaPS
0.3
LTexGA#1 LTexGA#2
0.2
LTexGA#3 AGA
0.1 0 0
100 Generations
200
Fig. 13. OneMax problem—genotype diversity of each algorithm.
5. Conclusions and future work This paper presented the Fuzzy-based lifetime extension of GA, which enables the use of ill-structured experience and vague information, as well as the human intuition. We used this capability to improve the emulation of the biological process by dynamically adapting the crossover probability. However, other parameters, preferable to those derived from nature, may be modeled as well. The Fuzzy-based lifetime extension of GA outperformed the Simple GA, the AGA, and the GAVaPS in all case studies: at least one fuzzy-based configuration reached better results in fewer generations. With a fixed number of generations, it reached higher fitness than GAVaPS in two cases out of three, while in the OneMax problem it was only slightly worse than GAVaPS. One explanation is that in some problems, there exists a relationship between the membership function and the objective function. Logically, it is possible that with a suitable membership function the lifetime extended GA would perform better than GAVaPS in the third case study as well, but it is an open issue to be explored. Within the fuzzy-based lifetime extension of GA, the first configuration, which defines up to 30% as young and 70% as old, was found to be preferable most of the times, which seems reasonable when we consider the biological process. Still, the study of these parameters, as well as the settings of the “expert knowledge”, requires further research and experimentation.
146
M. Last, S. Eyal / Fuzzy Sets and Systems 149 (2005) 131 – 147
Knowledge base
Fuzzification Interface
State Variables
Inference System
Defuzzification Interface
Controlled System
Control parameters
Fig. 14. Fuzzy logic controller generic structure.
Appendix A. The generic structure of an FLC is shown in Fig. 14. The FLC components are: • Knowledge base—Database, containing the definition of the fuzzy control rules linguistics labels (i.e. the membership functions of the fuzzy sets specifying the meaning of the linguistic terms) and a rule base created by collection of fuzzy control rules representing the expert knowledge. • Fuzzification interface—transformation of crisp data into fuzzy sets based on pre-defined membership functions. • Inference system—reasoning about the fuzzy sets based on the knowledge base. Inference methods consist of forward chaining inference. • Defuzzification interface—translation of the fuzzy control action into a real control action using defuzzification method. • Controlled system—the Genetic algorithm in our case. In addition, it is most important to define the FLC state variables, which include, in the case of a GA, measures describing the GA behavior, such as population diversity, maximum, average, and minimum fitness, as well as the controlled parameters, such as population size, crossover and mutation probability, etc. References [1] M. Annuziato, S. Pizzuti, Adaptive parameterization of evolutionary algorithms driven by reproduction and competition, atti di ESIT2000 European Symp. on Intelligent Techniques, Aachen (Germania), September 2000. [2] J. Arabas, Z. Michalewicz, J. Mulawka, GAVaPS—a Genetic Algorithm with varying population size, in: Proc. of the first IEEE Conf. on Evolutionary Computation,1994, pp. 73–78. [3] S. Arnone, M. Dell’Orto, A. Tettamanzi, Toward a fuzzy government of genetic populations, in: Proc. of the 6th IEEE Conf. on Tools with Artificial Intelligence,1994, pp. 581–585. [4] T. B”ack, Optimal mutation rates in genetic search, in: S. Forrest (Ed.), Proc. of the Fifth Internat. Conf. on Genetic Algorithms, San Mateo, CA, Morgan Kaufmann, Los Altos, CA, 1993, pp. 2–9. [5] A. Bergman, W. Burgard, S. Hemker, Adjusting parameters of genetic algorithms by fuzzy control rules, in: K.-H. Becks, D. Perret-Gallix (Eds.), New Computing Techniques in Physics Research III, World Scientific, Singapore, 1993. [6] J. Costa, R. Tavares, A.C. Rosa, An experimental study on dynamic random variation of population size, in: Proc. of the 1999 IEEE Internat. Conf. on Systems, Man, and Cybernetics,1999, pp. 607–612.
M. Last, S. Eyal / Fuzzy Sets and Systems 149 (2005) 131 – 147
147
[7] K. Deb, S. Agrawal, Understanding interactions among genetic algorithm parameters, in: W. Banzhaf, C. Reeves (Eds.), Foundations of Genetic Algorithm 5, Morgan Kaufmann, San Francisco, CA, 1998, pp. 265–286. [8] A.E. Eibon, R. Hinterding, Z. Michalewicz, Parameter control in evolutionary algorithm, IEEE Trans. Evolution. Comput. (1999) 124-141. [9] S. Eyal, A fuzzy-based age extension of genetic algorithm, Master’s Thesis, Department of Information Systems Engineering, Faculty of Engineering, Ben-Gurion University, 2003. [10] C. Fernandes, A. Rosa, A study on non-random mating and varying population size in genetic algorithms using royal road function, in: Proc. of the 2001 Congress on Evolutionary Computation CEC2001,2001, pp. 60–66. [11] C. Fernandes, R. Tavares, A. Rosa, niGAVaPS—Outbreeding in genetic algorithms, in: Proc. of the 2000 ACM Symp. on Applied Computing, Villa Olmo, Como, Italy, 2000. [12] P. Giguere, D.E. Goldberg, Population sizing for optimum sampling with genetic algorithms: a case study of the onemax problem, in: J.R. Koza et al. (Eds.), Proc. Genetic Programming, Morgan Kaufmann, Los Altos, CA, 1998, pp. 496–503. [13] D.E. Goldberg, Algorithm in Search Optimization and Machine Learning, Addison-Wesley, Reading, MA, 1989. [14] F. Herrera, M. Lozano, E. Herrera-Viedma, J.L. Verdegay, Fuzzy tools to improve genetic algorithms, in: Proc. Second European Conf. on Intelligent Techniques and Soft Computing (EUFIT’94), Vol. 3, September 1994, pp. 1532–1539. [15] F. Herrera, M. Lozano, Adaptation of genetic algorithm parameters based on fuzzy logic controllers, in: F. Herrera, J. Verdegay (Eds.), Genetic Algorithms and Soft Computing, 1996, pp. 95–125. [16] F. Herrera, M. Lozano, Adaptive genetic operators based on coevolution with fuzzy behaviors, IEEE Trans. Evolution. Comput. 5 (2) (2001) 1–18. [17] F. Herrera, M. Lozano, Fuzzy genetic algorithms: issues and models, Technical Report DECSAI-98116, Department of Computer Science and A.I., University of Granada, June 1998. [18] J.H. Holland, Adaptation in Natural and Artificial Systems, MIT Press, Cambridge, MA, 1975. [19] J.H. Holland, Genetic algorithms, Sci. Am. 267 (1) (1992) 44–150. [20] J.H. Holland, Royal Road Functions. Genetic algorithms, GAList Internet mailing list, Digest, Vol. 7(22), Article dated August 7, 1993. [21] J. Horn, C.R. Reeves, The genetic crossover landscape for the onemax problem, in: J. Alander (Ed.), Proc. of the 2nd Nordic Workshop on Genetic Algorithms and their Applications, University of Vaasa Press, Vaasa, Finland, 1996, pp. 27–43. [22] G.J. Klir, B. Yuan, Fuzzy Sets and Fuzzy Logic: Theory and Applications, Prentice-Hall Inc., Englewood Cliffs, NJ, 1995. [23] Z. Michalewicz, Genetic algorithms + Data structures = Evolution programs, Verlag, Heidelberg, Berlin, Third Revised and Extended Edition, 1999. [24] M. Mitchell, An Introduction to Genetic Algorithms, MIT Press, Cambridge, MA, 1996. [25] G. Reinelt, ‘TheTSPLIB’, http://www.iwr.uni-heidelberg.de/iwr/comopt/soft/TSPLIB95/TSPLIB.html. [26] Y. Shi, R. Eberhart, Y. Chen, Implementation of evolutionary fuzzy systems, IEEE Trans. Fuzzy Syst. 7 (2) (1999) 109– 119. [27] W.M. Spears, Adapting crossover in evolutionary algorithms, in: Proc. of 4th Annu. Conf. on Evolutionary Computing, Ed Fogel,MIT Press, Cambridge, MA, 1995. [28] M. Srinivas, L.M. Patnaik, Adaptive probabilities of crossover and mutation in genetic algorithms, IEEE Trans. Systems, Man Cybernet. 24 (4) (1994) 656–667. [29] R.J. Streifel, R.J. Marks II, R. Reed, J.J. Choi, M. Healy, Dynamic fuzzy control of genetic algorithm parameter coding, IEEE Trans. Systems, Man Cybernet. Part B: Cybernetics 29 (3) (1999) 426–433. [30] G. Syswerda, Uniform crossover in genetic algorithms, in: J.D. Schaffer (Ed.), Proc. of the Third Internat. Conf. on Genetic Algorithms and their Applications, Morgan Kaufmann, San Mateo, 1989, pp. 2–9. [31] L. Takagi, Dynamic control of genetic algorithms using fuzzy logic techniques, in: S. Forrest (Ed.), Proc. of the 5th Internat. Conf. On Genetic Algorithms (ICGA), 1993, pp. 76–83.
