Hybrid krill herd algorithm with differential evolution for global ...
Neural Comput & Applic (2014) 25:297–308 DOI 10.1007/s00521-013-1485-9
ORIGINAL ARTICLE
Hybrid krill herd algorithm with differential evolution for global numerical optimization Gai-Ge Wang • Amir H. Gandomi Amir H. Alavi • Guo-Sheng Hao
•
Received: 9 December 2012 / Accepted: 10 September 2013 / Published online: 1 October 2013 Ó Springer-Verlag London 2013
Abstract In order to overcome the poor exploitation of the krill herd (KH) algorithm, a hybrid differential evolution KH (DEKH) method has been developed for function optimization. The improvement involves adding a new hybrid differential evolution (HDE) operator into the krill, updating process for the purpose of dealing with optimization problems more efficiently. The introduced HDE operator inspires the intensification and lets the krill perform local search within the defined region. DEKH is validated by 26 functions. From the results, the proposed methods are able to find more accurate solution than the KH and other methods. In addition, the robustness of the DEKH algorithm and the influence of the initial population size on convergence and performance are investigated by a series of experiments. Keywords Global optimization problem Krill herd (KH) Hybrid differential evolution (HDE) operator
G.-G. Wang (&) G.-S. Hao School of Computer Science and Technology, Jiangsu Normal University, Xuzhou, Jiangsu 221116, China e-mail: [email protected] G.-S. Hao e-mail: [email protected] A. H. Gandomi Department of Civil Engineering, The University of Akron, Akron, OH 44325, USA e-mail: [email protected]; [email protected] URL: http://www.gozips.uakron.edu/*ag72/ A. H. Alavi Department of Civil and Environmental Engineering, Michigan State University, Engineering Building, East Lansing, MI 48824, USA e-mail: [email protected]; [email protected] URL: http://www.egr.msu.edu/*alavi/
1 Introduction In the current world, the resources are becoming less and less. Due to the limited resources, all the people are striving for making most profit with least cost and resources. These problems can be modeled into maximum/minimum problem, which are also called optimization problems. And then, various technologies are well capable of solving these optimization problems. Generally speaking, there are various ways to divide these technologies; however, a simple way is to look at the property of the techniques. These techniques can be classified into two categories: traditional classical methods and modern intelligent algorithms. The prior has strict step and always generate the same solution with the same starting value. Contrarily, modern intelligent algorithms, based on some stochastic distribution, cannot generate the same solutions under any conditions. In most cases, both can find the satisfactory solution. Recently, nature-inspired metaheuristic methods have emerged, and they perform better than traditional methods in solving optimization problems. These robust metaheuristic methods have been designed to solve complicated problems, such as cluster [1], truss structure design [2], UCAV path planning [3], permutation flow shop scheduling [4], knapsack problem [5], task assignment problem [6], and engineering optimization [7, 8]. These kinds of metaheuristic methods can take advantage of the useful information from the whole population. Therefore, they have a higher probability of searching for the final satisfactory solution. After the emergence of the genetic algorithms (GAs) [9–11] during the 1960s and 1970s, serials of techniques have been developed in the field of optimization, such as differential evolution (DE) [12–14], harmony search (HS) [15–17], artificial bee colony (ABC) [18], particle swarm optimization (PSO) [19–21], evolutionary strategy (ES) [22, 23], ant colony optimization (ACO) [24], firefly algorithm (FA) [25], artificial
123
298
plant optimization algorithm (APOA) [26], artificial physics optimization [27], biogeography-based optimization (BBO) [28], genetic programming (GP) [29], cuckoo search (CS) [30–32], animal migration optimization (AMO) [33], probability-based incremental learning (PBIL) [34], big bang–big crunch algorithm [35–38], bat algorithm (BA) [39, 40], charged system search (CSS) [41], and the KH method [42]. According to the idealization of the social phenomenon of krill, KH has been developed for optimizing complicated functions [42]. In KH, the objective function can be idealized into the distances from food source and its herd density. The position of the krill involves three components. A major advantage of KH is that it requires few control variables compared with other methods. In some cases, KH may fail to find the best solution on multimodal functions. Herein, an effective hybrid differential evolution krill herd (DEKH) method is proposed so as to accelerate convergence speed. In DEKH, firstly, we use the KH to shrink the search apace and select a good promising candidate solution set. Thereafter, for making the krill proceed to the best solution, a more focused hybrid differential evolution (HDE) operator is introduced into the algorithm. This added operator is utilized to search promising region to get optimal solutions for solving optimization problems. Some solutions would be updated by HDE operator around the optimal krill, and this will accelerate the convergence speed. Proposed method is validated on twenty-six functions. Experimental results indicate that the DEKH is able to find the better solution than the KH, other eight methods.
2 A review of hybrid metaheuristic methods It is well-known that a hybrid method can take merits of the methods that it was originated from. Some most representative hybrid methods are reviewed here. El-Abd [43] investigated the capabilities of two swarm intelligence algorithms, namely ABC and PSO and combined into one method, called ABC–PSO. ABC–PSO is a component-based technique where the PSO is expanded with an ABC to enhance the optimal particles. Wang and Guo [44] enhanced the diversity of the bat swarm by adding pitch adjustment operation in HS to the BA method. Based on this mutation operator, a hybrid HS/BA method is proposed and tested on various benchmarks. Duan et al. [45] proposed a hybrid method by merging the BBO together with the TS (tabu search) to deal with the test-sheet composition problem. HS cannot always find a satisfactory solution. In order to overcome this limitation, Gao et al. [46] developed a new HS– PBIL method through the combination of HS and PBIL. Geem [47] proposed an improved HS incorporating particle swarm concept. Gong et al. [48] introduced a hybrid DE and BBO method to merge the exploration of DE together with the
123
Neural Comput & Applic (2014) 25:297–308
exploitation of BBO. In order to cope with the difficulty of setting the number of clusters in most of the clustering methods, Kuo [49] established a novel clustering method based on PSO and GA (DCPG). DCPG can cluster data by means of testing of the data without a cluster number. In 2012, a hybrid co-evolutionary cultural algorithm based on PSO, namely CECBPSO, is developed by Sun et al. [50]. In [51], two new models are proposed for short-term electricity load forecasting. These models use ACO and hybridization of GA and ACO for feature selection and multi-layer perceptron for load prediction. Marichelvam [52] developed an improved hybrid CS algorithm to deal with the permutation flow shop scheduling problems.
3 The proposed DEKH method KH [42] is a heuristic search technique for solving function optimization, which is the idealization of the krill swarm in the process of hunting for the food and communicating with each other. The position of a krill can be idealized into three actions: 1. 2. 3.
movement influenced by other krill; foraging action; physical diffusion. KH used the Lagrangian model in Eq. (1).
dXi ¼ Ni þ Fi þ Di dt
ð1Þ
where Ni is the motion induced by other krill individuals; Fi is the foraging motion, and Di is the physical diffusion of the ith krill individual. The direction of the first movement, ai, is estimated by the target effect, local effect, and a repulsive effect. For a krill individual, this movement can be formulated as: Ninew ¼ N max ai þ xn Niold
ð2Þ
And Nmax is the maximum induced speed, xn is the inertia weight, Niold is the last motion induced. The second movement is affected by the two factors: the current food location and the previous experience. For the ith krill, it can be given below: Fi ¼ Vf bi þ xf Fiold
ð3Þ
where bi ¼ bfood þ bbest i i
ð4Þ
and Vf is the foraging speed, xf is the inertia weight, Fiold is the last foraging motion. The third movement can be considered to be a random process. It can be expressed according to a diffusion speed and a directional factor. It can be formulated as follows:
Neural Comput & Applic (2014) 25:297–308
Di ¼ Dmax d
299
ð5Þ
where Dmax is the maximum diffusion speed and d is the random directional vector, and its arrays are values in [-1, 1]. Herein, the position of a krill in the Dt is formulated by the Eq. (6): Xi ðt þ DtÞ ¼ Xi ðtÞ þ Dt
dXi dt
ð6Þ
For more details on this method, readers are advised to refer to Gandomi and Alavi [42]. For the third motion of the KH method, how a krill moves toward next position is mainly based on random walks; therefore, the satisfactory solution cannot always be found every time. In order to remove this disadvantage, some kind of genetic reproduction mechanisms have been added to KH approach [42]. For most cases, KH is well capable of making full use of the population knowledge and has proven good performance on some benchmarks. However, sometimes, KH may not proceed to the optimal solutions on some complex problems. Here, in order to further enhance the exploitation of KH, a more focused hybrid mutation operator performing local search, called hybrid differential evolution (HDE) operator, is incorporated into the KH to form a novel differential evolution KH (DEKH) method. This more focused local search technique can make the population not converge prematurely. The greedy selection strategy is used in this operator. In other words, the newly generated krill replace the previous one only when its fitness is better than before. Therefore, in this way, the HDE operator works better than mutation. The HDE operator used in DEKH algorithm can be represented in Algorithm 1.
problem, if the newly generated krill has a smaller fitness, it will be in place of the old one. In other words, it can be considered as krill i in the next generation. In DEKH, standard KH algorithm is utilized to search globally for the purpose of making most krill move toward a more promising area. After exploration stage, HDE operator is utilized to search locally with a small step to get the final best solution. Based on the mainframe of DEKH, the KH emphasizes the diversification at the beginning of the search with a big step to explore the search space extensively and evade trapping into local optima, while later HDE operator focuses on the intensification and lets the individuals move toward the best solution at the later stage of the optimization process. In addition, this method is able to cope with the conflict global and local search effectively. In addition, except HDE operator, elitism strategy is also added to the DEKH. Like other methods, some kind of elitism is introduced with the aim of keeping the optimal krill in the population, which can forbid the optimal krill from being ruined by incorrect position updating operator. This elitism strategy is able to make all the krill always proceed to the population with better fitness than before. By incorporating the above-mentioned HDE operator and focused elitism strategy into the KH algorithm, the DEKH has been developed, as described in Fig. 1. Here, d is dimension of the function. NP is the parent population size. FW is the mutation scaling factor. CR is a crossover constant. Xi(j) is the j-dimension for Xi. Ui, Vi are the offspring. r1, r2, and r3 are random integers in [1, NP].
4 The results
≤
In the present work, various test functions, as shown in Table 1, are employed to evaluate our DEKH method. In order to get fair results, all the experiments were done under the same conditions as shown in [53]. More details about these test functions can be found in recent literatures [28, 54, 55]. 4.1 The robustness of DEKH
In Algorithm 1, the mutation and crossover operators are used to generate new position for krill i. For a minimum
As mentioned ‘‘robust’’ several times, it is required to show the robust feature of the DEKH method here. To do this, firstly select a certain benchmark problem (F24 Sum function). And then, we carry out this 100 times to get its statistic characteristics of the final solutions (mean, standard deviation of each variable in the final solution sets), as shown in Table 2. The relation between the evolutions of the fitness and number of generations is represented in Fig. 2. From Table 2, DEKH is able to get the optimal value for F24 Sum function among ten methods. For the
123
300
Neural Comput & Applic (2014) 25:297–308 Table 1 Benchmark functions No.
Name
F01
Ackley
F02
Alpine
F03
Brown
F04
Dixon & Price
F05
Fletcher-Powell
F06
Griewank
F07
Holzman 2 function
F08
Levy
F09
Pathological function
F10
Penalty #1
F11 F12
Penalty #2 Perm #1
F13
Perm #2
F14
Powel
F15
Quartic with noise
F16
Rastrigin
F17
Rosenbrock
F18
Schwefel 2.26
F19
Schwefel 1.2
F20
Schwefel 2.22
F21
Schwefel 2.21
F22
Sphere
F23
Step
F24
Sum function
F25
Zakharov
F26
Wavy1
worst values, the values of ABC, DE, DEKH, GA, KH, and PSO are slightly different, which are better than others. Similar to best value, DEKH is well capable of getting the minimum on the average. In addition, the Std of DEKH is the minimum. It is demonstrated that the values obtained by DEKH have little difference from different implementations. 4.2 The superiority of DEKH In order to prove the merits of DEKH, it was compared with nine methods, which are ABC [18], ACO [24], DE [12, 56, 57], ES [23], GA [9], HS [15], KH [42], PBIL [34], and PSO [19, 58, 59]. In addition, by combination of different genetic operators, four variants of KHs are developed, as described below [42]: KH KH KH KH Fig. 1 Flowchart of DEKH algorithm
123
I. KH without any genetic operators; II. KH with crossover operator; III. KH with mutation operator; IV. KH with crossover and mutation operators [42].
Through various experiments on benchmark problems, Gandomi and Alavi [42] have claimed that the KH II is the
Neural Comput & Applic (2014) 25:297–308
301
Table 2 The best, worst, mean, and standard of the F24 Sum function ABC
ACO
DE
DEKH
ES
GA
HS
KH
PBIL
PSO
Best
140.04
317.93
112.65
0.18
730.56
152.52
999.80
29.22
1.11E3
303.92
Worst
906.99
1.48E3
906.99
906.99
2.05E3
906.99
2.30E3
906.99
2.78E3
906.99
Mean
422.13
964.83
192.05
1.44
2.05E3
467.83
2.05E3
84.93
2.37E3
735.84
Std
133.74
288.96
41.39
1.00
374.75
172.91
355.51
26.67
364.61
176.15
4000 ABC ACO DE DEKH ES
benchmark function value
3500 3000
GA HS KH PBIL PSO
2500 2000 1500 1000 500 0
0
5
10
15
20
25
30
35
40
45
50
Number of generations
Fig. 2 The evolutions of the objective function for ten methods
best method among four variants of KHs. Herein, in the present work, KH II is considered as one of the comparative methods. In other words, in the next texts, when KH is mentioned, it means KH II. For KH and DEKH, the same parameters will be used: the Vf = 0.02, Dmax = 0.005, Nmax = 0.01, FW = 0.1, and CR = 0.4. For others, we set the same parameters as [53]. Note that, for each function, it has 20 variables. To get representative performances for each algorithm, we did 150 implementations on test functions. The results are recorded in Tables 3 and 4, which illustrate the average and mean minima, respectively. The function minimum is highlighted through bold font. The values in the following tables are normalized according to its own scale, the detailed process of which can be found in [55]. From Tables 3 and 4, it can be seen that, DEKH can find function minimum on twenty-four of the twenty-six benchmarks. GA performs the best on the function F19. Furthermore, ES and ACO overtakes all other methods on F09 for the best values and F24 for the average values, respectively. In addition, to provide more proof to testify the advantage of the DEKH, optimization process of ten
methods on some most typical benchmarks are also provided in this section, as shown in Figs. 3, 4, 5 and 6. The solutions are the true average objective function minimum. Figure 2 clearly indicates DEKH is the best method for solving F02 Alpine function from start to finish. For other methods, KH is simply inferior to the DEKH and it eventually finds the function value close to DEKH; while other methods except KH and DEKH, do not search for the satisfactory global minimum. Here, all the methods start the optimization process form almost same value; however, DEKH outperforms them soon. For this problem, DEKH outperforms all others from start to finish. Other methods do not find the satisfactory solution. Clearly, KH converges to the value that is only inferior to DEKH’s. For others, DE can search for the function value that is almost the same with ABC. Table 3 and Fig. 4 show that ABC performs slightly better than DE for this case. For this case, DEKH outperforms all other methods from start to finish. Other methods fail to find the satisfactory solution under certain conditions. Moreover, for other methods, similarly, ABC can search for the function value that is almost the same with KH. Table 2 and Fig. 5 also shows ABC is slightly superior to KH for this function. KH performs very well and ranks 3 among ten approaches. For this case, DEKH is the best method that can search for the global minimum and outperforms all others. ACO performs very well and ranks 2 among ten approaches. 4.3 The difference of initial population size The above comparisons are based on same number of initial population. If we use more individuals in the initial population in the methods, in other words, the diversity of methods is increased. Here, the influence of initial population size will be showed through various simulations on benchmark F01–F14. In the present work, the initial population size NP = 50, 100, 150, and 200 is selected. Because the initial population size NP = 50 is used in the experiments conducted in Sects. 4.1–4.2, the results can be found in the first 15 rows of the Tables 3 and 4 for
123
302
Neural Comput & Applic (2014) 25:297–308
Table 3 Mean normalized function values ABC
ACO
DE
DEKH
ES
GA
HS
KH
PBIL
PSO
F01
11.92
13.81
10.76
1.00
16.84
15.14
17.19
3.87
17.44
14.50
F02
5.63
9.12
10.47
1.00
19.63
8.91
17.40
5.90
20.49
13.36
F03
68.31
121.42
19.76
1.00
947.04
62.02
516.97
96.27
192.64
453.86
F04
754.82
1.2E3
222.33
1.00
8.9E3
467.10
9.0E3
96.78
1.0E4
1.2E3
F05
2.73
10.83
4.15
1.00
10.43
4.23
10.17
4.06
9.43
8.36
F06
38.24
12.10
21.33
1.00
98.94
40.45
197.31
6.11
218.71
81.30
F07
1.3E3
2.4E3
543.81
1.00
1.7E4
1.5E3
1.7E4
161.62
2.1E4
3.8E3
F08
22.43
37.70
24.73
1.00
119.08
32.84
108.72
13.03
130.50
68.53
F09
2.93
3.12
1.69
2.47
1.00
2.45
3.71
3.54
3.22
2.64
F10
2.6E5
1.2E7
5.7E4
1.00
7.6E6
8.3E4
1.3E7
2.7E3
1.6E7
1.4E6
F11 F12
3.5E5 1.1E11
7.5E6 3.8E10
9.5E4 1.1E6
1.00 1.00
4.5E6 2.6E8
2.8E5 1.3E10
6.7E6 1.7E8
9.7E3 1.6E11
8.7E6 7.4E9
1.3E6 1.3E8
F13
4.6E12
5.6E12
8.4E7
1.00
4.2E10
5.0E12
3.6E10
8.7E12
3.6E12
4.2E10
F14
14.81
90.22
36.61
1.00
128.34
20.83
98.18
12.90
107.42
42.65
F16
1.7E3
1.3E3
579.54
1.00
1.7E4
1.3E3
1.8E4
168.57
2.4E4
4.2E3
F17
1.38
2.68
2.34
1.00
3.63
2.45
3.46
1.45
3.74
2.72
F18
13.06
70.94
11.23
1.00
95.61
19.47
68.56
4.98
79.93
22.45
F19
1.60
1.04
2.03
1.32
2.49
1.00
3.09
1.94
3.16
3.03
F20
6.11
5.31
7.55
1.00
8.88
5.95
7.87
3.64
8.67
6.05
F21
7.65
21.34
9.23
1.00
31.92
15.68
25.97
12.01
26.01
18.86
F22
10.48
6.49
8.22
1.00
9.97
8.37
10.45
1.77
10.97
8.25
F23
615.61
1.6E3
320.20
1.00
3.4E3
1.2E3
3.1E3
97.12
3.5E3
1.2E3
F24
492.15
204.77
242.95
1.00
1.6E3
469.39
2.3E3
58.49
2.6E3
911.76
F25
262.03
594.33
129.14
1.00
1.4E3
296.50
1.3E3
57.32
1.6E3
476.61
F26
2.15
9.6E3
2.54
1.00
2.87
2.20
2.77
1.61
2.52
2.48
F16
9.24
7.28
10.00
1.00
20.83
11.53
26.21
9.10
27.29
17.28
Total
0
0
0
24
1
1
0
0
0
0
DE
DEKH
ES
GA
HS
KH
PBIL
PSO
Table 4 Best normalized function values ABC
ACO
F01
75.56
95.18
70.44
1.00
128.99
103.93
137.49
14.37
136.79
108.85
F02
43.86
55.28
81.07
1.00
140.49
49.28
140.24
36.34
176.68
104.85
F03
48.72
233.26
39.27
1.00
733.71
97.60
504.38
100.76
338.32
367.58
F04
446.05
1.3E3
438.99
1.00
1.6E4
87.45
2.2E4
144.87
2.3E4
2.7E3
F05
3.47
15.24
7.16
1.00
18.08
5.93
15.44
5.19
6.98
12.98
F06
12.52
5.49
12.11
1.00
60.99
13.58
129.45
2.47
148.09
54.87
F07
934.13
1.0E4
1.6E3
1.00
1.9E5
4.0E3
1.7E5
1.2E3
2.5E5
1.4E4
F08
16.39
34.15
36.15
1.00
153.36
15.45
167.48
14.92
209.22
62.20
F09
105.68
132.64
48.46
1.00
36.64
105.86
178.00
158.05
143.20
69.69
F10
716.91
1.05
6.3E3
1.00
5.8E6
5.59
1.0E7
11.00
1.1E7
1.3E5
F11 F12
3.8E4 8.2E16
1.00 4.6E21
8.1E4 4.5E12
1.31 1.00
8.7E6 1.1E16
3.7E3 4.6E21
1.8E7 4.0E15
5.6E3 1.9E8
2.1E7 4.6E21
2.3E6 2.6E16
F13
4.1E14
1.8E17
3.5E6
1.00
4.2E11
1.8E17
6.6E12
1.6E10
1.8E17
1.5E9
F14
19.95
84.67
74.08
1.00
201.05
25.93
257.77
18.68
288.11
56.54
F16
771.76
3.9E3
4.2E3
1.00
1.3E5
1.7E3
1.7E5
956.73
1.4E5
2.3E4
123
Neural Comput & Applic (2014) 25:297–308
303
Table 4 continued ABC
ACO
DE
DEKH
ES
GA
HS
KH
PBIL
PSO 4.08
F17
1.48
3.98
3.58
1.00
4.82
2.76
5.28
1.61
6.15
F18
6.95
41.32
8.17
1.00
72.96
8.00
44.77
3.11
46.99
14.35
F19
3.48
1.70
4.75
2.82
6.12
1.00
7.98
3.95
7.04
7.06 8.37
F20
7.36
3.47
8.14
1.00
14.75
3.91
8.85
3.25
11.63
F21
21.07
63.02
27.79
1.00
70.68
42.49
86.58
33.27
92.47
52.61
F22
17.83
8.77
12.55
1.00
15.57
7.62
17.65
2.11
20.41
13.31
F23
1.3E3
4.6E3
769.86
1.00
9.5E3
1.9E3
1.2E4
182.22
1.4E4
4.5E3
F24
1.9E3
920.00
1.1E3
1.00
1.2E4
1.3E3
1.6E4
195.00
1.3E4
6.0E3
F25
660.01
1.5E3
607.12
1.00
5.3E3
795.14
7.7E3
167.34
7.7E3
1.6E3
F26
3.87
1.48
3.50
1.00
4.32
1.63
3.52
1.77
4.63
2.06
F16
11.78
9.14
14.53
1.00
30.63
11.15
43.95
11.53
37.71
24.70
Total
0
1
0
24
0
1
0
0
0
0
300
40 35
benchmark function value
250
benchmark function value
30 25 20 15 10
ABC ACO DE DEKH ES
5 0
0
5
10
GA HS KH PBIL PSO
15
20
200
150
50
25
30
35
40
45
50
ABC ACO DE DEKH ES
100
0
5
10
140 GA HS KH PBIL PSO
80
60
40
20
0
5
10
15
20
25
30
35
40
45
Number of generations
Fig. 4 Performance comparison for the F08 Levy function
30
35
40
45
50
1200 1000 800 600 400
ABC ACO DE DEKH ES
200
0
25
1400
benchmark function value
benchmark function value
100
20
Fig. 5 Performance comparison for the F16 Rastrigin function
Fig. 3 Performance comparison for the F02 Alpine function
120
15
Number of generations
Number of generations
ABC ACO DE DEKH ES
GA HS KH PBIL PSO
50
0
0
5
10
GA HS KH PBIL PSO
15
20
25
30
35
40
45
50
Number of generations
Fig. 6 Performance comparison for the F26 Wavy1 function
123
304
Neural Comput & Applic (2014) 25:297–308
Table 5 Best normalized function values with initial population size NP = 100 ABC
ACO
DE
DEKH
ES
GA
HS
KH
PBIL
PSO
F01
88.33
109.24
86.04
1.00
147.50
110.69
147.68
14.70
148.08
120.55
F02
4.26
7.24
10.53
1.00
20.18
4.83
18.04
5.57
17.12
11.67
F03
24.94
80.01
18.45
1.00
272.27
37.04
397.37
47.70
132.06
219.58
F04
93.63
546.99
46.19
1.00
5.0E3
58.28
1.5E3
83.16
3.6E3
361.46
F05
3.19
12.22
5.27
1.00
11.79
1.72
10.27
6.34
9.11
9.93
F06
16.50
3.78
17.06
1.00
72.98
9.82
145.94
7.77
160.34
34.58
F07
258.72
697.47
252.52
1.00
7.9E3
36.31
5.5E3
199.76
9.6E3
675.49
F08
7.62
23.15
13.84
1.00
65.21
7.41
43.19
8.91
55.23
14.65
F09
3.65
4.19
1.32
2.61
1.00
2.78
5.00
4.34
3.98
3.22
F10
486.74
2.06
8.6E3
1.00
9.6E6
4.36
1.8E7
1.6E3
1.9E7
9.2E5
F11 F12
4.2E6 1.1E14
1.00 3.2E16
3.9E6 1.2E9
260.70 1.00
1.0E9 8.6E12
1.4E4 3.2E16
1.3E9 4.4E11
8.4E6 4.6E7
1.1E9 3.2E16
3.0E7 6.0E12
F13
1.8E12
1.9E14
2.8E6
1.00
1.3E7
1.9E14
8.9E7
4.4E10
1.9E14
2.5E9
F14
4.07
44.32
14.92
1.00
39.67
1.70
36.98
6.59
49.61
18.32
Total
0
1
0
12
1
0
0
0
0
0
Table 6 Mean normalized function values with initial population size NP = 100 ABC
ACO
DE
DEKH
ES
GA
HS
KH
PBIL
PSO
F01
54.69
65.22
52.11
1.00
82.79
70.34
82.51
13.39
82.69
69.40
F02
2.74
6.19
6.06
1.00
11.05
4.48
10.03
3.63
10.62
7.58
F03
34.17
51.60
10.56
1.00
428.17
27.11
320.45
52.51
79.64
169.13
F04
166.12
443.20
51.75
1.00
2.6E3
78.17
1.8E3
68.20
2.1E3
243.24
F05
1.73
7.16
2.88
1.00
8.54
2.17
6.10
3.41
5.97
6.06
F06
23.44
5.45
17.05
1.00
125.43
23.81
144.32
9.07
148.93
50.33
F07
244.23
783.74
120.62
1.00
5.4E3
167.08
3.6E3
126.13
4.5E3
412.07
F08
11.61
30.13
15.28
1.00
74.43
10.21
60.02
10.22
67.07
26.02
F09
3.44
3.86
1.90
2.95
1.00
2.80
4.47
4.15
3.76
3.25
F10
1.1E5
1.1E7
3.3E4
1.00
7.5E6
3.9E4
8.2E6
1.8E4
9.4E6
9.0E5
F11 F12
1.9E4 5.1E9
1.5E6 9.6E9
1.0E4 1.2E5
1.00 1.00
9.7E5 7.1E7
7.2E3 5.7E9
9.6E5 2.3E7
8.4E3 1.2E10
1.0E6 5.7E9
1.2E5 1.3E8
F13
2.3E10
6.9E10
1.4E6
1.00
5.4E8
5.7E10
2.9E8
7.4E9
5.7E10
3.1E7
F14
4.52
46.29
14.05
1.00
47.80
4.32
38.85
8.37
39.53
16.10
Total
0
0
0
13
1
0
0
0
0
0
NP = 50. For the initial population size NP = 100, 150, and 200, the normalized values are shown in Tables 5, 6, 7, 8, 9, and 10. From Tables 5, 6, 7, 8, 9, and 10, DEKH is always better than other nine methods on different initial population size for most functions. For other approaches, ABC, ACO, ES, and GA are able to get optimal value on one or two functions in initial population size. This indicates that the DEKH method is well capable of taking the use of the population information, and it is insensitive to the initial population size NP. It has been demonstrated that DEKH is a more robust method than other methods.
123
The experiments carried out in Sects. 4.1–4.3 and above analyses about the Figs. 1, 2, 3 and 4 show that, in most cases, the DEKH and KH are the first and second best method on most benchmark problems among ten approaches, respectively. ABC, ACO, DE, and GA are the third best methods that are simply inferior to the DEKH and KH.
5 Conclusions In order to overcome the disadvantage of the KH, a HDE operator has been incorporated into the KH to generate
Neural Comput & Applic (2014) 25:297–308
305
Table 7 Best normalized function values with initial population size NP = 150 ABC
ACO
DE
DEKH
ES
GA
HS
KH
PBIL
PSO
F01
63.32
92.47
64.50
1.00
133.78
99.00
127.66
13.12
128.83
100.10
F02
2.99
4.68
5.67
1.00
12.74
3.80
9.73
4.49
9.92
6.71
F03
5.98
28.76
4.85
1.00
231.23
8.91
160.03
25.94
58.26
62.11
F04
51.64
250.90
46.83
1.00
2.7E3
37.96
1.9E3
65.15
1.9E3
141.22
F05
1.96
5.73
3.90
1.00
7.46
1.94
7.00
4.38
8.51
7.57
F06
6.05
4.36
11.74
1.00
106.39
6.52
90.71
7.44
118.46
32.96
F07
61.52
210.98
50.56
1.00
2.7E3
4.59
1.8E3
104.31
2.3E3
174.51
F08
12.23
15.72
11.48
1.00
56.09
8.71
61.09
11.06
67.29
32.47
F09
3.86
3.96
1.32
2.69
1.00
2.36
5.06
4.80
3.86
2.31
F10
2.5E3
1.49
2.3E4
3.54
1.6E7
1.00
5.9E6
3.0E4
4.9E6
2.1E5
F11 F12
6.4E4 3.3E14
1.28 9.5E15
5.3E4 2.5E9
1.00 1.00
8.8E6 7.4E11
799.21 9.5E15
9.1E6 9.6E11
1.1E5 3.6E8
5.7E6 9.5E15
6.5E5 7.0E11
F13
1.4E10
1.9E14
7.0E6
1.00
3.6E8
1.9E14
1.4E10
1.1E11
1.9E14
4.9E8
F14
2.34
26.76
11.93
1.00
53.88
1.28
28.12
5.97
31.48
9.32
Total
0
0
0
12
1
1
0
0
0
0
Table 8 Mean normalized function values with initial population size NP = 150 ABC
ACO
DE
DEKH
ES
GA
HS
KH
PBIL
PSO
F01
58.97
74.87
61.00
1.00
106.18
89.12
103.64
13.59
104.20
82.46
F02 F03
2.24 20.71
4.08 21.51
4.32 4.70
1.00 1.00
9.02 298.94
3.31 14.93
8.09 149.08
3.58 33.53
8.38 48.49
5.34 80.79
F04
66.65
200.85
29.61
1.00
1.6E3
38.33
1.3E3
58.71
1.5E3
134.77
F05
1.02
4.17
1.79
1.00
4.45
1.57
3.61
2.24
3.97
3.70
F06
16.74
5.00
12.55
1.00
117.68
14.73
126.96
10.53
127.21
39.33
F07
96.01
563.16
42.31
1.00
2.4E3
64.45
1.6E3
101.11
1.8E3
162.67
F08
9.58
25.77
10.35
1.00
48.17
9.93
45.55
9.57
52.49
25.25
F09
3.37
3.75
1.72
3.06
1.00
2.44
4.32
4.17
3.64
2.83
F10
9.9E3
1.7E6
1.6E4
1.07
5.0E6
1.00
3.4E6
1.8E4
2.8E6
2.1E5
F11
4.2E3
2.9E4
1.7E3
1.00
2.6E5
412.97
2.2E5
3.6E3
2.2E5
1.9E4
F12
1.3E11
2.1E11
1.0E6
1.00
7.7E8
2.1E11
2.9E8
1.1E8
2.1E11
1.1E9
F13
1.4E9
1.2E10
7.4E4
1.00
5.5E7
1.2E10
6.3E7
3.0E8
1.2E10
3.6E6
F14
2.24
26.76
9.42
1.00
42.00
2.33
19.73
4.70
23.32
8.55
Total
0
0
0
12
1
1
0
0
0
0
an improved DEKH approach for function optimization. In DEKH, KH is utilized to perform global search that make most krill move toward a more promising area. Thereafter, the HDE operator searches locally to get better solutions. By using two techniques, DEKH can balance global and local search and cope with multimodal benchmarks. From the results of the ten methods on the functions discussed in this paper, the DEKH’s performance is better than, or at least comparable with, the other nine algorithms.
For DEKH, there are various issues that still deserve further study. Firstly, the influence of the number of elitism, crossover factor CR, and weighting factor FW is required to be investigated through a series of experiments. Through this, DEKH can be used to solve problems more efficiently. Secondly, the capability of this hybrid approach to improve the performance of other population-based optimization methods is worthy of further study. Thirdly, the performance of DEKH is validated only by comparing with some basic methods. Thus, for more verification of the
123
306
Neural Comput & Applic (2014) 25:297–308
Table 9 Best normalized function values with initial population size NP = 200 ABC
ACO
DE
DEKH
ES
GA
HS
KH
PBIL
PSO
F01
89.72
101.10
84.41
1.00
148.85
110.05
142.21
6.26
141.45
108.23
F02
2.33
5.18
5.41
1.00
9.09
2.79
9.67
3.62
8.92
6.96
F03
12.26
19.82
4.56
1.00
95.79
10.95
163.39
21.38
41.02
54.36
F04
42.63
211.26
16.32
1.00
1.1E3
8.50
761.96
55.62
801.46
75.63
F05
1.00
7.85
3.67
2.52
8.64
1.03
8.25
4.78
6.24
4.50
F06
9.45
3.68
12.47
1.00
111.24
4.72
100.66
12.30
96.89
38.54
F07
25.73
190.21
30.23
1.00
2.3E3
7.41
929.69
125.97
1.3E3
73.39
F08
5.39
13.06
7.76
1.00
40.64
2.65
37.05
10.68
31.79
13.34
F09
3.33
4.36
1.65
2.79
1.00
2.38
4.96
4.46
4.02
1.95
F10
66.92
1.00
2.7E3
3.31
1.3E7
2.15
1.0E7
1.3E5
7.3E6
9.6E4
F11 F12
1.1E4 5.5E12
1.00 1.3E14
3.3E5 1.6E7
293.84 1.00
4.4E7 4.9E8
63.80 1.3E14
1.8E7 1.3E9
2.1E5 1.3E9
2.2E7 1.3E14
2.7E5 3.0E10
F13
2.5E6
7.0E10
998.38
1.00
2.0E4
7.0E10
6.0E5
3.1E7
7.0E10
3.5E5
F14
3.06
34.64
10.38
1.60
56.49
1.00
33.91
8.48
29.06
8.30
Total
1
2
0
9
1
1
0
0
0
0
HS
KH
PBIL
PSO
Table 10 Mean normalized function values with initial population size NP = 200 ABC
ACO
DE
DEKH
ES
GA
F01
76.55
92.17
77.79
1.00
126.53
99.75
122.72
11.75
122.07
96.90
F02 F03
1.62 11.73
3.92 16.23
3.58 4.31
1.00 1.00
6.53 162.49
1.82 12.10
6.01 143.16
2.58 36.07
6.83 34.82
4.55 63.08 67.84
F04
35.82
252.42
18.25
1.00
1.0E3
20.82
702.28
46.78
713.12
F05
1.00
6.31
2.61
1.57
6.14
1.43
5.76
3.08
4.50
4.40
F06
15.22
3.78
12.16
1.00
112.73
7.30
112.98
12.31
103.72
37.21
F07
32.96
336.19
23.03
1.00
2.0E3
54.19
1.1E3
91.54
1.3E3
116.49
F08
5.59
14.82
7.72
1.00
47.52
5.02
33.80
10.13
39.79
16.14
F09
3.00
3.55
1.68
2.63
1.00
2.23
4.07
3.74
3.48
2.53
F10
1.9E3
2.7E6
4.2E3
1.00
5.5E6
21.41
3.2E6
6.0E4
3.0E6
2.0E5
F11
222.98
6.3E3
217.84
1.00
2.5E4
22.09
1.4E4
340.76
1.5E4
1.1E3
F12
2.3E9
1.9E9
2.9E4
1.00
1.5E6
1.9E9
6.5E6
3.8E8
1.9E9
3.8E6
F13
3.3E8
1.7E9
587.22
1.00
9.2E6
1.7E9
2.8E6
1.3E8
1.7E9
4.2E5
F14
2.23
19.33
6.93
1.00
32.29
1.43
18.84
6.58
17.12
5.52
Total
1
0
0
12
1
0
0
0
0
0
robustness of this hybrid method, it may be compared with other hybrid methods reviewed in Sect. 4.2.
References 1. Le´vano M, Nowak H (2011) New aspects of the elastic net algorithm for cluster analysis. Neural Comput Appl 20(6):835–850. doi:10.1007/s00521-010-0498-x 2. Talatahari S, Kheirollahi M, Farahmandpour C, Gandomi A (2012) A multi-stage particle swarm for optimum design of truss structures. Neural Comput Appl. doi:10.1007/s00521-012-1072-5
123
3. Wang G, Guo L, Duan H, Liu L, Wang H (2012) Path planning for UCAV using bat algorithm with mutation. Sci World J 2012:1–15. doi:10.1100/2012/418946 4. Li X, Yin M (2013) An opposition-based differential evolution algorithm for permutation flow shop scheduling based on diversity measure. Adv Eng Softw 55:10–31. doi:10.1016/j. advengsoft.2012.09.003 5. Zou D, Gao L, Li S, Wu J (2011) Solving 0-1 knapsack problem by a novel global harmony search algorithm. Appl Soft Comput 11(2):1556–1564. doi:10.1016/j.asoc.2010.07.019 6. Zou D, Liu H, Gao L, Li S (2011) An improved differential evolution algorithm for the task assignment problem. Eng Appl Artif Intell 24(4):616–624. doi:10.1016/j.engappai.2010.12.002
Neural Comput & Applic (2014) 25:297–308 7. Yang XS, Gandomi AH, Talatahari S, Alavi AH (2013) Metaheuristics in water. Geotechnical and Transport Engineering, Elsevier 8. Gandomi AH, Yang XS, Talatahari S, Alavi AH (2013) Metaheuristic applications in structures and infrastructures. Elsevier, Waltham 9. Goldberg DE (1998) Genetic algorithms in search. Optimization and machine learning. Addison-Wesley, New York 10. Zhao M, Ren J, Ji L, Fu C, Li J, Zhou M (2012) Parameter selection of support vector machines and genetic algorithm based on change area search. Neural Comput Appl 21(1):1–8. doi:10. 1007/s00521-011-0603-9 11. Loghmanian S, Jamaluddin H, Ahmad R, Yusof R, Khalid M (2012) Structure optimization of neural network for dynamic system modeling using multi-objective genetic algorithm. Neural Comput Appl 21(6):1281–1295. doi:10.1007/s00521-011-0560-3 12. Storn R, Price K (1997) Differential evolution-a simple and efficient heuristic for global optimization over continuous spaces. J Global Optim 11(4):341–359 13. Gandomi AH, Yang X-S, Talatahari S, Deb S (2012) Coupled eagle strategy and differential evolution for unconstrained and constrained global optimization. Comput Math Appl 63(1):191–200. doi:10.1016/j.camwa.2011.11.010 14. Khazraee S, Jahanmiri A, Ghorayshi S (2011) Model reduction and optimization of reactive batch distillation based on the adaptive neuro-fuzzy inference system and differential evolution. Neural Comput Appl 20(2):239–248. doi:10.1007/s00521-010-0364-x 15. Geem ZW, Kim JH, Loganathan GV (2001) A new heuristic optimization algorithm: harmony search. Simulation 76(2):60–68. doi:10.1177/003754970107600201 16. Wang H, Yuan X, Wang Y, Yang Y (2013) Harmony search algorithm-based fuzzy-PID controller for electronic throttle valve. Neural Comput Appl 22(2):329–336. doi:10.1007/s00521011-0678-3 17. Gholizadeh S, Barzegar A (2013) Shape optimization of structures for frequency constraints by sequential harmony search algorithm. Eng Optim 45(6):627–646. doi:10.1080/0305215x. 2012.704028 18. Karaboga D, Basturk B (2007) A powerful and efficient algorithm for numerical function optimization: artificial bee colony (ABC) algorithm. J Global Optim 39(3):459–471. doi:10.1007/s10898007-9149-x 19. Kennedy J, Eberhart R (1995) Particle swarm optimization. In: Paper presented at the proceeding of the IEEE international conference on neural networks, Perth, Australia, 27 Nov–1 Dec 20. Chen D, Zhao C, Zhang H (2011) An improved cooperative particle swarm optimization and its application. Neural Comput Appl 20(2):171–182. doi:10.1007/s00521-010-0503-4 21. Gandomi AH, Yun GJ, Yang XS, Talatahari S (2013) Chaosenhanced accelerated particle swarm algorithm. Commun Nonlinear Sci Numer Simul 18(2):327–340. doi:10.1016/j.cnsns.2012. 07.017 22. Back T (1996) Evolutionary algorithms in theory and practice. Oxford University Press, Oxford 23. Beyer H (2001) The theory of evolution strategies. Springer, New York 24. Dorigo M, Stutzle T (2004) Ant colony optimization. MIT Press, Cambridge 25. Gandomi AH, Yang X-S, Alavi AH (2011) Mixed variable structural optimization using Firefly Algorithm. Comput Struct 89(23–24):2325–2336. doi:10.1016/j.compstruc.2011.08.002 26. Cai X, Fan S, Tan Y (2012) Light responsive curve selection for photosynthesis operator of APOA. Int J Bio-Inspired Comput 4(6):373–379 27. Xie L, Zeng J, Formato RA (2012) Selection strategies for gravitational constant G in artificial physics optimisation based
307
28. 29.
30.
31.
32.
33.
34.
35.
36.
37.
38.
39.
40.
41.
42.
43.
44.
45.
46.
47.
on analysis of convergence properties. Int J Bio-Inspired Comput 4(6):380–391 Simon D (2008) Biogeography-based optimization. IEEE T Evol Comput 12(6):702–713. doi:10.1109/TEVC.2008.919004 Gandomi AH, Alavi AH (2011) Multi-stage genetic programming: a new strategy to nonlinear system modeling. Inf Sci 181(23):5227–5239. doi:10.1016/j.ins.2011.07.026 Yang XS, Deb S (2009) Cuckoo search via Le´vy flights. In: Proceeding of World congress on nature & biologically inspired computing (NaBIC 2009), Coimbatore, India, Dec 2009. IEEE Publications, USA, pp 210–214 Gandomi AH, Talatahari S, Yang XS, Deb S (2012) Design optimization of truss structures using cuckoo search algorithm. Struct Des Tall Spec. doi:10.1002/tal.1033 Gandomi AH, Yang X-S, Alavi AH (2013) Cuckoo search algorithm: a metaheuristic approach to solve structural optimization problems. Eng Comput 29(1):17–35. doi:10.1007/s00366011-0241-y Li X, Zhang J, Yin M (2013) Animal migration optimization: an optimization algorithm inspired by animal migration behavior. Neural Comput Appl 1–11. doi:10.1007/s00521-013-1433-8 Shumeet B (1994) Population-based incremental learning: a method for integrating genetic search based function optimization and competitive learning. Carnegie Mellon University, Pittsburgh Erol OK, Eksin I (2006) A new optimization method: Big BangBig Crunch. Adv Eng Softw 37(2):106–111. doi:10.1016/j. advengsoft.2005.04.005 Kaveh A, Talatahari S (2009) Size optimization of space trusses using Big Bang-Big Crunch algorithm. Comput Struct 87(17–18):1129–1140. doi:10.1016/j.compstruc.2009.04.011 Kaveh A, Talatahari S (2010) Optimal design of Schwedler and ribbed domes via hybrid Big Bang-Big Crunch algorithm. J Constr Steel Res 66(3):412–419. doi:10.1016/j.jcsr.2009.10.013 Kaveh A, Talatahari S (2010) A discrete big bang-big crunch algorithm for optimal design of skeletal structures. Asian J Civil Eng 11(1):103–122 Gandomi AH, Yang X-S, Alavi AH, Talatahari S (2013) Bat algorithm for constrained optimization tasks. Neural Comput Appl 22(6):1239–1255. doi:10.1007/s00521-012-1028-9 Yang XS, Gandomi AH (2012) Bat algorithm: a novel approach for global engineering optimization. Eng Comput 29(5):464–483. doi:10.1108/02644401211235834 Kaveh A, Talatahari S (2010) A novel heuristic optimization method: charged system search. Acta Mech 213(3–4):267–289. doi:10.1007/s00707-009-0270-4 Gandomi AH, Alavi AH (2012) Krill Herd: a new bio-inspired optimization algorithm. Commun Nonlinear Sci Numer Simulat 17(12):4831–4845. doi:10.1016/j.cnsns.2012.05.010 El-Abd M (2011) A hybrid ABC-SPSO algorithm for continuous function optimization. In: Swarm intelligence (SIS), 2011 IEEE symposium on, Paris, 11–15 Apr 2011. IEEE, pp 1–6. doi:10. 1109/SIS.2011.5952576 Wang G, Guo L (2013) A novel hybrid bat algorithm with harmony search for global numerical optimization. J Appl Math 2013:21. doi:10.1155/2013/696491 Duan H, Zhao W, Wang G, Feng X (2012) Test-sheet composition using analytic hierarchy process and hybrid metaheuristic algorithm TS/BBO. Math Probl Eng 2012:1–22. doi:10.1155/ 2012/712752 Gao X, Wang X, Jokinen T, Ovaska S, Arkkio A, Zenger K (2012) A hybrid PBIL-based harmony search method. Neural Comput Appl 21(5):1071–1083. doi:10.1007/s00521-011-0675-6 Geem ZW (2009) Particle-swarm harmony search for water network design. Eng Optim 41(4):297–311. doi:10.1080/ 03052150802449227
123
308 48. Gong W, Cai Z, Ling C (2010) DE/BBO: a hybrid differential evolution with biogeography-based optimization for global numerical optimization. Soft Comput 15(4):645–665. doi:10. 1007/s00500-010-0591-1 49. Kuo RJ, Syu YJ, Chen Z-Y, Tien FC (2012) Integration of particle swarm optimization and genetic algorithm for dynamic clustering. Inf Sci 195:124–140. doi:10.1016/j.ins.2012.01.021 50. Sun Y, Zhang L, Gu X (2012) A hybrid co-evolutionary cultural algorithm based on particle swarm optimization for solving global optimization problems. Neurocomputing 98:76–89. doi:10. 1016/j.neucom.2011.08.043 51. Sheikhan M, Mohammadi N (2012) Neural-based electricity load forecasting using hybrid of GA and ACO for feature selection. Neural Comput Appl 21(8):1961–1970. doi:10.1007/s00521-0110599-1 52. Marichelvam M (2012) An improved hybrid Cuckoo Search (IHCS) metaheuristics algorithm for permutation flow shop scheduling problems. Int J Bio-Inspired Comput 4(4):200–205. doi:10.1504/IJBIC.2012.048061 53. Wang G, Guo L, Wang H, Duan H, Liu L, Li J (2013) Incorporating mutation scheme into krill herd algorithm for global
123
Neural Comput & Applic (2014) 25:297–308
54. 55.
56.
57.
58.
59.
numerical optimization. Neural Comput Appl. doi:10.1007/ s00521-012-1304-8 Yao X, Liu Y, Lin G (1999) Evolutionary programming made faster. IEEE T Evol Comput 3(2):82–102 Yang X-S, Cui Z, Xiao R, Gandomi AH, Karamanoglu M (2013) Swarm intelligence and bio-inspired computation. Elsevier, Waltham Arslan M, C¸unkas¸ M, Sag˘ T (2012) Determination of induction motor parameters with differential evolution algorithm. Neural Comput Appl 21(8):1995–2004. doi:10.1007/s00521-011-0612-8 Li X, Yin M (2012) Application of differential evolution algorithm on self-potential data. PLoS ONE 7(12):e51199. doi:10. 1371/journal.pone.0051199 Jia L, Cheng D, Chiu M-S (2012) Pareto-optimal solutions based multi-objective particle swarm optimization control for batch processes. Neural Comput Appl 21(6):1107–1116. doi:10.1007/ s00521-011-0659-6 Zhang Y, Huang D, Ji M, Xie F (2011) Image segmentation using PSO and PCM with Mahalanobis distance. Expert Syst Appl 38(7):9036–9040. doi:10.1016/j.eswa.2011.01.041
ORIGINAL ARTICLE
Hybrid krill herd algorithm with differential evolution for global numerical optimization Gai-Ge Wang • Amir H. Gandomi Amir H. Alavi • Guo-Sheng Hao
•
Received: 9 December 2012 / Accepted: 10 September 2013 / Published online: 1 October 2013 Ó Springer-Verlag London 2013
Abstract In order to overcome the poor exploitation of the krill herd (KH) algorithm, a hybrid differential evolution KH (DEKH) method has been developed for function optimization. The improvement involves adding a new hybrid differential evolution (HDE) operator into the krill, updating process for the purpose of dealing with optimization problems more efficiently. The introduced HDE operator inspires the intensification and lets the krill perform local search within the defined region. DEKH is validated by 26 functions. From the results, the proposed methods are able to find more accurate solution than the KH and other methods. In addition, the robustness of the DEKH algorithm and the influence of the initial population size on convergence and performance are investigated by a series of experiments. Keywords Global optimization problem Krill herd (KH) Hybrid differential evolution (HDE) operator
G.-G. Wang (&) G.-S. Hao School of Computer Science and Technology, Jiangsu Normal University, Xuzhou, Jiangsu 221116, China e-mail: [email protected] G.-S. Hao e-mail: [email protected] A. H. Gandomi Department of Civil Engineering, The University of Akron, Akron, OH 44325, USA e-mail: [email protected]; [email protected] URL: http://www.gozips.uakron.edu/*ag72/ A. H. Alavi Department of Civil and Environmental Engineering, Michigan State University, Engineering Building, East Lansing, MI 48824, USA e-mail: [email protected]; [email protected] URL: http://www.egr.msu.edu/*alavi/
1 Introduction In the current world, the resources are becoming less and less. Due to the limited resources, all the people are striving for making most profit with least cost and resources. These problems can be modeled into maximum/minimum problem, which are also called optimization problems. And then, various technologies are well capable of solving these optimization problems. Generally speaking, there are various ways to divide these technologies; however, a simple way is to look at the property of the techniques. These techniques can be classified into two categories: traditional classical methods and modern intelligent algorithms. The prior has strict step and always generate the same solution with the same starting value. Contrarily, modern intelligent algorithms, based on some stochastic distribution, cannot generate the same solutions under any conditions. In most cases, both can find the satisfactory solution. Recently, nature-inspired metaheuristic methods have emerged, and they perform better than traditional methods in solving optimization problems. These robust metaheuristic methods have been designed to solve complicated problems, such as cluster [1], truss structure design [2], UCAV path planning [3], permutation flow shop scheduling [4], knapsack problem [5], task assignment problem [6], and engineering optimization [7, 8]. These kinds of metaheuristic methods can take advantage of the useful information from the whole population. Therefore, they have a higher probability of searching for the final satisfactory solution. After the emergence of the genetic algorithms (GAs) [9–11] during the 1960s and 1970s, serials of techniques have been developed in the field of optimization, such as differential evolution (DE) [12–14], harmony search (HS) [15–17], artificial bee colony (ABC) [18], particle swarm optimization (PSO) [19–21], evolutionary strategy (ES) [22, 23], ant colony optimization (ACO) [24], firefly algorithm (FA) [25], artificial
123
298
plant optimization algorithm (APOA) [26], artificial physics optimization [27], biogeography-based optimization (BBO) [28], genetic programming (GP) [29], cuckoo search (CS) [30–32], animal migration optimization (AMO) [33], probability-based incremental learning (PBIL) [34], big bang–big crunch algorithm [35–38], bat algorithm (BA) [39, 40], charged system search (CSS) [41], and the KH method [42]. According to the idealization of the social phenomenon of krill, KH has been developed for optimizing complicated functions [42]. In KH, the objective function can be idealized into the distances from food source and its herd density. The position of the krill involves three components. A major advantage of KH is that it requires few control variables compared with other methods. In some cases, KH may fail to find the best solution on multimodal functions. Herein, an effective hybrid differential evolution krill herd (DEKH) method is proposed so as to accelerate convergence speed. In DEKH, firstly, we use the KH to shrink the search apace and select a good promising candidate solution set. Thereafter, for making the krill proceed to the best solution, a more focused hybrid differential evolution (HDE) operator is introduced into the algorithm. This added operator is utilized to search promising region to get optimal solutions for solving optimization problems. Some solutions would be updated by HDE operator around the optimal krill, and this will accelerate the convergence speed. Proposed method is validated on twenty-six functions. Experimental results indicate that the DEKH is able to find the better solution than the KH, other eight methods.
2 A review of hybrid metaheuristic methods It is well-known that a hybrid method can take merits of the methods that it was originated from. Some most representative hybrid methods are reviewed here. El-Abd [43] investigated the capabilities of two swarm intelligence algorithms, namely ABC and PSO and combined into one method, called ABC–PSO. ABC–PSO is a component-based technique where the PSO is expanded with an ABC to enhance the optimal particles. Wang and Guo [44] enhanced the diversity of the bat swarm by adding pitch adjustment operation in HS to the BA method. Based on this mutation operator, a hybrid HS/BA method is proposed and tested on various benchmarks. Duan et al. [45] proposed a hybrid method by merging the BBO together with the TS (tabu search) to deal with the test-sheet composition problem. HS cannot always find a satisfactory solution. In order to overcome this limitation, Gao et al. [46] developed a new HS– PBIL method through the combination of HS and PBIL. Geem [47] proposed an improved HS incorporating particle swarm concept. Gong et al. [48] introduced a hybrid DE and BBO method to merge the exploration of DE together with the
123
Neural Comput & Applic (2014) 25:297–308
exploitation of BBO. In order to cope with the difficulty of setting the number of clusters in most of the clustering methods, Kuo [49] established a novel clustering method based on PSO and GA (DCPG). DCPG can cluster data by means of testing of the data without a cluster number. In 2012, a hybrid co-evolutionary cultural algorithm based on PSO, namely CECBPSO, is developed by Sun et al. [50]. In [51], two new models are proposed for short-term electricity load forecasting. These models use ACO and hybridization of GA and ACO for feature selection and multi-layer perceptron for load prediction. Marichelvam [52] developed an improved hybrid CS algorithm to deal with the permutation flow shop scheduling problems.
3 The proposed DEKH method KH [42] is a heuristic search technique for solving function optimization, which is the idealization of the krill swarm in the process of hunting for the food and communicating with each other. The position of a krill can be idealized into three actions: 1. 2. 3.
movement influenced by other krill; foraging action; physical diffusion. KH used the Lagrangian model in Eq. (1).
dXi ¼ Ni þ Fi þ Di dt
ð1Þ
where Ni is the motion induced by other krill individuals; Fi is the foraging motion, and Di is the physical diffusion of the ith krill individual. The direction of the first movement, ai, is estimated by the target effect, local effect, and a repulsive effect. For a krill individual, this movement can be formulated as: Ninew ¼ N max ai þ xn Niold
ð2Þ
And Nmax is the maximum induced speed, xn is the inertia weight, Niold is the last motion induced. The second movement is affected by the two factors: the current food location and the previous experience. For the ith krill, it can be given below: Fi ¼ Vf bi þ xf Fiold
ð3Þ
where bi ¼ bfood þ bbest i i
ð4Þ
and Vf is the foraging speed, xf is the inertia weight, Fiold is the last foraging motion. The third movement can be considered to be a random process. It can be expressed according to a diffusion speed and a directional factor. It can be formulated as follows:
Neural Comput & Applic (2014) 25:297–308
Di ¼ Dmax d
299
ð5Þ
where Dmax is the maximum diffusion speed and d is the random directional vector, and its arrays are values in [-1, 1]. Herein, the position of a krill in the Dt is formulated by the Eq. (6): Xi ðt þ DtÞ ¼ Xi ðtÞ þ Dt
dXi dt
ð6Þ
For more details on this method, readers are advised to refer to Gandomi and Alavi [42]. For the third motion of the KH method, how a krill moves toward next position is mainly based on random walks; therefore, the satisfactory solution cannot always be found every time. In order to remove this disadvantage, some kind of genetic reproduction mechanisms have been added to KH approach [42]. For most cases, KH is well capable of making full use of the population knowledge and has proven good performance on some benchmarks. However, sometimes, KH may not proceed to the optimal solutions on some complex problems. Here, in order to further enhance the exploitation of KH, a more focused hybrid mutation operator performing local search, called hybrid differential evolution (HDE) operator, is incorporated into the KH to form a novel differential evolution KH (DEKH) method. This more focused local search technique can make the population not converge prematurely. The greedy selection strategy is used in this operator. In other words, the newly generated krill replace the previous one only when its fitness is better than before. Therefore, in this way, the HDE operator works better than mutation. The HDE operator used in DEKH algorithm can be represented in Algorithm 1.
problem, if the newly generated krill has a smaller fitness, it will be in place of the old one. In other words, it can be considered as krill i in the next generation. In DEKH, standard KH algorithm is utilized to search globally for the purpose of making most krill move toward a more promising area. After exploration stage, HDE operator is utilized to search locally with a small step to get the final best solution. Based on the mainframe of DEKH, the KH emphasizes the diversification at the beginning of the search with a big step to explore the search space extensively and evade trapping into local optima, while later HDE operator focuses on the intensification and lets the individuals move toward the best solution at the later stage of the optimization process. In addition, this method is able to cope with the conflict global and local search effectively. In addition, except HDE operator, elitism strategy is also added to the DEKH. Like other methods, some kind of elitism is introduced with the aim of keeping the optimal krill in the population, which can forbid the optimal krill from being ruined by incorrect position updating operator. This elitism strategy is able to make all the krill always proceed to the population with better fitness than before. By incorporating the above-mentioned HDE operator and focused elitism strategy into the KH algorithm, the DEKH has been developed, as described in Fig. 1. Here, d is dimension of the function. NP is the parent population size. FW is the mutation scaling factor. CR is a crossover constant. Xi(j) is the j-dimension for Xi. Ui, Vi are the offspring. r1, r2, and r3 are random integers in [1, NP].
4 The results
≤
In the present work, various test functions, as shown in Table 1, are employed to evaluate our DEKH method. In order to get fair results, all the experiments were done under the same conditions as shown in [53]. More details about these test functions can be found in recent literatures [28, 54, 55]. 4.1 The robustness of DEKH
In Algorithm 1, the mutation and crossover operators are used to generate new position for krill i. For a minimum
As mentioned ‘‘robust’’ several times, it is required to show the robust feature of the DEKH method here. To do this, firstly select a certain benchmark problem (F24 Sum function). And then, we carry out this 100 times to get its statistic characteristics of the final solutions (mean, standard deviation of each variable in the final solution sets), as shown in Table 2. The relation between the evolutions of the fitness and number of generations is represented in Fig. 2. From Table 2, DEKH is able to get the optimal value for F24 Sum function among ten methods. For the
123
300
Neural Comput & Applic (2014) 25:297–308 Table 1 Benchmark functions No.
Name
F01
Ackley
F02
Alpine
F03
Brown
F04
Dixon & Price
F05
Fletcher-Powell
F06
Griewank
F07
Holzman 2 function
F08
Levy
F09
Pathological function
F10
Penalty #1
F11 F12
Penalty #2 Perm #1
F13
Perm #2
F14
Powel
F15
Quartic with noise
F16
Rastrigin
F17
Rosenbrock
F18
Schwefel 2.26
F19
Schwefel 1.2
F20
Schwefel 2.22
F21
Schwefel 2.21
F22
Sphere
F23
Step
F24
Sum function
F25
Zakharov
F26
Wavy1
worst values, the values of ABC, DE, DEKH, GA, KH, and PSO are slightly different, which are better than others. Similar to best value, DEKH is well capable of getting the minimum on the average. In addition, the Std of DEKH is the minimum. It is demonstrated that the values obtained by DEKH have little difference from different implementations. 4.2 The superiority of DEKH In order to prove the merits of DEKH, it was compared with nine methods, which are ABC [18], ACO [24], DE [12, 56, 57], ES [23], GA [9], HS [15], KH [42], PBIL [34], and PSO [19, 58, 59]. In addition, by combination of different genetic operators, four variants of KHs are developed, as described below [42]: KH KH KH KH Fig. 1 Flowchart of DEKH algorithm
123
I. KH without any genetic operators; II. KH with crossover operator; III. KH with mutation operator; IV. KH with crossover and mutation operators [42].
Through various experiments on benchmark problems, Gandomi and Alavi [42] have claimed that the KH II is the
Neural Comput & Applic (2014) 25:297–308
301
Table 2 The best, worst, mean, and standard of the F24 Sum function ABC
ACO
DE
DEKH
ES
GA
HS
KH
PBIL
PSO
Best
140.04
317.93
112.65
0.18
730.56
152.52
999.80
29.22
1.11E3
303.92
Worst
906.99
1.48E3
906.99
906.99
2.05E3
906.99
2.30E3
906.99
2.78E3
906.99
Mean
422.13
964.83
192.05
1.44
2.05E3
467.83
2.05E3
84.93
2.37E3
735.84
Std
133.74
288.96
41.39
1.00
374.75
172.91
355.51
26.67
364.61
176.15
4000 ABC ACO DE DEKH ES
benchmark function value
3500 3000
GA HS KH PBIL PSO
2500 2000 1500 1000 500 0
0
5
10
15
20
25
30
35
40
45
50
Number of generations
Fig. 2 The evolutions of the objective function for ten methods
best method among four variants of KHs. Herein, in the present work, KH II is considered as one of the comparative methods. In other words, in the next texts, when KH is mentioned, it means KH II. For KH and DEKH, the same parameters will be used: the Vf = 0.02, Dmax = 0.005, Nmax = 0.01, FW = 0.1, and CR = 0.4. For others, we set the same parameters as [53]. Note that, for each function, it has 20 variables. To get representative performances for each algorithm, we did 150 implementations on test functions. The results are recorded in Tables 3 and 4, which illustrate the average and mean minima, respectively. The function minimum is highlighted through bold font. The values in the following tables are normalized according to its own scale, the detailed process of which can be found in [55]. From Tables 3 and 4, it can be seen that, DEKH can find function minimum on twenty-four of the twenty-six benchmarks. GA performs the best on the function F19. Furthermore, ES and ACO overtakes all other methods on F09 for the best values and F24 for the average values, respectively. In addition, to provide more proof to testify the advantage of the DEKH, optimization process of ten
methods on some most typical benchmarks are also provided in this section, as shown in Figs. 3, 4, 5 and 6. The solutions are the true average objective function minimum. Figure 2 clearly indicates DEKH is the best method for solving F02 Alpine function from start to finish. For other methods, KH is simply inferior to the DEKH and it eventually finds the function value close to DEKH; while other methods except KH and DEKH, do not search for the satisfactory global minimum. Here, all the methods start the optimization process form almost same value; however, DEKH outperforms them soon. For this problem, DEKH outperforms all others from start to finish. Other methods do not find the satisfactory solution. Clearly, KH converges to the value that is only inferior to DEKH’s. For others, DE can search for the function value that is almost the same with ABC. Table 3 and Fig. 4 show that ABC performs slightly better than DE for this case. For this case, DEKH outperforms all other methods from start to finish. Other methods fail to find the satisfactory solution under certain conditions. Moreover, for other methods, similarly, ABC can search for the function value that is almost the same with KH. Table 2 and Fig. 5 also shows ABC is slightly superior to KH for this function. KH performs very well and ranks 3 among ten approaches. For this case, DEKH is the best method that can search for the global minimum and outperforms all others. ACO performs very well and ranks 2 among ten approaches. 4.3 The difference of initial population size The above comparisons are based on same number of initial population. If we use more individuals in the initial population in the methods, in other words, the diversity of methods is increased. Here, the influence of initial population size will be showed through various simulations on benchmark F01–F14. In the present work, the initial population size NP = 50, 100, 150, and 200 is selected. Because the initial population size NP = 50 is used in the experiments conducted in Sects. 4.1–4.2, the results can be found in the first 15 rows of the Tables 3 and 4 for
123
302
Neural Comput & Applic (2014) 25:297–308
Table 3 Mean normalized function values ABC
ACO
DE
DEKH
ES
GA
HS
KH
PBIL
PSO
F01
11.92
13.81
10.76
1.00
16.84
15.14
17.19
3.87
17.44
14.50
F02
5.63
9.12
10.47
1.00
19.63
8.91
17.40
5.90
20.49
13.36
F03
68.31
121.42
19.76
1.00
947.04
62.02
516.97
96.27
192.64
453.86
F04
754.82
1.2E3
222.33
1.00
8.9E3
467.10
9.0E3
96.78
1.0E4
1.2E3
F05
2.73
10.83
4.15
1.00
10.43
4.23
10.17
4.06
9.43
8.36
F06
38.24
12.10
21.33
1.00
98.94
40.45
197.31
6.11
218.71
81.30
F07
1.3E3
2.4E3
543.81
1.00
1.7E4
1.5E3
1.7E4
161.62
2.1E4
3.8E3
F08
22.43
37.70
24.73
1.00
119.08
32.84
108.72
13.03
130.50
68.53
F09
2.93
3.12
1.69
2.47
1.00
2.45
3.71
3.54
3.22
2.64
F10
2.6E5
1.2E7
5.7E4
1.00
7.6E6
8.3E4
1.3E7
2.7E3
1.6E7
1.4E6
F11 F12
3.5E5 1.1E11
7.5E6 3.8E10
9.5E4 1.1E6
1.00 1.00
4.5E6 2.6E8
2.8E5 1.3E10
6.7E6 1.7E8
9.7E3 1.6E11
8.7E6 7.4E9
1.3E6 1.3E8
F13
4.6E12
5.6E12
8.4E7
1.00
4.2E10
5.0E12
3.6E10
8.7E12
3.6E12
4.2E10
F14
14.81
90.22
36.61
1.00
128.34
20.83
98.18
12.90
107.42
42.65
F16
1.7E3
1.3E3
579.54
1.00
1.7E4
1.3E3
1.8E4
168.57
2.4E4
4.2E3
F17
1.38
2.68
2.34
1.00
3.63
2.45
3.46
1.45
3.74
2.72
F18
13.06
70.94
11.23
1.00
95.61
19.47
68.56
4.98
79.93
22.45
F19
1.60
1.04
2.03
1.32
2.49
1.00
3.09
1.94
3.16
3.03
F20
6.11
5.31
7.55
1.00
8.88
5.95
7.87
3.64
8.67
6.05
F21
7.65
21.34
9.23
1.00
31.92
15.68
25.97
12.01
26.01
18.86
F22
10.48
6.49
8.22
1.00
9.97
8.37
10.45
1.77
10.97
8.25
F23
615.61
1.6E3
320.20
1.00
3.4E3
1.2E3
3.1E3
97.12
3.5E3
1.2E3
F24
492.15
204.77
242.95
1.00
1.6E3
469.39
2.3E3
58.49
2.6E3
911.76
F25
262.03
594.33
129.14
1.00
1.4E3
296.50
1.3E3
57.32
1.6E3
476.61
F26
2.15
9.6E3
2.54
1.00
2.87
2.20
2.77
1.61
2.52
2.48
F16
9.24
7.28
10.00
1.00
20.83
11.53
26.21
9.10
27.29
17.28
Total
0
0
0
24
1
1
0
0
0
0
DE
DEKH
ES
GA
HS
KH
PBIL
PSO
Table 4 Best normalized function values ABC
ACO
F01
75.56
95.18
70.44
1.00
128.99
103.93
137.49
14.37
136.79
108.85
F02
43.86
55.28
81.07
1.00
140.49
49.28
140.24
36.34
176.68
104.85
F03
48.72
233.26
39.27
1.00
733.71
97.60
504.38
100.76
338.32
367.58
F04
446.05
1.3E3
438.99
1.00
1.6E4
87.45
2.2E4
144.87
2.3E4
2.7E3
F05
3.47
15.24
7.16
1.00
18.08
5.93
15.44
5.19
6.98
12.98
F06
12.52
5.49
12.11
1.00
60.99
13.58
129.45
2.47
148.09
54.87
F07
934.13
1.0E4
1.6E3
1.00
1.9E5
4.0E3
1.7E5
1.2E3
2.5E5
1.4E4
F08
16.39
34.15
36.15
1.00
153.36
15.45
167.48
14.92
209.22
62.20
F09
105.68
132.64
48.46
1.00
36.64
105.86
178.00
158.05
143.20
69.69
F10
716.91
1.05
6.3E3
1.00
5.8E6
5.59
1.0E7
11.00
1.1E7
1.3E5
F11 F12
3.8E4 8.2E16
1.00 4.6E21
8.1E4 4.5E12
1.31 1.00
8.7E6 1.1E16
3.7E3 4.6E21
1.8E7 4.0E15
5.6E3 1.9E8
2.1E7 4.6E21
2.3E6 2.6E16
F13
4.1E14
1.8E17
3.5E6
1.00
4.2E11
1.8E17
6.6E12
1.6E10
1.8E17
1.5E9
F14
19.95
84.67
74.08
1.00
201.05
25.93
257.77
18.68
288.11
56.54
F16
771.76
3.9E3
4.2E3
1.00
1.3E5
1.7E3
1.7E5
956.73
1.4E5
2.3E4
123
Neural Comput & Applic (2014) 25:297–308
303
Table 4 continued ABC
ACO
DE
DEKH
ES
GA
HS
KH
PBIL
PSO 4.08
F17
1.48
3.98
3.58
1.00
4.82
2.76
5.28
1.61
6.15
F18
6.95
41.32
8.17
1.00
72.96
8.00
44.77
3.11
46.99
14.35
F19
3.48
1.70
4.75
2.82
6.12
1.00
7.98
3.95
7.04
7.06 8.37
F20
7.36
3.47
8.14
1.00
14.75
3.91
8.85
3.25
11.63
F21
21.07
63.02
27.79
1.00
70.68
42.49
86.58
33.27
92.47
52.61
F22
17.83
8.77
12.55
1.00
15.57
7.62
17.65
2.11
20.41
13.31
F23
1.3E3
4.6E3
769.86
1.00
9.5E3
1.9E3
1.2E4
182.22
1.4E4
4.5E3
F24
1.9E3
920.00
1.1E3
1.00
1.2E4
1.3E3
1.6E4
195.00
1.3E4
6.0E3
F25
660.01
1.5E3
607.12
1.00
5.3E3
795.14
7.7E3
167.34
7.7E3
1.6E3
F26
3.87
1.48
3.50
1.00
4.32
1.63
3.52
1.77
4.63
2.06
F16
11.78
9.14
14.53
1.00
30.63
11.15
43.95
11.53
37.71
24.70
Total
0
1
0
24
0
1
0
0
0
0
300
40 35
benchmark function value
250
benchmark function value
30 25 20 15 10
ABC ACO DE DEKH ES
5 0
0
5
10
GA HS KH PBIL PSO
15
20
200
150
50
25
30
35
40
45
50
ABC ACO DE DEKH ES
100
0
5
10
140 GA HS KH PBIL PSO
80
60
40
20
0
5
10
15
20
25
30
35
40
45
Number of generations
Fig. 4 Performance comparison for the F08 Levy function
30
35
40
45
50
1200 1000 800 600 400
ABC ACO DE DEKH ES
200
0
25
1400
benchmark function value
benchmark function value
100
20
Fig. 5 Performance comparison for the F16 Rastrigin function
Fig. 3 Performance comparison for the F02 Alpine function
120
15
Number of generations
Number of generations
ABC ACO DE DEKH ES
GA HS KH PBIL PSO
50
0
0
5
10
GA HS KH PBIL PSO
15
20
25
30
35
40
45
50
Number of generations
Fig. 6 Performance comparison for the F26 Wavy1 function
123
304
Neural Comput & Applic (2014) 25:297–308
Table 5 Best normalized function values with initial population size NP = 100 ABC
ACO
DE
DEKH
ES
GA
HS
KH
PBIL
PSO
F01
88.33
109.24
86.04
1.00
147.50
110.69
147.68
14.70
148.08
120.55
F02
4.26
7.24
10.53
1.00
20.18
4.83
18.04
5.57
17.12
11.67
F03
24.94
80.01
18.45
1.00
272.27
37.04
397.37
47.70
132.06
219.58
F04
93.63
546.99
46.19
1.00
5.0E3
58.28
1.5E3
83.16
3.6E3
361.46
F05
3.19
12.22
5.27
1.00
11.79
1.72
10.27
6.34
9.11
9.93
F06
16.50
3.78
17.06
1.00
72.98
9.82
145.94
7.77
160.34
34.58
F07
258.72
697.47
252.52
1.00
7.9E3
36.31
5.5E3
199.76
9.6E3
675.49
F08
7.62
23.15
13.84
1.00
65.21
7.41
43.19
8.91
55.23
14.65
F09
3.65
4.19
1.32
2.61
1.00
2.78
5.00
4.34
3.98
3.22
F10
486.74
2.06
8.6E3
1.00
9.6E6
4.36
1.8E7
1.6E3
1.9E7
9.2E5
F11 F12
4.2E6 1.1E14
1.00 3.2E16
3.9E6 1.2E9
260.70 1.00
1.0E9 8.6E12
1.4E4 3.2E16
1.3E9 4.4E11
8.4E6 4.6E7
1.1E9 3.2E16
3.0E7 6.0E12
F13
1.8E12
1.9E14
2.8E6
1.00
1.3E7
1.9E14
8.9E7
4.4E10
1.9E14
2.5E9
F14
4.07
44.32
14.92
1.00
39.67
1.70
36.98
6.59
49.61
18.32
Total
0
1
0
12
1
0
0
0
0
0
Table 6 Mean normalized function values with initial population size NP = 100 ABC
ACO
DE
DEKH
ES
GA
HS
KH
PBIL
PSO
F01
54.69
65.22
52.11
1.00
82.79
70.34
82.51
13.39
82.69
69.40
F02
2.74
6.19
6.06
1.00
11.05
4.48
10.03
3.63
10.62
7.58
F03
34.17
51.60
10.56
1.00
428.17
27.11
320.45
52.51
79.64
169.13
F04
166.12
443.20
51.75
1.00
2.6E3
78.17
1.8E3
68.20
2.1E3
243.24
F05
1.73
7.16
2.88
1.00
8.54
2.17
6.10
3.41
5.97
6.06
F06
23.44
5.45
17.05
1.00
125.43
23.81
144.32
9.07
148.93
50.33
F07
244.23
783.74
120.62
1.00
5.4E3
167.08
3.6E3
126.13
4.5E3
412.07
F08
11.61
30.13
15.28
1.00
74.43
10.21
60.02
10.22
67.07
26.02
F09
3.44
3.86
1.90
2.95
1.00
2.80
4.47
4.15
3.76
3.25
F10
1.1E5
1.1E7
3.3E4
1.00
7.5E6
3.9E4
8.2E6
1.8E4
9.4E6
9.0E5
F11 F12
1.9E4 5.1E9
1.5E6 9.6E9
1.0E4 1.2E5
1.00 1.00
9.7E5 7.1E7
7.2E3 5.7E9
9.6E5 2.3E7
8.4E3 1.2E10
1.0E6 5.7E9
1.2E5 1.3E8
F13
2.3E10
6.9E10
1.4E6
1.00
5.4E8
5.7E10
2.9E8
7.4E9
5.7E10
3.1E7
F14
4.52
46.29
14.05
1.00
47.80
4.32
38.85
8.37
39.53
16.10
Total
0
0
0
13
1
0
0
0
0
0
NP = 50. For the initial population size NP = 100, 150, and 200, the normalized values are shown in Tables 5, 6, 7, 8, 9, and 10. From Tables 5, 6, 7, 8, 9, and 10, DEKH is always better than other nine methods on different initial population size for most functions. For other approaches, ABC, ACO, ES, and GA are able to get optimal value on one or two functions in initial population size. This indicates that the DEKH method is well capable of taking the use of the population information, and it is insensitive to the initial population size NP. It has been demonstrated that DEKH is a more robust method than other methods.
123
The experiments carried out in Sects. 4.1–4.3 and above analyses about the Figs. 1, 2, 3 and 4 show that, in most cases, the DEKH and KH are the first and second best method on most benchmark problems among ten approaches, respectively. ABC, ACO, DE, and GA are the third best methods that are simply inferior to the DEKH and KH.
5 Conclusions In order to overcome the disadvantage of the KH, a HDE operator has been incorporated into the KH to generate
Neural Comput & Applic (2014) 25:297–308
305
Table 7 Best normalized function values with initial population size NP = 150 ABC
ACO
DE
DEKH
ES
GA
HS
KH
PBIL
PSO
F01
63.32
92.47
64.50
1.00
133.78
99.00
127.66
13.12
128.83
100.10
F02
2.99
4.68
5.67
1.00
12.74
3.80
9.73
4.49
9.92
6.71
F03
5.98
28.76
4.85
1.00
231.23
8.91
160.03
25.94
58.26
62.11
F04
51.64
250.90
46.83
1.00
2.7E3
37.96
1.9E3
65.15
1.9E3
141.22
F05
1.96
5.73
3.90
1.00
7.46
1.94
7.00
4.38
8.51
7.57
F06
6.05
4.36
11.74
1.00
106.39
6.52
90.71
7.44
118.46
32.96
F07
61.52
210.98
50.56
1.00
2.7E3
4.59
1.8E3
104.31
2.3E3
174.51
F08
12.23
15.72
11.48
1.00
56.09
8.71
61.09
11.06
67.29
32.47
F09
3.86
3.96
1.32
2.69
1.00
2.36
5.06
4.80
3.86
2.31
F10
2.5E3
1.49
2.3E4
3.54
1.6E7
1.00
5.9E6
3.0E4
4.9E6
2.1E5
F11 F12
6.4E4 3.3E14
1.28 9.5E15
5.3E4 2.5E9
1.00 1.00
8.8E6 7.4E11
799.21 9.5E15
9.1E6 9.6E11
1.1E5 3.6E8
5.7E6 9.5E15
6.5E5 7.0E11
F13
1.4E10
1.9E14
7.0E6
1.00
3.6E8
1.9E14
1.4E10
1.1E11
1.9E14
4.9E8
F14
2.34
26.76
11.93
1.00
53.88
1.28
28.12
5.97
31.48
9.32
Total
0
0
0
12
1
1
0
0
0
0
Table 8 Mean normalized function values with initial population size NP = 150 ABC
ACO
DE
DEKH
ES
GA
HS
KH
PBIL
PSO
F01
58.97
74.87
61.00
1.00
106.18
89.12
103.64
13.59
104.20
82.46
F02 F03
2.24 20.71
4.08 21.51
4.32 4.70
1.00 1.00
9.02 298.94
3.31 14.93
8.09 149.08
3.58 33.53
8.38 48.49
5.34 80.79
F04
66.65
200.85
29.61
1.00
1.6E3
38.33
1.3E3
58.71
1.5E3
134.77
F05
1.02
4.17
1.79
1.00
4.45
1.57
3.61
2.24
3.97
3.70
F06
16.74
5.00
12.55
1.00
117.68
14.73
126.96
10.53
127.21
39.33
F07
96.01
563.16
42.31
1.00
2.4E3
64.45
1.6E3
101.11
1.8E3
162.67
F08
9.58
25.77
10.35
1.00
48.17
9.93
45.55
9.57
52.49
25.25
F09
3.37
3.75
1.72
3.06
1.00
2.44
4.32
4.17
3.64
2.83
F10
9.9E3
1.7E6
1.6E4
1.07
5.0E6
1.00
3.4E6
1.8E4
2.8E6
2.1E5
F11
4.2E3
2.9E4
1.7E3
1.00
2.6E5
412.97
2.2E5
3.6E3
2.2E5
1.9E4
F12
1.3E11
2.1E11
1.0E6
1.00
7.7E8
2.1E11
2.9E8
1.1E8
2.1E11
1.1E9
F13
1.4E9
1.2E10
7.4E4
1.00
5.5E7
1.2E10
6.3E7
3.0E8
1.2E10
3.6E6
F14
2.24
26.76
9.42
1.00
42.00
2.33
19.73
4.70
23.32
8.55
Total
0
0
0
12
1
1
0
0
0
0
an improved DEKH approach for function optimization. In DEKH, KH is utilized to perform global search that make most krill move toward a more promising area. Thereafter, the HDE operator searches locally to get better solutions. By using two techniques, DEKH can balance global and local search and cope with multimodal benchmarks. From the results of the ten methods on the functions discussed in this paper, the DEKH’s performance is better than, or at least comparable with, the other nine algorithms.
For DEKH, there are various issues that still deserve further study. Firstly, the influence of the number of elitism, crossover factor CR, and weighting factor FW is required to be investigated through a series of experiments. Through this, DEKH can be used to solve problems more efficiently. Secondly, the capability of this hybrid approach to improve the performance of other population-based optimization methods is worthy of further study. Thirdly, the performance of DEKH is validated only by comparing with some basic methods. Thus, for more verification of the
123
306
Neural Comput & Applic (2014) 25:297–308
Table 9 Best normalized function values with initial population size NP = 200 ABC
ACO
DE
DEKH
ES
GA
HS
KH
PBIL
PSO
F01
89.72
101.10
84.41
1.00
148.85
110.05
142.21
6.26
141.45
108.23
F02
2.33
5.18
5.41
1.00
9.09
2.79
9.67
3.62
8.92
6.96
F03
12.26
19.82
4.56
1.00
95.79
10.95
163.39
21.38
41.02
54.36
F04
42.63
211.26
16.32
1.00
1.1E3
8.50
761.96
55.62
801.46
75.63
F05
1.00
7.85
3.67
2.52
8.64
1.03
8.25
4.78
6.24
4.50
F06
9.45
3.68
12.47
1.00
111.24
4.72
100.66
12.30
96.89
38.54
F07
25.73
190.21
30.23
1.00
2.3E3
7.41
929.69
125.97
1.3E3
73.39
F08
5.39
13.06
7.76
1.00
40.64
2.65
37.05
10.68
31.79
13.34
F09
3.33
4.36
1.65
2.79
1.00
2.38
4.96
4.46
4.02
1.95
F10
66.92
1.00
2.7E3
3.31
1.3E7
2.15
1.0E7
1.3E5
7.3E6
9.6E4
F11 F12
1.1E4 5.5E12
1.00 1.3E14
3.3E5 1.6E7
293.84 1.00
4.4E7 4.9E8
63.80 1.3E14
1.8E7 1.3E9
2.1E5 1.3E9
2.2E7 1.3E14
2.7E5 3.0E10
F13
2.5E6
7.0E10
998.38
1.00
2.0E4
7.0E10
6.0E5
3.1E7
7.0E10
3.5E5
F14
3.06
34.64
10.38
1.60
56.49
1.00
33.91
8.48
29.06
8.30
Total
1
2
0
9
1
1
0
0
0
0
HS
KH
PBIL
PSO
Table 10 Mean normalized function values with initial population size NP = 200 ABC
ACO
DE
DEKH
ES
GA
F01
76.55
92.17
77.79
1.00
126.53
99.75
122.72
11.75
122.07
96.90
F02 F03
1.62 11.73
3.92 16.23
3.58 4.31
1.00 1.00
6.53 162.49
1.82 12.10
6.01 143.16
2.58 36.07
6.83 34.82
4.55 63.08 67.84
F04
35.82
252.42
18.25
1.00
1.0E3
20.82
702.28
46.78
713.12
F05
1.00
6.31
2.61
1.57
6.14
1.43
5.76
3.08
4.50
4.40
F06
15.22
3.78
12.16
1.00
112.73
7.30
112.98
12.31
103.72
37.21
F07
32.96
336.19
23.03
1.00
2.0E3
54.19
1.1E3
91.54
1.3E3
116.49
F08
5.59
14.82
7.72
1.00
47.52
5.02
33.80
10.13
39.79
16.14
F09
3.00
3.55
1.68
2.63
1.00
2.23
4.07
3.74
3.48
2.53
F10
1.9E3
2.7E6
4.2E3
1.00
5.5E6
21.41
3.2E6
6.0E4
3.0E6
2.0E5
F11
222.98
6.3E3
217.84
1.00
2.5E4
22.09
1.4E4
340.76
1.5E4
1.1E3
F12
2.3E9
1.9E9
2.9E4
1.00
1.5E6
1.9E9
6.5E6
3.8E8
1.9E9
3.8E6
F13
3.3E8
1.7E9
587.22
1.00
9.2E6
1.7E9
2.8E6
1.3E8
1.7E9
4.2E5
F14
2.23
19.33
6.93
1.00
32.29
1.43
18.84
6.58
17.12
5.52
Total
1
0
0
12
1
0
0
0
0
0
robustness of this hybrid method, it may be compared with other hybrid methods reviewed in Sect. 4.2.
References 1. Le´vano M, Nowak H (2011) New aspects of the elastic net algorithm for cluster analysis. Neural Comput Appl 20(6):835–850. doi:10.1007/s00521-010-0498-x 2. Talatahari S, Kheirollahi M, Farahmandpour C, Gandomi A (2012) A multi-stage particle swarm for optimum design of truss structures. Neural Comput Appl. doi:10.1007/s00521-012-1072-5
123
3. Wang G, Guo L, Duan H, Liu L, Wang H (2012) Path planning for UCAV using bat algorithm with mutation. Sci World J 2012:1–15. doi:10.1100/2012/418946 4. Li X, Yin M (2013) An opposition-based differential evolution algorithm for permutation flow shop scheduling based on diversity measure. Adv Eng Softw 55:10–31. doi:10.1016/j. advengsoft.2012.09.003 5. Zou D, Gao L, Li S, Wu J (2011) Solving 0-1 knapsack problem by a novel global harmony search algorithm. Appl Soft Comput 11(2):1556–1564. doi:10.1016/j.asoc.2010.07.019 6. Zou D, Liu H, Gao L, Li S (2011) An improved differential evolution algorithm for the task assignment problem. Eng Appl Artif Intell 24(4):616–624. doi:10.1016/j.engappai.2010.12.002
Neural Comput & Applic (2014) 25:297–308 7. Yang XS, Gandomi AH, Talatahari S, Alavi AH (2013) Metaheuristics in water. Geotechnical and Transport Engineering, Elsevier 8. Gandomi AH, Yang XS, Talatahari S, Alavi AH (2013) Metaheuristic applications in structures and infrastructures. Elsevier, Waltham 9. Goldberg DE (1998) Genetic algorithms in search. Optimization and machine learning. Addison-Wesley, New York 10. Zhao M, Ren J, Ji L, Fu C, Li J, Zhou M (2012) Parameter selection of support vector machines and genetic algorithm based on change area search. Neural Comput Appl 21(1):1–8. doi:10. 1007/s00521-011-0603-9 11. Loghmanian S, Jamaluddin H, Ahmad R, Yusof R, Khalid M (2012) Structure optimization of neural network for dynamic system modeling using multi-objective genetic algorithm. Neural Comput Appl 21(6):1281–1295. doi:10.1007/s00521-011-0560-3 12. Storn R, Price K (1997) Differential evolution-a simple and efficient heuristic for global optimization over continuous spaces. J Global Optim 11(4):341–359 13. Gandomi AH, Yang X-S, Talatahari S, Deb S (2012) Coupled eagle strategy and differential evolution for unconstrained and constrained global optimization. Comput Math Appl 63(1):191–200. doi:10.1016/j.camwa.2011.11.010 14. Khazraee S, Jahanmiri A, Ghorayshi S (2011) Model reduction and optimization of reactive batch distillation based on the adaptive neuro-fuzzy inference system and differential evolution. Neural Comput Appl 20(2):239–248. doi:10.1007/s00521-010-0364-x 15. Geem ZW, Kim JH, Loganathan GV (2001) A new heuristic optimization algorithm: harmony search. Simulation 76(2):60–68. doi:10.1177/003754970107600201 16. Wang H, Yuan X, Wang Y, Yang Y (2013) Harmony search algorithm-based fuzzy-PID controller for electronic throttle valve. Neural Comput Appl 22(2):329–336. doi:10.1007/s00521011-0678-3 17. Gholizadeh S, Barzegar A (2013) Shape optimization of structures for frequency constraints by sequential harmony search algorithm. Eng Optim 45(6):627–646. doi:10.1080/0305215x. 2012.704028 18. Karaboga D, Basturk B (2007) A powerful and efficient algorithm for numerical function optimization: artificial bee colony (ABC) algorithm. J Global Optim 39(3):459–471. doi:10.1007/s10898007-9149-x 19. Kennedy J, Eberhart R (1995) Particle swarm optimization. In: Paper presented at the proceeding of the IEEE international conference on neural networks, Perth, Australia, 27 Nov–1 Dec 20. Chen D, Zhao C, Zhang H (2011) An improved cooperative particle swarm optimization and its application. Neural Comput Appl 20(2):171–182. doi:10.1007/s00521-010-0503-4 21. Gandomi AH, Yun GJ, Yang XS, Talatahari S (2013) Chaosenhanced accelerated particle swarm algorithm. Commun Nonlinear Sci Numer Simul 18(2):327–340. doi:10.1016/j.cnsns.2012. 07.017 22. Back T (1996) Evolutionary algorithms in theory and practice. Oxford University Press, Oxford 23. Beyer H (2001) The theory of evolution strategies. Springer, New York 24. Dorigo M, Stutzle T (2004) Ant colony optimization. MIT Press, Cambridge 25. Gandomi AH, Yang X-S, Alavi AH (2011) Mixed variable structural optimization using Firefly Algorithm. Comput Struct 89(23–24):2325–2336. doi:10.1016/j.compstruc.2011.08.002 26. Cai X, Fan S, Tan Y (2012) Light responsive curve selection for photosynthesis operator of APOA. Int J Bio-Inspired Comput 4(6):373–379 27. Xie L, Zeng J, Formato RA (2012) Selection strategies for gravitational constant G in artificial physics optimisation based
307
28. 29.
30.
31.
32.
33.
34.
35.
36.
37.
38.
39.
40.
41.
42.
43.
44.
45.
46.
47.
on analysis of convergence properties. Int J Bio-Inspired Comput 4(6):380–391 Simon D (2008) Biogeography-based optimization. IEEE T Evol Comput 12(6):702–713. doi:10.1109/TEVC.2008.919004 Gandomi AH, Alavi AH (2011) Multi-stage genetic programming: a new strategy to nonlinear system modeling. Inf Sci 181(23):5227–5239. doi:10.1016/j.ins.2011.07.026 Yang XS, Deb S (2009) Cuckoo search via Le´vy flights. In: Proceeding of World congress on nature & biologically inspired computing (NaBIC 2009), Coimbatore, India, Dec 2009. IEEE Publications, USA, pp 210–214 Gandomi AH, Talatahari S, Yang XS, Deb S (2012) Design optimization of truss structures using cuckoo search algorithm. Struct Des Tall Spec. doi:10.1002/tal.1033 Gandomi AH, Yang X-S, Alavi AH (2013) Cuckoo search algorithm: a metaheuristic approach to solve structural optimization problems. Eng Comput 29(1):17–35. doi:10.1007/s00366011-0241-y Li X, Zhang J, Yin M (2013) Animal migration optimization: an optimization algorithm inspired by animal migration behavior. Neural Comput Appl 1–11. doi:10.1007/s00521-013-1433-8 Shumeet B (1994) Population-based incremental learning: a method for integrating genetic search based function optimization and competitive learning. Carnegie Mellon University, Pittsburgh Erol OK, Eksin I (2006) A new optimization method: Big BangBig Crunch. Adv Eng Softw 37(2):106–111. doi:10.1016/j. advengsoft.2005.04.005 Kaveh A, Talatahari S (2009) Size optimization of space trusses using Big Bang-Big Crunch algorithm. Comput Struct 87(17–18):1129–1140. doi:10.1016/j.compstruc.2009.04.011 Kaveh A, Talatahari S (2010) Optimal design of Schwedler and ribbed domes via hybrid Big Bang-Big Crunch algorithm. J Constr Steel Res 66(3):412–419. doi:10.1016/j.jcsr.2009.10.013 Kaveh A, Talatahari S (2010) A discrete big bang-big crunch algorithm for optimal design of skeletal structures. Asian J Civil Eng 11(1):103–122 Gandomi AH, Yang X-S, Alavi AH, Talatahari S (2013) Bat algorithm for constrained optimization tasks. Neural Comput Appl 22(6):1239–1255. doi:10.1007/s00521-012-1028-9 Yang XS, Gandomi AH (2012) Bat algorithm: a novel approach for global engineering optimization. Eng Comput 29(5):464–483. doi:10.1108/02644401211235834 Kaveh A, Talatahari S (2010) A novel heuristic optimization method: charged system search. Acta Mech 213(3–4):267–289. doi:10.1007/s00707-009-0270-4 Gandomi AH, Alavi AH (2012) Krill Herd: a new bio-inspired optimization algorithm. Commun Nonlinear Sci Numer Simulat 17(12):4831–4845. doi:10.1016/j.cnsns.2012.05.010 El-Abd M (2011) A hybrid ABC-SPSO algorithm for continuous function optimization. In: Swarm intelligence (SIS), 2011 IEEE symposium on, Paris, 11–15 Apr 2011. IEEE, pp 1–6. doi:10. 1109/SIS.2011.5952576 Wang G, Guo L (2013) A novel hybrid bat algorithm with harmony search for global numerical optimization. J Appl Math 2013:21. doi:10.1155/2013/696491 Duan H, Zhao W, Wang G, Feng X (2012) Test-sheet composition using analytic hierarchy process and hybrid metaheuristic algorithm TS/BBO. Math Probl Eng 2012:1–22. doi:10.1155/ 2012/712752 Gao X, Wang X, Jokinen T, Ovaska S, Arkkio A, Zenger K (2012) A hybrid PBIL-based harmony search method. Neural Comput Appl 21(5):1071–1083. doi:10.1007/s00521-011-0675-6 Geem ZW (2009) Particle-swarm harmony search for water network design. Eng Optim 41(4):297–311. doi:10.1080/ 03052150802449227
123
308 48. Gong W, Cai Z, Ling C (2010) DE/BBO: a hybrid differential evolution with biogeography-based optimization for global numerical optimization. Soft Comput 15(4):645–665. doi:10. 1007/s00500-010-0591-1 49. Kuo RJ, Syu YJ, Chen Z-Y, Tien FC (2012) Integration of particle swarm optimization and genetic algorithm for dynamic clustering. Inf Sci 195:124–140. doi:10.1016/j.ins.2012.01.021 50. Sun Y, Zhang L, Gu X (2012) A hybrid co-evolutionary cultural algorithm based on particle swarm optimization for solving global optimization problems. Neurocomputing 98:76–89. doi:10. 1016/j.neucom.2011.08.043 51. Sheikhan M, Mohammadi N (2012) Neural-based electricity load forecasting using hybrid of GA and ACO for feature selection. Neural Comput Appl 21(8):1961–1970. doi:10.1007/s00521-0110599-1 52. Marichelvam M (2012) An improved hybrid Cuckoo Search (IHCS) metaheuristics algorithm for permutation flow shop scheduling problems. Int J Bio-Inspired Comput 4(4):200–205. doi:10.1504/IJBIC.2012.048061 53. Wang G, Guo L, Wang H, Duan H, Liu L, Li J (2013) Incorporating mutation scheme into krill herd algorithm for global
123
Neural Comput & Applic (2014) 25:297–308
54. 55.
56.
57.
58.
59.
numerical optimization. Neural Comput Appl. doi:10.1007/ s00521-012-1304-8 Yao X, Liu Y, Lin G (1999) Evolutionary programming made faster. IEEE T Evol Comput 3(2):82–102 Yang X-S, Cui Z, Xiao R, Gandomi AH, Karamanoglu M (2013) Swarm intelligence and bio-inspired computation. Elsevier, Waltham Arslan M, C¸unkas¸ M, Sag˘ T (2012) Determination of induction motor parameters with differential evolution algorithm. Neural Comput Appl 21(8):1995–2004. doi:10.1007/s00521-011-0612-8 Li X, Yin M (2012) Application of differential evolution algorithm on self-potential data. PLoS ONE 7(12):e51199. doi:10. 1371/journal.pone.0051199 Jia L, Cheng D, Chiu M-S (2012) Pareto-optimal solutions based multi-objective particle swarm optimization control for batch processes. Neural Comput Appl 21(6):1107–1116. doi:10.1007/ s00521-011-0659-6 Zhang Y, Huang D, Ji M, Xie F (2011) Image segmentation using PSO and PCM with Mahalanobis distance. Expert Syst Appl 38(7):9036–9040. doi:10.1016/j.eswa.2011.01.041
