LNCS 7331 - Novel Binary Biogeography-Based ... - Springer Link
Novel Binary Biogeography-Based Optimization Algorithm for the Knapsack Problem BiBingyan Zhao1, Changshou Deng2,*, Yanling Yang2, and Hu Peng2 1
School of Business, Jiujiang University, Jiujiang 332005, China School of Information Science and Technology, Jiujiang University, Jiujiang 332005, China [email protected],{csdeng,yangyanlin,xx_penghu}@jju.edu.cn 2
Abstract. Mathematical models of biogeography inspired the development of the biogeography-based optimization algorithm. In this article we propose a binary version of biogeography-based optimization (BBO) for the Knapsack Problem. Two new mutation operators are proposed to extend the biogeography-based optimization algorithm to binary optimization problems. We also demonstrate the performance of the resulting new binary Biogeography-based optimization algorithm in solving four Knapsack problems and compare it with that of the standard Genetic Algorithm. The simulation results show that our new method is effective and efficient for the Knapsack problem. Keywords: Knapsack Problem, Biogeography-based optimization, Migration operator, mutation operator.
1
Introduction
Knapsack problems have been widely studied not only because of their immediate application in industry and financial management, but also more pronounced for theoretical reasons. Many industrial problems such as resource allocation, investment decision-making, and hold loading can be modeled as Knapsack problems. The Knapsack problem can be described as follows. Given a set of items, each with a value vi and a weight wi , determine how often each item should be included in a collection so that the total weight is less than or equal to a given limit ( W ) and the total value is as large as possible. The most common formulation of the problem is the 0-1 Knapsack problem, where each item can either be included one time or not at all, which can be mathematically formulated as: n
Maximize
f =
∑v x
i i
i =1
n
s.t.
∑w x ≤ W, i i
xi ∈ {0,1}
i =1
*
Corresponding author. [email protected]
Y. Tan, Y. Shi, and Z. Ji (Eds.): ICSI 2012, Part I, LNCS 7331, pp. 217–224, 2012. © Springer-Verlag Berlin Heidelberg 2012
(1)
218
BiB. Zhao et al.
Recently, many exact algorithms and nature-inspired algorithms have been proposed to cope with the Knapsack problem [1-8]. However, several methods of them often can only provide locally optimal solutions. The biogeography-based optimization algorithm is a new nature-inspired computation technique proposed by Simon in 2008 [9]. It is originally designed for unconstrained integer programming problems. In this paper, we propose a novel binary BBO for Knapsack problems. To the best of our knowledge, this is the first application of BBO to the Knapsack problem. The rest of the paper is organized as follows. Section 2 gives an overview of BBO. A novel Binary BBO with two mutation operators for the Knapsack problems is presented in section 3. Four Knapsack Problems with different sizes are used to compare the performance of the Binary BBO with that of the Genetic Algorithm in section 4. Section 5 concludes this paper.
2
Biogeography-Based Optimization Algorithm
2.1
Mathematical Model of BBO
Biogeography studies the migration, speciation, and extinction of species from one island to another. With the mathematical models in the Biogeography, Simon (2008) proposed the BBO algorithm [9], which is an example of how a natural process can be modeled to solve general optimization problems. Islands that are well suited as habitats for biological species are said to have a high habitat suitability index (HSI). Islands with a low HSI have a small number of species, whereas islands with a high HSI have many species that emigrate to nearby islands because of the accumulation of random effects on its large populations. Emigration occurs as animals ride flotsam, fly, or swim to neighboring islands [10, 11]. Suppose that we have some problem, and that we also have several candidate solutions. A good solution is analogous to an island with a high HSI, and a poor solution is like an island with a low HSI. High HSI solutions are more likely to share their features with other solutions, and low HSI solutions are more likely to accept shared features from other solutions. This approach to problem solving is called biogeography-based optimization. 2.2
Migration Operator
In biogeography, species may migrate between islands. In BBO, the solution features may affect each other between islands. For the sake of simplicity, we set E = I and two solutions are given in Figure 1. Figure 1 illustrates the migration curves along with two solutions. S1 represents a poor solution and S2 represents a good solution. The immigration rate for S1 will therefore be larger than the immigration rate for S2 , and the emigration rate for S1 will be smaller than the emigration rate for S2 .
Novel Binary BBO Algorithm for the Knapsack Problem
219
E=I λ
μ
rate
s1
species
s2
smax
Fig. 1. Illustration of two solutions
The BBO uses the migration rates of each solution to probabilistically share features between solutions. The migration operator in the BBO is as follows. The immigration curve is used to probabilistically decide each feature whether or not to immigrate in each solution. If a given solution feature is selected to be immigrated, then the emigrating island is selected probabilistically. The pseudo code of the migration operator is presented in Figure 2.
： ： ： ：
// NP is the size of population, D is the dimension size 1 For i=1 to NP 2 Select the H i with probability λ i 3 If H i is selected, then For j=1 to D 4 Select the H j with probability μ i 5: If H j is selected, then 6: to generate a random integer k ∈ [1, D ] 7: let H ik = H jk 8 endfor 9: endfor
：
Fig. 2. Migration Operator
2.3
Mutation Operator
Cataclysmic events can drastically change the HSI of a natural habitat. An island habitat’s HSI can therefore change suddenly due to apparently random events. In the BBO, a mutation operator can be derived from this. The species count
220
BiB. Zhao et al.
probabilities are used to determine mutation rates. The mutation rate is given in formula (2) [10]. m( S ) = mmax (
1 − Ps ) Pmax
.
(2)
where m max is a user-defined parameter. The mutation operator is given in figure 3.
： ： ： ： ：
// NP is the size of population, D is the dimension size 1 for i=1 to NP Using the formula (2)to calculate mutation 2 probability Pi 3 select SIV in H i ( j ) with probability Pi ( j ) 4 if H i ( j ) is selected, then 5 Replace the H i ( j ) with a random generated SIV 6: endfor Fig. 3. Mutation Operator
2.4
Elitism
In order to achieve global convergence, similar to the common practice in GAs, elitism is incorporated into BBO. The best solution in the current population is preserved from one generation to the next.
3
Binary Biogeography-Based Optimization Algorithm for Knapsack Problem
The traditional BBO uses an integer coding scheme. The Knapsack Problem is a constrained zero-one programming problem. The migration operator in the conventional BBO is inherited in the binary BBO and two new mutation operators were used as well. 3.1
Migration Operator
The migration operator in the BBO is used in the binary BBO as is. 3.2
Binary Mutation Operator
The mutation operator in binary BBO, defined in Figure 4, uses formula (2) to decide the probability of each bit number in the population to flip.
Novel Binary BBO Algorithm for the Knapsack Problem
221
： ： ： ： ：
// NP is the size of population, D is the dimension size 1 for i=1 to NP 2 Using the formula (2) to calculate mutation probability Pi 3 select SIV in H i ( j ) with probability Pi ( j ) 4 if H i ( j ) is selected, then 5 Replace the H i ( j ) with 1- H i ( j ) 6: endfor Fig. 4. Binary Mutation Operator
3.3
Greedy Mutation Operator for the Infeasible Solutions
During the optimization process in BBO, infeasible solutions may occur. An additional greedy mutation operator was defined in binary BBO to repair these infeasible solutions. Firstly, it sorts the items according to vi / wi in ascending order. Then, when an infeasible solution is found, it flips the features if they are ones in order of vi / wi until the solution becomes feasible.
4
Numerical Experiments
In this section, we investigate the performance of binary BBO solving Knapsack Problems of different scales. Four benchmark problems from [5-8] (thereafter they are denoted by Kp1, Kp2, Kp3 and Kp4 respectively) were used to compare the binary BBO to the standard Genetic Algorithm (GA). 4.1
Results
The parameter setting of the binary BBO for each problem is presented in table 1. Table 1. Parameters of the binary BBO Problems
E
I
Kp1 Kp2 Kp3 Kp4
1 1 1 1
1 1 1 1
Population size 100 250 250 250
m max 0.005 0.005 0.005 0.005
Elitism size 2 2 2 2
Maximum Generation 100 100 100 100
For comparison, the standard GA with single point crossover with a crossover probability 0.8, and a mutation probability of 0.05 is used. In the GA, we deal with
222
BiB. Zhao et al.
the infeasible solutions using the same greedy mutation operator from the binary BBO. The population size and the maximum generation number of the GA are the same as that of the binary BBO for each problem. For each of the four Knapsack Problems, 100 runs have been conducted and the best results (Best), average results (Avg), worst results (Worst), standard deviations (Dev) and the success rate (SR) of the binary BBO are shown in Table 2. Table 2. Compared results between binary BBO and GA Examples Kp1 Kp2 Kp3 Kp4
Algorithms Binary BBO GA Binary BBO GA Binary BBO GA Binary BBO GA
Best 1042 1042 3119 3116 26559 25435 5375 5326
Avg 1041.1 1040.2 3118.5 3090.4 26554 2432.2 5361.9 5298.5
Worst 1037 1037 3113 3060 26534 23076 5343 5269
Dev 1.9714 2.4121 1.4530 13.6569 9.4814 534.8347 13.1674 11.3649
SR 81% 64% 82% 0% 80% 0% 25% 0%
Figures 5 and 6 present the compared average convergence of the 100 independent trials for the four Knapsack Problems. They clearly show that the average convergence speed of binary BBO is superior to that of the GA for each benchmark problem.
Fig. 5. Convergence speed of binary BBO versus GA for Kp1 and Kp2
Novel Binary BBO Algorithm for the Knapsack Problem
223
Fig. 6. Convergence speed of binary BBO versus GA for Kp3 and Kp4
5
Conclusion
In this article, a binary version of BBO has been proposed for the solution of Knapsack Problems in this paper. A binary mutation operator was used to improve the exploration ability of the algorithm and a greedy mutation operator was used to handle infeasible solutions of the Knapsack Problem. Simulation results show that the performance of the binary BBO is superior to that of the standard GA. Application of this binary BBO to other combinatorial optimization problems will be our future work. Acknowledgment. This work is partially supported by the Science and Technology Foundation of Jiangxi Province under Grant No. GJJ11616.
References 1. Masako, F., Takeo, Y.: An exact algorithm for the Knapsack sharing problem with common items. European Journal of Operational Research 171(2), 693–707 (2006) 2. Didier, E.B., Moussa, E.: Load balancing methods and parallel dynamic programming algorithm using dominance technique applied to the 0-1 Knapsack Problem. Journal of Parallel and Distributed Computing 65(1), 74–84 (2005)
224
BiB. Zhao et al.
3. Stephen, C.H., Leung, Z.D.F., Zhou, C.L., Wu, T.: A hybrid simulated annealing metaheuristic algorithm for the two-dimensional Knapsack packing problem. Computers & Operations Research 39(1), 64–73 (2010) 4. Kong, M., Peng, T., Kao, Y.C.: A new ant colony optimization algorithm for the multidimensional Knapsack Problem. Computers & Operations Research 35(8), 2672–2683 (2008) 5. Maria, J.A., Marla, A., Motg, A.: A multiobjective Tchebycheff based genetic algorithm for the multidimensional Knapsack Problem. Computers & Operations Research 34(11), 3458–3470 (2007) 6. Zhang, L., Zhang, B.: Good point set based genetic algorithm. Chinese Journal of Computers 24(9), 917–922 (2001) 7. Shen, X.J., Wang, W.W., Zheng, P.J.: Modified particle swarm optimization for 0/1 Knapsack problem. Computer Engineering 32(18), 23–24, 38 (2006) 8. Gao, T., Wang, M.G.: The research for the reductive dimension and replacive variable algorithm of special restrict ILP. Systems Engineering Theory Methodology Applications 11(2), 125–130 (2002) 9. Simon, D.: Biogeography-based optimization. IEEE Trans. Evol. Comput. 12(6), 702–713 (2008) 10. Simon, D., Mehmet, E., Dawei, D.: Population Distributions in Biogeography-based Optimization algorithms with Elitism. In: SMC, pp. 1–6 (2009) 11. Simon, D., Mehmet, E., Dawei, D., Rick, R.: Markov Models for Biogeography-Based Optimization. IEEE Transactions on Systems, Man, and Cybernetics—PART B: Cybernetics 41(1), 299–306 (2011)
School of Business, Jiujiang University, Jiujiang 332005, China School of Information Science and Technology, Jiujiang University, Jiujiang 332005, China [email protected],{csdeng,yangyanlin,xx_penghu}@jju.edu.cn 2
Abstract. Mathematical models of biogeography inspired the development of the biogeography-based optimization algorithm. In this article we propose a binary version of biogeography-based optimization (BBO) for the Knapsack Problem. Two new mutation operators are proposed to extend the biogeography-based optimization algorithm to binary optimization problems. We also demonstrate the performance of the resulting new binary Biogeography-based optimization algorithm in solving four Knapsack problems and compare it with that of the standard Genetic Algorithm. The simulation results show that our new method is effective and efficient for the Knapsack problem. Keywords: Knapsack Problem, Biogeography-based optimization, Migration operator, mutation operator.
1
Introduction
Knapsack problems have been widely studied not only because of their immediate application in industry and financial management, but also more pronounced for theoretical reasons. Many industrial problems such as resource allocation, investment decision-making, and hold loading can be modeled as Knapsack problems. The Knapsack problem can be described as follows. Given a set of items, each with a value vi and a weight wi , determine how often each item should be included in a collection so that the total weight is less than or equal to a given limit ( W ) and the total value is as large as possible. The most common formulation of the problem is the 0-1 Knapsack problem, where each item can either be included one time or not at all, which can be mathematically formulated as: n
Maximize
f =
∑v x
i i
i =1
n
s.t.
∑w x ≤ W, i i
xi ∈ {0,1}
i =1
*
Corresponding author. [email protected]
Y. Tan, Y. Shi, and Z. Ji (Eds.): ICSI 2012, Part I, LNCS 7331, pp. 217–224, 2012. © Springer-Verlag Berlin Heidelberg 2012
(1)
218
BiB. Zhao et al.
Recently, many exact algorithms and nature-inspired algorithms have been proposed to cope with the Knapsack problem [1-8]. However, several methods of them often can only provide locally optimal solutions. The biogeography-based optimization algorithm is a new nature-inspired computation technique proposed by Simon in 2008 [9]. It is originally designed for unconstrained integer programming problems. In this paper, we propose a novel binary BBO for Knapsack problems. To the best of our knowledge, this is the first application of BBO to the Knapsack problem. The rest of the paper is organized as follows. Section 2 gives an overview of BBO. A novel Binary BBO with two mutation operators for the Knapsack problems is presented in section 3. Four Knapsack Problems with different sizes are used to compare the performance of the Binary BBO with that of the Genetic Algorithm in section 4. Section 5 concludes this paper.
2
Biogeography-Based Optimization Algorithm
2.1
Mathematical Model of BBO
Biogeography studies the migration, speciation, and extinction of species from one island to another. With the mathematical models in the Biogeography, Simon (2008) proposed the BBO algorithm [9], which is an example of how a natural process can be modeled to solve general optimization problems. Islands that are well suited as habitats for biological species are said to have a high habitat suitability index (HSI). Islands with a low HSI have a small number of species, whereas islands with a high HSI have many species that emigrate to nearby islands because of the accumulation of random effects on its large populations. Emigration occurs as animals ride flotsam, fly, or swim to neighboring islands [10, 11]. Suppose that we have some problem, and that we also have several candidate solutions. A good solution is analogous to an island with a high HSI, and a poor solution is like an island with a low HSI. High HSI solutions are more likely to share their features with other solutions, and low HSI solutions are more likely to accept shared features from other solutions. This approach to problem solving is called biogeography-based optimization. 2.2
Migration Operator
In biogeography, species may migrate between islands. In BBO, the solution features may affect each other between islands. For the sake of simplicity, we set E = I and two solutions are given in Figure 1. Figure 1 illustrates the migration curves along with two solutions. S1 represents a poor solution and S2 represents a good solution. The immigration rate for S1 will therefore be larger than the immigration rate for S2 , and the emigration rate for S1 will be smaller than the emigration rate for S2 .
Novel Binary BBO Algorithm for the Knapsack Problem
219
E=I λ
μ
rate
s1
species
s2
smax
Fig. 1. Illustration of two solutions
The BBO uses the migration rates of each solution to probabilistically share features between solutions. The migration operator in the BBO is as follows. The immigration curve is used to probabilistically decide each feature whether or not to immigrate in each solution. If a given solution feature is selected to be immigrated, then the emigrating island is selected probabilistically. The pseudo code of the migration operator is presented in Figure 2.
： ： ： ：
// NP is the size of population, D is the dimension size 1 For i=1 to NP 2 Select the H i with probability λ i 3 If H i is selected, then For j=1 to D 4 Select the H j with probability μ i 5: If H j is selected, then 6: to generate a random integer k ∈ [1, D ] 7: let H ik = H jk 8 endfor 9: endfor
：
Fig. 2. Migration Operator
2.3
Mutation Operator
Cataclysmic events can drastically change the HSI of a natural habitat. An island habitat’s HSI can therefore change suddenly due to apparently random events. In the BBO, a mutation operator can be derived from this. The species count
220
BiB. Zhao et al.
probabilities are used to determine mutation rates. The mutation rate is given in formula (2) [10]. m( S ) = mmax (
1 − Ps ) Pmax
.
(2)
where m max is a user-defined parameter. The mutation operator is given in figure 3.
： ： ： ： ：
// NP is the size of population, D is the dimension size 1 for i=1 to NP Using the formula (2)to calculate mutation 2 probability Pi 3 select SIV in H i ( j ) with probability Pi ( j ) 4 if H i ( j ) is selected, then 5 Replace the H i ( j ) with a random generated SIV 6: endfor Fig. 3. Mutation Operator
2.4
Elitism
In order to achieve global convergence, similar to the common practice in GAs, elitism is incorporated into BBO. The best solution in the current population is preserved from one generation to the next.
3
Binary Biogeography-Based Optimization Algorithm for Knapsack Problem
The traditional BBO uses an integer coding scheme. The Knapsack Problem is a constrained zero-one programming problem. The migration operator in the conventional BBO is inherited in the binary BBO and two new mutation operators were used as well. 3.1
Migration Operator
The migration operator in the BBO is used in the binary BBO as is. 3.2
Binary Mutation Operator
The mutation operator in binary BBO, defined in Figure 4, uses formula (2) to decide the probability of each bit number in the population to flip.
Novel Binary BBO Algorithm for the Knapsack Problem
221
： ： ： ： ：
// NP is the size of population, D is the dimension size 1 for i=1 to NP 2 Using the formula (2) to calculate mutation probability Pi 3 select SIV in H i ( j ) with probability Pi ( j ) 4 if H i ( j ) is selected, then 5 Replace the H i ( j ) with 1- H i ( j ) 6: endfor Fig. 4. Binary Mutation Operator
3.3
Greedy Mutation Operator for the Infeasible Solutions
During the optimization process in BBO, infeasible solutions may occur. An additional greedy mutation operator was defined in binary BBO to repair these infeasible solutions. Firstly, it sorts the items according to vi / wi in ascending order. Then, when an infeasible solution is found, it flips the features if they are ones in order of vi / wi until the solution becomes feasible.
4
Numerical Experiments
In this section, we investigate the performance of binary BBO solving Knapsack Problems of different scales. Four benchmark problems from [5-8] (thereafter they are denoted by Kp1, Kp2, Kp3 and Kp4 respectively) were used to compare the binary BBO to the standard Genetic Algorithm (GA). 4.1
Results
The parameter setting of the binary BBO for each problem is presented in table 1. Table 1. Parameters of the binary BBO Problems
E
I
Kp1 Kp2 Kp3 Kp4
1 1 1 1
1 1 1 1
Population size 100 250 250 250
m max 0.005 0.005 0.005 0.005
Elitism size 2 2 2 2
Maximum Generation 100 100 100 100
For comparison, the standard GA with single point crossover with a crossover probability 0.8, and a mutation probability of 0.05 is used. In the GA, we deal with
222
BiB. Zhao et al.
the infeasible solutions using the same greedy mutation operator from the binary BBO. The population size and the maximum generation number of the GA are the same as that of the binary BBO for each problem. For each of the four Knapsack Problems, 100 runs have been conducted and the best results (Best), average results (Avg), worst results (Worst), standard deviations (Dev) and the success rate (SR) of the binary BBO are shown in Table 2. Table 2. Compared results between binary BBO and GA Examples Kp1 Kp2 Kp3 Kp4
Algorithms Binary BBO GA Binary BBO GA Binary BBO GA Binary BBO GA
Best 1042 1042 3119 3116 26559 25435 5375 5326
Avg 1041.1 1040.2 3118.5 3090.4 26554 2432.2 5361.9 5298.5
Worst 1037 1037 3113 3060 26534 23076 5343 5269
Dev 1.9714 2.4121 1.4530 13.6569 9.4814 534.8347 13.1674 11.3649
SR 81% 64% 82% 0% 80% 0% 25% 0%
Figures 5 and 6 present the compared average convergence of the 100 independent trials for the four Knapsack Problems. They clearly show that the average convergence speed of binary BBO is superior to that of the GA for each benchmark problem.
Fig. 5. Convergence speed of binary BBO versus GA for Kp1 and Kp2
Novel Binary BBO Algorithm for the Knapsack Problem
223
Fig. 6. Convergence speed of binary BBO versus GA for Kp3 and Kp4
5
Conclusion
In this article, a binary version of BBO has been proposed for the solution of Knapsack Problems in this paper. A binary mutation operator was used to improve the exploration ability of the algorithm and a greedy mutation operator was used to handle infeasible solutions of the Knapsack Problem. Simulation results show that the performance of the binary BBO is superior to that of the standard GA. Application of this binary BBO to other combinatorial optimization problems will be our future work. Acknowledgment. This work is partially supported by the Science and Technology Foundation of Jiangxi Province under Grant No. GJJ11616.
References 1. Masako, F., Takeo, Y.: An exact algorithm for the Knapsack sharing problem with common items. European Journal of Operational Research 171(2), 693–707 (2006) 2. Didier, E.B., Moussa, E.: Load balancing methods and parallel dynamic programming algorithm using dominance technique applied to the 0-1 Knapsack Problem. Journal of Parallel and Distributed Computing 65(1), 74–84 (2005)
224
BiB. Zhao et al.
3. Stephen, C.H., Leung, Z.D.F., Zhou, C.L., Wu, T.: A hybrid simulated annealing metaheuristic algorithm for the two-dimensional Knapsack packing problem. Computers & Operations Research 39(1), 64–73 (2010) 4. Kong, M., Peng, T., Kao, Y.C.: A new ant colony optimization algorithm for the multidimensional Knapsack Problem. Computers & Operations Research 35(8), 2672–2683 (2008) 5. Maria, J.A., Marla, A., Motg, A.: A multiobjective Tchebycheff based genetic algorithm for the multidimensional Knapsack Problem. Computers & Operations Research 34(11), 3458–3470 (2007) 6. Zhang, L., Zhang, B.: Good point set based genetic algorithm. Chinese Journal of Computers 24(9), 917–922 (2001) 7. Shen, X.J., Wang, W.W., Zheng, P.J.: Modified particle swarm optimization for 0/1 Knapsack problem. Computer Engineering 32(18), 23–24, 38 (2006) 8. Gao, T., Wang, M.G.: The research for the reductive dimension and replacive variable algorithm of special restrict ILP. Systems Engineering Theory Methodology Applications 11(2), 125–130 (2002) 9. Simon, D.: Biogeography-based optimization. IEEE Trans. Evol. Comput. 12(6), 702–713 (2008) 10. Simon, D., Mehmet, E., Dawei, D.: Population Distributions in Biogeography-based Optimization algorithms with Elitism. In: SMC, pp. 1–6 (2009) 11. Simon, D., Mehmet, E., Dawei, D., Rick, R.: Markov Models for Biogeography-Based Optimization. IEEE Transactions on Systems, Man, and Cybernetics—PART B: Cybernetics 41(1), 299–306 (2011)
