Application of Radial Basis Function Neural ... - Semantic Scholar
2009 International Asia Conference on Informatics in Control, Automation and Robotics
Application of Radial Basis Function Neural Network for Sales Forecasting Tung-Lai Hu
Department of Industrial Management
Department of Business Management
National Taiwan University of Science and Technology, Taipei, Taiwan [email protected]
National Taipei University of Technology, Taipei, Taiwan [email protected]
Keywords － sales forecasting; hybrid evolutionary algorithm; particle swarm optimization; genetic algorithm; radial basis function neural network.
I.
⎛ x − cj Φ ( x − c j ) = exp − ⎜ ⎜ σj ⎝
(1)
vector
x and a centre c j , and σ j represents the width of
the jth RBFnn hidden node. The key problems of RBFnn, including determining centers and widths of RBF, the number of hidden nodes, weights between hidden and output layers, and the parameters of hidden layer are optimized locally not globally [3]. Evolutionary computation (EC) techniques are a search algorithm using the concepts of evolutionary pressure to search for fit solutions to problems [24]. The emergence of various neural network (NN) topologies and efficient learning algorithms have led to a wide range of successful applications in forecasting [7]. Owing to its particular structure, a NN is very good in learning using some learning algorithms such as the GA [21] and PSO [10]. [8] focuses on the advantage of PSO into the mutation process of GA for improving the GA learning efficiency. [13] proposed a new hybrid genetical swarm optimization (GSO) approach, consists in a stronger cooperation of GA and PSO, maintaining the integration of them for the entire optimization run. This kind of updating results in an evolutionary process where individuals not only improve their score for natural selection of the fitness or for good-knowledge sharing, but for both of them at the same time.
LITERATURE REVIEW
When linearity or nonlinearity was considered, [23] concluded that the artificial neural network (ANN) model was significantly better than the reference average. [26] compares Corresponding author. E-mail: [email protected] (Zhen-Yao Chen).
978-0-7695-3519-7/09 $25.00 © 2009 IEEE DOI 10.1109/CAR.2009.97
2
⎞ ⎟ , ⎟ ⎠
where x − c j is the Euclidean distance between an input
INTRODUCTION
Box and Jenkins first developed the auto-regressive integrated moving average (ARIMA) methodology for forecasting time series events in 1976. Time series data are often examined in hopes of discovering a historical pattern that can be exploited in the forecast [2]. A linear correlation structure is assumed among the time series data and therefore, no nonlinear patterns can be captured by the auto-regressive moving average (ARMA (p, q)) (i.e., ARIMA (p, 0, q)) models [25]. However, artificial neural networks (ANN), like RBFnn, do not have such limitation. But, determinations of RBFnn structure and weights are time consuming. Therefore, due to promising results of PSO and GA, this study intends to combine PSO and GA for RBFnn learning. In this research, it mainly precedes practical short-term sales forecasting analysis of papaya milk in retail industry. The daily sales observations of 500 cm3 containers of papaya milk were offered by a parent company of chain convenience stores in Taiwan. The results being compared to others algorithms, PSO and GA, and traditional ARMA models, illustrate the accuracy of the proposed algorithm. II.
Institute of Industrial and Business Management National Taipei University of Technology, Taipei, Taiwan [email protected]
ANN, ARMA and the combined models in forecasting the wheat price of China Zhengzhou Grain Wholesale Market, and the results show the ANN model forecasts are considerably more accurate than the traditional ARMA models, which used as a benchmark [26]. There have been many reports where practitioners have applied radial basis functions (RBFs) to various time series problems [14]. A typical hidden node in an RBFnn is characterized by its centre, which is a vector with dimension equal to the number of inputs to the node. Gaussian basis functions ( Φ ) are the most frequently used RBFs with the following form [10],
Abstract—This paper proposes a hybrid evolutionary algorithm based radial basis function neural network (RBFnn) for sales forecasting. The proposed hybrid of particle swarm and genetic algorithm based optimization (HPSGO) algorithm gathers virtues of particle swarm optimization (PSO) and genetic algorithm (GA) to improve the learning performance of RBFnn. The diversity of chromosomes results in higher chance to search in the direction of global minimum instead of being confined to local minimum. Experimental results of papaya milk sales data show that the proposed HPSGO algorithm outperforms PSO, GA and Box-Jenkins model in accuracy.
＊
Zhen-Yao Chen ＊
R.J. Kuo
325
III.
METHODOLOGY
C. The proposed HPSGO algorithm The proposed hybrid of particle swarm and genetic algorithm based optimization (HPSGO) algorithm, which combines the evolutionary learning approaches of the PSO and GA, is designed to resolve the problem of network parameters training and solving with RBFnn. New generations are produced using the Roulette wheel selection [12] mechanism and four genetic operators: uniform crossover, one-point mutation, deletion and addition. The algorithm stops after a specified number of generations have been completed. This study adopted the root mean squared error (RMSE) to measure the error committed by the algorithm when used to predict the instances in the training set. We have used the inverse of RMSE as fitness function, i.e., Fitness = 1 / RMSE [5]. In this paper, the fitness value for above mentioned algorithms applied to sales forecasting are computed by maximizing the inverse of RMSE defined as [18] N , (4) −1
Since PSO and GA both work with a population of solutions, combining the searching abilities of both methods seems to be a good approach [20]. To begin with, the following goes to account for evolutionary procedure for PSO-based [10] and GA-based [21] algorithms respectively. A. The PSO-based algorithm [10] The PSO-based algorithm could be performed by the following equations [16, 18]: vin (t + 1) = k ⋅ vin (t ) + c1 ⋅ r1n (t ) ⋅ ( pin (t ) − xin (t ))
+c2 ⋅ r2 n (t ) ⋅ ( pgn (t ) − xin (t )), xin (t + 1) = xin (t ) + vin (t + 1),
(2) (3)
r1n (t ) and r2 n (t ) are random numbers uniformly distributed in the interval (0, 1), c1 and c2 are acceleration constants, and k denotes the inertia weight. The k is a userdefined parameter that controls, with c1 and c2 , the previous where
Fitness = RMSE =
values of particle velocities on its current ones. The c1 ∗ r1n (t ) ∗ ( pin (t ) − xin (t )) component in (2), referred to as the cognitive component, represents the distance that a particle is from the best solution, pin (t ) , found by itself. The
∑
N j =1
( y j − yˆ j ) 2
where y j is the actual output and yˆ j is the predicted output of the learned RBFnn model for the jth training pattern. Next, the nonlinear function that the RBFnn hidden layer adopts is the Gaussian function shown in (1), and the fitness value of individuals in population is calculated by (4). Then, the optimal values of parameters solution can be obtained and used in the algorithm with the network to solve the problem for sales forecasting. The pseudo code for the proposed HPSGO algorithm is illustrated as Fig. 3.
c2 ∗ r2 n (t ) ∗ ( pgn (t ) − xin (t )) component in (2), referred to as the
social component, represents the distance that a particle is from the best solution found by its neighbourhood. Equation (2) is used to calculate the new velocity of the particle according to its previous velocity and to the distances of its current position from its own best historical position and the collaborative effect of particles. It has cooperation between all particles to share information. Then, the particle updates the new position according to (3) [16, 18]. By the random selection, the number of RBFnn active nodes for each particle can be determined. It will cause operation trouble when the remaining vector length of global best (Gbest, pgn ) and local best (Lbest, pin ) in (2) are different from the others. The maximum value of Maximum selection type PSO learning method will be considered the active number of RBFs for all particles and ensure that the same vector length is achieved [10]. Thus, [10] used the method to decide the same vector length of pgn and pin in (2).
Figure 1. The pseudo code for the PSO-based algorithm.
Once the new position of a particle is calculated using (3) [10], the particle will fly towards it [22]. The pseudo code for the PSO-based algorithm [10] is showed as Fig. 1. B. The GA-based algorithm [21] In [21], the GA-based algorithm starts with an initial population of chromosomes, which represent possible network structures and contain the associated center locations. The chromosome that has produced the minimum value of the objective function is selected as the optimum NN model. The pseudo code for the GA-based algorithm [21] is showed as Fig. 2.
Figure 2. The pseudo code for the GA-based algorithm.
326
Application of Radial Basis Function Neural Network for Sales Forecasting Tung-Lai Hu
Department of Industrial Management
Department of Business Management
National Taiwan University of Science and Technology, Taipei, Taiwan [email protected]
National Taipei University of Technology, Taipei, Taiwan [email protected]
Keywords － sales forecasting; hybrid evolutionary algorithm; particle swarm optimization; genetic algorithm; radial basis function neural network.
I.
⎛ x − cj Φ ( x − c j ) = exp − ⎜ ⎜ σj ⎝
(1)
vector
x and a centre c j , and σ j represents the width of
the jth RBFnn hidden node. The key problems of RBFnn, including determining centers and widths of RBF, the number of hidden nodes, weights between hidden and output layers, and the parameters of hidden layer are optimized locally not globally [3]. Evolutionary computation (EC) techniques are a search algorithm using the concepts of evolutionary pressure to search for fit solutions to problems [24]. The emergence of various neural network (NN) topologies and efficient learning algorithms have led to a wide range of successful applications in forecasting [7]. Owing to its particular structure, a NN is very good in learning using some learning algorithms such as the GA [21] and PSO [10]. [8] focuses on the advantage of PSO into the mutation process of GA for improving the GA learning efficiency. [13] proposed a new hybrid genetical swarm optimization (GSO) approach, consists in a stronger cooperation of GA and PSO, maintaining the integration of them for the entire optimization run. This kind of updating results in an evolutionary process where individuals not only improve their score for natural selection of the fitness or for good-knowledge sharing, but for both of them at the same time.
LITERATURE REVIEW
When linearity or nonlinearity was considered, [23] concluded that the artificial neural network (ANN) model was significantly better than the reference average. [26] compares Corresponding author. E-mail: [email protected] (Zhen-Yao Chen).
978-0-7695-3519-7/09 $25.00 © 2009 IEEE DOI 10.1109/CAR.2009.97
2
⎞ ⎟ , ⎟ ⎠
where x − c j is the Euclidean distance between an input
INTRODUCTION
Box and Jenkins first developed the auto-regressive integrated moving average (ARIMA) methodology for forecasting time series events in 1976. Time series data are often examined in hopes of discovering a historical pattern that can be exploited in the forecast [2]. A linear correlation structure is assumed among the time series data and therefore, no nonlinear patterns can be captured by the auto-regressive moving average (ARMA (p, q)) (i.e., ARIMA (p, 0, q)) models [25]. However, artificial neural networks (ANN), like RBFnn, do not have such limitation. But, determinations of RBFnn structure and weights are time consuming. Therefore, due to promising results of PSO and GA, this study intends to combine PSO and GA for RBFnn learning. In this research, it mainly precedes practical short-term sales forecasting analysis of papaya milk in retail industry. The daily sales observations of 500 cm3 containers of papaya milk were offered by a parent company of chain convenience stores in Taiwan. The results being compared to others algorithms, PSO and GA, and traditional ARMA models, illustrate the accuracy of the proposed algorithm. II.
Institute of Industrial and Business Management National Taipei University of Technology, Taipei, Taiwan [email protected]
ANN, ARMA and the combined models in forecasting the wheat price of China Zhengzhou Grain Wholesale Market, and the results show the ANN model forecasts are considerably more accurate than the traditional ARMA models, which used as a benchmark [26]. There have been many reports where practitioners have applied radial basis functions (RBFs) to various time series problems [14]. A typical hidden node in an RBFnn is characterized by its centre, which is a vector with dimension equal to the number of inputs to the node. Gaussian basis functions ( Φ ) are the most frequently used RBFs with the following form [10],
Abstract—This paper proposes a hybrid evolutionary algorithm based radial basis function neural network (RBFnn) for sales forecasting. The proposed hybrid of particle swarm and genetic algorithm based optimization (HPSGO) algorithm gathers virtues of particle swarm optimization (PSO) and genetic algorithm (GA) to improve the learning performance of RBFnn. The diversity of chromosomes results in higher chance to search in the direction of global minimum instead of being confined to local minimum. Experimental results of papaya milk sales data show that the proposed HPSGO algorithm outperforms PSO, GA and Box-Jenkins model in accuracy.
＊
Zhen-Yao Chen ＊
R.J. Kuo
325
III.
METHODOLOGY
C. The proposed HPSGO algorithm The proposed hybrid of particle swarm and genetic algorithm based optimization (HPSGO) algorithm, which combines the evolutionary learning approaches of the PSO and GA, is designed to resolve the problem of network parameters training and solving with RBFnn. New generations are produced using the Roulette wheel selection [12] mechanism and four genetic operators: uniform crossover, one-point mutation, deletion and addition. The algorithm stops after a specified number of generations have been completed. This study adopted the root mean squared error (RMSE) to measure the error committed by the algorithm when used to predict the instances in the training set. We have used the inverse of RMSE as fitness function, i.e., Fitness = 1 / RMSE [5]. In this paper, the fitness value for above mentioned algorithms applied to sales forecasting are computed by maximizing the inverse of RMSE defined as [18] N , (4) −1
Since PSO and GA both work with a population of solutions, combining the searching abilities of both methods seems to be a good approach [20]. To begin with, the following goes to account for evolutionary procedure for PSO-based [10] and GA-based [21] algorithms respectively. A. The PSO-based algorithm [10] The PSO-based algorithm could be performed by the following equations [16, 18]: vin (t + 1) = k ⋅ vin (t ) + c1 ⋅ r1n (t ) ⋅ ( pin (t ) − xin (t ))
+c2 ⋅ r2 n (t ) ⋅ ( pgn (t ) − xin (t )), xin (t + 1) = xin (t ) + vin (t + 1),
(2) (3)
r1n (t ) and r2 n (t ) are random numbers uniformly distributed in the interval (0, 1), c1 and c2 are acceleration constants, and k denotes the inertia weight. The k is a userdefined parameter that controls, with c1 and c2 , the previous where
Fitness = RMSE =
values of particle velocities on its current ones. The c1 ∗ r1n (t ) ∗ ( pin (t ) − xin (t )) component in (2), referred to as the cognitive component, represents the distance that a particle is from the best solution, pin (t ) , found by itself. The
∑
N j =1
( y j − yˆ j ) 2
where y j is the actual output and yˆ j is the predicted output of the learned RBFnn model for the jth training pattern. Next, the nonlinear function that the RBFnn hidden layer adopts is the Gaussian function shown in (1), and the fitness value of individuals in population is calculated by (4). Then, the optimal values of parameters solution can be obtained and used in the algorithm with the network to solve the problem for sales forecasting. The pseudo code for the proposed HPSGO algorithm is illustrated as Fig. 3.
c2 ∗ r2 n (t ) ∗ ( pgn (t ) − xin (t )) component in (2), referred to as the
social component, represents the distance that a particle is from the best solution found by its neighbourhood. Equation (2) is used to calculate the new velocity of the particle according to its previous velocity and to the distances of its current position from its own best historical position and the collaborative effect of particles. It has cooperation between all particles to share information. Then, the particle updates the new position according to (3) [16, 18]. By the random selection, the number of RBFnn active nodes for each particle can be determined. It will cause operation trouble when the remaining vector length of global best (Gbest, pgn ) and local best (Lbest, pin ) in (2) are different from the others. The maximum value of Maximum selection type PSO learning method will be considered the active number of RBFs for all particles and ensure that the same vector length is achieved [10]. Thus, [10] used the method to decide the same vector length of pgn and pin in (2).
Figure 1. The pseudo code for the PSO-based algorithm.
Once the new position of a particle is calculated using (3) [10], the particle will fly towards it [22]. The pseudo code for the PSO-based algorithm [10] is showed as Fig. 1. B. The GA-based algorithm [21] In [21], the GA-based algorithm starts with an initial population of chromosomes, which represent possible network structures and contain the associated center locations. The chromosome that has produced the minimum value of the objective function is selected as the optimum NN model. The pseudo code for the GA-based algorithm [21] is showed as Fig. 2.
Figure 2. The pseudo code for the GA-based algorithm.
326
