An Improved Continuous-Action Extended Classifier ... - ScienceDirect
Available online at www.sciencedirect.com
ScienceDirect Procedia Computer Science 61 (2015) 361 – 366
Complex Adaptive Systems, Publication 5 Cihan H. Dagli, Editor in Chief Conference Organized by Missouri University of Science and Technology 2015-San Jose, CA
An Improved Continuous-Action Extended Classifier Systems for Function Approximation Omid Saremia*, Masoud Shariat Panahia, Amin Sabzehzarb b
a Mechanical Engineering Department, University of Tehran, Tehran, Iran Mechanical Engineering Department, University of Nevada, Reno, Nevada, USA
Abstract Due to their structural simplicity and superior generalization capability, Extended Classifier Systems (XCSs) are gaining popularity within the Artificial Intelligence community. In this study an improved XCS with continuous actions is introduced for function approximation purposes. The proposed XCSF uses “prediction zones,” rather than distinct “prediction values,” to enable multi-member match sets that would allow multiple rules to be evaluated per training step. It is shown that this would accelerate the training procedure and reduce the computational cost associated with the training phase. The improved XCSF is also shown to produce more accurate rules than the classical classifier system when it comes to approximating complex nonlinear functions. © 2015 2015The TheAuthors. Authors.Published Published Elsevier © byby Elsevier B.V.B.V. This is an open access article under the CC BY-NC-ND license (http://creativecommons.org/licenses/by-nc-nd/4.0/). Peer-review under responsibility of scientific committee of Missouri University of Science and Technology. Peer-review under responsibility of scientific committee of Missouri University of Science and Technology Keywords: Improved XCSF; Classifier Systems; Function approximation
1. Introduction Learning classifier systems were first introduced by John Holland in 1992[1]. The early variants used a set of randomly generated classifiers (rules) that were gradually evolved into more specialized rules using Genetic Algorithms. In 1994 Wilson introduced ZCS which improved the performance of LCS and reduce its computational cost[2]. In 1995 Wilson proposed a new way to calculate the classifier’s fitness which was a quantitative measure of a rule’s desirability. He presented eXtended Classifier Systems (XCS) in which accuracy plays a major role in
* Corresponding author. Tel.: +98-935-558-2736 E-mail address: [email protected]
1877-0509 © 2015 The Authors. Published by Elsevier B.V. This is an open access article under the CC BY-NC-ND license (http://creativecommons.org/licenses/by-nc-nd/4.0/). Peer-review under responsibility of scientific committee of Missouri University of Science and Technology doi:10.1016/j.procs.2015.09.160
362
Omid Saremi et al. / Procedia Computer Science 61 (2015) 361 – 366
computing classifier fitness[3]. Later studies mostly focused on XCS and its improvements like XCSR[4], TCS[5], and etc. XCSF is one of modified version of XCS, which presented by Wilson in 2002[6]. It evolves classifiers represent piecewise linear approximations of reward that is commonly the problem solution or the function value[7]. It firstly had been used as approximator[6]. Later, Lanzi et al[8, 9] have done some generalizations on XCSF prediction updating method. Some advanced approximators such as polynomials[10], neural networks[11] and … were studied by them. Other studies were conducted on combining several types of conditions like ellipsoids[12], convex hull[13] and … with computed prediction. They showed that an adequate combination of approximator and representation results in having a more competitive XCSF and conducts it to have superior noise-robust performance[14]. In this study, a new simple low cost method in order to increase XCSF performance and decrease the algorithm error as well as its convergence time is suggested. In the next sections, a brief description of XCSF is given. In the section 3, an improvement on XCSF will be introduced. Section 4 is dedicated to improved algorithm results and discussing them. And at last, conclusion will be presented in section 5. 2. The original XCSF As expressed before XCSF is an extent to eXtended Classifier Systems. It is distinguished from XCS in three respects[6]: i) using continuous integers instead of binary coding in condition part, (this method has also been used for XCSI, too)[15]. ii) A vector of weights ω is used to compute classifier prediction. And at last, iii) the weights ω are updated instead of the classifier prediction. XCSF includes a population of rules. Each rule contains condition part, action part, calculated prediction, classifier error, classifier fitness, classifier experience and its numerosity. Unlike XCS that uses binary genes, XCSF uses two real genes for each input. Forming condition part is done using upper and lower limits of each interval. th Therefore, u i and li (the upper and lower limits of the i interval) are used to bound each interval. Another section of a classifier condition part is weight vector. This vector is used to calculate each classifier prediction. By presenting an example, classifiers which cover the inputs; form the Match set [M]. Each classifier prediction is calculated using linear approximation method, which is brought as below:
h x Z.xc
(1)
Where ω is the weight vector Z0 , Z1 ,..., Zn and xc is the input vector x augmented by a constant x0 , x0 , x1 ,..., xn . In the case of function approximation, the function can be mapped by calculated prediction. So, the action part is omitted according to prediction mapping. Like XCS, prediction array in which each rule could be selected to form Action set [A] with the probability extracted from its prediction, is composed of classifiers with calculated prediction. A difference between XCSF and XCS, which has a great impact on convergence period, is XCSF disability to form subsets in prediction array. This problem causes the algorithm to select just one classifier for Action set [A] formation. In order to overcome this problem, a solution, which is the main idea of this paper, will be presented. Applying the Action set to the environment results a reward for updating the rule properties such as classifier error and fitness. In order to update each classifier error and experience, the following formulas (like in XCS) are used:
exp i
Hi
exp i 1 R Pi H i if exp 1 °°H i i exp i E ® 1 ° H i R Pi H i if exp i t E ¯°
(2)
(3)
363
Omid Saremi et al. / Procedia Computer Science 61 (2015) 361 – 366
Where H i and Pi are the error and calculated prediction of the i classifier, respectively and R is the environment reward. Experience of each classifier is shown by exp i . β is the learning rate of the algorithm. To update each classifier fitness, the formulas mentioned in Equations (4 and 5) are used: th
Ki
1 ° J ® §Hi · E ¨ ¸ ° ¯ © H0 ¹
fi
fi >Ki fi @
if
Hi H0
if
Hi t H0
(4)
(5)
Where, K i is classifier accuracy, H 0 is the initial error. J is accuracy power and f i is the i classifier fitness. If the number of classifiers which form Match set is less than the minimum action size, covering operator will be used to create new classifiers. Condition part of new classifiers is produced as below: th
li
xi rand r0
(6)
ui
xi rand r0
(7)
Zi
rand 1
(8)
xi is the i th input and r0 is a constant integer which is used for covering. The function, rand r0 , picks a random integer from [0, r0 ].
One of the three main differences between XCS and XCSF is updating the weight vector instead of the classifier prediction. This is so important because learning in XCSF is based on prediction approximation and its correction is done by updating weight vector as an element of prediction approximating function. There are different ways to update weight vector ω such as Delta rule, linear least square (LS), recursive least square (RLS) and …[8]. In this paper we used modified delta rule to update weight vector ω. In this method, the Delta rule improved in order to decrease effects of inputs range on weight changes[6]. Updating weights are done as follow:
'Zi
§ K · ¨ ¸ 2 R Pi xi ¨ xc ¸ © ¹
(9)
Zi and xi are the i th component of Z and xc , K is the correction rate, and xc 2 is square of input vector (include x0 ). According to the updating method, overall prediction changes could be calculated as following: 'Pi
'Z.xc K R Pi
size
(10)
Reaching GA threshold, enable Genetic Algorithm to produce new classifiers according to generation gap. Genetic Algorithm is applied to Action set. Number of parent pairs selected from Action set is computed as below:
N pair
GG u N A 2
(11)
Where GG is generation gap, N A and Npair are numbers of Action set population and parent pairs. Mutation in XCSF is done by adding/subtracting r rand m0 to the muted allele in which a random integer picks from [0,m0]
364
Omid Saremi et al. / Procedia Computer Science 61 (2015) 361 – 366
and randomly gets positive or negative sign. m0 is a fix integer. However, XCS mutation is done by changing the allele with 0, 1 and # with the probability of 50%. 3. The Modified XCSF As discussed in aforementioned paragraphs, having a single member Action set cause the convergence time get longer than the case a subset of classifiers are selected to form [A]. Furthermore, GA operator loses its efficiency as it operates on just one single rule (classifier) because both parents are the same. It causes children get the same genes as their parents. So the only difference between them takes place when mutation is applied on an allele. To overcome this problem, an improvement is applied on XCSF to form subsets in prediction array. By selecting a subset in prediction array, more classifiers will be attended in Action set. To reach this goal, a prediction tolerance, Pt is employed to form a subset around the selected classifier in prediction array. The classifier is selected according to inverse of its error, as it suggests a good measure for classifier accuracy. Errors are updated according to classifier prediction of environment rewards. Therefore, a classifier will be more accurate and cause better approximation, if its error is lower in comparison to others with the same experience. In the next step, classifiers that their predictions are in [Pf – Pt, Pf + Pt], form a subset which attend in [A]. Pf is the prediction of selected classifier. 4. Results and Discussion To implement the improvement on function approximation using eXtended Classifier Systems, two versions of XCSF were used. In the first one, initial XCSF which has been introduced by Wilson (2002)[6] is got to work. As mentioned before, rule selection procedure is done by using inverse classifiers error. To have an efficient GA operator, it uses Match set classifiers to apply GA on. The second version that is the improved version uses prediction tolerance to form subsets and probably compose Action set with more than one classifier. In this version, GA operator is applied on Action set [A] like original XCS. The algorithms use examples that were selected randomly from [0,100]. In the other hand, a batch of test data, which its number is 10% of examples number was picked randomly too. Making equal conditions for each version experiments causes better and accurate results. In order to achieve this, examples and test data were the same for all experiments. In the other hands, each of these versions, tests over 10 times to decrease statistical errors. The parameters used in XCSF algorithms are as followings, (according to Butz and Wilson notation[16].) Population size Npop=500, learning rate β=0.2, error threshold H 0 10 , accuracy power γ=5, minimum action size ASmin=7, correction rate η=0.9, generation gap GG=0.1, GA threshold θGA=1009, crossover probability χ=1, mutation probability μ=0.05, deletion threshold θdel=50. In addition: mutation increment m0=20 and covering interval r0=10. For improved version of XCSF, the prediction tolerance was Pt=25. 9 XCSF v.2 XCSF v.1
Error of Test data batch
8 7 6 5 4 3 2 1
0
50
100
150
200
250
300
350
400
Number of epochs
Fig1 Performance history of the original and the improved XCSF (cumulative average error)
In Figure 1, the train trend of each two versions is shown. As mentioned before, each algorithm tests over 10 times; and their Test data batch errors were recorded. To decrease effects of statistical errors, the recorded data was averaged. Each experiment examined through 400 epochs. This procedure makes the comparison reasonable.
Omid Saremi et al. / Procedia Computer Science 61 (2015) 361 – 366
As shown in figure 1, improved version of XCSF that uses prediction tolerance, has better trend. In classical XCSF, after about 40 epochs, the train trend almost stopped while improved one continued its train with adequate trend. Furthermore, according to the train trend slopes, its speed is higher than another one. In the other hand, test data cumulative average error converge about 6 in original XCSF, while it is 1.5 for improved one.
Fig2 Performances of (a) original XCSF and (b) improved XCSF approximating sine function from 10 experiments
To show train results better, function approximation was employed after each experiment for both two algorithms. Selecting fixed points to approximate the mentioned function, helps us to use statistical tools to analyze results. In order to achieve this, the function approximated by each algorithm experiment was recorded. Then, the recorded data got averaged and their variances were calculated. Figure 2 (a & b) shows approximations using original and improved XCSF with their calculated variances, respectively.
Fig3 Performances of (a) original XCSF and (b) improved XCSF approximating sine function from a single run
The approximations and their variances are in good agreement with their train trend results. Bigger variances in each ends of interval in comparison to other parts are reasonable. This is because in middle parts more classifiers got used, so selection probability of classifiers that cover them is higher and consequently training rate is higher too. Two single approximations for both versions of XCSF algorithm are shown in Figure 3.
Fig4 Evolution of the classifier population corresponding to Fig 3a and 3b
365
366
Omid Saremi et al. / Procedia Computer Science 61 (2015) 361 – 366
Figure 4 reports the two populations corresponding to the approximation in Figure 3a and Figure 3b where each classifier is depicted by the segment it represents. As shown in figure 4, population in improved version of XCSF matched better with the function than in original XCSF. In the other hand, the figure expresses that an accurate approximation can be expected from classical XCSF algorithm, if it employs more populations in order to have an accurate classifier for each point. While the function coincident populations in improved XCSF represent that the result will be the same, even if fewer populations are employed. 5. Conclusions An improved Extended Classifier System for Function Approximation (XCSF) with continuous actions was introduced. The proposed algorithm uses pre-defined tolerances (zones) for the output (action) values so that the diversity of rules in a query’s match set is preserved and hence multiple rules could be evaluated per training step. Results from numerous experiments showed that the proposed algorithm outperforms the original classifier system in terms of both accuracy and computational cost. The average error associated with the improved XCSF from several case studies was shown to be less than half the error associated with the original classifier system; and the average number of generations to convergence, using the same stopping criteria, was shown to be 25% less than that of the original algorithm. References 1. 2. 3. 4. 5. 6. 7. 8. 9. 10. 11. 12. 13. 14. 15. 16.
John, H., Holland, Adaptation in natural and artificial systems. 1992, MIT Press, Cambridge, MA. Wilson, S.W., ZCS: A zeroth level classifier system. Evolutionary computation, 1994. 2(1): p. 1-18. Wilson, S.W., Classifier fitness based on accuracy. Evolutionary computation, 1995. 3(2): p. 149-175. Wilson, S.W., Get real! XCS with continuous-valued inputs, in Learning Classifier Systems. 2000, Springer. p. 209-219. Hurst, J., L. Bull, and C. Melhuish, TCS learning classifier system controller on a real robot, in Parallel problem solving from nature—PPSN VII. 2002, Springer. p. 588-597. Wilson, S.W., Classifiers that approximate functions. Natural Computing, 2002. 1(2-3): p. 211-234. Lanzi, P.L., Learning classifier systems: then and now. Evolutionary Intelligence, 2008. 1(1): p. 63-82. Lanzi, P.L., et al., Generalization in the XCSF classifier system: Analysis, improvement, and extension. Evolutionary Computation, 2007. 15(2): p. 133-168. Lanzi, P.L., et al. Prediction update algorithms for XCSF: RLS, kalman filter, and gain adaptation. in Proceedings of the 8th annual conference on Genetic and evolutionary computation. 2006. ACM. Lanzi, P.L., et al. Extending XCSF beyond linear approximation. in Proceedings of the 2005 conference on Genetic and evolutionary computation. 2005. ACM. Lanzi, P.L. and D. Loiacono. Xcsf with neural prediction. in Evolutionary Computation, 2006. CEC 2006. IEEE Congress on. 2006. IEEE. Butz, M.V. Kernel-based, ellipsoidal conditions in the real-valued XCS classifier system. in Proceedings of the 2005 conference on Genetic and evolutionary computation. 2005. ACM. Lanzi, P.L. and S.W. Wilson. Using convex hulls to represent classifier conditions. in Proceedings of the 8th annual conference on Genetic and evolutionary computation. 2006. ACM. Butz, M.V., P.L. Lanzi, and S.W. Wilson, Function approximation with XCS: Hyperellipsoidal conditions, recursive least squares, and compaction. Evolutionary Computation, IEEE Transactions on, 2008. 12(3): p. 355-376. Wilson, S.W., Mining oblique data with XCS, in Advances in Learning Classifier Systems. 2001, Springer. p. 158-174. Butz, M.V. and S.W. Wilson, An algorithmic description of XCS, in Advances in Learning Classifier Systems. 2001, Springer. p. 253-272.
ScienceDirect Procedia Computer Science 61 (2015) 361 – 366
Complex Adaptive Systems, Publication 5 Cihan H. Dagli, Editor in Chief Conference Organized by Missouri University of Science and Technology 2015-San Jose, CA
An Improved Continuous-Action Extended Classifier Systems for Function Approximation Omid Saremia*, Masoud Shariat Panahia, Amin Sabzehzarb b
a Mechanical Engineering Department, University of Tehran, Tehran, Iran Mechanical Engineering Department, University of Nevada, Reno, Nevada, USA
Abstract Due to their structural simplicity and superior generalization capability, Extended Classifier Systems (XCSs) are gaining popularity within the Artificial Intelligence community. In this study an improved XCS with continuous actions is introduced for function approximation purposes. The proposed XCSF uses “prediction zones,” rather than distinct “prediction values,” to enable multi-member match sets that would allow multiple rules to be evaluated per training step. It is shown that this would accelerate the training procedure and reduce the computational cost associated with the training phase. The improved XCSF is also shown to produce more accurate rules than the classical classifier system when it comes to approximating complex nonlinear functions. © 2015 2015The TheAuthors. Authors.Published Published Elsevier © byby Elsevier B.V.B.V. This is an open access article under the CC BY-NC-ND license (http://creativecommons.org/licenses/by-nc-nd/4.0/). Peer-review under responsibility of scientific committee of Missouri University of Science and Technology. Peer-review under responsibility of scientific committee of Missouri University of Science and Technology Keywords: Improved XCSF; Classifier Systems; Function approximation
1. Introduction Learning classifier systems were first introduced by John Holland in 1992[1]. The early variants used a set of randomly generated classifiers (rules) that were gradually evolved into more specialized rules using Genetic Algorithms. In 1994 Wilson introduced ZCS which improved the performance of LCS and reduce its computational cost[2]. In 1995 Wilson proposed a new way to calculate the classifier’s fitness which was a quantitative measure of a rule’s desirability. He presented eXtended Classifier Systems (XCS) in which accuracy plays a major role in
* Corresponding author. Tel.: +98-935-558-2736 E-mail address: [email protected]
1877-0509 © 2015 The Authors. Published by Elsevier B.V. This is an open access article under the CC BY-NC-ND license (http://creativecommons.org/licenses/by-nc-nd/4.0/). Peer-review under responsibility of scientific committee of Missouri University of Science and Technology doi:10.1016/j.procs.2015.09.160
362
Omid Saremi et al. / Procedia Computer Science 61 (2015) 361 – 366
computing classifier fitness[3]. Later studies mostly focused on XCS and its improvements like XCSR[4], TCS[5], and etc. XCSF is one of modified version of XCS, which presented by Wilson in 2002[6]. It evolves classifiers represent piecewise linear approximations of reward that is commonly the problem solution or the function value[7]. It firstly had been used as approximator[6]. Later, Lanzi et al[8, 9] have done some generalizations on XCSF prediction updating method. Some advanced approximators such as polynomials[10], neural networks[11] and … were studied by them. Other studies were conducted on combining several types of conditions like ellipsoids[12], convex hull[13] and … with computed prediction. They showed that an adequate combination of approximator and representation results in having a more competitive XCSF and conducts it to have superior noise-robust performance[14]. In this study, a new simple low cost method in order to increase XCSF performance and decrease the algorithm error as well as its convergence time is suggested. In the next sections, a brief description of XCSF is given. In the section 3, an improvement on XCSF will be introduced. Section 4 is dedicated to improved algorithm results and discussing them. And at last, conclusion will be presented in section 5. 2. The original XCSF As expressed before XCSF is an extent to eXtended Classifier Systems. It is distinguished from XCS in three respects[6]: i) using continuous integers instead of binary coding in condition part, (this method has also been used for XCSI, too)[15]. ii) A vector of weights ω is used to compute classifier prediction. And at last, iii) the weights ω are updated instead of the classifier prediction. XCSF includes a population of rules. Each rule contains condition part, action part, calculated prediction, classifier error, classifier fitness, classifier experience and its numerosity. Unlike XCS that uses binary genes, XCSF uses two real genes for each input. Forming condition part is done using upper and lower limits of each interval. th Therefore, u i and li (the upper and lower limits of the i interval) are used to bound each interval. Another section of a classifier condition part is weight vector. This vector is used to calculate each classifier prediction. By presenting an example, classifiers which cover the inputs; form the Match set [M]. Each classifier prediction is calculated using linear approximation method, which is brought as below:
h x Z.xc
(1)
Where ω is the weight vector Z0 , Z1 ,..., Zn and xc is the input vector x augmented by a constant x0 , x0 , x1 ,..., xn . In the case of function approximation, the function can be mapped by calculated prediction. So, the action part is omitted according to prediction mapping. Like XCS, prediction array in which each rule could be selected to form Action set [A] with the probability extracted from its prediction, is composed of classifiers with calculated prediction. A difference between XCSF and XCS, which has a great impact on convergence period, is XCSF disability to form subsets in prediction array. This problem causes the algorithm to select just one classifier for Action set [A] formation. In order to overcome this problem, a solution, which is the main idea of this paper, will be presented. Applying the Action set to the environment results a reward for updating the rule properties such as classifier error and fitness. In order to update each classifier error and experience, the following formulas (like in XCS) are used:
exp i
Hi
exp i 1 R Pi H i if exp 1 °°H i i exp i E ® 1 ° H i R Pi H i if exp i t E ¯°
(2)
(3)
363
Omid Saremi et al. / Procedia Computer Science 61 (2015) 361 – 366
Where H i and Pi are the error and calculated prediction of the i classifier, respectively and R is the environment reward. Experience of each classifier is shown by exp i . β is the learning rate of the algorithm. To update each classifier fitness, the formulas mentioned in Equations (4 and 5) are used: th
Ki
1 ° J ® §Hi · E ¨ ¸ ° ¯ © H0 ¹
fi
fi >Ki fi @
if
Hi H0
if
Hi t H0
(4)
(5)
Where, K i is classifier accuracy, H 0 is the initial error. J is accuracy power and f i is the i classifier fitness. If the number of classifiers which form Match set is less than the minimum action size, covering operator will be used to create new classifiers. Condition part of new classifiers is produced as below: th
li
xi rand r0
(6)
ui
xi rand r0
(7)
Zi
rand 1
(8)
xi is the i th input and r0 is a constant integer which is used for covering. The function, rand r0 , picks a random integer from [0, r0 ].
One of the three main differences between XCS and XCSF is updating the weight vector instead of the classifier prediction. This is so important because learning in XCSF is based on prediction approximation and its correction is done by updating weight vector as an element of prediction approximating function. There are different ways to update weight vector ω such as Delta rule, linear least square (LS), recursive least square (RLS) and …[8]. In this paper we used modified delta rule to update weight vector ω. In this method, the Delta rule improved in order to decrease effects of inputs range on weight changes[6]. Updating weights are done as follow:
'Zi
§ K · ¨ ¸ 2 R Pi xi ¨ xc ¸ © ¹
(9)
Zi and xi are the i th component of Z and xc , K is the correction rate, and xc 2 is square of input vector (include x0 ). According to the updating method, overall prediction changes could be calculated as following: 'Pi
'Z.xc K R Pi
size
(10)
Reaching GA threshold, enable Genetic Algorithm to produce new classifiers according to generation gap. Genetic Algorithm is applied to Action set. Number of parent pairs selected from Action set is computed as below:
N pair
GG u N A 2
(11)
Where GG is generation gap, N A and Npair are numbers of Action set population and parent pairs. Mutation in XCSF is done by adding/subtracting r rand m0 to the muted allele in which a random integer picks from [0,m0]
364
Omid Saremi et al. / Procedia Computer Science 61 (2015) 361 – 366
and randomly gets positive or negative sign. m0 is a fix integer. However, XCS mutation is done by changing the allele with 0, 1 and # with the probability of 50%. 3. The Modified XCSF As discussed in aforementioned paragraphs, having a single member Action set cause the convergence time get longer than the case a subset of classifiers are selected to form [A]. Furthermore, GA operator loses its efficiency as it operates on just one single rule (classifier) because both parents are the same. It causes children get the same genes as their parents. So the only difference between them takes place when mutation is applied on an allele. To overcome this problem, an improvement is applied on XCSF to form subsets in prediction array. By selecting a subset in prediction array, more classifiers will be attended in Action set. To reach this goal, a prediction tolerance, Pt is employed to form a subset around the selected classifier in prediction array. The classifier is selected according to inverse of its error, as it suggests a good measure for classifier accuracy. Errors are updated according to classifier prediction of environment rewards. Therefore, a classifier will be more accurate and cause better approximation, if its error is lower in comparison to others with the same experience. In the next step, classifiers that their predictions are in [Pf – Pt, Pf + Pt], form a subset which attend in [A]. Pf is the prediction of selected classifier. 4. Results and Discussion To implement the improvement on function approximation using eXtended Classifier Systems, two versions of XCSF were used. In the first one, initial XCSF which has been introduced by Wilson (2002)[6] is got to work. As mentioned before, rule selection procedure is done by using inverse classifiers error. To have an efficient GA operator, it uses Match set classifiers to apply GA on. The second version that is the improved version uses prediction tolerance to form subsets and probably compose Action set with more than one classifier. In this version, GA operator is applied on Action set [A] like original XCS. The algorithms use examples that were selected randomly from [0,100]. In the other hand, a batch of test data, which its number is 10% of examples number was picked randomly too. Making equal conditions for each version experiments causes better and accurate results. In order to achieve this, examples and test data were the same for all experiments. In the other hands, each of these versions, tests over 10 times to decrease statistical errors. The parameters used in XCSF algorithms are as followings, (according to Butz and Wilson notation[16].) Population size Npop=500, learning rate β=0.2, error threshold H 0 10 , accuracy power γ=5, minimum action size ASmin=7, correction rate η=0.9, generation gap GG=0.1, GA threshold θGA=1009, crossover probability χ=1, mutation probability μ=0.05, deletion threshold θdel=50. In addition: mutation increment m0=20 and covering interval r0=10. For improved version of XCSF, the prediction tolerance was Pt=25. 9 XCSF v.2 XCSF v.1
Error of Test data batch
8 7 6 5 4 3 2 1
0
50
100
150
200
250
300
350
400
Number of epochs
Fig1 Performance history of the original and the improved XCSF (cumulative average error)
In Figure 1, the train trend of each two versions is shown. As mentioned before, each algorithm tests over 10 times; and their Test data batch errors were recorded. To decrease effects of statistical errors, the recorded data was averaged. Each experiment examined through 400 epochs. This procedure makes the comparison reasonable.
Omid Saremi et al. / Procedia Computer Science 61 (2015) 361 – 366
As shown in figure 1, improved version of XCSF that uses prediction tolerance, has better trend. In classical XCSF, after about 40 epochs, the train trend almost stopped while improved one continued its train with adequate trend. Furthermore, according to the train trend slopes, its speed is higher than another one. In the other hand, test data cumulative average error converge about 6 in original XCSF, while it is 1.5 for improved one.
Fig2 Performances of (a) original XCSF and (b) improved XCSF approximating sine function from 10 experiments
To show train results better, function approximation was employed after each experiment for both two algorithms. Selecting fixed points to approximate the mentioned function, helps us to use statistical tools to analyze results. In order to achieve this, the function approximated by each algorithm experiment was recorded. Then, the recorded data got averaged and their variances were calculated. Figure 2 (a & b) shows approximations using original and improved XCSF with their calculated variances, respectively.
Fig3 Performances of (a) original XCSF and (b) improved XCSF approximating sine function from a single run
The approximations and their variances are in good agreement with their train trend results. Bigger variances in each ends of interval in comparison to other parts are reasonable. This is because in middle parts more classifiers got used, so selection probability of classifiers that cover them is higher and consequently training rate is higher too. Two single approximations for both versions of XCSF algorithm are shown in Figure 3.
Fig4 Evolution of the classifier population corresponding to Fig 3a and 3b
365
366
Omid Saremi et al. / Procedia Computer Science 61 (2015) 361 – 366
Figure 4 reports the two populations corresponding to the approximation in Figure 3a and Figure 3b where each classifier is depicted by the segment it represents. As shown in figure 4, population in improved version of XCSF matched better with the function than in original XCSF. In the other hand, the figure expresses that an accurate approximation can be expected from classical XCSF algorithm, if it employs more populations in order to have an accurate classifier for each point. While the function coincident populations in improved XCSF represent that the result will be the same, even if fewer populations are employed. 5. Conclusions An improved Extended Classifier System for Function Approximation (XCSF) with continuous actions was introduced. The proposed algorithm uses pre-defined tolerances (zones) for the output (action) values so that the diversity of rules in a query’s match set is preserved and hence multiple rules could be evaluated per training step. Results from numerous experiments showed that the proposed algorithm outperforms the original classifier system in terms of both accuracy and computational cost. The average error associated with the improved XCSF from several case studies was shown to be less than half the error associated with the original classifier system; and the average number of generations to convergence, using the same stopping criteria, was shown to be 25% less than that of the original algorithm. References 1. 2. 3. 4. 5. 6. 7. 8. 9. 10. 11. 12. 13. 14. 15. 16.
John, H., Holland, Adaptation in natural and artificial systems. 1992, MIT Press, Cambridge, MA. Wilson, S.W., ZCS: A zeroth level classifier system. Evolutionary computation, 1994. 2(1): p. 1-18. Wilson, S.W., Classifier fitness based on accuracy. Evolutionary computation, 1995. 3(2): p. 149-175. Wilson, S.W., Get real! XCS with continuous-valued inputs, in Learning Classifier Systems. 2000, Springer. p. 209-219. Hurst, J., L. Bull, and C. Melhuish, TCS learning classifier system controller on a real robot, in Parallel problem solving from nature—PPSN VII. 2002, Springer. p. 588-597. Wilson, S.W., Classifiers that approximate functions. Natural Computing, 2002. 1(2-3): p. 211-234. Lanzi, P.L., Learning classifier systems: then and now. Evolutionary Intelligence, 2008. 1(1): p. 63-82. Lanzi, P.L., et al., Generalization in the XCSF classifier system: Analysis, improvement, and extension. Evolutionary Computation, 2007. 15(2): p. 133-168. Lanzi, P.L., et al. Prediction update algorithms for XCSF: RLS, kalman filter, and gain adaptation. in Proceedings of the 8th annual conference on Genetic and evolutionary computation. 2006. ACM. Lanzi, P.L., et al. Extending XCSF beyond linear approximation. in Proceedings of the 2005 conference on Genetic and evolutionary computation. 2005. ACM. Lanzi, P.L. and D. Loiacono. Xcsf with neural prediction. in Evolutionary Computation, 2006. CEC 2006. IEEE Congress on. 2006. IEEE. Butz, M.V. Kernel-based, ellipsoidal conditions in the real-valued XCS classifier system. in Proceedings of the 2005 conference on Genetic and evolutionary computation. 2005. ACM. Lanzi, P.L. and S.W. Wilson. Using convex hulls to represent classifier conditions. in Proceedings of the 8th annual conference on Genetic and evolutionary computation. 2006. ACM. Butz, M.V., P.L. Lanzi, and S.W. Wilson, Function approximation with XCS: Hyperellipsoidal conditions, recursive least squares, and compaction. Evolutionary Computation, IEEE Transactions on, 2008. 12(3): p. 355-376. Wilson, S.W., Mining oblique data with XCS, in Advances in Learning Classifier Systems. 2001, Springer. p. 158-174. Butz, M.V. and S.W. Wilson, An algorithmic description of XCS, in Advances in Learning Classifier Systems. 2001, Springer. p. 253-272.
