New Encoding/Converting Methods of Binary GA ... - Semantic Scholar
IEICE TRANS. FUNDAMENTALS, VOL.E88–A, NO.6 JUNE 2005
1554
PAPER
New Encoding /Converting Methods of Binary GA/Real-Coded GA Jong-Wook KIM†a) and Sang Woo KIM†† , Members
SUMMARY This paper presents new encoding methods for the binary genetic algorithm (BGA) and new converting methods for the real-coded genetic algorithm (RCGA). These methods are developed for the specific case in which some parameters have to be searched in wide ranges since their actual values are not known. The oversampling effect which occurs at large values in the wide range search are reduced by adjustment of resolutions in mantissa and exponent of real numbers mapped by BGA. Owing to an intrinsic similarity in chromosomal operations, the proposed encoding methods are also applied to RCGA with remapping (converting as named above) from real numbers generated in RCGA. A simple probabilistic analysis and benchmark with two ill-scaled test functions are carried out. System identification of a simple electrical circuit is also undertaken to testify effectiveness of the proposed methods to real world problems. All the optimization results show that the proposed encoding/converting methods are more suitable for problems with ill-scaled parameters or wide parameter ranges for searching. key words: binary genetic algorithm, real-coded genetic algorithm, illscaled function, system identification, encoding
1.
Introduction
System identification is the field of mathematical modelling of systems from experimental data, and is categorized into nonparametric and parametric methods according to model structures [1]. In the case the model structure is considerably complex or largely scaled like power transformers, nonparametric identification is deemed to be suitable. Nonparametric identification is characterized by the property that the resulting models are curves or functions, and its good example is the spectral analysis for the diagnosis of a power transformer [2]. On the other hand, parametric identification is favorable to a system whose model structure is simply described with mathematical static and/or dynamic equations. Most researches in parametric identification area focus on discrete-time systems due to the easiness and simplicity of required mathematics. However, the performance of discrete-time identification depends on the sampling time, and the obtained parameters give little physical insight into the system. Therefore, in the case of diagnosis or modeling, transformation of parameters in a discrete-time system Manuscript received March 17, 2004. Manuscript revised October 24, 2004. Final manuscript received March 4, 2005. † The author is with the Electrical Steel Sheet Research Group, POSCO Technical Research Laboratories, Pohang 790-785, Korea. †† The author is with the Electrical and Computer Engineering Division, Pohang University of Science and Technology (POSTECH), Pohang, Korea. a) E-mail: [email protected] DOI: 10.1093/ietfec/e88–a.6.1554
into those in a continuous-time system is required, which is quite cumbersome or even impossible as system dimension is raised. To this end, continuous-time prediction error method free from parameter transformation was provided, which estimates the system parameters by minimizing prediction errors with exact derivatives of the objective function with respect to the adjustable parameters [3]. However, the prediction error method requires a good guess of initial parameters, and update rule design is system dependent. The genetic algorithm whose chromosome is formed with model parameters and initial states is applied to the parametric identification of a simple electrical circuit [4] and an induction motor [5]. These systems contain physical electrical elements such as resistors, capacitors, and inductors whose values range from micro to kilo units. If actual values of each element are known a priori, which is unrealistic, parameter bounds for search with GA can be set up moderately. For the real systems however it is difficult to determine proper bounds owing to time variance of actual values or inaccuracy of name plate values. Therefore, the parameter bounds have to be selected wide enough not to exclude global minima prior to executing GA, while it is widely acknowledged that as parameter space grows wider optimization performance degenerates as much. The delta coding is an efficient alternative for this problem, but uses a multistage strategy where the global minima can be discarded at an initial search phase [6]. This paper proposes new encoding methods of BGA specifically designed to overcome the wide bound search problem. As search space grows wider, the general linear sampling (or linear encoding) of BGA suffers from inevitable unbalance between small and large numbers owing to the limited chromosome length, i.e., resolution of real numbers. In this paper logarithmic and hybrid encoding methods for search space are developed by modifying the conventional encoding (or decoding, equivalently) function of BGA. These changes in phenotypes enlarge the small numbers often ignored and reduce large numbers overly estimated both in linear sampling. Furthermore, the proposed encoding methods include the linear encoding method such that for narrow search space the effects of all the encoding methods are almost identical. Since the electrical circuit elements addressed in this paper are assigned positive real values, RCGA is more efficient than BGA in terms of search space resolution, run time, simplicity, convenience of program coding, and so on. RCGA has strings of real numbers instead of binary chro-
c 2005 The Institute of Electronics, Information and Communication Engineers Copyright
KIM and KIM: NEW ENCODING /CONVERTING METHODS OF BINARY GA/REAL-CODED GA
1555
mosomes of BGA, and takes three similar operators for real numbers; reproduction, crossover, and mutation operators. Hence it is assumed that RCGA and BGA will have similar optimization aspects under the expansion of search space. To verify this postulate, strings of RCGA are reprocessed (or converted as termed in this paper) in accordance with the logarithmic and hybrid encoding methods of BGA. For an investigation of these changes, a simple probabilistic analysis is made for logarithmic effects on the expansion of search space, and benchmark is undertaken on two ill-scaled test functions. In addition, as the first step to continuous-time parametric identification, a simple series-connected electrical circuit is identified with simulated input and output data. Improvement of optimization results assures that the proposed methods in GAs are effective for wide bound search or ill-scaled problems. This paper is organized as follows: Sect. 2 describes some details of BGA and RCGA. Section 3 explains the proposed encoding methods with a simple analysis on linear and logarithmic search spaces under space expansion. Section 4 provides performance comparison of benchmark and system identification. Section 5 concludes the work with comments on future works. 2.
BGA and RCGA
GA has three operators of reproduction, crossover, and mutation. Reproduction is devised to inherit superior individuals to the next generation by the fitness-based selection rule. This operator is a simulation of natural selection. In the crossover, two individuals selected randomly in the present population exchange their bits after random crossing sites. During the mutation, every bit is assigned an equal probability to be replaced by its complement digit. This is an imitation of natural mutation, and is very rare in GA processes as well. In BGA and RCGA, some similarities and differences exist as is written in the following subsections. 2.1 Representation and Crossover BGA has chromosomes made of binary digits 0 and 1, while RCGA has strings of real numbers. For instance, if the total number of parameters is k and if l bits in particular for BGA are assigned to every parameter, chromosomes are illustrated as follows: l
l
l
p1
p2
pk
s1b = (b111 · · · b11l ) · · · (b1i1 · ·b1im | b1i(m+1) · ·b1il ) · · · (b1k1 · · · b1kl ) s2b = (b211 · · · b21l ) · · · (b2i1 · ·b2im | b2i(m+1) · ·b2il ) · · · (b2k1 · · · b2kl ) ⇓ s1b = (b111 · · · b11l ) · · · (b1i1 · ·b1im | b2i(m+1) · ·b2il ) · · · (b2k1 · · · b2kl )
s2b = (b211 · · · b21l ) · · · (b2i1 · ·b2im | b1i(m+1) · ·b1il ) · · · (b1k1 · · · b1kl ), where bim ∈ {0, 1} is an m-th binary bit in an i-th substring. Since chromosome representation in RCGA is different from that of BGA, the crossover operation has to be modified. Up till now, many kinds of crossover operators, including simple crossover, arithmetical crossover [7], BLXα crossover, and modified simple crossover [8], have been suggested. Among them, the most similar operator with BGA is the modified simple crossover described as follows: 1 · · · rk1 ) s1r = (r11 · · · ri1 | ri+1 2 s2r = (r12 · · · ri2 | ri+1 · · · rk2 ) ⇓ 1 1 s = (r · · · r1 | r2 · · · r2 ) r
s2r
p1
p2
pk
where sb and sr represent concatenated strings of BGA and RCGA, respectively. Let s1b and s2b are selected parents for crossover to pro duce their offsprings s1b and s2b . After crossover, a change occurs as follows:
=
(r12
i
i+1
k
1 · · · ri2 | ri+1 · · · rk1 )
where ri1 = λc1 ri1 + (1 − λc1 )ri2 r2 = λ r2 + (1 − λ )r1 i
c2 i
c2
i
ri ∈ R represents an i-th parameter, and λc1 , λc2 denote uniformly distributed random numbers between 0 and 1. 2.2 Mutation Mutation is a unary operator which change genes in chromosomes. The mechanism of uniform mutation in an i-th substring of BGA is illustrated as follows: (bi1 · · · bim · · · bil ) ⇒ (bi1 · · · bim · · · bil ), where bim represents a flipped bit of bim , i.e., from 0 to 1 or from 1 to 0. Unlike BGA, various mutation operators have been provided in RCGA such as uniform mutation [7], real number creep mutation [9], dynamic mutation [10], M¨uhlenbein mutation [11], and so on. The analogous operator to BGA is the uniform mutation by which a selected real-valued gene is transformed randomly within its upper and lower bounds. The principle of the uniform mutation is illustrated as
· · · 01 10 · · · 10 · · · 00 · · · 01) sb = (11 sr = (15.279 0.858 · · · 296.99),
1
(r1 · · · r j · · · rk ) ⇒ (r1 · · · r j · · · rk ) r j = λm (rUj − r Lj ) + r Lj
(1)
where r j ∈ [r Lj , rUj ] is the mutated j-th gene from r j , and λm ∈ [0, 1] is the random number. 3.
Search Space
3.1 Encoding Methods of BGA The general decoding function which linearly transforms an
IEICE TRANS. FUNDAMENTALS, VOL.E88–A, NO.6 JUNE 2005
1556
l-bit-long binary string (a1 · · · al ) into a bounded real number x ∈ [xL , xU ] is written as x=
l−1 xU − x L a(l−i) 2i + xL 2l − 1 i=0
(2)
where a1 is a most significant bit (MSB) and al is a least significant bit (LSB). Specifically, (xU − xL )/(2l − 1) in (2) plays a role as search resolution or precision. To avoid terminological ambiguity hereafter, definitions are stated for each type of sampled space. The first is linear space. Definition 1 (Linear Space): The n-dimensional discretized linear space with finite resolution δL is defined as: L : {x | x = (x1 · · · xn )T ∈ Rn } xi = xiL + kδL , i = 1, · · · , n U xi − xiL k∈ 0, 1, · · · , , δL
(3)
L where xU i and xi are predetermined upper and lower bounds of xi , respectively.
The followings are the proposed encoding methods of BGA with corresponding definitions. One is exponential encoding, which transforms a binary string (b1 · · · bl ) into an exponent of 10 as y = 10
eU −eL 2l −1
(
l−1 i=0
b(l−i) 2i )+eL
,
(4)
where eL = log10 yL and eU = log10 yU . In (4), y is regarded as equidistant in terms of logarithmic scale. Thus, the corresponding space is denominated logarithmic space rather than exponential space. Definition 2 (Logarithmic Space): An n-dimensional discretized logarithmic space with finite resolution δG is defined as: G : {y | y = (y1 · · · yn )T ∈ Rn } yi = 10ei +kδG , i = 1, · · · , n U ei − eiL k∈ 0, 1, · · · , δG L
U L L where eU i = log10 yi and ei = log10 yi are the exponents of upper and lower bounds of yi , respectively.
The other is hybrid encoding, which is also a common way to represent floating-point numbers by a fixed-length memory. To implement the hybrid encoding method, a binary string must be partitioned into two parts; mantissa and exponent. If a majority of bits are assigned to a mantissa, this encoding turns closely into a linear encoding. On the other hand, if almost all bits are referred to as those for an exponent, this is approximately exponential encoding. Therefore, we conjecture that the hybrid encoding lies between linear and logarithmic encodings depending on the bit ratio of mantissa and exponent.
Suppose we assign p and q bits for the exponent and the mantissa of a real number, respectively, and a binary string is described as (c1 · · · c p d1 · · · dq ), whose total length is l(= p + q). The hybrid encoding is described as z = mant × 10expo , p−1 eU − e L c(p−i) 2i + eL , δe = expo = δe 2p i=0 mant = δm
q−1
d(q− j) 2 j + 1, δm =
j=0
10δe − 1 2q
(5) (6)
(7)
where δe and δm are the resolutions of exponent and mantissa, respectively, that are devised to avoid overlapping of decoding. The details of how δe and δm are calculated are provided in Appendix. The hybrid space is defined in accordance with (5)–(7). Definition 3 (Hybrid Space): An n-dimensional discretized hybrid space with finite resolution vector δH = (δm δe )T is defined as: H : {z | z = (z1 · · · zn )T ∈ Rn } L zi = (1 + lδm ) × 10ei +kδe , i = 1, · · · , n U ei − eiL k∈ 0, 1, · · · , δe − 1 δe 10 − 1 − 1 , j = k max(l) + l l ∈ 0, 1, · · · , δm U L L where eU i = log10 zi and ei = log10 zi are the exponents of upper and lower bounds of zi , respectively.
3.2 Converting Methods of RCGA Since individuals of RCGA consist of real numbers unlike BGA, the conventional RCGAs require no encoding methods. However, we apply in parallel to RCGA the concepts of the linear, exponential, and hybrid encoding methods based on the intrinsic similarities between BGA and RCGA. A new term for ‘encoding’ is denominated ‘conversion’ for RCGA. Then, except the linear conversion, exponential and hybrid conversion are accomplished using converting functions imitating (4)–(7). The exponential conversion of RCGA is described as yi = 10ri , ri ∈ [log10 yiL , log10 yU i ]
(8)
where ri is an i-th gene of RCGA, and the hybrid conversion for ri is written as zi = (9hi + 1)10ni , hi = ri − ni , ni = ri , 0 ≤ hi < 1.
(9) (10)
As shown in (8)–(10), converting functions of RCGA are considerably simple compared with the encoding functions of BGA.
KIM and KIM: NEW ENCODING /CONVERTING METHODS OF BINARY GA/REAL-CODED GA
1557
Fig. 1
Relative area of crest set C.
=
3.3 Probabilistic Analysis In searching for a whole solution space whose dimension equals the number of parameters, both the GAs have finite space resolution. In BGA, the resolution depends on the number of bits per parameter, while in RCGA it depends on the resolution of random numbers during crossover and mutation. Therefore, it can be regarded that search space is divided by a finite number of grids, whose amount remains constant irrespective of the size of search space. For a brief geometric analysis, we define the level set C as C : {θ | F(θ) ≥ F0 , θ ∈ Rn }
(11)
where θ is the n-dimensional parameter vector, F(·) is the fitness function, and F0 is the constant fitness value. Note that a sufficiently large F0 makes C a single and closed set even though a multimodal optimization problem is addressed. For an explanation of space expansion, Fig. 1(a) illustrates an optimization problem, whose parameters are p11 ∈ [p11min , p11max ] and p12 ∈ [p12min , p12max ] in a feasible space S1 . Let the only upper bounds of each parameter be expanded by factors of k1 and k2 , respectively. Then new bounds are described as p21max = k1 p11max , p21min = p11min p22max = k2 p12max , p22min = p12min
(12)
where superscripts 1 and 2 represent ‘before expansion’ and ‘after expansion,’ respectively. Fig. 1(b) shows an expanded solution space S2 , where it is notable that despite the expansion of the feasible space the crest set C remains the same. Thus, the relative portion of C decreases for S2 , and this contraction may cause difficulty in exploring solutions inside C in terms of probability. A relative probability between the two cases, i.e. before and after expansion, is computed for a 2-dimensional problem as P2 (C) A(C)/A(S 2 ) A(S 1 ) = = P1 (C) A(C)/A(S 1 ) A(S 2 )
(p11max − p11min )(p12max − p12min )
(13)
(p21max − p21min )(p22max − p22min )
where P1 (C) and P2 (C) represent the probabilities of selecting solutions inside C for corresponding search spaces S1 and S2 , respectively, and A(·) stands for area. In the case of p11max p11min and p12max p12min , (13) is simplified as (p11max − p11min )(p12max − p12min ) P2 (C) = P1 (C) k1 k2 (p11max − k11 p11min )(p12max − k12 p12min ) 1 . (14) k1 k2 Thus in an n-dimensional problem (13) can be generalized as 1 P2 (C) ≈ . (15) P1 (C) k1 k2 · · · kn ≈
This result indicates that the linear scale expansion of upper bounds in each parameter can cause performance degradation of the random search during the initialization and reproduction operations of GA. However, owing to the intrinsic mechanism of survival of the fittest, the overall performance of GA is not deteriorated with as much ratio as (15). To analyze the relative probability (13) for the logarithmic space, the lower and upper bounds in (12) are interpreted on a logarithmic scale as 1 1L p2L 1max = log k1 + log p1max = log k1 + p1max 2L 1 1L p1min = log p1min = p1min 1 1L p2L 2max = log k2 + log p2max = log k2 + p2max 1 1L p2L 2min = log p2min = p2min
(16)
where the superscript L stands for a logarithmic opeartion. The relative probability on logarithmic space is computed in a similar manner as 1L 1L 1L P2 (C L ) (p1L 1max − p1min )(p2max − p2min ) = 2L 2L 2L P1 (C L ) (p2L 1max − p1min )(p2max − p2min )
=
1L 1L 1L (p1L 1max − p1min )(p2max − p2min ) 1L 1L 1L (log k1 + p1L 1max − p1min )(log k2 + p2max − p2min )
. (17)
IEICE TRANS. FUNDAMENTALS, VOL.E88–A, NO.6 JUNE 2005
1558
For a more intuitive simplification, if log-scaled expansion ratios are defined as m1 =
log k1 log k2 , m2 = 1L 1L − p1min p2max − p1L 2min
(18)
p1L 1max
then (17) is simplified as P2 (C L ) 1 = . P1 (C L ) (m1 + 1)(m2 + 1)
(19)
For a simple comparison of (15) and (19) suppose that p1 and p2 were presumed to exist between 1 and 100 and 0.001 and 0.1, respectively, but search results show that those ranges fail to include acceptable solutions. As the next step the upper bounds are expanded simultaneously by a factor of 100, which leads the relative probability to enormous decrease of about 1/10000 for linear space, while only a slight decrease of 1/4 for logarithmic space. Thus, logarithmic space is far more reasonable for searching a wide solution space. 4.
converges to a global optimum, while the modified versions of maintaining the best solution in the population do. Therefore the adaptive GA (AGA) scheme [19] is adopted for BGA in this paper. GA parameters are set as • • • • • • • • • • • • •
maximum generation = 2000 number of variables = 2 population size = 50 number of restarts = 50 selection: Roulette wheel selection bit length of a substring in BGA = 13 bit ratio of exponent and mantissa in hybrid encoding of BGA = 4/9 parameters of adaptive BGA : k1 = 0.85, k2 = 0.5, k3 = 1.0, k4 = 0.05 crossover of BGA: one-point crossover crossover of RCGA: modified simple crossover mutation of BGA and RCGA: uniform mutation crossover rate of RCGA = 0.95 mutation rate of RCGA = 0.01.
For a performance comparison, the following measures are used:
Experimental Results
4.1 Test Functions and Performance Measure After developing an optimization method, one has to test it on a suite of benchmark functions at a validation step in accordance with the ‘no-free-lunch’ theorem [12]. In the literature of evolutionary computation [13]–[15], 14 or 23 test functions, varying from unimodal to multimodal and from low- to high-dimensional cost functions, are provided with their global optima acknoledged. However, the test functions are confined in well-scaled search space, i.e., their bounds are of similar order. Therefore, badly scaled functions are necessary for the specific optimization problem addressed in this paper. This paper adopts the following functions for test of the proposed encoding/converting methods: • Brown badly scaled function:
(20)
• Powell badly scaled function: f p (x1 , x2 ) = (104 x1 x2 − 1)2 + (exp[−x1 ] + exp[−x2 ] − 1.0001)2
In the above measures, LM (logarithmic mean) represents exponent averaging. That is, the LM of x = (x1 x2 · · · xm )T is described as LM(x) = 10
−6 2
fb (x1 , x2 ) = (x1 − 10 ) + (x2 − 2 × 10 ) + (x1 x2 − 2)2 6 2
a) MEAN BEST COST: mean of the best costs obtained at each restart. b) STD DEV: standard deviation of the best costs obtained at each restart. c) LM BEST COST: logarithmic mean of the best costs. d) FINAL BEST COST: best of the best costs obtained at each restart. e) MEAN BEST SOL: mean of the best solutions obtained at each restart. f) LM BEST SOL: logarithmic mean of each best solution. g) FINAL BEST SOL: solution for d).
(21)
The global minima of (20) and (21) are known to be exactly and almost 0 at the points (x1 , x2 ) = (106 , 2 × 10−6 ) and (1.098 · · · 10−5 , 9.106 · · · ), respectively. Search is undertaken within [10−8 , 108 ] for (20) and [10−8 , 104 ] for (21), whose upper bounds are huge values for general optimization problems. Rudolph [18] has shown that the classical BGA never
m log10 xi i=1 m
(22)
which is quite reasonable for averaging real numbers whose magnitudes are different in their exponents. For example, the general averaging ignores such a small number as 10−8 in presence of a relatively large number of 105 , but the LM balances the two numbers with their exponents. Thus, LM is thought to be a more reliable medium for a set of numbers with large variance. Table 1 gives performance comparison of the proposed encoding/converting methods on the Brown function for each solution space. Note in the case of RCGA that MEAN BEST COST values are quite different from LM BEST COST values owing to an ill-scaled property of attained cost values. This discrepancy increases as GA finds more numbers of bad solutions of high cost values. In linear solution space, both GAs fail to find out a
KIM and KIM: NEW ENCODING /CONVERTING METHODS OF BINARY GA/REAL-CODED GA
1559
GA
SPACE
LINEAR LOG HYBRID LINEAR RCGA LOG HYBRID BGA
Table 1 Performance comparison of BGA and RCGA on linear, logarithmic, and hybrid search space for the Brown function. MEAN STD LM FINAL MEAN BEST SOL LM BEST SOL BEST COST DEV BEST COST BEST COST x1 x2 x1 x2 1.92×108 4.33×108 5.57×106 1.21×106 9.95×105 1.00×10−8 9.94×105 1.00×10−8 7.63×106 7.61×106 2.05×106 3.16×105 1.00×106 1.96×10−6 1.00×106 1.94×10−6 2.53×106 1.19×106 9.13×104 3.81×10−6 9.99×105 1.66×10−6 9.99×105 1.57×10−6 6.80×1022 4.30×1023 5.36×1012 3.96 3.28×105 1.79×105 7.05×10−4 9.21 5 6 −1 −11 4.97×10 2.52×10 3.21×10 8.54×10 1.00×106 6.62×10−6 1.00×106 2.44×10−6 4.84×106 2.80×107 1.12×10−1 2.67×10−13 9.99×105 4.05×10−5 9.99×105 4.97×10−6
FINAL BEST SOL x1 x2 1.00×106 1.00×10−8 9.99×105 2.00×10−6 1.00×106 2.00×10−6 1.00×106 1.00×10−8 1.00×106 2.00×10−6 1.00×106 2.00×10−6
Table 2 Performance comparison of BGA and RCGA on linear, logarithmic, and hybrid search space for the Powell function. GA
SPACE
MEAN BEST COST
STD DEV
LM BEST COST
FINAL BEST COST
BGA
LINEAR LOG HYBRID
3.45×10−3 4.94×10−6 2.02×10−6
1.26×10−2 5.59×10−6 3.58×10−6
9.27×10−6 2.13×10−7 3.64×10−7
1.22×10−6 2.60×10−10 4.17×10−9
6.88×103 7.92×102 1.20×103
2.92×103 1.18×103 7.15×102
2.48 5.31×10−4 1.56×10−2
3.95 ×10−5 1.88×10−1 6.39×10−3
1.00×10−8 1.11×10−5 1.00×10−5
1.00×104 8.97 1.00×101
RCGA
LINEAR LOG HYBRID
8.99×1021 3.82×10−3 1.19×10−3
3.73×1022 2.12×10−2 4.68×10−3
3.73×104 4.49×10−7 1.16×10−7
1.00×10−8 4.85×10−9 2.41×10−11
3.89×103 1.97×101 1.43×102
1.29×104 1.26×101 6.87
2.00×10−1 1.93×10−3 5.91×10−4
8.97×10−1 5.20×10−2 1.70×10−1
1.00×10−8 1.01×101 1.10×10−5
1.00×104 9.88×10−6 9.06
true value x2 , 2 × 10−6 , and just approximate it as 0 because of limited resolution around such a small value. However, in logarithmic and hybrid search spaces, the precise value of x2 is obtained, which shows the effectiveness of the proposed encdong/converting methods for abnormally small and large numbers. Moreover, RCGA searches for better solutions than BGA on logarithmic and hybrid spaces. As is evidenced in the literature, RCGA, which has higher search resolution, is more suitable for nonlinear optimization problems, and the data in Table 1 verifies this result. Table 2 provides search results of the Powell function under the same search condition with Table 1. Unlike the Brown function, the Powell function appears to contain local minima at (x1 , x2 ) = (10.1, 9.88 × 10−6 ) as shown in logarithmic space with RCGA. The diversity of LM BEST SOL values also demonstrates the ruggedness of this function. Despite this difficulty, the proposed methods have found acceptable solutions among which a global minimum is (x1 , x2 ) = (1.1035 × 10−5 , 9.0624) with the cost of 2.41 × 10−11 found in hybrid space by RCGA. 4.2 System Identification of a Simple Electrical Circuit The main concern of this paper is to identify a simple electrical circuit whose elements are a resistor with resistance R, an inductor with inductance L, and a capacitor with capacitance C, as shown in Fig. 2. The state space and output equations of this system are given as 1 R x˙1 − L1 x1 − = 1L + L u (23) 0 x2 x˙2 0 C x1 1 0 (24) y= x2 where x1 is input current and x2 is capacitor voltage. Because initial state is necessary for determining unique states at each time, parameter vector in (11) is determined as
MEAN BEST SOL x1 x2
LM BEST SOL x1 x2
Fig. 2
FINAL BEST SOL x1 x2
Simple RLC circuit.
Table 3 Characteristics of PRBS. Nr p Ts N 6 17 100 × 10−6 1500
θ = (R L C x1 (0) x2 (0))T . The fitness function in GAs is designed to minimize the sum squared estimation error as F(θ) =
1 2
1 2 i=1 {y(θ) − y(θ0 )}
N
(25)
where N is the number of data points and the true parameter vector is θ0 = (1.22 × 101 9.60 × 10−2 5.00 × 10−5 7.28 × 10−4 2.98 × 10−1 )T . Input signal used for system identification is PRBS (Pseudo-Random Binary Signal) [1], and its characteristic parameters are given in Table 3 where Nr is the number of shifting registers, p is the frequency divider, and T s is the sampling period in seconds. GA parameters are the same with those of the test functions with the exceptions of • maximum generation = 5000 • number of parameters = 5
IEICE TRANS. FUNDAMENTALS, VOL.E88–A, NO.6 JUNE 2005
1560 Table 4 Comparison of cost values between BGA and RCGA on linear, logarithmic, and hybrid search space with respect to narrow, medium, and wide parameter ranges. BGA
RCGA
SPACE
BOUND
MEAN BEST COST
STD DEV
LM BEST COST
FINAL BEST COST
MEAN BEST COST
STD DEV
LM BEST COST
FINAL BEST COST
LINEAR
NARROW MEDIUM WIDE
2.81×10−4 2.30×10−3 9.37×10−3
2.45×10−5 2.05×10−3 9.11×10−3
1.44×10−4 1.22×10−3 4.55×10−3
1.98×10−6 4.63×10−5 4.69×10−5
2.86×10−4 7.08×10−2 8.89×10−2
2.71×10−4 6.52×10−3 1.08×10−3
1.83×10−4 7.05×10−2 8.89×10−2
1.25×10−5 4.55×10−2 8.42×10−2
LOG
NARROW MEDIUM WIDE
3.11×10−4 1.86×10−3 1.98×10−3
3.55×10−4 3.35×10−3 3.79×10−3
1.05×10−4 3.96×10−4 4.10×10−4
3.19×10−6 4.84×10−9 6.96×10−6
3.51×10−2 3.87×10−4 2.17×10−4
3.78×10−2 6.26×10−4 4.06×10−4
6.03×10−3 3.90×10−5 7.56×10−5
1.16×10−4 1.38×10−8 3.39×10−7
HYBRID
NARROW MEDIUM WIDE
1.97×10−4 2.18×10−3 2.49×10−3
2.91×10−4 2.88×10−3 3.72×10−3
4.31×10−5 6.16×10−4 4.70×10−4
4.40×10−8 8.35×10−6 1.41×10−5
5.22×10−4 3.07×10−4 4.47×10−4
6.08×10−4 3.79×10−4 5.12×10−4
2.60×10−4 1.14×10−4 2.02×10−4
4.72×10−5 1.08×10−6 1.91×10−6
Table 5 Comparison of obtained solutions between BGA and RCGA on linear, logarithmic, and hybrid search space with respect to narrow, medium, and wide parameter ranges. The true values are R=12.20 (Ω), L=96.00 (mH), and C=50.00 (µF). SPACE
BOUND
NARROW MEDIUM WIDE NARROW LOG MEDIUM WIDE NARROW HYBRID MEDIUM WIDE LINEAR
BGA R (Ω) L (mH) LM BEST LM BEST 12.90 12.21 94.90 96.53 12.71 12.22 94.49 98.57 14.25 11.99 94.91 93.78 12.23 12.20 93.08 95.71 12.94 12.20 103.5 95.98 12.33 12.18 101.83 95.12 12.16 12.20 93.51 96.07 12.74 12.24 98.63 94.87 12.57 12.18 105.61 94.66
RCGA C (µF) R (Ω) L (mH) C (µF) LM BEST LM BEST LM BEST LM BEST 50.63 49.72 12.38 12.20 90.30 94.52 53.46 50.79 51.38 48.81 95.67 48.95 145.85 153.85 300.59 29.97 53.49 51.05 986.94 280.97 1775.4 570.94 4095.6 4610.1 51.81 50.17 4.54 12.26 129.55 91.82 11.485 52.48 46.11 50.01 12.31 12.20 95.68 95.95 50.17 50.03 47.28 50.51 12.28 12.19 97.76 96.23 49.08 49.87 51.51 49.96 12.26 12.21 88.73 93.24 54.52 51.63 48.73 50.60 12.28 12.20 90.83 95.56 52.97 50.24 45.30 50.85 12.31 12.21 89.38 95.44 53.94 50.32
Table 6 Parameter bounds. PARAMETERS NARROW MEDIUM WIDE R 1 ∼ 102 1 ∼ 103 1 ∼ 104 L 10−3 ∼ 10−1 10−3 ∼ 1 10−3 ∼ 10 C 10−6 ∼ 10−4 10−6 ∼ 10−3 10−6 ∼ 10−2
• bit ratio of exponent and mantissa particularly in hybrid encoding of BGA = 3/10. Experiment results are summarized in Tables 4 and 5. The terms NARROW, MEDIUM, and WIDE represent that the order differences of upper and lower bounds in physical parameters including R, L, and C are 2, 3, and 4, respectively, as shown in Table 6. In linear space, as search bounds are expanded to medium and wide bounds, LM BEST COST values increases about ten times as large as those in BGA, and several hundred times in RCGA. On the contrary, the LM BEST COST values in logarithmic and hybrid space are constant or even decreased under the same condition. Moreover, as to the FINAL BEST COST values, as search bounds are expanded, average exponent number in linear space decreases from −5.33 to −7 (LOG) and −6.33 (HYBRID) in BGA, and from −3 to −6.33 (LOG) and −5.67 (HYBRID) for RCGA. In other words, as search space grows wider by factor of 10n where n is the dimension of a parameter vector, BGA and RCGA in linear search space tend to give poor performances, while in logarithmic and hybrid search spaces their performances are either unaffected or improved. Table 5 shows that as cost is minimized each parameter approaches
to its true value. Figure 3 and Fig. 4 are provided to illustrate behavior of the cost values during search processes at each search space and encoding/converting method. Figure 3 shows mean values of average cost values (for each restart) versus generation axis, which supplies a general inspection to GA practitioners. The performance of BGA, as shown in Figs. 3(a), 3(c), and 3(e), is raised in logarithmic and hybrid spaces, which can be confirmed by noting that their cost differences between narrow, medium, and wide ranges are reduced. The superiority of the proposed methods is more prominent for RCGA as shown in Figs. 3(b), 3(d), and 3(f). Unlike BGA, the hybrid converting of RCGA shown in Fig. 3(f) outperforms other methods in that search performance is unaffected by assigned searching ranges. Despite this superiority, overall average cost profiles represent that the proposed encoding methods for BGA is more reliable than those for RCGA. Figure 4 shows mean values of best-so-far costs versus generation axis, which illustrate a similar minimization pattern in a smoothed fashion. Owing to the perturbations of average cost values in narrow range shown in Fig. 3, the mean best-so-far costs of BGA in logarithmic and hybrid space shown in Figs. 4(c) and 4(e) are almost equal and better than the costs of RCGA shown in Figs. 4(d) and 4(f). Overall comparison of BGA and RCGA ensures that the performance of BGA is slightly better than RCGA owing to the chromosomal structures and the adaptive scheme used for crossover and mutation in BGA. However, development of
KIM and KIM: NEW ENCODING /CONVERTING METHODS OF BINARY GA/REAL-CODED GA
1561
Fig. 3
Comparison of average costs of BGA and RCGA in each best run.
improved crossover and mutation operators in RCGA would overcome this weakness. It should be noted that the best identification result is
attained at the MEDIUM bounds in logarithmic search space for both GAs. This may be not general but problem specific, since the benchmark results indicate that hybrid search
IEICE TRANS. FUNDAMENTALS, VOL.E88–A, NO.6 JUNE 2005
1562
Fig. 4
Comparison of best-so-far costs of BGA and RCGA in each best run.
space is superior. This observation however implies that if logarithmic or hybrid solution space is adopted for GA, parameter bounds can be set a little larger than those in linear search space with conventional GAs. This paper specifically concerns real positive parameters to be sought by GA. This constraint is due to the fact
that a target of the proposed optimization methods is identification of an electrical system composed of real positive elements such as resistors, inductors, and capacitors. However, to expand the application field of the proposed methods search range should contain negative real values. Among several alternatives, adding a polarity bit to each parameter
KIM and KIM: NEW ENCODING /CONVERTING METHODS OF BINARY GA/REAL-CODED GA
1563
is recommended for BGA referring to the microprocessor technique. That is, if the polarity bit is assigned 0 (1) for a string, the decoding processes refer to the negative (positive) upper and lower bounds for the corresponding parameter and multiply a decoded value by −1(+1). This additional operation can be simplified for the case of symmetrical search range, e.g. −105 ∼ 105 . This principle of division can be also applied to RCGA in a different manner. Considering that the proposed converting methods of RCGA can be understood as mapping, some modification of definition in (8)–(10) will allow a mapping to negative real values. In case of exponential conversion, for example, upper and lower exponent bounds for negative values are specified and added to those of positive values. Assume that an optimization problem contains a parameter whose range is −105 ∼ 104 , and one assigns −2 ∼ 5 for exponents of negative values and −3 ∼ 4 for those of positive values. Then a total exponent range is modified from −3 ∼ 4 (positive only) into −3 ∼ 11(= 4 + 7). The added number 7 denotes a difference of exponent ranges of negative values. During RCGA an exponent number of higher than 4 (upper limit of positive values) is regarded as corresponding to a negative value, which is transformed and decoded to a negative value. For instance, a gene of 7 is finally decoded to −101 , if a negative exponent is computed from −2 (lower limit) and added by the overflow number of 3(= 7 − 4). A remaining problem of containing zero can be simply resolved by extending lower exponent limit as small as possible for approximation, e.g. 10−10 0. As a summary, the proposed encoding/converting methods for BGA/RCGA are shown to be more effective than the conventional linear encoding/converting methods for the problems of ill-scaled parameters and of wide parameter bounds. The proposed methods will fall under a similar method with the linear encoding in narrow parameter bounds, which means that the proposed methods are to be considered as general. 5.
Conclusion
This paper proposes logarithmic and hybrid encodings/converting methods for BGA/RCGA considering identification of electrical systems whose physical parameters are the real numbers of different order. A simple probabilistic analysis is provided to investigate what occurrs in terms of random sampling as the search space is expanded. Function optimization and system identification results show that the use of proposed encoding/converting methods guarantee a robust optimization performance irrelevant to the size of entire search space. In the case of searching for real numbers, RCGA is preferable for its simple coding, fast computation, and various mutation and crossover operators. Our future work will focuss on improving crossover and mutation operators in RCGA, and applying to the identification of high-order realworld systems.
References [1] T. S¨oderstr¨om and P. Stoica, System Identification, Prentice-Hall, Englewood Cliffs, NJ, 1989. [2] J.-W. Kim, B. Park, S.C. Jeong, S.W. Kim, and P. Park, “Fault diagnosis of a power transformer using an improved frequency response analysis,” IEEE Trans. Power Deliv., vol.20, no.1, pp.169–178, Jan. 2005. [3] S.W. Sung and I.-B. Lee, “Prediction error identification method for continuous-time process with time delay,” J. Industrial and Engineering Chemistry Research, vol.40, pp.5743–5751, 2001. [4] J.-W. Kim and S.W. Kim, “A new approach to system identification using hybrid genetic algorithm,” International Conference on Control, Automation and Systems, pp.887–890, Korea, Oct. 2001. [5] F. Alonge, F. D’Ippolito, G. Ferrante, and F.M. Raimondi, “Parameter identification of induction motor model using genetic algorithms,” Proc. Inst. Elect. Eng. Contr. Theory Applicat., vol.145, no.6, pp.587–593, Nov. 1998. [6] D. Whitley, K. Mathias, and P. Fitzhorn, “Delta coding: An iterative search strategy for genetic algorithms,” Proc. Fourth Int. Conf. on Genetic Algorithms, pp.77–84, 1991. [7] Z. Michalewicz, Genetic algorithm + data structure = evolution program, Springer-Verlag, Heidelberg, Berlin, 1996. [8] G.-G. Jin and S.-R. Joo, “A study on a real-coded genetic algorithm,” J. Control, Automation, and Systems Engineering, vol.6, no.4, pp.268–274, April 2000. [9] L. Davis, Handbook of Genetic Algorithms, Von Nostrand Reinhold, N.Y., 1991. [10] C.Z. Janikow and Z. Michalewicz, “An experimental comparison of binary and floating point representations in genetic algorithms,” Proc. 4th Int. Conf. on Genetic Algorithm, pp.31–36, Morgan Kauffmann Publishers, CA, 1991. [11] H. M¨uhlenbein and D. Schlierkamp-Voosen, “Predictive models for the breeder genetic algorithm I. continuous parameter optimization,” Evolutionary Computation, vol.1, pp.25–49, 1993. [12] D.H. Wolpert and W.G. Macready, “No free lunch theorems for optimization,” IEEE Trans. Evol. Comput., vol.1, no.1, pp.67–82, April 1997. [13] X. Yao, Y. Liu, and G. Lin, “Evolutionary programming made faster,” IEEE Trans. Evol. Comput., vol.3, no.2, pp.82–102, July 1999. [14] J.G. Digalakis and K.G. Margaritis, “An experimental study of benchmarking functions for genetic algorithms,” IEEE International Conference on System, Man, and Cybernetics, vol.5, pp.3810–3815, 2000. [15] J.-M. Yang and C.Y. Kao, “A combined evolutionary algorithm for real parameter optimization,” Proc. IEEE International Conference on Evolutionary Computation, pp.732–737, 1996. [16] J.J. Mor´e, B.S. Garbow, and K.E. Hillstrom, “Testing unconstrained optimization software,” ACM Trans. Math. Softw., vol.7, no.1, pp.17–41, March 1981. [17] M.J.D. Powell, “A hybrid method for nonlinear equations,” in Numerical Methods for Nonlinear Algebraic Equations, ed. P. Rabinowitz, pp.87–114, Gordon & Breach, New York, 1970. [18] G. Rudolph, “Convergence analysis of canonical genetic algorithms,” IEEE Trans. Neural Netw., vol.5, no.1, pp.96–101, 1994. [19] M. Srinivas and L.M. Patnaik, “Adaptive probabilities of crossover and mutation in genetic algorithms,” IEEE Trans. Syst. Man Cybern., vol.24, no.4, pp.656–667, April 1994.
Appendix:
Calculation of δe and δ m in Hybrid Encoding of BGA
The resolution of an exponent and a mantissa, δe and δm ,
IEICE TRANS. FUNDAMENTALS, VOL.E88–A, NO.6 JUNE 2005
1564
respectively, can be calculated by boundary conditions. Assume p and q bits in GA are assigned, respectively, to the exponent and the mantissa of a real number x ∈ [xL , xU ], and eU = log10 xU and eL = log10 xL are upper and lower bound exponents of x. Since we fixed δe and δm to be constant, general expression of transformed real numbers using p bits for the exponent and q bits for the mantissa is given by x = (1 + jδm )10e +iδe , 0 ≤ i ≤ 2 p − 1, 0 ≤ j ≤ 2q − 1. L
(A· 1)
In (A· 1), the upper bound condition is satisfied if (1 + 2q δm )10e
L
+(2 p −1)δe
U
= 10e .
(A· 2)
Note in (A· 2) that 2q is (2q − 1) + 1, i.e., larger by 1 than the maximum integer which can be reached by q bits. This setting is specifically devised to avoid the overlap that occurs when the exponent is increased from iδe to (i + 1)δe : (1 + 2q δm )10e
L
+iδe
= 1 · 10e +(i+1)δe L = 10δe 10e +iδe . L
(A· 3)
From (A· 3), the following relation is obtained: (1 + 2q δm ) = 10δe .
(A· 4)
Inserting this into (A· 2) gives 10e
L
+2 p δe
U
= 10e ,
(A· 5)
thus, δe =
eU − e L . 2p
(A· 6)
Since δe is obtained, δm is easily calculated from (A· 4): δm =
10δe − 1 . 2q
(A· 7)
Jong-Wook Kim was born in Youngpoong, Korea, in 1970. He received the B.S., M.S., and Ph.D. degrees from the Electrical and Computer Engineering Division at Pohang University of Science and Technology (POSTECH), Pohang, Korea, in 1998, 2000, and 2004, respectively. Currently, he is a Researcher with the Electrical Steel Sheet Research Group in POSCO Technical Research Laboratories. His research interests include core design of transformers, diagnosis of electrical systems, intelligent control, optimization algorithm, and system identification.
Sang Woo Kim was born in Pyungtaek, Korea, in 1961. He received the B.S., M.S., and Ph.D. degrees from Department of Control and Instrumentation Engineering, Seoul National University, in 1983, 1985, and 1990, respectively. Currently, he is an Associate Professor in the Department of Electronics and Electrical Engineering at Pohang University of Science and Technology (POSTECH), Pohang, Korea. He joined POSTECH in 1992 as an Assistant Professor and was a Visiting Fellow in the Department of Systems Engineering, Australian National University, Canberra, Australia, in 1993. His current research interests are in optimal control, optimization algorithm, intelligent control, wireless communication, and process automation.
1554
PAPER
New Encoding /Converting Methods of Binary GA/Real-Coded GA Jong-Wook KIM†a) and Sang Woo KIM†† , Members
SUMMARY This paper presents new encoding methods for the binary genetic algorithm (BGA) and new converting methods for the real-coded genetic algorithm (RCGA). These methods are developed for the specific case in which some parameters have to be searched in wide ranges since their actual values are not known. The oversampling effect which occurs at large values in the wide range search are reduced by adjustment of resolutions in mantissa and exponent of real numbers mapped by BGA. Owing to an intrinsic similarity in chromosomal operations, the proposed encoding methods are also applied to RCGA with remapping (converting as named above) from real numbers generated in RCGA. A simple probabilistic analysis and benchmark with two ill-scaled test functions are carried out. System identification of a simple electrical circuit is also undertaken to testify effectiveness of the proposed methods to real world problems. All the optimization results show that the proposed encoding/converting methods are more suitable for problems with ill-scaled parameters or wide parameter ranges for searching. key words: binary genetic algorithm, real-coded genetic algorithm, illscaled function, system identification, encoding
1.
Introduction
System identification is the field of mathematical modelling of systems from experimental data, and is categorized into nonparametric and parametric methods according to model structures [1]. In the case the model structure is considerably complex or largely scaled like power transformers, nonparametric identification is deemed to be suitable. Nonparametric identification is characterized by the property that the resulting models are curves or functions, and its good example is the spectral analysis for the diagnosis of a power transformer [2]. On the other hand, parametric identification is favorable to a system whose model structure is simply described with mathematical static and/or dynamic equations. Most researches in parametric identification area focus on discrete-time systems due to the easiness and simplicity of required mathematics. However, the performance of discrete-time identification depends on the sampling time, and the obtained parameters give little physical insight into the system. Therefore, in the case of diagnosis or modeling, transformation of parameters in a discrete-time system Manuscript received March 17, 2004. Manuscript revised October 24, 2004. Final manuscript received March 4, 2005. † The author is with the Electrical Steel Sheet Research Group, POSCO Technical Research Laboratories, Pohang 790-785, Korea. †† The author is with the Electrical and Computer Engineering Division, Pohang University of Science and Technology (POSTECH), Pohang, Korea. a) E-mail: [email protected] DOI: 10.1093/ietfec/e88–a.6.1554
into those in a continuous-time system is required, which is quite cumbersome or even impossible as system dimension is raised. To this end, continuous-time prediction error method free from parameter transformation was provided, which estimates the system parameters by minimizing prediction errors with exact derivatives of the objective function with respect to the adjustable parameters [3]. However, the prediction error method requires a good guess of initial parameters, and update rule design is system dependent. The genetic algorithm whose chromosome is formed with model parameters and initial states is applied to the parametric identification of a simple electrical circuit [4] and an induction motor [5]. These systems contain physical electrical elements such as resistors, capacitors, and inductors whose values range from micro to kilo units. If actual values of each element are known a priori, which is unrealistic, parameter bounds for search with GA can be set up moderately. For the real systems however it is difficult to determine proper bounds owing to time variance of actual values or inaccuracy of name plate values. Therefore, the parameter bounds have to be selected wide enough not to exclude global minima prior to executing GA, while it is widely acknowledged that as parameter space grows wider optimization performance degenerates as much. The delta coding is an efficient alternative for this problem, but uses a multistage strategy where the global minima can be discarded at an initial search phase [6]. This paper proposes new encoding methods of BGA specifically designed to overcome the wide bound search problem. As search space grows wider, the general linear sampling (or linear encoding) of BGA suffers from inevitable unbalance between small and large numbers owing to the limited chromosome length, i.e., resolution of real numbers. In this paper logarithmic and hybrid encoding methods for search space are developed by modifying the conventional encoding (or decoding, equivalently) function of BGA. These changes in phenotypes enlarge the small numbers often ignored and reduce large numbers overly estimated both in linear sampling. Furthermore, the proposed encoding methods include the linear encoding method such that for narrow search space the effects of all the encoding methods are almost identical. Since the electrical circuit elements addressed in this paper are assigned positive real values, RCGA is more efficient than BGA in terms of search space resolution, run time, simplicity, convenience of program coding, and so on. RCGA has strings of real numbers instead of binary chro-
c 2005 The Institute of Electronics, Information and Communication Engineers Copyright
KIM and KIM: NEW ENCODING /CONVERTING METHODS OF BINARY GA/REAL-CODED GA
1555
mosomes of BGA, and takes three similar operators for real numbers; reproduction, crossover, and mutation operators. Hence it is assumed that RCGA and BGA will have similar optimization aspects under the expansion of search space. To verify this postulate, strings of RCGA are reprocessed (or converted as termed in this paper) in accordance with the logarithmic and hybrid encoding methods of BGA. For an investigation of these changes, a simple probabilistic analysis is made for logarithmic effects on the expansion of search space, and benchmark is undertaken on two ill-scaled test functions. In addition, as the first step to continuous-time parametric identification, a simple series-connected electrical circuit is identified with simulated input and output data. Improvement of optimization results assures that the proposed methods in GAs are effective for wide bound search or ill-scaled problems. This paper is organized as follows: Sect. 2 describes some details of BGA and RCGA. Section 3 explains the proposed encoding methods with a simple analysis on linear and logarithmic search spaces under space expansion. Section 4 provides performance comparison of benchmark and system identification. Section 5 concludes the work with comments on future works. 2.
BGA and RCGA
GA has three operators of reproduction, crossover, and mutation. Reproduction is devised to inherit superior individuals to the next generation by the fitness-based selection rule. This operator is a simulation of natural selection. In the crossover, two individuals selected randomly in the present population exchange their bits after random crossing sites. During the mutation, every bit is assigned an equal probability to be replaced by its complement digit. This is an imitation of natural mutation, and is very rare in GA processes as well. In BGA and RCGA, some similarities and differences exist as is written in the following subsections. 2.1 Representation and Crossover BGA has chromosomes made of binary digits 0 and 1, while RCGA has strings of real numbers. For instance, if the total number of parameters is k and if l bits in particular for BGA are assigned to every parameter, chromosomes are illustrated as follows: l
l
l
p1
p2
pk
s1b = (b111 · · · b11l ) · · · (b1i1 · ·b1im | b1i(m+1) · ·b1il ) · · · (b1k1 · · · b1kl ) s2b = (b211 · · · b21l ) · · · (b2i1 · ·b2im | b2i(m+1) · ·b2il ) · · · (b2k1 · · · b2kl ) ⇓ s1b = (b111 · · · b11l ) · · · (b1i1 · ·b1im | b2i(m+1) · ·b2il ) · · · (b2k1 · · · b2kl )
s2b = (b211 · · · b21l ) · · · (b2i1 · ·b2im | b1i(m+1) · ·b1il ) · · · (b1k1 · · · b1kl ), where bim ∈ {0, 1} is an m-th binary bit in an i-th substring. Since chromosome representation in RCGA is different from that of BGA, the crossover operation has to be modified. Up till now, many kinds of crossover operators, including simple crossover, arithmetical crossover [7], BLXα crossover, and modified simple crossover [8], have been suggested. Among them, the most similar operator with BGA is the modified simple crossover described as follows: 1 · · · rk1 ) s1r = (r11 · · · ri1 | ri+1 2 s2r = (r12 · · · ri2 | ri+1 · · · rk2 ) ⇓ 1 1 s = (r · · · r1 | r2 · · · r2 ) r
s2r
p1
p2
pk
where sb and sr represent concatenated strings of BGA and RCGA, respectively. Let s1b and s2b are selected parents for crossover to pro duce their offsprings s1b and s2b . After crossover, a change occurs as follows:
=
(r12
i
i+1
k
1 · · · ri2 | ri+1 · · · rk1 )
where ri1 = λc1 ri1 + (1 − λc1 )ri2 r2 = λ r2 + (1 − λ )r1 i
c2 i
c2
i
ri ∈ R represents an i-th parameter, and λc1 , λc2 denote uniformly distributed random numbers between 0 and 1. 2.2 Mutation Mutation is a unary operator which change genes in chromosomes. The mechanism of uniform mutation in an i-th substring of BGA is illustrated as follows: (bi1 · · · bim · · · bil ) ⇒ (bi1 · · · bim · · · bil ), where bim represents a flipped bit of bim , i.e., from 0 to 1 or from 1 to 0. Unlike BGA, various mutation operators have been provided in RCGA such as uniform mutation [7], real number creep mutation [9], dynamic mutation [10], M¨uhlenbein mutation [11], and so on. The analogous operator to BGA is the uniform mutation by which a selected real-valued gene is transformed randomly within its upper and lower bounds. The principle of the uniform mutation is illustrated as
· · · 01 10 · · · 10 · · · 00 · · · 01) sb = (11 sr = (15.279 0.858 · · · 296.99),
1
(r1 · · · r j · · · rk ) ⇒ (r1 · · · r j · · · rk ) r j = λm (rUj − r Lj ) + r Lj
(1)
where r j ∈ [r Lj , rUj ] is the mutated j-th gene from r j , and λm ∈ [0, 1] is the random number. 3.
Search Space
3.1 Encoding Methods of BGA The general decoding function which linearly transforms an
IEICE TRANS. FUNDAMENTALS, VOL.E88–A, NO.6 JUNE 2005
1556
l-bit-long binary string (a1 · · · al ) into a bounded real number x ∈ [xL , xU ] is written as x=
l−1 xU − x L a(l−i) 2i + xL 2l − 1 i=0
(2)
where a1 is a most significant bit (MSB) and al is a least significant bit (LSB). Specifically, (xU − xL )/(2l − 1) in (2) plays a role as search resolution or precision. To avoid terminological ambiguity hereafter, definitions are stated for each type of sampled space. The first is linear space. Definition 1 (Linear Space): The n-dimensional discretized linear space with finite resolution δL is defined as: L : {x | x = (x1 · · · xn )T ∈ Rn } xi = xiL + kδL , i = 1, · · · , n U xi − xiL k∈ 0, 1, · · · , , δL
(3)
L where xU i and xi are predetermined upper and lower bounds of xi , respectively.
The followings are the proposed encoding methods of BGA with corresponding definitions. One is exponential encoding, which transforms a binary string (b1 · · · bl ) into an exponent of 10 as y = 10
eU −eL 2l −1
(
l−1 i=0
b(l−i) 2i )+eL
,
(4)
where eL = log10 yL and eU = log10 yU . In (4), y is regarded as equidistant in terms of logarithmic scale. Thus, the corresponding space is denominated logarithmic space rather than exponential space. Definition 2 (Logarithmic Space): An n-dimensional discretized logarithmic space with finite resolution δG is defined as: G : {y | y = (y1 · · · yn )T ∈ Rn } yi = 10ei +kδG , i = 1, · · · , n U ei − eiL k∈ 0, 1, · · · , δG L
U L L where eU i = log10 yi and ei = log10 yi are the exponents of upper and lower bounds of yi , respectively.
The other is hybrid encoding, which is also a common way to represent floating-point numbers by a fixed-length memory. To implement the hybrid encoding method, a binary string must be partitioned into two parts; mantissa and exponent. If a majority of bits are assigned to a mantissa, this encoding turns closely into a linear encoding. On the other hand, if almost all bits are referred to as those for an exponent, this is approximately exponential encoding. Therefore, we conjecture that the hybrid encoding lies between linear and logarithmic encodings depending on the bit ratio of mantissa and exponent.
Suppose we assign p and q bits for the exponent and the mantissa of a real number, respectively, and a binary string is described as (c1 · · · c p d1 · · · dq ), whose total length is l(= p + q). The hybrid encoding is described as z = mant × 10expo , p−1 eU − e L c(p−i) 2i + eL , δe = expo = δe 2p i=0 mant = δm
q−1
d(q− j) 2 j + 1, δm =
j=0
10δe − 1 2q
(5) (6)
(7)
where δe and δm are the resolutions of exponent and mantissa, respectively, that are devised to avoid overlapping of decoding. The details of how δe and δm are calculated are provided in Appendix. The hybrid space is defined in accordance with (5)–(7). Definition 3 (Hybrid Space): An n-dimensional discretized hybrid space with finite resolution vector δH = (δm δe )T is defined as: H : {z | z = (z1 · · · zn )T ∈ Rn } L zi = (1 + lδm ) × 10ei +kδe , i = 1, · · · , n U ei − eiL k∈ 0, 1, · · · , δe − 1 δe 10 − 1 − 1 , j = k max(l) + l l ∈ 0, 1, · · · , δm U L L where eU i = log10 zi and ei = log10 zi are the exponents of upper and lower bounds of zi , respectively.
3.2 Converting Methods of RCGA Since individuals of RCGA consist of real numbers unlike BGA, the conventional RCGAs require no encoding methods. However, we apply in parallel to RCGA the concepts of the linear, exponential, and hybrid encoding methods based on the intrinsic similarities between BGA and RCGA. A new term for ‘encoding’ is denominated ‘conversion’ for RCGA. Then, except the linear conversion, exponential and hybrid conversion are accomplished using converting functions imitating (4)–(7). The exponential conversion of RCGA is described as yi = 10ri , ri ∈ [log10 yiL , log10 yU i ]
(8)
where ri is an i-th gene of RCGA, and the hybrid conversion for ri is written as zi = (9hi + 1)10ni , hi = ri − ni , ni = ri , 0 ≤ hi < 1.
(9) (10)
As shown in (8)–(10), converting functions of RCGA are considerably simple compared with the encoding functions of BGA.
KIM and KIM: NEW ENCODING /CONVERTING METHODS OF BINARY GA/REAL-CODED GA
1557
Fig. 1
Relative area of crest set C.
=
3.3 Probabilistic Analysis In searching for a whole solution space whose dimension equals the number of parameters, both the GAs have finite space resolution. In BGA, the resolution depends on the number of bits per parameter, while in RCGA it depends on the resolution of random numbers during crossover and mutation. Therefore, it can be regarded that search space is divided by a finite number of grids, whose amount remains constant irrespective of the size of search space. For a brief geometric analysis, we define the level set C as C : {θ | F(θ) ≥ F0 , θ ∈ Rn }
(11)
where θ is the n-dimensional parameter vector, F(·) is the fitness function, and F0 is the constant fitness value. Note that a sufficiently large F0 makes C a single and closed set even though a multimodal optimization problem is addressed. For an explanation of space expansion, Fig. 1(a) illustrates an optimization problem, whose parameters are p11 ∈ [p11min , p11max ] and p12 ∈ [p12min , p12max ] in a feasible space S1 . Let the only upper bounds of each parameter be expanded by factors of k1 and k2 , respectively. Then new bounds are described as p21max = k1 p11max , p21min = p11min p22max = k2 p12max , p22min = p12min
(12)
where superscripts 1 and 2 represent ‘before expansion’ and ‘after expansion,’ respectively. Fig. 1(b) shows an expanded solution space S2 , where it is notable that despite the expansion of the feasible space the crest set C remains the same. Thus, the relative portion of C decreases for S2 , and this contraction may cause difficulty in exploring solutions inside C in terms of probability. A relative probability between the two cases, i.e. before and after expansion, is computed for a 2-dimensional problem as P2 (C) A(C)/A(S 2 ) A(S 1 ) = = P1 (C) A(C)/A(S 1 ) A(S 2 )
(p11max − p11min )(p12max − p12min )
(13)
(p21max − p21min )(p22max − p22min )
where P1 (C) and P2 (C) represent the probabilities of selecting solutions inside C for corresponding search spaces S1 and S2 , respectively, and A(·) stands for area. In the case of p11max p11min and p12max p12min , (13) is simplified as (p11max − p11min )(p12max − p12min ) P2 (C) = P1 (C) k1 k2 (p11max − k11 p11min )(p12max − k12 p12min ) 1 . (14) k1 k2 Thus in an n-dimensional problem (13) can be generalized as 1 P2 (C) ≈ . (15) P1 (C) k1 k2 · · · kn ≈
This result indicates that the linear scale expansion of upper bounds in each parameter can cause performance degradation of the random search during the initialization and reproduction operations of GA. However, owing to the intrinsic mechanism of survival of the fittest, the overall performance of GA is not deteriorated with as much ratio as (15). To analyze the relative probability (13) for the logarithmic space, the lower and upper bounds in (12) are interpreted on a logarithmic scale as 1 1L p2L 1max = log k1 + log p1max = log k1 + p1max 2L 1 1L p1min = log p1min = p1min 1 1L p2L 2max = log k2 + log p2max = log k2 + p2max 1 1L p2L 2min = log p2min = p2min
(16)
where the superscript L stands for a logarithmic opeartion. The relative probability on logarithmic space is computed in a similar manner as 1L 1L 1L P2 (C L ) (p1L 1max − p1min )(p2max − p2min ) = 2L 2L 2L P1 (C L ) (p2L 1max − p1min )(p2max − p2min )
=
1L 1L 1L (p1L 1max − p1min )(p2max − p2min ) 1L 1L 1L (log k1 + p1L 1max − p1min )(log k2 + p2max − p2min )
. (17)
IEICE TRANS. FUNDAMENTALS, VOL.E88–A, NO.6 JUNE 2005
1558
For a more intuitive simplification, if log-scaled expansion ratios are defined as m1 =
log k1 log k2 , m2 = 1L 1L − p1min p2max − p1L 2min
(18)
p1L 1max
then (17) is simplified as P2 (C L ) 1 = . P1 (C L ) (m1 + 1)(m2 + 1)
(19)
For a simple comparison of (15) and (19) suppose that p1 and p2 were presumed to exist between 1 and 100 and 0.001 and 0.1, respectively, but search results show that those ranges fail to include acceptable solutions. As the next step the upper bounds are expanded simultaneously by a factor of 100, which leads the relative probability to enormous decrease of about 1/10000 for linear space, while only a slight decrease of 1/4 for logarithmic space. Thus, logarithmic space is far more reasonable for searching a wide solution space. 4.
converges to a global optimum, while the modified versions of maintaining the best solution in the population do. Therefore the adaptive GA (AGA) scheme [19] is adopted for BGA in this paper. GA parameters are set as • • • • • • • • • • • • •
maximum generation = 2000 number of variables = 2 population size = 50 number of restarts = 50 selection: Roulette wheel selection bit length of a substring in BGA = 13 bit ratio of exponent and mantissa in hybrid encoding of BGA = 4/9 parameters of adaptive BGA : k1 = 0.85, k2 = 0.5, k3 = 1.0, k4 = 0.05 crossover of BGA: one-point crossover crossover of RCGA: modified simple crossover mutation of BGA and RCGA: uniform mutation crossover rate of RCGA = 0.95 mutation rate of RCGA = 0.01.
For a performance comparison, the following measures are used:
Experimental Results
4.1 Test Functions and Performance Measure After developing an optimization method, one has to test it on a suite of benchmark functions at a validation step in accordance with the ‘no-free-lunch’ theorem [12]. In the literature of evolutionary computation [13]–[15], 14 or 23 test functions, varying from unimodal to multimodal and from low- to high-dimensional cost functions, are provided with their global optima acknoledged. However, the test functions are confined in well-scaled search space, i.e., their bounds are of similar order. Therefore, badly scaled functions are necessary for the specific optimization problem addressed in this paper. This paper adopts the following functions for test of the proposed encoding/converting methods: • Brown badly scaled function:
(20)
• Powell badly scaled function: f p (x1 , x2 ) = (104 x1 x2 − 1)2 + (exp[−x1 ] + exp[−x2 ] − 1.0001)2
In the above measures, LM (logarithmic mean) represents exponent averaging. That is, the LM of x = (x1 x2 · · · xm )T is described as LM(x) = 10
−6 2
fb (x1 , x2 ) = (x1 − 10 ) + (x2 − 2 × 10 ) + (x1 x2 − 2)2 6 2
a) MEAN BEST COST: mean of the best costs obtained at each restart. b) STD DEV: standard deviation of the best costs obtained at each restart. c) LM BEST COST: logarithmic mean of the best costs. d) FINAL BEST COST: best of the best costs obtained at each restart. e) MEAN BEST SOL: mean of the best solutions obtained at each restart. f) LM BEST SOL: logarithmic mean of each best solution. g) FINAL BEST SOL: solution for d).
(21)
The global minima of (20) and (21) are known to be exactly and almost 0 at the points (x1 , x2 ) = (106 , 2 × 10−6 ) and (1.098 · · · 10−5 , 9.106 · · · ), respectively. Search is undertaken within [10−8 , 108 ] for (20) and [10−8 , 104 ] for (21), whose upper bounds are huge values for general optimization problems. Rudolph [18] has shown that the classical BGA never
m log10 xi i=1 m
(22)
which is quite reasonable for averaging real numbers whose magnitudes are different in their exponents. For example, the general averaging ignores such a small number as 10−8 in presence of a relatively large number of 105 , but the LM balances the two numbers with their exponents. Thus, LM is thought to be a more reliable medium for a set of numbers with large variance. Table 1 gives performance comparison of the proposed encoding/converting methods on the Brown function for each solution space. Note in the case of RCGA that MEAN BEST COST values are quite different from LM BEST COST values owing to an ill-scaled property of attained cost values. This discrepancy increases as GA finds more numbers of bad solutions of high cost values. In linear solution space, both GAs fail to find out a
KIM and KIM: NEW ENCODING /CONVERTING METHODS OF BINARY GA/REAL-CODED GA
1559
GA
SPACE
LINEAR LOG HYBRID LINEAR RCGA LOG HYBRID BGA
Table 1 Performance comparison of BGA and RCGA on linear, logarithmic, and hybrid search space for the Brown function. MEAN STD LM FINAL MEAN BEST SOL LM BEST SOL BEST COST DEV BEST COST BEST COST x1 x2 x1 x2 1.92×108 4.33×108 5.57×106 1.21×106 9.95×105 1.00×10−8 9.94×105 1.00×10−8 7.63×106 7.61×106 2.05×106 3.16×105 1.00×106 1.96×10−6 1.00×106 1.94×10−6 2.53×106 1.19×106 9.13×104 3.81×10−6 9.99×105 1.66×10−6 9.99×105 1.57×10−6 6.80×1022 4.30×1023 5.36×1012 3.96 3.28×105 1.79×105 7.05×10−4 9.21 5 6 −1 −11 4.97×10 2.52×10 3.21×10 8.54×10 1.00×106 6.62×10−6 1.00×106 2.44×10−6 4.84×106 2.80×107 1.12×10−1 2.67×10−13 9.99×105 4.05×10−5 9.99×105 4.97×10−6
FINAL BEST SOL x1 x2 1.00×106 1.00×10−8 9.99×105 2.00×10−6 1.00×106 2.00×10−6 1.00×106 1.00×10−8 1.00×106 2.00×10−6 1.00×106 2.00×10−6
Table 2 Performance comparison of BGA and RCGA on linear, logarithmic, and hybrid search space for the Powell function. GA
SPACE
MEAN BEST COST
STD DEV
LM BEST COST
FINAL BEST COST
BGA
LINEAR LOG HYBRID
3.45×10−3 4.94×10−6 2.02×10−6
1.26×10−2 5.59×10−6 3.58×10−6
9.27×10−6 2.13×10−7 3.64×10−7
1.22×10−6 2.60×10−10 4.17×10−9
6.88×103 7.92×102 1.20×103
2.92×103 1.18×103 7.15×102
2.48 5.31×10−4 1.56×10−2
3.95 ×10−5 1.88×10−1 6.39×10−3
1.00×10−8 1.11×10−5 1.00×10−5
1.00×104 8.97 1.00×101
RCGA
LINEAR LOG HYBRID
8.99×1021 3.82×10−3 1.19×10−3
3.73×1022 2.12×10−2 4.68×10−3
3.73×104 4.49×10−7 1.16×10−7
1.00×10−8 4.85×10−9 2.41×10−11
3.89×103 1.97×101 1.43×102
1.29×104 1.26×101 6.87
2.00×10−1 1.93×10−3 5.91×10−4
8.97×10−1 5.20×10−2 1.70×10−1
1.00×10−8 1.01×101 1.10×10−5
1.00×104 9.88×10−6 9.06
true value x2 , 2 × 10−6 , and just approximate it as 0 because of limited resolution around such a small value. However, in logarithmic and hybrid search spaces, the precise value of x2 is obtained, which shows the effectiveness of the proposed encdong/converting methods for abnormally small and large numbers. Moreover, RCGA searches for better solutions than BGA on logarithmic and hybrid spaces. As is evidenced in the literature, RCGA, which has higher search resolution, is more suitable for nonlinear optimization problems, and the data in Table 1 verifies this result. Table 2 provides search results of the Powell function under the same search condition with Table 1. Unlike the Brown function, the Powell function appears to contain local minima at (x1 , x2 ) = (10.1, 9.88 × 10−6 ) as shown in logarithmic space with RCGA. The diversity of LM BEST SOL values also demonstrates the ruggedness of this function. Despite this difficulty, the proposed methods have found acceptable solutions among which a global minimum is (x1 , x2 ) = (1.1035 × 10−5 , 9.0624) with the cost of 2.41 × 10−11 found in hybrid space by RCGA. 4.2 System Identification of a Simple Electrical Circuit The main concern of this paper is to identify a simple electrical circuit whose elements are a resistor with resistance R, an inductor with inductance L, and a capacitor with capacitance C, as shown in Fig. 2. The state space and output equations of this system are given as 1 R x˙1 − L1 x1 − = 1L + L u (23) 0 x2 x˙2 0 C x1 1 0 (24) y= x2 where x1 is input current and x2 is capacitor voltage. Because initial state is necessary for determining unique states at each time, parameter vector in (11) is determined as
MEAN BEST SOL x1 x2
LM BEST SOL x1 x2
Fig. 2
FINAL BEST SOL x1 x2
Simple RLC circuit.
Table 3 Characteristics of PRBS. Nr p Ts N 6 17 100 × 10−6 1500
θ = (R L C x1 (0) x2 (0))T . The fitness function in GAs is designed to minimize the sum squared estimation error as F(θ) =
1 2
1 2 i=1 {y(θ) − y(θ0 )}
N
(25)
where N is the number of data points and the true parameter vector is θ0 = (1.22 × 101 9.60 × 10−2 5.00 × 10−5 7.28 × 10−4 2.98 × 10−1 )T . Input signal used for system identification is PRBS (Pseudo-Random Binary Signal) [1], and its characteristic parameters are given in Table 3 where Nr is the number of shifting registers, p is the frequency divider, and T s is the sampling period in seconds. GA parameters are the same with those of the test functions with the exceptions of • maximum generation = 5000 • number of parameters = 5
IEICE TRANS. FUNDAMENTALS, VOL.E88–A, NO.6 JUNE 2005
1560 Table 4 Comparison of cost values between BGA and RCGA on linear, logarithmic, and hybrid search space with respect to narrow, medium, and wide parameter ranges. BGA
RCGA
SPACE
BOUND
MEAN BEST COST
STD DEV
LM BEST COST
FINAL BEST COST
MEAN BEST COST
STD DEV
LM BEST COST
FINAL BEST COST
LINEAR
NARROW MEDIUM WIDE
2.81×10−4 2.30×10−3 9.37×10−3
2.45×10−5 2.05×10−3 9.11×10−3
1.44×10−4 1.22×10−3 4.55×10−3
1.98×10−6 4.63×10−5 4.69×10−5
2.86×10−4 7.08×10−2 8.89×10−2
2.71×10−4 6.52×10−3 1.08×10−3
1.83×10−4 7.05×10−2 8.89×10−2
1.25×10−5 4.55×10−2 8.42×10−2
LOG
NARROW MEDIUM WIDE
3.11×10−4 1.86×10−3 1.98×10−3
3.55×10−4 3.35×10−3 3.79×10−3
1.05×10−4 3.96×10−4 4.10×10−4
3.19×10−6 4.84×10−9 6.96×10−6
3.51×10−2 3.87×10−4 2.17×10−4
3.78×10−2 6.26×10−4 4.06×10−4
6.03×10−3 3.90×10−5 7.56×10−5
1.16×10−4 1.38×10−8 3.39×10−7
HYBRID
NARROW MEDIUM WIDE
1.97×10−4 2.18×10−3 2.49×10−3
2.91×10−4 2.88×10−3 3.72×10−3
4.31×10−5 6.16×10−4 4.70×10−4
4.40×10−8 8.35×10−6 1.41×10−5
5.22×10−4 3.07×10−4 4.47×10−4
6.08×10−4 3.79×10−4 5.12×10−4
2.60×10−4 1.14×10−4 2.02×10−4
4.72×10−5 1.08×10−6 1.91×10−6
Table 5 Comparison of obtained solutions between BGA and RCGA on linear, logarithmic, and hybrid search space with respect to narrow, medium, and wide parameter ranges. The true values are R=12.20 (Ω), L=96.00 (mH), and C=50.00 (µF). SPACE
BOUND
NARROW MEDIUM WIDE NARROW LOG MEDIUM WIDE NARROW HYBRID MEDIUM WIDE LINEAR
BGA R (Ω) L (mH) LM BEST LM BEST 12.90 12.21 94.90 96.53 12.71 12.22 94.49 98.57 14.25 11.99 94.91 93.78 12.23 12.20 93.08 95.71 12.94 12.20 103.5 95.98 12.33 12.18 101.83 95.12 12.16 12.20 93.51 96.07 12.74 12.24 98.63 94.87 12.57 12.18 105.61 94.66
RCGA C (µF) R (Ω) L (mH) C (µF) LM BEST LM BEST LM BEST LM BEST 50.63 49.72 12.38 12.20 90.30 94.52 53.46 50.79 51.38 48.81 95.67 48.95 145.85 153.85 300.59 29.97 53.49 51.05 986.94 280.97 1775.4 570.94 4095.6 4610.1 51.81 50.17 4.54 12.26 129.55 91.82 11.485 52.48 46.11 50.01 12.31 12.20 95.68 95.95 50.17 50.03 47.28 50.51 12.28 12.19 97.76 96.23 49.08 49.87 51.51 49.96 12.26 12.21 88.73 93.24 54.52 51.63 48.73 50.60 12.28 12.20 90.83 95.56 52.97 50.24 45.30 50.85 12.31 12.21 89.38 95.44 53.94 50.32
Table 6 Parameter bounds. PARAMETERS NARROW MEDIUM WIDE R 1 ∼ 102 1 ∼ 103 1 ∼ 104 L 10−3 ∼ 10−1 10−3 ∼ 1 10−3 ∼ 10 C 10−6 ∼ 10−4 10−6 ∼ 10−3 10−6 ∼ 10−2
• bit ratio of exponent and mantissa particularly in hybrid encoding of BGA = 3/10. Experiment results are summarized in Tables 4 and 5. The terms NARROW, MEDIUM, and WIDE represent that the order differences of upper and lower bounds in physical parameters including R, L, and C are 2, 3, and 4, respectively, as shown in Table 6. In linear space, as search bounds are expanded to medium and wide bounds, LM BEST COST values increases about ten times as large as those in BGA, and several hundred times in RCGA. On the contrary, the LM BEST COST values in logarithmic and hybrid space are constant or even decreased under the same condition. Moreover, as to the FINAL BEST COST values, as search bounds are expanded, average exponent number in linear space decreases from −5.33 to −7 (LOG) and −6.33 (HYBRID) in BGA, and from −3 to −6.33 (LOG) and −5.67 (HYBRID) for RCGA. In other words, as search space grows wider by factor of 10n where n is the dimension of a parameter vector, BGA and RCGA in linear search space tend to give poor performances, while in logarithmic and hybrid search spaces their performances are either unaffected or improved. Table 5 shows that as cost is minimized each parameter approaches
to its true value. Figure 3 and Fig. 4 are provided to illustrate behavior of the cost values during search processes at each search space and encoding/converting method. Figure 3 shows mean values of average cost values (for each restart) versus generation axis, which supplies a general inspection to GA practitioners. The performance of BGA, as shown in Figs. 3(a), 3(c), and 3(e), is raised in logarithmic and hybrid spaces, which can be confirmed by noting that their cost differences between narrow, medium, and wide ranges are reduced. The superiority of the proposed methods is more prominent for RCGA as shown in Figs. 3(b), 3(d), and 3(f). Unlike BGA, the hybrid converting of RCGA shown in Fig. 3(f) outperforms other methods in that search performance is unaffected by assigned searching ranges. Despite this superiority, overall average cost profiles represent that the proposed encoding methods for BGA is more reliable than those for RCGA. Figure 4 shows mean values of best-so-far costs versus generation axis, which illustrate a similar minimization pattern in a smoothed fashion. Owing to the perturbations of average cost values in narrow range shown in Fig. 3, the mean best-so-far costs of BGA in logarithmic and hybrid space shown in Figs. 4(c) and 4(e) are almost equal and better than the costs of RCGA shown in Figs. 4(d) and 4(f). Overall comparison of BGA and RCGA ensures that the performance of BGA is slightly better than RCGA owing to the chromosomal structures and the adaptive scheme used for crossover and mutation in BGA. However, development of
KIM and KIM: NEW ENCODING /CONVERTING METHODS OF BINARY GA/REAL-CODED GA
1561
Fig. 3
Comparison of average costs of BGA and RCGA in each best run.
improved crossover and mutation operators in RCGA would overcome this weakness. It should be noted that the best identification result is
attained at the MEDIUM bounds in logarithmic search space for both GAs. This may be not general but problem specific, since the benchmark results indicate that hybrid search
IEICE TRANS. FUNDAMENTALS, VOL.E88–A, NO.6 JUNE 2005
1562
Fig. 4
Comparison of best-so-far costs of BGA and RCGA in each best run.
space is superior. This observation however implies that if logarithmic or hybrid solution space is adopted for GA, parameter bounds can be set a little larger than those in linear search space with conventional GAs. This paper specifically concerns real positive parameters to be sought by GA. This constraint is due to the fact
that a target of the proposed optimization methods is identification of an electrical system composed of real positive elements such as resistors, inductors, and capacitors. However, to expand the application field of the proposed methods search range should contain negative real values. Among several alternatives, adding a polarity bit to each parameter
KIM and KIM: NEW ENCODING /CONVERTING METHODS OF BINARY GA/REAL-CODED GA
1563
is recommended for BGA referring to the microprocessor technique. That is, if the polarity bit is assigned 0 (1) for a string, the decoding processes refer to the negative (positive) upper and lower bounds for the corresponding parameter and multiply a decoded value by −1(+1). This additional operation can be simplified for the case of symmetrical search range, e.g. −105 ∼ 105 . This principle of division can be also applied to RCGA in a different manner. Considering that the proposed converting methods of RCGA can be understood as mapping, some modification of definition in (8)–(10) will allow a mapping to negative real values. In case of exponential conversion, for example, upper and lower exponent bounds for negative values are specified and added to those of positive values. Assume that an optimization problem contains a parameter whose range is −105 ∼ 104 , and one assigns −2 ∼ 5 for exponents of negative values and −3 ∼ 4 for those of positive values. Then a total exponent range is modified from −3 ∼ 4 (positive only) into −3 ∼ 11(= 4 + 7). The added number 7 denotes a difference of exponent ranges of negative values. During RCGA an exponent number of higher than 4 (upper limit of positive values) is regarded as corresponding to a negative value, which is transformed and decoded to a negative value. For instance, a gene of 7 is finally decoded to −101 , if a negative exponent is computed from −2 (lower limit) and added by the overflow number of 3(= 7 − 4). A remaining problem of containing zero can be simply resolved by extending lower exponent limit as small as possible for approximation, e.g. 10−10 0. As a summary, the proposed encoding/converting methods for BGA/RCGA are shown to be more effective than the conventional linear encoding/converting methods for the problems of ill-scaled parameters and of wide parameter bounds. The proposed methods will fall under a similar method with the linear encoding in narrow parameter bounds, which means that the proposed methods are to be considered as general. 5.
Conclusion
This paper proposes logarithmic and hybrid encodings/converting methods for BGA/RCGA considering identification of electrical systems whose physical parameters are the real numbers of different order. A simple probabilistic analysis is provided to investigate what occurrs in terms of random sampling as the search space is expanded. Function optimization and system identification results show that the use of proposed encoding/converting methods guarantee a robust optimization performance irrelevant to the size of entire search space. In the case of searching for real numbers, RCGA is preferable for its simple coding, fast computation, and various mutation and crossover operators. Our future work will focuss on improving crossover and mutation operators in RCGA, and applying to the identification of high-order realworld systems.
References [1] T. S¨oderstr¨om and P. Stoica, System Identification, Prentice-Hall, Englewood Cliffs, NJ, 1989. [2] J.-W. Kim, B. Park, S.C. Jeong, S.W. Kim, and P. Park, “Fault diagnosis of a power transformer using an improved frequency response analysis,” IEEE Trans. Power Deliv., vol.20, no.1, pp.169–178, Jan. 2005. [3] S.W. Sung and I.-B. Lee, “Prediction error identification method for continuous-time process with time delay,” J. Industrial and Engineering Chemistry Research, vol.40, pp.5743–5751, 2001. [4] J.-W. Kim and S.W. Kim, “A new approach to system identification using hybrid genetic algorithm,” International Conference on Control, Automation and Systems, pp.887–890, Korea, Oct. 2001. [5] F. Alonge, F. D’Ippolito, G. Ferrante, and F.M. Raimondi, “Parameter identification of induction motor model using genetic algorithms,” Proc. Inst. Elect. Eng. Contr. Theory Applicat., vol.145, no.6, pp.587–593, Nov. 1998. [6] D. Whitley, K. Mathias, and P. Fitzhorn, “Delta coding: An iterative search strategy for genetic algorithms,” Proc. Fourth Int. Conf. on Genetic Algorithms, pp.77–84, 1991. [7] Z. Michalewicz, Genetic algorithm + data structure = evolution program, Springer-Verlag, Heidelberg, Berlin, 1996. [8] G.-G. Jin and S.-R. Joo, “A study on a real-coded genetic algorithm,” J. Control, Automation, and Systems Engineering, vol.6, no.4, pp.268–274, April 2000. [9] L. Davis, Handbook of Genetic Algorithms, Von Nostrand Reinhold, N.Y., 1991. [10] C.Z. Janikow and Z. Michalewicz, “An experimental comparison of binary and floating point representations in genetic algorithms,” Proc. 4th Int. Conf. on Genetic Algorithm, pp.31–36, Morgan Kauffmann Publishers, CA, 1991. [11] H. M¨uhlenbein and D. Schlierkamp-Voosen, “Predictive models for the breeder genetic algorithm I. continuous parameter optimization,” Evolutionary Computation, vol.1, pp.25–49, 1993. [12] D.H. Wolpert and W.G. Macready, “No free lunch theorems for optimization,” IEEE Trans. Evol. Comput., vol.1, no.1, pp.67–82, April 1997. [13] X. Yao, Y. Liu, and G. Lin, “Evolutionary programming made faster,” IEEE Trans. Evol. Comput., vol.3, no.2, pp.82–102, July 1999. [14] J.G. Digalakis and K.G. Margaritis, “An experimental study of benchmarking functions for genetic algorithms,” IEEE International Conference on System, Man, and Cybernetics, vol.5, pp.3810–3815, 2000. [15] J.-M. Yang and C.Y. Kao, “A combined evolutionary algorithm for real parameter optimization,” Proc. IEEE International Conference on Evolutionary Computation, pp.732–737, 1996. [16] J.J. Mor´e, B.S. Garbow, and K.E. Hillstrom, “Testing unconstrained optimization software,” ACM Trans. Math. Softw., vol.7, no.1, pp.17–41, March 1981. [17] M.J.D. Powell, “A hybrid method for nonlinear equations,” in Numerical Methods for Nonlinear Algebraic Equations, ed. P. Rabinowitz, pp.87–114, Gordon & Breach, New York, 1970. [18] G. Rudolph, “Convergence analysis of canonical genetic algorithms,” IEEE Trans. Neural Netw., vol.5, no.1, pp.96–101, 1994. [19] M. Srinivas and L.M. Patnaik, “Adaptive probabilities of crossover and mutation in genetic algorithms,” IEEE Trans. Syst. Man Cybern., vol.24, no.4, pp.656–667, April 1994.
Appendix:
Calculation of δe and δ m in Hybrid Encoding of BGA
The resolution of an exponent and a mantissa, δe and δm ,
IEICE TRANS. FUNDAMENTALS, VOL.E88–A, NO.6 JUNE 2005
1564
respectively, can be calculated by boundary conditions. Assume p and q bits in GA are assigned, respectively, to the exponent and the mantissa of a real number x ∈ [xL , xU ], and eU = log10 xU and eL = log10 xL are upper and lower bound exponents of x. Since we fixed δe and δm to be constant, general expression of transformed real numbers using p bits for the exponent and q bits for the mantissa is given by x = (1 + jδm )10e +iδe , 0 ≤ i ≤ 2 p − 1, 0 ≤ j ≤ 2q − 1. L
(A· 1)
In (A· 1), the upper bound condition is satisfied if (1 + 2q δm )10e
L
+(2 p −1)δe
U
= 10e .
(A· 2)
Note in (A· 2) that 2q is (2q − 1) + 1, i.e., larger by 1 than the maximum integer which can be reached by q bits. This setting is specifically devised to avoid the overlap that occurs when the exponent is increased from iδe to (i + 1)δe : (1 + 2q δm )10e
L
+iδe
= 1 · 10e +(i+1)δe L = 10δe 10e +iδe . L
(A· 3)
From (A· 3), the following relation is obtained: (1 + 2q δm ) = 10δe .
(A· 4)
Inserting this into (A· 2) gives 10e
L
+2 p δe
U
= 10e ,
(A· 5)
thus, δe =
eU − e L . 2p
(A· 6)
Since δe is obtained, δm is easily calculated from (A· 4): δm =
10δe − 1 . 2q
(A· 7)
Jong-Wook Kim was born in Youngpoong, Korea, in 1970. He received the B.S., M.S., and Ph.D. degrees from the Electrical and Computer Engineering Division at Pohang University of Science and Technology (POSTECH), Pohang, Korea, in 1998, 2000, and 2004, respectively. Currently, he is a Researcher with the Electrical Steel Sheet Research Group in POSCO Technical Research Laboratories. His research interests include core design of transformers, diagnosis of electrical systems, intelligent control, optimization algorithm, and system identification.
Sang Woo Kim was born in Pyungtaek, Korea, in 1961. He received the B.S., M.S., and Ph.D. degrees from Department of Control and Instrumentation Engineering, Seoul National University, in 1983, 1985, and 1990, respectively. Currently, he is an Associate Professor in the Department of Electronics and Electrical Engineering at Pohang University of Science and Technology (POSTECH), Pohang, Korea. He joined POSTECH in 1992 as an Assistant Professor and was a Visiting Fellow in the Department of Systems Engineering, Australian National University, Canberra, Australia, in 1993. His current research interests are in optimal control, optimization algorithm, intelligent control, wireless communication, and process automation.
