MIMO beam-forming with neural network channel ... - Semantic Scholar
Missouri University of Science and Technology
Scholars' Mine Faculty Research & Creative Works
6-1-2008
MIMO beam-forming with neural network channel prediction trained by a novel PSO-EADEPSO algorithm Christopher Potter Ganesh K. Venayagamoorthy Missouri University of Science and Technology, [email protected]
Kurt Louis Kosbar Missouri University of Science and Technology, [email protected]
Follow this and additional works at: http://scholarsmine.mst.edu/faculty_work Part of the Electrical and Computer Engineering Commons Recommended Citation Potter, Christopher; Venayagamoorthy, Ganesh K.; and Kosbar, Kurt Louis, "MIMO beam-forming with neural network channel prediction trained by a novel PSO-EA-DEPSO algorithm" (2008). Faculty Research & Creative Works. Paper 867. http://scholarsmine.mst.edu/faculty_work/867
This Article - Conference proceedings is brought to you for free and open access by Scholars' Mine. It has been accepted for inclusion in Faculty Research & Creative Works by an authorized administrator of Scholars' Mine. For more information, please contact [email protected].
MIMO Beam-forming with Neural Network Channel Prediction Trained By a Novel PSO-EA-DEPSO Algorithm Chris Potter, Ganesh K. Venayagamoorthy, and Kurt Kosbar Abstract— A new hybrid algorithm based on particle swarm optimization (PSO), evolutionary algorithm (EA), and differential evolution (DE) is presented for training a recurrent neural network (RNN) for multiple-input multiple-output (MIMO) channel prediction. The hybrid algorithm is shown to be superior in performance to PSO and differential evolution PSO (DEPSO) for different channel environments. The received signal-to-noise ratio is derived for un-coded and beam-forming MIMO systems to see how the RNN error affects the performance. This error is shown to deteriorate the accuracy of the weak singular modes, making beam-forming more desirable. Bit error rate simulations are performed to validate these results.
I. I NTRODUCTION Multiple-input multiple-output (MIMO) systems have been shown to provide significant gains in both spectral efficiency and reliability [1]. This is based on the assumption that the receiver and transmitter have knowledge of the channel coefficients. In reality they must either be estimated or predicted. A few popular ways to estimate the channel are by using pilot symbols [2] and space time block codes (STBC) [3]. Both methods waste time learning the channel when meaningful data can be sent. Channel prediction does not suffer from these aforementioned setbacks. Unlike the use of conventional prediction techniques such as [4], a recurrent neural network (RNN) is used for prediction. Neural networks have the ability of being robust to different wireless channels as long as they are trained properly. In [5] an extended Kalman filter (EKF) was employed for training a RNN for time series prediction. A hybrid particle swarm optimization evolutionary algorithm (PSOEA) was utilized in [6] for time series. In this work, a novel hybrid algorithm composed of PSO, EA, and differential evolution (DE) is proposed for MIMO channel prediction. It is shown that this hybrid algorithm outperforms both PSO and DEPSO. Also, beam-forming is shown to be ideal for RNN predicted MIMO channels, since the strongest singular mode is more accurately predicted than the weaker ones. These results are verified with bit error rate (BER) simulations The rest of this paper is organized as follows. The next section describes the MIMO received symbol model for the Chris Potter is a PhD candidate in Electrical Engineering at the Missouri University of Science and Technology, Rolla, MO 65409 USA (email: [email protected]) Ganesh K. Venayagamoorthy is an Associate Professor and Director of the Real-Time Power and Intelligent Systems Laboratory, Department of Electrical and Computer Engineering, Missouri University of Science and Technology, Rolla, MO 65409 USA (e-mail: [email protected]) . Kurt Kosbar is an Associate Professor in Electrical Engineering at the Missouri University of Science and Technology, Rolla, MO 65409 USA (email: [email protected])
un-coded and beam-forming systems. This is followed by the fast fading channel representation. After this the RNN used for MIMO channel prediction is introduced . This is followed by the proposal of a novel PSO-EA-DE hybrid training algorithm. The received SNR for both systems are then derived and the prediction error is analyzed. BER simulations are then performed. This is followed by our concluding remarks. II. MIMO R ECEIVED M ODEL In this section, the received symbols for the un-coded and beam-forming systems are introduced. For the beam-forming case the received symbols are expressed in two scenarios, when the transmitter and receiver have full channel state information (CSI) and when they have the prediction matrix. A. Un-coded A MIMO wireless flat fading communication system with Nr receive antennas and Nt transmit antennas is modeled by y = Hx + n
(1)
where y is the Nr × 1 received vector, x is the Nt × 1 transmitted symbol vector with each xi belonging to constellation C with symbol energy Es , and n is the white noise iid vector of size Nr × 1 with ni ∼ CN (0, No ). The Nr × Nt channel matrix H = {hmn } describes the complex channel gain between the mth receiver antenna and the nth transmit antenna. B. Beam-forming Let H = U DV H be the singular value decomposition (SVD) where U and V are unitary matrices and ui and v i , are the left and right singular vectors corresponding to the ith non-zero singular value σH (i). Note that σH (1) ≤ . . . ≤ σH (M ), where M = rank(H). If x ˜ = v 1 x and one post-multiplies (1) by uH 1 the received symbol is H uH 1 y = σH (1)x + u1 n.
Letting n ˜=
uH 1 n
(2)
one can easily show that E|˜ n |2 = N r N o
(3)
and remains white. It has been shown [7] that this technique maximizes the received SNR for a single mode. When the transmitter and receiver have the prediction =U D V H , then by writing the MIMO channel matrix H + E the received symbol is written as as H = H H H v 1 )x + uH u 1 n. 1 y = (σH (1) + u 1 E
3338 c 978-1-4244-1821-3/08/$25.002008 IEEE
Authorized licensed use limited to: University of Missouri. Downloaded on December 2, 2008 at 10:12 from IEEE Xplore. Restrictions apply.
(4)
III. C HANNEL M ODEL When the doppler spread is greater than the pulse bandwidth, the MIMO channel undergoes fast fading [8]. In this work a MIMO flat fast fading wireless environment is employed. Each sub-channel is represented by [9] hmn (k) = hImn (k) + jhQ mn (k) where
hImn (k)
=
M 2 cos(2πfd k cos(αn ) + φn ) M n=1
is the in-phase component, M 2 hQ (k) = cos(2πfd k sin(αn ) + ψn ) mn M n=1
(5)
(6)
z−1 z−1
d1(k −1)
d2(k−1) h(k − 1)
h(k − N p )
A
is the quadrature component, and 2πn − π + θ αn = (8) 4M where φn , ψn , and θ are U [−π, π). The coefficients satisfy the first and second order statistics of Clarke’s reference model [9] IV. R ECURRENT N EURAL N ETWORK To accurately predict a wireless channel, it is necessary to have past values of the channel to learn the statistics of the fading process. It has become customary to use a linear filter to perform the prediction. This method requires the receiver know the statistics of the channel which greatly depends on the random process at hand. For example, the linear prediction filters for an autoregressive (AR) process and an autoregressive moving average (ARMA) process are quite different. When the wireless environment changes, the receiver has to redetermine the statistics of the environment and subsequently change the prediction model. When trained properly the neural network is robust to any wireless environment. The neural network used for prediction is shown in Figure 1. For each run, the RNN consists of two stages. In the first only the previous Np channel coefficients are used, thus d1 (k − 1) = d2 (k − 1) = 0. For the second, the RNN uses both the past channel coefficients and the previous h(k) represents the two outputs. The output of d1 (k) = prediction. All neurons are fully connected in the forward direction. The non-linear activation functions are tansig(·) which can be described by tansig(x) =
exp(x) − exp(−x) . exp(x) + exp(−x)
(9)
The output of the activation functions are written as dj = tansig
N p +2
aji si ,
j = 1, . . . , 2
d2(k)
+
• • •
(7)
d1(k) hˆ(k )
+
(10)
i=1
where
Fig. 1.
Neural Network used for channel prediction.
In matrix form this is represented as d = f (As)
(12)
where A=
a1,1 a2,1
. . . a1,(Np +2) . . . a2,(Np +2)
(13)
is a 2×(Np +2) matrix whose entries are the neural network weights and ⎤ ⎡ tansig(·) ⎥ ⎢ .. (14) f =⎣ ⎦ . tansig(·) is a 2 × 1 vector that evaluates the tansig of each component. The total number of weights are Nw = (Np +2)·2 = 2Np +4. V. T RAINING A LGORITHMS To obtain useful channel predictions the RNN must be trained. Before proceeding to the proposed hybrid algorithm, PSO, EA, and DEPSO are briefly summarized. A. PSO PSO is a evolutionary computation technique developed by Eberhart and Kennedy in 1995. It was inspired by swarm intelligence where a collection of unsophisticated individuals (particles) can solve complex problems by interacting with one another. Some examples of this behavior are a flock of birds or a school of fish. Although simple to implement, PSO has been shown to be an algorithm that when used properly can perform multi-parameter optimization [10]. Some applications of PSO include artificial life, social psychology, engineering, and computer science. The population of particles fly through the solution space with certain velocities.
s = [d1 (k − 1) d2 (k − 1) h(k − 1) · · · h(k − Np )]T . (11)
2008 International Joint Conference on Neural Networks (IJCNN 2008)
Authorized licensed use limited to: University of Missouri. Downloaded on December 2, 2008 at 10:12 from IEEE Xplore. Restrictions apply.
3339
While not Converged
While not Converged Evaluate Fitness
PSO Y Done!
Evaluate Fitness
DEPSO
EA-PSO
Winners
Losers
PSO
Eliminate EA
Converged?
N Fig. 2.
New Population Fig. 3.
New PSO-EA-DEPSO hybrid training algorithm.
Let P be the number of particles with alphabet P = {1, . . . , P } and D the dimension of each particle. At each epoch the position and velocity components for the ith particle and dth dimension are updated according to [11]
vid = wvid
+ c1 rand1 (·)(pbestid − xid ) + c2 rand2 (·)(gbestd − xid )
(15)
where xid = xid + vid
PSO-EA algorithm.
TABLE I PARAMETERS VALUES FOR PROPOSED HYBRID PSO-EA-DEPSO ALGORITHM
Parameter Vmax Xmax w c1 c2 P Pc δN τ
Value 2 4 .8 2 2 40 0.5 δ7 .3265
Description Maximum PSO velocity Maximum PSO position PSO Inertia Weight PSO Cognitive Weight PSO Social Weight Number of PSO Particles Crossover Probability for DEPSO DEPSO operator EA Parameter
(16)
and w, c1 , and c2 are the inertia and acceleration constants respectively. B. EA EA algorithm is evolving the population through mutation and selection operations. Each parent or offspring is represented as a chromosome which is made up of genes which represent characteristics of the individual. In terms of optimization, a gene represents a parameter such as a neural network weight. Given a population N of neural networks, for every generation each parent Pi , n = 1, . . . , N has a Nw ×1 self-adaptive parameter vector σ i , i = 1, . . . , N . Each parent generates an offspring P´i whose Nw × 1 self-adaptive ´ i is updated according to parameter vector σ
C. DEPSO DEPSO is a hybrid of DE and PSO which provides diversity on the population while keeping the swarm searching capabilities intact [12]. The pbest of each particle is updated by IF(rand(·) < Pc )OR(i == k) THEN pbestid = gbestd + ΔN
(19)
where k ∈ P is chosen randomly and ΔN =
N 1 pbestAj − pbestBj N j=1
(20)
with A, B ∈ P. D. A Hybrid PSO-EA-DEPSO Algorithm
σ ´i (j) = σi (j) exp(τ N (0, 1)) , j = 1, . . . , Nw .
(17)
The weights are updated according to w ´i (j) = wi (j) + σ ´i (j)N (0, 1) , j = 1, . . . , Nw √ where τ = 1/ 2 Nw .
3340
(18)
In this work, a new algorithm is proposed that is a hybrid version of PSO, EA, and DEPSO. The block diagram is displayed in Figure 2. The idea behind the algorithm is to alternate between PSO, EA, and DEPSO to continually provide diversity for the particles/parents. This in theory prevents the particles/parents from reaching a premature convergence. The PSO algorithm is implemented on odd
2008 International Joint Conference on Neural Networks (IJCNN 2008)
Authorized licensed use limited to: University of Missouri. Downloaded on December 2, 2008 at 10:12 from IEEE Xplore. Restrictions apply.
To show the prediction capability of the RNN trained by the proposed hybrid PSO-EA-DEPSO algorithm, a numerical example is provided. In the first experiment a Rayleigh flat fast fading 2 × 2 MIMO channel with fd Ts = 0.05 is generated. Since the sub-channels are uncorrelated, to save space and redundancy the in-phase component of h11 (k) is only considered. Looking at Figures 4 and 5, it is obvious that the proposed algorithm is superior to PSO and DEPSO. For the second experiment fd Ts = 0.1. Once again, Figures 6 and 7 validate that the proposed algorithm is superior. To quantify these observations, the MSE between the prediction and actual coefficients for each algorithm is displayed in Table II. The MSE of the hybrid algorithm for both channel environments is significantly lower than the others.
0.8 0.6 0.4 Amplitude
VI. T RAINING THE RNN The RNN in Figure 1 is trained by using the proposed PSO-EA-DEPSO algorithm for each MIMO sub-channel. The in-phase and quadrature components are trained separately. For each prediction, Np = 9 and the previous 10 channel coefficients and their respective predictions are used for the fitness function 10 1 (hmn − hmn )2 . (21) M SE(k) = 10 i=1
Actual PSO DEPSO
1
0.2 0 −0.2 −0.4 −0.6 −0.8
0
50
100
150
Samples
Fig. 4. Comparison of PSO and DEPSO predictions with the actual inphase channel coefficients for fd Ts = 0.05.
Actual PSO−EA−DEPSO
1 0.8 0.6 0.4 Amplitude
iterations while PSO-EA and DEPSO alternate on even iterations. For PSO, the velocity and position weights are all initialized from a uniform distribution. The pseudo-code for the PSO-EA algorithm is displayed in Figure 3. For more details of the PSO-EA algorithm, the reader is referred to [6]. The parameters for the proposed hybrid algorithm are displayed in Table I.
0.2 0 −0.2 −0.4 −0.6
VII. R ECEIVED SNR FOR D IFFERENT MIMO S CHEMES For a MIMO system using channel prediction, the received SNR takes on the general form ρ=
σx2 σe2 + σn2
(22)
where σx2 is the average received signal power, σe2 is the prediction error, and σn2 is the average noise variance. The received SNR for the un-coded system is defined as ρuc
2 E||Hx|| 2 . E||Ex||22 + E||n||22
E{trace(·)}
= trace(AH bbH A)
(24)
= trace(E{·})
(25)
we obtain after several manipulations M 2 i=1 E{σH (i)} ρuc = N Nr N o 2 i=1 E{σE (i)} + Es
0
th non-zero singular values where σH (i) and σE (i) are the i of H and E respectively, and N = rank(E).
100
150
Fig. 5. Comparison of new hybrid PSO-EA-DPSO with the actual in-phase channel coefficients for fd Ts = 0.05.
Recall that for the MIMO beam-forming system the received symbol was H H σ12 + u v 1 )x + n ˜. u 1 y = ( 1 E
(27)
Using similar techniques as before, the received SNR for the beam-forming case can be written as
ρbf = (26)
50 Samples
(23)
Using the independence assumption with x and H along with the identities ||bH AAH b||22
−0.8
E{σ12 } H 1 E| uH 1 U DV v
−σ max |2 +
Nr No Es
.
(28)
When comparing (26) with (28) one can see although there is less signal power in the beam-forming case, the prediction error has the capability of being significantly less. This can be seen by writing the prediction error in (23) as
2008 International Joint Conference on Neural Networks (IJCNN 2008)
Authorized licensed use limited to: University of Missouri. Downloaded on December 2, 2008 at 10:12 from IEEE Xplore. Restrictions apply.
3341
16 Actual PSO DEPSO
1
0.6
12
0.4
10
Received SNR
Amplitude
0.8
Actual Pred.
14
0.2 0
Beamforming
8 Uncoded 6
−0.2 4 −0.4 2
−0.6 −0.8
0
50
100
150
0
0
5
Samples
s
Fig. 6. Comparison of PSO, and DEPSO predictions with the actual inphase channel coefficients for fd Ts = 0.1.
0.8 0.6
Amplitude
0.4 0.2 0 −0.2 −0.4 −0.6
0
50
100
150
Samples
Fig. 7. Comparison of new hybrid PSO-EA-DPSO with the actual in-phase channel coefficients for fd Ts = 0.1. TABLE II M EAN SQUARED ERROR COMPARISON OF PSO, DEPSO, AND PSO-EA-DEPSO ALGORITHMS WITH RESPECT TO ACTUAL CHANNEL COEFFICIENTS . fd Ts = .05 fd Ts = .1
σe2 = E||Ex||22
PSO 1.2212 21.8440
= =
=
DEPSO 2.0811 1.2472
PSO − EA − DEPSO 0.0712 0.1183
H
Es
2 E{σE (i)}
(29)
i=1
where the last equality results from observing (26). Comparing this to the beam-forming prediction error
3342
20
Fig. 8. Received SNR comparison for un-coded and beam-forming MIMO systems.
(30)
it is clear that if u1 , v 1 , and σ1 are well predicted but the other modes are not, a smaller prediction error will result. This is implicitly true for the channel matrices predicted by the RNN. To show this behavior a numerical example is given. The received SNR using (26) and (28) is tabulated for 10000 binary phase shift keying (BPSK) symbol vectors with Es = 1 and Nt = Nr = 2. The results are displayed in Figure 8. As one can see the received SNR for the beam-forming case is significantly better at higher Es /No , suggesting that the stronger singular mode is more accurately 2 2 (1) and σE (2) are calculated predicted. To justify this σE and graphed in Figures 9 and 10. Clearly there are samples 2 2 2 2 (2) σE (1) suggesting E{σE (2)} ≥ E{σE (1)}. where σE 2 Comparing the two values we have E{σE (1)} ≈ 3 × 10−3 2 and E{σE (2)} ≈ 6 × 10−3 , indicating that the stronger singular value is predicted better by a factor of two. From this observation one can see that beam-forming provides two advantages over the un-coded system. The first 2 (1) affects the prediction error. The second is is only σE 2 (2)} > when the RNN predicts the MIMO channel, E{σE 2 (1)} which does not affect the beam-forming case since E{σE 2 (1) affects the received SNR only σE VIII. BER C OMPARISON
E V x||2 E||U 2 M H 2 H 1 − σ i U DV v E i xi u i=1 M
15 o
H 2 1 − σ 1 U DV H v σe2 = E u max x
Actual PSO−EA−DEPSO
1
−0.8
10 E /N
Before showing the difference in BER performance between the two MIMO systems, we briefly review the decoding procedures. A. Vector ML Detection For the MIMO un-coded system the received symbols are decoded according to 2. = arg min ||y − Hx|| x 2 x∈C Nt
This is known as the nearest neighbor decoding rule.
2008 International Joint Conference on Neural Networks (IJCNN 2008)
Authorized licensed use limited to: University of Missouri. Downloaded on December 2, 2008 at 10:12 from IEEE Xplore. Restrictions apply.
(31)
0
0.3
10 −3
5 Amplitude
Actual Prediction
2 E
E{σ (1)}
4
0.25
0.2
2 E
σ (1)
x 10
−1
3
10
2
0.15
0 800
900 Samples
BER
Amplitude
1 1000
−2
10
0.1 −3
10 0.05
−4
0 800
850
Fig. 9.
900 Samples
950
10
1000
5
10 E /N s
Maximum singular value of error matrix.
Fig. 11.
15
20
o
BER for a un-coded 2 × 2 MIMO fast fading channel.
−1
0.3
10 −3
2 E
2 E
8 Amplitude
0.25
Actual Prediction
σ (2)
x 10
0.2
0
E{σ (2)}
6 4
−2
10
0.15
0 800
900 Samples
BER
Amplitude
2 1000
−3
0.1
10
0.05
−4
0 800
850
Fig. 10.
900 Samples
950
1000
Fig. 12.
B. Beam-forming When beam-forming is performed at the transmitter and receiver the received symbol is found by x∈C
0
2
4
6
8 E /N s
Minimum singular value of error matrix.
uH H (1)x|2 . x = arg min | 1 y−σ
10
(32)
C. BER Simulations If the received SNR is large for a given Es /No then one expects the BER to be low. From the previous section it was established that the prediction error had a significant impact on the received SNR. By observing Figure 8 some observations about the BER can be inferred. One would expect that at low Es /No the prediction BER for the uncoded MIMO system will be comparable to the actual BER. As Es /No is increased, the received SNR does not increase as fast as the error free received SNR, which should cause the prediction BER to deteriorate. To illustrate this the BER was calculated via Monte Carlo simulations for a 2 x 2 MIMO flat fast fading channel using 10000 BPSK symbols for which
10
12
14
16
o
BER for a 2 × 2 MIMO beam-forming fast fading channel.
Es = 1. Looking at Figure 11, at low Es /No the prediction BER agrees nicely with the error free BER. But as Es /No increases the prediction BER begins to level off, since the effective noise floor due to prediction error is significant. This verifies the results in the previous section describing how the 2 (2) negatively affects the received SNR. addition of σE For the beam-forming case the received SNR remains close to the ideal SNR throughout the range of Es /No . This suggests the the BER for both the ideal and predicted channel will be close for all Es /No . To illustrate this the BER is calculated for a 2 × 2 MIMO beam-forming system using the same parameters as the un-coded case. As one can see the predicted BER differs only slightly from the error free BER. This verifies our observation that the strong singular mode is more accurately predicted by the RNN. IX. C ONCLUSION A recursive neural network trained by a novel PSO-EADEPSO was used to predict a MIMO channel. This training
2008 International Joint Conference on Neural Networks (IJCNN 2008)
Authorized licensed use limited to: University of Missouri. Downloaded on December 2, 2008 at 10:12 from IEEE Xplore. Restrictions apply.
3343
algorithm was shown to be superior to both PSO and DEPSO for two different fast fading scenarios. The received SNR for the un-coded and beam-forming MIMO systems was derived. The beam-forming technique was shown to be superior to the un-coded case for two reasons. The first was that only the dominant mode plays a role in the prediction error. The second is that the predicted channel predicts the dominant mode better than the minor ones. These two reasons lead to a higher received SNR and lower BER than the un-coded case, making the beam-forming system ideal. R EFERENCES [1] E. Telatar, “Capacity of multi-antenna gaussian channels,” AT&T Bell Laboratories Internal Tech. Memo., Jun. 1995. [2] H. Huang, “Spatial channel model for multiple input multiple output (mimo) simulations,” 3rd Generation Partnership Project Technical Report 25.996, V6.0.0, 2003. [3] D. Gesbert et al., “From theory to practice: An overview of mimo space-time coded wireless systems,” IEEE J. Select. Areas Commun., vol. 21, no. 3, pp. 281–302, Apr. 2003. [4] G. Bauch, “Mimo capacity loss for real world signal constellations and channel degradations,” Proc. IEEE Personal, Indoor and Mobile Radio Communications Conference (PIMRC’04), vol. 2, pp. 1439–1443, Sep. 2004. [5] A. Jazwinski, Stochastic Process and Filtering Theory. Acadamic Press, 1993. [6] X. Cai, N. Zhang, G. K. Venayagamoorthy, and D. C. Wunsch, “Time series prediction with recurrent neural networks trained by a hybrid pso-ea algorithm,” Neurocomputing, vol. 70, pp. 2342–2353, Aug. 2007. [7] A. Goldsmith, Wireless Communications. New York: Cambridge University Press, 2005. [8] T. S. Rappaport, Wireless Communications: Principles and Practice. Upper Saddle River,NJ: Prentice Hall, 2002. [9] Y. Zheng and C. Xiao, “Improved models for the generation of multiple uncorrelated rayleigh fading waveforms,” IEEE Commun. Lett., vol. 6, no. 6, 2002. [10] Y. Valle et al., “Particle swarm optimization: Basic concepts, variants and applications in power systems,” 2007. [11] J. Kennedy and R. Eberhart, “Particle swarm optimization,” IEEE Int. Conf. Neural Networks 4, vol. 49, pp. 1942–1948, 1995. [12] R. Xu, G. K. Venayagamoorthy, and D. C. Wunsch, “Modeling of gene regulatory networks with hybrid differential evolution and particle swarm optimization,” Neural Networks Journal, vol. 21, pp. 917–927, Oct. 2007.
3344
2008 International Joint Conference on Neural Networks (IJCNN 2008)
Authorized licensed use limited to: University of Missouri. Downloaded on December 2, 2008 at 10:12 from IEEE Xplore. Restrictions apply.
Scholars' Mine Faculty Research & Creative Works
6-1-2008
MIMO beam-forming with neural network channel prediction trained by a novel PSO-EADEPSO algorithm Christopher Potter Ganesh K. Venayagamoorthy Missouri University of Science and Technology, [email protected]
Kurt Louis Kosbar Missouri University of Science and Technology, [email protected]
Follow this and additional works at: http://scholarsmine.mst.edu/faculty_work Part of the Electrical and Computer Engineering Commons Recommended Citation Potter, Christopher; Venayagamoorthy, Ganesh K.; and Kosbar, Kurt Louis, "MIMO beam-forming with neural network channel prediction trained by a novel PSO-EA-DEPSO algorithm" (2008). Faculty Research & Creative Works. Paper 867. http://scholarsmine.mst.edu/faculty_work/867
This Article - Conference proceedings is brought to you for free and open access by Scholars' Mine. It has been accepted for inclusion in Faculty Research & Creative Works by an authorized administrator of Scholars' Mine. For more information, please contact [email protected].
MIMO Beam-forming with Neural Network Channel Prediction Trained By a Novel PSO-EA-DEPSO Algorithm Chris Potter, Ganesh K. Venayagamoorthy, and Kurt Kosbar Abstract— A new hybrid algorithm based on particle swarm optimization (PSO), evolutionary algorithm (EA), and differential evolution (DE) is presented for training a recurrent neural network (RNN) for multiple-input multiple-output (MIMO) channel prediction. The hybrid algorithm is shown to be superior in performance to PSO and differential evolution PSO (DEPSO) for different channel environments. The received signal-to-noise ratio is derived for un-coded and beam-forming MIMO systems to see how the RNN error affects the performance. This error is shown to deteriorate the accuracy of the weak singular modes, making beam-forming more desirable. Bit error rate simulations are performed to validate these results.
I. I NTRODUCTION Multiple-input multiple-output (MIMO) systems have been shown to provide significant gains in both spectral efficiency and reliability [1]. This is based on the assumption that the receiver and transmitter have knowledge of the channel coefficients. In reality they must either be estimated or predicted. A few popular ways to estimate the channel are by using pilot symbols [2] and space time block codes (STBC) [3]. Both methods waste time learning the channel when meaningful data can be sent. Channel prediction does not suffer from these aforementioned setbacks. Unlike the use of conventional prediction techniques such as [4], a recurrent neural network (RNN) is used for prediction. Neural networks have the ability of being robust to different wireless channels as long as they are trained properly. In [5] an extended Kalman filter (EKF) was employed for training a RNN for time series prediction. A hybrid particle swarm optimization evolutionary algorithm (PSOEA) was utilized in [6] for time series. In this work, a novel hybrid algorithm composed of PSO, EA, and differential evolution (DE) is proposed for MIMO channel prediction. It is shown that this hybrid algorithm outperforms both PSO and DEPSO. Also, beam-forming is shown to be ideal for RNN predicted MIMO channels, since the strongest singular mode is more accurately predicted than the weaker ones. These results are verified with bit error rate (BER) simulations The rest of this paper is organized as follows. The next section describes the MIMO received symbol model for the Chris Potter is a PhD candidate in Electrical Engineering at the Missouri University of Science and Technology, Rolla, MO 65409 USA (email: [email protected]) Ganesh K. Venayagamoorthy is an Associate Professor and Director of the Real-Time Power and Intelligent Systems Laboratory, Department of Electrical and Computer Engineering, Missouri University of Science and Technology, Rolla, MO 65409 USA (e-mail: [email protected]) . Kurt Kosbar is an Associate Professor in Electrical Engineering at the Missouri University of Science and Technology, Rolla, MO 65409 USA (email: [email protected])
un-coded and beam-forming systems. This is followed by the fast fading channel representation. After this the RNN used for MIMO channel prediction is introduced . This is followed by the proposal of a novel PSO-EA-DE hybrid training algorithm. The received SNR for both systems are then derived and the prediction error is analyzed. BER simulations are then performed. This is followed by our concluding remarks. II. MIMO R ECEIVED M ODEL In this section, the received symbols for the un-coded and beam-forming systems are introduced. For the beam-forming case the received symbols are expressed in two scenarios, when the transmitter and receiver have full channel state information (CSI) and when they have the prediction matrix. A. Un-coded A MIMO wireless flat fading communication system with Nr receive antennas and Nt transmit antennas is modeled by y = Hx + n
(1)
where y is the Nr × 1 received vector, x is the Nt × 1 transmitted symbol vector with each xi belonging to constellation C with symbol energy Es , and n is the white noise iid vector of size Nr × 1 with ni ∼ CN (0, No ). The Nr × Nt channel matrix H = {hmn } describes the complex channel gain between the mth receiver antenna and the nth transmit antenna. B. Beam-forming Let H = U DV H be the singular value decomposition (SVD) where U and V are unitary matrices and ui and v i , are the left and right singular vectors corresponding to the ith non-zero singular value σH (i). Note that σH (1) ≤ . . . ≤ σH (M ), where M = rank(H). If x ˜ = v 1 x and one post-multiplies (1) by uH 1 the received symbol is H uH 1 y = σH (1)x + u1 n.
Letting n ˜=
uH 1 n
(2)
one can easily show that E|˜ n |2 = N r N o
(3)
and remains white. It has been shown [7] that this technique maximizes the received SNR for a single mode. When the transmitter and receiver have the prediction =U D V H , then by writing the MIMO channel matrix H + E the received symbol is written as as H = H H H v 1 )x + uH u 1 n. 1 y = (σH (1) + u 1 E
3338 c 978-1-4244-1821-3/08/$25.002008 IEEE
Authorized licensed use limited to: University of Missouri. Downloaded on December 2, 2008 at 10:12 from IEEE Xplore. Restrictions apply.
(4)
III. C HANNEL M ODEL When the doppler spread is greater than the pulse bandwidth, the MIMO channel undergoes fast fading [8]. In this work a MIMO flat fast fading wireless environment is employed. Each sub-channel is represented by [9] hmn (k) = hImn (k) + jhQ mn (k) where
hImn (k)
=
M 2 cos(2πfd k cos(αn ) + φn ) M n=1
is the in-phase component, M 2 hQ (k) = cos(2πfd k sin(αn ) + ψn ) mn M n=1
(5)
(6)
z−1 z−1
d1(k −1)
d2(k−1) h(k − 1)
h(k − N p )
A
is the quadrature component, and 2πn − π + θ αn = (8) 4M where φn , ψn , and θ are U [−π, π). The coefficients satisfy the first and second order statistics of Clarke’s reference model [9] IV. R ECURRENT N EURAL N ETWORK To accurately predict a wireless channel, it is necessary to have past values of the channel to learn the statistics of the fading process. It has become customary to use a linear filter to perform the prediction. This method requires the receiver know the statistics of the channel which greatly depends on the random process at hand. For example, the linear prediction filters for an autoregressive (AR) process and an autoregressive moving average (ARMA) process are quite different. When the wireless environment changes, the receiver has to redetermine the statistics of the environment and subsequently change the prediction model. When trained properly the neural network is robust to any wireless environment. The neural network used for prediction is shown in Figure 1. For each run, the RNN consists of two stages. In the first only the previous Np channel coefficients are used, thus d1 (k − 1) = d2 (k − 1) = 0. For the second, the RNN uses both the past channel coefficients and the previous h(k) represents the two outputs. The output of d1 (k) = prediction. All neurons are fully connected in the forward direction. The non-linear activation functions are tansig(·) which can be described by tansig(x) =
exp(x) − exp(−x) . exp(x) + exp(−x)
(9)
The output of the activation functions are written as dj = tansig
N p +2
aji si ,
j = 1, . . . , 2
d2(k)
+
• • •
(7)
d1(k) hˆ(k )
+
(10)
i=1
where
Fig. 1.
Neural Network used for channel prediction.
In matrix form this is represented as d = f (As)
(12)
where A=
a1,1 a2,1
. . . a1,(Np +2) . . . a2,(Np +2)
(13)
is a 2×(Np +2) matrix whose entries are the neural network weights and ⎤ ⎡ tansig(·) ⎥ ⎢ .. (14) f =⎣ ⎦ . tansig(·) is a 2 × 1 vector that evaluates the tansig of each component. The total number of weights are Nw = (Np +2)·2 = 2Np +4. V. T RAINING A LGORITHMS To obtain useful channel predictions the RNN must be trained. Before proceeding to the proposed hybrid algorithm, PSO, EA, and DEPSO are briefly summarized. A. PSO PSO is a evolutionary computation technique developed by Eberhart and Kennedy in 1995. It was inspired by swarm intelligence where a collection of unsophisticated individuals (particles) can solve complex problems by interacting with one another. Some examples of this behavior are a flock of birds or a school of fish. Although simple to implement, PSO has been shown to be an algorithm that when used properly can perform multi-parameter optimization [10]. Some applications of PSO include artificial life, social psychology, engineering, and computer science. The population of particles fly through the solution space with certain velocities.
s = [d1 (k − 1) d2 (k − 1) h(k − 1) · · · h(k − Np )]T . (11)
2008 International Joint Conference on Neural Networks (IJCNN 2008)
Authorized licensed use limited to: University of Missouri. Downloaded on December 2, 2008 at 10:12 from IEEE Xplore. Restrictions apply.
3339
While not Converged
While not Converged Evaluate Fitness
PSO Y Done!
Evaluate Fitness
DEPSO
EA-PSO
Winners
Losers
PSO
Eliminate EA
Converged?
N Fig. 2.
New Population Fig. 3.
New PSO-EA-DEPSO hybrid training algorithm.
Let P be the number of particles with alphabet P = {1, . . . , P } and D the dimension of each particle. At each epoch the position and velocity components for the ith particle and dth dimension are updated according to [11]
vid = wvid
+ c1 rand1 (·)(pbestid − xid ) + c2 rand2 (·)(gbestd − xid )
(15)
where xid = xid + vid
PSO-EA algorithm.
TABLE I PARAMETERS VALUES FOR PROPOSED HYBRID PSO-EA-DEPSO ALGORITHM
Parameter Vmax Xmax w c1 c2 P Pc δN τ
Value 2 4 .8 2 2 40 0.5 δ7 .3265
Description Maximum PSO velocity Maximum PSO position PSO Inertia Weight PSO Cognitive Weight PSO Social Weight Number of PSO Particles Crossover Probability for DEPSO DEPSO operator EA Parameter
(16)
and w, c1 , and c2 are the inertia and acceleration constants respectively. B. EA EA algorithm is evolving the population through mutation and selection operations. Each parent or offspring is represented as a chromosome which is made up of genes which represent characteristics of the individual. In terms of optimization, a gene represents a parameter such as a neural network weight. Given a population N of neural networks, for every generation each parent Pi , n = 1, . . . , N has a Nw ×1 self-adaptive parameter vector σ i , i = 1, . . . , N . Each parent generates an offspring P´i whose Nw × 1 self-adaptive ´ i is updated according to parameter vector σ
C. DEPSO DEPSO is a hybrid of DE and PSO which provides diversity on the population while keeping the swarm searching capabilities intact [12]. The pbest of each particle is updated by IF(rand(·) < Pc )OR(i == k) THEN pbestid = gbestd + ΔN
(19)
where k ∈ P is chosen randomly and ΔN =
N 1 pbestAj − pbestBj N j=1
(20)
with A, B ∈ P. D. A Hybrid PSO-EA-DEPSO Algorithm
σ ´i (j) = σi (j) exp(τ N (0, 1)) , j = 1, . . . , Nw .
(17)
The weights are updated according to w ´i (j) = wi (j) + σ ´i (j)N (0, 1) , j = 1, . . . , Nw √ where τ = 1/ 2 Nw .
3340
(18)
In this work, a new algorithm is proposed that is a hybrid version of PSO, EA, and DEPSO. The block diagram is displayed in Figure 2. The idea behind the algorithm is to alternate between PSO, EA, and DEPSO to continually provide diversity for the particles/parents. This in theory prevents the particles/parents from reaching a premature convergence. The PSO algorithm is implemented on odd
2008 International Joint Conference on Neural Networks (IJCNN 2008)
Authorized licensed use limited to: University of Missouri. Downloaded on December 2, 2008 at 10:12 from IEEE Xplore. Restrictions apply.
To show the prediction capability of the RNN trained by the proposed hybrid PSO-EA-DEPSO algorithm, a numerical example is provided. In the first experiment a Rayleigh flat fast fading 2 × 2 MIMO channel with fd Ts = 0.05 is generated. Since the sub-channels are uncorrelated, to save space and redundancy the in-phase component of h11 (k) is only considered. Looking at Figures 4 and 5, it is obvious that the proposed algorithm is superior to PSO and DEPSO. For the second experiment fd Ts = 0.1. Once again, Figures 6 and 7 validate that the proposed algorithm is superior. To quantify these observations, the MSE between the prediction and actual coefficients for each algorithm is displayed in Table II. The MSE of the hybrid algorithm for both channel environments is significantly lower than the others.
0.8 0.6 0.4 Amplitude
VI. T RAINING THE RNN The RNN in Figure 1 is trained by using the proposed PSO-EA-DEPSO algorithm for each MIMO sub-channel. The in-phase and quadrature components are trained separately. For each prediction, Np = 9 and the previous 10 channel coefficients and their respective predictions are used for the fitness function 10 1 (hmn − hmn )2 . (21) M SE(k) = 10 i=1
Actual PSO DEPSO
1
0.2 0 −0.2 −0.4 −0.6 −0.8
0
50
100
150
Samples
Fig. 4. Comparison of PSO and DEPSO predictions with the actual inphase channel coefficients for fd Ts = 0.05.
Actual PSO−EA−DEPSO
1 0.8 0.6 0.4 Amplitude
iterations while PSO-EA and DEPSO alternate on even iterations. For PSO, the velocity and position weights are all initialized from a uniform distribution. The pseudo-code for the PSO-EA algorithm is displayed in Figure 3. For more details of the PSO-EA algorithm, the reader is referred to [6]. The parameters for the proposed hybrid algorithm are displayed in Table I.
0.2 0 −0.2 −0.4 −0.6
VII. R ECEIVED SNR FOR D IFFERENT MIMO S CHEMES For a MIMO system using channel prediction, the received SNR takes on the general form ρ=
σx2 σe2 + σn2
(22)
where σx2 is the average received signal power, σe2 is the prediction error, and σn2 is the average noise variance. The received SNR for the un-coded system is defined as ρuc
2 E||Hx|| 2 . E||Ex||22 + E||n||22
E{trace(·)}
= trace(AH bbH A)
(24)
= trace(E{·})
(25)
we obtain after several manipulations M 2 i=1 E{σH (i)} ρuc = N Nr N o 2 i=1 E{σE (i)} + Es
0
th non-zero singular values where σH (i) and σE (i) are the i of H and E respectively, and N = rank(E).
100
150
Fig. 5. Comparison of new hybrid PSO-EA-DPSO with the actual in-phase channel coefficients for fd Ts = 0.05.
Recall that for the MIMO beam-forming system the received symbol was H H σ12 + u v 1 )x + n ˜. u 1 y = ( 1 E
(27)
Using similar techniques as before, the received SNR for the beam-forming case can be written as
ρbf = (26)
50 Samples
(23)
Using the independence assumption with x and H along with the identities ||bH AAH b||22
−0.8
E{σ12 } H 1 E| uH 1 U DV v
−σ max |2 +
Nr No Es
.
(28)
When comparing (26) with (28) one can see although there is less signal power in the beam-forming case, the prediction error has the capability of being significantly less. This can be seen by writing the prediction error in (23) as
2008 International Joint Conference on Neural Networks (IJCNN 2008)
Authorized licensed use limited to: University of Missouri. Downloaded on December 2, 2008 at 10:12 from IEEE Xplore. Restrictions apply.
3341
16 Actual PSO DEPSO
1
0.6
12
0.4
10
Received SNR
Amplitude
0.8
Actual Pred.
14
0.2 0
Beamforming
8 Uncoded 6
−0.2 4 −0.4 2
−0.6 −0.8
0
50
100
150
0
0
5
Samples
s
Fig. 6. Comparison of PSO, and DEPSO predictions with the actual inphase channel coefficients for fd Ts = 0.1.
0.8 0.6
Amplitude
0.4 0.2 0 −0.2 −0.4 −0.6
0
50
100
150
Samples
Fig. 7. Comparison of new hybrid PSO-EA-DPSO with the actual in-phase channel coefficients for fd Ts = 0.1. TABLE II M EAN SQUARED ERROR COMPARISON OF PSO, DEPSO, AND PSO-EA-DEPSO ALGORITHMS WITH RESPECT TO ACTUAL CHANNEL COEFFICIENTS . fd Ts = .05 fd Ts = .1
σe2 = E||Ex||22
PSO 1.2212 21.8440
= =
=
DEPSO 2.0811 1.2472
PSO − EA − DEPSO 0.0712 0.1183
H
Es
2 E{σE (i)}
(29)
i=1
where the last equality results from observing (26). Comparing this to the beam-forming prediction error
3342
20
Fig. 8. Received SNR comparison for un-coded and beam-forming MIMO systems.
(30)
it is clear that if u1 , v 1 , and σ1 are well predicted but the other modes are not, a smaller prediction error will result. This is implicitly true for the channel matrices predicted by the RNN. To show this behavior a numerical example is given. The received SNR using (26) and (28) is tabulated for 10000 binary phase shift keying (BPSK) symbol vectors with Es = 1 and Nt = Nr = 2. The results are displayed in Figure 8. As one can see the received SNR for the beam-forming case is significantly better at higher Es /No , suggesting that the stronger singular mode is more accurately 2 2 (1) and σE (2) are calculated predicted. To justify this σE and graphed in Figures 9 and 10. Clearly there are samples 2 2 2 2 (2) σE (1) suggesting E{σE (2)} ≥ E{σE (1)}. where σE 2 Comparing the two values we have E{σE (1)} ≈ 3 × 10−3 2 and E{σE (2)} ≈ 6 × 10−3 , indicating that the stronger singular value is predicted better by a factor of two. From this observation one can see that beam-forming provides two advantages over the un-coded system. The first 2 (1) affects the prediction error. The second is is only σE 2 (2)} > when the RNN predicts the MIMO channel, E{σE 2 (1)} which does not affect the beam-forming case since E{σE 2 (1) affects the received SNR only σE VIII. BER C OMPARISON
E V x||2 E||U 2 M H 2 H 1 − σ i U DV v E i xi u i=1 M
15 o
H 2 1 − σ 1 U DV H v σe2 = E u max x
Actual PSO−EA−DEPSO
1
−0.8
10 E /N
Before showing the difference in BER performance between the two MIMO systems, we briefly review the decoding procedures. A. Vector ML Detection For the MIMO un-coded system the received symbols are decoded according to 2. = arg min ||y − Hx|| x 2 x∈C Nt
This is known as the nearest neighbor decoding rule.
2008 International Joint Conference on Neural Networks (IJCNN 2008)
Authorized licensed use limited to: University of Missouri. Downloaded on December 2, 2008 at 10:12 from IEEE Xplore. Restrictions apply.
(31)
0
0.3
10 −3
5 Amplitude
Actual Prediction
2 E
E{σ (1)}
4
0.25
0.2
2 E
σ (1)
x 10
−1
3
10
2
0.15
0 800
900 Samples
BER
Amplitude
1 1000
−2
10
0.1 −3
10 0.05
−4
0 800
850
Fig. 9.
900 Samples
950
10
1000
5
10 E /N s
Maximum singular value of error matrix.
Fig. 11.
15
20
o
BER for a un-coded 2 × 2 MIMO fast fading channel.
−1
0.3
10 −3
2 E
2 E
8 Amplitude
0.25
Actual Prediction
σ (2)
x 10
0.2
0
E{σ (2)}
6 4
−2
10
0.15
0 800
900 Samples
BER
Amplitude
2 1000
−3
0.1
10
0.05
−4
0 800
850
Fig. 10.
900 Samples
950
1000
Fig. 12.
B. Beam-forming When beam-forming is performed at the transmitter and receiver the received symbol is found by x∈C
0
2
4
6
8 E /N s
Minimum singular value of error matrix.
uH H (1)x|2 . x = arg min | 1 y−σ
10
(32)
C. BER Simulations If the received SNR is large for a given Es /No then one expects the BER to be low. From the previous section it was established that the prediction error had a significant impact on the received SNR. By observing Figure 8 some observations about the BER can be inferred. One would expect that at low Es /No the prediction BER for the uncoded MIMO system will be comparable to the actual BER. As Es /No is increased, the received SNR does not increase as fast as the error free received SNR, which should cause the prediction BER to deteriorate. To illustrate this the BER was calculated via Monte Carlo simulations for a 2 x 2 MIMO flat fast fading channel using 10000 BPSK symbols for which
10
12
14
16
o
BER for a 2 × 2 MIMO beam-forming fast fading channel.
Es = 1. Looking at Figure 11, at low Es /No the prediction BER agrees nicely with the error free BER. But as Es /No increases the prediction BER begins to level off, since the effective noise floor due to prediction error is significant. This verifies the results in the previous section describing how the 2 (2) negatively affects the received SNR. addition of σE For the beam-forming case the received SNR remains close to the ideal SNR throughout the range of Es /No . This suggests the the BER for both the ideal and predicted channel will be close for all Es /No . To illustrate this the BER is calculated for a 2 × 2 MIMO beam-forming system using the same parameters as the un-coded case. As one can see the predicted BER differs only slightly from the error free BER. This verifies our observation that the strong singular mode is more accurately predicted by the RNN. IX. C ONCLUSION A recursive neural network trained by a novel PSO-EADEPSO was used to predict a MIMO channel. This training
2008 International Joint Conference on Neural Networks (IJCNN 2008)
Authorized licensed use limited to: University of Missouri. Downloaded on December 2, 2008 at 10:12 from IEEE Xplore. Restrictions apply.
3343
algorithm was shown to be superior to both PSO and DEPSO for two different fast fading scenarios. The received SNR for the un-coded and beam-forming MIMO systems was derived. The beam-forming technique was shown to be superior to the un-coded case for two reasons. The first was that only the dominant mode plays a role in the prediction error. The second is that the predicted channel predicts the dominant mode better than the minor ones. These two reasons lead to a higher received SNR and lower BER than the un-coded case, making the beam-forming system ideal. R EFERENCES [1] E. Telatar, “Capacity of multi-antenna gaussian channels,” AT&T Bell Laboratories Internal Tech. Memo., Jun. 1995. [2] H. Huang, “Spatial channel model for multiple input multiple output (mimo) simulations,” 3rd Generation Partnership Project Technical Report 25.996, V6.0.0, 2003. [3] D. Gesbert et al., “From theory to practice: An overview of mimo space-time coded wireless systems,” IEEE J. Select. Areas Commun., vol. 21, no. 3, pp. 281–302, Apr. 2003. [4] G. Bauch, “Mimo capacity loss for real world signal constellations and channel degradations,” Proc. IEEE Personal, Indoor and Mobile Radio Communications Conference (PIMRC’04), vol. 2, pp. 1439–1443, Sep. 2004. [5] A. Jazwinski, Stochastic Process and Filtering Theory. Acadamic Press, 1993. [6] X. Cai, N. Zhang, G. K. Venayagamoorthy, and D. C. Wunsch, “Time series prediction with recurrent neural networks trained by a hybrid pso-ea algorithm,” Neurocomputing, vol. 70, pp. 2342–2353, Aug. 2007. [7] A. Goldsmith, Wireless Communications. New York: Cambridge University Press, 2005. [8] T. S. Rappaport, Wireless Communications: Principles and Practice. Upper Saddle River,NJ: Prentice Hall, 2002. [9] Y. Zheng and C. Xiao, “Improved models for the generation of multiple uncorrelated rayleigh fading waveforms,” IEEE Commun. Lett., vol. 6, no. 6, 2002. [10] Y. Valle et al., “Particle swarm optimization: Basic concepts, variants and applications in power systems,” 2007. [11] J. Kennedy and R. Eberhart, “Particle swarm optimization,” IEEE Int. Conf. Neural Networks 4, vol. 49, pp. 1942–1948, 1995. [12] R. Xu, G. K. Venayagamoorthy, and D. C. Wunsch, “Modeling of gene regulatory networks with hybrid differential evolution and particle swarm optimization,” Neural Networks Journal, vol. 21, pp. 917–927, Oct. 2007.
3344
2008 International Joint Conference on Neural Networks (IJCNN 2008)
Authorized licensed use limited to: University of Missouri. Downloaded on December 2, 2008 at 10:12 from IEEE Xplore. Restrictions apply.
