CONSTANT MODULUS BLIND SOURCE SEPARATION TECHNIQUE: A NEW APPROACH. A. Belouchrani and K. Abed Meraim
Department of Electrical Engineering and Computer Sciences, University of California, Berkeley CA 94720, U.S.A,
[email protected] Department of Electrical and Electronics Engineering, University of Melbourne, Parkville, Victoria 3052, Australia,
[email protected] ABSTRACT
In this paper, we present a new approach for the blind source separation problem. Recently, several new techniques have emerged for the Multi-Input-Multi-Output (MIMO) blind identi cation problem [1, 2, 3] which is an important issue in communications. These techniques estimate the transfer function up to a constant matrix. The purpose of the source separation [4, 5] consists of the estimation of such matrix and provides an estimate of the emitted signals. The proposed method estimates the mixing matrix by an Input-Output (IO) identi cation using as inputs a nonlinear version of the estimated sources. Herein, the nonlinear distortion consists of constraining the Modulus of the inputs of the IO-Identi cation device to a Constant. Hence the name of the proposed technique. This approach presents many bene ts, i) Simple implementations (see below) ii) No ill convergence as indicated by simulations, but no proof is available yet. iii) Implementations are possible either in a block processing or in an adaptive fashion. The eectiveness of the proposed method is illustrated by some numerical simulations.
1. INTRODUCTION The blind source separation problem has recently become an intense area of research. It consists of recovering original waveforms of independent sources from only an observed linear mixture of them. The rst solution to this problem was proposed in 1985 [6] and was based on cancelation of higher order moments assuming non-Gaussian and i.i.d source signals. Other criteria based on minimization of cost functions, such as the sum of square fourth order cumulants [4, 7], contrast functions [7, 8] or likelihood function [9], have been used by several researchers. Note that in the case of non i.i.d. source signals and even Gaussian sources, solutions based on second order statistics are possible [10, 11].
In this paper, we propose a new approach based on a recurrent Input Output (IO)-Identi cation where the inputs are a nonlinear transformation of current estimated sources s^(t) (see Fig.1). This nonlinearity is taken into account here by constraining the modulus of these inputs to be a constant. Hence, this approach will be referred to as a Constant Modulus Source Separation (CMSS). This paper is organized as follows. In section 2, the problem of blind source separation is stated together with the relevant assumptions. Section 3 presents a new blind source separation technique based on a recurrent Input-Output Identi cation. In section 4, an iterative and adaptive implementation of our approach are proposed. Finally, some computer simulations illustrating the eectiveness of our approach are presented in section 5.
2. PROBLEM FORMULATION The blind source separation problem consists in recovering a set of n independent signals from m n observed instantaneous mixtures of these signals without
knowledge of any structure of the mixture.
Let x(t) be m 1 vector of observations (sensor signals) at time instant t which may be corrupted by an additive noise n(t). The instantaneous mixture model is given by: x(t) = As(t) + n(t); (1) where the mn unknown matrix A is called the `mixing matrix' and where the n independent zero-mean signals are collected in a n 1 vector denoted s(t) which is referred to as the source signal vector. The source signal vector is assumed to be (H1) an i.i.d. stationary multivariate process. The additive noise n(t) is assumed (H2) stationary, temporally white, zero mean complex random process with a pos-
sibly unknown covariance matrix, Rnn = E[n(t)n(t)]; (2) where the superscript denotes the conjugate transpose of a vector and the notation E[:] is used for ensemble averaging under hypothesis (H1). The noise is assumed to be (H3) independent of the source signals. The m n complex matrix A is assumed to have (H4) full rank but is otherwise unknown. In contrast with traditional parametric methods, no speci c structure of the mixture matrix is assumed. Let us point out that this problem of blind source separation has two inherent ambiguities. First of all, there is no way of knowing the original labeling of the sources, hence any permutation of the estimated sources is also a satisfactory solution. The second ambiguity is that it is inherently impossible to uniquely identify the source signals. This is because the exchange of a xed scalar factor between a source signal and the corresponding column of the mixture matrix A does not aect the observations as is shown by the following relation,
x(t) = As(t) + n(t) =
Xn ai s (t) + n(t); i=1
i
i i
(3)
where i is an arbitrary non zero complex factor and ai denotes the i-th column of A. Hence, the blind source separation must be understood as the identi cation of the mixing matrix and/or the recovering of the source signals up to a xed permutation and some complex factors. In the following, we propose a new separation approach based on a recurrent Input-Output Identi cation.
3. RECURRENT BLIND INPUT-OUTPUT IDENTIFICATION So far, the proposed blind source separation methods [6, 8, 5] based on non-linear function attempt in general to decorrelate the estimated source vector with a nonlinear version of it, i.e. E[g(^si (t))^sj (t) ] = 0; i 6= j (4) Due to the use of the non-linearity which introduces implicitly higher order moments, the decorrelation (4) provides in fact the statistical independence of the components of the estimated source vector. Hence, the problem of the source separation is solved. One can notice that since the observations contains already a linear transform of the desired source vector, the above decorrelation (4) can be seen as equivalent
to the decorrelation between the non-linear version of the estimated source vector and the observations since the additive noise is independent from the sources. We propose in the sequel a recurrent blind InputOutput Identi cation using implicitlythis kind of decorrelation and provides some very interesting features. To start, let us rst assume that the emitted signals are available. In this case, it would be a simple matter to estimate the mixing matrix A using an Input-Output Identi cation. The solution consists of evaluating the statistics: XT XT Rxx = T1 x(t)x(t) ; Rsx = T1 s(t)x(t) t=1 t=1 and then computing the estimated mixing matrix by A^ = (Rxx ? 2 I)R#sx (5) where # denotes the Moore-Penrose pseudo-inverse operator. In our problem, the source signals s(t) are of course not available, otherwise we will not have any need to perform the source separation. In this case, one can use instead of the original source signals, a nonlinear version of the current estimated sources as inputs to the IO-Identi cation device. b(t) s(t)
A
+
x(t)
^s(t)
A^-1
Non-linearity Input -Output Out Identification
Figure 1
In
In other words, as depicted in Fig.1, we have the following iterative estimation procedure: At the initialization step, compute the noise-free covariance matrix Rxx ?2 I and its pseudo-inverse matrix (2 can be computed as the mean value of the least eigen-values of Rxx), and set ^s(t)(0) = [x1(t); ; xn(t)]T . Estimate the non-linear transformation ~s(t) t = 1; : : :; T of the source signals ^s(t)(i?1) , t = 1; : : :; T, estimated at iteration (i ? 1). Compute the statistics: Rsx = T1 PTt=1 ~s(t)x(t) Estimate the source signals by ^s(t)(i) = Rsx (Rxx? 2 I)# x(t); t = 1; : : :; T
At the end of the iterative procedure, compute
the mixing matrix (if needed) as: A^ = (Rxx ? 2I)R#sx . In this paper, the non-linear distortion is taken into account by constraining the inputs of the IO-Identi cation device to have Constant Modulus. Hence, the proposed Method is referred to as Constant Modulus blind Source Separation. Note that any other properly chosen non-linearity can be used.
The colored noise case: The above procedure can be modi ed to deal with the case of spatially colored additive noise with unknown covariance matrix Rnn. Note that, when the emitted signals are available, we can estimate both of the mixing matrix A and the noise covariance matrix Rnn using an Input-Output Identi cation. The solution consists of evaluating the statistics: XT (6) Rxx = T1 x(t)x(t) ; t=1 XT (7) Rxs = T1 x(t)s(t) ; t=1 XT (8) Rss = T1 s(t)s(t) t=1 and then estimate the unknown parameters by A^ = Rxs R?ss1; R^ nn = Rxx ? Rxs R?ss1RHxs (9) where H denotes the transpose conjugate of a matrix. Notice that, solution (9) is statistically better than (5), since it corresponds to the mean-squares solution of the IO-identi cation criterion, i.e. A^ = RxsR?ss1 = arg min E(kx(t) ? Bs(t)k2 ) B
It requires however (see next section) more computational cost than solution (5). The above identi cation scheme can be modi ed according to the new equations (9). Two dierent implementation of this approach are proposed in the next section.
4. IMPLEMENTATION We present here the implementation of the proposed approach in the case of colored noise. Iterative Implementation: The rst implementation is iterative and based on the processing of T received samples. This implementation is refereed to as Bloc Constant Modulus Separation (B-CMS) and goes as follows:
At iteration i
1 - Constrain the Modulus of the source signals ^s(t)(i?1), t = 1; : : :; T, estimated at iteration (i ? 1) to a constant r (e.g r=1) and let ~s(t) t = 1; : : :; T, be the constrained signals. 2 - ComputePthe statistics: Rxs = T1 PTt=1 x(t)~s(t) and Rss = T1 Tt=1 ~s(t)~s(t) . 3 - Estimate the source signals by ^s(t)(i) = RssR#xs x(t); t = 1; : : :; T. If needed, the
mixing matrix and the noise covariance matrix can be estimated from (9). Adaptive Implementations: The adaptive implementations is based on the processing of L samples received previously to the time-instant t. This implementation, refered to as Adaptive-Constant Modulus Separation (A-CMS) goes as follows: At time-instant t: 1 - Constrain the Modulus of the previous estimated source signal ^s(t ? 1) to a constant r (e.g r=1) and let s~(t ? 1) be the constrained signals. 2 - Compute the statistics:P (t?1) + t R(xst) = (1 ? )Rxs s(ti ) and L Pt =t?L+1 x(ti )~ ( t) ( t?1) t Rss = (1 ? )Rss + L t =t?L+1 s~(ti )~s(ti ) where is a decreasing and positive sequence. 4 - Estimate the# source signals at time t by (t) x(t). The mixing matrix and the noise ^s(t) = R(sst) Rxs covariance matrix can be estimated from (9). i
i
5. SIMULATIONS
We present in the following some numerical simulations to assess the performance of our algorithm. We have simulated two i.i.d. source signals respectively 4-QAM1 and 16-QAM constellation with unit variance. The pth element of the q-th column of the mixture matrix is expf2ipq g. Of course, we do not use here this a priori information. We choose a 2 2 mixing matrix with 1 = 0:2 and 2 = 0:4. The additive noise is generated from a zero mean and white Gaussian process. The SNR is taken equal to 30 dB and the sample size is equal to 1000. In Fig.2, a sample run of B-CMS algorithm is presented, and Fig.3 shows the evolution of the mean rejection level [5] during the iterations of the previous sample runs. In Fig.4, a sample run of A-CMS algorithm is presented. A fast convergence of the two proposed implementations is observed.
6. CONCLUSION This paper presents a new approach for blind separation of signals based on a recurrent Input-Output Identi cation using as inputs a nonlinear transformation of 1
QAM: Quadratic Amplitude Modulation.
One of the source signals
One of the source signals
Im.
1 0.5
Im.
1 0.5 0
0
−0.5
−0.5
−1
−1 −1 0 1 One of the mixed signals
1 0.5 0 −0.5 −1
Im.
Im.
−1 0 1 One of the mixed signals
−1 0 1 One of the separated signals
1 0.5 0 −0.5 −1
1 Im.
Im.
7.
−1 0 1 One of the separated signals
1 0 −1
0 −1
−1
0 Re.
1
−1
Figure 2
0 Re.
1
0
Mean rejection level (dB)
−5
−10
−15
−20
−25
−30 0
5
10
15
20 25 Iteration number
30
35
40
45
900
1000
Figure 3 0 −5 −10
Rejection level (dB)
−15 −20 −25 −30 −35 −40 −45 −50 0
100
200
300
400
500 Samples
Figure 4
600
700
current estimated sources. In contrast to other existing techniques, the covariance of the noise do not need to be modeled and can be estimated as a regular parameter if needed. Ecient iterative and adaptive algorithms were proposed for the implementation of the proposed approach. The validity and the performance of the proposed approach have been illustrated by some computer experiments.
800
REFERENCES
[1] D. Slock, \Blind fractionally-spaced equalization, perfect-reconstruction lter-banks and multichannel linear prediction," in Proc. ICASSP, vol. 4, pp. 585{ 588, 1994. [2] A. Gorokhov, P. Loubaton, \Subspace based techniques for second order blind separation of convolutive mixtures with temporally correlated sources.." submitted to IEEE Tr. on CS. [3] K. Abed Meraim, P. Loubaton and E. Moulines, \A subspace algorithm for certain blind identi cation problems.." accepted in IEEE Tr. on IT. [4] M. Gaeta and J.-L. Lacoume, \Source separation without a priori knowledge: the maximum likelihood solution," in Proc. EUSIPCO, pp. 621{624, 1990. [5] J.-F. Cardoso, A. Belouchrani, and B. Laheld, \A new composite criterion for adaptive and iterative blind source separation," in Proc. ICASSP, pp. 273{276, 1994. [6] C. Jutten and J. Herault, \Detection de grandeurs primitives dans un message composite par une architecture de calcul neuromimetrique en apprentissage non supervise," in Proc. Gretsi, (Nice), 1985. [7] P. Comon, \Independent component analysis, a new concept?," Signal Processing, vol. 36, pp. 287{314, 1994. [8] E. Moreau and O. Macchi, \New self-adaptive algorithms for source separation based on contrast functions," in Proc. IEEE SP Workshop on Higher-Order Stat., Lake Tahoe, USA, 1993. [9] A. Belouchrani and J.-F. Cardoso, \Maximum likelihood source separation for discrete sources," in Proc. EUSIPCO, pp. 768{771, 1994. [10] L. Tong and R. Liu, \Blind estimation of correlated source signals," in Proc. Asilomar conference, Nov. 1990. [11] A. Belouchrani and K. Abed Meraim and J.-F. Cardoso and E. Moulines, \Second-order blind separation of correlated sources," in Proc. Int. Conf. on Digital Sig. Proc., (Cyprus), pp. 346{351, 1993.