LECTURE 2 Signal Space Analysis
L ECTURE 2
Signal Space Analysis 2.1 Introduction In this lecture we consider an idealized digital communication system (see Figure 1.1) where the input to a modulator is one of M discrete alphabets or symbols denoted by m0 , m1 , m1 , · · · , mM−1 . Modulation is the process by which these symbols are converted into a form suitable for transmission over the channel. This may involve a single stage where each symbol is independently mapped to a waveform, or several stages where each symbol is first mapped to some waveform which is further processed before transmission. Some examples are as follows: Example 2.1: BPSK In binary phase shift keying (BPSK), a bit “0” is mapped to a waveform s1 (t) = cos(2π fct), 0 ≤ t < T and a “1” is mapped to a waveform s2 (t) = cos(2π fct + π), 0 ≤ t < T . Here fc is the carrier frequency and T is the bit duration. Note that there are different ways in which you can view the process of modulation. Here we have viewed this process as involving a mapping of the bits to waveforms. Since s2 (t) = −s1 (t), it is also possible to view this as a non-return to zero (NRZ) bit stream continuously multiplying a carrier cos(2π fct) which is a form of amplitude modulation (AM). Yet another way to look at this is to think of the information contained in the bits being carried in the phase of the carrier (phase modulation or PM), the phase being zero if the bit is a “0” and the phase being π if the bit is a “1”. These various methods of considering modulation schemes are useful in analyzing different aspects of the communication system. Example 2.2: BFSK In binary frequency shift keying (BSFSK), a bit “0” is mapped to a waveform s1 (t) = cos(2π f1t), 0 ≤ t < T and a “1” is mapped to a waveform s2 (t) = cos(2π f2t), 0 ≤ t < T . Once again, fc is the carrier frequency and T is the bit duration. Instead of viewing it as a mapping, we can represent BFSK as a form of frequency modulation (FM) with the bit zero selecting the frequency f1 and the bit one selecting the frequency f2 . Example 2.3: Spread Spectrum Direct sequence spread spectrum (DSSS) involves two levels of modulation, where the baseband information signal is first spread or modulated by a spreading signal. Once the information signal is spread, it is further modulated onto an RF carrier using either BPSK or QPSK.
10
LECTURE 2. SIGNAL SPACE ANALYSIS
11
In the previous lecture we had put forward a set of questions related to digital communication systems. One of the questions was how we could model a communication system and what the limitations of a model were. Initially, we will only consider a mapping between the symbols m1 , m2 , m3 , · · · , mM to the corresponding waveforms s1 (t), s2 (t), s3 (t), · · · , sM (t). In this lecture, we also make the following assumptions (limitations): 1. The communication system is memoryless. That is each symbol that is transmitted independently of the others and is also received independently of the others. 2. Each waveform si (t) occupies a duration of time T , which we call the symbol duration. 3. Each waveform si (t) is a real valued energy signal with finite energy given by: Z T
Ei =
0
s2i (t)dt
(2.1)
4. The channel is linear with sufficient bandwidth to eliminate ISI. 5. The system is perfectly synchronized and the noise introduced by the channel is additive, white and Gaussian (AWGN) with two-sided power-spectral density of N0 /2. Under these assumptions, an important method of characterizing digital communication systems is to use signal space analysis. In particular, this provides a general technique for analyzing thye bit-error rate performance of several modulation schemes. The objective of the signal space analysis can be summarized as follows: Given the finite set of waveforms si (t), i = 1, 2, 3, · · · , M, we need to establish that they can be represented as elements of a finite vector space spanned by a set of orthonormal basis functions φ1 (t), φ2 (t), · · · , φN (t) where N ≤ M. Also the received signal that has additive white Gaussian noise should also be represented as an element of the same vector space so that we can analyze the system.
2.2 Vector Spaces We are familiar with the concept of vectors as quantities that not only have magnitude but also direction. In Figure 2.1, we show some examples of vectors and their common properties. In Figure 2.1(a) the addition of two vectors is shown as the diagonal joining the edges of the parallelogram created by the two vectors. In Figure 2.1(b), we show the effect of multiplying a vector by a scalar number and in Figure 2.1(c), we look at the familiar concept of representing a vector in terms of its components (or projections on some unit vectors). In this particular example, we can write u = 2ˆi + 3ˆj. We can also write this as an ordered pair in two-dimensional Euclidean space R 2 as u = (2, 3). It is not only the usual vectors we know that belong to vector spaces. It is also possible to generalize this concept to other types of mathematical elements like polynomials and functions. For example, all polynomials p(x) = α0 + α1 x + α2 x2 can be written as ordered triples in three-dimensional Euclidean space R 3 . For this example, the polynomial p(x) would be written as p = (α0 , α1 , α2 ). The common properties of all such mathematical elements can be summarized as follows, and is the definition of vector spaces. Let V be a set of elements on which a binary operation called addition “+” is defined. If F is a field (real numbers or complex numbers), let a multiplication operation “.” be defined between the elements of F and V . The set V is called a vector space over the field F if it satisfies:
LECTURE 2. SIGNAL SPACE ANALYSIS
12
Figure 2.1: Vectors and their properties 1. For all elements u and v ∈ V , u + v = v + u (Commutative Property). 2. For all elements u, v, and w ∈ V , (u + v) + w = v + (u + w) (Associative Property). 3. There exists a “zero element” 0 ∈ V such that for any u ∈ V , u + 0 = 0 + u = u (Additive Identity). 4. There exists an “inverse element” −u ∈ V such that for any u ∈ V , u + −u = −u + u = 0 (Additive Inverse). 5. There exists a scalar “1”∈ F such that 1.u = u for all u ∈ V . 6. For any scalars α, β ∈ F , α(βu) = (αβ)u for all u ∈ V . 7. For any scalars α, β ∈ F , (α + β)u = αu + βu and α(u + v) = αu + αv for all u, v ∈ V With the above definition of a vector space, it is possible for us to provide a unified treatment of all signals by representing them as elements of a vector space. Before we do that, let us consider some general concepts related to vector spaces.
2.2.1 Concepts related to vector spaces Linear dependence: A collection of vectors u1 , u2 , u3 , · · · , uM in a vector space V is lineraly dependent if there exist scalars in F like α1 , α2 , α3 , · · · , αM not all zero such that: α1 u1 + α2 u2 + · · · + αM uM = 0
(2.2)
The set of these vectors is called linearly independent if the above equation holds for only one case: α1 = α2 = α3 = · · · = αM = 0
(2.3)
LECTURE 2. SIGNAL SPACE ANALYSIS
13
Dimensionality and basis: A vector space V has dimension N if there exists at least one collection of N vectors in it that are linearly independent but no collection of N + 1 or more vectors is linearly independent. Such a collection of N vectors is called a basis and the basis is said to span the vector space V . Example 2.4: The three dimensional Euclidean space R 3 In R 3 , the vectors (2,0,0), (0,3,0) and (0,0,4) are linearly independent, but (2,0,0), (0,3,0), (0,0,4), and (1,2,0) are not. Consider the latter set. We can write:
α0 (2, 0, 0) + α1 (0, 3, 0) + α2 (0, 0, 4) + α3 (1, 2, 0) = (0, 0, 0)
(2.4)
From this equation, we can write 2α0 + α3 = 0, 3α1 + 2α3 = 0, 4α2 = 0. We can easily find nonzero values of α0 , α1 , α2 , and α3 that satisfy these equations. For instance, letting α3 = 1, we find α0 = −1, α1 = −0.67, and α2 = 0. In fact no set of four vectors in R 3 is linearly independent since the dimension of R 3 is three. Also the vectors (2,0,0), (0,3,0) and (0,0,4) form a basis in R 3 .
Any vector in a vector space V can be expressed as a linear combination of its basis vectors. That is, given a set of basis vectors v1 , v2 , · · · , vN , we can write any vector u as: u = u1 v1 + u2 v2 + · · · + uN vN
(2.5)
In such a case, we can simply write the vector in terms of its components (u1 , u2 , · · · , uN ). Inner product and norm: The inner product of two vectors is defined as the projection of one vector on another, which is just a scalar number. It is written as u.v. In Euclidean space, where the vectors can be written in terms of their components, N
u.v = (u1 , u2 , · · · , uN ).(v1 , v2 , · · · , vN ) = u1 v1 + u2 v2 + · · · uN vN = ∑ ui vi
(2.6)
i=1
The norm or length of a vector is defined as s 1/2
kuk = (u.u)
=
N
∑ u2i
(2.7)
i=1
Two vectors u and v are said to be orthogonal if u.v = 0. The basis vectors v1 , v2 , · · · , vN are said to be orthonormal if kvi k = 1
(2.8)
vi .vj = 0, for i 6= j
(2.9)
A common example of an orthonormal basis is the set of unit vectors in N-dimensional Euclidean space R N given by: e1
=
(1, 0, 0, 0, · · · , 0)
e2
=
(0, 1, 0, 0, · · · , 0)
··· eN
=
(0, 0, 0, 0, · · · , 1)
(2.10) (2.11)
LECTURE 2. SIGNAL SPACE ANALYSIS
14
Triangle and Cauchy-Schwartz Inequalities here without proof:
Given two vectors u and v, two inequalities exist that are stated
The Triangle Inequality: ku + vk ≤ kuk + kvk The Cauchy-Schwartz Inequality: |u.v| ≤ kuk.kvk
2.2.2
(2.12) (2.13)
The Gram-Schmidt Orthonormalization Procedure
In the previous section, we discussed the fact that any vector can be represented as a linear combination of the basis vectors and as such, we can express an vector simply in terms of the scalar coefficients (components) that multiply the basis vectors. The Gram-Schmidt Orthonormalization procedure is a neat way of obtaining an orthonormal basis from any given set of vectors. Suppose we assume that we have a basis of N vectors, but they are not orthonormal. Let us call these vectors that form the basis as v1 , v2 , · · · , vN . We can generate an orthonormal set as follows: 1. Step 1: Lete1 =
v1 kv1 k
; ⇒ ke1 k = 1
0
2. Step 2: Let e2 = v2 − (v2 .e1 )e1 . This eliminates the component of v2 along e1 . We can then define: 0
e2 =
e2 0 ke2 k
(2.14)
We can easily show that e2 and e1 are orthogonal. we do that by demonstrating that their inner product is zero. Consider the inner product of e2 and e1 : e2 .e1 =
v2 − (v2 .e1 )e1 v2 .e1 − (v2 .e1 )(e1 .e1 ) .e1 = =0 0 0 ke2 k ke2 k
(2.15)
0
because e1 .e1 = 1 by definition. Note that e2 can never be zero since we have assumed that v1 , v2 , · · · , vN are linearly independent. 0
3. Step 3: Repeat Step 2 by creating e3 = v3 − (v3 .e1 )e1 − (v3 .e2 )e2 . The third vector of the orthonormal set will be e3 =
2.3
0 e3 0 ke3 k
. This process can be continued till all the orthonormal vectors are determined.
Communication Signals as Vectors in a Vector Space
Consider a communication system as shown in Figure 2.2. In this simplified communication system, we assume that the source produces one of M symbols or alphabets denoted by mi . When the transmitter receives the alphabet mi , it maps it into a finite duration energy waveform si (t) for i = 1, 2, 3, · · · , M. Each of these waveforms lasts for a duration of T seconds. We first establish that the M waveforms can be represented as elements of a vector space that is spanned by a set of N orthonormal functions φ1 (t), φ2 (t), φ3 (t), · · · , φN (t) where N ≤ M and each of the orthonormal functions is a finite energy signal that lasts for T seconds. It suffices to say that the set of functions s1 (t), s2 (t), · · · , sM (t) belong to a subspace of the vector space L2 defined as the set of all square integrable functions. In L2 , the inner product of two “vectors” f = f (t) and g = g(t) is given by: Z ∞
f.g = −∞
f (t)g∗ (t)dt
(2.16)
LECTURE 2. SIGNAL SPACE ANALYSIS
15
Figure 2.2: Simple communication system and the norm of a vector f = f (t) is defined as: kfk =
√
rZ
∞
| f (t)|2 dt
f.f =
(2.17)
−∞
The triangle inequality here becomes: Z ∞ 1/2 Z 2 | f (t) + g(t)| dt ≤ −∞
∞
2
1/2
Z
| f (t)| dt
+
−∞
and the Cauchy-Schwartz inequality becomes: Z ∞ Z ∗ f (t)g (t)dt ≤ −∞
∞
| f (t)|2 dt
∞
2
1/2
|g(t)| dt
(2.18)
−∞
1/2 Z
−∞
∞
|g(t)|2 dt
1/2 (2.19)
−∞
When we talk about the subspace spanned by the N orthonormal functions, we restrict our limits to [0, T ].
2.3.1
Geometric representation of signals
The set of signals si (t) for i = 1, 2, 3, · · · , M belong to a subspace of L2 . Suppose we can determine N orthonormal basis functions φi (t) for i = 1, 2, 3, · · · , N that span this subspace. We can then write every one of the signals si (t) for i = 1, 2, 3, · · · , M as a linear combination of the basis functions. That is: N
si (t) = ∑ si j φ j (t)
(2.20)
i=1
for 0 ≤ t ≤ T and for i = 1, 2, 3, · · · , M. Here si j is a scalar. In other words, we can simply express each signal si (t) in terms of its components si j . For example, we can write si (t) = si = (si1 , si2 , · · · , siN ). In the case of all real signals, certain properties become clear: Z T 1 if i = j φi (t)φ j (t)dt = δi j = (2.21) 0 if i 6= j 0 Also, the components of the vector si are obtained using the following relation: " # Z Z Z T
si j =
0
T
si (t)φ j (t)dt =
0
N
∑ sik φk (t) φ j (t)dt =
k=1
N
∑ sik
k=1
T
0
φk (t)φ j (t)dt
Clearly, from (2.21) the integral exists and it is one only if k = j giving us the desired result.
(2.22)
LECTURE 2. SIGNAL SPACE ANALYSIS
2.3.2
16
Gram-Schmidt Orthonormalization Procedure for Signals
In order to derive the N orthonormal basis functions, we resort to the Gram-Schmidt orthonormalization procedure. In this case however, we have made no assumption about our set of signals (for example that they are linearly independent and they span their own subspace). Consequently, the procedure will be slightly different. 1. Step 1: Let E1 = follows:
RT 2 0 s1 (t)dt be the energy in the signal s1 (t). Then we find the first basis function as
s1 (t) φ1 (t) = √ E1
(2.23)
Note that 0T φ21 (t)dt = E11 0T s21 (t)dt = 1 and the first function is already normalized. Clearly, s1 (t) = √ √ E1 φ1 (t) ⇒ s1 = ( E1 , 0, 0, · · · , 0). You can draw a parallel to the way in which we looked at the GramSchmidt orthonormalization procedure for vectors. We first divide the given vector by its norm to obtain the first element of the orthonormal basis, and so on. R
R
2. Step 2: Let g2 (t) = s2 (t) − s21 φ1 (t). We can calculate s21 using the relation s21 = 0T s2 (t)φ1 (t)dt. If s2 (t) and s1 (t) are linearly dependent, g2 (t) will be zero. If this is thye case, we proceed to compute g3 (t) = s3 (t) − s31 φ1 (t). Otherwise, we let: R
g2 (t) s2 (t) − s21 φ1 (t) φ2 (t) = qR = q T 2 E2 − s221 0 g2 (t)dt We arrive at the last result using the following relations: Let E2 = Z T 0
g22 (t)dt =
Z T 0
s22 (t)dt + s221 − 2s21
Z T 0
(2.24)
RT 2 0 s2 (t)dt be the energy in s2 (t). Then
s2 (t)φ1 (t)dt = E2 − s221
(2.25)
We know that φ2 (t) has unit energy by definition. We now need to show that φ1 (t) and φ2 (t) are orthogonal to one another. Orthogonality can be shown directly as follows: Z T 0
φ1 (t)φ2 (t)dt = q
1
Z T 0
E2 − s221
[s2 (t) − s21 φ1 (t)] φ1 (t)dt = q
s2 (t)−s21 φ1 (t) √ , we can write s2 (t) = E2 −s221
Also, since φ2 (t) = q and s2 = (s21 , E2 − s221 , 0, 0, · · · , 0).
q
E2 − s221
1 E2 − s221
(s21 − s21 ) = 0
φ2 (t)+s21 φ1 (t) so that s22 =
(2.26)
q
E2 − s221
3. Step 3:The general step in this procedure is as follows. Assume that l − 1 orthonormal basis functions have been determined using k − 1 signals s1 (t), · · · , sk−1 (t). Note that l ≤ k. If none of the gi (t) computed is zero, l = k. We let: l−1
gk (t) = sk (t) − ∑ sk j φ j (t) j=1
(2.27)
LECTURE 2. SIGNAL SPACE ANALYSIS where sk j =
RT 0
17
sk (t)φ j (t)dt for j = 1, 2, 3, · · · , l − 1. If gk (t) 6= 0, we can then compute: gk (t) φl (t) = qR T 2 0 gk (t)dt
(2.28)
4. Step 4: Repeat Step 3 till all the signals are exhausted and we determine the N orthonormal basis functions. Example 2.5: Gram-Schmidt orthonormalization for four signals
Figure 2.3: Signals to be represented as vectors We are required to determine the orthonormal basis for the signals shown in Figure 2.3 and represent them in a signal space.
√
1. Step 1: p Compute φ1 (t) = s1 (t)/ E1 . The energy in the signal s1 (t) is T /3. Thus φ1 (t) = s1 (t)/ T /3. 2. Step 2: Compute g2 (t) = s2 (t) − s21 φ1 (t). From the expressions for calculating the components, Z T
s21 =
0
Z T
E2 =
0
1 T /3
s2 (t)φ1 (t)dt = p
Z T /3 0
s2 (t)s1 (t)dt =
p
T /3
s22 (t)dt = 2T /3
(2.29) (2.30)
The signal s21 φ1 (t) and g2 (t) are shown in Figure 2.4. The energy in g2 (t) is p R φ2 (t) = g2 (t)/[ 0T g22 (t)dt] = g2 (t)/ T /3. This is also shown in Figure 2.4.
p T /3 so that
3. Step 3: Compute g3 (t) = s3 (t) − s31 φ1 (t) − s32 φ2 (t). Here Z T
s31 =
0
s3 (t)φ1 (t)dt = 0
Z T
s32 =
0
s3 (t)φ2 (t)dt =
p
(2.31) T /3
(2.32)
Figure 2.4 shows g3 (t) = s3 (t) − s32 φ2 (t) since s31 p p= 0. Once again, the energy in g3 (t) is R T /3 so that φ3 (t) = g3 (t)/[ 0T g23 (t)dt] = g3 (t)/ T /3 as shown in Figure 2.4.
LECTURE 2. SIGNAL SPACE ANALYSIS
18
4. Step 4: Now we consider the last signal. When we compute g4 (t) = s4 (t) p− −s41 φ1 (t) − s42 φ2 (t) − s43 φ3 (t), we find that g4 (t) = 0. This is because s41 = s42 = s43 = T /3 and s4 (t) can be expressed as a linear combination of φ1 (t), φ2 (t) and φ3 (t). There is no need for a fourth basis function.
Figure 2.4: Signals computed to determine the orthonormal basis Now that we know all the coefficients si j , we can easily express the signals as vectors as follows:
p s1 = T /3, 0, 0 p p s3 = 0, T /3, T /3
T /3, 0 p p p s4 = T /3, T /3, T /3
s2 =
p
T /3,
p
(2.33) (2.34)
LECTURE 2. SIGNAL SPACE ANALYSIS The four signals are plotted in the vector subspace spanned by φ1 (t), φ2 (t) and φ3 (t) as shown in Figure 2.5.
Figure 2.5: Signals computed to determine the orthonormal basis
19
Signal Space Analysis 2.1 Introduction In this lecture we consider an idealized digital communication system (see Figure 1.1) where the input to a modulator is one of M discrete alphabets or symbols denoted by m0 , m1 , m1 , · · · , mM−1 . Modulation is the process by which these symbols are converted into a form suitable for transmission over the channel. This may involve a single stage where each symbol is independently mapped to a waveform, or several stages where each symbol is first mapped to some waveform which is further processed before transmission. Some examples are as follows: Example 2.1: BPSK In binary phase shift keying (BPSK), a bit “0” is mapped to a waveform s1 (t) = cos(2π fct), 0 ≤ t < T and a “1” is mapped to a waveform s2 (t) = cos(2π fct + π), 0 ≤ t < T . Here fc is the carrier frequency and T is the bit duration. Note that there are different ways in which you can view the process of modulation. Here we have viewed this process as involving a mapping of the bits to waveforms. Since s2 (t) = −s1 (t), it is also possible to view this as a non-return to zero (NRZ) bit stream continuously multiplying a carrier cos(2π fct) which is a form of amplitude modulation (AM). Yet another way to look at this is to think of the information contained in the bits being carried in the phase of the carrier (phase modulation or PM), the phase being zero if the bit is a “0” and the phase being π if the bit is a “1”. These various methods of considering modulation schemes are useful in analyzing different aspects of the communication system. Example 2.2: BFSK In binary frequency shift keying (BSFSK), a bit “0” is mapped to a waveform s1 (t) = cos(2π f1t), 0 ≤ t < T and a “1” is mapped to a waveform s2 (t) = cos(2π f2t), 0 ≤ t < T . Once again, fc is the carrier frequency and T is the bit duration. Instead of viewing it as a mapping, we can represent BFSK as a form of frequency modulation (FM) with the bit zero selecting the frequency f1 and the bit one selecting the frequency f2 . Example 2.3: Spread Spectrum Direct sequence spread spectrum (DSSS) involves two levels of modulation, where the baseband information signal is first spread or modulated by a spreading signal. Once the information signal is spread, it is further modulated onto an RF carrier using either BPSK or QPSK.
10
LECTURE 2. SIGNAL SPACE ANALYSIS
11
In the previous lecture we had put forward a set of questions related to digital communication systems. One of the questions was how we could model a communication system and what the limitations of a model were. Initially, we will only consider a mapping between the symbols m1 , m2 , m3 , · · · , mM to the corresponding waveforms s1 (t), s2 (t), s3 (t), · · · , sM (t). In this lecture, we also make the following assumptions (limitations): 1. The communication system is memoryless. That is each symbol that is transmitted independently of the others and is also received independently of the others. 2. Each waveform si (t) occupies a duration of time T , which we call the symbol duration. 3. Each waveform si (t) is a real valued energy signal with finite energy given by: Z T
Ei =
0
s2i (t)dt
(2.1)
4. The channel is linear with sufficient bandwidth to eliminate ISI. 5. The system is perfectly synchronized and the noise introduced by the channel is additive, white and Gaussian (AWGN) with two-sided power-spectral density of N0 /2. Under these assumptions, an important method of characterizing digital communication systems is to use signal space analysis. In particular, this provides a general technique for analyzing thye bit-error rate performance of several modulation schemes. The objective of the signal space analysis can be summarized as follows: Given the finite set of waveforms si (t), i = 1, 2, 3, · · · , M, we need to establish that they can be represented as elements of a finite vector space spanned by a set of orthonormal basis functions φ1 (t), φ2 (t), · · · , φN (t) where N ≤ M. Also the received signal that has additive white Gaussian noise should also be represented as an element of the same vector space so that we can analyze the system.
2.2 Vector Spaces We are familiar with the concept of vectors as quantities that not only have magnitude but also direction. In Figure 2.1, we show some examples of vectors and their common properties. In Figure 2.1(a) the addition of two vectors is shown as the diagonal joining the edges of the parallelogram created by the two vectors. In Figure 2.1(b), we show the effect of multiplying a vector by a scalar number and in Figure 2.1(c), we look at the familiar concept of representing a vector in terms of its components (or projections on some unit vectors). In this particular example, we can write u = 2ˆi + 3ˆj. We can also write this as an ordered pair in two-dimensional Euclidean space R 2 as u = (2, 3). It is not only the usual vectors we know that belong to vector spaces. It is also possible to generalize this concept to other types of mathematical elements like polynomials and functions. For example, all polynomials p(x) = α0 + α1 x + α2 x2 can be written as ordered triples in three-dimensional Euclidean space R 3 . For this example, the polynomial p(x) would be written as p = (α0 , α1 , α2 ). The common properties of all such mathematical elements can be summarized as follows, and is the definition of vector spaces. Let V be a set of elements on which a binary operation called addition “+” is defined. If F is a field (real numbers or complex numbers), let a multiplication operation “.” be defined between the elements of F and V . The set V is called a vector space over the field F if it satisfies:
LECTURE 2. SIGNAL SPACE ANALYSIS
12
Figure 2.1: Vectors and their properties 1. For all elements u and v ∈ V , u + v = v + u (Commutative Property). 2. For all elements u, v, and w ∈ V , (u + v) + w = v + (u + w) (Associative Property). 3. There exists a “zero element” 0 ∈ V such that for any u ∈ V , u + 0 = 0 + u = u (Additive Identity). 4. There exists an “inverse element” −u ∈ V such that for any u ∈ V , u + −u = −u + u = 0 (Additive Inverse). 5. There exists a scalar “1”∈ F such that 1.u = u for all u ∈ V . 6. For any scalars α, β ∈ F , α(βu) = (αβ)u for all u ∈ V . 7. For any scalars α, β ∈ F , (α + β)u = αu + βu and α(u + v) = αu + αv for all u, v ∈ V With the above definition of a vector space, it is possible for us to provide a unified treatment of all signals by representing them as elements of a vector space. Before we do that, let us consider some general concepts related to vector spaces.
2.2.1 Concepts related to vector spaces Linear dependence: A collection of vectors u1 , u2 , u3 , · · · , uM in a vector space V is lineraly dependent if there exist scalars in F like α1 , α2 , α3 , · · · , αM not all zero such that: α1 u1 + α2 u2 + · · · + αM uM = 0
(2.2)
The set of these vectors is called linearly independent if the above equation holds for only one case: α1 = α2 = α3 = · · · = αM = 0
(2.3)
LECTURE 2. SIGNAL SPACE ANALYSIS
13
Dimensionality and basis: A vector space V has dimension N if there exists at least one collection of N vectors in it that are linearly independent but no collection of N + 1 or more vectors is linearly independent. Such a collection of N vectors is called a basis and the basis is said to span the vector space V . Example 2.4: The three dimensional Euclidean space R 3 In R 3 , the vectors (2,0,0), (0,3,0) and (0,0,4) are linearly independent, but (2,0,0), (0,3,0), (0,0,4), and (1,2,0) are not. Consider the latter set. We can write:
α0 (2, 0, 0) + α1 (0, 3, 0) + α2 (0, 0, 4) + α3 (1, 2, 0) = (0, 0, 0)
(2.4)
From this equation, we can write 2α0 + α3 = 0, 3α1 + 2α3 = 0, 4α2 = 0. We can easily find nonzero values of α0 , α1 , α2 , and α3 that satisfy these equations. For instance, letting α3 = 1, we find α0 = −1, α1 = −0.67, and α2 = 0. In fact no set of four vectors in R 3 is linearly independent since the dimension of R 3 is three. Also the vectors (2,0,0), (0,3,0) and (0,0,4) form a basis in R 3 .
Any vector in a vector space V can be expressed as a linear combination of its basis vectors. That is, given a set of basis vectors v1 , v2 , · · · , vN , we can write any vector u as: u = u1 v1 + u2 v2 + · · · + uN vN
(2.5)
In such a case, we can simply write the vector in terms of its components (u1 , u2 , · · · , uN ). Inner product and norm: The inner product of two vectors is defined as the projection of one vector on another, which is just a scalar number. It is written as u.v. In Euclidean space, where the vectors can be written in terms of their components, N
u.v = (u1 , u2 , · · · , uN ).(v1 , v2 , · · · , vN ) = u1 v1 + u2 v2 + · · · uN vN = ∑ ui vi
(2.6)
i=1
The norm or length of a vector is defined as s 1/2
kuk = (u.u)
=
N
∑ u2i
(2.7)
i=1
Two vectors u and v are said to be orthogonal if u.v = 0. The basis vectors v1 , v2 , · · · , vN are said to be orthonormal if kvi k = 1
(2.8)
vi .vj = 0, for i 6= j
(2.9)
A common example of an orthonormal basis is the set of unit vectors in N-dimensional Euclidean space R N given by: e1
=
(1, 0, 0, 0, · · · , 0)
e2
=
(0, 1, 0, 0, · · · , 0)
··· eN
=
(0, 0, 0, 0, · · · , 1)
(2.10) (2.11)
LECTURE 2. SIGNAL SPACE ANALYSIS
14
Triangle and Cauchy-Schwartz Inequalities here without proof:
Given two vectors u and v, two inequalities exist that are stated
The Triangle Inequality: ku + vk ≤ kuk + kvk The Cauchy-Schwartz Inequality: |u.v| ≤ kuk.kvk
2.2.2
(2.12) (2.13)
The Gram-Schmidt Orthonormalization Procedure
In the previous section, we discussed the fact that any vector can be represented as a linear combination of the basis vectors and as such, we can express an vector simply in terms of the scalar coefficients (components) that multiply the basis vectors. The Gram-Schmidt Orthonormalization procedure is a neat way of obtaining an orthonormal basis from any given set of vectors. Suppose we assume that we have a basis of N vectors, but they are not orthonormal. Let us call these vectors that form the basis as v1 , v2 , · · · , vN . We can generate an orthonormal set as follows: 1. Step 1: Lete1 =
v1 kv1 k
; ⇒ ke1 k = 1
0
2. Step 2: Let e2 = v2 − (v2 .e1 )e1 . This eliminates the component of v2 along e1 . We can then define: 0
e2 =
e2 0 ke2 k
(2.14)
We can easily show that e2 and e1 are orthogonal. we do that by demonstrating that their inner product is zero. Consider the inner product of e2 and e1 : e2 .e1 =
v2 − (v2 .e1 )e1 v2 .e1 − (v2 .e1 )(e1 .e1 ) .e1 = =0 0 0 ke2 k ke2 k
(2.15)
0
because e1 .e1 = 1 by definition. Note that e2 can never be zero since we have assumed that v1 , v2 , · · · , vN are linearly independent. 0
3. Step 3: Repeat Step 2 by creating e3 = v3 − (v3 .e1 )e1 − (v3 .e2 )e2 . The third vector of the orthonormal set will be e3 =
2.3
0 e3 0 ke3 k
. This process can be continued till all the orthonormal vectors are determined.
Communication Signals as Vectors in a Vector Space
Consider a communication system as shown in Figure 2.2. In this simplified communication system, we assume that the source produces one of M symbols or alphabets denoted by mi . When the transmitter receives the alphabet mi , it maps it into a finite duration energy waveform si (t) for i = 1, 2, 3, · · · , M. Each of these waveforms lasts for a duration of T seconds. We first establish that the M waveforms can be represented as elements of a vector space that is spanned by a set of N orthonormal functions φ1 (t), φ2 (t), φ3 (t), · · · , φN (t) where N ≤ M and each of the orthonormal functions is a finite energy signal that lasts for T seconds. It suffices to say that the set of functions s1 (t), s2 (t), · · · , sM (t) belong to a subspace of the vector space L2 defined as the set of all square integrable functions. In L2 , the inner product of two “vectors” f = f (t) and g = g(t) is given by: Z ∞
f.g = −∞
f (t)g∗ (t)dt
(2.16)
LECTURE 2. SIGNAL SPACE ANALYSIS
15
Figure 2.2: Simple communication system and the norm of a vector f = f (t) is defined as: kfk =
√
rZ
∞
| f (t)|2 dt
f.f =
(2.17)
−∞
The triangle inequality here becomes: Z ∞ 1/2 Z 2 | f (t) + g(t)| dt ≤ −∞
∞
2
1/2
Z
| f (t)| dt
+
−∞
and the Cauchy-Schwartz inequality becomes: Z ∞ Z ∗ f (t)g (t)dt ≤ −∞
∞
| f (t)|2 dt
∞
2
1/2
|g(t)| dt
(2.18)
−∞
1/2 Z
−∞
∞
|g(t)|2 dt
1/2 (2.19)
−∞
When we talk about the subspace spanned by the N orthonormal functions, we restrict our limits to [0, T ].
2.3.1
Geometric representation of signals
The set of signals si (t) for i = 1, 2, 3, · · · , M belong to a subspace of L2 . Suppose we can determine N orthonormal basis functions φi (t) for i = 1, 2, 3, · · · , N that span this subspace. We can then write every one of the signals si (t) for i = 1, 2, 3, · · · , M as a linear combination of the basis functions. That is: N
si (t) = ∑ si j φ j (t)
(2.20)
i=1
for 0 ≤ t ≤ T and for i = 1, 2, 3, · · · , M. Here si j is a scalar. In other words, we can simply express each signal si (t) in terms of its components si j . For example, we can write si (t) = si = (si1 , si2 , · · · , siN ). In the case of all real signals, certain properties become clear: Z T 1 if i = j φi (t)φ j (t)dt = δi j = (2.21) 0 if i 6= j 0 Also, the components of the vector si are obtained using the following relation: " # Z Z Z T
si j =
0
T
si (t)φ j (t)dt =
0
N
∑ sik φk (t) φ j (t)dt =
k=1
N
∑ sik
k=1
T
0
φk (t)φ j (t)dt
Clearly, from (2.21) the integral exists and it is one only if k = j giving us the desired result.
(2.22)
LECTURE 2. SIGNAL SPACE ANALYSIS
2.3.2
16
Gram-Schmidt Orthonormalization Procedure for Signals
In order to derive the N orthonormal basis functions, we resort to the Gram-Schmidt orthonormalization procedure. In this case however, we have made no assumption about our set of signals (for example that they are linearly independent and they span their own subspace). Consequently, the procedure will be slightly different. 1. Step 1: Let E1 = follows:
RT 2 0 s1 (t)dt be the energy in the signal s1 (t). Then we find the first basis function as
s1 (t) φ1 (t) = √ E1
(2.23)
Note that 0T φ21 (t)dt = E11 0T s21 (t)dt = 1 and the first function is already normalized. Clearly, s1 (t) = √ √ E1 φ1 (t) ⇒ s1 = ( E1 , 0, 0, · · · , 0). You can draw a parallel to the way in which we looked at the GramSchmidt orthonormalization procedure for vectors. We first divide the given vector by its norm to obtain the first element of the orthonormal basis, and so on. R
R
2. Step 2: Let g2 (t) = s2 (t) − s21 φ1 (t). We can calculate s21 using the relation s21 = 0T s2 (t)φ1 (t)dt. If s2 (t) and s1 (t) are linearly dependent, g2 (t) will be zero. If this is thye case, we proceed to compute g3 (t) = s3 (t) − s31 φ1 (t). Otherwise, we let: R
g2 (t) s2 (t) − s21 φ1 (t) φ2 (t) = qR = q T 2 E2 − s221 0 g2 (t)dt We arrive at the last result using the following relations: Let E2 = Z T 0
g22 (t)dt =
Z T 0
s22 (t)dt + s221 − 2s21
Z T 0
(2.24)
RT 2 0 s2 (t)dt be the energy in s2 (t). Then
s2 (t)φ1 (t)dt = E2 − s221
(2.25)
We know that φ2 (t) has unit energy by definition. We now need to show that φ1 (t) and φ2 (t) are orthogonal to one another. Orthogonality can be shown directly as follows: Z T 0
φ1 (t)φ2 (t)dt = q
1
Z T 0
E2 − s221
[s2 (t) − s21 φ1 (t)] φ1 (t)dt = q
s2 (t)−s21 φ1 (t) √ , we can write s2 (t) = E2 −s221
Also, since φ2 (t) = q and s2 = (s21 , E2 − s221 , 0, 0, · · · , 0).
q
E2 − s221
1 E2 − s221
(s21 − s21 ) = 0
φ2 (t)+s21 φ1 (t) so that s22 =
(2.26)
q
E2 − s221
3. Step 3:The general step in this procedure is as follows. Assume that l − 1 orthonormal basis functions have been determined using k − 1 signals s1 (t), · · · , sk−1 (t). Note that l ≤ k. If none of the gi (t) computed is zero, l = k. We let: l−1
gk (t) = sk (t) − ∑ sk j φ j (t) j=1
(2.27)
LECTURE 2. SIGNAL SPACE ANALYSIS where sk j =
RT 0
17
sk (t)φ j (t)dt for j = 1, 2, 3, · · · , l − 1. If gk (t) 6= 0, we can then compute: gk (t) φl (t) = qR T 2 0 gk (t)dt
(2.28)
4. Step 4: Repeat Step 3 till all the signals are exhausted and we determine the N orthonormal basis functions. Example 2.5: Gram-Schmidt orthonormalization for four signals
Figure 2.3: Signals to be represented as vectors We are required to determine the orthonormal basis for the signals shown in Figure 2.3 and represent them in a signal space.
√
1. Step 1: p Compute φ1 (t) = s1 (t)/ E1 . The energy in the signal s1 (t) is T /3. Thus φ1 (t) = s1 (t)/ T /3. 2. Step 2: Compute g2 (t) = s2 (t) − s21 φ1 (t). From the expressions for calculating the components, Z T
s21 =
0
Z T
E2 =
0
1 T /3
s2 (t)φ1 (t)dt = p
Z T /3 0
s2 (t)s1 (t)dt =
p
T /3
s22 (t)dt = 2T /3
(2.29) (2.30)
The signal s21 φ1 (t) and g2 (t) are shown in Figure 2.4. The energy in g2 (t) is p R φ2 (t) = g2 (t)/[ 0T g22 (t)dt] = g2 (t)/ T /3. This is also shown in Figure 2.4.
p T /3 so that
3. Step 3: Compute g3 (t) = s3 (t) − s31 φ1 (t) − s32 φ2 (t). Here Z T
s31 =
0
s3 (t)φ1 (t)dt = 0
Z T
s32 =
0
s3 (t)φ2 (t)dt =
p
(2.31) T /3
(2.32)
Figure 2.4 shows g3 (t) = s3 (t) − s32 φ2 (t) since s31 p p= 0. Once again, the energy in g3 (t) is R T /3 so that φ3 (t) = g3 (t)/[ 0T g23 (t)dt] = g3 (t)/ T /3 as shown in Figure 2.4.
LECTURE 2. SIGNAL SPACE ANALYSIS
18
4. Step 4: Now we consider the last signal. When we compute g4 (t) = s4 (t) p− −s41 φ1 (t) − s42 φ2 (t) − s43 φ3 (t), we find that g4 (t) = 0. This is because s41 = s42 = s43 = T /3 and s4 (t) can be expressed as a linear combination of φ1 (t), φ2 (t) and φ3 (t). There is no need for a fourth basis function.
Figure 2.4: Signals computed to determine the orthonormal basis Now that we know all the coefficients si j , we can easily express the signals as vectors as follows:
p s1 = T /3, 0, 0 p p s3 = 0, T /3, T /3
T /3, 0 p p p s4 = T /3, T /3, T /3
s2 =
p
T /3,
p
(2.33) (2.34)
LECTURE 2. SIGNAL SPACE ANALYSIS The four signals are plotted in the vector subspace spanned by φ1 (t), φ2 (t) and φ3 (t) as shown in Figure 2.5.
Figure 2.5: Signals computed to determine the orthonormal basis
19
