ASYMPTOTIC APPROXIMATIONS TO THE ERROR PROBABILITY FOR DETECTING GAUSSIAN SIGNALS by LEWIS DYE COLLINS
B.S.E.E., Purdue University (1963)
S.M., Massachusetts Institute of Technology (1965)
SUBMITTED IN PARTIAL FULFILLMENT OF THE REQUIREMENTS FOR THE DEGREE OF DOCTOR OF SCIENCE at the
MASSACHUSETTS INSTITUTE OF TECHNOLOGY June, 1968
Signature of Author . . . . . . . . . . . . . . . . . . . . . . . . . Department of Electrical Engineering, May 20, 1968
- ..... Certified by. Accepted
by
.
..
·
· ·
·
·
·
Thesis Supervisor
·
r
Chairman, Departmental Committee on Graduate Students
Archives
( JUL ;; a.
ASYMPTOTIC APPROXIMATIONS TO THE ERROR PROBABILITY FOR DETECTING GAUSSIAN SIGNALS
by LEWIS DYE COLLINS
Submitted to the Department of Electrical Engineering on May 20, 1968 in partial fulfillment of the requirements for the Degree of Doctor of Science.
ABSTRACT
The optimum detector for Gaussian signals in Gaussian noise has been known for many years. However, due to the nonlinear nature of the receiver,
it is extremely
difficult
to calculate
the probability
of making decision errors. Over the years, a number of alternative performance measures have been prepared, none of which are of universal applicability. A new technique is described which combines "tilting" of probability densities with the Edgeworth expansion to obtain an asymptotic expansion for the error probabilities. The unifying thread throughout this discussion of performance For the problem is the semi-invariant moment generating function (s). of detecting Gaussian signals, (s) can be expressed in terms of the Fredholm determinant. Several methods for evaluating the Fredholm determinant are discussed. For the important class of Gaussian random processes which can be modeled via state variables, a straightforward technique for evaluating the Fredholm determinant is presented. A number of examples are given illustrating application of the error approximations
to all three
levels
in the hierarchy
of
detection problems, with the emphasis being on random process problems. The approximation to the error probability is used as a performance index for optimal signal design for Doppler-spread channels subject to an energy constraint. Pontryagin's minimum principle is used to demonstrate that there is no waveform which achieves the best performance. However, a set of signals is exhibited which is capable of near optimum performance. The thesis concludes with a list of topics for future research. THESIS SUPERVISOR: TITLE:
Harry L. Van Trees Associate Professor of Electrical Engineering
3
ACKNOWLEDGMENT
I wish to acknowledge the supervision and encouragement of my doctoral research by Prof. Harry L. Van Trees.
I have
profited immeasurably from my close association with Prof. Van Trees, and for this I am indeed grateful. Profs. Robert G. Gallager and Donald L. Snyder served as readers of this thesis and their encouragement and suggestions are gratefully acknowledged.
I also wish to thank my fellow graduate
students for their helpful suggestions:
Dr. Arthur Baggeroer,
Dr. Michael Austin, Mr. Theodore Cruise, and Mr. Richard Kurth. Mrs. Vera Conwicke skillfully typed the final manuscript. This work was supported by the National Science Foundation and by a Research Laboratory of Electronics Industrial Electronics Fellowship.
TABLE OF CONTENTS
Page ABSTRACT ACKNOWLEDGMENT
3
TABLE OF CONTENTS
4
LIST OF ILLUSTRATIONS
8
CHAPTER I
INTRODUCTION A.
The Detection
B.
The Gaussian Model
12
C.
The Optimum Receiver
13
1.
The Estimator-Correlator
16
2.
Eigenfunction Diversity
18
3.
Optimum Realizable Filter Realization
20
D.
CHAPTER II
CHAPTER III
11 Problem
11
Performance
22
APPROXIMATIONS TO THE ERROR PROBABILITIES FOR BINARY HYPOTHESIS TESTING
26
A.
Tilting of Probability Distributions
26
B.
Expansion in Terms of Edgeworth Series
34
C.
Application
40
D.
Summary
to Suboptimum
Receivers
GAUSSIAN SIGNALS IN GAUSSIAN NOISE A.
Calculation of (s) for FiniteDimensional Gaussian Signals
45
46
46
Page B.
C.
Transition from Finite Set of Observables
48
Special Cases
53
1.
Simple Binary Hypothesis, Zero Mean
53
2.
Symmetric Binary Problem
56
3.
Stationary Bandpass Signals
57
D. Simplified Evaluation for Finite State RandomProcesses
61
E. F.
CHAPTER IV
to Infinite
Semi-Invariant
Moment Generating
Function for Suboptimum Receivers
68
Summary
76
APPROXIMATIONS TO THE PROBABILITY OF ERROR FOR M-ARY ORTHOGONAL DIGITAL COMMUNICATION
77
A.
77
Model
B. Bounds on the Probability C. CHAPTER V
Exponent-Rate
of Error
Curves
79
80
EXAMPLES
85
A. General Binary Problem, KnownSignals, Colored Noise
86
B. Slow Rayleigh Fading with Diversity
88
1.
Simple Binary Problem,
Bandpass Signals 2. 3.
4.
88
Symmetric Binary Problem, Bandpass Signals
92
The Effect of Neglecting Higher Order Terms in the Asymptotic Expansion
95
Optimum Diversity
95
Page
5. 6.
The Inadequacy of the Bhattacharyya Dis tance
99
The Inadequacy of the KullbachLeibler Information Number as a Performance Measure
C.
D.
101 105
Random Phase Angle 1.
Simple Binary, No Diversity
105
2.
Simple Binary, With Diversity
110
Random Process Detection Problems
114
1.
A simple Binary (Radar) Problem,
Symmetric Bandpass Spectrum 2.
A Symmetrical
117
(Communication)
Problem, Symmetric Bandpass Spectra
120
3.
Optimum Time-Bandwidth Product
124
4.
Long Time Intervals, Stationary
Processes (Bandpass Signals, 5. 6.
Symmetric Hypotheses)
124
Pierce's Upper and Lower Bounds (Bandpass, Binary Symmetric)
131
Bounds on the Accuracy
of Our
Asymptotic Approximation for Symmetric Hypotheses and Bandpass Spectra
140
A Suboptimum Receiver for SinglePole Fading
142
8.
A Class of Dispersive Channels
146
9.
The Low Energy Coherence Case
149
7.
E.
Summary
152
7 i
Page CHAPTER VI
OPTIMUM SIGNAL DESIGN FOR SINGLYSPREAD TARGETS (CHANNELS)
154
A. A Lower Bound on
156
B.
Formulation of a Typical Optimization Prob lem
158
Application of Pontryagin's Minimum Principle
162
A Nearly Optimum Set of Signals for Binary Communication
170
TOPICS FOR FURTHER RESEARCH AND SUMMARY
178
A. Error Analysis
178
B.
More than Two Hypotheses
178
C.
Markov (Non-Gaussian) Detection Problems
179
D.
Delay-Spread Targets
180
E.
Doubly-Spread Targets
181
F.
Suboptimum Receivers
181
G.
Bandpass Signals
181
H.
Summary
181
C.
D.
CHAPTER VII
APPENDIX A
APPENDIX B
APPENDIX C
APPENDIX D
(1)
ERROR PROBABILITIES IN TERMS OF TILTED RANDOM VARIABLES
185
p(s) FOR OPTIMAL DETECTION OF GAUSSIAN RANDOM VECTORS
192
u(s) FOR OPTIMAL DETECTION OF GAUSSIAN RANDOM PROCESSES
198
REALIZABLE WHITENING FILTERS
205
REFERENCES
213
BIOGRAPHY
221
8
LIST OF ILLUSTRATIONS Page Receiver
14
1.1
Form of the Optimum
1.2
Estimator-Correlator Realization
19
1.3
Eigenfunction Diversity Realization
19
1.4
Optimum Realizable Filter Realization
21
2.1
Graphical Interpretation of Exponents
33
3.1
State-Variable Random Process Generation
62
3.2
A Class of Suboptimum Receivers
69
3.3
Suboptimum Receiver: Model
4.1
Exponent-Rate Curve for Fading Channel
83
5.1
Bounds and Approximations for Known Signals
89
5.2 5.3
5.4
State-Variable
73
Approximate Receiver Operating Characteristic, Slow Rayleigh Fading
93
Error Probability for Slow Rayleigh Fading with Diversity
96
Optimum Signal-to-Noise Ratio per Channel, Slow Rayleigh Fading
98 102
5.5
p(s)
5.6
Receiver Operating Characteristic, Random Phase Channel
111
Receiver Operating Characteristic, Random 1 Phase Channel, N
115
Receiver Operating Characteristic, Random Phase Channel, N = 2.
116
5.7
5.8
Curves for Two Diversity Channels
9
Page 5.9
5.10
5.11
5.12
5.13
Approximate Receiver Operating Characteristic, Single-Pole Rayleigh Fading
121
Probability of Error, Single-Pole Rayleigh Fading
123
Optimum Time-Bandwidth Product, Single-Pole Rayleigh Fading
125
Optimum Exponent vs. Signal-to-Noise Ratio, Single Square Pulse, Single-Pole Rayleigh Fading Optimum Signal-to-Noise Ratio vs. Order of
Butterworth 5.14
126
Spectrum
129
Optimum Exponent vs. Order of Butterworth
Spectrum
130
5.15
Single-Pole Fading, Suboptimum Receiver
143
5.16
Tapped Delay Line Channel Model
147
5.17
u(s)
for Three Tap Channel
150
6.1
Communication System Model
155
6.2
General Behavior of Optimum Signal
167
6.3
A Nearly Optimum Set of Signals
172
6.4
Optimum Exponent vs. kT, N Samples, SinglePole Fading
174
Optimum Exponent vs. kT, Sequence of Square Pulses, Single-Pole Fading
176
D-1
Realizable Whitening Filter
206
D-2
State-Variable Realizable Whitening Filter
212
6.5
.Lo
DEDI CATION
This thesis is dedicated to
my parents.
i.L
I.
INTRODUCTION
In this chapter, we give a brief discussion of the detection
problem. motivate
Webriefly the
review the form of the optimumreceiver
discussion
and
of performance which occupies the remainder of
this thesis. A.
The Detection
Problem
We shall be concerned with a subclass of the general
problem
which statisticians for a couple of centuries have called "hypothesis testing" [1-31.
Since World War II this mathematical framework has
been applied by engineers to a wide variety of problems in the design and analysis of radar, sonar, and communication systems
4-71.
More
recently it has been applied to other problems as well, such as seismic detection. The general mathematical model for the class of problems that
we shall consider is as follows:
We assume there are M hypotheses
Hi, H2 , a..
a priori probabilitiesP1 , p2
PM
, HM which occur with hypothesis
the
j:r(t) = sj(t) +
m(t)
On the j
where s(t)
is a sample function
...
received waveform is:
+ w(t),
T
0, with equality occurring only in the unin-
is a constant
with probability
one.
Thus,
convex downwardin all cases of interest to us.
29
V(o)
(1)
=
(2.6)
= 0,
so that
u(s)
"tilting"
{tR.(R)
(L)
exp[i(s)-sL]dL
s
f
Pr[ EH21 {R:~ 0
(2.47)
Pr[sIH
< exp[p 2 (s 2 )-s 2Y
for s2 < 0.
(2.48)
2]
The first-order
approximations are,
P'
44
(2.49) 2
Pr[cH 2l] X~
(+s2 xi
(s ))exp[p2 (s2 )-s2 j2 (s2 )+
2* 2 2 22 2
2
2~~T
2
'2 2
(2.50)
where
(2.51) and 2 (s
2
)
(2.52)
y
Since, "l(S)
Var(Z)
42(s)
= Var(L2s)
0,
(2.53)
> 0,
(2.54)
>
l(S) and u 2 (s) are monotone increasing functions of s. Eq. 2.51 has a unique solution
Y > '1(0)
for s > 0 if,
(2.55)
E[ iH1
and Eq. 252 has a unique solution
< 2(o)
Then,
E[ZIH21.
Thus, just as before, we require
for s
·-l-l-LI^---(lll---L---^··Y^LTI-C '
I___
-iOto,4
i
.·--·-·11---·---·1^1_1-i-·---···---1
1-I1
li
. 0 ._r
:_4
"
0 144
.~~~~~ ..
.
1..
.
..
.
. ._.
....
.
-
.
_.
....
.
O:
.
.....
..
.
_._..._. ...
,
...
c....
.. _......... _.:. .~qS~..-\.--
i
-~.l.
- -- -;--
--
-
-So-
.ii.
-
-
0
'
4-)
-
s
C .C -
S
II 2.4
IsM (z -
---
V a0 4,
-4 -
,
l
-
!
~i
*
in f
I
*
l_
-
-
99
the optimum average received signal-to-noise energy ratio per channel
diversity
as a function of the total average received signal-
to-noise ratio for these three different criteria. A couple of comments are in order regarding Fig. 5.4 First, we can achieve the optimum solution only for discrete values In particular, the optimum can be
of total signal-to-noise ratio. achieved only for those values of of the optimum ratio per channel.
2
ErT/N0 which are integer multiples The curves in Fig. 5.4 are smooth
curves drawn through these points. Second, the minimum is not particularly sensitive to the exact value of signal-to-noise ratio per channel.
This
is
demonstrated
graphically in Fig. 7.44 of Ref.[35]. Therefore, the probability of error which results from using any one of the three criteria is practically the same.
The only difference in the two approximate
methods is that the probability of error computed from the Chernoff bound is not as good an approximation to Eq. 5.27 as is Eq. 5.26. It is also instructive to compare these results with those obtained using output signal-to-noise ratio d2 as the performance criterion.
d2
Straightforward calculations yields
2NErT
(5.31)
Hence, the maximum d 2 occurs for N = 1, which does not yield the minimum probability of error, when
5.
2
Er /NO > 3.
The Inadequacy of the Bhattacharyya Distance.
Eq. 2.21, we saw that _(1)
B, the
Bhattacharyya
Distance.
In Thus,
100
the above example is one for which using the Bhattacharyya distance as a performance measure leads to an optimum diversity system, while the "divergence" or d2 does not.
However, a slight modification of
this example produces a situation in which the Bhattacharyya distance is not a good performance measure. We assume that we have available two independent slow Rayleigh fading channels, which are modeled as described at the beginning of this section. transmitted; on H
On hypothesis H2, a signal of energy E is Again, there is
no signal is transmitted.
We wish to investigate
additive white Gaussian noise in each channel.
the consequences of dividing the available energy between the two channels.
For example, we might like to achieve the optimum division
of energy.
We assume the total average received energy is Er, with average received energy E 1 and E2 in the two channels, respectively.
E1
aErr
E 2 = (l-a)Er,
(5.32a)
for 0 < a < 1.
(5.32b)
Note that this problem is symmetric about sufficient to consider the range 0 < a
1 i .j-4 4.J
.X a)
5
LO4 *
$ 0
Un
M
i
tnn 13O i
.r4 LO S4
a
,
E-1I $4I
0
WI i I t
U)
_) _4
105
F the extra effort required for our performance technique is justified. C.
Random Phase Angle In this section, we consider a second example where there
is a finite set of random parameters.
We assume that the received
signal is a band-pass signal about some carrier frequency. random parameter is the phase of the carrier.
The
This phase uncertainty
might arise in practice from instability in the carrier oscillator at either the transmitter or receiver.
This problem has been studied
in great detail [42,581 and the error probabilities have been computed numerically. 1.
Simple Binary, No Diversity.
The mathematical
model for the problem is
H2: r(t) =T
H1: r(t)
2Er
f(t)cos(wct +
(t) +
) +
(t)
(t)
-
0
< t
exp[-
Y],
IH 1]
I
N
i)
Note that
PM
=
' > 0, hence we restrict y to be positive.
Pr[cIH2 ] is not so
variables
(5.41)
0,
tan 0)21 n
N/2
Z n-I
exp
F
in
n
)2
tan
2+
n
I
.J
B
2vr
and f'(x) = 0 only
at x =
'
- 2
2J v7
140
Eq. 5.103
is Pierce's result in our notation.
bounds in Eq. 5.103 differ at most by /F' 6.
Bounds on the Accuracy
The upper and lower
.
of Our Asymptotic
for Symmetric Hypotheses and Bandpass Sectra.
A topic
Approximation
of prime
of error importance is the analysis of the accuracy of our probability approximations.
We would like to be able to make some quantitative
statements about the accuracy which results number of terms in Eq. 2.26.
from retaining
given
For several examples where the tilted
density was far from being Gaussian (e.g. slow Rayleigh fading with no diversity) the first order approximation differs from the exact expressions by only about 10 percent.
In some other examples where
the "exact" error probability is computedfrom the significant eigenvalues of typical random processes, even smaller errors result
I
from our first-order approximation. We have made several attempts at using techniques similar to various Central Limit Theorem proofs [38, 63-65] to
tiI
estimate the accuracy of the first-order approximation, e.g., I
Eqs. 2.32 and 2.33.
However, the estimates we obtained are far
too conservative to be useful.
I I
In this section,
first-order
we obtain upper and lower bounds on our
error approximations for an important subclass of namely, the symmetric hypotheses
problems,
(communication)
with equally likely hypotheses, bandpass signals,
total probability of error criterion.
problem
and a minimum
For this problem (-)(the
Bhattacharrya distance) in the appropriate performance measure, and several of our examples in the previous section were in this class. I I
Furthermore, this is the problem which Turin and Pierce
qasdiscussed in the nreviots section. treated_ 3
_
_
---
-'-------
__
Pierce's upper and __
-I
'
----
_
lower bounds are
exp i(1) 2
)
+ )")2
eP
exp[p(l) 1
Pr i (½A1< Pr[E]
(5.104a)
and
Pr[Ee
0,
(6.26a)
0
0, thiscondition implies nothing
(6.26b)
about s(t) is
Since we have an energy constraint implies the use of an arbitrarily
unbounded.
on s(t),
(6.26c)
the third alternative
short pulse of the required energy.
This gives rise to only one non-zero eigenvalue in our observation.
L.
166
We have seen that for signal-to-noise ratios greater than about three, this is not optimum. Thus, either
s(t)
-
Hence, we discard the third possibility.
0 or 86(t)
0.
The general behavior of the optimum signal is indicated in Fig. 6.2. from (i) to
Denoting the set of times where the solution switches (ii) or vice-versa by (T,
conclude from the differential
1, 2, ... ,
j
N
we then
equations and boundary conditions for
fP(t) and P2(t) that
P1(TN) = P2 (TN) = o
regardless of whether TN
(6.27)
=
Tf.
Wenow shall consider the first-order of algebraic
case in the interest
simplicity, and we shall attempt to obtain additional
information about s(t)in the interval
TN_1 < t
TN. In this interval,
a(t) = 0 P;-[, (t) - z.(t)
- E(t)pl(t) - Z2(t) 2 +
N
(6.28)
Thus, all the derivatives of a(t) must also vanish on the interval
N-I -
i
1 l-sK +SK 2
nm+sKima j 9 I
F--, 2
-
smp(1-s)Kl m! + S2 -2
- 1/2 niK 1i-I 1si|