Optimal binary communication with nonequal probabilities ...
IEEE TRANSACTIONS ON COMMUNICATIONS, VOL. 51, NO. 9, SEPTEMBER 2003
1435
Optimal Binary Communication With Nonequal Probabilities Israel Korn, John P. Fonseka, and Shaohui Xing
Abstract—Optimal signal energies are derived for optimal binary digital communication systems with arbitrary signal probabilities and correlation with both coherent and noncoherent detection. The resulting bit-error probability (BEP) is computed and compared with the BEP of the same systems with equal signal energies. One of the conclusions is that for the coherent system with nonegative correlation, and for noncoherent system with arbitrary correlation, the optimal signals are on-off keying (OOK), i.e., the 0 5 has energy , while the second signal with probability signal has zero energy where is the average signal energy. The proposed system is also better than a system with source coding and equiprobable signals. Index Terms—Coherent detection, noncoherent detection, nonequal probabilities, optimal detection.
where is the conditional probability of making a deciwhen is transmitted. sion in favor of This problem can be discussed with coherent detection and noncoherent detection separately. We will first consider coherent detection in detail in the remainder of this section and in Section II, and extend the study to one particular noncoherent detection technique in Section III. The problem with coherent detection is discussed in many textbooks on digital communications and to the best of our knowledge, the complete solution to the problem has never been presented. The optimal coherent receiver (see, for example, [1, Ch. 4], [2], and [3, Ch. 3]) computes
I. INTRODUCTION
S
INCE the beginning of digital communications, an important problem was finding the optimal binary system and the computation of the resulting bit-error probability (BEP). The problem can be formulated as follows. Let
(5) is transmitted if the result is positive, and and decides that is transmitted if the result is negative. The BEP is ([1, Eq. 4.79(b)], [2, Eq. 4.3.5], or [3, Eq. 3.99])
(1) be arbitrary binary signals with energies , (if ities and ) where and and correlation
,
, and probabil, we shall call have unit energy
(2) The average energy per bit is (3) , where is either The received signal is or and is zero-mean, white, Gaussian noise with . The problem is to find the optimal power spectral density [maximum a posteriori (MAP)] receiver, the optimal , the optimal energies , , so that the BEP is minimum for a given value of . The BEP is (4) Paper approved by A. Ahlen, the Editor for Modulation and Signal Design of the IEEE Communications Society. Manuscript received May 15, 2002; revised December 15, 2002; February 15, 2003; and March 15, 2003. I. Korn is with the School of Electrical Engineering and Telecommunications, The University of New South Wales, Sydney, NSW 2052, Australia. J. P. Fonseka and S. Xing are with the School of Engineering and Computer Science, University of Texas at Dallas, Richardson, TX 75080 USA (e-mail: [email protected]). Digital Object Identifier 10.1109/TCOMM.2003.816985
(6) where
(7) is the standard -function [1]. At this stage it is asand or . sumed in all textbooks that either for arbiThe problem of finding the optimal values of , , , and has not been trary discussed in the literature. The solution to this problem is presented in the next section. This problem of nonuniform symbol ) arises in many practical image and probabilities (i.e., compression techniques. This is discussed in more detail in [4], which contains more references on this subject. II. SOLUTION TO THE PROBLEM The BEP is minimized when is maximized because is a decreasing function of . For arbitrary , we have from (3) and (7)
(8) and the problem is to find the optimal value of ( cannot exceed because ) so that The solution is presented as two special cases.
0090-6778/03$17.00 © 2003 IEEE
is maximum.
1436
IEEE TRANSACTIONS ON COMMUNICATIONS, VOL. 51, NO. 9, SEPTEMBER 2003
Fig. 1. Variation of A with E steps of 0.1, when p : .
= 01
when is varied between
01 and +1 in
Case 1) Orthogonal Signals. , thus In this case, (9) The maximum value occurs at the boundary and . If we seas is usually done in all textlect . Since , books, with equality only if . This implies the following unexpected optimal solution , , i.e., we do not use the second orthogonal signal. In fact, we have , the two solutions on-off keying (OOK). For , and , are spe, cial cases of the general solution , where is an arbitrary angle. Case 2) Nonorthogonal Signals. The first and second derivatives of in (8) with are, respectively respect to (10) (11) We see that the sign of the second derivative is de, is a termined by the sign of . Thus, for (with a minimum), while for convex function of , is a concave function of (with a maximum). In Fig. 1, we plot a normalized
as a function of normalized several values of and
,
(12) for . We can observe
Fig. 2.
p
Variation of BEP of optimal and conventional systems for
= 0 :1 .
0, when
the convexity for and concavity for . Equating (10) to zero, we obtain the following and a two roots which give a minimum for : maximum for (13) (14) , the maximum of occurs at the , , , which is OOK. In Fig. 2, we plot the for optimal and , BEP as a function of and also for conventional system (with , hence, ) for several values . We can see from this of nonnegative and figure that for nonnegative , the BEP of the optimal system (OOK) is much better than that of the conventional system, particularly at lower values of . , the improveFor example, when the BEP is is about 7.1 dB for and is even ment in greater for higher values of . , the maximum of occurs when For of (14). From (14) and (3) Thus for edge with
(15) When we substitute (14) and (16) in (8), we obtain the maximum value
(16)
IEEE TRANSACTIONS ON COMMUNICATIONS, VOL. 51, NO. 9, SEPTEMBER 2003
1437
(17) and (18). Similar to (5) in coherent detection, the optimal receiver in noncoherent orthogonal FSK computes (19) if positive, and in favor of and decides in favor of otherwise. The error probability of the optimal receiver can be and according calculated from (4) by calculating to (17) and (18) as
(20)
Fig. 3. Variation of BEP of optimal and conventional systems for < 0, when p = 0:1.
In Fig. 3, we plot the BEP as a function of for and , and for the conventional system, optimal . We for several values of negative when can see from this figure that for negative values of , there is also a significant improvement in performance relative to the conventional system, particularly for lower values of . For example, when BEP , the improvement in is about 4.5 dB is and about 6.0 dB for . for III. OPTIMAL NONCOHERENT ORTHOGONAL FSK In this section, we extend the study to noncoherent orthogonal [1]–[3], [5]. It is known frequency-shift keying (FSK) that in noncoherent orthogonal FSK, the decision is based on and , at the envelopes of the two matched filter outputs, and , which are separated by the respective frequencies . For the other values of the signaling rate, , the correlation lies in the range [5]. The optimal noncoherent receiver is well documented for the , which reduces to a comparison equal energy signals and for equiprobable symbols ([5, of the two envelopes pp. 302–311]). is transmitted, and follow It is known that when Rician and Rayleigh distributions, respectively, and similarly, is transmitted, the two distributions are reversed. when Specifically, the corresponding conditional probability density functions (pdfs) are [5]
(17)
(18) and are independent for orthogonal sigObserving that naling, the optimal MAP decision rule can be derived by maxiover the two signals ( 1 and 2) using mizing
(21) and are the decision regions of and , where respectively, derived from (19). For given values of and we can find the optimal values of and that mininumerically. The optimal solution is, interestingly, mize the same as in coherent detection with nonegative correlation, and . Hence, the optimal receiver namely, consists of a single envelope detector followed by a decision devise that uses in place of (19). The overall error probability of the above optimal receiver can be written as (22) by solving , and is the first-order Marcum -function [5]. Fig. 4 shows the BEP 0.1, 0.4, and 0.5 variation of the optimal receiver when along with that of the conventional envelope detector with which has the same variation for all . It is , the optimal receiver performs seen that even when about 0.25 dB better than the conventional receiver at a BEP . In other cases, the improvement in over the of is about 1.3 dB when conventional receiver at a BEP of 0.4 and 7.5 dB when 0.1. , Even though OOK has been found to be optimal when it can be deduced that it is optimal for any arbitrary value of . It can be deduced by following the analysis of [3, Sec. 5.2.4] or [5, , the BEP is a decreasing function of Sec. 5.4.3] that if (in [3] and [5] this has been demonstrated for and , but the same dependence on can be obtained for arbitrary and ). Therefore, the BEP for any other arbitrary value of cannot be less than that in (22), which is calculated . Since OOK converts the initial system with arbitrary for , signals into a system with orthogonal signals because OOK becomes the optimal noncoherent signaling scheme for arbitrary . OOK has an additional advantage over conventional FSK as it has a narrower bandwidth because it is the same as that of amplitude-shift keying (ASK). where
can
be
numerically
found
1438
IEEE TRANSACTIONS ON COMMUNICATIONS, VOL. 51, NO. 9, SEPTEMBER 2003
when 1. Similarly, it is seen from Fig. 4 that in noncoperherent orthogonal FSK, optimal transmission when forms 4.2 dB better than optimal noncoherent transmission with source coding. Hence, it is seen that a coded system requires channel coding in addition to optimal source coding to outperform optimal transmission of nonequal binary digits. However, it is mentioned here that the uncoded scheme requires a higher bandwidth than the coded scheme, specifically by a factor equal . to V. CONCLUSIONS
Fig. 4. Variation of BEP of optimal and conventional noncoherent orthogonal FSK when p 0.1, 0.4, and 0.5.
=
IV. COMPARISON WITH CODED SYSTEMS In this section, we compare the performance of the above optimal schemes with schemes that employ source coding to compress data before transmission. It is known that source coding reduces the transmission rate by attempting to make the encoded symbols equiprobable [6]. It follows from the source coding theorem that an optimal binary source encoder generates equiprobable bits with a reduction in the transmission rate by a factor equal to the entropy of the source [6] , which results in an increased coded energy . Hence, the performance of a scheme with optimal source encoding can be found by using a modified bit energy along with a known conventional receiver for . , and the performance For example, when with optimal source coding can be obtained by shifting the BEP by dB. Hence, variation for it is seen from Figs. 2 and 3 that optimal transmission performs better than a scheme that employs an optimal source encoder, by 0, 2.7 dB when 0.25, and 1.2 dB about 3.8 dB when
We have computed the optimal energies and BEP variations of two optimal binary communication systems with arbitrary probabilities and correlations. For an optimal system with cothe optimal herent detection, we have shown that for has energy and , system is OOK, i.e., signal is the probability of where is the average energy and . The optimal system performs better than the consignal ventional system, and the improvement increases with and decreases with . In case of envelope detection, the optimal system is again OOK for any arbitrary correlation. We have shown that the optimal system performs better than a system that employs source coding to make signals equiprobable and use equal energies on the two signals. REFERENCES [1] J. M. Wozencraft and I. M. Jacobs, Principles of Communication Engineering. Prospect Heights, IL: Waveland Press, 1990. [2] P. Z. Peebles, Jr., Digital Communication Systems. Englewood Cliffs, NJ: Prentice-Hall, 1987. [3] M. K. Simon, M. K. Hinedi, and W. C. Lindsey, Digital Communication Techniques. Englewood Cliffs, NJ: Prentice-Hall, 1995. [4] H. Kuai, F. Alajaji, and G. Takahara, “Tight error bounds for nonuniform signaling over AWGN channels,” IEEE Trans. Inform. Theory, vol. 46, pp. 2712–2718, Nov. 2000. [5] J. B. Proakis, Digital Communications, 4th ed. New York: McGrawHill, 2001. [6] R. G. Gallager, Information Theory and Reliable Communication. New York: Wiley, 1968.
1435
Optimal Binary Communication With Nonequal Probabilities Israel Korn, John P. Fonseka, and Shaohui Xing
Abstract—Optimal signal energies are derived for optimal binary digital communication systems with arbitrary signal probabilities and correlation with both coherent and noncoherent detection. The resulting bit-error probability (BEP) is computed and compared with the BEP of the same systems with equal signal energies. One of the conclusions is that for the coherent system with nonegative correlation, and for noncoherent system with arbitrary correlation, the optimal signals are on-off keying (OOK), i.e., the 0 5 has energy , while the second signal with probability signal has zero energy where is the average signal energy. The proposed system is also better than a system with source coding and equiprobable signals. Index Terms—Coherent detection, noncoherent detection, nonequal probabilities, optimal detection.
where is the conditional probability of making a deciwhen is transmitted. sion in favor of This problem can be discussed with coherent detection and noncoherent detection separately. We will first consider coherent detection in detail in the remainder of this section and in Section II, and extend the study to one particular noncoherent detection technique in Section III. The problem with coherent detection is discussed in many textbooks on digital communications and to the best of our knowledge, the complete solution to the problem has never been presented. The optimal coherent receiver (see, for example, [1, Ch. 4], [2], and [3, Ch. 3]) computes
I. INTRODUCTION
S
INCE the beginning of digital communications, an important problem was finding the optimal binary system and the computation of the resulting bit-error probability (BEP). The problem can be formulated as follows. Let
(5) is transmitted if the result is positive, and and decides that is transmitted if the result is negative. The BEP is ([1, Eq. 4.79(b)], [2, Eq. 4.3.5], or [3, Eq. 3.99])
(1) be arbitrary binary signals with energies , (if ities and ) where and and correlation
,
, and probabil, we shall call have unit energy
(2) The average energy per bit is (3) , where is either The received signal is or and is zero-mean, white, Gaussian noise with . The problem is to find the optimal power spectral density [maximum a posteriori (MAP)] receiver, the optimal , the optimal energies , , so that the BEP is minimum for a given value of . The BEP is (4) Paper approved by A. Ahlen, the Editor for Modulation and Signal Design of the IEEE Communications Society. Manuscript received May 15, 2002; revised December 15, 2002; February 15, 2003; and March 15, 2003. I. Korn is with the School of Electrical Engineering and Telecommunications, The University of New South Wales, Sydney, NSW 2052, Australia. J. P. Fonseka and S. Xing are with the School of Engineering and Computer Science, University of Texas at Dallas, Richardson, TX 75080 USA (e-mail: [email protected]). Digital Object Identifier 10.1109/TCOMM.2003.816985
(6) where
(7) is the standard -function [1]. At this stage it is asand or . sumed in all textbooks that either for arbiThe problem of finding the optimal values of , , , and has not been trary discussed in the literature. The solution to this problem is presented in the next section. This problem of nonuniform symbol ) arises in many practical image and probabilities (i.e., compression techniques. This is discussed in more detail in [4], which contains more references on this subject. II. SOLUTION TO THE PROBLEM The BEP is minimized when is maximized because is a decreasing function of . For arbitrary , we have from (3) and (7)
(8) and the problem is to find the optimal value of ( cannot exceed because ) so that The solution is presented as two special cases.
0090-6778/03$17.00 © 2003 IEEE
is maximum.
1436
IEEE TRANSACTIONS ON COMMUNICATIONS, VOL. 51, NO. 9, SEPTEMBER 2003
Fig. 1. Variation of A with E steps of 0.1, when p : .
= 01
when is varied between
01 and +1 in
Case 1) Orthogonal Signals. , thus In this case, (9) The maximum value occurs at the boundary and . If we seas is usually done in all textlect . Since , books, with equality only if . This implies the following unexpected optimal solution , , i.e., we do not use the second orthogonal signal. In fact, we have , the two solutions on-off keying (OOK). For , and , are spe, cial cases of the general solution , where is an arbitrary angle. Case 2) Nonorthogonal Signals. The first and second derivatives of in (8) with are, respectively respect to (10) (11) We see that the sign of the second derivative is de, is a termined by the sign of . Thus, for (with a minimum), while for convex function of , is a concave function of (with a maximum). In Fig. 1, we plot a normalized
as a function of normalized several values of and
,
(12) for . We can observe
Fig. 2.
p
Variation of BEP of optimal and conventional systems for
= 0 :1 .
0, when
the convexity for and concavity for . Equating (10) to zero, we obtain the following and a two roots which give a minimum for : maximum for (13) (14) , the maximum of occurs at the , , , which is OOK. In Fig. 2, we plot the for optimal and , BEP as a function of and also for conventional system (with , hence, ) for several values . We can see from this of nonnegative and figure that for nonnegative , the BEP of the optimal system (OOK) is much better than that of the conventional system, particularly at lower values of . , the improveFor example, when the BEP is is about 7.1 dB for and is even ment in greater for higher values of . , the maximum of occurs when For of (14). From (14) and (3) Thus for edge with
(15) When we substitute (14) and (16) in (8), we obtain the maximum value
(16)
IEEE TRANSACTIONS ON COMMUNICATIONS, VOL. 51, NO. 9, SEPTEMBER 2003
1437
(17) and (18). Similar to (5) in coherent detection, the optimal receiver in noncoherent orthogonal FSK computes (19) if positive, and in favor of and decides in favor of otherwise. The error probability of the optimal receiver can be and according calculated from (4) by calculating to (17) and (18) as
(20)
Fig. 3. Variation of BEP of optimal and conventional systems for < 0, when p = 0:1.
In Fig. 3, we plot the BEP as a function of for and , and for the conventional system, optimal . We for several values of negative when can see from this figure that for negative values of , there is also a significant improvement in performance relative to the conventional system, particularly for lower values of . For example, when BEP , the improvement in is about 4.5 dB is and about 6.0 dB for . for III. OPTIMAL NONCOHERENT ORTHOGONAL FSK In this section, we extend the study to noncoherent orthogonal [1]–[3], [5]. It is known frequency-shift keying (FSK) that in noncoherent orthogonal FSK, the decision is based on and , at the envelopes of the two matched filter outputs, and , which are separated by the respective frequencies . For the other values of the signaling rate, , the correlation lies in the range [5]. The optimal noncoherent receiver is well documented for the , which reduces to a comparison equal energy signals and for equiprobable symbols ([5, of the two envelopes pp. 302–311]). is transmitted, and follow It is known that when Rician and Rayleigh distributions, respectively, and similarly, is transmitted, the two distributions are reversed. when Specifically, the corresponding conditional probability density functions (pdfs) are [5]
(17)
(18) and are independent for orthogonal sigObserving that naling, the optimal MAP decision rule can be derived by maxiover the two signals ( 1 and 2) using mizing
(21) and are the decision regions of and , where respectively, derived from (19). For given values of and we can find the optimal values of and that mininumerically. The optimal solution is, interestingly, mize the same as in coherent detection with nonegative correlation, and . Hence, the optimal receiver namely, consists of a single envelope detector followed by a decision devise that uses in place of (19). The overall error probability of the above optimal receiver can be written as (22) by solving , and is the first-order Marcum -function [5]. Fig. 4 shows the BEP 0.1, 0.4, and 0.5 variation of the optimal receiver when along with that of the conventional envelope detector with which has the same variation for all . It is , the optimal receiver performs seen that even when about 0.25 dB better than the conventional receiver at a BEP . In other cases, the improvement in over the of is about 1.3 dB when conventional receiver at a BEP of 0.4 and 7.5 dB when 0.1. , Even though OOK has been found to be optimal when it can be deduced that it is optimal for any arbitrary value of . It can be deduced by following the analysis of [3, Sec. 5.2.4] or [5, , the BEP is a decreasing function of Sec. 5.4.3] that if (in [3] and [5] this has been demonstrated for and , but the same dependence on can be obtained for arbitrary and ). Therefore, the BEP for any other arbitrary value of cannot be less than that in (22), which is calculated . Since OOK converts the initial system with arbitrary for , signals into a system with orthogonal signals because OOK becomes the optimal noncoherent signaling scheme for arbitrary . OOK has an additional advantage over conventional FSK as it has a narrower bandwidth because it is the same as that of amplitude-shift keying (ASK). where
can
be
numerically
found
1438
IEEE TRANSACTIONS ON COMMUNICATIONS, VOL. 51, NO. 9, SEPTEMBER 2003
when 1. Similarly, it is seen from Fig. 4 that in noncoperherent orthogonal FSK, optimal transmission when forms 4.2 dB better than optimal noncoherent transmission with source coding. Hence, it is seen that a coded system requires channel coding in addition to optimal source coding to outperform optimal transmission of nonequal binary digits. However, it is mentioned here that the uncoded scheme requires a higher bandwidth than the coded scheme, specifically by a factor equal . to V. CONCLUSIONS
Fig. 4. Variation of BEP of optimal and conventional noncoherent orthogonal FSK when p 0.1, 0.4, and 0.5.
=
IV. COMPARISON WITH CODED SYSTEMS In this section, we compare the performance of the above optimal schemes with schemes that employ source coding to compress data before transmission. It is known that source coding reduces the transmission rate by attempting to make the encoded symbols equiprobable [6]. It follows from the source coding theorem that an optimal binary source encoder generates equiprobable bits with a reduction in the transmission rate by a factor equal to the entropy of the source [6] , which results in an increased coded energy . Hence, the performance of a scheme with optimal source encoding can be found by using a modified bit energy along with a known conventional receiver for . , and the performance For example, when with optimal source coding can be obtained by shifting the BEP by dB. Hence, variation for it is seen from Figs. 2 and 3 that optimal transmission performs better than a scheme that employs an optimal source encoder, by 0, 2.7 dB when 0.25, and 1.2 dB about 3.8 dB when
We have computed the optimal energies and BEP variations of two optimal binary communication systems with arbitrary probabilities and correlations. For an optimal system with cothe optimal herent detection, we have shown that for has energy and , system is OOK, i.e., signal is the probability of where is the average energy and . The optimal system performs better than the consignal ventional system, and the improvement increases with and decreases with . In case of envelope detection, the optimal system is again OOK for any arbitrary correlation. We have shown that the optimal system performs better than a system that employs source coding to make signals equiprobable and use equal energies on the two signals. REFERENCES [1] J. M. Wozencraft and I. M. Jacobs, Principles of Communication Engineering. Prospect Heights, IL: Waveland Press, 1990. [2] P. Z. Peebles, Jr., Digital Communication Systems. Englewood Cliffs, NJ: Prentice-Hall, 1987. [3] M. K. Simon, M. K. Hinedi, and W. C. Lindsey, Digital Communication Techniques. Englewood Cliffs, NJ: Prentice-Hall, 1995. [4] H. Kuai, F. Alajaji, and G. Takahara, “Tight error bounds for nonuniform signaling over AWGN channels,” IEEE Trans. Inform. Theory, vol. 46, pp. 2712–2718, Nov. 2000. [5] J. B. Proakis, Digital Communications, 4th ed. New York: McGrawHill, 2001. [6] R. G. Gallager, Information Theory and Reliable Communication. New York: Wiley, 1968.
