Nonlinear Channel Equalization Using Concurrent Support Vector
Nonlinear Channel Equalization Using Concurrent Support Vector Machine Processor Jae Woo Wee1, Tae Seon Kim2, Sung Soo Dong3, and Chong Ho Lee4 1
Korean Intellectual Property Office, Dunsan-dong, Seo-gu, DaeJeon, 302-701, Korea [email protected] 2 Catholic University of Korea, Bucheon, 420-743, Korea [email protected] 3 Yongin-Songdam College, Yongin, 449-710, Korea [email protected] 4 Inha University, Yonghyun-dong, Nam-gu, Incheon, 402-751, Korea [email protected]
Abstract. We developed a high-speed concurrent support vector machine (CSVM) processor for real-time nonlinear channel equalization. All phases of the recognition process, including kernel computing, learning, and recall of the support vector machine (SVM) are performed on a single chip. The concurrent operation of this CSVM using a parallel architecture of elements allows it to achieve high speed. The hardware-friendly kernel adatron (KA) SVM learning algorithms are embedded on a chip. The results of the nonlinear channel equalization obtained by the KA algorithm are compared with those obtained by the quadratic programming (QP) method. The CSVM using the KA learning algorithm is designed and implemented using the FPGA chip. The CSVM processor performs 20% faster than the existing SVM processors.
1 Introduction Nonlinear channel equalization is a major issue in digital communications, because it affects the transmitted sequence by causing both linear (inter-symbol interference) and nonlinear (amplifiers and converters) distortions. The support vector machine (SVM) has shown promise in the area of nonlinear equalization or nonlinear detection [1]. The objective of this study is to design and implement a nonlinear channel equalizer based on the support vector machine. The equalization performances obtained using the SVM and KA learning technique are verified, and a fast hardware architecture is presented based on parallelism, a shared bus and distributed memory. The circuit is implemented using hardware description language (HDL). We chose the FPGA chip for the hardware realization of our channel equalizer. The application of the SVM to nonlinear channel communication systems was first proposed by Sebald et al. [1], who compared the equalization performance between the SVM and Artificial Neural Network (ANN). According to [1], the SVM shows similar or superior performance as compared to the ANN. They trained an SVM using J. Wang et al. (Eds.): ISNN 2006, LNCS 3973, pp. 120 – 127, 2006. © Springer-Verlag Berlin Heidelberg 2006
Nonlinear Channel Equalization Using CSVM Processor
121
the QP method, which is computationally intensive. In this study, to avoid the high computational cost of training the SVM associated with its solving a quadratic optimization problem, we use a fast and simple procedure known as the Kernel Adatron (KA) algorithm [2]. This procedure leads to exactly the same solution, but with an exponential rate of convergence in the number of iterations. Nonlinear channel equalization requires specialized hardware, owing to the intensive and real-time computations involved. Various hardware implementations of the SVM have recently been proposed by many researchers based on parallel architectures, such as that used in the ANN chip. These hardware architectures of the SVM have been published, and have involved the development of both analog and digital hardware [3], [4], [5]. The Kerneltron [3], which internally performs analog and massively parallel kernel computation, successfully deals with real-time applications involving object detection and recognition. This analog hardware performs the recall processes on-chip, leaving the learning process off-chip. A digital hardware version of the SVM (DSVM) [4], [5] proposes a hardware-friendly SVM learning algorithm and performs the learning process on-chip. However, this digital hardware performs the kernel computation off-chip. The concurrent SVM (CSVM) processor is the first SVM processor to conduct all three processes of the SVM on a single chip.
2 Support Vector Machine The SVM expresses the classification or regression output in terms of a linear combination of examples in the training data, in which only a fraction of the data points, called ‘support vectors’, have non-zero coefficients. The support vectors thus capture all the relevant data contained in the training set. Quadratic programming in original SVM training algorithm is not suitable for hardware implementation, due to its complexity and large memory consumption. Thus we use kernel adatron (KA) [2] learning algorithms can be implemented on a silicon chip since these algorithms make use of recursive-updating equations instead of quadratic programming. KA algorithm uses the gradient ascent routine to maximize the margin in feature space. This algorithm is easy to implement and theoretically guaranteed to converge in a finite number of steps to the maximal margin. Consider a two-class classification problem with training set (x1, y1), … ,(xn, yn), where xi ℜd and yi = {+1, -1}. In its basic form, an SVM classifies the pattern vector, x, into class, y, based on the support vectors, xm, and corresponding classes, ym, as in
∈
M
f ( x ) = sign (∑ α m y m K ( x , x m ) + b)
(1)
m =1
Here, {αi} are Lagrange multipliers and the kernel function K( ⋅ , ⋅ ) realizes a dot product in the feature space. KA algorithm is described as follows.
122
J.W. Wee et al.
{Initialization} Set αi = 0 for all i and select learning rate parameter η {Learning loop} repeat for i =1,…n l
zi = ∑ α j y j K ( xi , x j ), j =1
Δα i = η (1 − yi zi )
if (αi + Δαi )>0 then αi ←αi + Δαi else αi ← 0 end for {Termination condition} until (maximum number of iterations reached or γ = 0.5 × [ min ( zi ) − max ( zi )] ≈ 1 ) {i | yi = +1}
{i | y i = −1}
3 CSVM Architecture 3.1 System Overview The hardware architecture, which is composed of the Support Vector Elements (SVEs) holding the support vector streams, is shown in Fig. 1. All of the SVEs are connected via a shared data bus, and the parallel operation of the SVEs generates high throughput. The extension of the SVEs using multiple-chips is also possible, when numerous training patterns are involved. Operating phases of the hardware consists of loading, kernel computation, learning, and recall. The computation of kernel function requires data from one SVE sent to other SVEs via shared data bus and the kernel computations are simultaneously done at all SVEs. The kernel computations are performed faster than those done sequentially by n times. In the learning phase, alphas and bias value are generated in the Learning Element (LE) and the updated parameters are sent to corresponding SVEs. The sign of margin that determines classes of the test data is computed in parallel during recall phase. Also recall phase by parallel operation is faster than by sequential operation by n times. CSVM Equalizer Class 1
shared bus
x (n − M + 1)
SVE1
Σ
uˆ(n − D)
>0
Korean Intellectual Property Office, Dunsan-dong, Seo-gu, DaeJeon, 302-701, Korea [email protected] 2 Catholic University of Korea, Bucheon, 420-743, Korea [email protected] 3 Yongin-Songdam College, Yongin, 449-710, Korea [email protected] 4 Inha University, Yonghyun-dong, Nam-gu, Incheon, 402-751, Korea [email protected]
Abstract. We developed a high-speed concurrent support vector machine (CSVM) processor for real-time nonlinear channel equalization. All phases of the recognition process, including kernel computing, learning, and recall of the support vector machine (SVM) are performed on a single chip. The concurrent operation of this CSVM using a parallel architecture of elements allows it to achieve high speed. The hardware-friendly kernel adatron (KA) SVM learning algorithms are embedded on a chip. The results of the nonlinear channel equalization obtained by the KA algorithm are compared with those obtained by the quadratic programming (QP) method. The CSVM using the KA learning algorithm is designed and implemented using the FPGA chip. The CSVM processor performs 20% faster than the existing SVM processors.
1 Introduction Nonlinear channel equalization is a major issue in digital communications, because it affects the transmitted sequence by causing both linear (inter-symbol interference) and nonlinear (amplifiers and converters) distortions. The support vector machine (SVM) has shown promise in the area of nonlinear equalization or nonlinear detection [1]. The objective of this study is to design and implement a nonlinear channel equalizer based on the support vector machine. The equalization performances obtained using the SVM and KA learning technique are verified, and a fast hardware architecture is presented based on parallelism, a shared bus and distributed memory. The circuit is implemented using hardware description language (HDL). We chose the FPGA chip for the hardware realization of our channel equalizer. The application of the SVM to nonlinear channel communication systems was first proposed by Sebald et al. [1], who compared the equalization performance between the SVM and Artificial Neural Network (ANN). According to [1], the SVM shows similar or superior performance as compared to the ANN. They trained an SVM using J. Wang et al. (Eds.): ISNN 2006, LNCS 3973, pp. 120 – 127, 2006. © Springer-Verlag Berlin Heidelberg 2006
Nonlinear Channel Equalization Using CSVM Processor
121
the QP method, which is computationally intensive. In this study, to avoid the high computational cost of training the SVM associated with its solving a quadratic optimization problem, we use a fast and simple procedure known as the Kernel Adatron (KA) algorithm [2]. This procedure leads to exactly the same solution, but with an exponential rate of convergence in the number of iterations. Nonlinear channel equalization requires specialized hardware, owing to the intensive and real-time computations involved. Various hardware implementations of the SVM have recently been proposed by many researchers based on parallel architectures, such as that used in the ANN chip. These hardware architectures of the SVM have been published, and have involved the development of both analog and digital hardware [3], [4], [5]. The Kerneltron [3], which internally performs analog and massively parallel kernel computation, successfully deals with real-time applications involving object detection and recognition. This analog hardware performs the recall processes on-chip, leaving the learning process off-chip. A digital hardware version of the SVM (DSVM) [4], [5] proposes a hardware-friendly SVM learning algorithm and performs the learning process on-chip. However, this digital hardware performs the kernel computation off-chip. The concurrent SVM (CSVM) processor is the first SVM processor to conduct all three processes of the SVM on a single chip.
2 Support Vector Machine The SVM expresses the classification or regression output in terms of a linear combination of examples in the training data, in which only a fraction of the data points, called ‘support vectors’, have non-zero coefficients. The support vectors thus capture all the relevant data contained in the training set. Quadratic programming in original SVM training algorithm is not suitable for hardware implementation, due to its complexity and large memory consumption. Thus we use kernel adatron (KA) [2] learning algorithms can be implemented on a silicon chip since these algorithms make use of recursive-updating equations instead of quadratic programming. KA algorithm uses the gradient ascent routine to maximize the margin in feature space. This algorithm is easy to implement and theoretically guaranteed to converge in a finite number of steps to the maximal margin. Consider a two-class classification problem with training set (x1, y1), … ,(xn, yn), where xi ℜd and yi = {+1, -1}. In its basic form, an SVM classifies the pattern vector, x, into class, y, based on the support vectors, xm, and corresponding classes, ym, as in
∈
M
f ( x ) = sign (∑ α m y m K ( x , x m ) + b)
(1)
m =1
Here, {αi} are Lagrange multipliers and the kernel function K( ⋅ , ⋅ ) realizes a dot product in the feature space. KA algorithm is described as follows.
122
J.W. Wee et al.
{Initialization} Set αi = 0 for all i and select learning rate parameter η {Learning loop} repeat for i =1,…n l
zi = ∑ α j y j K ( xi , x j ), j =1
Δα i = η (1 − yi zi )
if (αi + Δαi )>0 then αi ←αi + Δαi else αi ← 0 end for {Termination condition} until (maximum number of iterations reached or γ = 0.5 × [ min ( zi ) − max ( zi )] ≈ 1 ) {i | yi = +1}
{i | y i = −1}
3 CSVM Architecture 3.1 System Overview The hardware architecture, which is composed of the Support Vector Elements (SVEs) holding the support vector streams, is shown in Fig. 1. All of the SVEs are connected via a shared data bus, and the parallel operation of the SVEs generates high throughput. The extension of the SVEs using multiple-chips is also possible, when numerous training patterns are involved. Operating phases of the hardware consists of loading, kernel computation, learning, and recall. The computation of kernel function requires data from one SVE sent to other SVEs via shared data bus and the kernel computations are simultaneously done at all SVEs. The kernel computations are performed faster than those done sequentially by n times. In the learning phase, alphas and bias value are generated in the Learning Element (LE) and the updated parameters are sent to corresponding SVEs. The sign of margin that determines classes of the test data is computed in parallel during recall phase. Also recall phase by parallel operation is faster than by sequential operation by n times. CSVM Equalizer Class 1
shared bus
x (n − M + 1)
SVE1
Σ
uˆ(n − D)
>0
