Multilevel Filtering Elliptic Preconditioners - University of Southern ...
SIAM J. MATRIX ANAL. APPL. Vol. 11, No. 3, pp. 403-429, July 1990
(C) 1990 Society for Industrial and Applied Mathematics
006
Downloaded 01/26/14 to 132.174.255.3. Redistribution subject to SIAM license or copyright; see http://www.siam.org/journals/ojsa.php
MULTILEVEL FILTERING ELLIPTIC PRECONDITIONERS* C.-C. JAY KUOf, TONY F. CHAN:I:,
AND
CHARLES
TONG
Abstract. A class of preconditioners for elliptic problems built on ideas borrowed from the digital filtering theory and implemented on a multilevel grid structure is presented. These preconditioners are designed to be both rapidly convergent and highly parallelizable. The digital filtering viewpoint allows for the use of filter design techniques for constructing elliptic preconditioners and also provides an alternative framework for understanding several other recently proposed multilevel preconditioners. Numerical results are presented to assess the convergence behavior of the new methods and to compare them with other preconditioners of multilevel type, including the usual multigrid method as preconditioner, the hierarchical basis method, and a recent method proposed by Bramble-Pasciak-Xu.
Key words, filtering, multigrid, multilevel, parallel computation, preconditioned conjugate gradient, preconditioners
AMS(MOS) subject classifications. 65N20, 65F10
1. Introduction. Preconditioned conjugate gradient (PCG) methods have been a very popular and successful class of methods for solving large systems of equations arising from discretizations of elliptic partial differential equations. With the advent of parallel computers in recent years, there has been increased research into effectively implementing these methods on various parallel architectures. In this paper, we present a class of preconditioners for elliptic problems built on ideas from the digital filtering theory and implemented on a multilevel grid structure. Our goal is to work towards preconditioners that are both highly parellelizable and rapidly convergent. The idea of preconditioning is a simple one, but it is now recognized as critical to the effectiveness of PCG methods. Suppose we would like to solve the symmetric positive definite linear system Ax b, where A arises from discretizing a second-order self-adjoint elliptic partial differential operator. A good preconditioner for A is a matrix M that approximates A well (in the sense that the spectrum for the preconditioned matrix M- A is clustered around and has a small condition number), and for which the matrix vector product M-Iv can be computed efficiently for a given vector v. With such a where preconditioner, one then solves in principle the preconditioned system .3Y M-I/2AM -1/2, ml/2x and M-/2b, by the conjugate gradient method. Since an effective preconditioner plays a critical role in PCG methods, many classical preconditioners have been proposed and studied, especially for second-order elliptic problems. Among these are the Jacobi preconditioner (diagonal scaling), the symmetric successive overrelaxation (SSOR) preconditioner [3], and the incomplete factorization
,
,
.
Received by the editors September 7, 1989; accepted for publication (in revised form) December 27,
1989.
f Signal and Image Processing Institute and Department of Electrical Engineering Systems, University of Southern California, Los Angeles, California 90089-0272 ([email protected]). The work of this author was supported by the University of Southern California Faculty Research and Innovation Fund. Department of Mathematics, University of California at Los Angeles, Los Angeles, California 90024. chan@math, ucla.edu Department of Computer Science, University of California at Los Angeles, Los Angeles, California 90024 tong@cs, ucla.edu ). The work of the second and third authors was supported in part by the National Science Foundation under contract NSF-DMS87-14612, and by the Army Research Office under contract DAAL03-88-K-0085. Part of this work was performed while they were visiting the Research Institute for Advanced Computer Science, National Aeronautics and Space Administration Research Center. 403
Downloaded 01/26/14 to 132.174.255.3. Redistribution subject to SIAM license or copyright; see http://www.siam.org/journals/ojsa.php
404
C.-C. J. KUO, T. F. CHAN, AND C. TONG
preconditioners (ILU [25] and MILU [15]). These preconditioners have been very successful, especially when implemented on sequential computers. In the parallel implementation of PCG methods, the major bottleneck is often the parallelization of the preconditioner, since the rest ofthe PCG methods can be parallelized in a straightforward way (the only potential bottleneck is the need for innerproducts, but many parallel computers do support fast inner-product evaluations). Unfortunately, previous works [12], [16] have shown that for many of the classical preconditioners, there is a fundamental trade-off in the ease of parallelization and the rate of convergence. A principal obstacle to parallelization is the sequential manner in which many preconditioners traverse the computational gridthe data dependence implicitly prescribed by the method fundamentally limits the amount of parallelism available. Reordering the grid traversal (e.g., from natural to red-black ordering) or inventing new methods (e.g., polynomial preconditioners 2 ], 19 to improve parallelization usually has an adverse effect on the rate of convergence 12 ], 23 ]. The fundamental difficulty can be traced to the global dependence of elliptic problems. An effective preconditioner must account for the global coupling inherent in the original elliptic problem. Preconditioners that use purely local information (such as redblack orderings and polynomial preconditioners) are fundamentally limited in their ability to improve the convergence rate. On the other hand, global coupling through a natural ordering grid traversal is not highly parallelizable. The fundamental challenge is therefore to construct preconditioners that maintain global coupling and are highly parallelizable. Ideas along this line have of course been explored in the development of multigrid methods as solution 10 ], 17 as well as preconditioning techniques 20 ], 21 ], and the more recently proposed hierarchical basis preconditioner 8 ], 29 ]. We are thus led to the consideration of preconditioners that share global information through a multilevel grid structure (ensuring a good convergence rate) but perform only local operations on each grid level (and hence highly parallelizable). Compared with a purely multigrid iteration, we have more flexibility in terms of the choice of inter- and intragrid level operators (such as interpolation, projection, and smoothing), since we are using the multilevel iteration within an outer conjugate gradient iteration. One preconditioner of this type has been proposed recently by Bramble, Pasciak, and Xu [9] and Xu 28]. The methods that we propose in this paper are quite similar to their preconditioner, and our digital filtering framework can be looked at as providing an alternative view of their method. It also allows the flexibility in deriving several variants. The approach taken in this paper and that of Bramble, Pasciak, and Xu differs from that of multigrid methods in that the smoothing operation in multigrid methods is replaced by a simple scaling operation. Other types of multilevel preconditioners have been studied by Vassilevski 27 ], Axelsson and Vassilevski 6 ], 7 ], Kuznetsov 24 and Axelsson 4 ]. The outline of the paper is as follows. In 2, we describe our framework for deriving multilevel filtering preconditioners for a model problem on a single discretization grid. The basic framework is then extended to the multigrid discretization case in 3. In 4, we briefly survey several other preconditioners of the multilevel type. Numerical results for (model, variable coefficient, and discontinuous coefficient) problems in two and three dimensions are presented in 5, comparing the performance of several multilevel preconditioners, including the usual multigrid method as a preconditioner, the hierarchical basis preconditioner, and the method of Bramble-Pasciak-Xu. Some brief concluding remarks are given in 6. We note that the main emphasis of the present paper is on the convergence behavior of these multilevel preconditionersno attempt is made to assess their parallel efficiency. That will be the subject of a forthcoming paper.
Downloaded 01/26/14 to 132.174.255.3. Redistribution subject to SIAM license or copyright; see http://www.siam.org/journals/ojsa.php
MULTILEVEL FILTERING PRECONDITIONERS
405
2. Multilevel filtering preconditioners: Fundamentals. 2.1. Motivation. Consider the one-dimensional discrete Poisson equation on [0, 1] with zero boundary conditions on a uniform grid fh,
(2.1)
-
1-E
-
un=f,
n =1,...,N-I,
2 L, with integer L > 1, and E is the shift operator on 2h. We denote where N h the above system by
Au =f,
where A, u, and f correspond, respectively, to the discrete Laplacian, solution, and forcing functions. Clearly, A is a tridiagonal matrix with diagonal elements -1/2, and It is well known that the matrix A can be diagonalized as
-.
where
AA is a diagonal matrix diag (X,
w=
,
Xu- ),
cos (kh),
)2 ohogonal matrix whose kth row is
and W is an order (N-
(2.3)
WTAA W,
A
(2.2)
sin (k(N- 1)h)).
sin (knh),
(sin (kh),
The diagonalization of the matrix A can be intereted as the decomposition of the driving and solution functions into their Fourier components, i.e.,
a
sin (knh),
u sin (knh), n=l
n=l
k= 1,2,
One can easily verify that
and
,N-1.
are related via
A(I=,
= ,,
,-
,
where
(2.4)
(k)= Xk
cos (krh),
is known as the spectrum of the discrete Laplacian. In order to invert A, we can make use of (2.2) and obtain a fast Poisson solver:
(2.5)
A-l
WrAS W.
The above procedure also serves as the general framework for fast Poisson solvers in cases of higher dimension. However, fast Poisson solvers are not generally applicable for nonseparable elliptic operators and irregular domains. Instead, we want to find good approximations to this solution procedure that are extensible to more general problems and then use them as preconditioners. The fundamental idea is to avoid the use of fast Fourier transform (FFT) and to use instead a sequence of filtering operations to approximate the desired spectral decomposition. This explains the motivation and the name of the multilevel filtering (MF) preconditioner proposed in this paper. Our main idea for deriving the MF preconditioner for A is to divide all admissible wavenumbers into bands and to approximate the spectrum 4 (k) at each band with some
406
C.-C. J. KUO, T. F. CHAN, AND C. TONG
Downloaded 01/26/14 to 132.174.255.3. Redistribution subject to SIAM license or copyright; see http://www.siam.org/journals/ojsa.php
constant. To be more precise, consider the following piecewise constant function in the wavenumber domain
P(k)=Cl,
0.5, z> 0.5, x- 0.5, z_-< 0.5 or y=< 0.5, z> 0.5, x>0.5 with
elsewhere.
10
426
C.-C. J. KUO, T. F. CHAN, AND C. TONG
Downloaded 01/26/14 to 132.174.255.3. Redistribution subject to SIAM license or copyright; see http://www.siam.org/journals/ojsa.php
100
90 80 70
60 504030 20-
10-
10
10 n= 1/h (a)
350O
2500200015001000-
.......
..
MGM’F2
RIC
500 0 10o
I0
I0 2
n=l/h
(b)
FIG. 5.5. (a) Iteration and b operation counts for Test Problem 5.
The number of iterations and operation counts per grid point are plotted in Figs. 5.15.6 (a) and (b), respectively. We can make the following observations from these figures. The BPX and MGMF preconditioners have better convergence behavior than the HB preconditioner, especially for three-dimensional problems. The HB method is competitive with the other multilevel methods only for the discontinuous coefficient problem in two dimensions.
427
Downloaded 01/26/14 to 132.174.255.3. Redistribution subject to SIAM license or copyright; see http://www.siam.org/journals/ojsa.php
MULTILEVEL FILTERING PRECONDITIONERS
600500400-
300-
MGMI?
200/
.-’""
MGMF3
100’"i’","i"’i""
0
10 2
I0
100
n=l/h
xl04 2.5
1.5
MGMF MGMF1 MGMF3
RIC 0
1( o
101
10
n=l/h
(b)
FIG. 5.6. (a) Iteration and (b) operation counts for Test Problem 6.
2 The O(log" n convergence rate for all the multilevel methods is evident, except for the three-dimensional HB method. The three-dimensional HB method behaves like O(h -’59) and O(h -’7) for problems (5.4) and (5.5), which are close to the predicted theoretical result O(h-5). However, for the discontinuous coefficient problem 5.6 ), it converges more slowly, like O(h-126).
428
C.-C. J. KUO, T. F. CHAN, AND C. TONG
Downloaded 01/26/14 to 132.174.255.3. Redistribution subject to SIAM license or copyright; see http://www.siam.org/journals/ojsa.php
(3) In general, the MGMF methods perform slightly better than the corresponding BPX methods. Recall that the only difference between the two methods is the choice of the elementary filters. (4) Filtering twice (BPX2, BPX3, and MGMF2) does improve the convergence rates for the model Poisson problem in either two or three dimensions (the MGMF2 and BPX3 preconditioners appear to be spectrally equivalent.) For variable and discontinuous coefficient problems, filtering twice does not seem to improve the convergence rates enough to compensate for the extra work involved. (5) The MGMF3 method is designed to incorporate the desired features of MGMF and MGMF2, i.e., the good convergence property due to filtering twice and the smaller amount of work due to filtering once at the finest grid level. It turns out that it works very well. MGMF3 behaves better than MGMF but worse than MGMF2 in the number of iterations required. However, in terms of amount of work, MGMF3 is better than MGMF1 and MGMF2. (6) For small n (
(C) 1990 Society for Industrial and Applied Mathematics
006
Downloaded 01/26/14 to 132.174.255.3. Redistribution subject to SIAM license or copyright; see http://www.siam.org/journals/ojsa.php
MULTILEVEL FILTERING ELLIPTIC PRECONDITIONERS* C.-C. JAY KUOf, TONY F. CHAN:I:,
AND
CHARLES
TONG
Abstract. A class of preconditioners for elliptic problems built on ideas borrowed from the digital filtering theory and implemented on a multilevel grid structure is presented. These preconditioners are designed to be both rapidly convergent and highly parallelizable. The digital filtering viewpoint allows for the use of filter design techniques for constructing elliptic preconditioners and also provides an alternative framework for understanding several other recently proposed multilevel preconditioners. Numerical results are presented to assess the convergence behavior of the new methods and to compare them with other preconditioners of multilevel type, including the usual multigrid method as preconditioner, the hierarchical basis method, and a recent method proposed by Bramble-Pasciak-Xu.
Key words, filtering, multigrid, multilevel, parallel computation, preconditioned conjugate gradient, preconditioners
AMS(MOS) subject classifications. 65N20, 65F10
1. Introduction. Preconditioned conjugate gradient (PCG) methods have been a very popular and successful class of methods for solving large systems of equations arising from discretizations of elliptic partial differential equations. With the advent of parallel computers in recent years, there has been increased research into effectively implementing these methods on various parallel architectures. In this paper, we present a class of preconditioners for elliptic problems built on ideas from the digital filtering theory and implemented on a multilevel grid structure. Our goal is to work towards preconditioners that are both highly parellelizable and rapidly convergent. The idea of preconditioning is a simple one, but it is now recognized as critical to the effectiveness of PCG methods. Suppose we would like to solve the symmetric positive definite linear system Ax b, where A arises from discretizing a second-order self-adjoint elliptic partial differential operator. A good preconditioner for A is a matrix M that approximates A well (in the sense that the spectrum for the preconditioned matrix M- A is clustered around and has a small condition number), and for which the matrix vector product M-Iv can be computed efficiently for a given vector v. With such a where preconditioner, one then solves in principle the preconditioned system .3Y M-I/2AM -1/2, ml/2x and M-/2b, by the conjugate gradient method. Since an effective preconditioner plays a critical role in PCG methods, many classical preconditioners have been proposed and studied, especially for second-order elliptic problems. Among these are the Jacobi preconditioner (diagonal scaling), the symmetric successive overrelaxation (SSOR) preconditioner [3], and the incomplete factorization
,
,
.
Received by the editors September 7, 1989; accepted for publication (in revised form) December 27,
1989.
f Signal and Image Processing Institute and Department of Electrical Engineering Systems, University of Southern California, Los Angeles, California 90089-0272 ([email protected]). The work of this author was supported by the University of Southern California Faculty Research and Innovation Fund. Department of Mathematics, University of California at Los Angeles, Los Angeles, California 90024. chan@math, ucla.edu Department of Computer Science, University of California at Los Angeles, Los Angeles, California 90024 tong@cs, ucla.edu ). The work of the second and third authors was supported in part by the National Science Foundation under contract NSF-DMS87-14612, and by the Army Research Office under contract DAAL03-88-K-0085. Part of this work was performed while they were visiting the Research Institute for Advanced Computer Science, National Aeronautics and Space Administration Research Center. 403
Downloaded 01/26/14 to 132.174.255.3. Redistribution subject to SIAM license or copyright; see http://www.siam.org/journals/ojsa.php
404
C.-C. J. KUO, T. F. CHAN, AND C. TONG
preconditioners (ILU [25] and MILU [15]). These preconditioners have been very successful, especially when implemented on sequential computers. In the parallel implementation of PCG methods, the major bottleneck is often the parallelization of the preconditioner, since the rest ofthe PCG methods can be parallelized in a straightforward way (the only potential bottleneck is the need for innerproducts, but many parallel computers do support fast inner-product evaluations). Unfortunately, previous works [12], [16] have shown that for many of the classical preconditioners, there is a fundamental trade-off in the ease of parallelization and the rate of convergence. A principal obstacle to parallelization is the sequential manner in which many preconditioners traverse the computational gridthe data dependence implicitly prescribed by the method fundamentally limits the amount of parallelism available. Reordering the grid traversal (e.g., from natural to red-black ordering) or inventing new methods (e.g., polynomial preconditioners 2 ], 19 to improve parallelization usually has an adverse effect on the rate of convergence 12 ], 23 ]. The fundamental difficulty can be traced to the global dependence of elliptic problems. An effective preconditioner must account for the global coupling inherent in the original elliptic problem. Preconditioners that use purely local information (such as redblack orderings and polynomial preconditioners) are fundamentally limited in their ability to improve the convergence rate. On the other hand, global coupling through a natural ordering grid traversal is not highly parallelizable. The fundamental challenge is therefore to construct preconditioners that maintain global coupling and are highly parallelizable. Ideas along this line have of course been explored in the development of multigrid methods as solution 10 ], 17 as well as preconditioning techniques 20 ], 21 ], and the more recently proposed hierarchical basis preconditioner 8 ], 29 ]. We are thus led to the consideration of preconditioners that share global information through a multilevel grid structure (ensuring a good convergence rate) but perform only local operations on each grid level (and hence highly parallelizable). Compared with a purely multigrid iteration, we have more flexibility in terms of the choice of inter- and intragrid level operators (such as interpolation, projection, and smoothing), since we are using the multilevel iteration within an outer conjugate gradient iteration. One preconditioner of this type has been proposed recently by Bramble, Pasciak, and Xu [9] and Xu 28]. The methods that we propose in this paper are quite similar to their preconditioner, and our digital filtering framework can be looked at as providing an alternative view of their method. It also allows the flexibility in deriving several variants. The approach taken in this paper and that of Bramble, Pasciak, and Xu differs from that of multigrid methods in that the smoothing operation in multigrid methods is replaced by a simple scaling operation. Other types of multilevel preconditioners have been studied by Vassilevski 27 ], Axelsson and Vassilevski 6 ], 7 ], Kuznetsov 24 and Axelsson 4 ]. The outline of the paper is as follows. In 2, we describe our framework for deriving multilevel filtering preconditioners for a model problem on a single discretization grid. The basic framework is then extended to the multigrid discretization case in 3. In 4, we briefly survey several other preconditioners of the multilevel type. Numerical results for (model, variable coefficient, and discontinuous coefficient) problems in two and three dimensions are presented in 5, comparing the performance of several multilevel preconditioners, including the usual multigrid method as a preconditioner, the hierarchical basis preconditioner, and the method of Bramble-Pasciak-Xu. Some brief concluding remarks are given in 6. We note that the main emphasis of the present paper is on the convergence behavior of these multilevel preconditionersno attempt is made to assess their parallel efficiency. That will be the subject of a forthcoming paper.
Downloaded 01/26/14 to 132.174.255.3. Redistribution subject to SIAM license or copyright; see http://www.siam.org/journals/ojsa.php
MULTILEVEL FILTERING PRECONDITIONERS
405
2. Multilevel filtering preconditioners: Fundamentals. 2.1. Motivation. Consider the one-dimensional discrete Poisson equation on [0, 1] with zero boundary conditions on a uniform grid fh,
(2.1)
-
1-E
-
un=f,
n =1,...,N-I,
2 L, with integer L > 1, and E is the shift operator on 2h. We denote where N h the above system by
Au =f,
where A, u, and f correspond, respectively, to the discrete Laplacian, solution, and forcing functions. Clearly, A is a tridiagonal matrix with diagonal elements -1/2, and It is well known that the matrix A can be diagonalized as
-.
where
AA is a diagonal matrix diag (X,
w=
,
Xu- ),
cos (kh),
)2 ohogonal matrix whose kth row is
and W is an order (N-
(2.3)
WTAA W,
A
(2.2)
sin (k(N- 1)h)).
sin (knh),
(sin (kh),
The diagonalization of the matrix A can be intereted as the decomposition of the driving and solution functions into their Fourier components, i.e.,
a
sin (knh),
u sin (knh), n=l
n=l
k= 1,2,
One can easily verify that
and
,N-1.
are related via
A(I=,
= ,,
,-
,
where
(2.4)
(k)= Xk
cos (krh),
is known as the spectrum of the discrete Laplacian. In order to invert A, we can make use of (2.2) and obtain a fast Poisson solver:
(2.5)
A-l
WrAS W.
The above procedure also serves as the general framework for fast Poisson solvers in cases of higher dimension. However, fast Poisson solvers are not generally applicable for nonseparable elliptic operators and irregular domains. Instead, we want to find good approximations to this solution procedure that are extensible to more general problems and then use them as preconditioners. The fundamental idea is to avoid the use of fast Fourier transform (FFT) and to use instead a sequence of filtering operations to approximate the desired spectral decomposition. This explains the motivation and the name of the multilevel filtering (MF) preconditioner proposed in this paper. Our main idea for deriving the MF preconditioner for A is to divide all admissible wavenumbers into bands and to approximate the spectrum 4 (k) at each band with some
406
C.-C. J. KUO, T. F. CHAN, AND C. TONG
Downloaded 01/26/14 to 132.174.255.3. Redistribution subject to SIAM license or copyright; see http://www.siam.org/journals/ojsa.php
constant. To be more precise, consider the following piecewise constant function in the wavenumber domain
P(k)=Cl,
0.5, z> 0.5, x- 0.5, z_-< 0.5 or y=< 0.5, z> 0.5, x>0.5 with
elsewhere.
10
426
C.-C. J. KUO, T. F. CHAN, AND C. TONG
Downloaded 01/26/14 to 132.174.255.3. Redistribution subject to SIAM license or copyright; see http://www.siam.org/journals/ojsa.php
100
90 80 70
60 504030 20-
10-
10
10 n= 1/h (a)
350O
2500200015001000-
.......
..
MGM’F2
RIC
500 0 10o
I0
I0 2
n=l/h
(b)
FIG. 5.5. (a) Iteration and b operation counts for Test Problem 5.
The number of iterations and operation counts per grid point are plotted in Figs. 5.15.6 (a) and (b), respectively. We can make the following observations from these figures. The BPX and MGMF preconditioners have better convergence behavior than the HB preconditioner, especially for three-dimensional problems. The HB method is competitive with the other multilevel methods only for the discontinuous coefficient problem in two dimensions.
427
Downloaded 01/26/14 to 132.174.255.3. Redistribution subject to SIAM license or copyright; see http://www.siam.org/journals/ojsa.php
MULTILEVEL FILTERING PRECONDITIONERS
600500400-
300-
MGMI?
200/
.-’""
MGMF3
100’"i’","i"’i""
0
10 2
I0
100
n=l/h
xl04 2.5
1.5
MGMF MGMF1 MGMF3
RIC 0
1( o
101
10
n=l/h
(b)
FIG. 5.6. (a) Iteration and (b) operation counts for Test Problem 6.
2 The O(log" n convergence rate for all the multilevel methods is evident, except for the three-dimensional HB method. The three-dimensional HB method behaves like O(h -’59) and O(h -’7) for problems (5.4) and (5.5), which are close to the predicted theoretical result O(h-5). However, for the discontinuous coefficient problem 5.6 ), it converges more slowly, like O(h-126).
428
C.-C. J. KUO, T. F. CHAN, AND C. TONG
Downloaded 01/26/14 to 132.174.255.3. Redistribution subject to SIAM license or copyright; see http://www.siam.org/journals/ojsa.php
(3) In general, the MGMF methods perform slightly better than the corresponding BPX methods. Recall that the only difference between the two methods is the choice of the elementary filters. (4) Filtering twice (BPX2, BPX3, and MGMF2) does improve the convergence rates for the model Poisson problem in either two or three dimensions (the MGMF2 and BPX3 preconditioners appear to be spectrally equivalent.) For variable and discontinuous coefficient problems, filtering twice does not seem to improve the convergence rates enough to compensate for the extra work involved. (5) The MGMF3 method is designed to incorporate the desired features of MGMF and MGMF2, i.e., the good convergence property due to filtering twice and the smaller amount of work due to filtering once at the finest grid level. It turns out that it works very well. MGMF3 behaves better than MGMF but worse than MGMF2 in the number of iterations required. However, in terms of amount of work, MGMF3 is better than MGMF1 and MGMF2. (6) For small n (
