MORPHOLOGICAL PYRAMIDS AND WAVELETS ... - Semantic Scholar
MORPHOLOGICAL PYRAMIDS AND WAVELETS BASED ON THE QUINCUNX LATTICE HENK J.A.M. HEIJMANS
CWI, P.O. Box 94079, 1090 GB Amsterdam, The Netherlands
and JOHN GOUTSIAS
Center for Imaging Science, Dept. of Electrical and Comp. Eng. The Johns Hopkins University, Baltimore, MD 21218, USA
Abstract. This paper is concerned with two types of multiresolution image decompositions, pyramids and wavelets. We present an axiomatic approach for both cases, encompassing linear as well as nonlinear decompositions. A wavelet decomposition is more speci c in the sense that it always involves a pyramid transform. Both families will be illustrated by means of concrete examples using the quincunx scheme in two dimensions. One nonlinear wavelet transform will be discussed in more detail: it uses the lifting scheme and has the intriguing property that it preserves local maxima over a range of scales. Key words: multiresolution decomposition, pyramid, wavelet, lifting scheme, quincunx grid.
1. Introduction It is widely accepted that multiresolution approaches are extremely useful in various image processing applications. This is due to the fact that most images contain physically relevant features at di erent scales. For their proper understanding, multiresolution (or multiscale) techniques are indispensable. Another good reasons to take recourse to multiresolution approaches is that the corresponding algorithms o er various computational advantages. In this paper we present a brief overview of the axiomatic framework for the pyramid and the wavelet transform, which has been discussed in great detail in [5, 8], and we present worked-out examples for both cases based on the two-dimensional quincunx sampling scheme. An important feature of our framework is that it allows linear as well as nonlinear transforms. This is important to us, since our interest primarily goes to decompositions which are based on morphological operators. In Section 2 we introduce the pyramid transform and in Section 3 we discuss a particular example based on a morphological adjunction. The general wavelet transform is introduced in Section 4. There we also explain how the lifting scheme can be used to design (linear and nonlinear) wavelet transforms. An interesting example, called max-lifting, based on morphological operators is discussed in Section 5. As we will show, the major characteristic of this scheme is that it preserves local maxima. This work was supported in part by NATO Collaborative Research Grant CRG.971503. John Goutsias was also supported by the Oce of Naval Research (U.S.A.), Mathematical, Computer, and Information Sciences Division, under ONR Grant N00014-90-1345.
2
HENK J.A.M. HEIJMANS AND JOHN GOUTSIAS
2. The pyramid transform In this section we will formalize the concept of a pyramid, as rst introduced by Burt and Adelson [1], using general analysis and synthesis operators. For a comprehensive discussion we refer to [5]. Thus, consider a family of image spaces Vj ; j 0. Assume that we can go from level j to the next level j + 1 by means of an analysis operator " j : Vj ! Vj+1 . To go back from level j + 1 to the lower level j we need to dispose of a synthesis operator j# : Vj+1 ! Vj . In this scheme, every analysis operator " j is designed to reduce the information contained in images at level j , and as such, they are not invertible in general: the composition j# j" (x) does only yield an approximation of the input image x 2 Vj . On the other hand, we demand that the synthesis operator does not cause a further reduction of the information content of an image. To achieve this, we will make the following assumption to which we refer as the pyramid condition: for every j 0, the operators j" ; j# satisfy " # (1) j j (x) = x for x 2 Vj+1 : This condition yields that the composition j# j" is idempotent. The pyramid scheme
introduced above can be used to obtain an alternative representation of an image x, _ x2 ,_ provided that we have an addition +_ and a subtraction ,_ on Vj such that x1 +( x1 ) = x2 for x1 ; x2 2 Vj . Given an input signal x0 2 V0 , we consider the following recursive signal analysis scheme:
x0 ! fy0 ; x1 g ! fy0 ; y1; x2 g ! where xj+1 = j" (xj ); yj = xj ,_ j# (xj+1 ) : We refer to this scheme as the pyramid transform. The original signal x0 2 V0 can be exactly reconstructed from xj+1 and y0 ; y1 ; : : : ; yj by means of the backward recursion
xj = j# (xj+1 ) +_ yj ; j 0;
the inverse pyramid transform. For grey-level images, one can choose the standard addition and subtraction for +_ and ,_ , respectively. In the case of nitely many grey-levels, say n, one can also use the cyclic addition (i.e. addition modulus n). In the binary case, this corresponds with the `exclusive or'.
3. Morphological pyramids Here we investigate pyramid decompositions based on two basic morphological operators, erosion and dilation. Recall that for two complete lattices L and M, an operator pair ("; ), with " : L ! M and : M ! L, is called an adjunction if (y) x if and only if y "(x) for x 2 L and y 2 M. In that case " is an erosion and a dilation; see [7]. We consider pyramids that satisfy the following constraints: (i) the image domains Vj are complete lattices; (ii) the analysis/synthesis pair ( j" ; j# ) forms an adjunction. Assume that there exist two sets S; Q and a binary relation on S Q denoted by s ! q. Given a complete lattice T (later, T will have the interpretation of grey-level
MORPHOLOGICAL PYRAMIDS AND WAVELETS
3
set), we can de ne an adjunction ( " ; # ) between T S and T Q in the following way: " (x)(q) =
^
s:s!q
V
_ 0 x(s) and # (x0 )(s) = x (q) q:s!q
(2)
for x 2 T S and x0 2 T Q . Here s:s!q x(s) stands for minfx(s) j s 2 S and s ! qg (or `inf' if the set is in nite). Indeed, the pair ( " ; # ) constitutes an adjunction. In general it will not satisfy the pyramid condition, however. Thereto we need an additional assumption [5]. Proposition 1 The pair ( "; # ) given by (2) satis es the pyramid condition (i.e. " # = id on T Q ) if and only if for all q 2 Q there exists an s 2 S such that (i) s ! q and (ii) s ! q0 where q0 2 Q, implies q0 = q. In the remainder of this section we are exclusively concerned with morphological adjunction pyramids which correspond with a 2D quincunx sampling approach. Let S denote the square lattice comprising the integer points, i.e., S = f(s1 ; s2 ) j s1 ; s2 2 Zg. Furthermore, let Q be the subset of S resulting from a quincunx sampling scheme, i.e., Q = f(q1 ; q2 ) j q1 ; q2 2 Z and q1 + q2 even g. Finally, let S 0 Q be the set resulting after a second quincunx sampling step, i.e., S 0 = f(s01 ; s02 ) j s01 ; s02 2 2Zg. We de ne the following two norms on S :
Fig. 1. Arrows express the relations s !0 q between s 2 S and q 2 Q (left), and q !1 s0 between q 2 Q and s0 2 S 0 (right). The larger disks comprise the quincunx grid.
ksk1 = js1 j + js2 j and ksk1 = maxfjs1 j; js2 jg; where s = (s1 ; s2 ) 2 S . Consider the binary relations s !0 q i ks , qk1 1 and q !1 s0 i kq , s0 k1 1 on S Q and Q S 0, respectively. These relations are illustrated in Fig. 1. Observe that both relations satisfy the conditions (i) , (ii) of Proposition 1. Putting V0 = T S and V1 = T Q , the analysis and synthesis operator given by (2), i.e., ^ _ 0 " x(s) and 0# (x0 )(s) = x (q); 0 (x)(q ) = s:s!0 q
q:s!0 q
4
HENK J.A.M. HEIJMANS AND JOHN GOUTSIAS
form an adjunction between V0 and V1 and satisfy the pyramid condition. Similarly, putting V2 = T S and de ning 0
"
0
1 (x)(s ) =
^
q:q!1 s
0
_ 0 0 x(q) and 1# (x0 )(q) = x (s ); s :q!1 s 0
0
we obtain an analysis/synthesis pair ( 1" ; 1# ) which is an adjunction between V1 and V2 and satis es the pyramid condition. Now, since S 0 = 2S , we can repeat the same procedure by putting Q0 = 2Q and S 00 = 2S 0. In Fig. 2 we compute 2 levels of the corresponding pyramid transform. The odd levels in the pyramid are shown after a 45 clockwise rotation. We make two
Fig. 2. Morphological adjunction pyramid for the quincunx scheme. From left to right: image xj (with j = 0 at the bottom level), approximation x^j = j# j" (xj ), and detail yj = xj , x^j .
important observations regarding this example: (1) since every analysis/synthesis pair constitutes a morphological adjunction, the approximation operator j# j" is a morphological opening [7]. This means in particular that the approximation image x^j = j# j" (xj ) is never larger than the original image xj , and therefore the detail image yj = xj , x^j is nonnegative. (2) In the expressions for j" and j# we only need to consider those points that lie in the domain of the image, i.e., a square window. This does not a ect the validity of the pyramid condition, but it does destroy translation invariance.
5
MORPHOLOGICAL PYRAMIDS AND WAVELETS
4. Wavelets and lifting A serious drawback of the pyramid transform is that its output signal comprises more data than the input signal. The wavelet transform (see [9] for a comprehensive account in the linear case), to be discussed below, does not have this drawback. In this case the detail signal is generated by a second analysis operator !j" , mapping Vj into another space Wj+1 , in such a way that x0 = j" (x) and y0 = !j" (x) jointly contain the same amount of data as x. Furthermore, there exists a synthesis operator #j such that the perfect reconstruction condition #j ( j" (x); !j" (x)) = x; x 2 Vj
(3)
holds, as well as
" # " # j ( j (x; y)) = x; and !j ( j (x; y)) = y; for x 2 Vj+1 ; y 2 Wj+1 : Refer to Fig. 3 for an illustration. Often, e.g. in the linear case, #j is of the form #j (x0 ; y0) = j# (x0 ) +_ !j# (y0 ); (4)
and in this case we say that the wavelet transform is uncoupled [8]. V j+1
Wj+1
Synthesis Analysis
ψjA
Ψ Bj
ωjA
Analysis
Vj
Fig. 3. The general wavelet transform.
The lifting scheme introduced by Sweldens [10], provides a simple, exible, and ecient tool for the construction of linear as well as nonlinear wavelet transforms. A general lifting scheme starts with an invertible transformation of the input data x0 into two or more channels (or bands). We restrict ourselves to the two-channel case for simplicity. Thus application of to x0 yields two output signals, a coarse signal x1 and a detail signal y1 . Application of ,1 to x1 ; y1 returns the input signal: x0 = ,1 (x1 ; y1 ). In practice, will often be a known wavelet transform, for example the lazy wavelet which splits the input data into even and odd samples. By a concatenation of so-called prediction and update lifting steps (see Fig. 4) one arrives at a lifted wavelet. In Fig. 4 the prediction operators are denoted by i and the update operators by i . If the lifting scheme consists of one prediction step followed by one update step , and x01 ; y10 are computed by the following scheme, (x1 ; y1 ) = (x0 ); y10 = y1 , (x1 ); x01 = x1 + (y10 );
6
HENK J.A.M. HEIJMANS AND JOHN GOUTSIAS x1 +
x0
Σ y1
−
+
x1' +
λ1
π1 +
+
+
πi +
y1'
−
+
+
λi
+
Fig. 4. Lifting scheme.
then we arrive at a lifted transform 0 (x0 ) = (x01 ; y10 ). This can be inverted by
x1 = x01 , (y10 ); y1 = y10 + (x1 ); x0 = ,1 (x1 ; y1 ):
Later we use this scheme to obtain a two-dimensional wavelet transform for the case that splits an input image according to the quincunx sampling lattice. In [8] we have shown that a linear transformation followed by one nonlinear prediction or update step yields an uncoupled wavelet transform. However, two nonlinear lifting steps result in a coupled wavelet transform, in general.
5. Max-lifting for the quincunx lattice In this section we present a particular example of a nonlinear wavelet scheme associated with the two-dimensional quincunx sampling lattice. Consider the partition of the square lattice S into two disjoint subsets, Q and R = S n Q. We de ne an adjacency relation expressing when s is a neighbour of s0 , i.e., s s0 if ks , s0k1 = 1. Thus is a symmetric relation on S S , and s s0 can only hold if either s or s0 (but not both) is an element of Q. Observe that r !0 q if and only if r q for r 2 R and q 2 Q. Now let the operator govern the splitting of a signal x0 on S into two subsampled signals, x1 de ned on Q and y1 de ned on R, i.e., x1 (q) = x0 (q) for q 2 Q and y1 (r) = x0 (r) for r 2 R. Consider the coupled wavelet transform obtained by applying rst a prediction lifting
(x)(r) =
_
q:qr
and then an update lifting
(y)(q) = maxf0;
x(q)
_ r:rq
y ( r )g
(5)
(6)
This means that the prediction of the signal at a point r is given by the maximum of its 4 neighbours. The update operator is chosen so that local maxima of the input signal x0 are mapped to the next level x01 . The next result, a proof of which can be found in [8], gives a formal statement. But rst we need two more de nitions. We
MORPHOLOGICAL PYRAMIDS AND WAVELETS
7
Fig. 5. Wavelet decomposition of an image based on the max-lifting scheme. Bottom row: original image x0 . Middle row: wavelet transformed images x01 and y10 (after rotation). Top row: images x001 and y100 resulting from wavelet transform of x01 . Note that the detail image may contain positive (bright) and negative (dark) greyvalues.
write s s0 if s = s0 or ks , s0 k1 = 2. Given a signal x on S and a point s 2 S we denote by A(x j s) the set of neighbours s0 of s such that x(s0 ) x(s00 ) for all neighbours s00 of s. Observe that A(x j s) contains at least one point. Proposition 2 Let x0 be an input signal on S , let (x1 ; y1) be its splitting into subsignals on Q and R, respectively, and y10 (r) = y1 (r) , (x1 )(r); x01 (q) = x1 (q) + (y10 )(q); where and are given by (5)-(6). Then the following holds: (a) x0 (q) x01 (q) maxfx0 (s) j s = q or s qg, for q 2 Q. (b) x0 (r) maxfx01 (q) j q rg, for r 2 R. (c) Assume that r 2 R is such that x0 (r) x0 (s) for s r and s r, then x01 (q) = x0 (r) for every q 2 A(x0 j r). Thus the max-lifting scheme preserves local maxima. But we can also show that this scheme will never create new maxima.
8
HENK J.A.M. HEIJMANS AND JOHN GOUTSIAS
Proposition 3 Suppose that the coarse signal x01 has a local maximum at q 2 Q in the following sense: x01 (q0 ) x01 (q) for q0 2 Q with kq , q0 k1 2 (with every point q there correspond nine such points, including q itself). Then x0 has a local maximum at some s 2 S with s = q or s q and x0 (s) = x01 (q). In Fig. 5 we apply the max-lifting scheme to a particular image.
6. Summary and Conclusions The pyramid condition given in (1), though seemingly straightforward, imposes relatively strong conditions on the analysis and synthesis operators. In the linear case, this condition is necessary and sucient in order that the pyramid transform can be extended to a wavelet transform. For the nonlinear case, this problem is open. Construction of nonlinear wavelets with prescribed properties (in the spirit of vanishing moments conditions in the linear case) is a research area which is almost entirely unexplored; some early work in this direction can be found in [2, 3, 4, 6]. The main tool, and to the best of our knowledge the only tool so far, to address this problem is the lifting scheme. A great deal of future research on nonlinear wavelets will have to face the question how to build schemes that yield decompositions which are useful in applications such as compression, denoising, image fusion, or image retrieval. We believe that the max-lifting scheme will turn out useful for some of these applications. We are currently exploring this issue.
References 1. P. J. Burt and E. H. Adelson, The Laplacian pyramid as a compact image code. IEEE Transactions on Communications 31 (1983), 532{540. 2. R. Claypoole, R. G. Baraniuk, and R. D. Nowak, Lifting construction of non-linear wavelet transforms. In IEEE International Symposium on Time-Frequency and Time-Scale Analysis (Pittsburgh, 1998), pp. 49{52. 3. R. L. de Queiroz, D. A. Florencio and R. W. Schafer, Nonexpansive pyramid for image coding using a nonlinear lterbank. IEEE Transactions on Image Processing 7, 2 (1998), 246{252. 4. O. Egger and W. Li, Very low bit rate image coding using morphological operators and adaptive decompositions. In Proceedings of the IEEE International Conference on Image Processing, volume II (Austin, Texas, 1994), IEEE, pp. 326{330. 5. J. Goutsias and H. J. A. M. Heijmans, Multiresolution signal decomposition schemes. Part 1: Morphological pyramids. To appear in IEEE Transactions on Image Processing. 6. F. J. Hampson and J.-C. Pesquet, M-band nonlinear subband decompositions with perfect reconstruction. IEEE Transactions on Image Processing 7, 11 (1998), 1547{1560. 7. H. J. A. M. Heijmans, Morphological Image Operators. Academic Press, Boston, 1994. 8. H. J. A. M. Heijmans and J. Goutsias, Multiresolution signal decomposition schemes. Part 2: Morphological wavelets. Research Report PNA-R9905, CWI, 1999. 9. S. Mallat, A wavelet tour of signal processing. Academic Press, San Diego, 1998. 10. W. Sweldens, The lifting scheme: A custom-design construction of biorthogonal wavelets. Appl. Comput. Harmon. Anal. 3, 2 (1996), 186{200.
CWI, P.O. Box 94079, 1090 GB Amsterdam, The Netherlands
and JOHN GOUTSIAS
Center for Imaging Science, Dept. of Electrical and Comp. Eng. The Johns Hopkins University, Baltimore, MD 21218, USA
Abstract. This paper is concerned with two types of multiresolution image decompositions, pyramids and wavelets. We present an axiomatic approach for both cases, encompassing linear as well as nonlinear decompositions. A wavelet decomposition is more speci c in the sense that it always involves a pyramid transform. Both families will be illustrated by means of concrete examples using the quincunx scheme in two dimensions. One nonlinear wavelet transform will be discussed in more detail: it uses the lifting scheme and has the intriguing property that it preserves local maxima over a range of scales. Key words: multiresolution decomposition, pyramid, wavelet, lifting scheme, quincunx grid.
1. Introduction It is widely accepted that multiresolution approaches are extremely useful in various image processing applications. This is due to the fact that most images contain physically relevant features at di erent scales. For their proper understanding, multiresolution (or multiscale) techniques are indispensable. Another good reasons to take recourse to multiresolution approaches is that the corresponding algorithms o er various computational advantages. In this paper we present a brief overview of the axiomatic framework for the pyramid and the wavelet transform, which has been discussed in great detail in [5, 8], and we present worked-out examples for both cases based on the two-dimensional quincunx sampling scheme. An important feature of our framework is that it allows linear as well as nonlinear transforms. This is important to us, since our interest primarily goes to decompositions which are based on morphological operators. In Section 2 we introduce the pyramid transform and in Section 3 we discuss a particular example based on a morphological adjunction. The general wavelet transform is introduced in Section 4. There we also explain how the lifting scheme can be used to design (linear and nonlinear) wavelet transforms. An interesting example, called max-lifting, based on morphological operators is discussed in Section 5. As we will show, the major characteristic of this scheme is that it preserves local maxima. This work was supported in part by NATO Collaborative Research Grant CRG.971503. John Goutsias was also supported by the Oce of Naval Research (U.S.A.), Mathematical, Computer, and Information Sciences Division, under ONR Grant N00014-90-1345.
2
HENK J.A.M. HEIJMANS AND JOHN GOUTSIAS
2. The pyramid transform In this section we will formalize the concept of a pyramid, as rst introduced by Burt and Adelson [1], using general analysis and synthesis operators. For a comprehensive discussion we refer to [5]. Thus, consider a family of image spaces Vj ; j 0. Assume that we can go from level j to the next level j + 1 by means of an analysis operator " j : Vj ! Vj+1 . To go back from level j + 1 to the lower level j we need to dispose of a synthesis operator j# : Vj+1 ! Vj . In this scheme, every analysis operator " j is designed to reduce the information contained in images at level j , and as such, they are not invertible in general: the composition j# j" (x) does only yield an approximation of the input image x 2 Vj . On the other hand, we demand that the synthesis operator does not cause a further reduction of the information content of an image. To achieve this, we will make the following assumption to which we refer as the pyramid condition: for every j 0, the operators j" ; j# satisfy " # (1) j j (x) = x for x 2 Vj+1 : This condition yields that the composition j# j" is idempotent. The pyramid scheme
introduced above can be used to obtain an alternative representation of an image x, _ x2 ,_ provided that we have an addition +_ and a subtraction ,_ on Vj such that x1 +( x1 ) = x2 for x1 ; x2 2 Vj . Given an input signal x0 2 V0 , we consider the following recursive signal analysis scheme:
x0 ! fy0 ; x1 g ! fy0 ; y1; x2 g ! where xj+1 = j" (xj ); yj = xj ,_ j# (xj+1 ) : We refer to this scheme as the pyramid transform. The original signal x0 2 V0 can be exactly reconstructed from xj+1 and y0 ; y1 ; : : : ; yj by means of the backward recursion
xj = j# (xj+1 ) +_ yj ; j 0;
the inverse pyramid transform. For grey-level images, one can choose the standard addition and subtraction for +_ and ,_ , respectively. In the case of nitely many grey-levels, say n, one can also use the cyclic addition (i.e. addition modulus n). In the binary case, this corresponds with the `exclusive or'.
3. Morphological pyramids Here we investigate pyramid decompositions based on two basic morphological operators, erosion and dilation. Recall that for two complete lattices L and M, an operator pair ("; ), with " : L ! M and : M ! L, is called an adjunction if (y) x if and only if y "(x) for x 2 L and y 2 M. In that case " is an erosion and a dilation; see [7]. We consider pyramids that satisfy the following constraints: (i) the image domains Vj are complete lattices; (ii) the analysis/synthesis pair ( j" ; j# ) forms an adjunction. Assume that there exist two sets S; Q and a binary relation on S Q denoted by s ! q. Given a complete lattice T (later, T will have the interpretation of grey-level
MORPHOLOGICAL PYRAMIDS AND WAVELETS
3
set), we can de ne an adjunction ( " ; # ) between T S and T Q in the following way: " (x)(q) =
^
s:s!q
V
_ 0 x(s) and # (x0 )(s) = x (q) q:s!q
(2)
for x 2 T S and x0 2 T Q . Here s:s!q x(s) stands for minfx(s) j s 2 S and s ! qg (or `inf' if the set is in nite). Indeed, the pair ( " ; # ) constitutes an adjunction. In general it will not satisfy the pyramid condition, however. Thereto we need an additional assumption [5]. Proposition 1 The pair ( "; # ) given by (2) satis es the pyramid condition (i.e. " # = id on T Q ) if and only if for all q 2 Q there exists an s 2 S such that (i) s ! q and (ii) s ! q0 where q0 2 Q, implies q0 = q. In the remainder of this section we are exclusively concerned with morphological adjunction pyramids which correspond with a 2D quincunx sampling approach. Let S denote the square lattice comprising the integer points, i.e., S = f(s1 ; s2 ) j s1 ; s2 2 Zg. Furthermore, let Q be the subset of S resulting from a quincunx sampling scheme, i.e., Q = f(q1 ; q2 ) j q1 ; q2 2 Z and q1 + q2 even g. Finally, let S 0 Q be the set resulting after a second quincunx sampling step, i.e., S 0 = f(s01 ; s02 ) j s01 ; s02 2 2Zg. We de ne the following two norms on S :
Fig. 1. Arrows express the relations s !0 q between s 2 S and q 2 Q (left), and q !1 s0 between q 2 Q and s0 2 S 0 (right). The larger disks comprise the quincunx grid.
ksk1 = js1 j + js2 j and ksk1 = maxfjs1 j; js2 jg; where s = (s1 ; s2 ) 2 S . Consider the binary relations s !0 q i ks , qk1 1 and q !1 s0 i kq , s0 k1 1 on S Q and Q S 0, respectively. These relations are illustrated in Fig. 1. Observe that both relations satisfy the conditions (i) , (ii) of Proposition 1. Putting V0 = T S and V1 = T Q , the analysis and synthesis operator given by (2), i.e., ^ _ 0 " x(s) and 0# (x0 )(s) = x (q); 0 (x)(q ) = s:s!0 q
q:s!0 q
4
HENK J.A.M. HEIJMANS AND JOHN GOUTSIAS
form an adjunction between V0 and V1 and satisfy the pyramid condition. Similarly, putting V2 = T S and de ning 0
"
0
1 (x)(s ) =
^
q:q!1 s
0
_ 0 0 x(q) and 1# (x0 )(q) = x (s ); s :q!1 s 0
0
we obtain an analysis/synthesis pair ( 1" ; 1# ) which is an adjunction between V1 and V2 and satis es the pyramid condition. Now, since S 0 = 2S , we can repeat the same procedure by putting Q0 = 2Q and S 00 = 2S 0. In Fig. 2 we compute 2 levels of the corresponding pyramid transform. The odd levels in the pyramid are shown after a 45 clockwise rotation. We make two
Fig. 2. Morphological adjunction pyramid for the quincunx scheme. From left to right: image xj (with j = 0 at the bottom level), approximation x^j = j# j" (xj ), and detail yj = xj , x^j .
important observations regarding this example: (1) since every analysis/synthesis pair constitutes a morphological adjunction, the approximation operator j# j" is a morphological opening [7]. This means in particular that the approximation image x^j = j# j" (xj ) is never larger than the original image xj , and therefore the detail image yj = xj , x^j is nonnegative. (2) In the expressions for j" and j# we only need to consider those points that lie in the domain of the image, i.e., a square window. This does not a ect the validity of the pyramid condition, but it does destroy translation invariance.
5
MORPHOLOGICAL PYRAMIDS AND WAVELETS
4. Wavelets and lifting A serious drawback of the pyramid transform is that its output signal comprises more data than the input signal. The wavelet transform (see [9] for a comprehensive account in the linear case), to be discussed below, does not have this drawback. In this case the detail signal is generated by a second analysis operator !j" , mapping Vj into another space Wj+1 , in such a way that x0 = j" (x) and y0 = !j" (x) jointly contain the same amount of data as x. Furthermore, there exists a synthesis operator #j such that the perfect reconstruction condition #j ( j" (x); !j" (x)) = x; x 2 Vj
(3)
holds, as well as
" # " # j ( j (x; y)) = x; and !j ( j (x; y)) = y; for x 2 Vj+1 ; y 2 Wj+1 : Refer to Fig. 3 for an illustration. Often, e.g. in the linear case, #j is of the form #j (x0 ; y0) = j# (x0 ) +_ !j# (y0 ); (4)
and in this case we say that the wavelet transform is uncoupled [8]. V j+1
Wj+1
Synthesis Analysis
ψjA
Ψ Bj
ωjA
Analysis
Vj
Fig. 3. The general wavelet transform.
The lifting scheme introduced by Sweldens [10], provides a simple, exible, and ecient tool for the construction of linear as well as nonlinear wavelet transforms. A general lifting scheme starts with an invertible transformation of the input data x0 into two or more channels (or bands). We restrict ourselves to the two-channel case for simplicity. Thus application of to x0 yields two output signals, a coarse signal x1 and a detail signal y1 . Application of ,1 to x1 ; y1 returns the input signal: x0 = ,1 (x1 ; y1 ). In practice, will often be a known wavelet transform, for example the lazy wavelet which splits the input data into even and odd samples. By a concatenation of so-called prediction and update lifting steps (see Fig. 4) one arrives at a lifted wavelet. In Fig. 4 the prediction operators are denoted by i and the update operators by i . If the lifting scheme consists of one prediction step followed by one update step , and x01 ; y10 are computed by the following scheme, (x1 ; y1 ) = (x0 ); y10 = y1 , (x1 ); x01 = x1 + (y10 );
6
HENK J.A.M. HEIJMANS AND JOHN GOUTSIAS x1 +
x0
Σ y1
−
+
x1' +
λ1
π1 +
+
+
πi +
y1'
−
+
+
λi
+
Fig. 4. Lifting scheme.
then we arrive at a lifted transform 0 (x0 ) = (x01 ; y10 ). This can be inverted by
x1 = x01 , (y10 ); y1 = y10 + (x1 ); x0 = ,1 (x1 ; y1 ):
Later we use this scheme to obtain a two-dimensional wavelet transform for the case that splits an input image according to the quincunx sampling lattice. In [8] we have shown that a linear transformation followed by one nonlinear prediction or update step yields an uncoupled wavelet transform. However, two nonlinear lifting steps result in a coupled wavelet transform, in general.
5. Max-lifting for the quincunx lattice In this section we present a particular example of a nonlinear wavelet scheme associated with the two-dimensional quincunx sampling lattice. Consider the partition of the square lattice S into two disjoint subsets, Q and R = S n Q. We de ne an adjacency relation expressing when s is a neighbour of s0 , i.e., s s0 if ks , s0k1 = 1. Thus is a symmetric relation on S S , and s s0 can only hold if either s or s0 (but not both) is an element of Q. Observe that r !0 q if and only if r q for r 2 R and q 2 Q. Now let the operator govern the splitting of a signal x0 on S into two subsampled signals, x1 de ned on Q and y1 de ned on R, i.e., x1 (q) = x0 (q) for q 2 Q and y1 (r) = x0 (r) for r 2 R. Consider the coupled wavelet transform obtained by applying rst a prediction lifting
(x)(r) =
_
q:qr
and then an update lifting
(y)(q) = maxf0;
x(q)
_ r:rq
y ( r )g
(5)
(6)
This means that the prediction of the signal at a point r is given by the maximum of its 4 neighbours. The update operator is chosen so that local maxima of the input signal x0 are mapped to the next level x01 . The next result, a proof of which can be found in [8], gives a formal statement. But rst we need two more de nitions. We
MORPHOLOGICAL PYRAMIDS AND WAVELETS
7
Fig. 5. Wavelet decomposition of an image based on the max-lifting scheme. Bottom row: original image x0 . Middle row: wavelet transformed images x01 and y10 (after rotation). Top row: images x001 and y100 resulting from wavelet transform of x01 . Note that the detail image may contain positive (bright) and negative (dark) greyvalues.
write s s0 if s = s0 or ks , s0 k1 = 2. Given a signal x on S and a point s 2 S we denote by A(x j s) the set of neighbours s0 of s such that x(s0 ) x(s00 ) for all neighbours s00 of s. Observe that A(x j s) contains at least one point. Proposition 2 Let x0 be an input signal on S , let (x1 ; y1) be its splitting into subsignals on Q and R, respectively, and y10 (r) = y1 (r) , (x1 )(r); x01 (q) = x1 (q) + (y10 )(q); where and are given by (5)-(6). Then the following holds: (a) x0 (q) x01 (q) maxfx0 (s) j s = q or s qg, for q 2 Q. (b) x0 (r) maxfx01 (q) j q rg, for r 2 R. (c) Assume that r 2 R is such that x0 (r) x0 (s) for s r and s r, then x01 (q) = x0 (r) for every q 2 A(x0 j r). Thus the max-lifting scheme preserves local maxima. But we can also show that this scheme will never create new maxima.
8
HENK J.A.M. HEIJMANS AND JOHN GOUTSIAS
Proposition 3 Suppose that the coarse signal x01 has a local maximum at q 2 Q in the following sense: x01 (q0 ) x01 (q) for q0 2 Q with kq , q0 k1 2 (with every point q there correspond nine such points, including q itself). Then x0 has a local maximum at some s 2 S with s = q or s q and x0 (s) = x01 (q). In Fig. 5 we apply the max-lifting scheme to a particular image.
6. Summary and Conclusions The pyramid condition given in (1), though seemingly straightforward, imposes relatively strong conditions on the analysis and synthesis operators. In the linear case, this condition is necessary and sucient in order that the pyramid transform can be extended to a wavelet transform. For the nonlinear case, this problem is open. Construction of nonlinear wavelets with prescribed properties (in the spirit of vanishing moments conditions in the linear case) is a research area which is almost entirely unexplored; some early work in this direction can be found in [2, 3, 4, 6]. The main tool, and to the best of our knowledge the only tool so far, to address this problem is the lifting scheme. A great deal of future research on nonlinear wavelets will have to face the question how to build schemes that yield decompositions which are useful in applications such as compression, denoising, image fusion, or image retrieval. We believe that the max-lifting scheme will turn out useful for some of these applications. We are currently exploring this issue.
References 1. P. J. Burt and E. H. Adelson, The Laplacian pyramid as a compact image code. IEEE Transactions on Communications 31 (1983), 532{540. 2. R. Claypoole, R. G. Baraniuk, and R. D. Nowak, Lifting construction of non-linear wavelet transforms. In IEEE International Symposium on Time-Frequency and Time-Scale Analysis (Pittsburgh, 1998), pp. 49{52. 3. R. L. de Queiroz, D. A. Florencio and R. W. Schafer, Nonexpansive pyramid for image coding using a nonlinear lterbank. IEEE Transactions on Image Processing 7, 2 (1998), 246{252. 4. O. Egger and W. Li, Very low bit rate image coding using morphological operators and adaptive decompositions. In Proceedings of the IEEE International Conference on Image Processing, volume II (Austin, Texas, 1994), IEEE, pp. 326{330. 5. J. Goutsias and H. J. A. M. Heijmans, Multiresolution signal decomposition schemes. Part 1: Morphological pyramids. To appear in IEEE Transactions on Image Processing. 6. F. J. Hampson and J.-C. Pesquet, M-band nonlinear subband decompositions with perfect reconstruction. IEEE Transactions on Image Processing 7, 11 (1998), 1547{1560. 7. H. J. A. M. Heijmans, Morphological Image Operators. Academic Press, Boston, 1994. 8. H. J. A. M. Heijmans and J. Goutsias, Multiresolution signal decomposition schemes. Part 2: Morphological wavelets. Research Report PNA-R9905, CWI, 1999. 9. S. Mallat, A wavelet tour of signal processing. Academic Press, San Diego, 1998. 10. W. Sweldens, The lifting scheme: A custom-design construction of biorthogonal wavelets. Appl. Comput. Harmon. Anal. 3, 2 (1996), 186{200.
