Automated Image Registration Based on ... - Semantic Scholar
3908
IEEE TRANSACTIONS ON GEOSCIENCE AND REMOTE SENSING, VOL. 46, NO. 11, NOVEMBER 2008
Automated Image Registration Based on Pseudoinvariant Metrics of Dynamic Land-Surface Features Chintan A. Shah, Yongwei Sheng, and Laurence C. Smith
Abstract—Accurate assessment of land-cover/land-use change is essential for understanding the impacts of global change and necessitates the use of satellite data. Satellite change detection requires large volumes of multitemporal images to be precisely registered. Image registration is particularly difficult in dynamic (i.e., rapidly time varying) landscapes since the changes themselves interfere with the process of tie-point identification. Despite the existence of sophisticated registration algorithms, it is still problematic to register images acquired over such areas due to a dearth of stable features. Hence, we propose an automated image registration method using tie points derived from pseudoinvariant features (PIFs) and apply the method to register satellite images for hydrologic change detection in the Arctic, where abundant shallow lakes dominate the landscape but change significantly over time. A key to the method is the identification of “shape-stable” lakes as PIFs, which preserve their geometric shape even though the shorelines may migrate significantly. The proposed method automatically identifies PIFs based on scale-invariant shape descriptors and employs their center points for establishing the registration model. Our method thus consists of water-body feature extraction, PIF detection based on feature shape criteria, and image registration using tie points derived from the PIFs. The approach is used to register 1978 and 2000 Landsat images in Alaska, where lakes dominate the landscape and change significantly over time. The performance of the proposed approach is evaluated quantitatively, and a high subpixel registration accuracy of 0.66 pixel at Enhanced Thematic Mapper Plus resolution (i.e., 19 m) is achieved. A comparative evaluation indicates that the proposed approach outcompetes the conventional manual tie-point selection method and automated image registration techniques based on fast Fourier transform. Index Terms—Centroid, invariant moments, lake dynamics, land-cover/land-use change, multitemporal image analysis, precise registration, pseudoinvariant feature (PIF).
Manuscript received February 13, 2008; revised April 8, 2008. Current version published October 30, 2008. This work was supported in part by the National Aeronautics and Space Administration through the Terrestrial Hydrology Program under Contract NNX08AE51G and in part by the National Science Foundation through the Arctic System Science Program under Contract ARC-0713903. C. A. Shah and Y. Sheng are with the Department of Geography, University of California, Los Angeles, CA 90095-1524 USA (e-mail: [email protected]). L. C. Smith is with the Department of Geography, University of California, Los Angeles, CA 90095-1524 USA, and also with the Department of Earth and Space Sciences, University of California, Los Angeles, CA 90095-1567 USA. Digital Object Identifier 10.1109/TGRS.2008.2000636
I. I NTRODUCTION
E
NVIRONMENTAL monitoring of the Earth’s surface requires detecting changes using multitemporal remotely sensed images. Change detection techniques quantify surface features’ radiometric and/or spatial changes over time and involve a pixel-by-pixel comparison of multitemporal images acquired over the same ground area at different time instances. Therefore, it is essential for the multitemporal images to be spatially registered to each other [1]. When multitemporal images are not precisely registered, the residual misregistration presents a source of error, known as the “registration noise” [2], which greatly influences change detection accuracy [3]. Traditionally, image registration relies on manual selection of tie points (i.e., control points). The performance of manual registration techniques depends on the accuracy of tie-point selection from the image pair to be registered. Linear features and feature intersection points are often ideal candidates for tie points. However, when the number of image pairs to be registered is large, human intervention becomes impractical, necessitating the use of automated image registration techniques. A significant amount of research has been focused on the development of sophisticated multitemporal image registration algorithms [4]–[9]. However, it still remains challenging to obtain precise registration of multitemporal remote sensing images [5], [10], particularly in dynamic landscapes [4]. Hence, this paper particularly focuses on automated image registration for change detection in dynamic landscapes. Automated image registration techniques may be classified into two broad categories: 1) area based and 2) feature based [7]. Area-based matching techniques employ metrics [8], [11] to measure the similarity between two images in terms of their radiometric characteristics. Sum of absolute differences, normalized cross correlation, and mutual information are examples of widely used similarity metrics. However, their performance is limited by various factors, including atmospheric degradations, illumination effects, and sensor response differences in multitemporal images, necessitating the use of scene-to-scene radiometric normalization [5], [12]. In contrast, feature-based methods that use points, edges, and contours are relatively less sensitive to such effects. Hence, feature-based registration techniques establish correspondence between images without the need for scene normalization [13], [14]. The performance of current registration algorithms is limited in dynamic landscapes where land cover changes dramatically [4]. A good example of such environments is the Arctic and
0196-2892/$25.00 © 2008 IEEE
Authorized licensed use limited to: Univ of Calif Los Angeles. Downloaded on March 13, 2009 at 16:51 from IEEE Xplore. Restrictions apply.
SHAH et al.: IMAGE REGISTRATION BASED ON PSEUDOINVARIANT METRICS OF LAND-SURFACE FEATURES
3909
Fig. 1. (a) Color composite image formed by MSS and ETM+ NIR images displayed as the red and green channels, respectively. Stable lakes appear black, and stable land appears yellow. New (or expanded) lakes appear red, whereas vanished (or shrunk) lakes appear green. (b) and (c) Close visual inspection of subimages reveals a misregistration offset of ∼2–3 pixels, as indicated by the red and green edges on the opposite lake banks.
sub-Arctic, characterized by vast area, few man-made features, and numerous shallow ponds, lakes, and wetlands. They come and go in response to seasonal and annual climate cycles. Scientific interest in these environments is at an all-time high, owing to their acute sensitivity to global climate change [15]. Lakes and their shorelines are the primary features that are visible in satellite images of lake-rich Arctic environments. However, neither of these features can be assumed to be temporally stable. Lakeshores are quite dynamic, subject to changing water balance, shoreline erosion, tapping, and emergent vegetation growth, preventing their use as matching primitives. Feature-based algorithms that use lakeshores as feature primitives are therefore unsatisfactory. Hence, Sheng et al. [4] proposed the use of stable lakes with an area change of 3%
or less and employed the centroids of these stable lakes as tie points for automated image registration. However, it still may not be possible to find a sufficient number of stable lakes, particularly when environmental conditions have changed dramatically, e.g., a wet or dry year. Hence, this paper introduces the concept of pseudoinvariant features (PIFs)—lakes that have not undergone significant changes in shape over time—and, thus, uses both stable and dynamic lakes for image registration. The proposed method includes procedures of water-body extraction, PIF detection based on feature shape criteria, and image registration using tie points derived from the PIFs. The approach is then employed to register Landsat images for lake change detection in the Yukon flats region of Alaska, where lakes dominate the landscape and change significantly over time.
Authorized licensed use limited to: Univ of Calif Los Angeles. Downloaded on March 13, 2009 at 16:51 from IEEE Xplore. Restrictions apply.
3910
IEEE TRANSACTIONS ON GEOSCIENCE AND REMOTE SENSING, VOL. 46, NO. 11, NOVEMBER 2008
II. S TUDY A REA AND D ATA S ET U SED Recent Arctic warming has caused significant regional changes in lake distribution in the Arctic and sub-Arctic regions [15], [16]. Therefore, a systematic pan-Arctic-scale inventory of lake changes is essential for us to understand the overall impacts of the Arctic climate warming. This requires automated registration of thousands of Landsat Multi-Spectral Scanner (MSS), Thematic Mapper (TM), and Enhanced Thematic Mapper Plus (ETM+) images archived over the past ∼30 years. To test the proposed image registration approach, this study uses one MSS image (Path:77/Row:13) and one ETM+ image (Path:70/Row:14) in Alaska acquired on August 3, 1978, and August 16, 2000, respectively. They are both GeoCover orthorectified image products. Our study area is in the Yukon flats region of Alaska, with latitudes and longitudes ranging from 66◦ 6 N to 66◦ 21 N and 148◦ 30 W to 149◦ 24 W. Fig. 1(a) provides the color composite image by displaying the MSS and the ETM+ near-infrared (NIR) bands in red and green, respectively. This false-color image illustrates both real lake changes and misregistration errors. Stable water bodies appear black, and stable land appears yellow. New (or expanded) water bodies appear red, whereas vanished (or shrunk) water bodies appear green. The red edges on one side of the lakes and the green ones on the other side indicate a misregistration offset between the two satellite images. A close visual inspection of Fig. 1(b) and (c) reveals an offset of ∼2–3 pixels between the multitemporal images. The misregistration offset has to be removed before real lake changes may be quantified. III. M ETHOD To establish correspondence between features in multitemporal images, it is essential to have a priori knowledge about the characteristics of their spatial change. For example, lakes may shrink, grow, or stay stable over a period of time. To employ the lakes themselves as tie points in an automated featurebased registration, it is straightforward to use only the stable lakes. The centroids of such unchanged lakes can be effectively employed to estimate the registration parameters [4]. However, as described earlier, such unchanged lakes may not be sufficient in quantity and/or sufficiently distributed throughout the images in dynamic landscapes to obtain an unbiased registration model, thus requiring relaxation of the criteria for “stable” lakes. Observations of lake changes in flat terrain areas such as the broad Arctic lowlands indicate, in general, that many lakes grow or shrink radially outward or inward, with their shapes and centroids remaining relatively stable (Fig. 2). The lake boundary (solid line) at time t1 migrates to a new location (dashed line) at t2 after Δt period of time. The centroids Ct1 and Ct2 of the lake at time t1 and t2 only migrate by a small distance ΔC, and the lake shape remains relatively stable even though the size of the lake changes significantly [Fig. 2(b) and (c)]. This lake at t2 can be expressed as a scaled version of the lake at t1 . In such cases, scale-invariant descriptors such as invariant moments can be employed to describe the shape of dynamic lakes for subsequent registration tie-point development. To this end, a lake is considered to be a PIF if it is stable or its change is invariant to a shape descriptor.
Fig. 2. Illustration of typical lake changes. Their shapes and centers remain relatively stable, even though the lakes change significantly. The contour with solid line corresponds to the boundary of a lake at time t1 , whereas the dashed line corresponds to the boundary of the same lake at time t2 = t1 + Δt. Furthermore, Ct1 and Ct2 are the centroids of these lake at time t1 and t2 , respectively, and ΔC is the Euclidean distance between the two centroids. (a) Lake has undergone insignificant changes. (b) Lake has expanded radially outward. (c) Lake has shrunk radially inward.
Fig. 3 illustrates the workflow of the proposed method. Each of the multitemporal images is processed independently through the steps of feature extraction and centroid detection. Next, lake correspondence between the images is established to identify identical lakes. PIF lakes are screened from these identical lakes using invariant moments, and their centroids are extracted as tie points for subsequent image registration. These major steps are detailed in the following sections. A. Feature Extraction Feature extraction aims at segmenting the features of interest from background features in the image. Since the objective here is to employ the delineated features for subsequent image registration, the spectral characteristics of the features should be known beforehand. For example, open water exhibits a very low spectral response in the NIR wavelengths relative to the surrounding terrain. Thus, a simple threshold may be applied to an NIR image to segment water bodies from the background. Although several thresholding techniques have been proposed in the literature [17], none of these techniques have yet been proven to be optimal, their performances vary depending upon the characteristics of the images being processed, and the segmentation results are sensitive to the selected threshold value. With multispectral images, the additional spectral bands may be used to improve water-body segmentation. Hence, this paper first applies a thresholding segmentation method and then refines the initial segmentation using the maximumlikelihood classifier, a supervised classification technique using all the available spectral information relevant to water bodies. A simple filtering technique is then employed to remove isolated pixels for clear identification of the lakes.
Authorized licensed use limited to: Univ of Calif Los Angeles. Downloaded on March 13, 2009 at 16:51 from IEEE Xplore. Restrictions apply.
SHAH et al.: IMAGE REGISTRATION BASED ON PSEUDOINVARIANT METRICS OF LAND-SURFACE FEATURES
3911
Fig. 3. Major steps involved in the proposed approach (a) to be applied on each of the multitemporal data sets and (b) to be applied to identify PIF lakes and generate tie points for registration model development and assessment.
B. Centroid Detection Once the lakes are detected in each multitemporal image, they are matched between the images using their centroids. The lake centroids are calculated as
xc = m10 /m00 yc = m01 /m00
(1)
where mpq = x y xp y q f (x, y) is the (p + q)th-order moment of an image f (x, y). When applied to a binary lake image, mpq becomes [18] mpq =
x
xp y q .
(2)
y
From (2), note that m00 corresponds to the area of the lake object, whereas the centroid coordinates xc and yc are the averages of the x and the y coordinates of the pixels within the lake, respectively. The steps in Fig. 3(b) operate on the output of individually processed images as outlined in Fig. 3(a). For each centroid at t1 , the corresponding centroid with the minimum Euclidean distance is determined in the image acquired at t2 . These centroids are used to identify identical lakes in the next step. C. Lake Matching As many satellite images are georeferenced, this paper assumes that the input images have been coarsely registered. For images that have not been georeferenced, automated coarse registration techniques can be employed to roughly register them. Once the two input images are roughly aligned, lakes on the two images may be easily identified using their centroids. If the distance between the lake centroids (ΔC) in the two images is less than a certain centroid distance threshold, then the two lake objects at t1 and t2 are considered to be the same lake. A large value of ΔC indicates that either the shape of the lake has changed significantly or it has completely disappeared, making ΔC the distance of that lake at t1 to the nearest lake in t2 . Thus, a centroid threshold facilitates the elimination of dynamic lakes,
Fig. 4. (Light gray) PIF lakes identified from ETM+. (Dark gray) Remaining lake objects that failed to satisfy the PIF criteria.
which are not PIFs. Furthermore, ΔC values when averaged over all the PIFs (μΔC ) indicate the approximate amount of spatial misalignment between the images prior to registration, whereas their standard deviation σΔC indicates the offset variation. Hence, we estimate the centroid distance threshold as μΔC + 2σΔC . For the two images used in this study, the values for μΔC and σΔC are calculated to be 2.87 pixels and 0.91 pixel, respectively. Therefore, the centroid distance threshold is determined to be 4.7 pixels. D. PIF Lake Detection The PIF detection criterion tests the shape similarity of the corresponding lakes at t1 and t2 . As illustrated in Fig. 2, a lake is considered to be a PIF if the lake either has not changed significantly or has expanded or contracted radially with little geometric shape change. In each case, the lake at t2 can be expressed as a scaled version of the lake at t1 . A shape descriptor establishes a quantitative representation of the shape of an object [20]. Since the expanding or shrinking lakes vary in their scale over time, the descriptor employed for encoding their shape should be invariant to scale differences,
Authorized licensed use limited to: Univ of Calif Los Angeles. Downloaded on March 13, 2009 at 16:51 from IEEE Xplore. Restrictions apply.
3912
IEEE TRANSACTIONS ON GEOSCIENCE AND REMOTE SENSING, VOL. 46, NO. 11, NOVEMBER 2008
Fig. 5. Color composite image after registration. Offsets found in the original images have been removed by the proposed registration method.
in addition to being invariant to translation changes. Widely used shape descriptors include Fourier descriptors, chain codes, shape signatures, centroidal profiles, and invariant moments [20]. A large body of research on translational, rotational, and scale-invariant descriptors has recommended the use of invariant moments and chain codes [18], [20], [21]. In this paper, invariant moments [22] have been used for shape description as also recommended in some of the earlier literature [13], [14], [23]. The moment-based shape descriptors use moments for shape description and, thus, have been established in a statistical framework. Furthermore, compared to other descriptors such as chain codes, the moment-based descriptors have been found to be more robust to the effects of noise [20]. With the (p + q)th-order moment of a binary lake
image given in (2), the corresponding central moments are defined as [20] μpq =
x
(x − xc )p (y − yc )q
(3)
y
where xc and yc are the centroid coordinates of the object being subtracted from the object coordinates (x, y) to make the moments invariant to translation (hence, the term “central moments”). The advantage of using these moments is that their characteristics are easily interpreted with reference to the standard statistical moments [22]. For example, the first-order statistical moment corresponds to the mean, and the second-, third-, and fourth-order moments correspond to the variance,
Authorized licensed use limited to: Univ of Calif Los Angeles. Downloaded on March 13, 2009 at 16:51 from IEEE Xplore. Restrictions apply.
SHAH et al.: IMAGE REGISTRATION BASED ON PSEUDOINVARIANT METRICS OF LAND-SURFACE FEATURES
3913
skewness, and kurtosis, respectively. Similarly, the zero-order image moment represents the area of the binary object, the first-order moment is employed in estimating the corresponding centroids of the object, and the second-order moment expresses the distribution of matter around the centroid. The third-order moment expresses the symmetry of the object, and the fourthorder moment determines its flatness. To make these moments invariant to scale, the normalized central moments, denoted as ηpq , are defined as ηpq =
μpq μγ00
(4)
where γ = (p + q/2) + 1, for p + q ≥ 2. Since higher order moments are more sensitive to noise, only the first seven moments based on the second and third normalized central moments are used [22], which are given as follows: 1 (μ20 + μ02 ) μ200 1 φ2 = 4 (μ20 − μ02 )2 + 4μ211 μ00 1 φ3 = 5 (μ30 − 3μ12 )2 + (3μ21 − μ03 )2 μ00 1 φ4 = 5 (μ30 + μ12 )2 + (μ21 + μ03 )2 μ00
If the metric d of a lake is less than a defined threshold, the lake is considered to be a PIF. Fig. 4 shows the PIF lakes (in light gray) identified from the ETM+ images with a shape similarity metric threshold of 0.01. This threshold was selected to produce a desired number (about 70 PIF lakes in this paper) of welldistributed high-quality tie points for both registration model development and accuracy assessment. A lower threshold leads to the selection of fewer but more reliable PIF lakes. Once the PIFs are determined, the centroid pairs of PIF lakes are used as tie points for the registration model development and assessment to precisely register the images.
φ1 =
φ5 =
1 {(μ30 − 3μ12 )(μ30 + μ12 ) μ10 00 × (μ30 + μ12 )2 − 3(μ21 + μ03 )2
IV. E XPERIMENTAL R ESULTS
+ (3μ21 − μ03 )(μ21 + μ03 ) × 3(μ30 + μ12 )2 − (μ21 + μ03 )2 φ6 =
1 (μ20 − μ02 ) (μ30 + μ12 )2 − (μ21 + μ03 )2 7 μ00 + 4μ11 (μ30 + μ12 )(μ21 + μ03 )}
φ7 =
1 {(3μ21 − μ03 )(μ30 + μ12 ) μ10 00 × (μ30 + μ12 )2 − 3(μ21 + μ03 )2 + (3μ12 − μ30 )(μ21 + μ03 ) × 3(μ30 + μ12 )2 − (μ21 + μ03 )2 .
(5)
Thus, a set of seven moments φ = {φ1 , . . . , φ7 } invariant to translation and scale changes are subsequently employed for shape representation. Once the invariant moments are determined for a correspondent lake in the images, an invariant moment metric is computed as the lake shape similarity measure between times t1 and t2
7
d= [ϕi (t1 ) − ϕi (t2 )]2 . i=1
Fig. 6. Plot of moment distance versus area ratio of the identified PIFs.
(6)
For the two Alaskan images used in this paper, a total of 71 PIF lakes are automatically detected to derive tie points. Approximately two-thirds (48) of the generated tie points are randomly selected to develop a linear registration model to register the MSS image to the ETM+ image using bilinear resampling; the rest (23) are used for registration accuracy assessment. Subpixel accuracy is achieved with a root-mean-square error (RMSE) of 0.66 pixel at ETM+ resolution (i.e., 19 m). This low RMSE produced by the linear model using a large number of automatically generated tie points indicates that the centroids of the PIF serve as accurate tie points. Fig. 5 is a color composite image obtained by displaying the registered MSS and ETM+ NIR images in red and green, respectively. This image, compared with the image in Fig. 1, exhibits a high degree of matching, thus indicating high registration performance. The normalized correlation coefficient between the MSS and ETM+ NIR images has significantly increased after registration from 0.61 to 0.87. The noticeable offsets around lake shorelines before registration [Fig. 1(b) and (c)] are successfully removed in Fig. 5(b) and (c). The two images are ready for lake change detection. The effectiveness of invariant moments as a shape descriptor for PIF detection is illustrated in Fig. 6, which shows the plot of moment distance versus area ratio of the PIFs. Although certain PIF lakes have a comparatively large change in area,
Authorized licensed use limited to: Univ of Calif Los Angeles. Downloaded on March 13, 2009 at 16:51 from IEEE Xplore. Restrictions apply.
3914
IEEE TRANSACTIONS ON GEOSCIENCE AND REMOTE SENSING, VOL. 46, NO. 11, NOVEMBER 2008
Fig. 7. Two PIF lakes with significant area changes. (a) Expanding lake (marked Lake 1 in Fig. 6). (b) Shrinking lake (Lake 2 in Fig. 6).
their corresponding moment distances are relatively small. For example, the data point marked with a circle on the extreme right of the plot (Fig. 6) corresponds to a lake (Lake 1) with an area ratio of 1.99 (i.e., a 99% expansion) and a moment distance of 0.008. This particular lake is further analyzed for a visual inspection of its shape in Fig. 7(a). It can be seen that this lake has expanded uniformly outward between 1978 and 2000, making it a PIF lake. Similarly, the PIF (Lake 2) corresponding to the data point circled on the extreme left of the plot has an area ratio of 0.69 and a moment distance of 0.003 [see Fig. 7(b)]. The absolute errors of these two lake centroids after registration were found to be 0.57 and 0.63 pixel, respectively. This indicates the effectiveness of employing the centroids of these dynamic lakes as tie points. Sheng et al. [4] found lakes with an area change of 3% or less to be sufficiently stable for the purpose of image registration. Crosses in Fig. 6 correspond to the PIFs with an area change of more than 3%. Our study area represents a dynamic landscape with an insufficient number of stable lakes for registration, as we could identify only four lakes with an area of change of less than 3%. To further test the robustness of the proposed technique in regions with significant lake changes, a registration model was developed by excluding all the stable lakes with an area change of 3% or less. This led to a total of 67 tie points, 44 of which were used to develop the registration model and the remaining 23 for accuracy assessment. Thus, the model developed with no stable lakes as tie points led to a registration accuracy of 0.69 pixel, indicating the robustness of the proposed method in registering remote sensing images in dynamic landscapes. Therefore, the current PIF approach
provides a substantial improvement over that of Sheng et al. [4] and can be widely applied to dynamic landscapes. The performance of the proposed technique is compared with that of an automated image registration technique based on fast Fourier transform (FFT) [24]. The registration accuracy of the FFT-based technique is evaluated at 2.41 pixels (RMSE), which is significantly lower than that of the proposed method. The degraded performance of the FFT-based method may be explained by the significant variations of high-frequency information in the images. The FFT-based method requires application of high-pass filtering to the input images to remove low-frequency components [24], [25]. The high-frequency components in these images are largely lake edges in the images, which have changed significantly between the two time instances. To further evaluate the performance of the proposed registration approach, 71 (i.e., the same number of automatically generated tie points) tie points were manually selected from the original MSS and ETM+ images through a careful and tedious process. One-third (i.e., 23) of them were used as checkpoints. By using these manually selected checkpoints, the registration accuracy of the proposed technique was also estimated at the subpixel level, i.e., 0.75 pixel (RMSE). A similar linear registration model was developed using the remaining 48 (i.e., two-thirds) manually selected tie points. The same 23 manual checkpoints were used for registration accuracy assessment, which led to an RMSE of 1.42 pixels. This suggests that the quality of the automatically generated tie points is better than that of the manually selected tie points and that the proposed automated method outcompetes the conventional manual registration method.
Authorized licensed use limited to: Univ of Calif Los Angeles. Downloaded on March 13, 2009 at 16:51 from IEEE Xplore. Restrictions apply.
SHAH et al.: IMAGE REGISTRATION BASED ON PSEUDOINVARIANT METRICS OF LAND-SURFACE FEATURES
V. D ISCUSSION AND C ONCLUSION Current feature-based image registration techniques do not consider in tie-point generation the temporal stability of actual surface features to be analyzed for change detection. This can have severe impacts on the performance of change detection in dynamic landscapes, since the changes themselves interfere with the process of tie-point identification. This study employs a feature-based approach explicitly targeted to water bodies, which are highly dynamic in time and space. The procedure consists of water-body extraction, PIF detection based on object shape criteria, and image registration based on the tie points derived from the PIFs. The method is powerful in that even shrinking or contracting lakes can yield useful PIFs, as long as their overall shape is preserved. Such PIFs are identified using invariant moment shape descriptors, which are invariant to translation and scale changes. The high performance of the proposed method is demonstrated by using images acquired by sensors (MSS and ETM+) with different spectral bands, spatial resolution, and geolocation accuracy. A registration accuracy (RMSE) of 0.66 pixel at ETM+ resolution (i.e., 19 m) is achieved using tie points that are automatically generated from these two images. Results show that the offset was eliminated using the described approach. The proposed technique outcompetes the manual registration method and the FFT-based automated registration technique. The robustness of the proposed method in dynamic landscapes is indicated by the high registration accuracy (RMSE of 0.69 pixel) achieved using dynamic features only. The proposed automated image registration technique is powerful and reliable in terms of its registration accuracy, computational efficiency, and degree of automation. The uniqueness of this approach, i.e., using dynamic scale-invariant features for generating tie points, is also its strength since it overcomes the main difficulty that dynamic landscapes pose to other widely used image registration methods. Although the new approach is used here for a hydrologic application, it has great potential for other land-cover/land-use applications in dynamic landscapes lacking stable features.
3915
[8] J. P. Kern and M. S. Pattichis, “Robust multispectral image registration using mutual-information models,” IEEE Trans. Geosci. Remote Sens., vol. 45, no. 5, pp. 1494–1505, May 2007. [9] Y. Bentoutou, N. Taleb, K. Kpalma, and J. Ronsin, “An automatic image registration for applications in remote sensing,” IEEE Trans. Geosci. Remote Sens., vol. 43, no. 9, pp. 2127–2137, Sep. 2005. [10] J. Inglada, V. Muron, D. Pichard, and T. Feuvrier, “Analysis of artifacts in subpixel remote sensing image registration,” IEEE Trans. Geosci. Remote Sens., vol. 45, no. 1, pp. 254–264, Jan. 2007. [11] M. Boninsegna and M. Rossi, “Similarity measures in computer vision,” Pattern Recognit. Lett., vol. 15, no. 12, pp. 1255–1260, Dec. 1994. [12] H. J. Buiten and B. vanPutten, “Quality assessment of remote sensing image registration—Analysis and testing of control point residuals,” ISPRS J. Photogramm. Remote Sens., vol. 52, no. 2, pp. 57–73, Apr. 1997. [13] X. L. Dai and S. Khorram, “A feature-based image registration algorithm using improved chain-code representation combined with invariant moments,” IEEE Trans. Geosci. Remote Sens., vol. 37, no. 5, pp. 2351–2362, Sep. 1999. [14] H. Li, B. S. Manjunath, and S. K. Mitra, “A contour-based approach to multisensor image registration,” IEEE Trans. Image Process., vol. 4, no. 3, pp. 320–334, Mar. 1995. [15] L. C. Smith, Y. Sheng, G. M. MacDonald, and L. D. Hinzman, “Disappearing arctic lakes,” Science, vol. 308, no. 5727, p. 1429, Jun. 2005. [16] J. P. Smol and M. S. V. Douglas, “Crossing the final ecological threshold in high Arctic ponds,” Proc. Nat. Acad. Sci. U.S.A., vol. 104, no. 30, pp. 12 395–12 397, Jul. 2007. [17] M. Sezgin and B. Sankur, “Survey over image thresholding techniques and quantitative performance evaluation,” J. Electron. Imaging, vol. 13, no. 1, pp. 146–168, Jan. 2004. [18] R. C. Gonzalez and R. E. Woods, Digital Image Processing, 2nd ed. Englewood Cliffs, NJ: Prentice-Hall, 2002. [19] L. M. G. Fonseca and B. S. Manjunath, “Registration techniques for multisensor remotely sensed imagery,” Photogramm. Eng. Remote Sens., vol. 62, no. 9, pp. 1049–1056, Sep. 1996. [20] R. O. Duda, P. E. Hart, and D. G. Stork, Pattern Classification, 2nd ed. Hoboken, NJ: Wiley-Interscience, 2000. [21] V. V. Kindratenko, “On using functions to describe the shape,” J. Math. Imaging Vis., vol. 18, no. 3, pp. 225–245, May 2003. [22] M. Hu, “Visual-pattern recognition by moment invariants,” IEEE Trans. Inf. Theory, vol. IT-8, no. 2, pp. 179–187, 1962. [23] J. Flusser and T. Suk, “A moment-based approach to registration of images with affine geometric distortion,” IEEE Trans. Geosci. Remote Sens., vol. 32, no. 2, pp. 382–387, Mar. 1994. [24] B. S. Reddy and B. N. Chatterji, “An FFT-based technique for translation, rotation, and scale-invariant image registration,” IEEE Trans. Image Process., vol. 5, no. 8, pp. 1266–1271, Aug. 1996. [25] H. Xie, N. Hicks, G. Randy Keller, H. Huang, and V. Kreinovich, “An IDL/ENVI implementation of the FFT-based algorithm for automatic image registration,” Comput. Geosci., vol. 29, no. 8, pp. 1045– 1055, 2003.
R EFERENCES [1] A. Singh, “Review article digital change detection techniques using remotely-sensed data,” Int. J. Remote Sens., vol. 10, no. 6, pp. 989–1003, Jun. 1989. [2] L. Bruzzone and R. Cossu, “An adaptive approach to reducing registration noise effects in unsupervised change detection,” IEEE Trans. Geosci. Remote Sens., vol. 41, no. 11, pp. 2455–2465, Nov. 2003. [3] X. L. Dai and S. Khorram, “The effects of image misregistration on the accuracy of remotely sensed change detection,” IEEE Trans. Geosci. Remote Sens., vol. 36, no. 5, pp. 1566–1577, Sep. 1998. [4] Y. Sheng, C. A. Shah, and L. C. Smith, “Automated image registration for hydrologic change detection in the lake-rich Arctic,” IEEE Geosci. Remote Sens. Lett., vol. 5, no. 3, pp. 414–418, Jul. 2008. [5] D. Robinson and P. Milanfar, “Fundamental performance limits in image registration,” IEEE Trans. Image Process., vol. 13, no. 9, pp. 1185–1199, Sep. 2004. [6] A. Wong and D. A. Clausi, “ARRSI: Automatic registration of remotesensing images,” IEEE Trans. Geosci. Remote Sens., vol. 45, no. 5, pp. 1483–1493, May 2007. [7] J. Inglada and A. Giros, “On the possibility of automatic multisensor image registration,” IEEE Trans. Geosci. Remote Sens., vol. 42, no. 10, pp. 2104–2120, Oct. 2004.
Chintan A. Shah received the B.E. degree in computer engineering from Mumbai University, Mumbai, Maharashtra, India, in 2000 and the M.S. degree in computer science from Syracuse University, Syracuse, NY, in 2003. He is currently working toward the Ph.D. degree in the Department of Geography, University of California, Los Angeles (UCLA). His research interests include remote sensing, statistical pattern recognition, spatial statistics, data fusion, change detection, and automatic image registration and mosaicking. His current research involves analyzing multitemporal images to understand global lake dynamics, particularly in the regions sensitive to climate change. Mr. Shah is the recipient of the All-University Master’s Prize for his M.S. thesis on multispectral/hyperspectral image classification based on independent component analysis. He received the American Society for Photogrammetry and Remote Sensing Central New York Region Kodak Student of the Year Award in 2004, the UCLA Fellowship in 2007, and the UCLA Extramural Grant Award in 2007.
Authorized licensed use limited to: Univ of Calif Los Angeles. Downloaded on March 13, 2009 at 16:51 from IEEE Xplore. Restrictions apply.
3916
IEEE TRANSACTIONS ON GEOSCIENCE AND REMOTE SENSING, VOL. 46, NO. 11, NOVEMBER 2008
Yongwei Sheng received the B.S. and M.E. degrees in remote sensing from Zhejiang University, Hangzhou, China, in 1988 and 1991, respectively, and the Ph.D. degree in environmental science, policy, and management from the University of California, Berkeley, in 2000. From 2004 to 2006, he was with the faculty of the State University of New York College of Environmental Science and Forestry, Syracuse. He is currently an Assistant Professor with the Department of Geography, University of California, Los Angeles. He has authored about 35 published refereed journal articles. His research interests include remote sensing, photogrammetry, GIS, and their applications addressing natural resources management and environmental assessment, particularly regional and global-scale lake dynamics. Dr. Sheng is the recipient of the 1997 American Society for Photogrammetry and Remote Sensing William A. Fischer Remote Sensing Scholarship Award. He received the NASA New Investigator Award in 2006.
Laurence C. Smith received the B.S. degree in Earth sciences from the University of Illinois, UrbanaChampaign, the M.S. degree in Earth sciences from Indiana University, Bloomington, and the Ph.D. degree in Earth and atmospheric sciences from Cornell University, Ithaca, NY, in 1989, 1991, and 1996, respectively. He is currently a Professor with the Department of Geography and the Department of Earth and Space Sciences, University of California, Los Angeles. His research focuses on the effects of global warming in northern environments. He is the author of 45 journal articles. Dr. Smith received the Guggenheim Fellowship in 2006.
Authorized licensed use limited to: Univ of Calif Los Angeles. Downloaded on March 13, 2009 at 16:51 from IEEE Xplore. Restrictions apply.
IEEE TRANSACTIONS ON GEOSCIENCE AND REMOTE SENSING, VOL. 46, NO. 11, NOVEMBER 2008
Automated Image Registration Based on Pseudoinvariant Metrics of Dynamic Land-Surface Features Chintan A. Shah, Yongwei Sheng, and Laurence C. Smith
Abstract—Accurate assessment of land-cover/land-use change is essential for understanding the impacts of global change and necessitates the use of satellite data. Satellite change detection requires large volumes of multitemporal images to be precisely registered. Image registration is particularly difficult in dynamic (i.e., rapidly time varying) landscapes since the changes themselves interfere with the process of tie-point identification. Despite the existence of sophisticated registration algorithms, it is still problematic to register images acquired over such areas due to a dearth of stable features. Hence, we propose an automated image registration method using tie points derived from pseudoinvariant features (PIFs) and apply the method to register satellite images for hydrologic change detection in the Arctic, where abundant shallow lakes dominate the landscape but change significantly over time. A key to the method is the identification of “shape-stable” lakes as PIFs, which preserve their geometric shape even though the shorelines may migrate significantly. The proposed method automatically identifies PIFs based on scale-invariant shape descriptors and employs their center points for establishing the registration model. Our method thus consists of water-body feature extraction, PIF detection based on feature shape criteria, and image registration using tie points derived from the PIFs. The approach is used to register 1978 and 2000 Landsat images in Alaska, where lakes dominate the landscape and change significantly over time. The performance of the proposed approach is evaluated quantitatively, and a high subpixel registration accuracy of 0.66 pixel at Enhanced Thematic Mapper Plus resolution (i.e., 19 m) is achieved. A comparative evaluation indicates that the proposed approach outcompetes the conventional manual tie-point selection method and automated image registration techniques based on fast Fourier transform. Index Terms—Centroid, invariant moments, lake dynamics, land-cover/land-use change, multitemporal image analysis, precise registration, pseudoinvariant feature (PIF).
Manuscript received February 13, 2008; revised April 8, 2008. Current version published October 30, 2008. This work was supported in part by the National Aeronautics and Space Administration through the Terrestrial Hydrology Program under Contract NNX08AE51G and in part by the National Science Foundation through the Arctic System Science Program under Contract ARC-0713903. C. A. Shah and Y. Sheng are with the Department of Geography, University of California, Los Angeles, CA 90095-1524 USA (e-mail: [email protected]). L. C. Smith is with the Department of Geography, University of California, Los Angeles, CA 90095-1524 USA, and also with the Department of Earth and Space Sciences, University of California, Los Angeles, CA 90095-1567 USA. Digital Object Identifier 10.1109/TGRS.2008.2000636
I. I NTRODUCTION
E
NVIRONMENTAL monitoring of the Earth’s surface requires detecting changes using multitemporal remotely sensed images. Change detection techniques quantify surface features’ radiometric and/or spatial changes over time and involve a pixel-by-pixel comparison of multitemporal images acquired over the same ground area at different time instances. Therefore, it is essential for the multitemporal images to be spatially registered to each other [1]. When multitemporal images are not precisely registered, the residual misregistration presents a source of error, known as the “registration noise” [2], which greatly influences change detection accuracy [3]. Traditionally, image registration relies on manual selection of tie points (i.e., control points). The performance of manual registration techniques depends on the accuracy of tie-point selection from the image pair to be registered. Linear features and feature intersection points are often ideal candidates for tie points. However, when the number of image pairs to be registered is large, human intervention becomes impractical, necessitating the use of automated image registration techniques. A significant amount of research has been focused on the development of sophisticated multitemporal image registration algorithms [4]–[9]. However, it still remains challenging to obtain precise registration of multitemporal remote sensing images [5], [10], particularly in dynamic landscapes [4]. Hence, this paper particularly focuses on automated image registration for change detection in dynamic landscapes. Automated image registration techniques may be classified into two broad categories: 1) area based and 2) feature based [7]. Area-based matching techniques employ metrics [8], [11] to measure the similarity between two images in terms of their radiometric characteristics. Sum of absolute differences, normalized cross correlation, and mutual information are examples of widely used similarity metrics. However, their performance is limited by various factors, including atmospheric degradations, illumination effects, and sensor response differences in multitemporal images, necessitating the use of scene-to-scene radiometric normalization [5], [12]. In contrast, feature-based methods that use points, edges, and contours are relatively less sensitive to such effects. Hence, feature-based registration techniques establish correspondence between images without the need for scene normalization [13], [14]. The performance of current registration algorithms is limited in dynamic landscapes where land cover changes dramatically [4]. A good example of such environments is the Arctic and
0196-2892/$25.00 © 2008 IEEE
Authorized licensed use limited to: Univ of Calif Los Angeles. Downloaded on March 13, 2009 at 16:51 from IEEE Xplore. Restrictions apply.
SHAH et al.: IMAGE REGISTRATION BASED ON PSEUDOINVARIANT METRICS OF LAND-SURFACE FEATURES
3909
Fig. 1. (a) Color composite image formed by MSS and ETM+ NIR images displayed as the red and green channels, respectively. Stable lakes appear black, and stable land appears yellow. New (or expanded) lakes appear red, whereas vanished (or shrunk) lakes appear green. (b) and (c) Close visual inspection of subimages reveals a misregistration offset of ∼2–3 pixels, as indicated by the red and green edges on the opposite lake banks.
sub-Arctic, characterized by vast area, few man-made features, and numerous shallow ponds, lakes, and wetlands. They come and go in response to seasonal and annual climate cycles. Scientific interest in these environments is at an all-time high, owing to their acute sensitivity to global climate change [15]. Lakes and their shorelines are the primary features that are visible in satellite images of lake-rich Arctic environments. However, neither of these features can be assumed to be temporally stable. Lakeshores are quite dynamic, subject to changing water balance, shoreline erosion, tapping, and emergent vegetation growth, preventing their use as matching primitives. Feature-based algorithms that use lakeshores as feature primitives are therefore unsatisfactory. Hence, Sheng et al. [4] proposed the use of stable lakes with an area change of 3%
or less and employed the centroids of these stable lakes as tie points for automated image registration. However, it still may not be possible to find a sufficient number of stable lakes, particularly when environmental conditions have changed dramatically, e.g., a wet or dry year. Hence, this paper introduces the concept of pseudoinvariant features (PIFs)—lakes that have not undergone significant changes in shape over time—and, thus, uses both stable and dynamic lakes for image registration. The proposed method includes procedures of water-body extraction, PIF detection based on feature shape criteria, and image registration using tie points derived from the PIFs. The approach is then employed to register Landsat images for lake change detection in the Yukon flats region of Alaska, where lakes dominate the landscape and change significantly over time.
Authorized licensed use limited to: Univ of Calif Los Angeles. Downloaded on March 13, 2009 at 16:51 from IEEE Xplore. Restrictions apply.
3910
IEEE TRANSACTIONS ON GEOSCIENCE AND REMOTE SENSING, VOL. 46, NO. 11, NOVEMBER 2008
II. S TUDY A REA AND D ATA S ET U SED Recent Arctic warming has caused significant regional changes in lake distribution in the Arctic and sub-Arctic regions [15], [16]. Therefore, a systematic pan-Arctic-scale inventory of lake changes is essential for us to understand the overall impacts of the Arctic climate warming. This requires automated registration of thousands of Landsat Multi-Spectral Scanner (MSS), Thematic Mapper (TM), and Enhanced Thematic Mapper Plus (ETM+) images archived over the past ∼30 years. To test the proposed image registration approach, this study uses one MSS image (Path:77/Row:13) and one ETM+ image (Path:70/Row:14) in Alaska acquired on August 3, 1978, and August 16, 2000, respectively. They are both GeoCover orthorectified image products. Our study area is in the Yukon flats region of Alaska, with latitudes and longitudes ranging from 66◦ 6 N to 66◦ 21 N and 148◦ 30 W to 149◦ 24 W. Fig. 1(a) provides the color composite image by displaying the MSS and the ETM+ near-infrared (NIR) bands in red and green, respectively. This false-color image illustrates both real lake changes and misregistration errors. Stable water bodies appear black, and stable land appears yellow. New (or expanded) water bodies appear red, whereas vanished (or shrunk) water bodies appear green. The red edges on one side of the lakes and the green ones on the other side indicate a misregistration offset between the two satellite images. A close visual inspection of Fig. 1(b) and (c) reveals an offset of ∼2–3 pixels between the multitemporal images. The misregistration offset has to be removed before real lake changes may be quantified. III. M ETHOD To establish correspondence between features in multitemporal images, it is essential to have a priori knowledge about the characteristics of their spatial change. For example, lakes may shrink, grow, or stay stable over a period of time. To employ the lakes themselves as tie points in an automated featurebased registration, it is straightforward to use only the stable lakes. The centroids of such unchanged lakes can be effectively employed to estimate the registration parameters [4]. However, as described earlier, such unchanged lakes may not be sufficient in quantity and/or sufficiently distributed throughout the images in dynamic landscapes to obtain an unbiased registration model, thus requiring relaxation of the criteria for “stable” lakes. Observations of lake changes in flat terrain areas such as the broad Arctic lowlands indicate, in general, that many lakes grow or shrink radially outward or inward, with their shapes and centroids remaining relatively stable (Fig. 2). The lake boundary (solid line) at time t1 migrates to a new location (dashed line) at t2 after Δt period of time. The centroids Ct1 and Ct2 of the lake at time t1 and t2 only migrate by a small distance ΔC, and the lake shape remains relatively stable even though the size of the lake changes significantly [Fig. 2(b) and (c)]. This lake at t2 can be expressed as a scaled version of the lake at t1 . In such cases, scale-invariant descriptors such as invariant moments can be employed to describe the shape of dynamic lakes for subsequent registration tie-point development. To this end, a lake is considered to be a PIF if it is stable or its change is invariant to a shape descriptor.
Fig. 2. Illustration of typical lake changes. Their shapes and centers remain relatively stable, even though the lakes change significantly. The contour with solid line corresponds to the boundary of a lake at time t1 , whereas the dashed line corresponds to the boundary of the same lake at time t2 = t1 + Δt. Furthermore, Ct1 and Ct2 are the centroids of these lake at time t1 and t2 , respectively, and ΔC is the Euclidean distance between the two centroids. (a) Lake has undergone insignificant changes. (b) Lake has expanded radially outward. (c) Lake has shrunk radially inward.
Fig. 3 illustrates the workflow of the proposed method. Each of the multitemporal images is processed independently through the steps of feature extraction and centroid detection. Next, lake correspondence between the images is established to identify identical lakes. PIF lakes are screened from these identical lakes using invariant moments, and their centroids are extracted as tie points for subsequent image registration. These major steps are detailed in the following sections. A. Feature Extraction Feature extraction aims at segmenting the features of interest from background features in the image. Since the objective here is to employ the delineated features for subsequent image registration, the spectral characteristics of the features should be known beforehand. For example, open water exhibits a very low spectral response in the NIR wavelengths relative to the surrounding terrain. Thus, a simple threshold may be applied to an NIR image to segment water bodies from the background. Although several thresholding techniques have been proposed in the literature [17], none of these techniques have yet been proven to be optimal, their performances vary depending upon the characteristics of the images being processed, and the segmentation results are sensitive to the selected threshold value. With multispectral images, the additional spectral bands may be used to improve water-body segmentation. Hence, this paper first applies a thresholding segmentation method and then refines the initial segmentation using the maximumlikelihood classifier, a supervised classification technique using all the available spectral information relevant to water bodies. A simple filtering technique is then employed to remove isolated pixels for clear identification of the lakes.
Authorized licensed use limited to: Univ of Calif Los Angeles. Downloaded on March 13, 2009 at 16:51 from IEEE Xplore. Restrictions apply.
SHAH et al.: IMAGE REGISTRATION BASED ON PSEUDOINVARIANT METRICS OF LAND-SURFACE FEATURES
3911
Fig. 3. Major steps involved in the proposed approach (a) to be applied on each of the multitemporal data sets and (b) to be applied to identify PIF lakes and generate tie points for registration model development and assessment.
B. Centroid Detection Once the lakes are detected in each multitemporal image, they are matched between the images using their centroids. The lake centroids are calculated as
xc = m10 /m00 yc = m01 /m00
(1)
where mpq = x y xp y q f (x, y) is the (p + q)th-order moment of an image f (x, y). When applied to a binary lake image, mpq becomes [18] mpq =
x
xp y q .
(2)
y
From (2), note that m00 corresponds to the area of the lake object, whereas the centroid coordinates xc and yc are the averages of the x and the y coordinates of the pixels within the lake, respectively. The steps in Fig. 3(b) operate on the output of individually processed images as outlined in Fig. 3(a). For each centroid at t1 , the corresponding centroid with the minimum Euclidean distance is determined in the image acquired at t2 . These centroids are used to identify identical lakes in the next step. C. Lake Matching As many satellite images are georeferenced, this paper assumes that the input images have been coarsely registered. For images that have not been georeferenced, automated coarse registration techniques can be employed to roughly register them. Once the two input images are roughly aligned, lakes on the two images may be easily identified using their centroids. If the distance between the lake centroids (ΔC) in the two images is less than a certain centroid distance threshold, then the two lake objects at t1 and t2 are considered to be the same lake. A large value of ΔC indicates that either the shape of the lake has changed significantly or it has completely disappeared, making ΔC the distance of that lake at t1 to the nearest lake in t2 . Thus, a centroid threshold facilitates the elimination of dynamic lakes,
Fig. 4. (Light gray) PIF lakes identified from ETM+. (Dark gray) Remaining lake objects that failed to satisfy the PIF criteria.
which are not PIFs. Furthermore, ΔC values when averaged over all the PIFs (μΔC ) indicate the approximate amount of spatial misalignment between the images prior to registration, whereas their standard deviation σΔC indicates the offset variation. Hence, we estimate the centroid distance threshold as μΔC + 2σΔC . For the two images used in this study, the values for μΔC and σΔC are calculated to be 2.87 pixels and 0.91 pixel, respectively. Therefore, the centroid distance threshold is determined to be 4.7 pixels. D. PIF Lake Detection The PIF detection criterion tests the shape similarity of the corresponding lakes at t1 and t2 . As illustrated in Fig. 2, a lake is considered to be a PIF if the lake either has not changed significantly or has expanded or contracted radially with little geometric shape change. In each case, the lake at t2 can be expressed as a scaled version of the lake at t1 . A shape descriptor establishes a quantitative representation of the shape of an object [20]. Since the expanding or shrinking lakes vary in their scale over time, the descriptor employed for encoding their shape should be invariant to scale differences,
Authorized licensed use limited to: Univ of Calif Los Angeles. Downloaded on March 13, 2009 at 16:51 from IEEE Xplore. Restrictions apply.
3912
IEEE TRANSACTIONS ON GEOSCIENCE AND REMOTE SENSING, VOL. 46, NO. 11, NOVEMBER 2008
Fig. 5. Color composite image after registration. Offsets found in the original images have been removed by the proposed registration method.
in addition to being invariant to translation changes. Widely used shape descriptors include Fourier descriptors, chain codes, shape signatures, centroidal profiles, and invariant moments [20]. A large body of research on translational, rotational, and scale-invariant descriptors has recommended the use of invariant moments and chain codes [18], [20], [21]. In this paper, invariant moments [22] have been used for shape description as also recommended in some of the earlier literature [13], [14], [23]. The moment-based shape descriptors use moments for shape description and, thus, have been established in a statistical framework. Furthermore, compared to other descriptors such as chain codes, the moment-based descriptors have been found to be more robust to the effects of noise [20]. With the (p + q)th-order moment of a binary lake
image given in (2), the corresponding central moments are defined as [20] μpq =
x
(x − xc )p (y − yc )q
(3)
y
where xc and yc are the centroid coordinates of the object being subtracted from the object coordinates (x, y) to make the moments invariant to translation (hence, the term “central moments”). The advantage of using these moments is that their characteristics are easily interpreted with reference to the standard statistical moments [22]. For example, the first-order statistical moment corresponds to the mean, and the second-, third-, and fourth-order moments correspond to the variance,
Authorized licensed use limited to: Univ of Calif Los Angeles. Downloaded on March 13, 2009 at 16:51 from IEEE Xplore. Restrictions apply.
SHAH et al.: IMAGE REGISTRATION BASED ON PSEUDOINVARIANT METRICS OF LAND-SURFACE FEATURES
3913
skewness, and kurtosis, respectively. Similarly, the zero-order image moment represents the area of the binary object, the first-order moment is employed in estimating the corresponding centroids of the object, and the second-order moment expresses the distribution of matter around the centroid. The third-order moment expresses the symmetry of the object, and the fourthorder moment determines its flatness. To make these moments invariant to scale, the normalized central moments, denoted as ηpq , are defined as ηpq =
μpq μγ00
(4)
where γ = (p + q/2) + 1, for p + q ≥ 2. Since higher order moments are more sensitive to noise, only the first seven moments based on the second and third normalized central moments are used [22], which are given as follows: 1 (μ20 + μ02 ) μ200 1 φ2 = 4 (μ20 − μ02 )2 + 4μ211 μ00 1 φ3 = 5 (μ30 − 3μ12 )2 + (3μ21 − μ03 )2 μ00 1 φ4 = 5 (μ30 + μ12 )2 + (μ21 + μ03 )2 μ00
If the metric d of a lake is less than a defined threshold, the lake is considered to be a PIF. Fig. 4 shows the PIF lakes (in light gray) identified from the ETM+ images with a shape similarity metric threshold of 0.01. This threshold was selected to produce a desired number (about 70 PIF lakes in this paper) of welldistributed high-quality tie points for both registration model development and accuracy assessment. A lower threshold leads to the selection of fewer but more reliable PIF lakes. Once the PIFs are determined, the centroid pairs of PIF lakes are used as tie points for the registration model development and assessment to precisely register the images.
φ1 =
φ5 =
1 {(μ30 − 3μ12 )(μ30 + μ12 ) μ10 00 × (μ30 + μ12 )2 − 3(μ21 + μ03 )2
IV. E XPERIMENTAL R ESULTS
+ (3μ21 − μ03 )(μ21 + μ03 ) × 3(μ30 + μ12 )2 − (μ21 + μ03 )2 φ6 =
1 (μ20 − μ02 ) (μ30 + μ12 )2 − (μ21 + μ03 )2 7 μ00 + 4μ11 (μ30 + μ12 )(μ21 + μ03 )}
φ7 =
1 {(3μ21 − μ03 )(μ30 + μ12 ) μ10 00 × (μ30 + μ12 )2 − 3(μ21 + μ03 )2 + (3μ12 − μ30 )(μ21 + μ03 ) × 3(μ30 + μ12 )2 − (μ21 + μ03 )2 .
(5)
Thus, a set of seven moments φ = {φ1 , . . . , φ7 } invariant to translation and scale changes are subsequently employed for shape representation. Once the invariant moments are determined for a correspondent lake in the images, an invariant moment metric is computed as the lake shape similarity measure between times t1 and t2
7
d= [ϕi (t1 ) − ϕi (t2 )]2 . i=1
Fig. 6. Plot of moment distance versus area ratio of the identified PIFs.
(6)
For the two Alaskan images used in this paper, a total of 71 PIF lakes are automatically detected to derive tie points. Approximately two-thirds (48) of the generated tie points are randomly selected to develop a linear registration model to register the MSS image to the ETM+ image using bilinear resampling; the rest (23) are used for registration accuracy assessment. Subpixel accuracy is achieved with a root-mean-square error (RMSE) of 0.66 pixel at ETM+ resolution (i.e., 19 m). This low RMSE produced by the linear model using a large number of automatically generated tie points indicates that the centroids of the PIF serve as accurate tie points. Fig. 5 is a color composite image obtained by displaying the registered MSS and ETM+ NIR images in red and green, respectively. This image, compared with the image in Fig. 1, exhibits a high degree of matching, thus indicating high registration performance. The normalized correlation coefficient between the MSS and ETM+ NIR images has significantly increased after registration from 0.61 to 0.87. The noticeable offsets around lake shorelines before registration [Fig. 1(b) and (c)] are successfully removed in Fig. 5(b) and (c). The two images are ready for lake change detection. The effectiveness of invariant moments as a shape descriptor for PIF detection is illustrated in Fig. 6, which shows the plot of moment distance versus area ratio of the PIFs. Although certain PIF lakes have a comparatively large change in area,
Authorized licensed use limited to: Univ of Calif Los Angeles. Downloaded on March 13, 2009 at 16:51 from IEEE Xplore. Restrictions apply.
3914
IEEE TRANSACTIONS ON GEOSCIENCE AND REMOTE SENSING, VOL. 46, NO. 11, NOVEMBER 2008
Fig. 7. Two PIF lakes with significant area changes. (a) Expanding lake (marked Lake 1 in Fig. 6). (b) Shrinking lake (Lake 2 in Fig. 6).
their corresponding moment distances are relatively small. For example, the data point marked with a circle on the extreme right of the plot (Fig. 6) corresponds to a lake (Lake 1) with an area ratio of 1.99 (i.e., a 99% expansion) and a moment distance of 0.008. This particular lake is further analyzed for a visual inspection of its shape in Fig. 7(a). It can be seen that this lake has expanded uniformly outward between 1978 and 2000, making it a PIF lake. Similarly, the PIF (Lake 2) corresponding to the data point circled on the extreme left of the plot has an area ratio of 0.69 and a moment distance of 0.003 [see Fig. 7(b)]. The absolute errors of these two lake centroids after registration were found to be 0.57 and 0.63 pixel, respectively. This indicates the effectiveness of employing the centroids of these dynamic lakes as tie points. Sheng et al. [4] found lakes with an area change of 3% or less to be sufficiently stable for the purpose of image registration. Crosses in Fig. 6 correspond to the PIFs with an area change of more than 3%. Our study area represents a dynamic landscape with an insufficient number of stable lakes for registration, as we could identify only four lakes with an area of change of less than 3%. To further test the robustness of the proposed technique in regions with significant lake changes, a registration model was developed by excluding all the stable lakes with an area change of 3% or less. This led to a total of 67 tie points, 44 of which were used to develop the registration model and the remaining 23 for accuracy assessment. Thus, the model developed with no stable lakes as tie points led to a registration accuracy of 0.69 pixel, indicating the robustness of the proposed method in registering remote sensing images in dynamic landscapes. Therefore, the current PIF approach
provides a substantial improvement over that of Sheng et al. [4] and can be widely applied to dynamic landscapes. The performance of the proposed technique is compared with that of an automated image registration technique based on fast Fourier transform (FFT) [24]. The registration accuracy of the FFT-based technique is evaluated at 2.41 pixels (RMSE), which is significantly lower than that of the proposed method. The degraded performance of the FFT-based method may be explained by the significant variations of high-frequency information in the images. The FFT-based method requires application of high-pass filtering to the input images to remove low-frequency components [24], [25]. The high-frequency components in these images are largely lake edges in the images, which have changed significantly between the two time instances. To further evaluate the performance of the proposed registration approach, 71 (i.e., the same number of automatically generated tie points) tie points were manually selected from the original MSS and ETM+ images through a careful and tedious process. One-third (i.e., 23) of them were used as checkpoints. By using these manually selected checkpoints, the registration accuracy of the proposed technique was also estimated at the subpixel level, i.e., 0.75 pixel (RMSE). A similar linear registration model was developed using the remaining 48 (i.e., two-thirds) manually selected tie points. The same 23 manual checkpoints were used for registration accuracy assessment, which led to an RMSE of 1.42 pixels. This suggests that the quality of the automatically generated tie points is better than that of the manually selected tie points and that the proposed automated method outcompetes the conventional manual registration method.
Authorized licensed use limited to: Univ of Calif Los Angeles. Downloaded on March 13, 2009 at 16:51 from IEEE Xplore. Restrictions apply.
SHAH et al.: IMAGE REGISTRATION BASED ON PSEUDOINVARIANT METRICS OF LAND-SURFACE FEATURES
V. D ISCUSSION AND C ONCLUSION Current feature-based image registration techniques do not consider in tie-point generation the temporal stability of actual surface features to be analyzed for change detection. This can have severe impacts on the performance of change detection in dynamic landscapes, since the changes themselves interfere with the process of tie-point identification. This study employs a feature-based approach explicitly targeted to water bodies, which are highly dynamic in time and space. The procedure consists of water-body extraction, PIF detection based on object shape criteria, and image registration based on the tie points derived from the PIFs. The method is powerful in that even shrinking or contracting lakes can yield useful PIFs, as long as their overall shape is preserved. Such PIFs are identified using invariant moment shape descriptors, which are invariant to translation and scale changes. The high performance of the proposed method is demonstrated by using images acquired by sensors (MSS and ETM+) with different spectral bands, spatial resolution, and geolocation accuracy. A registration accuracy (RMSE) of 0.66 pixel at ETM+ resolution (i.e., 19 m) is achieved using tie points that are automatically generated from these two images. Results show that the offset was eliminated using the described approach. The proposed technique outcompetes the manual registration method and the FFT-based automated registration technique. The robustness of the proposed method in dynamic landscapes is indicated by the high registration accuracy (RMSE of 0.69 pixel) achieved using dynamic features only. The proposed automated image registration technique is powerful and reliable in terms of its registration accuracy, computational efficiency, and degree of automation. The uniqueness of this approach, i.e., using dynamic scale-invariant features for generating tie points, is also its strength since it overcomes the main difficulty that dynamic landscapes pose to other widely used image registration methods. Although the new approach is used here for a hydrologic application, it has great potential for other land-cover/land-use applications in dynamic landscapes lacking stable features.
3915
[8] J. P. Kern and M. S. Pattichis, “Robust multispectral image registration using mutual-information models,” IEEE Trans. Geosci. Remote Sens., vol. 45, no. 5, pp. 1494–1505, May 2007. [9] Y. Bentoutou, N. Taleb, K. Kpalma, and J. Ronsin, “An automatic image registration for applications in remote sensing,” IEEE Trans. Geosci. Remote Sens., vol. 43, no. 9, pp. 2127–2137, Sep. 2005. [10] J. Inglada, V. Muron, D. Pichard, and T. Feuvrier, “Analysis of artifacts in subpixel remote sensing image registration,” IEEE Trans. Geosci. Remote Sens., vol. 45, no. 1, pp. 254–264, Jan. 2007. [11] M. Boninsegna and M. Rossi, “Similarity measures in computer vision,” Pattern Recognit. Lett., vol. 15, no. 12, pp. 1255–1260, Dec. 1994. [12] H. J. Buiten and B. vanPutten, “Quality assessment of remote sensing image registration—Analysis and testing of control point residuals,” ISPRS J. Photogramm. Remote Sens., vol. 52, no. 2, pp. 57–73, Apr. 1997. [13] X. L. Dai and S. Khorram, “A feature-based image registration algorithm using improved chain-code representation combined with invariant moments,” IEEE Trans. Geosci. Remote Sens., vol. 37, no. 5, pp. 2351–2362, Sep. 1999. [14] H. Li, B. S. Manjunath, and S. K. Mitra, “A contour-based approach to multisensor image registration,” IEEE Trans. Image Process., vol. 4, no. 3, pp. 320–334, Mar. 1995. [15] L. C. Smith, Y. Sheng, G. M. MacDonald, and L. D. Hinzman, “Disappearing arctic lakes,” Science, vol. 308, no. 5727, p. 1429, Jun. 2005. [16] J. P. Smol and M. S. V. Douglas, “Crossing the final ecological threshold in high Arctic ponds,” Proc. Nat. Acad. Sci. U.S.A., vol. 104, no. 30, pp. 12 395–12 397, Jul. 2007. [17] M. Sezgin and B. Sankur, “Survey over image thresholding techniques and quantitative performance evaluation,” J. Electron. Imaging, vol. 13, no. 1, pp. 146–168, Jan. 2004. [18] R. C. Gonzalez and R. E. Woods, Digital Image Processing, 2nd ed. Englewood Cliffs, NJ: Prentice-Hall, 2002. [19] L. M. G. Fonseca and B. S. Manjunath, “Registration techniques for multisensor remotely sensed imagery,” Photogramm. Eng. Remote Sens., vol. 62, no. 9, pp. 1049–1056, Sep. 1996. [20] R. O. Duda, P. E. Hart, and D. G. Stork, Pattern Classification, 2nd ed. Hoboken, NJ: Wiley-Interscience, 2000. [21] V. V. Kindratenko, “On using functions to describe the shape,” J. Math. Imaging Vis., vol. 18, no. 3, pp. 225–245, May 2003. [22] M. Hu, “Visual-pattern recognition by moment invariants,” IEEE Trans. Inf. Theory, vol. IT-8, no. 2, pp. 179–187, 1962. [23] J. Flusser and T. Suk, “A moment-based approach to registration of images with affine geometric distortion,” IEEE Trans. Geosci. Remote Sens., vol. 32, no. 2, pp. 382–387, Mar. 1994. [24] B. S. Reddy and B. N. Chatterji, “An FFT-based technique for translation, rotation, and scale-invariant image registration,” IEEE Trans. Image Process., vol. 5, no. 8, pp. 1266–1271, Aug. 1996. [25] H. Xie, N. Hicks, G. Randy Keller, H. Huang, and V. Kreinovich, “An IDL/ENVI implementation of the FFT-based algorithm for automatic image registration,” Comput. Geosci., vol. 29, no. 8, pp. 1045– 1055, 2003.
R EFERENCES [1] A. Singh, “Review article digital change detection techniques using remotely-sensed data,” Int. J. Remote Sens., vol. 10, no. 6, pp. 989–1003, Jun. 1989. [2] L. Bruzzone and R. Cossu, “An adaptive approach to reducing registration noise effects in unsupervised change detection,” IEEE Trans. Geosci. Remote Sens., vol. 41, no. 11, pp. 2455–2465, Nov. 2003. [3] X. L. Dai and S. Khorram, “The effects of image misregistration on the accuracy of remotely sensed change detection,” IEEE Trans. Geosci. Remote Sens., vol. 36, no. 5, pp. 1566–1577, Sep. 1998. [4] Y. Sheng, C. A. Shah, and L. C. Smith, “Automated image registration for hydrologic change detection in the lake-rich Arctic,” IEEE Geosci. Remote Sens. Lett., vol. 5, no. 3, pp. 414–418, Jul. 2008. [5] D. Robinson and P. Milanfar, “Fundamental performance limits in image registration,” IEEE Trans. Image Process., vol. 13, no. 9, pp. 1185–1199, Sep. 2004. [6] A. Wong and D. A. Clausi, “ARRSI: Automatic registration of remotesensing images,” IEEE Trans. Geosci. Remote Sens., vol. 45, no. 5, pp. 1483–1493, May 2007. [7] J. Inglada and A. Giros, “On the possibility of automatic multisensor image registration,” IEEE Trans. Geosci. Remote Sens., vol. 42, no. 10, pp. 2104–2120, Oct. 2004.
Chintan A. Shah received the B.E. degree in computer engineering from Mumbai University, Mumbai, Maharashtra, India, in 2000 and the M.S. degree in computer science from Syracuse University, Syracuse, NY, in 2003. He is currently working toward the Ph.D. degree in the Department of Geography, University of California, Los Angeles (UCLA). His research interests include remote sensing, statistical pattern recognition, spatial statistics, data fusion, change detection, and automatic image registration and mosaicking. His current research involves analyzing multitemporal images to understand global lake dynamics, particularly in the regions sensitive to climate change. Mr. Shah is the recipient of the All-University Master’s Prize for his M.S. thesis on multispectral/hyperspectral image classification based on independent component analysis. He received the American Society for Photogrammetry and Remote Sensing Central New York Region Kodak Student of the Year Award in 2004, the UCLA Fellowship in 2007, and the UCLA Extramural Grant Award in 2007.
Authorized licensed use limited to: Univ of Calif Los Angeles. Downloaded on March 13, 2009 at 16:51 from IEEE Xplore. Restrictions apply.
3916
IEEE TRANSACTIONS ON GEOSCIENCE AND REMOTE SENSING, VOL. 46, NO. 11, NOVEMBER 2008
Yongwei Sheng received the B.S. and M.E. degrees in remote sensing from Zhejiang University, Hangzhou, China, in 1988 and 1991, respectively, and the Ph.D. degree in environmental science, policy, and management from the University of California, Berkeley, in 2000. From 2004 to 2006, he was with the faculty of the State University of New York College of Environmental Science and Forestry, Syracuse. He is currently an Assistant Professor with the Department of Geography, University of California, Los Angeles. He has authored about 35 published refereed journal articles. His research interests include remote sensing, photogrammetry, GIS, and their applications addressing natural resources management and environmental assessment, particularly regional and global-scale lake dynamics. Dr. Sheng is the recipient of the 1997 American Society for Photogrammetry and Remote Sensing William A. Fischer Remote Sensing Scholarship Award. He received the NASA New Investigator Award in 2006.
Laurence C. Smith received the B.S. degree in Earth sciences from the University of Illinois, UrbanaChampaign, the M.S. degree in Earth sciences from Indiana University, Bloomington, and the Ph.D. degree in Earth and atmospheric sciences from Cornell University, Ithaca, NY, in 1989, 1991, and 1996, respectively. He is currently a Professor with the Department of Geography and the Department of Earth and Space Sciences, University of California, Los Angeles. His research focuses on the effects of global warming in northern environments. He is the author of 45 journal articles. Dr. Smith received the Guggenheim Fellowship in 2006.
Authorized licensed use limited to: Univ of Calif Los Angeles. Downloaded on March 13, 2009 at 16:51 from IEEE Xplore. Restrictions apply.
