3,D Building Reconstruction Using IKONOS ... - Semantic Scholar

Report 1 Downloads 53 Views
3-D Building Reconstruction Using IKONOS Multispectral Stereo Images Hong-Gyoo Sohn, Choung-Hwan Park, and Joon Heo School of Civil Engineering, Yonsei University, Seoul, Korea {sohn1,c142520,jheo}@yonsei.ac.kr

Abstract. This paper presents an effective strategy to extract the buildings and to reconstruct 3-D buildings using high-resolution multispectral stereo satellite images. Proposed scheme containes three major steps: building enhancement and segmentation using both Background Discriminant Transformation (BDT) and ISODATA algorithm, conjugate building identification using the object matching with Hausdorff distance and color indexing, and 3-D building reconstruction using photogrammetric techniques. IKONOS multispectral stereo images were used to evaluate the scheme. As a result, the BDT technique was verified as an effective tool for enhancing building areas since BDT suppressed the dominance of background to enhance the building as a non-background. In building recognition, color information itself was not enough to identify the conjugate building pairs since most buildings are composed of similar materials such as concrete. When both Hausdorff distance for edge information and color indexing for color information were combined, all segmented buildings in the stereo images were correctly identified. Finally, 3-D building models were successfully generated using the space intersection by the forward Rational Function Model (RFM).

1 Introduction 3-D building reconstruction in urban areas is one of the highlighted issues in photogrammetry and remote sensing. Generated 3-D building information can be used in various fields such as urban planning, disaster management, navigation system, and cyber city. In order to extract 3-D building information, various attempts have been performed using aerial images, satellite images, LIDAR data, Digital Surface Model (DSM), Digital Elevation Model (DEM), and Geospatial Information System (GIS) thematic maps. Aerial and satellite images are the primary data sets used in most researches. Conventional 3-D building reconstruction techniques are divided into image-based and model-based approaches. Image-based approach utilizes all possible extracted data from the images. Several attempts to extract and construct 3-D building models using the aerial imagery were investigated [6], [8]. These trials were failed to match the extracted buildings simultaneously since only edge information is not enough for successful matching. Subsequently, a system that detects and constructs 3-D models from a single image was developed [5]. In this case, shadow information was used to support the building hypotheses. However, shadow is not always available information. Although approaches using color information in satellite imagery were also performed [7], extraction method was limited only to color information. R. Khosla et al. (Eds.): KES 2005, LNAI 3683, pp. 62–68, 2005. © Springer-Verlag Berlin Heidelberg 2005

3-D Building Reconstruction Using IKONOS Multispectral Stereo Images

63

In model-based approach, buildings in the image are extracted using the prior building models. Therefore, model-based approach still requires interacting with the system and input information such as the approximate building position, the initial building model, and model parameters. Fischer and others [2] solved the building extraction problem using semi-automated approach. This has a deficiency in that users manually define the building model and find the building elements in one image. This paper presents an effective strategy to extract building and to establish 3-D building models using high-resolution multispectral stereo satellite images. Proposed scheme consists of three major processes.

2 Building Extraction and Recognition The detection and extraction of objects in images is often dependent on the suppression of the background. Also, this problem is of great importance in pattern recognition systems. The success of automated pattern recognition systems depends on the enhancement of significant features in relation to the irrelevant background information in the pre-processing stage. 2.1 Background Discriminant Transformation (BDT) A large variety of techniques are in use for enhancing significant features in images. The commonly used techniques are linear and non-linear stretching, histogram equalization, spatial filtering in univariate space, and linear transformation in multivariate space [9]. The BDT is one of the linear transformations for image enhancement. The BDT is designed to discriminate between the background and the nonbackground (for example, buildings of interest). One of the importance aspects of this technique is that it is scale-invariant. In the BDT technique, the image is assumed to have two classes: background and non-background. In order to enhance the non-background class, the axes in spectral space are to be rotated to reduce the background variability and to increase the nonbackground variability, much like in the Principal Component Transformation (PCT). In other words, BDT coefficients are computed to maximize the variance of the nonbackground relative to the background. The theoretical basis of this algorithm is well described in Carroll and others [1]. The BDT computes the same number of new calculated bands as the original multispectral bands. First band means that the ratio of variances of non-background class and background class is the maximum. This band shows the dominance of nonbackground over the background. Last band means that the ratio of variances is the minimum, and thus background class is dominant over the non-background class. After enhancing the original multispectral image using BDT, it is possible to segment the image into a limited number of clusters corresponding to non-background and background. In this study, the BDT is used to enhance building areas and ISODATA algorithm is used to segment buildings from the enhanced images.

64

Hong-Gyoo Sohn, Choung-Hwan Park, and Joon Heo

2.2 Hausdorff Distance and Color Indexing The operation of object matching consists of deciding if two objects observed in different scenes are identical. Object matching often plays important role in object recognition. In this study, the Hausdorff distance and color indexing value are used to compute matching scores in object matching. Given two finite point sets A = {a1 , ! , a p } and B = {b1 , ! , bq } , the Hausdorff distance is defined as [4]: D( A, B) = max(d ( A, B), d ( B, A))

where d ( A, B ) = max min a − b and ⋅ a∈ A

b∈B

(1)

is Euclidean norm on the points of A and

B . The function d ( A, B ) is called the direct Hausdorff distance from A to B . The

distance d ( A, B ) ≥ 0 and if d ( A, B ) = 0 , the two data sets are identical. The distance d ( A, B ) depends on the relative translation, rotation, and scale change between two data sets. The segmented objects of interest do not necessarily have the same position, orientation, and scale in each data set. Therefore, the two objects need to be registered before the distance between them can be computed. Final edge matching score between the two edge images E1 and E2 is then rewritten as: Dnew = D( E1T , E2 )

(2) T 1

where Dnew is new Hausdorff distance between E and E2 . Color indexing algorithm identifies an object by comparing its colors with the colors in other image. However, crucial point for this processing is that the total area covered by each color must be taken into account. The areas are computed and compared by histogramming the images and intersecting the histograms. A color histogram is three dimensional and simply represents the count of the number of pixels in image having a particular RGB value. Color histograms of images of every object are computed and stored. Presented with image of an unknown object, the color indexing algorithm computes its color histogram and intersects it with every one of the stored histograms in order to find the one that matched best. For two objects with color histograms, the color matching score is defined as [3]: C=

∑∑∑ r

g

b

min( H1 ( R, G, B ), H 2 ( R, G, B)) min(| H1 |, | H 2 |)

(3)

where H i ( R, G, B ) is the color histogram of i th object. 0 ≤ C ≤ 1 and if C = 1 , one of the histograms is totally included in the other one.

3 3-D Building Reconstruction In order to generate 3-D building model, 3-D position information are essential. 3-D position of buildings can be calculated using photogrammetric techniques. 3-D positioning is performed with conjugate points in stereo images and geometric model expressed relationships between image space and ground space.

3-D Building Reconstruction Using IKONOS Multispectral Stereo Images

65

For acquiring accurate conjugate points, least squares image matching technique is introduced. Least squares image matching is applied along the edges of identified building pairs. For calculating 3-D position of buildings, forward Rational Function Model (RFM) is used. RFM, one of the replacement sensor models, has been extensively used in IKONOS imagery. The RFM defines the relation between the image space and ground space in the form of polynomial ratios. r=

p (ϕ , λ , h) p1 (ϕ , λ , h) , c= 3 p2 (ϕ , λ , h) p4 (ϕ , λ , h)

(4)

where r and c are the normalized row and column indices of pixels in the image space and ϕ , λ , and h are the normalized coordinates in the ground space. The detailed algorithm about space intersection by RFM is presented in Sohn and others [10].

4 Experiments and Results To test the proposed 3-D building reconstruction scheme, IKONOS multispectral stereo images taken in February 7, 2000 (4 bands/1 m resolution) were used. The stereo pairs cover the San Diego City, U.S.A. as shown in Figure 1. The test area highlighted in Figure 1 was selected for 3-D building reconstruction. Unlike conventional satellite, which takes cross-track stereo images from different orbital passes, IKONOS collects same pass stereo pairs. That is the two images constituting the stereo pair are taken on the same orbital pass. Stereo pairs used in this study are scanned at reverse direction. The nominal elevation angle of each image is 62.1° and 64.6°. Also, it is appropriate to test proposed algorithms since stereo images contain various man-made features and natural topography.

Fig. 1. IKONOS multispectral stereo images and test area

For building enhancement and segmentation, five training sites for background class are collected in stereo images. Mean vector and covariance matrices for background class and those of whole image are calculated. After enhancing images using the BDT technique, buildings are segmented using ISODATA algorithm. Figure 2 shows extracted buildings in the stereo images. Total 34 buildings in left image are extracted, while 36 buildings in right image are extracted. However, a small group of buildings are not extracted since some buildings are obscured by the shadow of neighboring buildings.

66

Hong-Gyoo Sohn, Choung-Hwan Park, and Joon Heo

Extracted buildings first have to be identified as conjugate building pairs before performing point matching for accurate conjugate points. Especially, building recognition is effective when extracted buildings have not same number in each image. Object matching algorithm using Hausdorff and color indexing technique is used for building recognition. From a practical point of view, this approach has an advantage that it does not need an additional process such as epipolar image resampling. To combine the color matching score and edge matching score into a single matching score, it should be noted that the color matching score is a similarity index while the edge matching score is dissimilarity index.

Fig. 2. Extracted buildings in test area (left: 34 buildings. right: 36 buildings)

Therefore, we have chosen the following equation to compute the total matching score as follows: m m T = Pcolor ⋅ C − Pedge ⋅D (6) where T is the total matching score, C and D are color indexing score and Hausm m and Pedge are ratios of successful matched objects over the dorff distance, and Pcolor candidate matching objects. Table 1 summarizes the results of building recognition when both matching scores are combined. R a is the average rank in actual pairs, R m is the maximum rank in actual pairs, M t (or M f ) is the number of true (or false) matches. Table 1. Building recognition results using the object matching technique

Edge Color Edge+Color

Ra

Rm

Mt

Mf

P m (%)

1.1 19.8 1.0

2 32 1

32 4 34

2 30 0

94.1 11.8 100

As shown in Table 1, the accuracy of building recognition using only color information is considerably low. The result may be caused by the similar reflectance characteristics of the building. In case when buildings are composed of materials such as concrete, they generally have the homogeneous spectral characteristics. However, all buildings are correctly recognized when both edge information and color information are combined.

3-D Building Reconstruction Using IKONOS Multispectral Stereo Images

67

Once the initial position for point matching is defined by building recognition process, least squares image matching is further performed to obtain the accurate conjugate points. Total 34 corresponding building pairs in stereo images were successfully matched. The final 34 building pairs, which contain 4,892 conjugate points, were used to calculate 3-D position by the forward RFM. Calculated 3-D coordinates have geodetic coordinates on WGS-84 ellipsoid. Therefore, additional coordinate conversion to UTM is followed. Final 3-D building reconstruction results are overlaid on geocoded reference image as shown in Figure 3. In Figure 3, the calculated heights of buildings were three-times vertically exaggerated.

Fig. 3. 3-D building reconstruction results of the test area

5 Conclusions Automatic or semiautomatic techniques for building extraction and 3-D building reconstruction have evolved recently. They showed great potential for generating 3-D building models. In this paper, we presented a new method for 3-D building reconstruction using high-resolution multispectral stereo images. Our scheme focused on finding solution about following question: the effect of color information in 3-D building reconstruction. As a result, color information provided a useful tool for building extraction. Background and non-background such as building can be surely separated in multispectral imagery. Especially, the BDT technique based on spectral characteristics of objects is suitable for multispectral imagery. However, color information in building recognition did not provide satisfactory result since buildings are often composed of almost homogeneous materials, which causes similar spectral characteristics in resulting images. It was also confirmed that IKONOS stereo images, which adopt replacement sensor model as a basic sensor model, are suitable for the 3-D building reconstruction. Our approach can be extended to generate 3-D building model using other multi-source data sets.

68

Hong-Gyoo Sohn, Choung-Hwan Park, and Joon Heo

References 1. Carroll, J. D., Green, P. E., and Chaturvedi, A.: Mathematical tools for applied multivariate analysis, Elsevier Science, Revised edition, (1997) 259-294 2. Fischer, A., Kolbe, T. H., and Lang, F.: On the use of geometric and semantic models for component-based building reconstruction proceedings, SMATI (Semantic Modeling for the Acquisition of Topographic Information from Images and Maps) Workshop, (1999) 101119 3. Funt, B. V., Finlayson, G. D.: Color constant color indexing, IEEE Transaction on Pattern Analysis and Machine Intelligence, 17(5), (1995) 522-529 4. Huttenlocher, D. P., Klanderman, G. A., and Rucklidge, W. J.: Comparing images using the hausdorff distance, IEEE Transaction on Pattern Analysis and Machine Intelligence, 15(9), (1993) 850-863 5. Lin, C., Nevatia, R.: Building detection and description from a single intensity image, Computer Vision and Image Understanding, 72(2), (1998) 101-121 6. Noronha, S., Nevatia, R.: Detection and description of buildings from multiple aerial images, IEEE Transaction on Pattern Analysis and Machine Intelligence, 23(5), (2001) 501518 7. Ormsby, J. P.: Evaluation of natural and man-made features using Landsat TM data, International Journal of Remote Sensing, 13, (1992) 303-318 8. Roux, M., McKeown, D. M.: Feature matching for building extraction from multiple view, IEEE Proceeding of Computer Vision and Pattern Recognition, (1994) 46-53 9. Shettigara, V. K.: Image enhancement using background discriminant transformation, International Journal of Remote Sensing, 12(10), (1991) 2153-2167 10. Sohn, H.-G., Park, C.-H., and Chang, H.: RFM-based image matching technique using object-space approach, Journal of Japan Society of Photogrammetry and Remote Sensing, 43(4), (2004) 4-12.