digital elevation model production by stereo ... - Semantic Scholar
DIGITAL ELEVATION MODEL PRODUCTION BY STEREO-MATCHING SPOT IMAGE-PAIRS: A COMPARISON OF ALGORITHMS. T Day J-P Muller Department of Photogrammetry and Surveying University College London Gower Street London WC1E 6BT, U.K. JANET: [email protected] Figure 1: Reference DEM
Three published stereo-matching algorithms have been implemented and tested on SPOT images of an area for which we have an accurate digital elevation model (DEM) available with 30m spacing. We present the results of comparison of stereo-matcher output with the DEM, and examine the errors arising and their
1
INTRODUCTION
Alvey MMI-137 is concerned with producing accurate DEMs from sources such level 1A ("raw") SPOT image data within a couple of hours for a whole 6000 by 6000 pixel image pair using a transputer-based MIMD machine '. This paper presents an empirical comparison of the performance of three stereo-matching algorithms when applied to SPOT image data.
Intensity range image
Previous empirical assessments of quality of automated stereomatcher output have suffered from noise-correlated data * ', limited area data 4 or a-priori data of dubious quality * * T. Sub-pixel accuracy stereo-matching has already been demonstrated by many authors using quality assessment techniques based, on the use of isolated random check-points (see, for example, *). However, the potential of stereo-matchers to produce millions of point pairs implies that reliability and blunder rate (and the ease with which blunders can be filtered out) are also important factors.
2 2.1
TEST AREA Images
Lambertian shaded nadir view
Our test data consists of three SPOT images (scene number 502S2) of Aix-en-Provence in the South of France, provided as part of the SPOT-PEPS programme. Extracts from these are shown in Figure 2. While the vertical image is free of atmospheric effects, the left image (angle of incidence —17.5°) is affected by haze which acts as multiplicative noise and lowers the contrast range of features. The right image (+22.6°) contains several completely opaque clouds, but is clear away from these. All images are affected by horizontal and vertical striping originating in the pushbroom sensor and uncorrectable by SPOT-Image for these images. 2.2
and Otto tc Chau on conventional hardware by UCL's Department of Computer Science * 10 and Barnard and Thompson on transputers by the Royal Signals and Radar Establishment at Malvern »>). 3.1
This is an interest operator based matcher l a , using the Moravec operator " u > feature extractor. Since the Moravec operator does not locate features to sub-pixel accuracies, the algorithm is unable to resolve elevations in steps smaller than the height change due to one pixel disparity (e.g approximately 30m for the left and vertical image pair used below). However, the matched points obtained can be used as initial seed points for the Otto & Chau algorithm.
Independent Reference Data
We also have a 12.42km by 6.9km DEM, with 30m spacing, of a region (Montagne Sainte Victoire) within the area covered by the images. The range of elevations is 191.71m to 1010.99m. This was produced within the department by manual photogrammetric measurement of spot heights from aerial photographs with much higher resolution than SPOT. The DEM is unusual in that the operator measured the top of the tree canopy, where it was present, rather than attempting to measure the underlying ground level; this makes it more directly comparable with stereo-matcher output. By repeated measurement of several blocks, the accuracy of the DEM (i.e the standard deviation from the unknown groundtruth) was estimated as 1.3m. Comparison with a lower resolution DEM of the area, supplied by IGN, indicated no systematic offset.
3
Barnard and Thompson
3.2
PMF
This is an edge-based matcher developed at Sheffield University as part of Alvey IKBS-02S (3D surface representations and 3D model based vision from stereo) l*. It operates only along scanlines and therefore requires epipolar images as input. However, SPOT images are generally rotated at different angles as a function of scanline; we currently use an affine transformation to warp to near-epipolar. In practice it appears we cannot resample to true epipolar without iterative adjustment '°.
STEREO-MATCHERS TESTED
Software to produce epipolar images when supplied with a DEM has recently been written IS , but the PMF algorithm has not yet been applied to these due to difficulties associated with the afore-
All the stereo-matchers used are adaptations of existing published algorithms, and were implemented by project collaborators (PMF
AVC 1988 doi:10.5244/C.2.18
117
mentioned striping effects. S.S
Figure 2: Extracts from left, vertical and right SPOT images showing the location of the 30m DEM. Images copyright CNES.
Otto tc Chau
This stereo matcher is based on the Gruen adaptive least-squares correlator (ALSO) 1T, claimed to be of extremely high accuracy (approximately 0.0B pixels, based on figures reported for aerial photography *). The correlator can only refine an initial estimate of the disparity at a point, and has a limited pull-in range (on the order of two pixels). However, it also produces shaping information which is used in the Otto & Chau algorithm to estimate the disparity locally and so the matcher can sheet-grow out from a few initial seed-points ". These could be obtained by manual measurement, or from some other stereo matcher.
4
QUALITY ASSESSMENT
Since we are comparing our stereo-matcher output for SPOT with (assumed accurate) manual measurements of aerial photography, it must be stressed that the results given below will also include errors due to the camera model (which transforms SPOT image co-ordinates to and from ground co-ordinates). We are therefore testing the accuracy of our stereo-matching system as a whole, and not just the stereo-matching component. 4.1
Disparity-space analysis
To evaluate stereo-matcher output in image co-ordinates we first transform the DEM to a 'digital disparity model' (DDM) using an appropriate camera model 1*. The DDM takes the form of a list of corresponding points in each image (thus there is a different DDM for each image pair). Given a stereo-matcher output point in one image of the pair (usually the closest to nadir, since this contains the most even distribution) we compare the stereomatcher derived disparity vector with the disparity predicted from the manual measurements.
Left image
4.2
Analysis in ground co-ordinates
Here we consider the quality of stereo-matcher output transformed to ground co-ordinates through the camera model. We examine both the error statistics of the raw transformed points, and of a regular gridded DEM interpolated from them. In the first case we simply compare the elevation of a stereo matched point with the nearest reference point, provided one exists within a specified distance. Obviously we would like to make this distance small to minimise the additional error due to the variation of the terrain away from the manually measured reference points (we can estimate this error from the terrain's variogram 3 0 ), but this will in turn reduce the number of points we are evaluating. A problem with previous methods is that they only assess the quality of the points the stereo-matcher chose to output. If the points are unevenly distributed, a gridded DEM produced from those points could be of significantly lower quality than indicated by the previous method. So we also interpolate the irregularly spaced stereo-matcher output to the same grid as the 30m DEM, using Laserscan's 'Matrix' package; the quality of the stereo-matcher derived DEM is then assessed by comparing coinciding points.
5
RESULTS
To compare the performance of each stereo-matcher implemented we have applied each to the images in Figure 2. S.I
Barnard and Thompson
Barnard and Thompson produces relatively sparse output; 115 points from 240 by 240 pixel extracts taken from the left and vertical images, compared with an expected 560 from filtered PMF and 1800 from S pixel spaced Otto & Chau. We are currently unable to test this algorithm on larger images due to it being implemented on transputers virtual storage. Since it does not produce sub-pixel disparities or sufficient density for DEM generation we intend to use it only as a source of seed-points for the sheetgrowing Otto & Chau stereo-matcher (and so we give no ground co-ordinate results).
118
From Table 1 it can be seen that the majority of the points are accurate to within 2 pixels and suitable for use as seed-points for the Otto & Chau algorithm (see below). The Chau & Otto algorithm itself filters bad seed-points and corrects those within the pull-in range.
Table 2: PMF disparity errors Filtered PMF disparity errors Number of points 2950 Line error mean m -2.20 Line error S.D Sv
8380
28053
2675
2903 3.80m 46.19m 45.35m 548.70m -669.77m 1.24%
9794 -0.09m 11.24m 11.24m 126.42m -115.05m 1.54%
930 -0.07r 12.71r 12.71T
547.81 r -347.94» 3.035
Table 5: Error statistics of stereo-matcher derived DEMs (95865 points)
PMF Mean S.D R.M.S Max. Min. | error — /i |> 3
Three published stereo-matching algorithms have been implemented and tested on SPOT images of an area for which we have an accurate digital elevation model (DEM) available with 30m spacing. We present the results of comparison of stereo-matcher output with the DEM, and examine the errors arising and their
1
INTRODUCTION
Alvey MMI-137 is concerned with producing accurate DEMs from sources such level 1A ("raw") SPOT image data within a couple of hours for a whole 6000 by 6000 pixel image pair using a transputer-based MIMD machine '. This paper presents an empirical comparison of the performance of three stereo-matching algorithms when applied to SPOT image data.
Intensity range image
Previous empirical assessments of quality of automated stereomatcher output have suffered from noise-correlated data * ', limited area data 4 or a-priori data of dubious quality * * T. Sub-pixel accuracy stereo-matching has already been demonstrated by many authors using quality assessment techniques based, on the use of isolated random check-points (see, for example, *). However, the potential of stereo-matchers to produce millions of point pairs implies that reliability and blunder rate (and the ease with which blunders can be filtered out) are also important factors.
2 2.1
TEST AREA Images
Lambertian shaded nadir view
Our test data consists of three SPOT images (scene number 502S2) of Aix-en-Provence in the South of France, provided as part of the SPOT-PEPS programme. Extracts from these are shown in Figure 2. While the vertical image is free of atmospheric effects, the left image (angle of incidence —17.5°) is affected by haze which acts as multiplicative noise and lowers the contrast range of features. The right image (+22.6°) contains several completely opaque clouds, but is clear away from these. All images are affected by horizontal and vertical striping originating in the pushbroom sensor and uncorrectable by SPOT-Image for these images. 2.2
and Otto tc Chau on conventional hardware by UCL's Department of Computer Science * 10 and Barnard and Thompson on transputers by the Royal Signals and Radar Establishment at Malvern »>). 3.1
This is an interest operator based matcher l a , using the Moravec operator " u > feature extractor. Since the Moravec operator does not locate features to sub-pixel accuracies, the algorithm is unable to resolve elevations in steps smaller than the height change due to one pixel disparity (e.g approximately 30m for the left and vertical image pair used below). However, the matched points obtained can be used as initial seed points for the Otto & Chau algorithm.
Independent Reference Data
We also have a 12.42km by 6.9km DEM, with 30m spacing, of a region (Montagne Sainte Victoire) within the area covered by the images. The range of elevations is 191.71m to 1010.99m. This was produced within the department by manual photogrammetric measurement of spot heights from aerial photographs with much higher resolution than SPOT. The DEM is unusual in that the operator measured the top of the tree canopy, where it was present, rather than attempting to measure the underlying ground level; this makes it more directly comparable with stereo-matcher output. By repeated measurement of several blocks, the accuracy of the DEM (i.e the standard deviation from the unknown groundtruth) was estimated as 1.3m. Comparison with a lower resolution DEM of the area, supplied by IGN, indicated no systematic offset.
3
Barnard and Thompson
3.2
PMF
This is an edge-based matcher developed at Sheffield University as part of Alvey IKBS-02S (3D surface representations and 3D model based vision from stereo) l*. It operates only along scanlines and therefore requires epipolar images as input. However, SPOT images are generally rotated at different angles as a function of scanline; we currently use an affine transformation to warp to near-epipolar. In practice it appears we cannot resample to true epipolar without iterative adjustment '°.
STEREO-MATCHERS TESTED
Software to produce epipolar images when supplied with a DEM has recently been written IS , but the PMF algorithm has not yet been applied to these due to difficulties associated with the afore-
All the stereo-matchers used are adaptations of existing published algorithms, and were implemented by project collaborators (PMF
AVC 1988 doi:10.5244/C.2.18
117
mentioned striping effects. S.S
Figure 2: Extracts from left, vertical and right SPOT images showing the location of the 30m DEM. Images copyright CNES.
Otto tc Chau
This stereo matcher is based on the Gruen adaptive least-squares correlator (ALSO) 1T, claimed to be of extremely high accuracy (approximately 0.0B pixels, based on figures reported for aerial photography *). The correlator can only refine an initial estimate of the disparity at a point, and has a limited pull-in range (on the order of two pixels). However, it also produces shaping information which is used in the Otto & Chau algorithm to estimate the disparity locally and so the matcher can sheet-grow out from a few initial seed-points ". These could be obtained by manual measurement, or from some other stereo matcher.
4
QUALITY ASSESSMENT
Since we are comparing our stereo-matcher output for SPOT with (assumed accurate) manual measurements of aerial photography, it must be stressed that the results given below will also include errors due to the camera model (which transforms SPOT image co-ordinates to and from ground co-ordinates). We are therefore testing the accuracy of our stereo-matching system as a whole, and not just the stereo-matching component. 4.1
Disparity-space analysis
To evaluate stereo-matcher output in image co-ordinates we first transform the DEM to a 'digital disparity model' (DDM) using an appropriate camera model 1*. The DDM takes the form of a list of corresponding points in each image (thus there is a different DDM for each image pair). Given a stereo-matcher output point in one image of the pair (usually the closest to nadir, since this contains the most even distribution) we compare the stereomatcher derived disparity vector with the disparity predicted from the manual measurements.
Left image
4.2
Analysis in ground co-ordinates
Here we consider the quality of stereo-matcher output transformed to ground co-ordinates through the camera model. We examine both the error statistics of the raw transformed points, and of a regular gridded DEM interpolated from them. In the first case we simply compare the elevation of a stereo matched point with the nearest reference point, provided one exists within a specified distance. Obviously we would like to make this distance small to minimise the additional error due to the variation of the terrain away from the manually measured reference points (we can estimate this error from the terrain's variogram 3 0 ), but this will in turn reduce the number of points we are evaluating. A problem with previous methods is that they only assess the quality of the points the stereo-matcher chose to output. If the points are unevenly distributed, a gridded DEM produced from those points could be of significantly lower quality than indicated by the previous method. So we also interpolate the irregularly spaced stereo-matcher output to the same grid as the 30m DEM, using Laserscan's 'Matrix' package; the quality of the stereo-matcher derived DEM is then assessed by comparing coinciding points.
5
RESULTS
To compare the performance of each stereo-matcher implemented we have applied each to the images in Figure 2. S.I
Barnard and Thompson
Barnard and Thompson produces relatively sparse output; 115 points from 240 by 240 pixel extracts taken from the left and vertical images, compared with an expected 560 from filtered PMF and 1800 from S pixel spaced Otto & Chau. We are currently unable to test this algorithm on larger images due to it being implemented on transputers virtual storage. Since it does not produce sub-pixel disparities or sufficient density for DEM generation we intend to use it only as a source of seed-points for the sheetgrowing Otto & Chau stereo-matcher (and so we give no ground co-ordinate results).
118
From Table 1 it can be seen that the majority of the points are accurate to within 2 pixels and suitable for use as seed-points for the Otto & Chau algorithm (see below). The Chau & Otto algorithm itself filters bad seed-points and corrects those within the pull-in range.
Table 2: PMF disparity errors Filtered PMF disparity errors Number of points 2950 Line error mean m -2.20 Line error S.D Sv
8380
28053
2675
2903 3.80m 46.19m 45.35m 548.70m -669.77m 1.24%
9794 -0.09m 11.24m 11.24m 126.42m -115.05m 1.54%
930 -0.07r 12.71r 12.71T
547.81 r -347.94» 3.035
Table 5: Error statistics of stereo-matcher derived DEMs (95865 points)
PMF Mean S.D R.M.S Max. Min. | error — /i |> 3
