Automatic detection of oil spills in ERS SAR images - Semantic Scholar
1916
IEEE TRANSACTIONS ON GEOSCIENCE AND REMOTE SENSING, VOL. 37, NO. 4, JULY 1999
Automatic Detection of Oil Spills in ERS SAR Images Anne H. Schistad Solberg, Member, IEEE, Geir Storvik, Rune Solberg, and Espen Volden
Abstract— We present algorithms for the automatic detection of oil spills in SAR images. The developed framework consists of first detecting dark spots in the image, then computing a set of features for each dark spot, before the spot is classified as either an oil slick or a “lookalike” (other oceanographic phenomena which resemble oil slicks). The classification rule is constructed by combining statistical modeling with a rule-based approach. Prior knowledge about the higher probability for the presence of oil slicks around ships and oil platforms is incorporated into the model. In addition, knowledge about the external conditions like wind level and slick surroundings are taken into account. The presented algorithms are tested on 84 SAR images. The algorithm can discriminate between oil slicks and lookalikes with high accuracy. 94% of the oil slicks and 99% of the lookalikes were correctly classified. Index Terms—Bayesian image classification, modeling of prior knowledge, oil spill detection, SAR image analysis.
I. INTRODUCTION
A
PROJECT for use of ERS-1 SAR data for the detection of marine oil spills was initiated in 1991 as part of the Norwegian Space Centre’s national ERS-1 program. Based on this activity, a near real-time service by Tromsø Satellite Station (TSS) has been established according to requirements set by the Norwegian Pollution Control Authority. Today’s manual service is a first step toward a fully operational system covering Norwegian and adjacent waters. The present study, which aims at determining the potential for automatic oil spill classification, is part of this project. Being able to use data from TSS collected regularly since 1994, we can present experimental results involving a large number of SAR scenes. Automatic identification of oil spills in SAR images is a very complex task because objects which resemble oil spills (in this paper, called “lookalikes”) frequently occur, particularly in low-wind conditions. Examples of oil spill lookalikes are organic film, grease ice, wind front areas, areas sheltered by land, rain cells, current shear zones, internal waves, and upwelling zones [1]. In some cases, even an experienced operator cannot determine if a slick actually is an oil slick or a lookalike. The manual oil spill service at TSS assigns possible oil spills into three categories representing slicks with high, medium, or low probability of being oil. This information Manuscript received July 28, 1998. This work was supported by Tromsø Satellite Station (TSS). A. H. Schistad Solberg, R. Solberg, and E. Volden are with the Norwegian Computing Center, 0314 Oslo, Norway (e-mail: [email protected]). G. Storvik is with the Norwegian Computing Center, 0314 Oslo, Norway and the Department of Mathematics, University of Oslo, 0316 Oslo, Norway. Publisher Item Identifier S 0196-2892(99)03483-X.
Fig. 1. Oil spill detection system.
is forwarded to the Norwegian Pollution Control Authority, which can use a surveillance aircraft to verify it. A trained human interpreter is able to discriminate between oil slicks and lookalikes based on experience and prior information concerning location, external information about weather conditions, differences in shape and contrast to surroundings between oil slicks and lookalikes, etc. To obtain the same results for an automatic slick classification, these aspects must be incorporated into the classifier. This poses a challenge in designing a classification algorithm. In this paper, we discuss the most important factors when discriminating between oil slicks and their lookalikes, and we present a classification algorithm which incorporates many of these factors. Our goal is to develop a semiautomatic system for oil spill detection, in which objects with a high probability of being oil slicks are automatically identified. These possible oil slicks are then presented to an operator for manual inspection. Presently, we do not believe that a fully automatic oil spill detection algorithm will be able to correctly discriminate between the most difficult oil slicks and their lookalikes. However, a semiautomatic procedure will greatly reduce the number of SAR images which need to be manually inspected compared to a fully manual detection procedure. A framework for automatic detection with the following main elements (see Fig. 1) is
0196–2892/99$10.00 1999 IEEE
SCHISTAD SOLBERG et al.: AUTOMATIC DETECTION OF OIL SPILLS
presented: 1) detection of dark spots, 2) spot feature extraction, and 3) dark spot classification. We illustrate the importance of incorporating prior knowledge by comparing the performance of the classification rules obtained without and with prior distributions and a classifier based on both prior distributions and rule-based corrections of the probability densities for the observed features. Only a small number of studies aiming at automatic oil spill detection have reported results on more than a few images. Weisteen et al. [2] describe our initial work on the problem on a very limited data set. Wahl et al. [3], [4] included an algorithm for automatic detection of dark spots which could possibly be oil spills in their ship detection system. A tree classifier for oil spill classification is presented in Kubat et al. [5], but only the classification part, without specifying the feature extraction part. Using a leave-one-image-out strategy on nine SAR images, they correctly classify 78% of the oil spills and 50% of the lookalikes. The remainder of the paper is organized as follows. Section II describes SAR imaging of oil spills and lookalikes, Section III the algorithm for detection of dark spots, and Section IV the feature extraction process. The classification problem and the modeling are discussed in detail in Section V, while Section VI contains the experimental results. Section VII presents a discussion and conclusions.
II. SAR IMAGING
OF
OIL SPILLS
A number of researchers have described SAR imaging of oil spills or lookalikes. Oil usually appears as dark spots in SAR images because it has a dampening effect on the Bragg waves. Wahl et al. [6] give an overview of radar satellites in terms of their oil spill detection capabilities. Bern et al. [7] describe SAR oil spill imaging in relation to wind speed. Wismann [8] has studied the radar return of more than 150 oil spills. These studies reach similar conclusions regarding the visibility of oil slicks in ERS SAR imagery. Hovland et al. [1] studied different kinds of lookalikes, and proposed a simple decision tree for discriminating between oil spills and lookalikes. The following description relies heavily on other studies performed as part of the Norwegian Oil Spill project (see [6] and the references therein). The SAR signature of an oil spill will depend on the external conditions. The contrast between the spill and its surroundings depends on a number of parameters like wind speed, wave height, and the amount and type of oil released. The shape of the spill will depend on whether the oil was released from a stationary object or from a moving ship, the amount of oil involved, and the wind and current history between the release and the image acquisition. When discriminating between oil slicks and lookalikes, we think that the following factors are the most important to model. • Known objects and detected point sources: If a bright object (ship or oil platform) is seen close to a spot, there is a higher probability for the presence of oil slicks. The North Sea, which has a large number of oil platforms, is part of the oil spill coverage area for Tromsø Satellite
1917
Station. There are also many examples of oil slicks directly connected to a ship in the test data used. • Wind and probability for lookalikes: One will often observe many lookalikes with low-wind conditions. With moderate and high wind, the number of lookalikes will be small. • Wind and spot detection: An oil slick will have a large contrast to the surroundings in low-wind conditions. With moderate wind, the oil will have smaller contrast to the surroundings. • Slick surroundings: The human eye is superior in observing a slick in the context of the surrounding sea. If the surroundings are homogeneous, the human observer will have more belief in that the spot is an oil slick than with heterogeneous surroundings. With heterogeneous surroundings, the human eye can easily determine if the spot is separated from the surroundings based on contrast or orientation. III. SPOT DETECTION The algorithm for detection of dark spots is based on adaptive thresholding. This thresholding is based on an estimate of the typical backscatter level in a large window. The adaptive dB below the estimated local mean threshold is set to pixels) is backscatter level. The window (of size moved across the image in small steps to threshold all pixels in the scene. Wind data (the wind level) is used to determine . Currently, the wind level is set manually as one of four categories: low, low/medium, medium, or high wind. This will be replaced by automatic methods for wind estimation based on the SAR image. As part of the Norwegian oil spill project, NORUT-IT are currently developing algorithms for estimation of wind speed and direction from ERS SAR images, and these algorithms will be used when the system is operated at TSS. This dark spot detection procedure did not always define the correct border between the oil slick and the surrounding sea when the surroundings were heterogeneous (particularly in low-wind conditions). Parts of the surroundings are sometimes included in the detected spot. A clustering step is used to avoid this. After spot detection, each spot is clustered into two clusters. The idea is that if the spot includes part of its surroundings, the oil slick will consist of the darkest cluster. If the two clusters are sufficiently separated and the darkest cluster is sufficiently large compared to the brightest, the darkest cluster is used as the spot; otherwise, the original spot is kept. This algorithm works well on most of the 84 images that we have tested it on. The algorithm detected all of the oil spills in the test data set. In certain cases, a thin, linear oil spill was detected as several smaller spots because the contrast varies within the spill. IV. SLICK FEATURE EXTRACTION For each region corresponding to a detected spot, a set of features is computed. The features constitute standard descriptors applied for regions in general image analysis
1918
IEEE TRANSACTIONS ON GEOSCIENCE AND REMOTE SENSING, VOL. 37, NO. 4, JULY 1999
applications, and some additional features particularly suited for oil spill detection. The following features are used. 1) Distance to a point source: A simple algorithm searches for bright points in the image, and the distance from a region corresponding to a dark spot to the closest bright object is computed. 2) Number of detected spots in the scene. 3) Number of neighboring spots in a larger window centered at the region. 4) Homogeneity (power-to-mean ratio) of the surroundings. , where is 5) Slick complexity is defined by the perimeter and is the area of the region. 6) Slick width is the ratio between the area of the region and the width of the branches of the skeleton of the region. 7) Slick area is the size (in number of pixels) of the region. 8) First invariant planar moment [9], which describes the shape of the region. , where 9) Local area contrast ratio is defined as is the local backscatter mean computed in a window is the mean backscatter centered at the region, and value of the region. 10) Border gradient is the mean of the magnitude of gradient values of the region border area. The Sobel operator is used to compute the gradients. , 11) Power-to-mean ratio of the slick, defined as and are the standard deviation and the mean where value of the slick, respectively. Feature 1) is actually prior information, and not observed data in our modeling assumptions (see Section V-A). The next three features are measures of the surroundings of the spot. Features 5)–8) measure the shape of the spot, while the last three features are measures of the actual backscatter values of the spot. V. SLICK CLASSIFICATION After spot detection and feature extraction, we have a set dark spots or regions which we want to classify as of oil slicks or lookalikes. For each region, we have a feature describing the shape, contrast, surroundings, etc., vector of the region. In addition, we have prior information about the likelihood of observing oil slicks and external information about wind level which will influence the observed features. The nature of the oil spill classification problem poses a set of challenges which needs to be handled during the design of the classifier. It is a problem in which context is extremely important. The classification algorithm must try to resemble the way a trained human operator views a possible oil slick in relation to the whole scene, and his knowledge about the weather conditions and oil spills in general. The need for controling how some important features are used during classification made us reject the automatically generated tree classifier used in a preliminary version of the system [10]. Instead, we have decided to use a classifier which combines a prior model for the number of lookalikes in a scene, a model for the presence of a slick in the vicinity of a bright object, a standard probability density for the observed features, and a rule-based modification of the probability density to take into
account combinations of features which are indications of a certain scene condition. A. Statistical Modeling The goal is to design a classifier which incorporates both prior information we can derive about the scene and the information contained in the observed features for the detected dark spots. In statistical classification, there are two main ingredients which need to be specified, the prior model and the conditional probability density for the features. In standard classification applications, one would specify a prior probability for an object being an oil slick or not. In our case, the number of objects to be classified is actually stochastic, implying that the standard approach is not the obvious choice. A further complication is that the presence of ships and oil platforms in the scene influences the presence of oil slicks, and that the wind level affects the number of lookalikes. In spite of these complications, it turns out that we are able to specify a (pseudo) prior for the probability of a detected spot being an oil slick (see the next subsection). We will , emphasizing the dependence on denote this prior by the information about the presence of ships and oil platforms which we assume is given through the vector . The second part to be specified is the probability density and for the observed features in classes oil slick and lookalikes, respectively. Note the dependence on the external information which will be used for describing the changes in the distribution of the feature values as a function of the wind level . The prior distribution and the probability density for the features are combined to obtain the posterior probability for a detected spot being an oil slick through Bayes theorem. Let be the unknown class membership of a detected spot. Then Pr
(1)
We want the classifier to find all spots with a certain probability of being oil. All spots classified as oil are then inspected by an operator. Misclassifying oil as lookalikes is considered more serious than misclassifying lookalikes as oil. This can be modeled in terms of a simple loss function. Let be the loss associated with misclassifying a true oil slick as a lookalike, and the loss associated with misclassifying a true lookalike as an oil slick. Then the optimal classification for spot is given by if Pr otherwise. B. Specification of “Prior” Probabilities Prior distributions for the presence of oil slicks can, in its simplest form, be specified through the number of oil slicks
SCHISTAD SOLBERG et al.: AUTOMATIC DETECTION OF OIL SPILLS
1919
in the scene. In particular, we will assume that the probability of oil slicks in the scene is given by (2)
TABLE I THE EXPECTED NUMBER OF LOOKALIKES IS GIVEN BY 1=(1 w ) WHERE w IS THE CORRESPONDING PARAMETER IN THE GEOMETRIC DISTRIBUTION FOR THE NUMBER OF LOOKALIKES
0
Wind level
is a parameter describing the expected number of where oil slicks in the scene. The presence of ships or oil platforms may increase the number of oil slicks expected in the area. Such ships or platforms can be seen as bright spots in the SAR images. Although there always will be some uncertainty involved, we assume that the positions of such objects can be estimated with sufficient accuracy. We can then construct a prior distribution for the presence of oil slicks in the neighborhood of the bright spot positions. Consider oil slicks in a scene with distances to the nearest bright spot. We then assume that the prior distribution (2) is modified to
where if if otherwise. This prior is not directly usable in order to calculate the posterior probability for a single detected spot being an oil slick or not. We need a probability statement about a detected spot prior to taking the observed features into account. In order to do this, the distribution of the number of lookalikes in a scene has to be handled. the number of detected spots. As previously Denote by noted, we assume that all of the present oil slicks will be , the number of oil slicks, the detected as dark spots. Given , the number of lookalikes stochastic variable is that will appear in the scene. We will assume that Pr that is, a geometric distribution with parameter depending on the wind level (given by the dependence on ). In particular, we have assumed that the wind level is divided into four categories. These categories are given in Table I together with . Based on these their corresponding values of assumptions, it is shown in the Appendix that Pr
which we will use as our “prior” an oil slick.
for an object being
C. Specification of Class-Conditional Probability Densities For each of the detected dark spots, a number of features are calculated, collected in the feature vector . These features are constructed such that they typically will be different, depending on whether the dark spot is an oil slick or a lookalike. However, a further complication is that the behavior
Low Low-to-moderate Moderate High
Expected no. lookalikes 10 000 100 20 7
of these features will change with different wind levels. Denote the wind level by . The simplest choice of densities is multivariate Gaussian ones. Even within each wind level, both the oil slicks and the lookalikes may vary quite a lot in shape and other features. Describing the feature density by a unimodal density such as the Gaussian is therefore not appropriate. Instead, we have assumed different densities, depending on the value of a shape descriptor (we have used the first invariant planar moment). was divided into four subgroups, The sample space of ranging from wide spots with regular shape to wide slicks with a complex boundary to thin, linear spots. The density for class and wind level is then given by if is in subgroup . Finally, the densities inside each subgroup are assumed Gaussian:
where , is the number of features, is and shape descriptor in the mean vector under wind level is a diagonal covariance subgroup for class , and . Only the matrix, common for both classes given features slick complexity, slick contrast ratio, border gradient, local smoothness contrast, homogeneity of surroundings, slick width, and number of neighboring spots are used to compute the probability density. The choice of making the covariance matrix equal for both the oil slick and the lookalike class is an important one. Lookalikes occur much more frequently than oil spills, and their features will vary much more than the features for oil slicks. If class-dependent covariance matrices are used, the variance of the lookalike class will be huge compared to the oil spill class. The resulting class-conditional probabilities for lookalikes will be of another magnitude than for oil, resulting in a large “bias” in the classification. A way of handling the very unbalanced data set is to avoid using a class-conditional covariance matrix. Another advantage of using a common covariance matrix is that the number of parameters involved will be reduced. Because of limited training data for oil slicks, a further reduction in the number of unknown parameters (which need to be estimated) is obtained by assuming the common covariance matrix to be diagonal. By doing so, we are ignoring the correlation between the features. D. Rule-Based Corrections of the Class-Conditional Densities Many combinations of features are known to give strong indications either toward an oil slick or toward a lookalike.
1920
IEEE TRANSACTIONS ON GEOSCIENCE AND REMOTE SENSING, VOL. 37, NO. 4, JULY 1999
Because of our need for using common diagonal covariance matrices in the Gaussian densities, knowledge of such combinations must be utilized in a different setting. Our choice has been to introduce a multiplicative factor adjusting the Gaussian densities [or actually the ratios between the densities of oil slick and lookalikes, see (1)] in a rule-based setting. The features listed below are used for this purpose. Many of these rules handle interactions between several features, which are indications of certain scene configurations. • Slick area: In low/low–moderate wind, we can have very large dark spots corresponding to large areas with no wind and almost no backscattering from the sea. If the area is very large, the ratio of densities is decreased (divided by a large factor). If the area is very small, the slick needs better contrast to be classified as oil; thus, the ratio of density is also decreased in this case. • Number of regions in a scene: If the number of detected spots in the scene is large, the scene is very complex, and most of the slicks will be lookalikes. The ratio of densities is decreased. If the number of regions is low, and the slick has homogeneous surroundings, the feature density for oil is increased. • Slick local contrast ratio: Used in combination with other features, the slick contrast affects the probability densities for the features. If the contrast is low for a slick with several neighboring regions in a scene with many regions, the ratio of densities is decreased. • Slick width: Used in combination with other features, the slick width affects the probability densities. For low-wind conditions, thick regions with large area, and not very high contrast the ratio of densities is decreased. : If is large, • Homogeneity of surroundings, the surroundings are heterogeneous. If, in addition, the number of regions in the scene and the number of neighboring regions are high, the ratio of densities is is low (homogeneous surrounding) decreased. If and the scene is not very complex, the ratio of densities is increased. Such modifications of the feature densities are not common in statistical classification. Rules like these are often used in knowledge-based classification methods, in which they are used to define boxes for a hard classification. We use them in combination with a statistical classifier. They allow the combination of several features, while a Gaussian classifier with a diagonal covariance matrix ignores the correlation between the features. The specification of the rules was performed by inspecting the results obtained by the classification only based on the Gaussian densities and the prior. Only a small subset of the images was used in order to avoid overfitting. VI. EXPERIMENTAL RESULTS The data set used to test the performance of the algorithm was designed to contain as many oil spills as possible, but also to include scenes containing only lookalikes. The images in the test set were selected by an experienced oil spill operator at Tromsø Satellite Station. The ground truth consisted of the
TABLE II DETECTION ACCURACIES. o IS THE NUMBER OF LOST OIL SPILLS, AND l IS THE NUMBER OF LOOKALIKES CLASSIFIED AS OILS SPILLS. MODEL 1 CORRESPONDS TO A CLASSIFICATION RULE BASED ON THE GAUSSIAN DENSITIES AND A PRIOR ONLY TAKING THE NUMBER OF OIL SPILLS AND THE NUMBER OF LOOKALIKES INTO ACCOUNT. MODEL 2 IS THE EXTENSION OF MODEL 1, ALSO TAKING THE DISTANCE TO A BRIGHT SPOT INTO ACCOUNT, WHILE MODEL 3 IS A FURTHER EXTENSION INCLUDING THE RULE-BASED CORRECTIONS OF THE DENSITIES
N
Model 1 Model 2 Model 3
N
N
o
19/71 11/71 4/71
N
l
310/6980 363/6980 75/6980
manual classification of each scene performed by the operators at TSS. Each scene was manually classified into the categories low, medium, or high probability of containing oil. The manual classification is used as ground truth, although we have no guarantee that the manual classification is correct. Out of a test set of 84 SAR scenes, 36 did not contain any oil spills, but were included to test the classifier on a high number of lookalikes. In addition, some scenes contained one or two oil spills and many (>100) lookalikes. The images were collected at TSS during a two-year period of operation of their manual oil spill detection service. The oil slicks belong to four main categories: 1) thin, linear slicks which might be caused by a ship or a stationary object releasing a small amount of oil; 2) wide, regular slicks caused by a stationary object releasing a larger amount of oil; 3) wide, irregular slicks for which the wind and/or current have altered the shape of the slick; 4) thin, piecewise linear slicks caused by a moving ship changing directions, or a thin slick altered by wind or current. The use of different densities depending on the value of our shape descriptor makes it possible to handle such a variety of cases automatically. To evaluate the performance of the classifier based on a SAR images, we used a leavedata set consisting of one-image-out strategy. We have trained the classifier (that is, estimation of the means and the covariance matrices in the images, then classified the Gaussian densities) on remaining image, and computed the number of correctly and incorrectly classified objects. The rule-based corrections were kept fixed all throughout the experiment. This procedure is times until all images have been classified. repeated It is illustrative to study the effect of the various elements in the classification model. Table II shows the results for three different classification rules. For the first model, only the distributions of the number of oil spills and the number of lookalikes are incorporated into the prior. Furthermore, Gaussian densities without the rule-based corrections are used. Most of the oil slicks in this case are correctly classified. However, a relatively high number of lookalikes is classified as oil slicks, although this number should also be compared to the number of lookalikes detected as dark spots (6980), showing that most of the lookalikes are discarded. If we incorporate the distance to a bright spot, the number of lost oil spills is reduced, although we obtain an increase
SCHISTAD SOLBERG et al.: AUTOMATIC DETECTION OF OIL SPILLS
Fig. 2. Oil slick on heterogeneous background. The region classified as oil is shown in the small frame (ESA/TSS/NR).
in the number of false alarms. Extending the model further by incorporating the rule-based corrections of the densities, both the number of lost oil spills and the number of false alarms were reduced significantly. The classification accuracy is so high that it is worthwhile to use the algorithms in an operational system. The rule-based adjustment of the densities is thus very important in reducing the number of false alarms, while also correctly identifying the oil slicks. Figs. 2–6 display the detection results in different types of scenes. A correctly classified slick on a very heterogeneous background is shown in Fig. 2. Fig. 3 shows a scene with several slicks connected to point sources. All of the slicks close to point sources were classified as oil, in addition to a small spot in the middle of the scene. Fig. 4 shows two correctly classified oil slicks connected to point sources, while Fig. 5 shows a complex scene with two lookalikes which were misclassified as oil. A complex scene with no alarms is shown in Fig. 6. VII. DISCUSSION
AND
CONCLUSIONS
A method for detecting oil slicks from SAR images has been presented. The method is based on a combination of using prior knowledge, Gaussian densities, and rule-based density corrections. The developed algorithms achieved a high accuracy on detecting oil spills. 94% (67 out of 71) of the oil slicks were correctly classified, while only 1% (75 out of 6980) of the lookalikes were wrongly classified. The system is currently being implemented at Tromsø Satellite Station. The advantage of using an automatic algorithm and not merely manual inspection of all possible SAR images depends on the number of SAR scenes to be analyzed and their complexity. Our test data set consisted of 84 images which all contained some objects with a certain probability of being oil. Manual inspection determined that 36 of these images contained only lookalikes, indicating that they should not cause alarms. 21 of these 36 images were automatically processed without causing any false alarms. Nine images contained alarms which were reasonable for a manual operator to have
1921
a closer look at, while the remaining six images resulted in false alarms (sending an image to manual inspection when the operator will determine that the alarm is false). The classifier had a tendency to classify many spots in medium wind with very homogeneous surroundings as oil. This number can probably be reduced by adjusting the parameters to reduce this effect. Our test data set did not contain any images with no possible oil objects, that is, scenes which are not classified into any of the categories low, medium, and high probability of oil at TSS. Such scenes will occur in an operational monitoring, but we do not know how frequent they are. This must be accounted for when determining the actual effect of an automatic algorithm versus manual inspection of all scenes. Our classification methods are based on formal statistical modeling, which also can be used as a foundation for further improvements. In particular, incorporating other types of prior knowledge and/or utilizing knowledge about the behavior of the dark spot features may further increase the accuracy. Because oil spills are relatively rare, we would like to design a system which can learn from experience when used operationally. If the system misclassifies a spot, as judged by the operator who examines all spots assigned to the oil category, the spot will be used to update the class description database when the system is used operationally.
CALCULATION
APPENDIX “PRIOR” PROBABILITIES
OF
In Section V, a prior distribution for oil slicks in a scene is introduced, together with a specification of the probability distribution for the number of lookalikes. We now show how we can combine these models in order to obtain a “prior” probability for a single detected spot being an oil slick. Since the probability distributions are defined through the total number of oils slicks and lookalikes, we need to work with the simultaneous distribution of all detected spots. The probability of interest is Pr
Pr
(3) Pr (We have Pr because of symmetry.) Furthermore, if we define
for all
Pr Pr
Pr
(4)
1922
IEEE TRANSACTIONS ON GEOSCIENCE AND REMOTE SENSING, VOL. 37, NO. 4, JULY 1999
Fig. 3. Several spots close to point sources classified as oil (ESA/TSS/NR).
Fig. 4. Two oil slicks connected to point sources (ESA/TSS/NR).
SCHISTAD SOLBERG et al.: AUTOMATIC DETECTION OF OIL SPILLS
Fig. 5. Two alarms in a very complex scene (ESA/TSS/NR).
Fig. 6. Complex scene with no alarms (ESA/TSS/NR).
1923
1924
IEEE TRANSACTIONS ON GEOSCIENCE AND REMOTE SENSING, VOL. 37, NO. 4, JULY 1999
where
Combining (4) with (3) results in
Anne H. Schistad Solberg (S’92–M’96) received the M.S. degree in computer science in 1989 and the Ph.D. degree in image analysis in 1995, both from the University of Oslo, Norway. She is currently a Senior Research Scientist at the Norwegian Computing Center, Oslo. During 1991–1992, she was a Visiting Scholar, Department of Computer Science, Michigan State University, East Lansing. Her research interests include statistical pattern recognition, multisensor image classification, texture analysis, and remote sensing.
Pr where
is given by
which is a constant not depending on . Using that Pr Pr , we end up with Pr
ACKNOWLEDGMENT The ERS SAR images were provided by TSS. The authors are grateful to T. Wahl (NDRE), H. Espedal (NERSC), H. Johnsen (NORUT-IT), J. Petter Pedersen, and T. Bauna (TSS) for sharing their knowledge about visually discriminating between oil slicks and lookalikes. REFERENCES [1] H. A. Hovland, J. A. Johannessen, and G. Digranes, “Slick detection in SAR images,” in Proc. IEEE Symp. Geosci. Remote Sensing (IGARSS), Pasadena, CA, Aug. 1994, pp. 2038–2040. [2] K. Weisteen, A. Solberg, and R. Solberg, “Detection of oil spills in SAR images using a statistical classification scheme,” in Proc. IEEE Symp. Geosci. Remote Sensing (IGARSS), Tokyo, Japan, Aug. 1993, pp. 943–945. [3] T. Wahl et al., “Oil spill detection using satellite based SAR, Phase 1b completion report,” Tech. Rep., Norwegian Defence Res. Establishment, 1993. [4] D. Lindberg and T. Wahl, “Automatic screening of SAR images for slick detection,” Tech. Rep., Norwegian Defence Res. Establishment, 1995. [5] M. Kubat, R. C. Holte, and S. Matwin, “Machine learning for the detection of oil spills in satellite radar images,” Mach. Learn., 1998, accepted for publication. ˚ Skøelv, J. P. Pedersen, L. Seljelv, J. H. Andersen, O. A. [6] T. Wahl, A. Follum, T. Anderssen, G. D. Strøm, T. Bern, H. H. Espedal, H. Hamnes, and R. Solberg, “Radar satellites: A new tool for pollution monitoring in coastal waters,” Coastal Manage., vol. 24, pp. 61–71, 1996. [7] T. I. Bern, T. Wahl, T. Anderssen, and R. Olsen, “Oil spill detection using satellite based SAR; Experience from a field experiment,” Photogramm. Eng. Remote Sensing, vol. 59, pp. 423–428, 1993. [8] V. Wismann, “Oil spill detection and monitoring with the ERS-1 SAR,” in Proc. 2nd ERS-1 Symp., Space at the Service of Our Environment, Hamburg, Germany, Oct. 1993. pp. 431–435. [9] M. K. Hu, “Visual pattern recognition by moment invariants,” IEEE Trans. Inform. Theory, vol. IT-8, pp. 179–187, 1962. [10] A. Solberg and R. Solberg, “A large-scale evaluation of features for automatic detection of oil spills in ERS SAR images,” in Proc. IEEE Symp. Geosci. Remote Sensing (IGARSS), Lincoln, NE, May 1996, pp. 1484–1486.
Geir Storvik was born in Hitra, Norway, in 1962. He received the Cand. Scient. degree in mathematical statistics from the University of Oslo, Norway, in 1986, and the Dr. Scient. degree in mathematical statistics from the University of Oslo in 1993. From 1987 to 1993, he was a Research Statistician at the Norwegian Computing Center (parttime), working on reservoir simulation and image analysis, and a part-time student for the Dr. Scient. degree in mathematical statistics at the University of Oslo working with contour detection in noisy images. He was a Visiting Scholar at Stanford University, Stanford, CA, in 1988–1989. Currently, he is an Associate Professor at the Mathematical institute, University of Oslo. His main research interests are in the fields of statistical image analysis, statistical classification, boundary detection, pattern recognition, sampling techniques, and stochastic modeling. Dr. Storvik is a member of the Institute of Mathematical Statistics, the American Statistical Association, and the Norwegian Statistical Association.
Rune Solberg received the Cand. Scient. degree in computer science in 1985 from the University of Oslo, Norway. He is currently a Senior Research Scientist at the Norwegian Computing Center, Oslo, where he is in charge of the remote sensing activities. His research interests include preprocessing of remote sensing data, feature extraction, segmentation, and imaging spectrometry.
Espen Volden was awarded scholarships from Total Marine AS and ELF Aquitaine AS to attend the Ecole Nationale Sup´erieure des Mines de Paris, France, where he received the Diplˆome d’Ing´enieur degree in 1991. The same year, he graduated from the Applied Mathematics Department of the Norwegian Institute of Technology (Sivil-ingeniør). After working on industrial organizational problems for Lancome (L’Or´eal), he was a Research Assistant at the Institut National de Recherche en Informatique et en Automatique (INRIA), Sophia Antipolis, France, from 1993 to 1995. He is currently a Research Scientist at the Norwegian Computing Center, Oslo, Norway. His research interests include Markov random fields, image segmentation, texture, cooccurrence matrices, information theory, and multisensor fusion.
IEEE TRANSACTIONS ON GEOSCIENCE AND REMOTE SENSING, VOL. 37, NO. 4, JULY 1999
Automatic Detection of Oil Spills in ERS SAR Images Anne H. Schistad Solberg, Member, IEEE, Geir Storvik, Rune Solberg, and Espen Volden
Abstract— We present algorithms for the automatic detection of oil spills in SAR images. The developed framework consists of first detecting dark spots in the image, then computing a set of features for each dark spot, before the spot is classified as either an oil slick or a “lookalike” (other oceanographic phenomena which resemble oil slicks). The classification rule is constructed by combining statistical modeling with a rule-based approach. Prior knowledge about the higher probability for the presence of oil slicks around ships and oil platforms is incorporated into the model. In addition, knowledge about the external conditions like wind level and slick surroundings are taken into account. The presented algorithms are tested on 84 SAR images. The algorithm can discriminate between oil slicks and lookalikes with high accuracy. 94% of the oil slicks and 99% of the lookalikes were correctly classified. Index Terms—Bayesian image classification, modeling of prior knowledge, oil spill detection, SAR image analysis.
I. INTRODUCTION
A
PROJECT for use of ERS-1 SAR data for the detection of marine oil spills was initiated in 1991 as part of the Norwegian Space Centre’s national ERS-1 program. Based on this activity, a near real-time service by Tromsø Satellite Station (TSS) has been established according to requirements set by the Norwegian Pollution Control Authority. Today’s manual service is a first step toward a fully operational system covering Norwegian and adjacent waters. The present study, which aims at determining the potential for automatic oil spill classification, is part of this project. Being able to use data from TSS collected regularly since 1994, we can present experimental results involving a large number of SAR scenes. Automatic identification of oil spills in SAR images is a very complex task because objects which resemble oil spills (in this paper, called “lookalikes”) frequently occur, particularly in low-wind conditions. Examples of oil spill lookalikes are organic film, grease ice, wind front areas, areas sheltered by land, rain cells, current shear zones, internal waves, and upwelling zones [1]. In some cases, even an experienced operator cannot determine if a slick actually is an oil slick or a lookalike. The manual oil spill service at TSS assigns possible oil spills into three categories representing slicks with high, medium, or low probability of being oil. This information Manuscript received July 28, 1998. This work was supported by Tromsø Satellite Station (TSS). A. H. Schistad Solberg, R. Solberg, and E. Volden are with the Norwegian Computing Center, 0314 Oslo, Norway (e-mail: [email protected]). G. Storvik is with the Norwegian Computing Center, 0314 Oslo, Norway and the Department of Mathematics, University of Oslo, 0316 Oslo, Norway. Publisher Item Identifier S 0196-2892(99)03483-X.
Fig. 1. Oil spill detection system.
is forwarded to the Norwegian Pollution Control Authority, which can use a surveillance aircraft to verify it. A trained human interpreter is able to discriminate between oil slicks and lookalikes based on experience and prior information concerning location, external information about weather conditions, differences in shape and contrast to surroundings between oil slicks and lookalikes, etc. To obtain the same results for an automatic slick classification, these aspects must be incorporated into the classifier. This poses a challenge in designing a classification algorithm. In this paper, we discuss the most important factors when discriminating between oil slicks and their lookalikes, and we present a classification algorithm which incorporates many of these factors. Our goal is to develop a semiautomatic system for oil spill detection, in which objects with a high probability of being oil slicks are automatically identified. These possible oil slicks are then presented to an operator for manual inspection. Presently, we do not believe that a fully automatic oil spill detection algorithm will be able to correctly discriminate between the most difficult oil slicks and their lookalikes. However, a semiautomatic procedure will greatly reduce the number of SAR images which need to be manually inspected compared to a fully manual detection procedure. A framework for automatic detection with the following main elements (see Fig. 1) is
0196–2892/99$10.00 1999 IEEE
SCHISTAD SOLBERG et al.: AUTOMATIC DETECTION OF OIL SPILLS
presented: 1) detection of dark spots, 2) spot feature extraction, and 3) dark spot classification. We illustrate the importance of incorporating prior knowledge by comparing the performance of the classification rules obtained without and with prior distributions and a classifier based on both prior distributions and rule-based corrections of the probability densities for the observed features. Only a small number of studies aiming at automatic oil spill detection have reported results on more than a few images. Weisteen et al. [2] describe our initial work on the problem on a very limited data set. Wahl et al. [3], [4] included an algorithm for automatic detection of dark spots which could possibly be oil spills in their ship detection system. A tree classifier for oil spill classification is presented in Kubat et al. [5], but only the classification part, without specifying the feature extraction part. Using a leave-one-image-out strategy on nine SAR images, they correctly classify 78% of the oil spills and 50% of the lookalikes. The remainder of the paper is organized as follows. Section II describes SAR imaging of oil spills and lookalikes, Section III the algorithm for detection of dark spots, and Section IV the feature extraction process. The classification problem and the modeling are discussed in detail in Section V, while Section VI contains the experimental results. Section VII presents a discussion and conclusions.
II. SAR IMAGING
OF
OIL SPILLS
A number of researchers have described SAR imaging of oil spills or lookalikes. Oil usually appears as dark spots in SAR images because it has a dampening effect on the Bragg waves. Wahl et al. [6] give an overview of radar satellites in terms of their oil spill detection capabilities. Bern et al. [7] describe SAR oil spill imaging in relation to wind speed. Wismann [8] has studied the radar return of more than 150 oil spills. These studies reach similar conclusions regarding the visibility of oil slicks in ERS SAR imagery. Hovland et al. [1] studied different kinds of lookalikes, and proposed a simple decision tree for discriminating between oil spills and lookalikes. The following description relies heavily on other studies performed as part of the Norwegian Oil Spill project (see [6] and the references therein). The SAR signature of an oil spill will depend on the external conditions. The contrast between the spill and its surroundings depends on a number of parameters like wind speed, wave height, and the amount and type of oil released. The shape of the spill will depend on whether the oil was released from a stationary object or from a moving ship, the amount of oil involved, and the wind and current history between the release and the image acquisition. When discriminating between oil slicks and lookalikes, we think that the following factors are the most important to model. • Known objects and detected point sources: If a bright object (ship or oil platform) is seen close to a spot, there is a higher probability for the presence of oil slicks. The North Sea, which has a large number of oil platforms, is part of the oil spill coverage area for Tromsø Satellite
1917
Station. There are also many examples of oil slicks directly connected to a ship in the test data used. • Wind and probability for lookalikes: One will often observe many lookalikes with low-wind conditions. With moderate and high wind, the number of lookalikes will be small. • Wind and spot detection: An oil slick will have a large contrast to the surroundings in low-wind conditions. With moderate wind, the oil will have smaller contrast to the surroundings. • Slick surroundings: The human eye is superior in observing a slick in the context of the surrounding sea. If the surroundings are homogeneous, the human observer will have more belief in that the spot is an oil slick than with heterogeneous surroundings. With heterogeneous surroundings, the human eye can easily determine if the spot is separated from the surroundings based on contrast or orientation. III. SPOT DETECTION The algorithm for detection of dark spots is based on adaptive thresholding. This thresholding is based on an estimate of the typical backscatter level in a large window. The adaptive dB below the estimated local mean threshold is set to pixels) is backscatter level. The window (of size moved across the image in small steps to threshold all pixels in the scene. Wind data (the wind level) is used to determine . Currently, the wind level is set manually as one of four categories: low, low/medium, medium, or high wind. This will be replaced by automatic methods for wind estimation based on the SAR image. As part of the Norwegian oil spill project, NORUT-IT are currently developing algorithms for estimation of wind speed and direction from ERS SAR images, and these algorithms will be used when the system is operated at TSS. This dark spot detection procedure did not always define the correct border between the oil slick and the surrounding sea when the surroundings were heterogeneous (particularly in low-wind conditions). Parts of the surroundings are sometimes included in the detected spot. A clustering step is used to avoid this. After spot detection, each spot is clustered into two clusters. The idea is that if the spot includes part of its surroundings, the oil slick will consist of the darkest cluster. If the two clusters are sufficiently separated and the darkest cluster is sufficiently large compared to the brightest, the darkest cluster is used as the spot; otherwise, the original spot is kept. This algorithm works well on most of the 84 images that we have tested it on. The algorithm detected all of the oil spills in the test data set. In certain cases, a thin, linear oil spill was detected as several smaller spots because the contrast varies within the spill. IV. SLICK FEATURE EXTRACTION For each region corresponding to a detected spot, a set of features is computed. The features constitute standard descriptors applied for regions in general image analysis
1918
IEEE TRANSACTIONS ON GEOSCIENCE AND REMOTE SENSING, VOL. 37, NO. 4, JULY 1999
applications, and some additional features particularly suited for oil spill detection. The following features are used. 1) Distance to a point source: A simple algorithm searches for bright points in the image, and the distance from a region corresponding to a dark spot to the closest bright object is computed. 2) Number of detected spots in the scene. 3) Number of neighboring spots in a larger window centered at the region. 4) Homogeneity (power-to-mean ratio) of the surroundings. , where is 5) Slick complexity is defined by the perimeter and is the area of the region. 6) Slick width is the ratio between the area of the region and the width of the branches of the skeleton of the region. 7) Slick area is the size (in number of pixels) of the region. 8) First invariant planar moment [9], which describes the shape of the region. , where 9) Local area contrast ratio is defined as is the local backscatter mean computed in a window is the mean backscatter centered at the region, and value of the region. 10) Border gradient is the mean of the magnitude of gradient values of the region border area. The Sobel operator is used to compute the gradients. , 11) Power-to-mean ratio of the slick, defined as and are the standard deviation and the mean where value of the slick, respectively. Feature 1) is actually prior information, and not observed data in our modeling assumptions (see Section V-A). The next three features are measures of the surroundings of the spot. Features 5)–8) measure the shape of the spot, while the last three features are measures of the actual backscatter values of the spot. V. SLICK CLASSIFICATION After spot detection and feature extraction, we have a set dark spots or regions which we want to classify as of oil slicks or lookalikes. For each region, we have a feature describing the shape, contrast, surroundings, etc., vector of the region. In addition, we have prior information about the likelihood of observing oil slicks and external information about wind level which will influence the observed features. The nature of the oil spill classification problem poses a set of challenges which needs to be handled during the design of the classifier. It is a problem in which context is extremely important. The classification algorithm must try to resemble the way a trained human operator views a possible oil slick in relation to the whole scene, and his knowledge about the weather conditions and oil spills in general. The need for controling how some important features are used during classification made us reject the automatically generated tree classifier used in a preliminary version of the system [10]. Instead, we have decided to use a classifier which combines a prior model for the number of lookalikes in a scene, a model for the presence of a slick in the vicinity of a bright object, a standard probability density for the observed features, and a rule-based modification of the probability density to take into
account combinations of features which are indications of a certain scene condition. A. Statistical Modeling The goal is to design a classifier which incorporates both prior information we can derive about the scene and the information contained in the observed features for the detected dark spots. In statistical classification, there are two main ingredients which need to be specified, the prior model and the conditional probability density for the features. In standard classification applications, one would specify a prior probability for an object being an oil slick or not. In our case, the number of objects to be classified is actually stochastic, implying that the standard approach is not the obvious choice. A further complication is that the presence of ships and oil platforms in the scene influences the presence of oil slicks, and that the wind level affects the number of lookalikes. In spite of these complications, it turns out that we are able to specify a (pseudo) prior for the probability of a detected spot being an oil slick (see the next subsection). We will , emphasizing the dependence on denote this prior by the information about the presence of ships and oil platforms which we assume is given through the vector . The second part to be specified is the probability density and for the observed features in classes oil slick and lookalikes, respectively. Note the dependence on the external information which will be used for describing the changes in the distribution of the feature values as a function of the wind level . The prior distribution and the probability density for the features are combined to obtain the posterior probability for a detected spot being an oil slick through Bayes theorem. Let be the unknown class membership of a detected spot. Then Pr
(1)
We want the classifier to find all spots with a certain probability of being oil. All spots classified as oil are then inspected by an operator. Misclassifying oil as lookalikes is considered more serious than misclassifying lookalikes as oil. This can be modeled in terms of a simple loss function. Let be the loss associated with misclassifying a true oil slick as a lookalike, and the loss associated with misclassifying a true lookalike as an oil slick. Then the optimal classification for spot is given by if Pr otherwise. B. Specification of “Prior” Probabilities Prior distributions for the presence of oil slicks can, in its simplest form, be specified through the number of oil slicks
SCHISTAD SOLBERG et al.: AUTOMATIC DETECTION OF OIL SPILLS
1919
in the scene. In particular, we will assume that the probability of oil slicks in the scene is given by (2)
TABLE I THE EXPECTED NUMBER OF LOOKALIKES IS GIVEN BY 1=(1 w ) WHERE w IS THE CORRESPONDING PARAMETER IN THE GEOMETRIC DISTRIBUTION FOR THE NUMBER OF LOOKALIKES
0
Wind level
is a parameter describing the expected number of where oil slicks in the scene. The presence of ships or oil platforms may increase the number of oil slicks expected in the area. Such ships or platforms can be seen as bright spots in the SAR images. Although there always will be some uncertainty involved, we assume that the positions of such objects can be estimated with sufficient accuracy. We can then construct a prior distribution for the presence of oil slicks in the neighborhood of the bright spot positions. Consider oil slicks in a scene with distances to the nearest bright spot. We then assume that the prior distribution (2) is modified to
where if if otherwise. This prior is not directly usable in order to calculate the posterior probability for a single detected spot being an oil slick or not. We need a probability statement about a detected spot prior to taking the observed features into account. In order to do this, the distribution of the number of lookalikes in a scene has to be handled. the number of detected spots. As previously Denote by noted, we assume that all of the present oil slicks will be , the number of oil slicks, the detected as dark spots. Given , the number of lookalikes stochastic variable is that will appear in the scene. We will assume that Pr that is, a geometric distribution with parameter depending on the wind level (given by the dependence on ). In particular, we have assumed that the wind level is divided into four categories. These categories are given in Table I together with . Based on these their corresponding values of assumptions, it is shown in the Appendix that Pr
which we will use as our “prior” an oil slick.
for an object being
C. Specification of Class-Conditional Probability Densities For each of the detected dark spots, a number of features are calculated, collected in the feature vector . These features are constructed such that they typically will be different, depending on whether the dark spot is an oil slick or a lookalike. However, a further complication is that the behavior
Low Low-to-moderate Moderate High
Expected no. lookalikes 10 000 100 20 7
of these features will change with different wind levels. Denote the wind level by . The simplest choice of densities is multivariate Gaussian ones. Even within each wind level, both the oil slicks and the lookalikes may vary quite a lot in shape and other features. Describing the feature density by a unimodal density such as the Gaussian is therefore not appropriate. Instead, we have assumed different densities, depending on the value of a shape descriptor (we have used the first invariant planar moment). was divided into four subgroups, The sample space of ranging from wide spots with regular shape to wide slicks with a complex boundary to thin, linear spots. The density for class and wind level is then given by if is in subgroup . Finally, the densities inside each subgroup are assumed Gaussian:
where , is the number of features, is and shape descriptor in the mean vector under wind level is a diagonal covariance subgroup for class , and . Only the matrix, common for both classes given features slick complexity, slick contrast ratio, border gradient, local smoothness contrast, homogeneity of surroundings, slick width, and number of neighboring spots are used to compute the probability density. The choice of making the covariance matrix equal for both the oil slick and the lookalike class is an important one. Lookalikes occur much more frequently than oil spills, and their features will vary much more than the features for oil slicks. If class-dependent covariance matrices are used, the variance of the lookalike class will be huge compared to the oil spill class. The resulting class-conditional probabilities for lookalikes will be of another magnitude than for oil, resulting in a large “bias” in the classification. A way of handling the very unbalanced data set is to avoid using a class-conditional covariance matrix. Another advantage of using a common covariance matrix is that the number of parameters involved will be reduced. Because of limited training data for oil slicks, a further reduction in the number of unknown parameters (which need to be estimated) is obtained by assuming the common covariance matrix to be diagonal. By doing so, we are ignoring the correlation between the features. D. Rule-Based Corrections of the Class-Conditional Densities Many combinations of features are known to give strong indications either toward an oil slick or toward a lookalike.
1920
IEEE TRANSACTIONS ON GEOSCIENCE AND REMOTE SENSING, VOL. 37, NO. 4, JULY 1999
Because of our need for using common diagonal covariance matrices in the Gaussian densities, knowledge of such combinations must be utilized in a different setting. Our choice has been to introduce a multiplicative factor adjusting the Gaussian densities [or actually the ratios between the densities of oil slick and lookalikes, see (1)] in a rule-based setting. The features listed below are used for this purpose. Many of these rules handle interactions between several features, which are indications of certain scene configurations. • Slick area: In low/low–moderate wind, we can have very large dark spots corresponding to large areas with no wind and almost no backscattering from the sea. If the area is very large, the ratio of densities is decreased (divided by a large factor). If the area is very small, the slick needs better contrast to be classified as oil; thus, the ratio of density is also decreased in this case. • Number of regions in a scene: If the number of detected spots in the scene is large, the scene is very complex, and most of the slicks will be lookalikes. The ratio of densities is decreased. If the number of regions is low, and the slick has homogeneous surroundings, the feature density for oil is increased. • Slick local contrast ratio: Used in combination with other features, the slick contrast affects the probability densities for the features. If the contrast is low for a slick with several neighboring regions in a scene with many regions, the ratio of densities is decreased. • Slick width: Used in combination with other features, the slick width affects the probability densities. For low-wind conditions, thick regions with large area, and not very high contrast the ratio of densities is decreased. : If is large, • Homogeneity of surroundings, the surroundings are heterogeneous. If, in addition, the number of regions in the scene and the number of neighboring regions are high, the ratio of densities is is low (homogeneous surrounding) decreased. If and the scene is not very complex, the ratio of densities is increased. Such modifications of the feature densities are not common in statistical classification. Rules like these are often used in knowledge-based classification methods, in which they are used to define boxes for a hard classification. We use them in combination with a statistical classifier. They allow the combination of several features, while a Gaussian classifier with a diagonal covariance matrix ignores the correlation between the features. The specification of the rules was performed by inspecting the results obtained by the classification only based on the Gaussian densities and the prior. Only a small subset of the images was used in order to avoid overfitting. VI. EXPERIMENTAL RESULTS The data set used to test the performance of the algorithm was designed to contain as many oil spills as possible, but also to include scenes containing only lookalikes. The images in the test set were selected by an experienced oil spill operator at Tromsø Satellite Station. The ground truth consisted of the
TABLE II DETECTION ACCURACIES. o IS THE NUMBER OF LOST OIL SPILLS, AND l IS THE NUMBER OF LOOKALIKES CLASSIFIED AS OILS SPILLS. MODEL 1 CORRESPONDS TO A CLASSIFICATION RULE BASED ON THE GAUSSIAN DENSITIES AND A PRIOR ONLY TAKING THE NUMBER OF OIL SPILLS AND THE NUMBER OF LOOKALIKES INTO ACCOUNT. MODEL 2 IS THE EXTENSION OF MODEL 1, ALSO TAKING THE DISTANCE TO A BRIGHT SPOT INTO ACCOUNT, WHILE MODEL 3 IS A FURTHER EXTENSION INCLUDING THE RULE-BASED CORRECTIONS OF THE DENSITIES
N
Model 1 Model 2 Model 3
N
N
o
19/71 11/71 4/71
N
l
310/6980 363/6980 75/6980
manual classification of each scene performed by the operators at TSS. Each scene was manually classified into the categories low, medium, or high probability of containing oil. The manual classification is used as ground truth, although we have no guarantee that the manual classification is correct. Out of a test set of 84 SAR scenes, 36 did not contain any oil spills, but were included to test the classifier on a high number of lookalikes. In addition, some scenes contained one or two oil spills and many (>100) lookalikes. The images were collected at TSS during a two-year period of operation of their manual oil spill detection service. The oil slicks belong to four main categories: 1) thin, linear slicks which might be caused by a ship or a stationary object releasing a small amount of oil; 2) wide, regular slicks caused by a stationary object releasing a larger amount of oil; 3) wide, irregular slicks for which the wind and/or current have altered the shape of the slick; 4) thin, piecewise linear slicks caused by a moving ship changing directions, or a thin slick altered by wind or current. The use of different densities depending on the value of our shape descriptor makes it possible to handle such a variety of cases automatically. To evaluate the performance of the classifier based on a SAR images, we used a leavedata set consisting of one-image-out strategy. We have trained the classifier (that is, estimation of the means and the covariance matrices in the images, then classified the Gaussian densities) on remaining image, and computed the number of correctly and incorrectly classified objects. The rule-based corrections were kept fixed all throughout the experiment. This procedure is times until all images have been classified. repeated It is illustrative to study the effect of the various elements in the classification model. Table II shows the results for three different classification rules. For the first model, only the distributions of the number of oil spills and the number of lookalikes are incorporated into the prior. Furthermore, Gaussian densities without the rule-based corrections are used. Most of the oil slicks in this case are correctly classified. However, a relatively high number of lookalikes is classified as oil slicks, although this number should also be compared to the number of lookalikes detected as dark spots (6980), showing that most of the lookalikes are discarded. If we incorporate the distance to a bright spot, the number of lost oil spills is reduced, although we obtain an increase
SCHISTAD SOLBERG et al.: AUTOMATIC DETECTION OF OIL SPILLS
Fig. 2. Oil slick on heterogeneous background. The region classified as oil is shown in the small frame (ESA/TSS/NR).
in the number of false alarms. Extending the model further by incorporating the rule-based corrections of the densities, both the number of lost oil spills and the number of false alarms were reduced significantly. The classification accuracy is so high that it is worthwhile to use the algorithms in an operational system. The rule-based adjustment of the densities is thus very important in reducing the number of false alarms, while also correctly identifying the oil slicks. Figs. 2–6 display the detection results in different types of scenes. A correctly classified slick on a very heterogeneous background is shown in Fig. 2. Fig. 3 shows a scene with several slicks connected to point sources. All of the slicks close to point sources were classified as oil, in addition to a small spot in the middle of the scene. Fig. 4 shows two correctly classified oil slicks connected to point sources, while Fig. 5 shows a complex scene with two lookalikes which were misclassified as oil. A complex scene with no alarms is shown in Fig. 6. VII. DISCUSSION
AND
CONCLUSIONS
A method for detecting oil slicks from SAR images has been presented. The method is based on a combination of using prior knowledge, Gaussian densities, and rule-based density corrections. The developed algorithms achieved a high accuracy on detecting oil spills. 94% (67 out of 71) of the oil slicks were correctly classified, while only 1% (75 out of 6980) of the lookalikes were wrongly classified. The system is currently being implemented at Tromsø Satellite Station. The advantage of using an automatic algorithm and not merely manual inspection of all possible SAR images depends on the number of SAR scenes to be analyzed and their complexity. Our test data set consisted of 84 images which all contained some objects with a certain probability of being oil. Manual inspection determined that 36 of these images contained only lookalikes, indicating that they should not cause alarms. 21 of these 36 images were automatically processed without causing any false alarms. Nine images contained alarms which were reasonable for a manual operator to have
1921
a closer look at, while the remaining six images resulted in false alarms (sending an image to manual inspection when the operator will determine that the alarm is false). The classifier had a tendency to classify many spots in medium wind with very homogeneous surroundings as oil. This number can probably be reduced by adjusting the parameters to reduce this effect. Our test data set did not contain any images with no possible oil objects, that is, scenes which are not classified into any of the categories low, medium, and high probability of oil at TSS. Such scenes will occur in an operational monitoring, but we do not know how frequent they are. This must be accounted for when determining the actual effect of an automatic algorithm versus manual inspection of all scenes. Our classification methods are based on formal statistical modeling, which also can be used as a foundation for further improvements. In particular, incorporating other types of prior knowledge and/or utilizing knowledge about the behavior of the dark spot features may further increase the accuracy. Because oil spills are relatively rare, we would like to design a system which can learn from experience when used operationally. If the system misclassifies a spot, as judged by the operator who examines all spots assigned to the oil category, the spot will be used to update the class description database when the system is used operationally.
CALCULATION
APPENDIX “PRIOR” PROBABILITIES
OF
In Section V, a prior distribution for oil slicks in a scene is introduced, together with a specification of the probability distribution for the number of lookalikes. We now show how we can combine these models in order to obtain a “prior” probability for a single detected spot being an oil slick. Since the probability distributions are defined through the total number of oils slicks and lookalikes, we need to work with the simultaneous distribution of all detected spots. The probability of interest is Pr
Pr
(3) Pr (We have Pr because of symmetry.) Furthermore, if we define
for all
Pr Pr
Pr
(4)
1922
IEEE TRANSACTIONS ON GEOSCIENCE AND REMOTE SENSING, VOL. 37, NO. 4, JULY 1999
Fig. 3. Several spots close to point sources classified as oil (ESA/TSS/NR).
Fig. 4. Two oil slicks connected to point sources (ESA/TSS/NR).
SCHISTAD SOLBERG et al.: AUTOMATIC DETECTION OF OIL SPILLS
Fig. 5. Two alarms in a very complex scene (ESA/TSS/NR).
Fig. 6. Complex scene with no alarms (ESA/TSS/NR).
1923
1924
IEEE TRANSACTIONS ON GEOSCIENCE AND REMOTE SENSING, VOL. 37, NO. 4, JULY 1999
where
Combining (4) with (3) results in
Anne H. Schistad Solberg (S’92–M’96) received the M.S. degree in computer science in 1989 and the Ph.D. degree in image analysis in 1995, both from the University of Oslo, Norway. She is currently a Senior Research Scientist at the Norwegian Computing Center, Oslo. During 1991–1992, she was a Visiting Scholar, Department of Computer Science, Michigan State University, East Lansing. Her research interests include statistical pattern recognition, multisensor image classification, texture analysis, and remote sensing.
Pr where
is given by
which is a constant not depending on . Using that Pr Pr , we end up with Pr
ACKNOWLEDGMENT The ERS SAR images were provided by TSS. The authors are grateful to T. Wahl (NDRE), H. Espedal (NERSC), H. Johnsen (NORUT-IT), J. Petter Pedersen, and T. Bauna (TSS) for sharing their knowledge about visually discriminating between oil slicks and lookalikes. REFERENCES [1] H. A. Hovland, J. A. Johannessen, and G. Digranes, “Slick detection in SAR images,” in Proc. IEEE Symp. Geosci. Remote Sensing (IGARSS), Pasadena, CA, Aug. 1994, pp. 2038–2040. [2] K. Weisteen, A. Solberg, and R. Solberg, “Detection of oil spills in SAR images using a statistical classification scheme,” in Proc. IEEE Symp. Geosci. Remote Sensing (IGARSS), Tokyo, Japan, Aug. 1993, pp. 943–945. [3] T. Wahl et al., “Oil spill detection using satellite based SAR, Phase 1b completion report,” Tech. Rep., Norwegian Defence Res. Establishment, 1993. [4] D. Lindberg and T. Wahl, “Automatic screening of SAR images for slick detection,” Tech. Rep., Norwegian Defence Res. Establishment, 1995. [5] M. Kubat, R. C. Holte, and S. Matwin, “Machine learning for the detection of oil spills in satellite radar images,” Mach. Learn., 1998, accepted for publication. ˚ Skøelv, J. P. Pedersen, L. Seljelv, J. H. Andersen, O. A. [6] T. Wahl, A. Follum, T. Anderssen, G. D. Strøm, T. Bern, H. H. Espedal, H. Hamnes, and R. Solberg, “Radar satellites: A new tool for pollution monitoring in coastal waters,” Coastal Manage., vol. 24, pp. 61–71, 1996. [7] T. I. Bern, T. Wahl, T. Anderssen, and R. Olsen, “Oil spill detection using satellite based SAR; Experience from a field experiment,” Photogramm. Eng. Remote Sensing, vol. 59, pp. 423–428, 1993. [8] V. Wismann, “Oil spill detection and monitoring with the ERS-1 SAR,” in Proc. 2nd ERS-1 Symp., Space at the Service of Our Environment, Hamburg, Germany, Oct. 1993. pp. 431–435. [9] M. K. Hu, “Visual pattern recognition by moment invariants,” IEEE Trans. Inform. Theory, vol. IT-8, pp. 179–187, 1962. [10] A. Solberg and R. Solberg, “A large-scale evaluation of features for automatic detection of oil spills in ERS SAR images,” in Proc. IEEE Symp. Geosci. Remote Sensing (IGARSS), Lincoln, NE, May 1996, pp. 1484–1486.
Geir Storvik was born in Hitra, Norway, in 1962. He received the Cand. Scient. degree in mathematical statistics from the University of Oslo, Norway, in 1986, and the Dr. Scient. degree in mathematical statistics from the University of Oslo in 1993. From 1987 to 1993, he was a Research Statistician at the Norwegian Computing Center (parttime), working on reservoir simulation and image analysis, and a part-time student for the Dr. Scient. degree in mathematical statistics at the University of Oslo working with contour detection in noisy images. He was a Visiting Scholar at Stanford University, Stanford, CA, in 1988–1989. Currently, he is an Associate Professor at the Mathematical institute, University of Oslo. His main research interests are in the fields of statistical image analysis, statistical classification, boundary detection, pattern recognition, sampling techniques, and stochastic modeling. Dr. Storvik is a member of the Institute of Mathematical Statistics, the American Statistical Association, and the Norwegian Statistical Association.
Rune Solberg received the Cand. Scient. degree in computer science in 1985 from the University of Oslo, Norway. He is currently a Senior Research Scientist at the Norwegian Computing Center, Oslo, where he is in charge of the remote sensing activities. His research interests include preprocessing of remote sensing data, feature extraction, segmentation, and imaging spectrometry.
Espen Volden was awarded scholarships from Total Marine AS and ELF Aquitaine AS to attend the Ecole Nationale Sup´erieure des Mines de Paris, France, where he received the Diplˆome d’Ing´enieur degree in 1991. The same year, he graduated from the Applied Mathematics Department of the Norwegian Institute of Technology (Sivil-ingeniør). After working on industrial organizational problems for Lancome (L’Or´eal), he was a Research Assistant at the Institut National de Recherche en Informatique et en Automatique (INRIA), Sophia Antipolis, France, from 1993 to 1995. He is currently a Research Scientist at the Norwegian Computing Center, Oslo, Norway. His research interests include Markov random fields, image segmentation, texture, cooccurrence matrices, information theory, and multisensor fusion.
