Radar Image Segmentation using Recurrent ... - Semantic Scholar
Radar Image Segmentation using Recurrent Artificial Neural Networks1
Tom Ziemke The Connectionist Research Group Dept. of Computer Science University of Skövde S-54128 Skövde/Sweden [email protected] December 1995
Abstract This paper discusses the application of artificial neural networks to the segmentation of Doppler radar images, in particular the detection of oil spills within sea environments, based on a classification of radar backscatter signals. Best results have been achieved with recurrent backpropagation networks of an architecture similar to that of Elman's Simple Recurrent Network (1990). These recurrent networks are shown to be very robust to variations in both sea state (weather conditions) as well as illumination distance, and their performance is analysed in further detail.
Keywords:
1
oil spill detection, radar image segmentation, recurrent artificial neural networks
to appear in a special issue 'Artificial Neural Networks in Computer Vision' of the journal Pattern Recognition Letters (to be published around May '96, paper will be retyped and -formatted)
1
1 Introduction The design of systems for the identification of objects from the measurements of their radar backscatter signals has traditionally been based upon mathematical/decision-theoretic methods of pattern recognition. The applicability of such methods, due to their mathematical nature, crucially depends on the availability of sufficiently accurate models for the statistical properties of the radar measurements (Ahalt et al., 1989). The problem domain discussed here however, the segmentation of Doppler radar images of sea environments into oil and water areas, is characterised by the lack of such formal knowledge. Therefore artificial neural networks (ANNs), due to their 'model-freedom', offer an attractive alternative to this problem, since they are able to learn to perform the classification from labelled training data, and do not require a-priori knowledge of statistical models. Model-freedom lifts us from the oldest burden of modern science - guessing at cause-and-effect relationships with functions, and with functions so simple and so tractable that our recently evolved brain can understand them and manipulate them in symbols. (Kosko, 1992)
2 Problem Domain 2.1
General idea
The overall goal of this work is the development of a reliable and robust technique for the segmentation of images obtained from the measurements of so-called Side Looking Airborne Radar (SLAR, a conventional Doppler radar illuminating a sea environment from a plane) into oil and water areas.
Figure 1: Backscattered energy from water and oil
The discrimination of oil from water is theoretically possible due to the fact that oil dampens the capillary waves present on a sea surface, such that the surface becomes smoother and acts more like a mirror (as illustrated in figure 1), i.e. more energy will be reflected away from the radar by an oil covered surface than by pure water.
2
In practice however, this task, and in particular the detection of small oil spills, is further complicated by the noise and other problems inherent in the nature of radar measurements.
2.2 Doppler radar The SLAR mentioned above is a Doppler radar, i.e. it detects objects by virtue of their velocity with respect to the radar itself. Doppler filtering makes use of the progressive phase shift (Doppler shift) of an object's return signal on successively transmitted pulses, which is caused by the object's motion towards or away from the radar. The radar echo from a target area consists of the returns of a multitude of point scatterers forming a spectrum of individual phase shifts, the so-called Doppler spectrum or Doppler signature. Hence, beside the mean radial velocity, depending on the waves' translational motion (which can be assumed to be the same for oil and water), the spectral shape and width to some degree reflect the 'internal dynamics' of an illuminated area/object. The ability to perform a, to some extent, automatic classification of illuminated objects/areas using a conventional Doppler radar could be considered a significant increase in the capabilities of the radar sensor (Martinez Madrid et al., 1992). The differences in oil's and water's backscatter signals become apparent in their Doppler spectra, as illustrated in figure 2, such that in this case two relevant values can be extracted from the Doppler spectrum and will be used as input values for a classification network :
Figure 2: Typical Doppler spectra of oil and water (illumination distance 20 km)
Intensity, reflecting the magnitude of the backscattered energy, which, as explained and illustrated above, is lower for oil than for water. Spectrum width, i.e. the velocity spread of the individual point scatterers, which, due to the waves' lesser internal motion, is also lower for oil than for water.
3
It should be noted that the absolute values cannot be used for a discrimination of oil from water as they vary with varying wind conditions and illumination distances (s. table 1 for details).
2.3 SLAR simulation model In this study a simulation model of an SLAR, originally developed by Ericsson Microwave Systems, Mölndal/Sweden, (Fälldin, 1993), has been used. This is due to the fact that the radar simulated here in reality is still under development. Due to the SLAR's technical specification the radar's resolution (i.e. the resolution of the radar images) is such that cells of 20 m (flight direction) * 75 m are illuminated. The SLAR model simulates the measurements of 64 Doppler channels for each resolution cell, such that one Doppler spectrum (as shown in figure 2) can be obtained for each cell, from which values for intensity and spectrum width can be calculated for each individual cell.
2.4 Detection of small oil spills - The real problem The difficulty of discriminating oil from water in Doppler radar images is probably illustrated best with an example. Figure 3 shows an environment of 1500m * 1500m, corresponding to 75 * 20 cells in the radar's resolution, containing two large and five small oil spills (dark areas). The left and right half are here separated by a line as they will later be used as independent test and training sets. Environment map (radar resolution)
70
rows (flight direction >>)
60
50
40
30
20
10
2
4
6
8
10 12 columns
14
16
18
20
Figure 3: Model environment
Measurements of intensity and spectrum width within this environment are shown in figure 4 (illumination distance 20 km, sea state 3, cf. later). Both maps show peaks of low values (dark areas) where the large oil spills have their greatest widths (in flight direction), but the small oil spills cannot be discriminated reliably from the noise.
4
Spectrum width map
70
60
60 rows (flight direction >>)
rows (flight direction >>)
Intensity map
70
50
40
30
50
40
30
20
20
10
10
2
4
6
8
10 12 columns
14
16
18
20
2
4
6
8
10 12 columns
14
16
18
20
Figure 4: Measurements of a) intensity and b) spectrum width in the model environment
The absolute measurements are normalised with regard to different sea states by calculating the relative deviations from reference values characteristic for a certain sea state. These reference values can be calculated as mean values of intensity and spectrum width from an area known to be water, in this case the 100 cells in rows 1 to 5, such that the normalised measurements are obtained as follows: 2 intensity deviation = 10 ^ ( (measured intensity - reference intensity) / 10 ) - 1 spectrum width deviation = (measured spectrum width - reference spectrum width) / ref. spectrum width The resulting normalised vectors for the above environment together with their (correct) binary classification (using a threshold of 0.5, i.e. 50% oil in a resolution cell) look as shown in figure 5a. Resolution cell vectors ( o : oil, + : water ) − columns 7 to 15 0.4
0.3
0.3
0.2
0.2
0.1
0.1
Spectrum width deviation (rel.)
Spectrum width deviation (rel.)
Resolution cell vectors ( o : oil, + : water ) 0.4
0 −0.1 −0.2 −0.3 −0.4
0 −0.1 −0.2 −0.3 −0.4
−0.5
−0.5
−0.6
−0.6
−1
−0.8
−0.6
−0.4 −0.2 0 Intensity deviation (rel.)
0.2
0.4
−1
0.6
−0.8
−0.6
−0.4 −0.2 0 Intensity deviation (rel.)
0.2
0.4
Figure 5: Normalised resolution cell vectors and their binary classification, a) whole environment and b) columns 7 to 15
Hence, the resolution cell vectors show the expected distribution: The water cells cluster around the (water) reference values, i.e. around point (0,0). The oil cells mostly have both lower intensity and spectrum width values. 2
NB: dB is a logarithmic measure, therefore the above normalisation formula for the intensity deviation
5
0.6
But there is a large 'overlap area', that contains both kinds of cells, such that oil and water vectors are definitely not linearly separable. This area mostly contains the vectors of the small oil spill cells, such that a reliable classification of an individual cell only on basis of its own values is not possible. This is further illustrated in figure 5b which shows the vectors of columns 7 to 15, an area containing only small oil spills. So, what is it that makes the detection of small oil spills so difficult? Apart from the fact that radar measurements, by their nature, are noisy, the task is further complicated by the fact that the radar cannot focus on a single resolution cell. That means that at every time step the radar echo does not consist only of the returns of one resolution cell, but those from a much larger area 3. The practically relevant area should be approximately that within the 6 dB beam width4 of 88m, the SLAR model used here however simulates measurements up to the 24 dB beam width of 349m. This means that the measurements of a particular cell are always influenced by the returns of a number of neighbour cells. Therefore the radar echo of a single resolution cell containing oil (itself only being 20m * 75 m in size) will actually only be a 'pure oil' echo if the whole area covered by the relevant radar beam contains oil. This has the effect that at the border of an oil spill both returns of oil and water are measured (as illustrated in figure 6), such that small oil spills (only consisting of 'borders') can 'get lost' in their water neighbourhood. 6 dB radar beam(s)
oil spill
flight direction >>
88 m 6 dB beam width
20 m
illumination distance 20 km
Figure 6: Beam width vs. cell size
2.5 Goals The task of detecting oil spills from radar imagery is usually carried out by visual inspection through human operators. This procedure is a) difficult due to the slight differences between oil's and water's
3
4
Actually this area is theoretically infinitely large. The SLAR model simulates returns up to the 24 dB beam width, being measured where the gain has fallen 24 dB below maximum. At a distance of 20 km the 24 dB beam width is 349 m. The 6 dB beam, being measured where the gain has fallen 6 dB below maximum, is 88m at 20 km distance.
6
backscatter signals and b) inefficient and not possible in real-time due to the large amount of data obtained. Therefore goal of this work is to develop an ANN technique that allows an automatic segmentation of radar images as those shown above (cf. figure 4) into oil and water areas, based on a classification of individual resolution cells. Hence, the main requirements are as follows: The detection should be reliable, i.e. it should be sensitive to oil spills as small as possible. The number of false alarms should be kept to a minimum, i.e. it should be insensitive to noise in the radar measurements. In order to be applicable under real world conditions the segmentation method has to be be robust, e.g. to variations in sea states/wind conditions as well as illumination distance.
3 Experiments 3.1 General approach / previous work It has been pointed out earlier that the classification of individual resolution cells cannot be performed reliably (with regard to the detection of small oil spills) only on basis of a single cell's own values. It has been shown in earlier work (Ziemke and Athley, 1995a) that best classification performance is achieved with ANNs taking as input the values of intensity and spectrum width for the cell to be classified itself (C) and its four direct neighbours (N), as shown in figure 7.
flight direction >>
75 m
20 m
N N
C
N
N
Figure 7: Neighbourhood context
Furthermore in earlier experiments with different ANN architectures (Learning Vector Quantization (LVQ), feed-forward backpropagation networks and recurrent backpropagation networks of two different architectures, similar to those used by Jordan (1986) and Elman (1990)) best performance was achieved with recurrent networks of an architecture similar to Elman's Simple Recurrent Network (SRN). It has been shown that these recurrent networks enable a reliable and robust detection of oil spills down to a size of about 50 m * 50 m (Ziemke and Athley, 1995b, Ziemke, 1995). Further experiments with this architecture are documented in this paper.
7
3.2 Architecture(s) As mentioned above, the architecture used here is a recurrent network similar to the SRN as proposed by Elman5 (1990), i.e. a three-layer backpropagation network with weighted, fully-connected (indirect) feedback from the hidden layer to itself : output unit (1)
fully connected copy connection
t hidden units (5) t+1
t t
context units (5)
input units (10)
Figure 8: Recurrent architecture
As discussed earlier, the networks discussed here have been trained to individually classify each resolution cell within an environment. The target output values have been taken from the oil spill maps as shown in figure 3, therefore one output unit is used. The networks use as input values the normalised intensity and spectrum width values of the cell to be classified and its four direct neighbours (cf. figure 7), as shown in the intensity and spectrum width maps in figure 4, therefore ten input units are used. Best performance has been achieved with five hidden units. The networks have been trained with a 'fast' backpropagation algorithm using momentum and an adaptive learning rate (Demuth and Beale, 1992), using the logistic activation function for hidden and output units (earlier experiments with linear activation functions (Ziemke and Athley, 1995a) proved less successful). This means that the networks are trained to generate an estimation of the degree to which each cell is covered by oil. The desired binary classification into oil and water is achieved by simply thresholding these estimates with 0.5 afterwards. These recurrent networks have been trained to individually classify sequences of resolution cells in flight direction (including their direct neighbours). So the general approach here is to make use of the context of the previous cell's classification, which is based on the fact that, as illustrated in figure 6, it is not possible to measure a clear water echo in one time step and a clear oil echo in the next time step. Instead the transition at the border from water to oil there will normally be a sequence of 4-5 time steps during which oil's 'share' of the returns grows from 0% to 100%. Correspondingly, there will be a similar sequence of time steps at the border from oil back to water.
5
Elman trained his SRNs t o predict the next element in a sequence presented to the network, therefore input and output layer were of the same size.
8
Resolution cell vectors ( o : oil, + : water ) − sea state 6 0.4
0.3
0.3
0.2
0.2
0.1
0.1
Spectrum width deviation (rel.)
Spectrum width deviation (rel.)
Resolution cell vectors ( o : oil, + : water ) − sea state 1 0.4
0 −0.1 −0.2 −0.3 −0.4
0 −0.1 −0.2 −0.3 −0.4
−0.5
−0.5
−0.6
−0.6
−1
−0.8
−0.6
−0.4 −0.2 0 Intensity deviation (rel.)
0.2
0.4
−1
0.6
−0.8
−0.6
−0.4 −0.2 0 Intensity deviation (rel.)
0.2
0.4
0.6
Hence, this recurrent architecture aims at capturing the information contained in the temporal sequence of changes over a few time steps, from water to oil echo and the other way round, and at exploiting this sequential information contained in this transition for the classification task.
3.3 Robustness to varying sea states For an evaluation of the ANNs' robustness to varying wind/weather conditions data for the following six sea states has been used in the experiments discussed here. Sea state
Wind velocity [m/s]
Approx. radial velocity [m/s]
Wave height [m]
Approx. spectrum width water1 [m/s]
RCS water2 [dB]
1
1-3
0.2
0.1 - 0.3
0.3
- 45
2
3-6
0.7
0.3 - 0.9
0.6
- 41
3
6-8
1
0.9 - 1.5
0.9
- 38
4
8 - 10
1.4
1.5 - 2.4
1.2
- 35
5
10 - 13
1.7
2.4 - 3.6
1.6
- 33
6
13 - 15
2.1
3.6 - 6.0
1.9
- 28
1
The spectrum width of oil is assumed to be 50% of that of water. (Experiments with values of 30% and 70% were equally successful, showing that this assumption is not critical.)
2
RCS (Radar Cross Section) is proportional to intensity. Values given for 20 km idistance, values for 10 km are up to 6 dB higher, those for 40 km up to 4 dB lower. At an illumination distance of 10/20/40 km oil's RCS can be assumed to be about 8/6/5 dB lower than that of water.
Table 1: Sea states
In all sea states the normalized measurements show a similar distribution (into two clusters plus an overlap area) as in sea state 3 (cf. figure 5), but in lower sea states the values spread out more whereas in higher sea states the clustering is even stronger. This can be seen pretty clearly in figure 9.
Figure 9: Resolution cell vectors in a) sea state 1 and b) sea state 6
To evaluate the ANNs' robustness to variations in sea states/wind conditions, ten recurrent networks have been trained, five of them on the left half, five of them on the right half of the above environment
9
Binary output map (threshold 0.5)
70
60
60
rows (flight direction >>)
rows (flight direction >>)
Continuous output map
70
50
40
30
50
40
30
20
20
10
10
2
4
6
8
10 12 columns
14
16
18
20
2
4
6
8
10 12 columns
14
16
18
20
(sea state 3, illumination distance 20 km). All networks have been trained for 500 epochs, which showed best results in initial experiments. These networks have been tested on the whole environment in all sea states. It has however turned out that the results obtained for sea state 1 were dramatically worse than those for other sea states. This can be explained by the fact that for very low waves (as in sea state 1) the rather small differences between oil and water can easily 'disappear' in the noise. The results (using a threshold of 0.5) are summarised in table 2. Sea state
Oil spills detected
False alarms
1
64 (91%)
> 20
2
68 (97%)
6
3
70 (100%)
0
4
70 (100%)
1
5
69 (99%)
1
6
70 (100%)
0
Table 2: Performance of 10 ANNs trained on 50% of the environment (sea state 3), tested on the whole environment in all sea states
It can be seen that, although the networks have only been trained for one particular sea state (3), the classification performance for sea states 3 to 6 is almost perfect, but performance decreases significantly for the lower states. The following figure shows the continuous output values (sea state 3) computed by one of these networks (trained on the left half) as well as the resulting binary classification (again using a threshold of 0.5). Intensity map (40 km)
70
70
60
60
rows (flight direction >>)
rows (flight direction >>)
Intensity map (10 km)
50
40
30
50
40
30
20
20
10
10
2
4
6
8
10 12 columns
14
16
18
20
2
4
6
8
10 12 columns
14
16
18
Figure 10: a) Continuous output map , b) resulting binary classification
10
20
When comparing these maps to the correct environment map (in figure 3) it can be seen that the output as generated by the recurrent network actually comes very close to the original.
3.4 Robustness to varying illuminations distances To evaluate the ANNs' robustness to varying illumination distances the same ten recurrent networks as above, each trained on one half of the radar images taken from a distance of 20 km (sea state 3), have been tested on data obtained from measurements from illumination distances of 10 and 40 km in all sea states. A halving/doubling of the illumination distance also leads to a halving/doubling of the (relevant) beam width. Hence, the images obtained from a shorter distance should be clearer and the detection/segmentation should be easier, and correspondingly for a longer illumination distance the detection of small oil spills should become more difficult. This effect is illustrated in figure 11, which shows the normalised measurements of intensity from distances of 10 and 40 km (for 20 km cf. figure 4).
Binary output map (40 km, threshold 0.5)
70
70
60
60
rows (flight direction >>)
rows (flight direction >>)
Binary output map (10 km, threshold 0.5)
50
40
30
50
40
30
20
20
10
10
2
4
6
8
10 12 columns
14
16
18
20
2
4
6
8
10 12 columns
14
16
18
20
Figure 11: Intensities from illumination distances of a) 10 km and b) 40 km
The results achieved by the recurrent networks are shown in table 3 (using a threshold of 0.5). It can be seen that the performance is clearly better for shorter illumination distances and very good results are achieved for the 10 km distance. Results for the 40 km distance are significantly worse, but still satisfactory for higher sea states. Sea state Oil spills detected (10 km)
False alarms (10 km)
Oil spills detected (40 km)
False alarms (40 km)
1
70 (100%)
11
42 (60%)
>> 20
2
70 (100%)
2
51 (73%)
18
3
70 (100%)
0
55 (79%)
3
4
70 (100%)
0
56 (80%)
0
5
70 (100%)
1
53 (75%)
2
6
70 (100%)
0
58 (83%)
1
Table 3: Performance of 10 ANNs trained on 50% of the environment (sea state 3, 20 km), tested on the whole environment in all sea states, illuminations distances 10 and 40 km
11
The results for an illumination distance of 40 km may look quite negative, but it should be noted that for sea states 3 to 6 most of the networks, although being trained on 20 km distance data, do detect three or four of the five small oil spills while not generating any false alarm. Typical binary output maps (sea state 3, network trained on the left half, threshold of 0.5) for both distances are shown in figure 12. Binary output map (40 km, threshold 0.5)
70
60
60
rows (flight direction >>)
rows (flight direction >>)
Binary output map (10 km, threshold 0.5)
70
50
40
30
50
40
30
20
20
10
10
2
4
6
8
10 12 columns
14
16
18
20
2
4
6
8
10 12 columns
14
16
18
20
Figure 12: Binary output maps (threshold 0.5) for distances of a) 10 km and b) 40 km
3.5 Analysis What has been shown so far is that the segmentation into oil and water areas does work (with some limitations), moreover the recurrent networks used here seem to be very robust to noise and variations in sea state and illumination distance. So, the obvious question is: How does it work? To some extent this question can be answered by looking at the networks' connection weights. Figure 13 shows the input-to-hidden (IH) weights for a typical network (weights are similar for other networks), which to some extent reflect the individual input values' relevance. The input units were dedicated as follows (odd numbers: normalised intensity, even numbers: normalised spectrum width): 1,2 cell to be classified; 3,4 - previous cell (in the sequence of cells in flight direction); 5,6 - next cell; 7,8 left neighbour cell; 9,10 - right neighbour cell. Columns 11 to 15 represent the connection weights between context and hidden layer.
1
2
3
4
5
1
3
5
7 9 Input and context units
12
11
15
Figure 13: Input-to-hidden-layer (IH) weights
The multiplication of the IH and hidden-to-output-layer (HO) weight matrices results in the following vector. This vector does of course not exactly quantify the individual input values' relevance for the classification output, as it disregards the influence of biases and activation functions, nevertheless it can be used as an indicator of the input units' impact on the output unit.
1
1
3
5
7 9 Input and context units
11
15
Figure 14: Matrix product of IH and HO weights
Figures 13 and 14 indicate that the recurrent network during training has learned to weight its connections such that the intensity values ('odd columns') have a significantly larger relevance for the classification than the spectrum width values, which corresponds to the fact that the intensity map (cf. figure 4) 'looks clearer'; it is the cell to be classified (columns 1,2) that has the highest impact on its own classification, followed by previous and next cell (almost equal), again followed by left and right cell; this pattern is valid for both intensity and spectrum width values and corresponds to the 'shorter distance' between cells in flight direction than along the x-axis. This is of course a rather informal and incomplete analysis, as it disregards the role of the context units, nevertheless it shows that the network has found its own 'reasonable' model of its input values´s individual relevance for the classification, which to some extent explains the good generalisation and robustness achieved in the experiments discussed here.
4 Conclusion A recurrent ANN architecture, similar to that of Elman's SRN (1990), for the detection of oils spills based on a segmentation of Doppler radar images into oil and water areas has been presented in this
13
paper. It has been shown that networks of this recurrent architecture have been successfully trained (using simulated radar measurements) such that a high detection reliability and sensitivity, even to relatively small oil spills, has been achieved; the number of false alarms could be kept very low, i.e. a low sensitivity to noise has been achieved; the networks are very robust to variations in wind conditions and illumination distances for sea states 3 to 6 (and, with some limitations, 2). The computational costs of the technique presented here are rather low as the only preprocessing required is the calculation of average intensity and spectrum width per resolution cell, i.e. the presented technique allows a real-time detection of oil spills. Hence, the results reported in this paper can be considered very satisfactory. The degree of robustness and generalisation achieved here across a large variety of simulated sea conditions is very promising for an application of this method under real world conditions. The approach taken here, to have an ANN generate a rather clear output map from two noisy input maps, is a rather general one as it could be applicable to a number of other pattern recognition problems.
Acknowledgement(s) The work presented in this paper is part of a long-term joint project on ANNs and radar signal processing between The Connectionist Research Group (CRG), University of Skövde, and Ericsson Microwave Systems, Mölndal/Sweden. The CRG's work on this project is supported by the Swedish Department of Education.
References Ahalt, S. C., Garber, F. D., Jouny, I., and Krishnamurthy, A. K. (1989). Performance of Synthetic Neural Network Classification of Noisy Radar Signals, In: Touretzky, David S., Ed., Advances in neural information processing systems I, Morgan Kauffmann, San Mateo Demuth, Howard and Beale, Mark (1992). Neural Network Toolbox User's Guide, The MathWorks, Inc., Natick, MA. Elman, Jeffrey L. (1990). Finding Structure in Time, Cognitive Science, 14, 179 - 211. Fälldin, Björn (1993). SLAR, Side Looking Airborne Radar - Signal Processing, Design and Evaluation, Master's Thesis, Chalmers Technical University, Gothenburg, Sweden
14
Jordan, Michael I. (1986). Attractor dynamics and parallelism in a connectionist sequential machine, In: Proceedings of the Eighth Conference of the Cognitive Science Society, 531-546. Kosko, Bart (1992). Neural networks for signal processing, Prentice Hall, Englewood Cliffs Martinez Madrid, Juan J., Casar Corredera, Jose R. & de Miguel Vela, G. (1992). A Neural Network approach to Doppler-based target classification, In: Proceedings of the IEEE International Radar Conference 'RADAR 92', 450-453. Ziemke, Tom and Athley, Fredrik (1995a). Connectionist Models for the Detection of Oil Spills from Doppler Radar Imagery, In: Niklasson, Lars F. and Bodén, Mikael B., Eds., Current Trends in Connectionism, Lawrence Erlbaum Associates, 355-370. Ziemke, Tom and Athley, Fredrik (1995b). Oil Spill Detection from Doppler Radar Imagery using Artificial Neural Networks, In: Bulsari, A.B. and Kallio, S., Eds., Engineering Applications of Neural Networks - Proceedings of the International Conference EANN '95, Finnish Artificial Intelligence Society, 83-86. Ziemke, Tom (1995). Recurrent Artificial Neural Networks for the Detection of Oil Spills from Doppler Radar Imagery, In: Keating, John G., Ed., Neural Computing Research III - Proceedings of the Irish Neural Networks Conference (INNC '95), St. Patrick's College, Maynooth, Co. Kildare, Ireland, 54-61.
15
Tom Ziemke The Connectionist Research Group Dept. of Computer Science University of Skövde S-54128 Skövde/Sweden [email protected] December 1995
Abstract This paper discusses the application of artificial neural networks to the segmentation of Doppler radar images, in particular the detection of oil spills within sea environments, based on a classification of radar backscatter signals. Best results have been achieved with recurrent backpropagation networks of an architecture similar to that of Elman's Simple Recurrent Network (1990). These recurrent networks are shown to be very robust to variations in both sea state (weather conditions) as well as illumination distance, and their performance is analysed in further detail.
Keywords:
1
oil spill detection, radar image segmentation, recurrent artificial neural networks
to appear in a special issue 'Artificial Neural Networks in Computer Vision' of the journal Pattern Recognition Letters (to be published around May '96, paper will be retyped and -formatted)
1
1 Introduction The design of systems for the identification of objects from the measurements of their radar backscatter signals has traditionally been based upon mathematical/decision-theoretic methods of pattern recognition. The applicability of such methods, due to their mathematical nature, crucially depends on the availability of sufficiently accurate models for the statistical properties of the radar measurements (Ahalt et al., 1989). The problem domain discussed here however, the segmentation of Doppler radar images of sea environments into oil and water areas, is characterised by the lack of such formal knowledge. Therefore artificial neural networks (ANNs), due to their 'model-freedom', offer an attractive alternative to this problem, since they are able to learn to perform the classification from labelled training data, and do not require a-priori knowledge of statistical models. Model-freedom lifts us from the oldest burden of modern science - guessing at cause-and-effect relationships with functions, and with functions so simple and so tractable that our recently evolved brain can understand them and manipulate them in symbols. (Kosko, 1992)
2 Problem Domain 2.1
General idea
The overall goal of this work is the development of a reliable and robust technique for the segmentation of images obtained from the measurements of so-called Side Looking Airborne Radar (SLAR, a conventional Doppler radar illuminating a sea environment from a plane) into oil and water areas.
Figure 1: Backscattered energy from water and oil
The discrimination of oil from water is theoretically possible due to the fact that oil dampens the capillary waves present on a sea surface, such that the surface becomes smoother and acts more like a mirror (as illustrated in figure 1), i.e. more energy will be reflected away from the radar by an oil covered surface than by pure water.
2
In practice however, this task, and in particular the detection of small oil spills, is further complicated by the noise and other problems inherent in the nature of radar measurements.
2.2 Doppler radar The SLAR mentioned above is a Doppler radar, i.e. it detects objects by virtue of their velocity with respect to the radar itself. Doppler filtering makes use of the progressive phase shift (Doppler shift) of an object's return signal on successively transmitted pulses, which is caused by the object's motion towards or away from the radar. The radar echo from a target area consists of the returns of a multitude of point scatterers forming a spectrum of individual phase shifts, the so-called Doppler spectrum or Doppler signature. Hence, beside the mean radial velocity, depending on the waves' translational motion (which can be assumed to be the same for oil and water), the spectral shape and width to some degree reflect the 'internal dynamics' of an illuminated area/object. The ability to perform a, to some extent, automatic classification of illuminated objects/areas using a conventional Doppler radar could be considered a significant increase in the capabilities of the radar sensor (Martinez Madrid et al., 1992). The differences in oil's and water's backscatter signals become apparent in their Doppler spectra, as illustrated in figure 2, such that in this case two relevant values can be extracted from the Doppler spectrum and will be used as input values for a classification network :
Figure 2: Typical Doppler spectra of oil and water (illumination distance 20 km)
Intensity, reflecting the magnitude of the backscattered energy, which, as explained and illustrated above, is lower for oil than for water. Spectrum width, i.e. the velocity spread of the individual point scatterers, which, due to the waves' lesser internal motion, is also lower for oil than for water.
3
It should be noted that the absolute values cannot be used for a discrimination of oil from water as they vary with varying wind conditions and illumination distances (s. table 1 for details).
2.3 SLAR simulation model In this study a simulation model of an SLAR, originally developed by Ericsson Microwave Systems, Mölndal/Sweden, (Fälldin, 1993), has been used. This is due to the fact that the radar simulated here in reality is still under development. Due to the SLAR's technical specification the radar's resolution (i.e. the resolution of the radar images) is such that cells of 20 m (flight direction) * 75 m are illuminated. The SLAR model simulates the measurements of 64 Doppler channels for each resolution cell, such that one Doppler spectrum (as shown in figure 2) can be obtained for each cell, from which values for intensity and spectrum width can be calculated for each individual cell.
2.4 Detection of small oil spills - The real problem The difficulty of discriminating oil from water in Doppler radar images is probably illustrated best with an example. Figure 3 shows an environment of 1500m * 1500m, corresponding to 75 * 20 cells in the radar's resolution, containing two large and five small oil spills (dark areas). The left and right half are here separated by a line as they will later be used as independent test and training sets. Environment map (radar resolution)
70
rows (flight direction >>)
60
50
40
30
20
10
2
4
6
8
10 12 columns
14
16
18
20
Figure 3: Model environment
Measurements of intensity and spectrum width within this environment are shown in figure 4 (illumination distance 20 km, sea state 3, cf. later). Both maps show peaks of low values (dark areas) where the large oil spills have their greatest widths (in flight direction), but the small oil spills cannot be discriminated reliably from the noise.
4
Spectrum width map
70
60
60 rows (flight direction >>)
rows (flight direction >>)
Intensity map
70
50
40
30
50
40
30
20
20
10
10
2
4
6
8
10 12 columns
14
16
18
20
2
4
6
8
10 12 columns
14
16
18
20
Figure 4: Measurements of a) intensity and b) spectrum width in the model environment
The absolute measurements are normalised with regard to different sea states by calculating the relative deviations from reference values characteristic for a certain sea state. These reference values can be calculated as mean values of intensity and spectrum width from an area known to be water, in this case the 100 cells in rows 1 to 5, such that the normalised measurements are obtained as follows: 2 intensity deviation = 10 ^ ( (measured intensity - reference intensity) / 10 ) - 1 spectrum width deviation = (measured spectrum width - reference spectrum width) / ref. spectrum width The resulting normalised vectors for the above environment together with their (correct) binary classification (using a threshold of 0.5, i.e. 50% oil in a resolution cell) look as shown in figure 5a. Resolution cell vectors ( o : oil, + : water ) − columns 7 to 15 0.4
0.3
0.3
0.2
0.2
0.1
0.1
Spectrum width deviation (rel.)
Spectrum width deviation (rel.)
Resolution cell vectors ( o : oil, + : water ) 0.4
0 −0.1 −0.2 −0.3 −0.4
0 −0.1 −0.2 −0.3 −0.4
−0.5
−0.5
−0.6
−0.6
−1
−0.8
−0.6
−0.4 −0.2 0 Intensity deviation (rel.)
0.2
0.4
−1
0.6
−0.8
−0.6
−0.4 −0.2 0 Intensity deviation (rel.)
0.2
0.4
Figure 5: Normalised resolution cell vectors and their binary classification, a) whole environment and b) columns 7 to 15
Hence, the resolution cell vectors show the expected distribution: The water cells cluster around the (water) reference values, i.e. around point (0,0). The oil cells mostly have both lower intensity and spectrum width values. 2
NB: dB is a logarithmic measure, therefore the above normalisation formula for the intensity deviation
5
0.6
But there is a large 'overlap area', that contains both kinds of cells, such that oil and water vectors are definitely not linearly separable. This area mostly contains the vectors of the small oil spill cells, such that a reliable classification of an individual cell only on basis of its own values is not possible. This is further illustrated in figure 5b which shows the vectors of columns 7 to 15, an area containing only small oil spills. So, what is it that makes the detection of small oil spills so difficult? Apart from the fact that radar measurements, by their nature, are noisy, the task is further complicated by the fact that the radar cannot focus on a single resolution cell. That means that at every time step the radar echo does not consist only of the returns of one resolution cell, but those from a much larger area 3. The practically relevant area should be approximately that within the 6 dB beam width4 of 88m, the SLAR model used here however simulates measurements up to the 24 dB beam width of 349m. This means that the measurements of a particular cell are always influenced by the returns of a number of neighbour cells. Therefore the radar echo of a single resolution cell containing oil (itself only being 20m * 75 m in size) will actually only be a 'pure oil' echo if the whole area covered by the relevant radar beam contains oil. This has the effect that at the border of an oil spill both returns of oil and water are measured (as illustrated in figure 6), such that small oil spills (only consisting of 'borders') can 'get lost' in their water neighbourhood. 6 dB radar beam(s)
oil spill
flight direction >>
88 m 6 dB beam width
20 m
illumination distance 20 km
Figure 6: Beam width vs. cell size
2.5 Goals The task of detecting oil spills from radar imagery is usually carried out by visual inspection through human operators. This procedure is a) difficult due to the slight differences between oil's and water's
3
4
Actually this area is theoretically infinitely large. The SLAR model simulates returns up to the 24 dB beam width, being measured where the gain has fallen 24 dB below maximum. At a distance of 20 km the 24 dB beam width is 349 m. The 6 dB beam, being measured where the gain has fallen 6 dB below maximum, is 88m at 20 km distance.
6
backscatter signals and b) inefficient and not possible in real-time due to the large amount of data obtained. Therefore goal of this work is to develop an ANN technique that allows an automatic segmentation of radar images as those shown above (cf. figure 4) into oil and water areas, based on a classification of individual resolution cells. Hence, the main requirements are as follows: The detection should be reliable, i.e. it should be sensitive to oil spills as small as possible. The number of false alarms should be kept to a minimum, i.e. it should be insensitive to noise in the radar measurements. In order to be applicable under real world conditions the segmentation method has to be be robust, e.g. to variations in sea states/wind conditions as well as illumination distance.
3 Experiments 3.1 General approach / previous work It has been pointed out earlier that the classification of individual resolution cells cannot be performed reliably (with regard to the detection of small oil spills) only on basis of a single cell's own values. It has been shown in earlier work (Ziemke and Athley, 1995a) that best classification performance is achieved with ANNs taking as input the values of intensity and spectrum width for the cell to be classified itself (C) and its four direct neighbours (N), as shown in figure 7.
flight direction >>
75 m
20 m
N N
C
N
N
Figure 7: Neighbourhood context
Furthermore in earlier experiments with different ANN architectures (Learning Vector Quantization (LVQ), feed-forward backpropagation networks and recurrent backpropagation networks of two different architectures, similar to those used by Jordan (1986) and Elman (1990)) best performance was achieved with recurrent networks of an architecture similar to Elman's Simple Recurrent Network (SRN). It has been shown that these recurrent networks enable a reliable and robust detection of oil spills down to a size of about 50 m * 50 m (Ziemke and Athley, 1995b, Ziemke, 1995). Further experiments with this architecture are documented in this paper.
7
3.2 Architecture(s) As mentioned above, the architecture used here is a recurrent network similar to the SRN as proposed by Elman5 (1990), i.e. a three-layer backpropagation network with weighted, fully-connected (indirect) feedback from the hidden layer to itself : output unit (1)
fully connected copy connection
t hidden units (5) t+1
t t
context units (5)
input units (10)
Figure 8: Recurrent architecture
As discussed earlier, the networks discussed here have been trained to individually classify each resolution cell within an environment. The target output values have been taken from the oil spill maps as shown in figure 3, therefore one output unit is used. The networks use as input values the normalised intensity and spectrum width values of the cell to be classified and its four direct neighbours (cf. figure 7), as shown in the intensity and spectrum width maps in figure 4, therefore ten input units are used. Best performance has been achieved with five hidden units. The networks have been trained with a 'fast' backpropagation algorithm using momentum and an adaptive learning rate (Demuth and Beale, 1992), using the logistic activation function for hidden and output units (earlier experiments with linear activation functions (Ziemke and Athley, 1995a) proved less successful). This means that the networks are trained to generate an estimation of the degree to which each cell is covered by oil. The desired binary classification into oil and water is achieved by simply thresholding these estimates with 0.5 afterwards. These recurrent networks have been trained to individually classify sequences of resolution cells in flight direction (including their direct neighbours). So the general approach here is to make use of the context of the previous cell's classification, which is based on the fact that, as illustrated in figure 6, it is not possible to measure a clear water echo in one time step and a clear oil echo in the next time step. Instead the transition at the border from water to oil there will normally be a sequence of 4-5 time steps during which oil's 'share' of the returns grows from 0% to 100%. Correspondingly, there will be a similar sequence of time steps at the border from oil back to water.
5
Elman trained his SRNs t o predict the next element in a sequence presented to the network, therefore input and output layer were of the same size.
8
Resolution cell vectors ( o : oil, + : water ) − sea state 6 0.4
0.3
0.3
0.2
0.2
0.1
0.1
Spectrum width deviation (rel.)
Spectrum width deviation (rel.)
Resolution cell vectors ( o : oil, + : water ) − sea state 1 0.4
0 −0.1 −0.2 −0.3 −0.4
0 −0.1 −0.2 −0.3 −0.4
−0.5
−0.5
−0.6
−0.6
−1
−0.8
−0.6
−0.4 −0.2 0 Intensity deviation (rel.)
0.2
0.4
−1
0.6
−0.8
−0.6
−0.4 −0.2 0 Intensity deviation (rel.)
0.2
0.4
0.6
Hence, this recurrent architecture aims at capturing the information contained in the temporal sequence of changes over a few time steps, from water to oil echo and the other way round, and at exploiting this sequential information contained in this transition for the classification task.
3.3 Robustness to varying sea states For an evaluation of the ANNs' robustness to varying wind/weather conditions data for the following six sea states has been used in the experiments discussed here. Sea state
Wind velocity [m/s]
Approx. radial velocity [m/s]
Wave height [m]
Approx. spectrum width water1 [m/s]
RCS water2 [dB]
1
1-3
0.2
0.1 - 0.3
0.3
- 45
2
3-6
0.7
0.3 - 0.9
0.6
- 41
3
6-8
1
0.9 - 1.5
0.9
- 38
4
8 - 10
1.4
1.5 - 2.4
1.2
- 35
5
10 - 13
1.7
2.4 - 3.6
1.6
- 33
6
13 - 15
2.1
3.6 - 6.0
1.9
- 28
1
The spectrum width of oil is assumed to be 50% of that of water. (Experiments with values of 30% and 70% were equally successful, showing that this assumption is not critical.)
2
RCS (Radar Cross Section) is proportional to intensity. Values given for 20 km idistance, values for 10 km are up to 6 dB higher, those for 40 km up to 4 dB lower. At an illumination distance of 10/20/40 km oil's RCS can be assumed to be about 8/6/5 dB lower than that of water.
Table 1: Sea states
In all sea states the normalized measurements show a similar distribution (into two clusters plus an overlap area) as in sea state 3 (cf. figure 5), but in lower sea states the values spread out more whereas in higher sea states the clustering is even stronger. This can be seen pretty clearly in figure 9.
Figure 9: Resolution cell vectors in a) sea state 1 and b) sea state 6
To evaluate the ANNs' robustness to variations in sea states/wind conditions, ten recurrent networks have been trained, five of them on the left half, five of them on the right half of the above environment
9
Binary output map (threshold 0.5)
70
60
60
rows (flight direction >>)
rows (flight direction >>)
Continuous output map
70
50
40
30
50
40
30
20
20
10
10
2
4
6
8
10 12 columns
14
16
18
20
2
4
6
8
10 12 columns
14
16
18
20
(sea state 3, illumination distance 20 km). All networks have been trained for 500 epochs, which showed best results in initial experiments. These networks have been tested on the whole environment in all sea states. It has however turned out that the results obtained for sea state 1 were dramatically worse than those for other sea states. This can be explained by the fact that for very low waves (as in sea state 1) the rather small differences between oil and water can easily 'disappear' in the noise. The results (using a threshold of 0.5) are summarised in table 2. Sea state
Oil spills detected
False alarms
1
64 (91%)
> 20
2
68 (97%)
6
3
70 (100%)
0
4
70 (100%)
1
5
69 (99%)
1
6
70 (100%)
0
Table 2: Performance of 10 ANNs trained on 50% of the environment (sea state 3), tested on the whole environment in all sea states
It can be seen that, although the networks have only been trained for one particular sea state (3), the classification performance for sea states 3 to 6 is almost perfect, but performance decreases significantly for the lower states. The following figure shows the continuous output values (sea state 3) computed by one of these networks (trained on the left half) as well as the resulting binary classification (again using a threshold of 0.5). Intensity map (40 km)
70
70
60
60
rows (flight direction >>)
rows (flight direction >>)
Intensity map (10 km)
50
40
30
50
40
30
20
20
10
10
2
4
6
8
10 12 columns
14
16
18
20
2
4
6
8
10 12 columns
14
16
18
Figure 10: a) Continuous output map , b) resulting binary classification
10
20
When comparing these maps to the correct environment map (in figure 3) it can be seen that the output as generated by the recurrent network actually comes very close to the original.
3.4 Robustness to varying illuminations distances To evaluate the ANNs' robustness to varying illumination distances the same ten recurrent networks as above, each trained on one half of the radar images taken from a distance of 20 km (sea state 3), have been tested on data obtained from measurements from illumination distances of 10 and 40 km in all sea states. A halving/doubling of the illumination distance also leads to a halving/doubling of the (relevant) beam width. Hence, the images obtained from a shorter distance should be clearer and the detection/segmentation should be easier, and correspondingly for a longer illumination distance the detection of small oil spills should become more difficult. This effect is illustrated in figure 11, which shows the normalised measurements of intensity from distances of 10 and 40 km (for 20 km cf. figure 4).
Binary output map (40 km, threshold 0.5)
70
70
60
60
rows (flight direction >>)
rows (flight direction >>)
Binary output map (10 km, threshold 0.5)
50
40
30
50
40
30
20
20
10
10
2
4
6
8
10 12 columns
14
16
18
20
2
4
6
8
10 12 columns
14
16
18
20
Figure 11: Intensities from illumination distances of a) 10 km and b) 40 km
The results achieved by the recurrent networks are shown in table 3 (using a threshold of 0.5). It can be seen that the performance is clearly better for shorter illumination distances and very good results are achieved for the 10 km distance. Results for the 40 km distance are significantly worse, but still satisfactory for higher sea states. Sea state Oil spills detected (10 km)
False alarms (10 km)
Oil spills detected (40 km)
False alarms (40 km)
1
70 (100%)
11
42 (60%)
>> 20
2
70 (100%)
2
51 (73%)
18
3
70 (100%)
0
55 (79%)
3
4
70 (100%)
0
56 (80%)
0
5
70 (100%)
1
53 (75%)
2
6
70 (100%)
0
58 (83%)
1
Table 3: Performance of 10 ANNs trained on 50% of the environment (sea state 3, 20 km), tested on the whole environment in all sea states, illuminations distances 10 and 40 km
11
The results for an illumination distance of 40 km may look quite negative, but it should be noted that for sea states 3 to 6 most of the networks, although being trained on 20 km distance data, do detect three or four of the five small oil spills while not generating any false alarm. Typical binary output maps (sea state 3, network trained on the left half, threshold of 0.5) for both distances are shown in figure 12. Binary output map (40 km, threshold 0.5)
70
60
60
rows (flight direction >>)
rows (flight direction >>)
Binary output map (10 km, threshold 0.5)
70
50
40
30
50
40
30
20
20
10
10
2
4
6
8
10 12 columns
14
16
18
20
2
4
6
8
10 12 columns
14
16
18
20
Figure 12: Binary output maps (threshold 0.5) for distances of a) 10 km and b) 40 km
3.5 Analysis What has been shown so far is that the segmentation into oil and water areas does work (with some limitations), moreover the recurrent networks used here seem to be very robust to noise and variations in sea state and illumination distance. So, the obvious question is: How does it work? To some extent this question can be answered by looking at the networks' connection weights. Figure 13 shows the input-to-hidden (IH) weights for a typical network (weights are similar for other networks), which to some extent reflect the individual input values' relevance. The input units were dedicated as follows (odd numbers: normalised intensity, even numbers: normalised spectrum width): 1,2 cell to be classified; 3,4 - previous cell (in the sequence of cells in flight direction); 5,6 - next cell; 7,8 left neighbour cell; 9,10 - right neighbour cell. Columns 11 to 15 represent the connection weights between context and hidden layer.
1
2
3
4
5
1
3
5
7 9 Input and context units
12
11
15
Figure 13: Input-to-hidden-layer (IH) weights
The multiplication of the IH and hidden-to-output-layer (HO) weight matrices results in the following vector. This vector does of course not exactly quantify the individual input values' relevance for the classification output, as it disregards the influence of biases and activation functions, nevertheless it can be used as an indicator of the input units' impact on the output unit.
1
1
3
5
7 9 Input and context units
11
15
Figure 14: Matrix product of IH and HO weights
Figures 13 and 14 indicate that the recurrent network during training has learned to weight its connections such that the intensity values ('odd columns') have a significantly larger relevance for the classification than the spectrum width values, which corresponds to the fact that the intensity map (cf. figure 4) 'looks clearer'; it is the cell to be classified (columns 1,2) that has the highest impact on its own classification, followed by previous and next cell (almost equal), again followed by left and right cell; this pattern is valid for both intensity and spectrum width values and corresponds to the 'shorter distance' between cells in flight direction than along the x-axis. This is of course a rather informal and incomplete analysis, as it disregards the role of the context units, nevertheless it shows that the network has found its own 'reasonable' model of its input values´s individual relevance for the classification, which to some extent explains the good generalisation and robustness achieved in the experiments discussed here.
4 Conclusion A recurrent ANN architecture, similar to that of Elman's SRN (1990), for the detection of oils spills based on a segmentation of Doppler radar images into oil and water areas has been presented in this
13
paper. It has been shown that networks of this recurrent architecture have been successfully trained (using simulated radar measurements) such that a high detection reliability and sensitivity, even to relatively small oil spills, has been achieved; the number of false alarms could be kept very low, i.e. a low sensitivity to noise has been achieved; the networks are very robust to variations in wind conditions and illumination distances for sea states 3 to 6 (and, with some limitations, 2). The computational costs of the technique presented here are rather low as the only preprocessing required is the calculation of average intensity and spectrum width per resolution cell, i.e. the presented technique allows a real-time detection of oil spills. Hence, the results reported in this paper can be considered very satisfactory. The degree of robustness and generalisation achieved here across a large variety of simulated sea conditions is very promising for an application of this method under real world conditions. The approach taken here, to have an ANN generate a rather clear output map from two noisy input maps, is a rather general one as it could be applicable to a number of other pattern recognition problems.
Acknowledgement(s) The work presented in this paper is part of a long-term joint project on ANNs and radar signal processing between The Connectionist Research Group (CRG), University of Skövde, and Ericsson Microwave Systems, Mölndal/Sweden. The CRG's work on this project is supported by the Swedish Department of Education.
References Ahalt, S. C., Garber, F. D., Jouny, I., and Krishnamurthy, A. K. (1989). Performance of Synthetic Neural Network Classification of Noisy Radar Signals, In: Touretzky, David S., Ed., Advances in neural information processing systems I, Morgan Kauffmann, San Mateo Demuth, Howard and Beale, Mark (1992). Neural Network Toolbox User's Guide, The MathWorks, Inc., Natick, MA. Elman, Jeffrey L. (1990). Finding Structure in Time, Cognitive Science, 14, 179 - 211. Fälldin, Björn (1993). SLAR, Side Looking Airborne Radar - Signal Processing, Design and Evaluation, Master's Thesis, Chalmers Technical University, Gothenburg, Sweden
14
Jordan, Michael I. (1986). Attractor dynamics and parallelism in a connectionist sequential machine, In: Proceedings of the Eighth Conference of the Cognitive Science Society, 531-546. Kosko, Bart (1992). Neural networks for signal processing, Prentice Hall, Englewood Cliffs Martinez Madrid, Juan J., Casar Corredera, Jose R. & de Miguel Vela, G. (1992). A Neural Network approach to Doppler-based target classification, In: Proceedings of the IEEE International Radar Conference 'RADAR 92', 450-453. Ziemke, Tom and Athley, Fredrik (1995a). Connectionist Models for the Detection of Oil Spills from Doppler Radar Imagery, In: Niklasson, Lars F. and Bodén, Mikael B., Eds., Current Trends in Connectionism, Lawrence Erlbaum Associates, 355-370. Ziemke, Tom and Athley, Fredrik (1995b). Oil Spill Detection from Doppler Radar Imagery using Artificial Neural Networks, In: Bulsari, A.B. and Kallio, S., Eds., Engineering Applications of Neural Networks - Proceedings of the International Conference EANN '95, Finnish Artificial Intelligence Society, 83-86. Ziemke, Tom (1995). Recurrent Artificial Neural Networks for the Detection of Oil Spills from Doppler Radar Imagery, In: Keating, John G., Ed., Neural Computing Research III - Proceedings of the Irish Neural Networks Conference (INNC '95), St. Patrick's College, Maynooth, Co. Kildare, Ireland, 54-61.
15
