A Swarm Intelligence inspired algorithm for contour detection in images
Applied Soft Computing 13 (2013) 3118–3129
Contents lists available at SciVerse ScienceDirect
Applied Soft Computing journal homepage: www.elsevier.com/locate/asoc
A Swarm Intelligence inspired algorithm for contour detection in images Ulrich Kirchmaier ∗ , Simon Hawe, Klaus Diepold Technische Universität München, Arcisstr. 21, 80290 Munich, Germany
a r t i c l e
i n f o
Article history: Received 31 December 2011 Received in revised form 2 July 2012 Accepted 11 July 2012 Available online 25 July 2012 Keywords: Contour detection Boundary detection Image segmentation Image processing Swarm Intelligence
a b s t r a c t Swarm Intelligence uses a set of agents which are able to move and gather local information in a search space and utilize communication, limited memory, and intelligence for problem solving. In this work, we present an agent-based algorithm which is specifically tailored to detect contours in images. Following a novel movement and communication scheme, the agents are able to position themselves distributed over the entire image to cover all important image positions. To generate global contours, the agents examine the local windowed image information, and based on a set of fitness functions and via communicating with each other, they establish connections. Instead of a centralized paradigm, the global solution is discovered by some principal rules each agent is following. The algorithm is independent of object models or training steps. In our evaluation we focus on boundary detection as a major step towards image segmentation. We therefore evaluate our algorithm using the Berkeley Segmentation Dataset (BSDS) and compare its performance to existing methods via the BSDS benchmark and Pratt’s Figure of Merit. © 2012 Published by Elsevier B.V.
1. Introduction Boundary Detection is commonly regarded as a major step towards Image Segmentation, which is a central problem in computer vision that has been studied for years. Image Segmentation again constitutes an important initial step for high-level tasks in computer vision, like object recognition, image analysis, and scene understanding. It pursues the goal of dividing an image into parts, regions, or objects, which are preferably self-consistent and of meaningful content. Application examples include separating objects considered as foreground from the background, or separating multiple objects from each other. One of the main difficulties in the field of boundary detection and image segmentation arises from the fact, that not even human beings are able to uniquely and unambiguously decide and agree over a correct segmentation [1]. Different persons are likely to have different opinions on the correct solution to the segmentation problem of any image, especially involving natural images. This lack of an unique ground-truth complicates the comparison of results descending form different segmentation methods. Nevertheless, efforts were taken to establish a framework for objective evaluation of segmentation algorithms. A prominent example is the Berkeley Segmentation Dataset (BSDS) [1], which provides a huge collection of natural images along with human-marked proposals of ground-truth, as well as a precision-recall based measurement
∗ Corresponding author. E-mail address: [email protected] (U. Kirchmaier). 1568-4946/$ – see front matter © 2012 Published by Elsevier B.V. http://dx.doi.org/10.1016/j.asoc.2012.07.017
for comparison [2]. Fig. 1 shows three example images along with three different human ground-truth suggestions. The plenty of solutions to the segmentation problem proposed in the literature descend from versatile fields, such as clustering [4], graph theory [5,6], region growing [7], optimization of an evaluation function like energy minimization [8], as well as image feature-based methods [9,10]. The majority of approaches found in the literature can be categorized into one or a subset of these fields, with each adding improved mechanisms to solve specific tasks. Boundary or contour detection is often performed as an initial step for image segmentation, as boundaries naturally correspond to borders between objects. It describes the process of identifying and locating sharp discontinuities in image regions, which closely relates it to the task of edge detection. Mere edge detection, however, registers on the one hand any type of abrupt changes in image brightness, and misses smooth or subtle transitions of image brightness on the other hand. Regarding the goal of detecting boundaries, this causes false positives like textured regions, or single highlighted spots and true negatives like region boundaries of low contrast. One recipe to overcome these drawbacks is to combine edge detection techniques with diverse soft computing approaches like Genetic Algorithms, or Fuzzy Logic, etc. [11,12]. Other methods combine oriented local image cues, like brightness, color, and texture descriptors with edge detection to get a global estimation. Such combination methods include multiscale cue combination [3], energy and probability weighting [13], or likelihood-methods [14].
U. Kirchmaier et al. / Applied Soft Computing 13 (2013) 3118–3129
3119
Fig. 1. Three sample images from the Berkeley Image Segmentation Dataset (BSDS 500) [3], each along with three different human-marked ground-truth images.
In this paper, we propose an algorithm that uses the concept of Swarm Intelligence to detect contours in images. We believe that the idea of interacting distributed agents to migrate from local information to global boundary estimates is a promising alternative to existing methods. While a variety of Swarm Intelligence inspired algorithms already exists (refer to Section 2), their swarms often seek to find one global optimal solution in a search space, as these algorithms descend from the field of optimization. However, it is our goal to detect a non-predefined number of boundaries with arbitrary positions, sizes, and shapes in an image, which all have to be considered as equal or even independent solutions. Therefore, we established a novel algorithm based on distributed agents being explicitly tailored to the demands of contour detection. The main contribution of this paper are the development of (1) a novel movement and positioning rule, along with (2) local information gathering mechanisms, and (3) a set of fitness functions and communication schemes. This paper is organized as follows: Section 2 gives an overview on approaches in the fields of image segmentation and contour detection, which utilize Swarm Intelligence. Section 3 investigates the principles of Swarm Intelligence and Section 4 provides details on our algorithm. Section 5 shows results of our algorithm tested on the task of boundary detection in natural images.
2. Related work In recent years, algorithms descending from the field of Evolutionary Computing, Genetic Algorithms, and also Swarm Intelligence are increasingly applied to the image segmentation problem. These optimization tools have mostly been used to improve the performance of important steps of segmentation algorithms. In [15,16], evolutionary and genetic algorithms were used to enhance the clustering process in segmentation. In [17], the authors combined high-level features generated with a visual
attention model with low-level features to guide region growing algorithm, where the optimal thresholds of the region growing process were detected using the Particle Swarm Optimization (PSO) algorithm. In [18], the PSO was applied to find the optimal fuzzy entropy thresholds to segment images into foreground and background. A related approach was introduced in [19], where PSO was used to tune thresholds in 2D-histograms, maximizing the entropy to segment infrared images. SI was also utilized to improve Segmentation based on clustering approaches, which can get stuck in local optima, depending on their initialization. In [20], the authors developed a hybrid combination of the Ant Colony Optimization algorithm (ACO) and PSO to make K-Means clustering and Single Competitive Learning more independent from their initial cluster centers, and learning rate, respectively. Besides deploying SI techniques to optimize specific steps in existing segmentation approaches, SI algorithms were also applied to the tasks of edge detection and contour detection, akin to our proposed algorithm. The authors of [21] proposed ant-based correlation for edge detection. Their method is capable of performing feature extraction for edge detection and segmentation, generating less distortion in the presence of noise, as compared to classical edge detectors, like Sobel, Prewitt, and Canny. In [22], a variant of the PSO algorithm is utilized for finding a proper edge detection filter size in noisy images. While combining edge filters of different sizes can improve the edge detection in noisy images, it is generally computationally expensive when applied to the entire image. Thus, the heuristic capabilities of PSO were used to adaptively decide, where on the image to use filters of various sizes. The evaluation of edge intensity and the proposed movement principle in this approach has analogies to our algorithm, as they divide the edge regions into sets and use averaged intensities. However, the major difference is the usage of discrete PSO instead of our communication scheme where the agents share their local information.
Contents lists available at SciVerse ScienceDirect
Applied Soft Computing journal homepage: www.elsevier.com/locate/asoc
A Swarm Intelligence inspired algorithm for contour detection in images Ulrich Kirchmaier ∗ , Simon Hawe, Klaus Diepold Technische Universität München, Arcisstr. 21, 80290 Munich, Germany
a r t i c l e
i n f o
Article history: Received 31 December 2011 Received in revised form 2 July 2012 Accepted 11 July 2012 Available online 25 July 2012 Keywords: Contour detection Boundary detection Image segmentation Image processing Swarm Intelligence
a b s t r a c t Swarm Intelligence uses a set of agents which are able to move and gather local information in a search space and utilize communication, limited memory, and intelligence for problem solving. In this work, we present an agent-based algorithm which is specifically tailored to detect contours in images. Following a novel movement and communication scheme, the agents are able to position themselves distributed over the entire image to cover all important image positions. To generate global contours, the agents examine the local windowed image information, and based on a set of fitness functions and via communicating with each other, they establish connections. Instead of a centralized paradigm, the global solution is discovered by some principal rules each agent is following. The algorithm is independent of object models or training steps. In our evaluation we focus on boundary detection as a major step towards image segmentation. We therefore evaluate our algorithm using the Berkeley Segmentation Dataset (BSDS) and compare its performance to existing methods via the BSDS benchmark and Pratt’s Figure of Merit. © 2012 Published by Elsevier B.V.
1. Introduction Boundary Detection is commonly regarded as a major step towards Image Segmentation, which is a central problem in computer vision that has been studied for years. Image Segmentation again constitutes an important initial step for high-level tasks in computer vision, like object recognition, image analysis, and scene understanding. It pursues the goal of dividing an image into parts, regions, or objects, which are preferably self-consistent and of meaningful content. Application examples include separating objects considered as foreground from the background, or separating multiple objects from each other. One of the main difficulties in the field of boundary detection and image segmentation arises from the fact, that not even human beings are able to uniquely and unambiguously decide and agree over a correct segmentation [1]. Different persons are likely to have different opinions on the correct solution to the segmentation problem of any image, especially involving natural images. This lack of an unique ground-truth complicates the comparison of results descending form different segmentation methods. Nevertheless, efforts were taken to establish a framework for objective evaluation of segmentation algorithms. A prominent example is the Berkeley Segmentation Dataset (BSDS) [1], which provides a huge collection of natural images along with human-marked proposals of ground-truth, as well as a precision-recall based measurement
∗ Corresponding author. E-mail address: [email protected] (U. Kirchmaier). 1568-4946/$ – see front matter © 2012 Published by Elsevier B.V. http://dx.doi.org/10.1016/j.asoc.2012.07.017
for comparison [2]. Fig. 1 shows three example images along with three different human ground-truth suggestions. The plenty of solutions to the segmentation problem proposed in the literature descend from versatile fields, such as clustering [4], graph theory [5,6], region growing [7], optimization of an evaluation function like energy minimization [8], as well as image feature-based methods [9,10]. The majority of approaches found in the literature can be categorized into one or a subset of these fields, with each adding improved mechanisms to solve specific tasks. Boundary or contour detection is often performed as an initial step for image segmentation, as boundaries naturally correspond to borders between objects. It describes the process of identifying and locating sharp discontinuities in image regions, which closely relates it to the task of edge detection. Mere edge detection, however, registers on the one hand any type of abrupt changes in image brightness, and misses smooth or subtle transitions of image brightness on the other hand. Regarding the goal of detecting boundaries, this causes false positives like textured regions, or single highlighted spots and true negatives like region boundaries of low contrast. One recipe to overcome these drawbacks is to combine edge detection techniques with diverse soft computing approaches like Genetic Algorithms, or Fuzzy Logic, etc. [11,12]. Other methods combine oriented local image cues, like brightness, color, and texture descriptors with edge detection to get a global estimation. Such combination methods include multiscale cue combination [3], energy and probability weighting [13], or likelihood-methods [14].
U. Kirchmaier et al. / Applied Soft Computing 13 (2013) 3118–3129
3119
Fig. 1. Three sample images from the Berkeley Image Segmentation Dataset (BSDS 500) [3], each along with three different human-marked ground-truth images.
In this paper, we propose an algorithm that uses the concept of Swarm Intelligence to detect contours in images. We believe that the idea of interacting distributed agents to migrate from local information to global boundary estimates is a promising alternative to existing methods. While a variety of Swarm Intelligence inspired algorithms already exists (refer to Section 2), their swarms often seek to find one global optimal solution in a search space, as these algorithms descend from the field of optimization. However, it is our goal to detect a non-predefined number of boundaries with arbitrary positions, sizes, and shapes in an image, which all have to be considered as equal or even independent solutions. Therefore, we established a novel algorithm based on distributed agents being explicitly tailored to the demands of contour detection. The main contribution of this paper are the development of (1) a novel movement and positioning rule, along with (2) local information gathering mechanisms, and (3) a set of fitness functions and communication schemes. This paper is organized as follows: Section 2 gives an overview on approaches in the fields of image segmentation and contour detection, which utilize Swarm Intelligence. Section 3 investigates the principles of Swarm Intelligence and Section 4 provides details on our algorithm. Section 5 shows results of our algorithm tested on the task of boundary detection in natural images.
2. Related work In recent years, algorithms descending from the field of Evolutionary Computing, Genetic Algorithms, and also Swarm Intelligence are increasingly applied to the image segmentation problem. These optimization tools have mostly been used to improve the performance of important steps of segmentation algorithms. In [15,16], evolutionary and genetic algorithms were used to enhance the clustering process in segmentation. In [17], the authors combined high-level features generated with a visual
attention model with low-level features to guide region growing algorithm, where the optimal thresholds of the region growing process were detected using the Particle Swarm Optimization (PSO) algorithm. In [18], the PSO was applied to find the optimal fuzzy entropy thresholds to segment images into foreground and background. A related approach was introduced in [19], where PSO was used to tune thresholds in 2D-histograms, maximizing the entropy to segment infrared images. SI was also utilized to improve Segmentation based on clustering approaches, which can get stuck in local optima, depending on their initialization. In [20], the authors developed a hybrid combination of the Ant Colony Optimization algorithm (ACO) and PSO to make K-Means clustering and Single Competitive Learning more independent from their initial cluster centers, and learning rate, respectively. Besides deploying SI techniques to optimize specific steps in existing segmentation approaches, SI algorithms were also applied to the tasks of edge detection and contour detection, akin to our proposed algorithm. The authors of [21] proposed ant-based correlation for edge detection. Their method is capable of performing feature extraction for edge detection and segmentation, generating less distortion in the presence of noise, as compared to classical edge detectors, like Sobel, Prewitt, and Canny. In [22], a variant of the PSO algorithm is utilized for finding a proper edge detection filter size in noisy images. While combining edge filters of different sizes can improve the edge detection in noisy images, it is generally computationally expensive when applied to the entire image. Thus, the heuristic capabilities of PSO were used to adaptively decide, where on the image to use filters of various sizes. The evaluation of edge intensity and the proposed movement principle in this approach has analogies to our algorithm, as they divide the edge regions into sets and use averaged intensities. However, the major difference is the usage of discrete PSO instead of our communication scheme where the agents share their local information.
