Optimal range segmentation parameters through genetic ... - Imagelab
Optimal Range Segmentation Parameters Through Genetic Algorithms Luigi Cinque Stefano Levialdi Gianluca Pignalberi Dipartimento di Scienze dell’lnformazione Universith di Roma “La Sapienza” Via Salaria, 113 - 00198 Roma, ITALY {cinque, levialdi, pignalberi}@dsi.uniromal .it
Abstract
the type of segmented surface (only planar [4, 51 or planar and curved [3,6]), and performance in terms of speed and number of parameters [4]. Our goal is to introduce a general approach to optimize parameters for range image segmentation; this approach has been applied and tested on different segmenters. The optimization is based on Genetic Algorithms (GAS) [2,9], which are a well known technique adopted in many NP-hard optimization problems for a suitable search in complex and non-monotone solution spaces. We propose a GA featured with two new crossover methods which are compared to the classic Single Point Crossover [2]. The proposed system is able to find the optimal parameter set for a given algorithm and for a given set of images.
A wide number of algorithmsfor surjtiace segmentation in range images have been recently proposed characterized by different approaches (edgefilling, region growing, . . . ), different su$ace types (either for planar or curved suifaces) and different parameters involved. Optimization of the parameter set is a particularly critical task since the range of parameter variability is often quite large: parameter selection depends on surface type, sensors and the required speed which strongly affect peqormance. A framework for parameters optimization is proposed based on genetic algorithms. Such algorithms allow a general approach that has been succesfully applied on different state-of-the-art segmenters and different range image databases.
1.1. Range image segmentation Many algorithms have been defined for segmenting surfaces in range images [4]; we tested two of them. The first is the USF that is simple to implement and precise for planar surfaces [4]. It consists of two parts: it firstly computes the normal vector for each pixel, selects between such elements a set of seed pixels for starting a region growing process. Next the region growing process detects planar surfaces until one, between five possible conditions, fails. It uses five parameters { Winsize, Maxptdist, Maxperpdist, Maxangle, Minregpix} to tune precision and speed, region growing criteria and dimension of legal surfaces. We also tested an interactively modified version of USF, called FCS (Focused Constrained Segmentation) from University of Ferrara [7]. It segments, surface by surface, by computing normal vectors only where it is strictly necessary and the surface extraction is guided by constraints. The FCS uses three more parameters than USF { Segwinsize, Mintpts, Maxtdist} to manage interactivity and focused attention with a wide variability range. The second is the UB in the new version allowing for curved surfaces [6] and its interactive version.
1. Introduction Many algorithms for image processing and image analysis require a set of a variable number of parameters which depends on the image type, the image size and often the image content. The parameter tuning strongly influences the effectiveness and performance of the algorithm. Particularly, in range image segmentation many algorithms have been proposed characterized by a variable set of parameters and different performances: in this work we address range image surface segmentation algorithms. Many algorithms have been proposed for segmenting object surfaces in range images [4]. Between them, we can quote the WSU from Washington State University [3], the USF algorithm of University of South Florida [4], the UB algorithms of the Bern University [5,6]. Surface segmentation algorithms can be based on: region growing [4], point clustering [4, 31 or edge filling techniques [5, 6, 11, lo],
0-7695-0750-6100 $10.00 0 2000 IEEE
Rita Cucchiara Stefano Martinz Dipartimento di Scienze dell’hgegneria Universith di Modena e Reggio Emilia Via Campi, 213/b - 41 100 Modena, ITALY [email protected] martinz@ dsi.unimo.it
474
I
Figure 1. The proposed approach
Dominant with probability p ~
ing images. Mating individuals
1
Indiv. B /'
.
1.2. GAS for parameters selection GAS have rapidly gained acceptance in the scientific community as powerful statistical search and optimization techniques, and their basic principles are now quite known [2,8,9]. GAS encode the concept of evolution in the algorithmic search: from a population of individuals representing possible problem solutions, evolution is carried out by means of selection and reproduction. The design of GAS for a given optimization should cope with some critical aspects:
I
Dominant with probability I-p
I
c
IC, zc, 3c,....., 8C (9C, IOC)
1. the problem and solution coding: the problem is coded by a chromosome (a string coding parameters we are looking for) and a solution by a genotype (a certain value of the chromosome); 2. the definition of afitnessfunction which monotonically describes the solution goodness (in our case the effectiveness of surface segmentation); such a function should have a large variability in order to improve the convergence. Moreover this function should suitably show the effectiveness of the genotype; 3. the recombination operators: many variations of standard recombination operators (such as crossover and mutation) can be proposed in order to comply with
The algorithm has been conceived for images acquired by K2T sensors, using parameters referred to the sensor scan lines; with simple modifications, it can be adopted also for other range images but requires a careful parameter tuning. It firstly computes crease and jump edges and then fills edges by trying an element interpolation with plane and superquadric equations (for curved surfaces). It has ten parameters with a high variability. During several experiments, we found that different classes of range images need very different parameter sets. The key idea is to select the best parameter set in an automatic way given a set of train-
475
Mating individuals
Ldiv. A
1 4 5 3 rCrossover(how
manyohildren?)
Indiv. B
I
Indw. A
Indiv. B
Figure 4. Segmentation comparison for adapter1; on the left Heuristic results, on the right GA improved; from top to bottom: segmented image, interactively detected edges
Poiots of division, 1 for one offrpnng.... 7 for 7 offsprings
Figure 3. RORD crossover method
point 1 and to produce solutions compatible with the solution space.
1. a part of the score will be assigned because of undersegmentation, oversegmentation, missed segmentation (a single ground truth region covered by no segmented region - only for USF/FCS) f a n d region sizelreal region size ratio
In [12] GAS are adopted to modify the random search of range image points used in a specific modification of the least median squares algorithm (namely RESC method) for robust estimation of range surfaces. In [l] GAS are used to set the control parameters involved in a region-growing based intensity image segmentation using some qualitative evaluation of the segmentation results for guiding the genetic search. In this paper we use the same rationale for range image segmentation, independently from the specific segmenter.
2. a second part of the score will be assigned as I #of real regions - #of found regions 1 3. a third part of the score will be assigned according to the variance of gray levels of pixels belonging to each region such parts are necessary to improve phenotype discrimination. As previously stated, in this paper we introduce two new crossover methods, based on the variability of the number of offsprings. These methods (figures 2 and 3), called, respectively, Random Offsprings Fixed Division (ROFD) and Random Offsprings Random Division (RORD), were compared to the classic method (single point crossover SPC) of the Simple Genetic Algorithm [91.
2. The proposed approach Similarly to [l], we find a set of parameters to obtain the best possible segmentationresults for each chosen algorithm and for each class of images. Figure 1 shows our system blocks diagram. We can recognize the peculiar parts of almost every GA (initial population creation, selection, crossover, mutation, fitness function); in addition we can see three databases: one to store all the individuals data (“Genotype + scores + phenotype ids repository” - reinitialized when starting a new population), useful in case of run halting and restart, one to store all the phenotypes (the filesystem - “Phenotypes repository”), one to store only the solution genotypes (“Final genotypes repository” - inizialized only once, at the beginning of the system activity). Our fitness function adapts its scoring capability to the given image. Once determined the worst score, by an uniform gray image and the ground truth one, the system will assign a score to a segmented image as follows:
3. Results We used, as the reference machine, a Pentium 133 PC running Linux. A generation is defined within a fixed interval of time. Because of the speed of the used segmenters we fixed the interval for UB system was 5 minutes, while was 24 hours for the system using FCS/USF moreover we established 30 generations as a maximum to obtain a solution. If the system is not able to converge before such limits we get, as the solutions for the class represented by the current image, 476
Table 1. Program performances
in the short of time than now. Obviously the agent should organize the whole work of supervising the schedule and the data exchange. We have observed that the genetic system is present only in the trial process, while it is not required in the application process. Once the trial is completed, we would have a database containing the sets for every case of our images of interest. Using the interactivity capability of FCS and of the modified version of UB, the user can select the primitives to guide the segmentation and discriminate by the program suggested sets of parameters to start an optimal segmentation.
the set obtaining the best score among the genotypes of the last generation and that one obtaining the best score ever. With the equation “Generation = Time interval” the obtained number of results is independent from the crossover method. In fact SPC always generates two offsprings per coupling, while RORD and ROFD generate one offspring in about 97% of couplings; in this way the system will be able to generate about the same number of offsprings per crossover method. Some interesting results have been obtained using images from the WSU range image database processed with the UB algorithm (figure 4). In case of non convergence within the fixed limits we obtain a complete population in about two hours and a half. Figure 4 contains one of the processed objects. We can observe that the heuristic parameter set has a good segmentation capability but performs some errors, while the optimized parameter set allqws more precise results, finding most accurate edges in particular it eliminates many spurious edges, provides a sort of edge thinning and drives a more precise segmentation. Table 1 shows the performancesof our system, relatively to the mean scores reached by the crossover methods and the number of offsprings, on different images. The last group of rows of the table (gray coloured) represents the performances of the system with the paradigm “Generation = Number of couplings”, while the others represent the performances of the system with the paradigm “Generation = Time interval”. In the studied cases ROFD seems to offer the most affordable results, either in terms of number of generated phenotypes or in terms of mean score (a lot of phenotypes with low score - high fitness). When it does not reach the best it still has good results, close to the actual best, while RORD and SPC seem not to offer the same robust performances.
References [ l ] B. Bhanu, S. Lee, and J. Ming. Adaptive Image Segmentation Using a Genetic Algorithm. IEEE Transactions on Systems, Man and Cybernetics, 25( 12):1543-1567, 1995. [2] D. E. Goldberg. Genetic algorithms in search optimization and machine learning. Addison-Wesley, Reading, MA, 1989. [3] R. Hoffman and A. K. Jain. Segmentation and Classification of Range Images. IEEE Transactions on Pattern Analysis and Machine Intelligence, 9(5):60&620, September 1987. [4] A. Hoover, G. Jean-Baptiste, X. Jiang, P. J. Flynn, H. Burke, D. B. Goldgof, K. Bowyer, D. W. Eggert, A. Fitzgibbon, and R. B. Fisher. An Experimental Comparison of Range Image Segmentation Algorithms. IEEE Transactions on Pattern Analysis and Machine Intelligence, 18(7):673-689, July 1996. [5] X. Y.Jiang and H. Burke. Fast Segmentation of Range Images into Planar Regions by Scan Line Grouping. Machine Vision and Applications, 7(2): 115-122, 1994. [6] X. Y. Jiang and H. Burke. Edge Detection in Range Images Based on Scan Line Approximation. Computer Vision and Image Understanding, 73(2): 183-199, February 1999. [7] E. Lamma, P. Mello, M. Milano, R. Cucchiara, M. Gavanelli, and M. Piccardi. Constraint Propagation and Value Acquisition: why we should do it Interactively. In Proceedings of International Joined Conference of Artifrcial Intelligence (IJCAI ’99),Stockholm, 1999. [SI 2. Michalewicz. Genetic Algorithms + Data Structures = Evolutionary Computation. Springer Verlag, New York, N.Y., 2nd edition, 1994. [9] M. Srinivas and L. M. Patnaik. Genetic Algorithms: a Survey. IEEE Computer, 27(6): 17-27, 1994. 101 M. A. Wani and B. G. Batchelor. Edge-Region-Based Segmentation of Range Images. IEEE Transactions on Pattern Analysis and Machine Intelligence, 16(3):314-320, March 1994. 111 J. H. Yi and D. M. Chelberg. Model-Based 3D Object Recognition Using Bayesian Indexing. Computer Vision, Graphics and Image Processing (CVGIP), 69( 1):87-106, January 1998. [12] X. Yu, T. D. Bui, and A. Krzyiak. Robust Estimation for Range Image Segmentation and Reconstruction. IEEE Transactions on Pattern Analysis and Machine Intelligence, 16(5):530-538, May 1994.
4. Discussion and conclusions We saw that a lot of time is required to have some good results during the tuning phase. The system could be faster than that presented in this work by, at least, an additional feature: the genetic stage could be parallelized. Due to its nature it will be possible to have an agent that schedules the couplings to several processors, so to resolve a generation
477
Abstract
the type of segmented surface (only planar [4, 51 or planar and curved [3,6]), and performance in terms of speed and number of parameters [4]. Our goal is to introduce a general approach to optimize parameters for range image segmentation; this approach has been applied and tested on different segmenters. The optimization is based on Genetic Algorithms (GAS) [2,9], which are a well known technique adopted in many NP-hard optimization problems for a suitable search in complex and non-monotone solution spaces. We propose a GA featured with two new crossover methods which are compared to the classic Single Point Crossover [2]. The proposed system is able to find the optimal parameter set for a given algorithm and for a given set of images.
A wide number of algorithmsfor surjtiace segmentation in range images have been recently proposed characterized by different approaches (edgefilling, region growing, . . . ), different su$ace types (either for planar or curved suifaces) and different parameters involved. Optimization of the parameter set is a particularly critical task since the range of parameter variability is often quite large: parameter selection depends on surface type, sensors and the required speed which strongly affect peqormance. A framework for parameters optimization is proposed based on genetic algorithms. Such algorithms allow a general approach that has been succesfully applied on different state-of-the-art segmenters and different range image databases.
1.1. Range image segmentation Many algorithms have been defined for segmenting surfaces in range images [4]; we tested two of them. The first is the USF that is simple to implement and precise for planar surfaces [4]. It consists of two parts: it firstly computes the normal vector for each pixel, selects between such elements a set of seed pixels for starting a region growing process. Next the region growing process detects planar surfaces until one, between five possible conditions, fails. It uses five parameters { Winsize, Maxptdist, Maxperpdist, Maxangle, Minregpix} to tune precision and speed, region growing criteria and dimension of legal surfaces. We also tested an interactively modified version of USF, called FCS (Focused Constrained Segmentation) from University of Ferrara [7]. It segments, surface by surface, by computing normal vectors only where it is strictly necessary and the surface extraction is guided by constraints. The FCS uses three more parameters than USF { Segwinsize, Mintpts, Maxtdist} to manage interactivity and focused attention with a wide variability range. The second is the UB in the new version allowing for curved surfaces [6] and its interactive version.
1. Introduction Many algorithms for image processing and image analysis require a set of a variable number of parameters which depends on the image type, the image size and often the image content. The parameter tuning strongly influences the effectiveness and performance of the algorithm. Particularly, in range image segmentation many algorithms have been proposed characterized by a variable set of parameters and different performances: in this work we address range image surface segmentation algorithms. Many algorithms have been proposed for segmenting object surfaces in range images [4]. Between them, we can quote the WSU from Washington State University [3], the USF algorithm of University of South Florida [4], the UB algorithms of the Bern University [5,6]. Surface segmentation algorithms can be based on: region growing [4], point clustering [4, 31 or edge filling techniques [5, 6, 11, lo],
0-7695-0750-6100 $10.00 0 2000 IEEE
Rita Cucchiara Stefano Martinz Dipartimento di Scienze dell’hgegneria Universith di Modena e Reggio Emilia Via Campi, 213/b - 41 100 Modena, ITALY [email protected] martinz@ dsi.unimo.it
474
I
Figure 1. The proposed approach
Dominant with probability p ~
ing images. Mating individuals
1
Indiv. B /'
.
1.2. GAS for parameters selection GAS have rapidly gained acceptance in the scientific community as powerful statistical search and optimization techniques, and their basic principles are now quite known [2,8,9]. GAS encode the concept of evolution in the algorithmic search: from a population of individuals representing possible problem solutions, evolution is carried out by means of selection and reproduction. The design of GAS for a given optimization should cope with some critical aspects:
I
Dominant with probability I-p
I
c
IC, zc, 3c,....., 8C (9C, IOC)
1. the problem and solution coding: the problem is coded by a chromosome (a string coding parameters we are looking for) and a solution by a genotype (a certain value of the chromosome); 2. the definition of afitnessfunction which monotonically describes the solution goodness (in our case the effectiveness of surface segmentation); such a function should have a large variability in order to improve the convergence. Moreover this function should suitably show the effectiveness of the genotype; 3. the recombination operators: many variations of standard recombination operators (such as crossover and mutation) can be proposed in order to comply with
The algorithm has been conceived for images acquired by K2T sensors, using parameters referred to the sensor scan lines; with simple modifications, it can be adopted also for other range images but requires a careful parameter tuning. It firstly computes crease and jump edges and then fills edges by trying an element interpolation with plane and superquadric equations (for curved surfaces). It has ten parameters with a high variability. During several experiments, we found that different classes of range images need very different parameter sets. The key idea is to select the best parameter set in an automatic way given a set of train-
475
Mating individuals
Ldiv. A
1 4 5 3 rCrossover(how
manyohildren?)
Indiv. B
I
Indw. A
Indiv. B
Figure 4. Segmentation comparison for adapter1; on the left Heuristic results, on the right GA improved; from top to bottom: segmented image, interactively detected edges
Poiots of division, 1 for one offrpnng.... 7 for 7 offsprings
Figure 3. RORD crossover method
point 1 and to produce solutions compatible with the solution space.
1. a part of the score will be assigned because of undersegmentation, oversegmentation, missed segmentation (a single ground truth region covered by no segmented region - only for USF/FCS) f a n d region sizelreal region size ratio
In [12] GAS are adopted to modify the random search of range image points used in a specific modification of the least median squares algorithm (namely RESC method) for robust estimation of range surfaces. In [l] GAS are used to set the control parameters involved in a region-growing based intensity image segmentation using some qualitative evaluation of the segmentation results for guiding the genetic search. In this paper we use the same rationale for range image segmentation, independently from the specific segmenter.
2. a second part of the score will be assigned as I #of real regions - #of found regions 1 3. a third part of the score will be assigned according to the variance of gray levels of pixels belonging to each region such parts are necessary to improve phenotype discrimination. As previously stated, in this paper we introduce two new crossover methods, based on the variability of the number of offsprings. These methods (figures 2 and 3), called, respectively, Random Offsprings Fixed Division (ROFD) and Random Offsprings Random Division (RORD), were compared to the classic method (single point crossover SPC) of the Simple Genetic Algorithm [91.
2. The proposed approach Similarly to [l], we find a set of parameters to obtain the best possible segmentationresults for each chosen algorithm and for each class of images. Figure 1 shows our system blocks diagram. We can recognize the peculiar parts of almost every GA (initial population creation, selection, crossover, mutation, fitness function); in addition we can see three databases: one to store all the individuals data (“Genotype + scores + phenotype ids repository” - reinitialized when starting a new population), useful in case of run halting and restart, one to store all the phenotypes (the filesystem - “Phenotypes repository”), one to store only the solution genotypes (“Final genotypes repository” - inizialized only once, at the beginning of the system activity). Our fitness function adapts its scoring capability to the given image. Once determined the worst score, by an uniform gray image and the ground truth one, the system will assign a score to a segmented image as follows:
3. Results We used, as the reference machine, a Pentium 133 PC running Linux. A generation is defined within a fixed interval of time. Because of the speed of the used segmenters we fixed the interval for UB system was 5 minutes, while was 24 hours for the system using FCS/USF moreover we established 30 generations as a maximum to obtain a solution. If the system is not able to converge before such limits we get, as the solutions for the class represented by the current image, 476
Table 1. Program performances
in the short of time than now. Obviously the agent should organize the whole work of supervising the schedule and the data exchange. We have observed that the genetic system is present only in the trial process, while it is not required in the application process. Once the trial is completed, we would have a database containing the sets for every case of our images of interest. Using the interactivity capability of FCS and of the modified version of UB, the user can select the primitives to guide the segmentation and discriminate by the program suggested sets of parameters to start an optimal segmentation.
the set obtaining the best score among the genotypes of the last generation and that one obtaining the best score ever. With the equation “Generation = Time interval” the obtained number of results is independent from the crossover method. In fact SPC always generates two offsprings per coupling, while RORD and ROFD generate one offspring in about 97% of couplings; in this way the system will be able to generate about the same number of offsprings per crossover method. Some interesting results have been obtained using images from the WSU range image database processed with the UB algorithm (figure 4). In case of non convergence within the fixed limits we obtain a complete population in about two hours and a half. Figure 4 contains one of the processed objects. We can observe that the heuristic parameter set has a good segmentation capability but performs some errors, while the optimized parameter set allqws more precise results, finding most accurate edges in particular it eliminates many spurious edges, provides a sort of edge thinning and drives a more precise segmentation. Table 1 shows the performancesof our system, relatively to the mean scores reached by the crossover methods and the number of offsprings, on different images. The last group of rows of the table (gray coloured) represents the performances of the system with the paradigm “Generation = Number of couplings”, while the others represent the performances of the system with the paradigm “Generation = Time interval”. In the studied cases ROFD seems to offer the most affordable results, either in terms of number of generated phenotypes or in terms of mean score (a lot of phenotypes with low score - high fitness). When it does not reach the best it still has good results, close to the actual best, while RORD and SPC seem not to offer the same robust performances.
References [ l ] B. Bhanu, S. Lee, and J. Ming. Adaptive Image Segmentation Using a Genetic Algorithm. IEEE Transactions on Systems, Man and Cybernetics, 25( 12):1543-1567, 1995. [2] D. E. Goldberg. Genetic algorithms in search optimization and machine learning. Addison-Wesley, Reading, MA, 1989. [3] R. Hoffman and A. K. Jain. Segmentation and Classification of Range Images. IEEE Transactions on Pattern Analysis and Machine Intelligence, 9(5):60&620, September 1987. [4] A. Hoover, G. Jean-Baptiste, X. Jiang, P. J. Flynn, H. Burke, D. B. Goldgof, K. Bowyer, D. W. Eggert, A. Fitzgibbon, and R. B. Fisher. An Experimental Comparison of Range Image Segmentation Algorithms. IEEE Transactions on Pattern Analysis and Machine Intelligence, 18(7):673-689, July 1996. [5] X. Y.Jiang and H. Burke. Fast Segmentation of Range Images into Planar Regions by Scan Line Grouping. Machine Vision and Applications, 7(2): 115-122, 1994. [6] X. Y. Jiang and H. Burke. Edge Detection in Range Images Based on Scan Line Approximation. Computer Vision and Image Understanding, 73(2): 183-199, February 1999. [7] E. Lamma, P. Mello, M. Milano, R. Cucchiara, M. Gavanelli, and M. Piccardi. Constraint Propagation and Value Acquisition: why we should do it Interactively. In Proceedings of International Joined Conference of Artifrcial Intelligence (IJCAI ’99),Stockholm, 1999. [SI 2. Michalewicz. Genetic Algorithms + Data Structures = Evolutionary Computation. Springer Verlag, New York, N.Y., 2nd edition, 1994. [9] M. Srinivas and L. M. Patnaik. Genetic Algorithms: a Survey. IEEE Computer, 27(6): 17-27, 1994. 101 M. A. Wani and B. G. Batchelor. Edge-Region-Based Segmentation of Range Images. IEEE Transactions on Pattern Analysis and Machine Intelligence, 16(3):314-320, March 1994. 111 J. H. Yi and D. M. Chelberg. Model-Based 3D Object Recognition Using Bayesian Indexing. Computer Vision, Graphics and Image Processing (CVGIP), 69( 1):87-106, January 1998. [12] X. Yu, T. D. Bui, and A. Krzyiak. Robust Estimation for Range Image Segmentation and Reconstruction. IEEE Transactions on Pattern Analysis and Machine Intelligence, 16(5):530-538, May 1994.
4. Discussion and conclusions We saw that a lot of time is required to have some good results during the tuning phase. The system could be faster than that presented in this work by, at least, an additional feature: the genetic stage could be parallelized. Due to its nature it will be possible to have an agent that schedules the couplings to several processors, so to resolve a generation
477
