Interpretability Based Interest Points Detection
Interpretability Based Interest Points Detection Ahmed REBAI
Alexis JOLY
Nozha BOUJEMAA
INRIA Rocquencourt 78153 Le chesnay France
INRIA Rocquencourt 78153 Le chesnay France
INRIA Rocquencourt 78153 Le chesnay France
[email protected]
[email protected]
ABSTRACT This paper deals with a new interest points detector. Unlike most standard detectors which concentrate on the local shape of the signal, the main objective of this new operator is to extract interpretable points from the image context. The basic principle of this operator was the detection of radial symmetries, but we have generalized it to cover other kind of interest points. Indeed, detected points constitute centers of circles or logarithmic spirals, intersections of curves and vanishing points. Added to that, this detector does not depend on the features’ size, what makes it possible to be robust to the scaling. Detection of such points is performed using a three dimensional space called θ-space. Experiments reveal that these points are more likely to be related to visual attention. We also applied these points to the object recognition problem and state of the art performances.
Categories and Subject Descriptors I.4.7 [Image Processing and Computer Vision]: Feature Measurement—Feature representation
General Terms Algorithms
Keywords Interpretability, symmetry, CBIR
1.
INTRODUCTION
In computer vision, interest points have been widely used to solve various problems, such as image retrieval [22], object recognition [14, 15], wide baseline matching for stero pair [2], etc. Detecting these points hasn’t been a trivial task since they have to be the more relevant features in the image. It is known, for instance, for generic objects recognition or object class recognition, that only a few percentage of initially detected features are used in the final object class.
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. To copy otherwise, to republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. WOODSTOCK ’97 El Paso, Texas USA Copyright 200X ACM X-XXXXX-XX-X/XX/XX ...$5.00.
[email protected]
This proves that the most of detected points are useless and unsuitable for such tasks; this is problematic both in term of efficiency and computation time. That’s why it is essential to reduce the input of local features by improving their visual interpretability in order to concentrate effort at key locations in the image. In this paper, we introduce a new interest points detector whose goal is to extract meaningful points. In other words, detected points can be easily interpreted visually.
1.1
Related research
At some extent, interpretability is related to symmetry which is considered to be an important factor in the human visual system. Many objects around us exhibit some form of symmetry. The best symmetric objects are the circular ones because they have an infinity of symmetry axes which all intersect at the center [12, 28]. That’s way the center is considered to be an interest point with a high visual interpretability. To detect symmetry, many operators have already been developed. The Reisfeld’s generalized symmetry transform [6] looks for local reflectional symmetries and it is based on image gradients. Loy and Zelinsky [16] use also gradients for searching local radial symmetry. Other techniques [13, 9, 11] use the Hough transform and some of them exploit gradients information in order to reduce computation time. Stentiford [17, 18] use an attention mechanism to detect reflectional symmetries. In this paper, we’ll present a new symmetry operator. It is based only upon gradients directions and it is not conceived to detect reflectional symmetries. The approach was inspired from the work of Loy and Zelinsky [16]. Unlike previous methods which used to quantize gradients into angular bins [13, 19, 24], their technique is enough stable because they take into account all the gradients directions. However, this works only for a set of radii belonging to a discrete values {rk | k = 1, · · · , m}. Then, the transform can not extract features having a radius superior to the maximum radius of this set. In the approach they used, transformed image is the sum of intermediate accumulator images that we can visualize as layers, one layer for each member of rk , whereas, in our approach, output image is formed by a single pass over original image, there is no need to use any intermediate images. We are not limited to a set of radii and so we can detect centers of circles of any size. The novelty of our operator comes from its capability to detect centers of other symmetric objects (cf. figures 3 and 5). It can also detect centers of logarithmic spirals or portions of them as well as intersection of such curves. Moreover, not only can this operator extract centers of objects,
but it can also detect the intersection of straight lines, typically junctions and even “fictive” junctions. Thus, it can be tuned to be a vanishing point detector. Vanishing points are defined as the image points at which the projection of parallel lines intersect. The detection of vanishing points allows to infer 3D structure from 2D images. That’s why, when existing in a scene, a vanishing point belongs to the best interest points in that image, if not to be the best one at all, and so, it has a high visual interpretability. In this context, the approach developed by Stentiford [26] is based upon a model of human visual attention. It computes the scarcity of a point according to its neighbours. A point that occur rarely (unusual) will be attributed a high attention score. Unlike the most developed methods which use the Gaussian sphere [5, 25, 1] for detecting vanishing points, the approach presented herein exploit gradients’ directions. It bares more similarity to the work of Virginio et al. [4] in which an accumulator image was used. However, their method needs a preprocess that performs an edge detection operation and the use of two masks instead of gradients information. Only by using one detector, we are able to extract a variety of interest points having different topological nature. Each point has a high visual interpretability and it is likely to be a point of visual attention. All of this is performed without any prior knowledge of the scene.
1.2
Overview
This work provides generalization, improvements and more detailed experiments of research previously reported in the author’s conference publication [21]. Section 2 redefines the new approach of the operator, discusses the nature of the detected points and provides some illustrative results. Section 3 describes the implementation and the selection of parameters. Section 4 exhibit the experiments. And finally, section 5 presents the conclusions and the future work.
2.
THEORY
Originally, the detector presented herein was a radial symmetry operator that has been generalized, on the one hand, to cover other kinds of symmetries and in the other hand, to detect centers of logarithmic spirals and junctions. Next paragraph introduces the basic radial symmetry operator.
2.1
Circular objects detection
To ensure the detection of circle’s center, we use an election principle based on gradients’ directions [16]. Each pixel will look for all the gradients whose directions point to its position (see Figure 1). The best elected point corresponds to the point having the highest number of gradients satisfying the condition. In this way, we can construct an accumulator image, in which, each pixel value represents the number of gradients that point to that pixel. Centers of circles correspond to the local maxima on this image. Algorithmically and in terms of computation time, this approach is not efficient since, for each point k, we need to loop over all pixels in the image and test whether the gradient’s direction passes by k. While this approach is prohibitive, we can obtain the same output image by changing the technique. Giving a gradient’s direction, since each point located along this direction will increment its accumulator by 1, it will be possible to make only one single pass over the image, in which and for a pixel (x, y), we increment by 1 all of the pixels located in the gradient’s direction of
Figure 1: Illustration of the election’s principle. The pixel k looks for gradients pointing to its position. Those satisfying this condition are highlighted. Note that not all gradients are represented here. If we take into account only these gradients, the score for the pixel k will be equal to 3.
(a)
(b)
(c)
Figure 2: Example of transformed image. (a) represents the original image, (b) is the transformed image and (c) is a 3D - visualization of (b).
(x, y). The result is absolutely the same whereas the second method is obviously fastest. In order to be robust with the changes of colour, in addition to making votes in the direction of the gradient, we make also votes in its opposite direction (not to take account with the convention of which the gradient is calculated: white towards black or conversely). In [16], votes are also made in the both directions of the gradient vector, nevertheless, those made in the opposite direction were negative. This is a problematic because in the case where a symmetric feature has a light colour in the right half and a dark colour in the left half for example, votes will cancel each other. Such kind of feature is represented in figure 2 (a). An example of our output transform is given in figure 2 (b). Due to the digitization and estimation of the gradients’ directions, the process described above will not lead to a proper signal. So, transformed image will have much noise and, consequently, the detection of interest points will not be so easy. Furthermore, truly circular objects — in the strictly usual mathematical notion — are rare in natural scene. It is then noticeable that the process described above will fail to detect a circular object which has some kind of distortions. To encounter this problem, instead of voting only in the gradient’s direction, votes will be made along an arc of circle centered with reference to the gradient’s direction and whose length L depends on the radius r of the vote: L=δ·r δ is a constant angle used to define the arc (see figure 8).
Now, we put the general equation of the transform as: XX ą − ć − → → ~ E(x, y) = |∇I(i, j)· R xy (i, j)| w k R xy (i, j)k, Nxy (i, j) i
j
(1) ~ designates the gradient where I represents original image, ∇ − → symbol, R xy is the vector defined by the two pixels (x, y) and (i, j): ţ ű − → x−i R xy (i, j) = y−j and Nxy (i, j) =
− → ~ |∇I(i, j) · R xy (i, j)| − → ~ k∇I(i, j)k k R xy (i, j)k
w(·, ·) is a weighting function depending on two parameters: − → the vote’s distance k R xy k and the angle δ deduced from Nxy . Note that we are using the absolute value of the scalar − → product between the gradient vector and R xy in order to achieve positive votes in both directions. To estimate the radius of a detected feature, we use an histogram accumulator for each pixel in the image. The length of each histogram is equal to the image’s diagonal which is theoritically the largest radius that could exist. If the gradient at the pixel (i, j) is an elector for the pixel (x, y), then, the histogram of pixel (x, y) will be incremented by 1 at − → the index r = round(k R xy k) which represents the distance separating the two points. Here, function round(·) rounds each number to the nearest integer. After processing the whole image, we compute the maximum of each histogram. Finally, the radius of a feature will be the index corresponding to the maximum of the histogram related to the center of that feature. In any case, if the maximum exits in two or many indexes, we choose the highest index. In this way, we obtain a second output image “Radii image” which stores information about the features’ radius. Radii image is considered as a good informative source regarding features’ geometry. This is why we will exploit it to determine whether a pixel is an interest point or not. In fact, the detection of local maxima (3 × 3 maxima) is no sufficient because of the noise that could be included in the transformed image. Fixing a threshold even adaptive doesn’t constitute a good solution too because a noisy value could be enough high to exceed the threshold. The approach we use here is very simple and efficient at the same time. Is regarded as point of interest any pixel having a value of symmetry higher than all the values of the pixels located in the disc defined by its symmetry’s radius. By this definition, even in case where a circle is included in an other one, both of two circles will be detected as the interior circle will have a symmetric value inferior to the biggest one because it has less electors than it.
2.2
~ θ I(i, j) the direction of the gradient’s vecWe denote by ∇ tor rotated by θ, equation 1 will be changed to: XX ą − ć → → ~ θ I(i, j)·− Eθ (x, y) = |∇ R xy (i, j)| w k R xy (i, j)k, Nxy (i, j) i
j
(2) Eθ is the transformed image with angle θ.
2.2.1
Variety of symmetric objects
Radial symmetry objects aren’t only circular objects. In figure 3 is represented another type of radial symmetric object whose center can not be detected using other existing techniques.
Figure 3: A kind of radial symmetric feature whose center can’t be detected using standard detectors. In blue are represented gradients and in red are represented electors which are the result of rotating gradients by the angle θ.
2.2.2
Characteristic of logarithmic spirals
The polar equation of a logarithmic spiral is given by: ρ = a · exp (kφ)
(3)
where k = cot(ψ). ψ is a constant representing the equiangular spiral. In the special case where ψ = π2 , we obtain a circle. Logarithmic spiral makes the constant angle ψ with any radius vector. This is an interesting geometric property, since, by rotating the gradient vector by θ = φ − α (where φ and α represent the polar and the gradient’s angle respectively), it will point to the center of the spiral (see figure 4).
Generalization of the election principle
The approach described above used for detecting circles is generalized here to cover other symmetric objects. It will also allow the detection of centers of logarithmic spirals, junctions and vanishing points. The generalization consists in keeping the election principle and using other directions rather than the gradient’s direction one. Given θ ∈ [− π2 , π2 ], if α designates the gradient’s direction, the elector vector’s direction will be (α + θ).
Figure 4: Example of logarithmic spiral (ψ =
2.2.3
7π ). 16
Junctions and vanishing points
Now, let θ = ± π2 , elector vectors will have perpendicular directions to that of gradients. In the case of straight lines, votes will be accumulated along that lines. Therefore,
if there is any intersection of these lines (even fictive), it will have a high score thanks to the votes from different directions. By fictive junction, we mean that there is no effective intersection presented in the scene but the extension of the lines will intersect (see figure 5). In case of vanishing points, the extension of 3D parallel lines projected in 2D image will be considered as a junction between segment lines, and so, vanishing point will be seen as junctions.
(a)
(b)
(c)
Figure 7: Example of detected points. The representation used here takes into account the estimation of feature’s radius. (a) gives an example of circles’ detection (θ = 0). The best elected point in (b) corresponds to θ = 8◦ while (c) reveals a vanishing point (θ = 90◦ ).
Figure 5: Example of junctions. It is essential to notice here that the radius’ estimation method described in section 2.1 will not give good results because the electors are not situated at the same radius. The maximum of histogram occur at the radius having the highest number of electors and it is not possible to predict whether this maximum will exist due to electors which are close to the junction or due to electors which are far from the junction.
2.2.4 θ-space analysis By applying the transform to several θ angles, we construct the θ-space which can theoritically be continuous if θ takes continuous values in [− π2 , π2 ]. θ-space is a good example of 3-dimensional space which maintain the energy unchanged (by neglecting side effect). Indeed, while going through this space, we notice that energy, initially dispersed for a given pixel, concentrates with a specific angle θ0 to disperse again while going away from θ0 . While the concept of a continuous θ-space is interesting, actually, taking only a subset of θ values is normally enough to yield a representative result. Figure 6 gives two examples of synthetic images with their θ-spaces sampled. It is noticeable that 3D − maxima in θ-space have to be more stable than 2D − maxima of a transformed image at a given value θ compared at what is usually done in scale-space theory [8]. This fact let θ-space be a powerful tool to detect interest points having different topological natures. However, 3D − maxima extraction requires that θ-space has a kind of continuity to ensure stable detection. Unfortunately, such a criterion necessitates the use of a high number of samples and so it will affect the computing time considerably.
2.3
Detection examples
In figure 7, we give some illustrative results showing the best elected points in three examples. In these examples, we didn’t perform a 3D−maxima detection. We’ve just located 2D − maxima in θ-space sampled images and then, we’ve sorted them in descending order according to the transform’s value. Only the best elected points are kept.
3.
IMPLEMENTATION DETAILS
Initially, we define a set of θ values Θ = {θmin , · · · , θmax } ⊂ [− π2 , π2 ]. For each θ ∈ Θ, we construct the transformed image Eθ and the radii image IR . Eθ is initially zero. To compute IR , we define an histogram accumulator for each pixel and we initialize it to zero. The first step in the process is to compute gradients’ directions. For a given pixel k, the gradient’s direction α is given by: ş ∇I(k) ~ · yť α = arctan ~ ∇I(k) ·x where x and y are the unit vectors according to the width and the height of the image respectively: ţ ű ţ ű 1 0 x= y= 0 1 Now, we will consider only gradients whose magnitude is higher than a defined threshold T . Indeed, small gradients are not significant and usually they result from the noise effect. If they are took into account, on the one hand, they lead to a noisy image transform, on the other hand, they will considerably increase computation time. Threshold T is determined empirically after several tests had been made on different images. Now, for each pixel k, we compute the elector vector’s direction which is equal to α + θ. Let δ be the angle used to determine the arc of votes. For each radius r and for each angle λ ∈ [α + θ − 2δ , α + θ + 2δ ], we compute coordinates of the two affected pixels p and q located on both sides of k (see figure 8). Equations are given by: ş ą ćť p = k + round r · cos(α + θ + λ)x + sin(β + θ + λ)y (4) ş ą ćť (5) q = k − round r · cos(α + θ + λ)x + sin(α + θ + λ)y In transformed image Eθ , each affected pixel will be incremented by 1 as: Eθ (p) = Eθ (p) + 1
(6)
Eθ (q) = Eθ (q) + 1
(7)
Meanwhile, we increment by 1 at index r the histogram which corresponds to pixel k. In order to reduce the excessive use of computer memory, we can restrict the histogram’s size by a factor of 2 or 4. The use of bins doesn’t
(a) The best elected point is represented by a red cross in original image. It is detected at θ = 40◦ .
(b) Best elected points are represented in red in original image. There are 5 points detected at θ = 0◦ and only one point detected at θ = 90◦ . Figure 6: Two examples of samples of θ-space. Original image is given in the top-left of each example and contains the best elected points. Representation takes into account the estimation of features’ radius. Values of θ used here from left to right: −90◦ , −81◦ , −65◦ , −57◦ , −40◦ , −24◦ , −16◦ , 0◦ , 16◦ , 24◦ , 40◦ , 57◦ , 65◦ , 81◦ , 90◦ . is related to the significance of extracted points and finally an application to the object recognition.
4.1
Figure 8: Representation of the affected pixels p and q at a distance r away from k. Note that votes will cover the entire highlighted area. affect considerably images having big dimensions where the histogram’s size grows. While normally the step of λ should be Lδ = r1 , it is possible to fix the step in order accelerate the algorithm. However, the quality of transformed image deteriorates and this is more clear when increasing the step and when image’s dimensions are excessively big. After achieving the voting process, we blur Eθ image in order to reduce noise and then we perform local maxima detection.
4.
EXPERIMENTS
To evaluate our detector, experiments were carried out using three criteria: ²-repeatability [23], interpretability which
Repeatability measurement
In these experiments, we used a database of 100 images from different categories: natural scenes such as animals, trees, grass, sea and man-made scenes such as building, roads, vehicles, etc. θ-space was sampled into the following set of values {− π2 , − π3 , − π6 , 0, π6 , π3 , π2 }. The ² used for computing the repeatability is equal to 5. Repeatability rate was calculated for some image transformations: gaussian noise, gaussian blur, jpeg compression, salt and pepper noise, negative, flip and mirror. Results show that junctions (θ = ± π2 ) have always higher repeatability rate than all interest points detected at other angles in any transformation considered. The distribution of repeatability rate according to θ is given in figure 9. Note that the value of repeatability rate for a given θ represents the average of repeatability rates of all the transformations cited above. In order to measure the seperability of detected features, we computed the number of interest points according to each angle θ. The average of these numbers between all database images is given in figure 10. This graph shows that circular objects are few and that, contrary, junctions are the most present features. Between the two types of points and while moving away from the angle θ = 0, the number of detected points increases while following a convex curve. According to the figure 9, one may deduce that junctions are the most stable points that could exist in nature. How-
Figure 9: Average of repeatability rate to some image transformations according to angle θ (² = 5).
Figure 10: Average of interest points’ number detected in the whole database according to angle θ. ever, if we calculate the contribution of each interest point to compute the repeatability rate for each angle θ (i.e. we divide the repeatability rate corresponding to a given θ by the number of points detected at this angle), we find that the greatest value is obtained at θ = 0. Moreover, this value is definitely higher than those obtained in other angles (see figure 11). According to this graph, we notice that although the circular objects are the least present, they are the most repeatable in detection. Such result confirms that repeatability measurement as defined in [23] is not an appropriate metric to evaluate the performance of our detector. This definition of repeatability, indeed, deals only with tractability problem. It doesn’t inform about global seperability and doesn’t characterize the distinctivenss of detected features [27].
4.2
Distinctiveness
In order to test whether our detected features are distinctive or not, we are based upon human decision. In fact, we chose some classes of objects which contain relevant features according to the human interpretation and we tried to detect theses features only by keeping the three best elected points using two values of θ: 0 and π2 . For these expermiments, we used Caltech 101 [7] and Caltech 256 [10] databases which contain a variety of symmetric objects. For the following classes: bowling ball, tennis ball, soccer ball, desk globe, frisbee, mars, watch and steering wheel, the objective was to determine the center of those features. For the touring bike and motorbikes classes, the objective was to detect one of the centers of the two wheels.
Figure 11: Repeatability rate per point according to angle θ (² = 5).
(a)
(b)
(c)
Figure 12: Example of transformed image. (a) represents the original image, (b) is the transformed image and (c) is a 3D - visualization of (b). Concerning “faces easy” class, it was necessary to detect the center of only one eye. And finally, for the camera class, we aimed to detect the lens. Results are illustrated in the table 1. Note that the decision defining whether a feature is detected or not was made by three persons. Object’s class Bowling ball Desk-globe Frisbee Mars Steering-wheel Tennis-ball Touring-bike Faces easy Motorbikes Camera Soccer ball Watch
Percentage of detected features 73.08 96.34 88.89 91.03 90.72 73.47 90 77.01 97.82 94 87.5 91.21
Table 1: Percentage of detected features according to the object’s class. Now, concerning the detection of junctions, we used the classes butterfly and dragonfly to ensure detection of the back of these insects which represents the wings’ intersec-
tion. For the class ceiling fan, the objective was to detect intersection of blades. In the same way, we tried to detect the intersection of tripod’s legs. Another object containing one junction is scissor. Above this, we used two other classes envolving more than one junction which are anchor and menorah. The objective here was to detect only one junction. Results of detection are given in table 2. Moreover, we created a database of 76 images of different sizes collected from Internet and where a vanishing point is present. Note that not all of images chosen contain perfect straight lines. However, we succeeded in detecting 72% of vanishing points which represents a good score. Object’s class Tripod Anchor Butterfly Ceiling fan Dragonfly Menorah Scissors Vanishing points base
Percentage of detected features 83.93 88.1 76.92 97.87 97.05 91.95 97.44 72
Table 2: Percentage of detected features according to the object’s class.
4.3 4.3.1
Basic Objects’ class recognition tests Description
We describe here some preliminary experiments for objects’ class recognition. Since we did not yet defined specific local descriptors for our new interest points, we used non-specialized histogram based descriptors. They are computed around each interest point in a square window with an arbitrary side equal to 50 pixels. Each local descriptor is a 128-dimensional vector F composed of the concatenation of six local histograms described in [3] (Fourier based histogram, weighted color histograms, edge orientation histogram, Hough based histogram). Obviously, these descriptors are not adapted to our interest points since they do not take care about the topology of the detected features and they do not characterize any geometrical structure. For the learning, we use a simple feature selection strategy that consists in keeping the three best local features in each image of the learning set, according to their distinctiveness. The distinctiveness of a local feature is determined in a way similar to the Weak-Hypothesis-Finder defined in the boosting approach of Opelt et al. [20]. A distance bewteen any feature d(Fqj and any image Ik of the training set is defined as:
4.3.2
Tests of objects’ recognition were applied on ImagEVAL databases. ImagEVAL1 is an evaluation compaign of contentbased image retrieval techniques and automatic description of images. The fourth task of its benchmark was dedicated to object recognition. There are ten classes used: armored vehicle, car, cow, Eiffel tower, minaret and mosque, plane, road signs, sunglasses, tree and US flag. Moreover, proposed objects had different poses, contexts and sizes. The final database is formed with 14,000 images. Note that some images contain more than one object’s class and that 5,000 images didn’t include any objects.
4.3.3
5.
Results
CONCLUSIONS AND FUTURE WORK
We have introduced a new interest points detector which focus on salient points having a high visual interpretability. Originally, the detector was a radial symmetry operator which has been generalized to allow the extraction of other interpretable points. Generalization leads to a three dimensional space whose third dimension depends on angle θ. This space called θ-space constitutes a rich analyze tool allowing the extraction of points having different topological natures. Experiments showed that our interest points concentrate on the best relevant features in a given object and that these features are almost detected for the same object’s class. This is how these points are more likely to be related to the visual attention mechanism. To verify this assumption, an interesting experiment will be the use of an eye-tracking system. This represents one goal of our future works. Another important task lies in the possibility of characterizing an object using our interest points since each point has a characteristic angle θ. The objective will be the creation of an invariant and robust description using geometric relations. Such description will not be affected by the precision of detection. Therefore, standard repeatability measurement is not appropriate to judge whether the detection is good or not. Justification of fiability could be derived from applying the detector to some computer vision problems, in particular, objects recognition gave good results while using non specialized histogram descriptors. Such result is very promising to define new local descriptors adapted to the extracted features.
6. REFERENCES [1] S. T. Barnard. Interpreting perspective images. Artificial intelligence, 21(4):435–462, 1983. [2] A. Baumberg. Reliable feature matching across widely separated views. In CVPR, pages 1774–1781, 2000. [3] N. Boujemaa, J. Fauqueur, M. Ferecatu, F. Fleuret, V. Gouet, B. L. Saux, and H. Sahbi. Ikona for interactive specific and generic image retrieval. In Proc. of Workshop on MultiMedia Content Based Indexing and Retrieval, 2001. [4] V. Cantoni, L. Lombardi, M. Porta, and N. Sicard. Vanishing point detection: representation analysis and new approaches. Image Analysis and Processing, 3(3):90–94, September 26-28 2001.
d(Fqj , Ik ) = min d(Fqj , Fki ) 1≤i≤nk
For each Fqj , the images Ik are then sorted with increasing distances and the best threshold θ on the distance is then chosen according to the classification error. The retrieval of the images belonging to a given class of objects consists in computing a matching kernel between the candidate images and all the learning images of a given class. The matching kernel is applied only on the features selected by the learning step and the final score is the average value of the kernel function on the learning images of the given class.
Evaluation
1
http://www.imageval.org/
[5] R. Collins and R. Weiss. Vanishing point calculation as a statistical inference on the unit sphere. In International Conference on Computer Vision, pages 400–403, December 1990. [6] R. D., W. H., and Y. Y. Context free attentional operators: the generalized symmetry transform. Int. J. of Computer Vision, Special Issue on Qualitative Vision, 1994. [7] L. Fei-Fei, R. Fergus, and P. Perona. Learning generative visual models from few training examples: an incremental bayesian approach tested on 101 object categories. IEEE. CVPR 2004, Workshop on Generative-Model Based Vision, 2004. [8] L. Florack and A. Kuijper. The topological structure of scale-space images. Journal of Mathematical Imaging and Vision, 12(1):65–79, February 2000. [9] J. Y. Goulermas and P. Liatsis. Incorporating gradient estimations in a circle-finding probabilistic hough transform. Pattern Analysis and Applications, 2(3):239–250, August 1999. [10] G. Griffin, A. Holub, and P. Perona. The caltech-256, caltech technical report. [11] N. Guil and E. Zapata. Lower order circle and ellipse hough transform, October 1997. [12] L. P. J. and N. C. F. Symmetry catches the eye. Eye movements: From physiology to cognition, pages 353–361, 1987. [13] C. Kimme, D. Ballard, and J. Sklansky. Finding circles by an array of accumulators. Commun. ACM, 18(2):120–122, 1975. [14] S. Lazebnik, C. Schmid, and J. Ponce. Semi-local affine parts for object recognition. In British Machine Vision Conference, volume volume 2, pages 779–788, 2004. [15] D. G. Lowe. Object recognition from local scale-invariant features. In Proc. of Int. Conf. on Computer Vision, pages 1150–1157, 1999. [16] G. Loy and A. Zelinsky. Fast radial symmetry for detecting points of interest. IEEE Transactions on Pattern Analysis and Machine Intelligence, 25(8), August 2003. [17] S. F. W. M. Attention based facial symmetry detection. International Conference on Advances in Pattern Recognition, pages 112–119, August 22-25 2005. [18] S. F. W. M. Attention based symmetry in colour images. IEEE International Workshop on Multimedia Signal Processing, Oct 30 - Nov 2 2005. [19] L. G. Minor and J. Sklansky. Detection and segmetaion of blobs in infrared images. IEEE Tranactions on Systems Man and Cyberneteics, 11(3):194–201, March 1981. [20] A. Opelt, MichaelFussenegger, A. Pinz, and P. Auer. Generic object recognition with boosting. IEEE Transactions on Pattern Recognition and Machine Intelligence, 28(3), 2006. [21] A. Rebai, A. Joly, and N. Boujemaa. Constant tangential angle elected interest points. In MIR ’06: Proceedings of the 8th ACM international workshop on Multimedia information retrieval, pages 203–212, New York, NY, USA, 2006. ACM Press.
[22] C. Schmid and R. Mohr. Local gray value invariants for image retrieval. IEEE Trans. on Pattern Analysis and Machine Intelligence, 19(5):530–535, 1997. [23] C. Schmid, R. Mohr, and C. Bauckhage. Evaluation of interest point detectors. Computer Vision, 37(2):151–172, 2000. [24] G. Sela and M. D. Levine. Real-time attention for robotic vision. Real-Time Imaging, 3(3):173–194, 1997. [25] J. A. Shufelt. Performance evaluation and analysis of vanishing point detection techniques. IEEE Trans. Pattern Anal. Mach. Intell., 21(3):282–288, 1999. [26] F. W. M. Stentiford. Attention-based vanishing point detection. Int. Conf. on Image Processing, October 8-11 2006. [27] L. Trujillo and G. Olague. Using evolution to learn how to perform interest point detection. In ICPR ’06: Proceedings of the 18th International Conference on Pattern Recognition (ICPR’06), pages 211–214, Washington, DC, USA, 2006. IEEE Computer Society. [28] R. W. and K. L. Center-of-gravity tendencies for fixations and flow patterns. Perception and Psychophysics, 5(2):81–84, September 26-28 1969.
Alexis JOLY
Nozha BOUJEMAA
INRIA Rocquencourt 78153 Le chesnay France
INRIA Rocquencourt 78153 Le chesnay France
INRIA Rocquencourt 78153 Le chesnay France
[email protected]
[email protected]
ABSTRACT This paper deals with a new interest points detector. Unlike most standard detectors which concentrate on the local shape of the signal, the main objective of this new operator is to extract interpretable points from the image context. The basic principle of this operator was the detection of radial symmetries, but we have generalized it to cover other kind of interest points. Indeed, detected points constitute centers of circles or logarithmic spirals, intersections of curves and vanishing points. Added to that, this detector does not depend on the features’ size, what makes it possible to be robust to the scaling. Detection of such points is performed using a three dimensional space called θ-space. Experiments reveal that these points are more likely to be related to visual attention. We also applied these points to the object recognition problem and state of the art performances.
Categories and Subject Descriptors I.4.7 [Image Processing and Computer Vision]: Feature Measurement—Feature representation
General Terms Algorithms
Keywords Interpretability, symmetry, CBIR
1.
INTRODUCTION
In computer vision, interest points have been widely used to solve various problems, such as image retrieval [22], object recognition [14, 15], wide baseline matching for stero pair [2], etc. Detecting these points hasn’t been a trivial task since they have to be the more relevant features in the image. It is known, for instance, for generic objects recognition or object class recognition, that only a few percentage of initially detected features are used in the final object class.
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. To copy otherwise, to republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. WOODSTOCK ’97 El Paso, Texas USA Copyright 200X ACM X-XXXXX-XX-X/XX/XX ...$5.00.
[email protected]
This proves that the most of detected points are useless and unsuitable for such tasks; this is problematic both in term of efficiency and computation time. That’s why it is essential to reduce the input of local features by improving their visual interpretability in order to concentrate effort at key locations in the image. In this paper, we introduce a new interest points detector whose goal is to extract meaningful points. In other words, detected points can be easily interpreted visually.
1.1
Related research
At some extent, interpretability is related to symmetry which is considered to be an important factor in the human visual system. Many objects around us exhibit some form of symmetry. The best symmetric objects are the circular ones because they have an infinity of symmetry axes which all intersect at the center [12, 28]. That’s way the center is considered to be an interest point with a high visual interpretability. To detect symmetry, many operators have already been developed. The Reisfeld’s generalized symmetry transform [6] looks for local reflectional symmetries and it is based on image gradients. Loy and Zelinsky [16] use also gradients for searching local radial symmetry. Other techniques [13, 9, 11] use the Hough transform and some of them exploit gradients information in order to reduce computation time. Stentiford [17, 18] use an attention mechanism to detect reflectional symmetries. In this paper, we’ll present a new symmetry operator. It is based only upon gradients directions and it is not conceived to detect reflectional symmetries. The approach was inspired from the work of Loy and Zelinsky [16]. Unlike previous methods which used to quantize gradients into angular bins [13, 19, 24], their technique is enough stable because they take into account all the gradients directions. However, this works only for a set of radii belonging to a discrete values {rk | k = 1, · · · , m}. Then, the transform can not extract features having a radius superior to the maximum radius of this set. In the approach they used, transformed image is the sum of intermediate accumulator images that we can visualize as layers, one layer for each member of rk , whereas, in our approach, output image is formed by a single pass over original image, there is no need to use any intermediate images. We are not limited to a set of radii and so we can detect centers of circles of any size. The novelty of our operator comes from its capability to detect centers of other symmetric objects (cf. figures 3 and 5). It can also detect centers of logarithmic spirals or portions of them as well as intersection of such curves. Moreover, not only can this operator extract centers of objects,
but it can also detect the intersection of straight lines, typically junctions and even “fictive” junctions. Thus, it can be tuned to be a vanishing point detector. Vanishing points are defined as the image points at which the projection of parallel lines intersect. The detection of vanishing points allows to infer 3D structure from 2D images. That’s why, when existing in a scene, a vanishing point belongs to the best interest points in that image, if not to be the best one at all, and so, it has a high visual interpretability. In this context, the approach developed by Stentiford [26] is based upon a model of human visual attention. It computes the scarcity of a point according to its neighbours. A point that occur rarely (unusual) will be attributed a high attention score. Unlike the most developed methods which use the Gaussian sphere [5, 25, 1] for detecting vanishing points, the approach presented herein exploit gradients’ directions. It bares more similarity to the work of Virginio et al. [4] in which an accumulator image was used. However, their method needs a preprocess that performs an edge detection operation and the use of two masks instead of gradients information. Only by using one detector, we are able to extract a variety of interest points having different topological nature. Each point has a high visual interpretability and it is likely to be a point of visual attention. All of this is performed without any prior knowledge of the scene.
1.2
Overview
This work provides generalization, improvements and more detailed experiments of research previously reported in the author’s conference publication [21]. Section 2 redefines the new approach of the operator, discusses the nature of the detected points and provides some illustrative results. Section 3 describes the implementation and the selection of parameters. Section 4 exhibit the experiments. And finally, section 5 presents the conclusions and the future work.
2.
THEORY
Originally, the detector presented herein was a radial symmetry operator that has been generalized, on the one hand, to cover other kinds of symmetries and in the other hand, to detect centers of logarithmic spirals and junctions. Next paragraph introduces the basic radial symmetry operator.
2.1
Circular objects detection
To ensure the detection of circle’s center, we use an election principle based on gradients’ directions [16]. Each pixel will look for all the gradients whose directions point to its position (see Figure 1). The best elected point corresponds to the point having the highest number of gradients satisfying the condition. In this way, we can construct an accumulator image, in which, each pixel value represents the number of gradients that point to that pixel. Centers of circles correspond to the local maxima on this image. Algorithmically and in terms of computation time, this approach is not efficient since, for each point k, we need to loop over all pixels in the image and test whether the gradient’s direction passes by k. While this approach is prohibitive, we can obtain the same output image by changing the technique. Giving a gradient’s direction, since each point located along this direction will increment its accumulator by 1, it will be possible to make only one single pass over the image, in which and for a pixel (x, y), we increment by 1 all of the pixels located in the gradient’s direction of
Figure 1: Illustration of the election’s principle. The pixel k looks for gradients pointing to its position. Those satisfying this condition are highlighted. Note that not all gradients are represented here. If we take into account only these gradients, the score for the pixel k will be equal to 3.
(a)
(b)
(c)
Figure 2: Example of transformed image. (a) represents the original image, (b) is the transformed image and (c) is a 3D - visualization of (b).
(x, y). The result is absolutely the same whereas the second method is obviously fastest. In order to be robust with the changes of colour, in addition to making votes in the direction of the gradient, we make also votes in its opposite direction (not to take account with the convention of which the gradient is calculated: white towards black or conversely). In [16], votes are also made in the both directions of the gradient vector, nevertheless, those made in the opposite direction were negative. This is a problematic because in the case where a symmetric feature has a light colour in the right half and a dark colour in the left half for example, votes will cancel each other. Such kind of feature is represented in figure 2 (a). An example of our output transform is given in figure 2 (b). Due to the digitization and estimation of the gradients’ directions, the process described above will not lead to a proper signal. So, transformed image will have much noise and, consequently, the detection of interest points will not be so easy. Furthermore, truly circular objects — in the strictly usual mathematical notion — are rare in natural scene. It is then noticeable that the process described above will fail to detect a circular object which has some kind of distortions. To encounter this problem, instead of voting only in the gradient’s direction, votes will be made along an arc of circle centered with reference to the gradient’s direction and whose length L depends on the radius r of the vote: L=δ·r δ is a constant angle used to define the arc (see figure 8).
Now, we put the general equation of the transform as: XX ą − ć − → → ~ E(x, y) = |∇I(i, j)· R xy (i, j)| w k R xy (i, j)k, Nxy (i, j) i
j
(1) ~ designates the gradient where I represents original image, ∇ − → symbol, R xy is the vector defined by the two pixels (x, y) and (i, j): ţ ű − → x−i R xy (i, j) = y−j and Nxy (i, j) =
− → ~ |∇I(i, j) · R xy (i, j)| − → ~ k∇I(i, j)k k R xy (i, j)k
w(·, ·) is a weighting function depending on two parameters: − → the vote’s distance k R xy k and the angle δ deduced from Nxy . Note that we are using the absolute value of the scalar − → product between the gradient vector and R xy in order to achieve positive votes in both directions. To estimate the radius of a detected feature, we use an histogram accumulator for each pixel in the image. The length of each histogram is equal to the image’s diagonal which is theoritically the largest radius that could exist. If the gradient at the pixel (i, j) is an elector for the pixel (x, y), then, the histogram of pixel (x, y) will be incremented by 1 at − → the index r = round(k R xy k) which represents the distance separating the two points. Here, function round(·) rounds each number to the nearest integer. After processing the whole image, we compute the maximum of each histogram. Finally, the radius of a feature will be the index corresponding to the maximum of the histogram related to the center of that feature. In any case, if the maximum exits in two or many indexes, we choose the highest index. In this way, we obtain a second output image “Radii image” which stores information about the features’ radius. Radii image is considered as a good informative source regarding features’ geometry. This is why we will exploit it to determine whether a pixel is an interest point or not. In fact, the detection of local maxima (3 × 3 maxima) is no sufficient because of the noise that could be included in the transformed image. Fixing a threshold even adaptive doesn’t constitute a good solution too because a noisy value could be enough high to exceed the threshold. The approach we use here is very simple and efficient at the same time. Is regarded as point of interest any pixel having a value of symmetry higher than all the values of the pixels located in the disc defined by its symmetry’s radius. By this definition, even in case where a circle is included in an other one, both of two circles will be detected as the interior circle will have a symmetric value inferior to the biggest one because it has less electors than it.
2.2
~ θ I(i, j) the direction of the gradient’s vecWe denote by ∇ tor rotated by θ, equation 1 will be changed to: XX ą − ć → → ~ θ I(i, j)·− Eθ (x, y) = |∇ R xy (i, j)| w k R xy (i, j)k, Nxy (i, j) i
j
(2) Eθ is the transformed image with angle θ.
2.2.1
Variety of symmetric objects
Radial symmetry objects aren’t only circular objects. In figure 3 is represented another type of radial symmetric object whose center can not be detected using other existing techniques.
Figure 3: A kind of radial symmetric feature whose center can’t be detected using standard detectors. In blue are represented gradients and in red are represented electors which are the result of rotating gradients by the angle θ.
2.2.2
Characteristic of logarithmic spirals
The polar equation of a logarithmic spiral is given by: ρ = a · exp (kφ)
(3)
where k = cot(ψ). ψ is a constant representing the equiangular spiral. In the special case where ψ = π2 , we obtain a circle. Logarithmic spiral makes the constant angle ψ with any radius vector. This is an interesting geometric property, since, by rotating the gradient vector by θ = φ − α (where φ and α represent the polar and the gradient’s angle respectively), it will point to the center of the spiral (see figure 4).
Generalization of the election principle
The approach described above used for detecting circles is generalized here to cover other symmetric objects. It will also allow the detection of centers of logarithmic spirals, junctions and vanishing points. The generalization consists in keeping the election principle and using other directions rather than the gradient’s direction one. Given θ ∈ [− π2 , π2 ], if α designates the gradient’s direction, the elector vector’s direction will be (α + θ).
Figure 4: Example of logarithmic spiral (ψ =
2.2.3
7π ). 16
Junctions and vanishing points
Now, let θ = ± π2 , elector vectors will have perpendicular directions to that of gradients. In the case of straight lines, votes will be accumulated along that lines. Therefore,
if there is any intersection of these lines (even fictive), it will have a high score thanks to the votes from different directions. By fictive junction, we mean that there is no effective intersection presented in the scene but the extension of the lines will intersect (see figure 5). In case of vanishing points, the extension of 3D parallel lines projected in 2D image will be considered as a junction between segment lines, and so, vanishing point will be seen as junctions.
(a)
(b)
(c)
Figure 7: Example of detected points. The representation used here takes into account the estimation of feature’s radius. (a) gives an example of circles’ detection (θ = 0). The best elected point in (b) corresponds to θ = 8◦ while (c) reveals a vanishing point (θ = 90◦ ).
Figure 5: Example of junctions. It is essential to notice here that the radius’ estimation method described in section 2.1 will not give good results because the electors are not situated at the same radius. The maximum of histogram occur at the radius having the highest number of electors and it is not possible to predict whether this maximum will exist due to electors which are close to the junction or due to electors which are far from the junction.
2.2.4 θ-space analysis By applying the transform to several θ angles, we construct the θ-space which can theoritically be continuous if θ takes continuous values in [− π2 , π2 ]. θ-space is a good example of 3-dimensional space which maintain the energy unchanged (by neglecting side effect). Indeed, while going through this space, we notice that energy, initially dispersed for a given pixel, concentrates with a specific angle θ0 to disperse again while going away from θ0 . While the concept of a continuous θ-space is interesting, actually, taking only a subset of θ values is normally enough to yield a representative result. Figure 6 gives two examples of synthetic images with their θ-spaces sampled. It is noticeable that 3D − maxima in θ-space have to be more stable than 2D − maxima of a transformed image at a given value θ compared at what is usually done in scale-space theory [8]. This fact let θ-space be a powerful tool to detect interest points having different topological natures. However, 3D − maxima extraction requires that θ-space has a kind of continuity to ensure stable detection. Unfortunately, such a criterion necessitates the use of a high number of samples and so it will affect the computing time considerably.
2.3
Detection examples
In figure 7, we give some illustrative results showing the best elected points in three examples. In these examples, we didn’t perform a 3D−maxima detection. We’ve just located 2D − maxima in θ-space sampled images and then, we’ve sorted them in descending order according to the transform’s value. Only the best elected points are kept.
3.
IMPLEMENTATION DETAILS
Initially, we define a set of θ values Θ = {θmin , · · · , θmax } ⊂ [− π2 , π2 ]. For each θ ∈ Θ, we construct the transformed image Eθ and the radii image IR . Eθ is initially zero. To compute IR , we define an histogram accumulator for each pixel and we initialize it to zero. The first step in the process is to compute gradients’ directions. For a given pixel k, the gradient’s direction α is given by: ş ∇I(k) ~ · yť α = arctan ~ ∇I(k) ·x where x and y are the unit vectors according to the width and the height of the image respectively: ţ ű ţ ű 1 0 x= y= 0 1 Now, we will consider only gradients whose magnitude is higher than a defined threshold T . Indeed, small gradients are not significant and usually they result from the noise effect. If they are took into account, on the one hand, they lead to a noisy image transform, on the other hand, they will considerably increase computation time. Threshold T is determined empirically after several tests had been made on different images. Now, for each pixel k, we compute the elector vector’s direction which is equal to α + θ. Let δ be the angle used to determine the arc of votes. For each radius r and for each angle λ ∈ [α + θ − 2δ , α + θ + 2δ ], we compute coordinates of the two affected pixels p and q located on both sides of k (see figure 8). Equations are given by: ş ą ćť p = k + round r · cos(α + θ + λ)x + sin(β + θ + λ)y (4) ş ą ćť (5) q = k − round r · cos(α + θ + λ)x + sin(α + θ + λ)y In transformed image Eθ , each affected pixel will be incremented by 1 as: Eθ (p) = Eθ (p) + 1
(6)
Eθ (q) = Eθ (q) + 1
(7)
Meanwhile, we increment by 1 at index r the histogram which corresponds to pixel k. In order to reduce the excessive use of computer memory, we can restrict the histogram’s size by a factor of 2 or 4. The use of bins doesn’t
(a) The best elected point is represented by a red cross in original image. It is detected at θ = 40◦ .
(b) Best elected points are represented in red in original image. There are 5 points detected at θ = 0◦ and only one point detected at θ = 90◦ . Figure 6: Two examples of samples of θ-space. Original image is given in the top-left of each example and contains the best elected points. Representation takes into account the estimation of features’ radius. Values of θ used here from left to right: −90◦ , −81◦ , −65◦ , −57◦ , −40◦ , −24◦ , −16◦ , 0◦ , 16◦ , 24◦ , 40◦ , 57◦ , 65◦ , 81◦ , 90◦ . is related to the significance of extracted points and finally an application to the object recognition.
4.1
Figure 8: Representation of the affected pixels p and q at a distance r away from k. Note that votes will cover the entire highlighted area. affect considerably images having big dimensions where the histogram’s size grows. While normally the step of λ should be Lδ = r1 , it is possible to fix the step in order accelerate the algorithm. However, the quality of transformed image deteriorates and this is more clear when increasing the step and when image’s dimensions are excessively big. After achieving the voting process, we blur Eθ image in order to reduce noise and then we perform local maxima detection.
4.
EXPERIMENTS
To evaluate our detector, experiments were carried out using three criteria: ²-repeatability [23], interpretability which
Repeatability measurement
In these experiments, we used a database of 100 images from different categories: natural scenes such as animals, trees, grass, sea and man-made scenes such as building, roads, vehicles, etc. θ-space was sampled into the following set of values {− π2 , − π3 , − π6 , 0, π6 , π3 , π2 }. The ² used for computing the repeatability is equal to 5. Repeatability rate was calculated for some image transformations: gaussian noise, gaussian blur, jpeg compression, salt and pepper noise, negative, flip and mirror. Results show that junctions (θ = ± π2 ) have always higher repeatability rate than all interest points detected at other angles in any transformation considered. The distribution of repeatability rate according to θ is given in figure 9. Note that the value of repeatability rate for a given θ represents the average of repeatability rates of all the transformations cited above. In order to measure the seperability of detected features, we computed the number of interest points according to each angle θ. The average of these numbers between all database images is given in figure 10. This graph shows that circular objects are few and that, contrary, junctions are the most present features. Between the two types of points and while moving away from the angle θ = 0, the number of detected points increases while following a convex curve. According to the figure 9, one may deduce that junctions are the most stable points that could exist in nature. How-
Figure 9: Average of repeatability rate to some image transformations according to angle θ (² = 5).
Figure 10: Average of interest points’ number detected in the whole database according to angle θ. ever, if we calculate the contribution of each interest point to compute the repeatability rate for each angle θ (i.e. we divide the repeatability rate corresponding to a given θ by the number of points detected at this angle), we find that the greatest value is obtained at θ = 0. Moreover, this value is definitely higher than those obtained in other angles (see figure 11). According to this graph, we notice that although the circular objects are the least present, they are the most repeatable in detection. Such result confirms that repeatability measurement as defined in [23] is not an appropriate metric to evaluate the performance of our detector. This definition of repeatability, indeed, deals only with tractability problem. It doesn’t inform about global seperability and doesn’t characterize the distinctivenss of detected features [27].
4.2
Distinctiveness
In order to test whether our detected features are distinctive or not, we are based upon human decision. In fact, we chose some classes of objects which contain relevant features according to the human interpretation and we tried to detect theses features only by keeping the three best elected points using two values of θ: 0 and π2 . For these expermiments, we used Caltech 101 [7] and Caltech 256 [10] databases which contain a variety of symmetric objects. For the following classes: bowling ball, tennis ball, soccer ball, desk globe, frisbee, mars, watch and steering wheel, the objective was to determine the center of those features. For the touring bike and motorbikes classes, the objective was to detect one of the centers of the two wheels.
Figure 11: Repeatability rate per point according to angle θ (² = 5).
(a)
(b)
(c)
Figure 12: Example of transformed image. (a) represents the original image, (b) is the transformed image and (c) is a 3D - visualization of (b). Concerning “faces easy” class, it was necessary to detect the center of only one eye. And finally, for the camera class, we aimed to detect the lens. Results are illustrated in the table 1. Note that the decision defining whether a feature is detected or not was made by three persons. Object’s class Bowling ball Desk-globe Frisbee Mars Steering-wheel Tennis-ball Touring-bike Faces easy Motorbikes Camera Soccer ball Watch
Percentage of detected features 73.08 96.34 88.89 91.03 90.72 73.47 90 77.01 97.82 94 87.5 91.21
Table 1: Percentage of detected features according to the object’s class. Now, concerning the detection of junctions, we used the classes butterfly and dragonfly to ensure detection of the back of these insects which represents the wings’ intersec-
tion. For the class ceiling fan, the objective was to detect intersection of blades. In the same way, we tried to detect the intersection of tripod’s legs. Another object containing one junction is scissor. Above this, we used two other classes envolving more than one junction which are anchor and menorah. The objective here was to detect only one junction. Results of detection are given in table 2. Moreover, we created a database of 76 images of different sizes collected from Internet and where a vanishing point is present. Note that not all of images chosen contain perfect straight lines. However, we succeeded in detecting 72% of vanishing points which represents a good score. Object’s class Tripod Anchor Butterfly Ceiling fan Dragonfly Menorah Scissors Vanishing points base
Percentage of detected features 83.93 88.1 76.92 97.87 97.05 91.95 97.44 72
Table 2: Percentage of detected features according to the object’s class.
4.3 4.3.1
Basic Objects’ class recognition tests Description
We describe here some preliminary experiments for objects’ class recognition. Since we did not yet defined specific local descriptors for our new interest points, we used non-specialized histogram based descriptors. They are computed around each interest point in a square window with an arbitrary side equal to 50 pixels. Each local descriptor is a 128-dimensional vector F composed of the concatenation of six local histograms described in [3] (Fourier based histogram, weighted color histograms, edge orientation histogram, Hough based histogram). Obviously, these descriptors are not adapted to our interest points since they do not take care about the topology of the detected features and they do not characterize any geometrical structure. For the learning, we use a simple feature selection strategy that consists in keeping the three best local features in each image of the learning set, according to their distinctiveness. The distinctiveness of a local feature is determined in a way similar to the Weak-Hypothesis-Finder defined in the boosting approach of Opelt et al. [20]. A distance bewteen any feature d(Fqj and any image Ik of the training set is defined as:
4.3.2
Tests of objects’ recognition were applied on ImagEVAL databases. ImagEVAL1 is an evaluation compaign of contentbased image retrieval techniques and automatic description of images. The fourth task of its benchmark was dedicated to object recognition. There are ten classes used: armored vehicle, car, cow, Eiffel tower, minaret and mosque, plane, road signs, sunglasses, tree and US flag. Moreover, proposed objects had different poses, contexts and sizes. The final database is formed with 14,000 images. Note that some images contain more than one object’s class and that 5,000 images didn’t include any objects.
4.3.3
5.
Results
CONCLUSIONS AND FUTURE WORK
We have introduced a new interest points detector which focus on salient points having a high visual interpretability. Originally, the detector was a radial symmetry operator which has been generalized to allow the extraction of other interpretable points. Generalization leads to a three dimensional space whose third dimension depends on angle θ. This space called θ-space constitutes a rich analyze tool allowing the extraction of points having different topological natures. Experiments showed that our interest points concentrate on the best relevant features in a given object and that these features are almost detected for the same object’s class. This is how these points are more likely to be related to the visual attention mechanism. To verify this assumption, an interesting experiment will be the use of an eye-tracking system. This represents one goal of our future works. Another important task lies in the possibility of characterizing an object using our interest points since each point has a characteristic angle θ. The objective will be the creation of an invariant and robust description using geometric relations. Such description will not be affected by the precision of detection. Therefore, standard repeatability measurement is not appropriate to judge whether the detection is good or not. Justification of fiability could be derived from applying the detector to some computer vision problems, in particular, objects recognition gave good results while using non specialized histogram descriptors. Such result is very promising to define new local descriptors adapted to the extracted features.
6. REFERENCES [1] S. T. Barnard. Interpreting perspective images. Artificial intelligence, 21(4):435–462, 1983. [2] A. Baumberg. Reliable feature matching across widely separated views. In CVPR, pages 1774–1781, 2000. [3] N. Boujemaa, J. Fauqueur, M. Ferecatu, F. Fleuret, V. Gouet, B. L. Saux, and H. Sahbi. Ikona for interactive specific and generic image retrieval. In Proc. of Workshop on MultiMedia Content Based Indexing and Retrieval, 2001. [4] V. Cantoni, L. Lombardi, M. Porta, and N. Sicard. Vanishing point detection: representation analysis and new approaches. Image Analysis and Processing, 3(3):90–94, September 26-28 2001.
d(Fqj , Ik ) = min d(Fqj , Fki ) 1≤i≤nk
For each Fqj , the images Ik are then sorted with increasing distances and the best threshold θ on the distance is then chosen according to the classification error. The retrieval of the images belonging to a given class of objects consists in computing a matching kernel between the candidate images and all the learning images of a given class. The matching kernel is applied only on the features selected by the learning step and the final score is the average value of the kernel function on the learning images of the given class.
Evaluation
1
http://www.imageval.org/
[5] R. Collins and R. Weiss. Vanishing point calculation as a statistical inference on the unit sphere. In International Conference on Computer Vision, pages 400–403, December 1990. [6] R. D., W. H., and Y. Y. Context free attentional operators: the generalized symmetry transform. Int. J. of Computer Vision, Special Issue on Qualitative Vision, 1994. [7] L. Fei-Fei, R. Fergus, and P. Perona. Learning generative visual models from few training examples: an incremental bayesian approach tested on 101 object categories. IEEE. CVPR 2004, Workshop on Generative-Model Based Vision, 2004. [8] L. Florack and A. Kuijper. The topological structure of scale-space images. Journal of Mathematical Imaging and Vision, 12(1):65–79, February 2000. [9] J. Y. Goulermas and P. Liatsis. Incorporating gradient estimations in a circle-finding probabilistic hough transform. Pattern Analysis and Applications, 2(3):239–250, August 1999. [10] G. Griffin, A. Holub, and P. Perona. The caltech-256, caltech technical report. [11] N. Guil and E. Zapata. Lower order circle and ellipse hough transform, October 1997. [12] L. P. J. and N. C. F. Symmetry catches the eye. Eye movements: From physiology to cognition, pages 353–361, 1987. [13] C. Kimme, D. Ballard, and J. Sklansky. Finding circles by an array of accumulators. Commun. ACM, 18(2):120–122, 1975. [14] S. Lazebnik, C. Schmid, and J. Ponce. Semi-local affine parts for object recognition. In British Machine Vision Conference, volume volume 2, pages 779–788, 2004. [15] D. G. Lowe. Object recognition from local scale-invariant features. In Proc. of Int. Conf. on Computer Vision, pages 1150–1157, 1999. [16] G. Loy and A. Zelinsky. Fast radial symmetry for detecting points of interest. IEEE Transactions on Pattern Analysis and Machine Intelligence, 25(8), August 2003. [17] S. F. W. M. Attention based facial symmetry detection. International Conference on Advances in Pattern Recognition, pages 112–119, August 22-25 2005. [18] S. F. W. M. Attention based symmetry in colour images. IEEE International Workshop on Multimedia Signal Processing, Oct 30 - Nov 2 2005. [19] L. G. Minor and J. Sklansky. Detection and segmetaion of blobs in infrared images. IEEE Tranactions on Systems Man and Cyberneteics, 11(3):194–201, March 1981. [20] A. Opelt, MichaelFussenegger, A. Pinz, and P. Auer. Generic object recognition with boosting. IEEE Transactions on Pattern Recognition and Machine Intelligence, 28(3), 2006. [21] A. Rebai, A. Joly, and N. Boujemaa. Constant tangential angle elected interest points. In MIR ’06: Proceedings of the 8th ACM international workshop on Multimedia information retrieval, pages 203–212, New York, NY, USA, 2006. ACM Press.
[22] C. Schmid and R. Mohr. Local gray value invariants for image retrieval. IEEE Trans. on Pattern Analysis and Machine Intelligence, 19(5):530–535, 1997. [23] C. Schmid, R. Mohr, and C. Bauckhage. Evaluation of interest point detectors. Computer Vision, 37(2):151–172, 2000. [24] G. Sela and M. D. Levine. Real-time attention for robotic vision. Real-Time Imaging, 3(3):173–194, 1997. [25] J. A. Shufelt. Performance evaluation and analysis of vanishing point detection techniques. IEEE Trans. Pattern Anal. Mach. Intell., 21(3):282–288, 1999. [26] F. W. M. Stentiford. Attention-based vanishing point detection. Int. Conf. on Image Processing, October 8-11 2006. [27] L. Trujillo and G. Olague. Using evolution to learn how to perform interest point detection. In ICPR ’06: Proceedings of the 18th International Conference on Pattern Recognition (ICPR’06), pages 211–214, Washington, DC, USA, 2006. IEEE Computer Society. [28] R. W. and K. L. Center-of-gravity tendencies for fixations and flow patterns. Perception and Psychophysics, 5(2):81–84, September 26-28 1969.
