Shape Retrieval using Contour Features and Distance Optimization
International Conference on Computer Vision Theory and Applications
VISAPP 2010 – Angers, France
Shape Retrieval using Contour Features and Distance Optimization Daniel Carlos Guimarães Pedronette [email protected]
Ricardo da S. Torres [email protected] Institute of Computing University of Campinas - Brazil
1
Outline • Shape Description and Contour Features – Local, Regional, Global and Histogram Features – Irrelevant Contour Segments – Matching and Distance Computation
• Distance Optimization Algorithm – The Algorithm – Clustering Approach
• Experimental Results • Conclusions 2
Outline • Shape Description and Contour Features – Local, Regional, Global and Histogram Features – Irrelevant Contour Segments – Matching and Distance Computation
• Distance Optimization Algorithm – The Algorithm – Clustering Approach
• Experimental Results • Conclusions 3
Contour Features • Features to describe the contour of an object – Local Features: properties in a small neighborhood of contour pixel pi – Global Features: properties that consider all the object – Regional Features: relation between local and global features
• Feature Vector Extraction – Feature Functions {F0, F1, . . . , Fm} – Contour pixels {p0,p1,…,pn} – Matrix • fv[i,j] = Fj (pi) 4
Local Features • Normal Angle - Θn – Angle between normal vector and horizontal line
• Concavity – Sample Lines • Region A + Region B • Considering radius of analysis
• Opposite Opening
MPEG-7 Shape: beetle
– Sample Lines • Region C 5
Global Features • Distance to Center of Mass - dc – Euclidean distance between pi (contour pixel) and pc (central pixel)
• Angle to Center of Mass Θc – Directional information about relation of pixels pi (contour pixel) and pc (central pixel)
6
Regional Features • Opposite Distance - do – Determine if pi is in main body of object (high values) or in branches (low values)
• Difference Normal Angle and Angle to Center of Mass – Directional information of contour
• Opening – Information about branches in a neighborhood – Sample lines (in blue)
7
Histogram Features • Histogram for Contour Features – Matching of feature vectors: different sizes of contour for corresponding segments (apple leaf) – Solution: histogram-based features for characterizing the shape complexity
8
Irrelevants Contour Segments • Irrelevants Contour Segments – Segments of contour which have some instability in the shape contour – Small changes in the angle/perspective that the image is seen, such segments may disappear in the visual perception of the contour of the object
• Detection: – Moving average of opening feature
9
Distance Computation • Normalization for Contour Features – Different features have values in different ranges – Normalization using max and avg values
• Matching Distance: – Features Distances used for Matching • Distance to Center of Mass • Opposite Distance
• Final Distance – Computed by a weighted average of distances for each contour feature 10
Outline • Shape Description and Contour Features – Local, Regional, Global and Histogram Features – Irrelevant Contour Segments – Matching and Distance Computation
• Distance Optimization Algorithm – The Algorithm – Clustering Approach
• Experimental Results • Conclusions 11
Distance Optimization Algorithm • Similarity of Ranked lists – If two images are similar, their ranked lists should be similar too
• Distance Optimization Algorithm 1. Create clusters: – by exploring information of ranked lists
2. Update distances: – distances among images of a same cluster are decreased
• Convergence – Process is repeated until the quality of clusters does not improve – Cohesion for measuring quality of ranked lists • Quantity of references among ranked lists of images of same cluster
12
Clustering Approach • Graph-based clustering using ranked lists – Two images are assigned to the same cluster if they are clustersimilar – Basically, two images are cluster-similar if they refer to each other at the first positions of their ranked lists
13
Clustering Approach • Clusters verification: – After initial clustering (by cluster-similar functions), the clusters are verified:
– Merge clusters • Two clusters are merged if cohesion of new (merged) cluster are greater than those of original clusters
– Divide clusters • If a cluster has a low cohesion, this cluster is split • Images of split clusters are processed with a more restrictive cluster-similar function
14
Outline • Shape Description and Contour Features – Local, Regional, Global and Histogram Features – Irrelevant Contour Segments – Matching and Distance Computation
• Distance Optimization Algorithm – The Algorithm – Clustering Approach
• Experimental Results • Conclusions 15
Experimental Results • Datasets – MPEG-7 – Kimia
16
Outline • Shape Description and Contour Features – Local, Regional, Global and Histogram Features – Irrelevant Contour Segments – Matching and Distance Computation
• Distance Optimization Algorithm – The Algorithm – Clustering Approach
• Experimental Results • Conclusions 17
Conclusions • Contributions – Shape Description and Contour Features • New approaches for: – Several contour features combination – Excluding irrelevant contour segments – Histogram features
– Distance Optimization Algorithm • Gain for CFD Descriptor: +8,13% for Recall at 40 / MPEG-7 Dataset • CFD Shape Descriptor + Distance Optimization: – Results comparable to state-of-the-art methods
• Future work – Application of Distance Optimization to color and texture descriptors 18
International Conference on Computer Vision Theory and Applications
VISAPP 2010 – Angers, France
Shape Retrieval using Contour Features and Distance Optimization Daniel Carlos Guimarães Pedronette [email protected]
Ricardo da S. Torres [email protected] Institute of Computing University of Campinas - Brazil
19
International Conference on Computer Vision Theory and Applications
VISAPP 2010 – Angers, France
• Thanks! Daniel Carlos Guimarães Pedronette º [email protected]
Ricardo da S. Torres º [email protected] º Institute of Computing University of Campinas - Brazil 20
Distance Computation • Normalization for Contour Features (curves) • Matching Distance: – Features: Distance to Center of Mass and Opposite Distance
• Final Distance (weighted average)
21
Cohesion Computation • Let C = {img1, img2, …, imgn} be a collection (or a cluster) of images, cohesion is defined as follows:
where S is a function S: i → {0,1}, that assumes value 1 if C contains the image at position i of ranked list and assumes value 0, otherwise.
22
Distance Optimization
23
VISAPP 2010 – Angers, France
Shape Retrieval using Contour Features and Distance Optimization Daniel Carlos Guimarães Pedronette [email protected]
Ricardo da S. Torres [email protected] Institute of Computing University of Campinas - Brazil
1
Outline • Shape Description and Contour Features – Local, Regional, Global and Histogram Features – Irrelevant Contour Segments – Matching and Distance Computation
• Distance Optimization Algorithm – The Algorithm – Clustering Approach
• Experimental Results • Conclusions 2
Outline • Shape Description and Contour Features – Local, Regional, Global and Histogram Features – Irrelevant Contour Segments – Matching and Distance Computation
• Distance Optimization Algorithm – The Algorithm – Clustering Approach
• Experimental Results • Conclusions 3
Contour Features • Features to describe the contour of an object – Local Features: properties in a small neighborhood of contour pixel pi – Global Features: properties that consider all the object – Regional Features: relation between local and global features
• Feature Vector Extraction – Feature Functions {F0, F1, . . . , Fm} – Contour pixels {p0,p1,…,pn} – Matrix • fv[i,j] = Fj (pi) 4
Local Features • Normal Angle - Θn – Angle between normal vector and horizontal line
• Concavity – Sample Lines • Region A + Region B • Considering radius of analysis
• Opposite Opening
MPEG-7 Shape: beetle
– Sample Lines • Region C 5
Global Features • Distance to Center of Mass - dc – Euclidean distance between pi (contour pixel) and pc (central pixel)
• Angle to Center of Mass Θc – Directional information about relation of pixels pi (contour pixel) and pc (central pixel)
6
Regional Features • Opposite Distance - do – Determine if pi is in main body of object (high values) or in branches (low values)
• Difference Normal Angle and Angle to Center of Mass – Directional information of contour
• Opening – Information about branches in a neighborhood – Sample lines (in blue)
7
Histogram Features • Histogram for Contour Features – Matching of feature vectors: different sizes of contour for corresponding segments (apple leaf) – Solution: histogram-based features for characterizing the shape complexity
8
Irrelevants Contour Segments • Irrelevants Contour Segments – Segments of contour which have some instability in the shape contour – Small changes in the angle/perspective that the image is seen, such segments may disappear in the visual perception of the contour of the object
• Detection: – Moving average of opening feature
9
Distance Computation • Normalization for Contour Features – Different features have values in different ranges – Normalization using max and avg values
• Matching Distance: – Features Distances used for Matching • Distance to Center of Mass • Opposite Distance
• Final Distance – Computed by a weighted average of distances for each contour feature 10
Outline • Shape Description and Contour Features – Local, Regional, Global and Histogram Features – Irrelevant Contour Segments – Matching and Distance Computation
• Distance Optimization Algorithm – The Algorithm – Clustering Approach
• Experimental Results • Conclusions 11
Distance Optimization Algorithm • Similarity of Ranked lists – If two images are similar, their ranked lists should be similar too
• Distance Optimization Algorithm 1. Create clusters: – by exploring information of ranked lists
2. Update distances: – distances among images of a same cluster are decreased
• Convergence – Process is repeated until the quality of clusters does not improve – Cohesion for measuring quality of ranked lists • Quantity of references among ranked lists of images of same cluster
12
Clustering Approach • Graph-based clustering using ranked lists – Two images are assigned to the same cluster if they are clustersimilar – Basically, two images are cluster-similar if they refer to each other at the first positions of their ranked lists
13
Clustering Approach • Clusters verification: – After initial clustering (by cluster-similar functions), the clusters are verified:
– Merge clusters • Two clusters are merged if cohesion of new (merged) cluster are greater than those of original clusters
– Divide clusters • If a cluster has a low cohesion, this cluster is split • Images of split clusters are processed with a more restrictive cluster-similar function
14
Outline • Shape Description and Contour Features – Local, Regional, Global and Histogram Features – Irrelevant Contour Segments – Matching and Distance Computation
• Distance Optimization Algorithm – The Algorithm – Clustering Approach
• Experimental Results • Conclusions 15
Experimental Results • Datasets – MPEG-7 – Kimia
16
Outline • Shape Description and Contour Features – Local, Regional, Global and Histogram Features – Irrelevant Contour Segments – Matching and Distance Computation
• Distance Optimization Algorithm – The Algorithm – Clustering Approach
• Experimental Results • Conclusions 17
Conclusions • Contributions – Shape Description and Contour Features • New approaches for: – Several contour features combination – Excluding irrelevant contour segments – Histogram features
– Distance Optimization Algorithm • Gain for CFD Descriptor: +8,13% for Recall at 40 / MPEG-7 Dataset • CFD Shape Descriptor + Distance Optimization: – Results comparable to state-of-the-art methods
• Future work – Application of Distance Optimization to color and texture descriptors 18
International Conference on Computer Vision Theory and Applications
VISAPP 2010 – Angers, France
Shape Retrieval using Contour Features and Distance Optimization Daniel Carlos Guimarães Pedronette [email protected]
Ricardo da S. Torres [email protected] Institute of Computing University of Campinas - Brazil
19
International Conference on Computer Vision Theory and Applications
VISAPP 2010 – Angers, France
• Thanks! Daniel Carlos Guimarães Pedronette º [email protected]
Ricardo da S. Torres º [email protected] º Institute of Computing University of Campinas - Brazil 20
Distance Computation • Normalization for Contour Features (curves) • Matching Distance: – Features: Distance to Center of Mass and Opposite Distance
• Final Distance (weighted average)
21
Cohesion Computation • Let C = {img1, img2, …, imgn} be a collection (or a cluster) of images, cohesion is defined as follows:
where S is a function S: i → {0,1}, that assumes value 1 if C contains the image at position i of ranked list and assumes value 0, otherwise.
22
Distance Optimization
23
