Affine invariant non-rigid shape analysis - Semantic Scholar
Technion - Computer Science Department - Tehnical Report CIS-2012-01 - 2012
Affine invariant non-rigid shape analysis Dan Raviv Dept. of Computer Science Technion, Israel
Ron Kimmel Dept. of Computer Science Technion, Israel
[email protected]
[email protected]
Figure 1: Voronoi diagrams of ten points selected by farthest point sampling. Diffusion distances were used based on Euclidean metric (top) and an affine invariant one (bottom).
Abstract Shape recognition deals with the study geometric structures. Modern surface processing methods can cope with non-rigidity - by measuring the lack of isometry, deal with similarity - by multiplying the Euclidean arc-length by the Gaussian curvature, and manage equi-affine transformations - by resorting to the special affine arc-length definition in classical affine geometry. Here, we propose a computational framework that is invariant to the affine group of transformations (similarity and equi-affine) and thus, by construction, can handle non-rigid shapes. Technically, we add the similarity invariant property to an equi-affine invariant one. Diffusion geometry encapsulates the resulting measure to robustly provide signatures and computational tools for affine invariant surface matching and comparison.
1. Introduction Differential invariants for planar shape matching and recognition were introduced to computer vision in the 80’s [49] and studied in the early 90’s [13, 12, 20, 14, 17, 11], where global invariants were computed in a local manner to overcome numerical sensitivity of the differential forms. Scale space entered 1
Technion - Computer Science Department - Tehnical Report CIS-2012-01 - 2012
the game as a stabilizing mechanism, for example in [15], where locality was tuned by a scalar indicating how far one should depart from the point of interest. Along a different path, using a point matching oracle to reduce the number of derivatives, semi-differential signatures were proposed in [46, 47, 33, 39, 18]. Non-local signatures, which are more sensitive to occlusions, were shown to perform favorably in holistic paradigms [36, 10, 35, 28]. At another end, image simplification through geometric invariant heat processes were introduced and experimented with during the late 90’s, [44, 2, 27]. In the beginning of this century, scale space theories gave birth to the celebrated scale invariant feature transform (SIFT) [30] and the affine-scale invariant feature transform ASIFT [34], that are used to successfully locate repeatable informative (invariant) features in images. Matching surfaces while accounting for deformations was performed with conformal mappings [29], embedding to finite dimensional Euclidean spaces [24] and infinite ones [4, 43], topological graphs [26, 48], and exploiting the Gromov-Hausdorff distance [31, 7, 19]. Which are just a subset of the numerous methods used in this exploding field. Another example, relevant to this paper is diffusion geometry, introduced in [21] for manifold learning and first applied for shape analysis in [8]. This geometry can be constructed by the eigen-strucutre of the Laplace-Beltrami operator. The same decomposition was recently used in [45, 37, 38, 8] to construct surface descriptors for shape retrieval and matching. In this paper, following the adoption of metric/differential geometry tools to image-analysis, we introduce a new geometry for affine invariant surface analysis. In [42, 3], an equi-affine invariant metric for surfaces with effective Gaussian curvature were presented, while a scale invariant metric was the main theme of [1, 9]. Here, we introduce a framework that handles both, that is, affine transformations in its most general form including similarity, that is scaling and isometry. The paper is organized as follows. In Section 2 we introduce the affine invariant metric. Section 3 briefly reviews the main concepts of diffusion geometry. Section 4 is dedicated to experimental results, and Section 5 concludes the paper.
2. Affine metric construction We model a surface (S, g) as a compact two dimensional Riemannian manifold S with a metric tensor g. We further assume that S is embedded in R3 by a regular map S : U ⊂ R2 → R3 . The construction of a Euclidean metric tensor can be obtained from the re-parameterization invariant arclength ds of a parametrized curve C(s) on S, as the simplest Euclidean invariant is length, we search for the parameterization s that would satisfy |Cs | = 1, or explicitly, 1 = hCs , Cs i = hSs , Ss i ∂S du ∂S dv ∂S du ∂S dv + , + = ∂u ds ∂v ds ∂u ds ∂v ds −2 2 = ds g11 du + 2g12 dudv + g22 dv 2 ,
(1)
where gij = hSi , Sj i,
(2)
using the short hand notation S1 = ∂S/∂u, S2 = ∂S/∂v, and u and v are the coordinates of U . An infinitesimal displacement ds on the surface is thereby given by ds2 = g11 du2 + 2g12 dudv + g22 dv 2 . 2
(3)
Technion - Computer Science Department - Tehnical Report CIS-2012-01 - 2012
The metric coefficients translate the surface parametrization into a Euclidean invariant distance measure on the surface. The equi-affine transformation, defined by the linear operator AS + b, where det(A) = 1, is a bit more tricky to deal with, see [5, 16]. Consider the curve C ∈ S parametrized by w. The equi-affine transformation is volume preserving, and thus, its invariant metric could be constructed by restricting the volume defined by Su , Sv , and Cww to one. That is, 1 = det(Su , Sv , Cww ) = det (S u , Sv , Sww ) 2 dv 2 du dv d2 u d2 v du + Svv 2 + Su 2 + Sv 2 = det Su , Sv , Suu 2 + 2Suv dw dw dw dw dw dw 2 = dw−2 det Su , Sv , Suu du2 + 2Suv dudv + S dv vv (4) = dw−2 r11 du2 + 2r12 dudv + r22 dv 2 , where now the metric elements are given by rij = det (S1 , S2 , Sij ), and we extended the short hand 2 2 ∂2S notation to second order derivatives by which S11 = ∂∂uS2 , S22 = ∂∂vS2 , and S12 = ∂u∂v . Note that √ the second fundamental form in the Euclidean case is given by bij = grij where g = det(gij ) = 2 . The equi-affine re-parametrization invariant metric [16, 5] reads g11 g22 − g12 1
qij = |r|− 4 rij ,
(5)
2 . where r = det(rij ) = r11 r22 − r12 In [42] others and us modified the equi-affine metric to accommodate for surfaces with effective Gaussian curvature. There, the idea was to project the metric tensor matrix qij onto the class of positive definite subspaces. Practically, we decompose the tensor into its eigen-structure, take the absolute value of its eigenvalues, and compose the tensor back into a valid invariant metric. Next, we resort to the similarity (scale and isometry) invariant metric proposed in [1], according to which, scale invariance is obtained by multiplying the metric by the Gaussian curvature. The Gaussian curvature is defined by the ratio between the determinants of the second and the first fundamental forms and is denoted by K. We propose to compute the Gaussian curvature of the equi-affine invariant metric, and construct a new metric by multiplying the metric elements by |K|. Specifically, consider the surface (S, q), where qij is the equi-affine invariant metric, and compute the Gaussian curvature K q (S, q) at each point. The affine invariant metric is defined by
hij = |K q | qij .
(6)
Let us justify the above construction of an affine invariant metric for surfaces. 2.1. Proof of invariance
In order to prove the affine invariance of our metric construction let us first justify the scale invariant metric constructed by multiplication of a given Euclidean metric by the Gaussian curvature. Assume that the surface S is scaled by a scalar α > 0, such that ˜ v) = αS(u, v). S(u,
(7)
In what follows, we omit the parameters u,v for brevity, and denote the quantities q computed for the scaled surface by q˜. The first and second fundamental forms are scaled by α2 and α respectfully, g˜ij = hαSi , αSj i = α2 hSi , Sj i = α2 gij , 3
Technion - Computer Science Department - Tehnical Report CIS-2012-01 - 2012
˜bij = hαSij , N i = α hSij , N i = αbij ,
(8)
which yields det(˜ g ) = α4 det(g) det(˜b) = α2 det(b).
(9)
Since the Gaussian curvature is the ratio between the determinants of the second and first fundamental forms, we readily have that α2 det(b) 1 det(˜b) ˜ = 4 = 2 K, K≡ det(˜ g) α det(g) α
(10)
from which we conclude that multiplying the Euclidean metric by the magnitude of its Gaussian curvature indeed provides a scale invariant metric. That is, 1 2 ˜ (11) K g˜ij = 2 K α gij = |K|gij . α Next, we prove that multiplying the equi-affine metric by the Gaussian curvature computed for the equi-affine metric provides an affine invariant metric. Using Brioschi formula [25] we can evaluate the Gaussian curvature directly from the metric and its first and second derivatives. Specifically, given the metric tensor qij , we have K ≡
β−γ , det2 (q)
(12)
where − 12 q11,vv + q12,vu − 21 q22,uu q12,v − 12 q22,u β = det 1 q 2 22,v 1 1 0 q q 11,v 22,u 2 2 q12 , γ = det 21 q11,v q11 1 q q12 q22 2 22,u
1 q 2 11,u
q11 q12
q12,u − 12 q11,v q12 q22 (13)
here qij,u denotes the derivation of qij with respect to u, and in a similar manner qij,uv is the second derivative w.r.t. u and v. Same notations follow for qij,v , qij,vv , and qij,uu . Scaling the surface S by α, the corresponding equi-affine invariant components are r˜ij = r˜ij,m = r˜ij,mn = det(˜ r) =
det(αS1 , αS2 , αSij ) = α3 rij α3 rij,m α3 rij,mn α6 det(r),
(14)
that leads to q˜ij =
r˜ij (det(˜ r))
1 4
=
α3 rij (α6 det(r)) 4
3
1 4
= α 2 qij ,
(15)
Technion - Computer Science Department - Tehnical Report CIS-2012-01 - 2012
3 2 that yields det(˜ q ) = α 2 det(q) = α3 det(q). Denote the Gaussian curvature constructed from the ˜ q . We have that equi-affine invariant metric of the scaled surface S˜ = αS by K β˜ =
γ˜ =
3
α2
3
3 3
α2
β γ
˜q = K
3
3
(β − γ) 3 β˜ − γ˜ = = α− 2 K q . 2 2 2 det (˜ q) (α3 ) det (q) α2
It immediately follows that 3 3 3 ˜ q ˜ q 2 K q˜ij = K α qij = α− 2 K q α 2 qij = |K q | qij
(16)
(17)
which concludes the proof. 2.2. Implementation considerations
Given a triangulated surface we use the Gaussian curvature approximation proposed in [32] while operating on the equi-affine metric tensor. The Gaussian curvature for smooth surfaces can be defined using the Global Gauss-Bonnet Theorem, see [22]. Polthier and Schmies used that connection in order to approximate the Gaussian curvature of triangulated surfaces in [40]. Given a vertex in a triangulated mesh that is shared by p triangles, such that the angle of each triangle at that vertex is given by θi , where i ∈ 1, ...p, the Gaussian curvature K at that vertex can be approximated by ! p X 1 2π − (18) θi , K∼ = 1 Pp A i i=1 3 i=1 where Ai is the area of the i-th triangle, and θi is the corresponding angle. Consider the triangle ABC which is one face of a triangulated surface S, defines by its vertices A, B, and C, as can be seen in Figure 2. Without loss of generality, we could define the surface tangent vectors at vertex A to be Su = B − A and Sv = C − A, where we have chosen the arbitrary local parametrization u, v to align with the corresponding edges AB and AC. Translating from the vertex A to an arbitrary point in the triangle could be measured in terms of du along the Su direction and dv along the Sv direction. We can thereby write the displacement ds by ds2 = |Su du + Sv dv|2 = |Su |2 du2 + 2hSu , Sv idudv + |Sv |2 dv 2 = g11 du2 + 2g12 dudv + g22 dv 2 . (19) We could also express the angle θA as a function of Su and Sv , cos θA =
hSu , Sv i g12 =√ , |Su ||Sv | g11 g22
(20)
and the area of the triangle AABC
q q 1 1 1p 1 2 2 2 2 = |Su × Sv | = |Su | |Sv | − |hSu , Sv i| = g11 g22 − g12 = det(gij ). 2 2 2 2 5
(21)
Technion - Computer Science Department - Tehnical Report CIS-2012-01 - 2012
Figure 2: One face of a triangulated surface S, defines by its vertices A, B, and C. In order to construct the eigenfunctions of the Laplace-Beltrami operator w.r.t. the affine metric using a finite elements method (FEM), the Gaussian curvature needs to be interpolated within each triangle for which we use a linear interpolation. Finally, following [42], we use the finite elements method (FEM) presented in [23] to compute the spectral decomposition of the affine invariant Laplace-Beltrami operator constructed from the metric in Eq. (6). The whole framework provides us with an affine invariant metric, invariant eigenvectors and corresponding invariant eigenvalues.
3. Invariant diffusion geometry Diffusion Geometry, see e.g. [21], deals with geometric analysis of metric spaces where usual distances are replaced by the way heat propagates in a given space. The heat equation ∂ + ∆h f (x, t) = 0, (22) ∂t describes the propagation of heat, where f (x, t) is the heat distribution at a point x in time t, with initial conditions at f (x, 0), and ∆h is the Laplace Beltrami Operator for our metric h. The fundamental solution of (22) is called the heat kernel, and using spectral decomposition it can be represented as X kt (x, x0 ) = e−λi t φi (x)φi (x0 ), (23) i≥0
where φi and λi are, respectively, the eigenfunctions and eigenvalues of the Laplace-Beltrami operator satisfying ∆h φi = λi φi . As the Laplace-Beltrami operator is an intrinsic geometric quantity, it can be expressed in terms of the metric of S. The value of the heat kernel kt (x, x0 ) can be interpreted as the transition probability density of a random walk of length t from the point x to the point x0 . This allows to construct a family of intrinsic metrics known as diffusion metrics, Z 2 2 0 dt (x, x ) = (kt (x, ·) − kt (x0 , ·)) da X = e−2λi t (φi (x) − φi (x0 ))2 , (24) i>0
that define the diffusion distance between two points x and x0 for a given time t. A special attention was given to the diagonal of the kernel kt (x, x), that was proposed as robust local descriptor, referred to as heat kernel signatures (HKS), by Sun et al. [45]. 6
Technion - Computer Science Department - Tehnical Report CIS-2012-01 - 2012
4. Experimental results The first experiment demonstrates resulting eigenfunctions of the Laplace Beltrami operator using different metrics and different deformations. In Figure 3 we present the 9’th eigenfunction textured mapped on the surface using an Euclidean metric, scale invariant metric, equi-affine metric and the proposed affine invariant metric. In each row a different deformation of the surface is presented. On the second row we applied local scaling, on the third row volume preserving stretching (equi-affine), and on the last row affine transformation (including local scale). At the top are plotted the accumulated histogram values of the eigenfunction. Blue color is used for the original shape, red for the locally scaled shape, green for the equi-affine transformed one, and black for the affine transformation. In the second experiment, Figure 4, we evaluate the Heat Kernel Signatures of the surface subject to local scaling, equi-affine and affine transformations, using the Euclidean and the affine metrics. In addition, we depict the accumulated values of the histograms, color coded as before. In the third experiment, Figure 5 shows diffusion distances measured from the nose of a cat after anisotropic scaling and stretching as well as an almost isomeric transformation. Next, we compute Voronoi diagrams for ten points selected by the farthest point sampling strategy, as seen in Figures 6 and 1. Distances are measured with the global scale invariant commute time distances [41], and diffusion distances receptively, using a Euclidean metric and the proposed affine version. Again, the affine metric outperforms the Euclidean one. In the next example we used the affine metric for finding the correspondence between two shapes. We used the GMDS framework [7] with diffusion distances with the same initialization for both experiments. Figure 7 displays the Voronoi cells of matching surface segments. Finally, we evaluated the proposed metric on the SHREC 2010 dataset [6] using the shapeGoogle framework [37], after adding four new deformations; equi-affine, isometry and equi-affine, affine and isometry and affine. Table 1 shows that the new affine metric discriminative power is as good as the Euclidean one, performs well on scaling as the scale invariant metric, and similar to the the equi-affine one for volume preserving affine transformations. More over, the new metric is the only one capable to handle full affine deformations added to SHREC dataset. Note that shapes which were considered locally scaled in that database were in fact treated with an offset operation (morphological erosion) rather than scaling. This explains some of the degradation in performances in the local scaled examples. Performances were evaluated using precision/recall characteristic. Precision P(r) is defined as the percentage of relevant P shapes in the first r top-ranked retrieved shapes. Mean average precision (mAP), defined as mAP = r P(r) · rel(r), where rel(r) is the relevance of a given rank, was used as a single measure of performance. Intuitively, mAP is interpreted as the area below the precision-recall curve. Ideal retrieval performance (mAP=100%) is achieved when all queries return relevant first matches. Performance results were broken down according to transformation class and strength.
5. Conclusions We introduced a new metric that gracefully handles the affine group of transformations. Its differential structure allows us to cope with local and global non-uniform stretching and scaling of the surfaces. We demonstrated that the proposed geometry and resulting computational methods could be useful for shape analysis like the SIFT and ASIFT operations were useful for image analysis.
7
Technion - Computer Science Department - Tehnical Report CIS-2012-01 - 2012
Figure 3: The 9’th LBO eigenfunction textured mapped on the surface using four different metrics, from left to right: Euclidean, scale-invariant, equi-affine and affine. Deformations from top to bottom: None, local scale, equi-affine and affine. At the top the accumulated (histogram) values of the eigenfunction are displayed. The blue graph is used for the original shape, red for a locally scaled one, green for equi-affine and black for an affine transformation.
References [1] Y. Aflalo, D. Raviv, and R. Kimmel. Scale invariant geometry for non-rigid shapes. Technical Report CIS2011-02, Technion, Israel Institute of Technology, 2011. 2, 3 [2] L. Alvarez, F. Guichard, P.-L. Lions, and J.-M. Morel. Axioms and fundamental equations of image processing. Archive for Rational Mechanics and Analysis, 123(3):199–257, 1993. 2
8
Technion - Computer Science Department - Tehnical Report CIS-2012-01 - 2012
Figure 4: Affined heat kernel signatures for the regular metric (left), and the invariant version (right). The blue circles represent the signatures for three points on the original surface, while the red plus signs are computed from the deformed version. Using a log-log axes we plot the scaled-HKS as a function of t.
Figure 5: Diffusion distances - Euclidean (left) and affine (right) - measured from the nose of the cat after non-uniform scaling and stretching. The accumulated (histogram) distances (at the bottom) are color coded as in Figure 3. [3] M. Andrade and T. Lewiner. Affine-invariant curvature estimators for implicit surfaces. Computer Aided Geometric Design, 29(2):162–173, 2012. 2
9
Technion - Computer Science Department - Tehnical Report CIS-2012-01 - 2012
Figure 6: Voronoi diagrams using ten points selected by the farthest point sampling strategy. The commute time distances were evaluated using the Euclidean metric (top) and the proposed affine one (bottom). We textured mapped the result to the original mesh for comparison.
Figure 7: Correspondence search between two shapes using the GMDS [7] framework with diffusion distances. The affine (right) metric clearly outperforms the Euclidean one (left) in the presence of non-uniform stretching. Corollated surfaces segments have the same color. [4] P. B´erard, G. Besson, and S. Gallot. Embedding riemannian manifolds by their heat kernel. Geometric and Functional Analysis, 4(4):373–398, 1994. 2 [5] W. Blaschke. Vorlesungen uber Differentialgeometrie und geometrische Grundlagen von Einsteins Relativitatstheorie, volume 2. Springer, 1923. 3 [6] A. M. Bronstein, M. M. Bronstein, U. Castellani, B. Falcidieno, A. Fusiello, A. Godil, L. J. Guibas, I. Kokkinos, Z. Lian, M. Ovsjanikov, G. Patan´e, M. Spagnuolo, and R. Toldo. SHREC 2010: robust large-scale shape retrieval benchmark. In Proc. 3DOR, 2010. 7 [7] A. M. Bronstein, M. M. Bronstein, and R. Kimmel. Efficient computation of isometry-invariant distances between surfaces. SIAM J. Scientific Computing, 28(5):1812–1836, 2006. 2, 7, 10 [8] A. M. Bronstein, M. M. Bronstein, R. Kimmel, M. Mahmoudi, and G. Sapiro. A Gromov-Hausdorff framework with diffusion geometry for topologically-robust non-rigid shape matching. International Journal of Computer Vision (IJCV), 89(2-3):266–286, 2010. 2 [9] M. M. Bronstein and I. Kokkinos. Scale-invariant heat kernel signatures for non-rigid shape recognition. In Proc. Computer Vision and Pattern Recognition (CVPR), 2010. 2
10
Technion - Computer Science Department - Tehnical Report CIS-2012-01 - 2012
Euclidean invariant metric Transformation Isometry Micro holes Noise Shot noise Scale Local scale Equi-Affine Iso+Equi-Afiine Affine Isometry+Affine
1 100.00 100.00 100.00 100.00 28.87 100.00 100.00 100.00 88.46 88.46
≤2 100.00 100.00 100.00 100.00 56.74 100.00 98.08 97.12 77.79 77.82
Strength ≤3 100.00 100.00 100.00 98.72 61.42 98.72 92.52 92.95 70.81 73.76
≤4 100.00 100.00 98.72 97.60 53.75 91.79 87.72 85.32 65.21 68.35
≤5 100.00 100.00 98.97 97.05 47.77 87.40 83.05 79.99 61.21 64.60
Equi-affine invariant metric Transformation Isometry Micro holes Noise Shot noise Scale Local scale Equi-Affine Iso+Equi-Affine Affine Isometry+Affine
1 100.00 100.00 100.00 100.00 29.82 100.00 100.00 100.00 75.90 75.46
≤2 100.00 100.00 100.00 100.00 50.81 100.00 100.00 100.00 60.35 59.59
Strength ≤3 100.00 100.00 91.88 100.00 48.34 95.51 100.00 100.00 52.54 52.08
≤4 100.00 100.00 83.72 97.12 43.16 87.81 100.00 100.00 47.82 47.39
≤5 100.00 100.00 75.62 92.31 39.89 81.08 100.00 100.00 43.86 44.34
Scale invariant metric 1 100.00 100.00 100.00 100.00 100.00 100.00 81.03 81.03 100.00 100.00
≤2 100.00 92.12 95.00 100.00 100.00 97.44 72.35 74.13 90.87 93.75
Strength ≤3 100.00 84.66 92.42 97.01 100.00 94.87 65.82 63.93 87.36 89.64
≤4 100.00 78.90 87.49 94.94 100.00 90.21 61.55 61.17 84.64 84.33
≤5 100.00 74.96 81.62 92.21 100.00 82.98 59.86 62.30 81.99 83.11
Affine invariant metric 1 100.00 100.00 100.00 100.00 94.87 100.00 100.00 100.00 96.15 96.15
Strength ≤2 ≤3 100.00 98.72 100.00 98.08 95.51 94.02 100.00 100.00 97.44 98.29 100.00 98.29 100.00 96.79 98.08 93.80 95.51 95.73 98.08 97.44
≤4 99.04 97.02 94.55 99.04 95.03 96.47 96.63 94.39 95.83 98.08
≤5 99.23 95.53 94.87 97.69 92.40 92.07 96.08 94.36 96.67 98.46
Table 1: Performance of different metrics with shapeGoogle framework(mAP in %). [10] A. Brook, A. Bruckstein, and R. Kimmel. On similarity-invariant fairness measures. In Scale-Space, LNCS 3459, pages 456–467, Hofgeismar, Germany, 7-9 April 2005. Springer-Verlag. 2 [11] A. Bruckstein and D. Shaked. Skew symmetry detection via invariant signatures. Pattern Recognition, 31(2):181–192, 1998. 1 [12] A. M. Bruckstein, R. J. Holt, A. N. Netravali, and T. J. Richardson. Invariant signatures for planar shape recognition under partial occlusion. Computer Vision, Graphics, and Image Processing: Image Understanding, 58:49–65, 1993. 1 [13] A. M. Bruckstein, N. Katzir, M. Lindenbaum, and M. Porat. Similarity-invariant signatures for partially occluded planar shapes. International Journal of Computer Vision, 7(3):271–285, 1992. 1 [14] A. M. Bruckstein and A. N. Netravali. On differential invariants of planar curves and recognizing partially occluded planar shapes. Annals of Mathematics and Artificial Intelligence (AMAI), 13(3-4):227–250, 1995. 1
11
Technion - Computer Science Department - Tehnical Report CIS-2012-01 - 2012
[15] A. M. Bruckstein, E. Rivlin, and I. Weiss. Scale-space local invariants. Image and Vision Computing, 15(5):335–344, 1997. 2 [16] S. Buchin. Affine differential geometry. Beijing, China: Science Press, 1983. 3 [17] E. Calabi, P. Olver, C. Shakiban, A. Tannenbaum, and S. Haker. Differential and numerically invariant signature curves applied to object recognition. International Journal of Computer Vision, 26:107–135, 1998. 1 [18] S. Carlsson, R. Mohr, T. Moons, L. Morin, C. A. Rothwell, M. Van Diest, L. Van Gool, F. Veillon, and A. Zisserman. Semi-local projective invariants for the recognition of smooth plane curves. International Journal of Computer Vision, 19(3):211–236, 1996. 2 [19] F. Chazal, D. Cohen-Steiner, L. Guibas, F. M´emoli, and S. Oudot. Gromov-Hausdorff stable signatures for shapes using persistence. Computer Graphics Forum, 28(5):1393–1403, July 2009. 2 [20] T. Cohignac, C. Lopez, and J. Morel. Integral and local affine invariant parameter and application to shape recognition. In Proceedings of the 12th IAPR International Conference on Pattern Recognition (ICPR), volume 1, pages 164–168, October 1994. 1 [21] R. R. Coifman, S. Lafon, A. B. Lee, M. Maggioni, B. Nadler, F. Warner, and S. W. Zucker. Geometric diffusions as a tool for harmonic analysis and structure definition of data: Diffusion maps. PNAS, 102(21):7426– 7431, 2005. 2, 6 [22] M. P. do Carmo. Differential geometry of curves and surfaces. Prentice-Hall, 1976. 5 [23] G. Dziuk. Finite elements for the Beltrami operator on arbitrary surfaces. In S. Hildebrandt and R. Leis, editors, Partial differential equations and calculus of variations, pages 142–155. 1988. 6 [24] A. Elad and R. Kimmel. Bending invariant representations for surfaces. In Proc. Computer Vision and Pattern Recognition (CVPR), pages 168–174, 2001. 2 [25] A. Gray, E. Abbena, and S. Salamon. Modern Differential Geometry of Curves and Surfaces with Mathematica. 3rd edition, 2006. 4 [26] A. Hamza and H. Krim. Geodesic matching of triangulated surfaces. IEEE Transactions on Image Processing, 15(8):2249–2258, 2006. 2 [27] R. Kimmel. Affine differential signatures for gray level images of planar shapes. In Proceedings of the 13th International Conference on Pattern Recognition, pages 45–49, vol. 1, Vienna, Austria, 25-30 August 1996. IEEE. 2 [28] H. Ling and D. Jacobs. Using the inner-distance for classification of articulated shapes. In Proc. Computer Vision and Pattern Recognition (CVPR), volume 2, pages 719–726, San Diego, USA, 20-26 June 2005. 2 [29] Y. Lipman and T. Funkhouser. M¨obius voting for surface correspondence. In Proc. ACM Transactions on Graphics (SIGGRAPH), volume 28, 2009. 2 [30] D. Lowe. Distinctive image features from scale-invariant keypoint. International Journal of Computer Vision (IJCV), 60(2):91–110, 2004. 2 [31] F. M´emoli and G. Sapiro. A theoretical and computational framework for isometry invariant recognition of point cloud data. Foundations of Computational Mathematics, 5(3):313–347, 2005. 2 [32] M. Meyer, M. Desbrun, P. Schroder, and A. H. Barr. Discrete differential-geometry operators for triangulated 2-manifolds. Visualization and Mathematics III, pages 35–57, 2003. 5 [33] T. Moons, E. Pauwels, L. Van Gool, and A. Oosterlinck. Foundations of semi-differential invariants. International Journal of Computer Vision (IJCV), 14(1):25–48, 1995. 2 [34] J. M. Morel and G.Yu. ASIFT: A new framework for fully affine invariant image comparison. Journal on Imaging Sciences (SIAM), 2:438–469, 2009. 2
12
Technion - Computer Science Department - Tehnical Report CIS-2012-01 - 2012
[35] P. Olver. A survey of moving frames. In H. Li, P. Olver, and G. Sommer, editors, Computer Algebra and Geometric Algebra with Applications, LNCS 3519, pages 105–138, New York, 2005. Springer-Verlag. 2 [36] P. J. Olver. Joint invariant signatures. Foundations of Computational Mathematics, 1:3–67, 1999. 2 [37] M. Ovsjanikov, A. M. Bronstein, M. M. Bronstein, and L. J. Guibas. Shape Google: a computer vision approach to invariant shape retrieval. In Proc. Non-Rigid Shape Analysis and Deformable Image Alignment (NORDIA), 2009. 2, 7 [38] M. Ovsjanikov, Q. M´erigot, F. M´emoli, and L. J. Guibas. One point isometric matching with the heat kernel. Proc. Symposium on Geometry Processing (SGP), 29(5):1555–1564, Jul 2010. 2 [39] E. Pauwels, T. Moons, L. Van Gool, P. Kempenaers, and A. Oosterlinck. Recognition of planar shapes under affine distortion. International Journal of Computer Vision (IJCV), 14(1):49–65, 1995. 2 [40] K. Polthier and M. Schmies. Straightest geodesics on polyhedral surfaces. Mathematical Visualization, 1998. 5 [41] H. Qiu and E. Hancock. Clustering and embedding using commute times. IEEE Transactions on Pattern Analysis and Machine Intelligence, 29(11):1873–1890, November 2007. 7 [42] D. Raviv, A. M. Bronstein, M. M. Bronstein, R. Kimmel, and N. Sochen. Affine-invariant diffusion geometry of deformable 3D shapes. In Proc. Computer Vision and Pattern Recognition (CVPR), 2011. 2, 3, 6 [43] R. Rustamov. Laplace-Beltrami eigenfunctions for deformation invariant shape representation. In Proc. Symposium on Geometry Processing (SGP), pages 225–233, 2007. 2 [44] G. Sapiro. Affine Invariant Shape Evolutions. PhD thesis, Technion - IIT, 1993. 2 [45] J. Sun, M. Ovsjanikov, and L. J. Guibas. A concise and provably informative multi-scale signature based on heat diffusion. In Proc. Symposium on Geometry Processing (SGP), 2009. 2, 6 [46] L. Van Gool, M. Brill, E. Barrett, T. Moons, and E. Pauwels. Semi-differential invariants for nonplanar curves. In J. Mundy and A. Zisserman, editors, Geometric Invariance in Computer Vision, pages Chapter 11: 293–309, Massachusetts / London, Englad, 1992. MIT Press: Cambridge. 2 [47] L. Van Gool, T. Moons, E. Pauwels, and A. Oosterlinck. Semi-differential invariants. In J. Mundy and A. Zisserman, editors, Geometric Invariance in Computer Vision, pages Chapter 8: 157–192, Massachusetts / London, England, 1992. MIT Press: Cambridge. 2 [48] Y. Wang, M. Gupta, S. Zhang, S. Wang, X. Gu, D. Samaras, and P. Huang. High resolution tracking of nonrigid motion of densely sampled 3D data using harmonic maps. International Journal of Computer Vision (IJCV), 76(3):283–300, 2008. 2 [49] I. Weiss. Projective invariants of shapes. Technical Report CAR-TR-339, University of Maryland - Center for Automation, January 1988. 1
13
Affine invariant non-rigid shape analysis Dan Raviv Dept. of Computer Science Technion, Israel
Ron Kimmel Dept. of Computer Science Technion, Israel
[email protected]
[email protected]
Figure 1: Voronoi diagrams of ten points selected by farthest point sampling. Diffusion distances were used based on Euclidean metric (top) and an affine invariant one (bottom).
Abstract Shape recognition deals with the study geometric structures. Modern surface processing methods can cope with non-rigidity - by measuring the lack of isometry, deal with similarity - by multiplying the Euclidean arc-length by the Gaussian curvature, and manage equi-affine transformations - by resorting to the special affine arc-length definition in classical affine geometry. Here, we propose a computational framework that is invariant to the affine group of transformations (similarity and equi-affine) and thus, by construction, can handle non-rigid shapes. Technically, we add the similarity invariant property to an equi-affine invariant one. Diffusion geometry encapsulates the resulting measure to robustly provide signatures and computational tools for affine invariant surface matching and comparison.
1. Introduction Differential invariants for planar shape matching and recognition were introduced to computer vision in the 80’s [49] and studied in the early 90’s [13, 12, 20, 14, 17, 11], where global invariants were computed in a local manner to overcome numerical sensitivity of the differential forms. Scale space entered 1
Technion - Computer Science Department - Tehnical Report CIS-2012-01 - 2012
the game as a stabilizing mechanism, for example in [15], where locality was tuned by a scalar indicating how far one should depart from the point of interest. Along a different path, using a point matching oracle to reduce the number of derivatives, semi-differential signatures were proposed in [46, 47, 33, 39, 18]. Non-local signatures, which are more sensitive to occlusions, were shown to perform favorably in holistic paradigms [36, 10, 35, 28]. At another end, image simplification through geometric invariant heat processes were introduced and experimented with during the late 90’s, [44, 2, 27]. In the beginning of this century, scale space theories gave birth to the celebrated scale invariant feature transform (SIFT) [30] and the affine-scale invariant feature transform ASIFT [34], that are used to successfully locate repeatable informative (invariant) features in images. Matching surfaces while accounting for deformations was performed with conformal mappings [29], embedding to finite dimensional Euclidean spaces [24] and infinite ones [4, 43], topological graphs [26, 48], and exploiting the Gromov-Hausdorff distance [31, 7, 19]. Which are just a subset of the numerous methods used in this exploding field. Another example, relevant to this paper is diffusion geometry, introduced in [21] for manifold learning and first applied for shape analysis in [8]. This geometry can be constructed by the eigen-strucutre of the Laplace-Beltrami operator. The same decomposition was recently used in [45, 37, 38, 8] to construct surface descriptors for shape retrieval and matching. In this paper, following the adoption of metric/differential geometry tools to image-analysis, we introduce a new geometry for affine invariant surface analysis. In [42, 3], an equi-affine invariant metric for surfaces with effective Gaussian curvature were presented, while a scale invariant metric was the main theme of [1, 9]. Here, we introduce a framework that handles both, that is, affine transformations in its most general form including similarity, that is scaling and isometry. The paper is organized as follows. In Section 2 we introduce the affine invariant metric. Section 3 briefly reviews the main concepts of diffusion geometry. Section 4 is dedicated to experimental results, and Section 5 concludes the paper.
2. Affine metric construction We model a surface (S, g) as a compact two dimensional Riemannian manifold S with a metric tensor g. We further assume that S is embedded in R3 by a regular map S : U ⊂ R2 → R3 . The construction of a Euclidean metric tensor can be obtained from the re-parameterization invariant arclength ds of a parametrized curve C(s) on S, as the simplest Euclidean invariant is length, we search for the parameterization s that would satisfy |Cs | = 1, or explicitly, 1 = hCs , Cs i = hSs , Ss i ∂S du ∂S dv ∂S du ∂S dv + , + = ∂u ds ∂v ds ∂u ds ∂v ds −2 2 = ds g11 du + 2g12 dudv + g22 dv 2 ,
(1)
where gij = hSi , Sj i,
(2)
using the short hand notation S1 = ∂S/∂u, S2 = ∂S/∂v, and u and v are the coordinates of U . An infinitesimal displacement ds on the surface is thereby given by ds2 = g11 du2 + 2g12 dudv + g22 dv 2 . 2
(3)
Technion - Computer Science Department - Tehnical Report CIS-2012-01 - 2012
The metric coefficients translate the surface parametrization into a Euclidean invariant distance measure on the surface. The equi-affine transformation, defined by the linear operator AS + b, where det(A) = 1, is a bit more tricky to deal with, see [5, 16]. Consider the curve C ∈ S parametrized by w. The equi-affine transformation is volume preserving, and thus, its invariant metric could be constructed by restricting the volume defined by Su , Sv , and Cww to one. That is, 1 = det(Su , Sv , Cww ) = det (S u , Sv , Sww ) 2 dv 2 du dv d2 u d2 v du + Svv 2 + Su 2 + Sv 2 = det Su , Sv , Suu 2 + 2Suv dw dw dw dw dw dw 2 = dw−2 det Su , Sv , Suu du2 + 2Suv dudv + S dv vv (4) = dw−2 r11 du2 + 2r12 dudv + r22 dv 2 , where now the metric elements are given by rij = det (S1 , S2 , Sij ), and we extended the short hand 2 2 ∂2S notation to second order derivatives by which S11 = ∂∂uS2 , S22 = ∂∂vS2 , and S12 = ∂u∂v . Note that √ the second fundamental form in the Euclidean case is given by bij = grij where g = det(gij ) = 2 . The equi-affine re-parametrization invariant metric [16, 5] reads g11 g22 − g12 1
qij = |r|− 4 rij ,
(5)
2 . where r = det(rij ) = r11 r22 − r12 In [42] others and us modified the equi-affine metric to accommodate for surfaces with effective Gaussian curvature. There, the idea was to project the metric tensor matrix qij onto the class of positive definite subspaces. Practically, we decompose the tensor into its eigen-structure, take the absolute value of its eigenvalues, and compose the tensor back into a valid invariant metric. Next, we resort to the similarity (scale and isometry) invariant metric proposed in [1], according to which, scale invariance is obtained by multiplying the metric by the Gaussian curvature. The Gaussian curvature is defined by the ratio between the determinants of the second and the first fundamental forms and is denoted by K. We propose to compute the Gaussian curvature of the equi-affine invariant metric, and construct a new metric by multiplying the metric elements by |K|. Specifically, consider the surface (S, q), where qij is the equi-affine invariant metric, and compute the Gaussian curvature K q (S, q) at each point. The affine invariant metric is defined by
hij = |K q | qij .
(6)
Let us justify the above construction of an affine invariant metric for surfaces. 2.1. Proof of invariance
In order to prove the affine invariance of our metric construction let us first justify the scale invariant metric constructed by multiplication of a given Euclidean metric by the Gaussian curvature. Assume that the surface S is scaled by a scalar α > 0, such that ˜ v) = αS(u, v). S(u,
(7)
In what follows, we omit the parameters u,v for brevity, and denote the quantities q computed for the scaled surface by q˜. The first and second fundamental forms are scaled by α2 and α respectfully, g˜ij = hαSi , αSj i = α2 hSi , Sj i = α2 gij , 3
Technion - Computer Science Department - Tehnical Report CIS-2012-01 - 2012
˜bij = hαSij , N i = α hSij , N i = αbij ,
(8)
which yields det(˜ g ) = α4 det(g) det(˜b) = α2 det(b).
(9)
Since the Gaussian curvature is the ratio between the determinants of the second and first fundamental forms, we readily have that α2 det(b) 1 det(˜b) ˜ = 4 = 2 K, K≡ det(˜ g) α det(g) α
(10)
from which we conclude that multiplying the Euclidean metric by the magnitude of its Gaussian curvature indeed provides a scale invariant metric. That is, 1 2 ˜ (11) K g˜ij = 2 K α gij = |K|gij . α Next, we prove that multiplying the equi-affine metric by the Gaussian curvature computed for the equi-affine metric provides an affine invariant metric. Using Brioschi formula [25] we can evaluate the Gaussian curvature directly from the metric and its first and second derivatives. Specifically, given the metric tensor qij , we have K ≡
β−γ , det2 (q)
(12)
where − 12 q11,vv + q12,vu − 21 q22,uu q12,v − 12 q22,u β = det 1 q 2 22,v 1 1 0 q q 11,v 22,u 2 2 q12 , γ = det 21 q11,v q11 1 q q12 q22 2 22,u
1 q 2 11,u
q11 q12
q12,u − 12 q11,v q12 q22 (13)
here qij,u denotes the derivation of qij with respect to u, and in a similar manner qij,uv is the second derivative w.r.t. u and v. Same notations follow for qij,v , qij,vv , and qij,uu . Scaling the surface S by α, the corresponding equi-affine invariant components are r˜ij = r˜ij,m = r˜ij,mn = det(˜ r) =
det(αS1 , αS2 , αSij ) = α3 rij α3 rij,m α3 rij,mn α6 det(r),
(14)
that leads to q˜ij =
r˜ij (det(˜ r))
1 4
=
α3 rij (α6 det(r)) 4
3
1 4
= α 2 qij ,
(15)
Technion - Computer Science Department - Tehnical Report CIS-2012-01 - 2012
3 2 that yields det(˜ q ) = α 2 det(q) = α3 det(q). Denote the Gaussian curvature constructed from the ˜ q . We have that equi-affine invariant metric of the scaled surface S˜ = αS by K β˜ =
γ˜ =
3
α2
3
3 3
α2
β γ
˜q = K
3
3
(β − γ) 3 β˜ − γ˜ = = α− 2 K q . 2 2 2 det (˜ q) (α3 ) det (q) α2
It immediately follows that 3 3 3 ˜ q ˜ q 2 K q˜ij = K α qij = α− 2 K q α 2 qij = |K q | qij
(16)
(17)
which concludes the proof. 2.2. Implementation considerations
Given a triangulated surface we use the Gaussian curvature approximation proposed in [32] while operating on the equi-affine metric tensor. The Gaussian curvature for smooth surfaces can be defined using the Global Gauss-Bonnet Theorem, see [22]. Polthier and Schmies used that connection in order to approximate the Gaussian curvature of triangulated surfaces in [40]. Given a vertex in a triangulated mesh that is shared by p triangles, such that the angle of each triangle at that vertex is given by θi , where i ∈ 1, ...p, the Gaussian curvature K at that vertex can be approximated by ! p X 1 2π − (18) θi , K∼ = 1 Pp A i i=1 3 i=1 where Ai is the area of the i-th triangle, and θi is the corresponding angle. Consider the triangle ABC which is one face of a triangulated surface S, defines by its vertices A, B, and C, as can be seen in Figure 2. Without loss of generality, we could define the surface tangent vectors at vertex A to be Su = B − A and Sv = C − A, where we have chosen the arbitrary local parametrization u, v to align with the corresponding edges AB and AC. Translating from the vertex A to an arbitrary point in the triangle could be measured in terms of du along the Su direction and dv along the Sv direction. We can thereby write the displacement ds by ds2 = |Su du + Sv dv|2 = |Su |2 du2 + 2hSu , Sv idudv + |Sv |2 dv 2 = g11 du2 + 2g12 dudv + g22 dv 2 . (19) We could also express the angle θA as a function of Su and Sv , cos θA =
hSu , Sv i g12 =√ , |Su ||Sv | g11 g22
(20)
and the area of the triangle AABC
q q 1 1 1p 1 2 2 2 2 = |Su × Sv | = |Su | |Sv | − |hSu , Sv i| = g11 g22 − g12 = det(gij ). 2 2 2 2 5
(21)
Technion - Computer Science Department - Tehnical Report CIS-2012-01 - 2012
Figure 2: One face of a triangulated surface S, defines by its vertices A, B, and C. In order to construct the eigenfunctions of the Laplace-Beltrami operator w.r.t. the affine metric using a finite elements method (FEM), the Gaussian curvature needs to be interpolated within each triangle for which we use a linear interpolation. Finally, following [42], we use the finite elements method (FEM) presented in [23] to compute the spectral decomposition of the affine invariant Laplace-Beltrami operator constructed from the metric in Eq. (6). The whole framework provides us with an affine invariant metric, invariant eigenvectors and corresponding invariant eigenvalues.
3. Invariant diffusion geometry Diffusion Geometry, see e.g. [21], deals with geometric analysis of metric spaces where usual distances are replaced by the way heat propagates in a given space. The heat equation ∂ + ∆h f (x, t) = 0, (22) ∂t describes the propagation of heat, where f (x, t) is the heat distribution at a point x in time t, with initial conditions at f (x, 0), and ∆h is the Laplace Beltrami Operator for our metric h. The fundamental solution of (22) is called the heat kernel, and using spectral decomposition it can be represented as X kt (x, x0 ) = e−λi t φi (x)φi (x0 ), (23) i≥0
where φi and λi are, respectively, the eigenfunctions and eigenvalues of the Laplace-Beltrami operator satisfying ∆h φi = λi φi . As the Laplace-Beltrami operator is an intrinsic geometric quantity, it can be expressed in terms of the metric of S. The value of the heat kernel kt (x, x0 ) can be interpreted as the transition probability density of a random walk of length t from the point x to the point x0 . This allows to construct a family of intrinsic metrics known as diffusion metrics, Z 2 2 0 dt (x, x ) = (kt (x, ·) − kt (x0 , ·)) da X = e−2λi t (φi (x) − φi (x0 ))2 , (24) i>0
that define the diffusion distance between two points x and x0 for a given time t. A special attention was given to the diagonal of the kernel kt (x, x), that was proposed as robust local descriptor, referred to as heat kernel signatures (HKS), by Sun et al. [45]. 6
Technion - Computer Science Department - Tehnical Report CIS-2012-01 - 2012
4. Experimental results The first experiment demonstrates resulting eigenfunctions of the Laplace Beltrami operator using different metrics and different deformations. In Figure 3 we present the 9’th eigenfunction textured mapped on the surface using an Euclidean metric, scale invariant metric, equi-affine metric and the proposed affine invariant metric. In each row a different deformation of the surface is presented. On the second row we applied local scaling, on the third row volume preserving stretching (equi-affine), and on the last row affine transformation (including local scale). At the top are plotted the accumulated histogram values of the eigenfunction. Blue color is used for the original shape, red for the locally scaled shape, green for the equi-affine transformed one, and black for the affine transformation. In the second experiment, Figure 4, we evaluate the Heat Kernel Signatures of the surface subject to local scaling, equi-affine and affine transformations, using the Euclidean and the affine metrics. In addition, we depict the accumulated values of the histograms, color coded as before. In the third experiment, Figure 5 shows diffusion distances measured from the nose of a cat after anisotropic scaling and stretching as well as an almost isomeric transformation. Next, we compute Voronoi diagrams for ten points selected by the farthest point sampling strategy, as seen in Figures 6 and 1. Distances are measured with the global scale invariant commute time distances [41], and diffusion distances receptively, using a Euclidean metric and the proposed affine version. Again, the affine metric outperforms the Euclidean one. In the next example we used the affine metric for finding the correspondence between two shapes. We used the GMDS framework [7] with diffusion distances with the same initialization for both experiments. Figure 7 displays the Voronoi cells of matching surface segments. Finally, we evaluated the proposed metric on the SHREC 2010 dataset [6] using the shapeGoogle framework [37], after adding four new deformations; equi-affine, isometry and equi-affine, affine and isometry and affine. Table 1 shows that the new affine metric discriminative power is as good as the Euclidean one, performs well on scaling as the scale invariant metric, and similar to the the equi-affine one for volume preserving affine transformations. More over, the new metric is the only one capable to handle full affine deformations added to SHREC dataset. Note that shapes which were considered locally scaled in that database were in fact treated with an offset operation (morphological erosion) rather than scaling. This explains some of the degradation in performances in the local scaled examples. Performances were evaluated using precision/recall characteristic. Precision P(r) is defined as the percentage of relevant P shapes in the first r top-ranked retrieved shapes. Mean average precision (mAP), defined as mAP = r P(r) · rel(r), where rel(r) is the relevance of a given rank, was used as a single measure of performance. Intuitively, mAP is interpreted as the area below the precision-recall curve. Ideal retrieval performance (mAP=100%) is achieved when all queries return relevant first matches. Performance results were broken down according to transformation class and strength.
5. Conclusions We introduced a new metric that gracefully handles the affine group of transformations. Its differential structure allows us to cope with local and global non-uniform stretching and scaling of the surfaces. We demonstrated that the proposed geometry and resulting computational methods could be useful for shape analysis like the SIFT and ASIFT operations were useful for image analysis.
7
Technion - Computer Science Department - Tehnical Report CIS-2012-01 - 2012
Figure 3: The 9’th LBO eigenfunction textured mapped on the surface using four different metrics, from left to right: Euclidean, scale-invariant, equi-affine and affine. Deformations from top to bottom: None, local scale, equi-affine and affine. At the top the accumulated (histogram) values of the eigenfunction are displayed. The blue graph is used for the original shape, red for a locally scaled one, green for equi-affine and black for an affine transformation.
References [1] Y. Aflalo, D. Raviv, and R. Kimmel. Scale invariant geometry for non-rigid shapes. Technical Report CIS2011-02, Technion, Israel Institute of Technology, 2011. 2, 3 [2] L. Alvarez, F. Guichard, P.-L. Lions, and J.-M. Morel. Axioms and fundamental equations of image processing. Archive for Rational Mechanics and Analysis, 123(3):199–257, 1993. 2
8
Technion - Computer Science Department - Tehnical Report CIS-2012-01 - 2012
Figure 4: Affined heat kernel signatures for the regular metric (left), and the invariant version (right). The blue circles represent the signatures for three points on the original surface, while the red plus signs are computed from the deformed version. Using a log-log axes we plot the scaled-HKS as a function of t.
Figure 5: Diffusion distances - Euclidean (left) and affine (right) - measured from the nose of the cat after non-uniform scaling and stretching. The accumulated (histogram) distances (at the bottom) are color coded as in Figure 3. [3] M. Andrade and T. Lewiner. Affine-invariant curvature estimators for implicit surfaces. Computer Aided Geometric Design, 29(2):162–173, 2012. 2
9
Technion - Computer Science Department - Tehnical Report CIS-2012-01 - 2012
Figure 6: Voronoi diagrams using ten points selected by the farthest point sampling strategy. The commute time distances were evaluated using the Euclidean metric (top) and the proposed affine one (bottom). We textured mapped the result to the original mesh for comparison.
Figure 7: Correspondence search between two shapes using the GMDS [7] framework with diffusion distances. The affine (right) metric clearly outperforms the Euclidean one (left) in the presence of non-uniform stretching. Corollated surfaces segments have the same color. [4] P. B´erard, G. Besson, and S. Gallot. Embedding riemannian manifolds by their heat kernel. Geometric and Functional Analysis, 4(4):373–398, 1994. 2 [5] W. Blaschke. Vorlesungen uber Differentialgeometrie und geometrische Grundlagen von Einsteins Relativitatstheorie, volume 2. Springer, 1923. 3 [6] A. M. Bronstein, M. M. Bronstein, U. Castellani, B. Falcidieno, A. Fusiello, A. Godil, L. J. Guibas, I. Kokkinos, Z. Lian, M. Ovsjanikov, G. Patan´e, M. Spagnuolo, and R. Toldo. SHREC 2010: robust large-scale shape retrieval benchmark. In Proc. 3DOR, 2010. 7 [7] A. M. Bronstein, M. M. Bronstein, and R. Kimmel. Efficient computation of isometry-invariant distances between surfaces. SIAM J. Scientific Computing, 28(5):1812–1836, 2006. 2, 7, 10 [8] A. M. Bronstein, M. M. Bronstein, R. Kimmel, M. Mahmoudi, and G. Sapiro. A Gromov-Hausdorff framework with diffusion geometry for topologically-robust non-rigid shape matching. International Journal of Computer Vision (IJCV), 89(2-3):266–286, 2010. 2 [9] M. M. Bronstein and I. Kokkinos. Scale-invariant heat kernel signatures for non-rigid shape recognition. In Proc. Computer Vision and Pattern Recognition (CVPR), 2010. 2
10
Technion - Computer Science Department - Tehnical Report CIS-2012-01 - 2012
Euclidean invariant metric Transformation Isometry Micro holes Noise Shot noise Scale Local scale Equi-Affine Iso+Equi-Afiine Affine Isometry+Affine
1 100.00 100.00 100.00 100.00 28.87 100.00 100.00 100.00 88.46 88.46
≤2 100.00 100.00 100.00 100.00 56.74 100.00 98.08 97.12 77.79 77.82
Strength ≤3 100.00 100.00 100.00 98.72 61.42 98.72 92.52 92.95 70.81 73.76
≤4 100.00 100.00 98.72 97.60 53.75 91.79 87.72 85.32 65.21 68.35
≤5 100.00 100.00 98.97 97.05 47.77 87.40 83.05 79.99 61.21 64.60
Equi-affine invariant metric Transformation Isometry Micro holes Noise Shot noise Scale Local scale Equi-Affine Iso+Equi-Affine Affine Isometry+Affine
1 100.00 100.00 100.00 100.00 29.82 100.00 100.00 100.00 75.90 75.46
≤2 100.00 100.00 100.00 100.00 50.81 100.00 100.00 100.00 60.35 59.59
Strength ≤3 100.00 100.00 91.88 100.00 48.34 95.51 100.00 100.00 52.54 52.08
≤4 100.00 100.00 83.72 97.12 43.16 87.81 100.00 100.00 47.82 47.39
≤5 100.00 100.00 75.62 92.31 39.89 81.08 100.00 100.00 43.86 44.34
Scale invariant metric 1 100.00 100.00 100.00 100.00 100.00 100.00 81.03 81.03 100.00 100.00
≤2 100.00 92.12 95.00 100.00 100.00 97.44 72.35 74.13 90.87 93.75
Strength ≤3 100.00 84.66 92.42 97.01 100.00 94.87 65.82 63.93 87.36 89.64
≤4 100.00 78.90 87.49 94.94 100.00 90.21 61.55 61.17 84.64 84.33
≤5 100.00 74.96 81.62 92.21 100.00 82.98 59.86 62.30 81.99 83.11
Affine invariant metric 1 100.00 100.00 100.00 100.00 94.87 100.00 100.00 100.00 96.15 96.15
Strength ≤2 ≤3 100.00 98.72 100.00 98.08 95.51 94.02 100.00 100.00 97.44 98.29 100.00 98.29 100.00 96.79 98.08 93.80 95.51 95.73 98.08 97.44
≤4 99.04 97.02 94.55 99.04 95.03 96.47 96.63 94.39 95.83 98.08
≤5 99.23 95.53 94.87 97.69 92.40 92.07 96.08 94.36 96.67 98.46
Table 1: Performance of different metrics with shapeGoogle framework(mAP in %). [10] A. Brook, A. Bruckstein, and R. Kimmel. On similarity-invariant fairness measures. In Scale-Space, LNCS 3459, pages 456–467, Hofgeismar, Germany, 7-9 April 2005. Springer-Verlag. 2 [11] A. Bruckstein and D. Shaked. Skew symmetry detection via invariant signatures. Pattern Recognition, 31(2):181–192, 1998. 1 [12] A. M. Bruckstein, R. J. Holt, A. N. Netravali, and T. J. Richardson. Invariant signatures for planar shape recognition under partial occlusion. Computer Vision, Graphics, and Image Processing: Image Understanding, 58:49–65, 1993. 1 [13] A. M. Bruckstein, N. Katzir, M. Lindenbaum, and M. Porat. Similarity-invariant signatures for partially occluded planar shapes. International Journal of Computer Vision, 7(3):271–285, 1992. 1 [14] A. M. Bruckstein and A. N. Netravali. On differential invariants of planar curves and recognizing partially occluded planar shapes. Annals of Mathematics and Artificial Intelligence (AMAI), 13(3-4):227–250, 1995. 1
11
Technion - Computer Science Department - Tehnical Report CIS-2012-01 - 2012
[15] A. M. Bruckstein, E. Rivlin, and I. Weiss. Scale-space local invariants. Image and Vision Computing, 15(5):335–344, 1997. 2 [16] S. Buchin. Affine differential geometry. Beijing, China: Science Press, 1983. 3 [17] E. Calabi, P. Olver, C. Shakiban, A. Tannenbaum, and S. Haker. Differential and numerically invariant signature curves applied to object recognition. International Journal of Computer Vision, 26:107–135, 1998. 1 [18] S. Carlsson, R. Mohr, T. Moons, L. Morin, C. A. Rothwell, M. Van Diest, L. Van Gool, F. Veillon, and A. Zisserman. Semi-local projective invariants for the recognition of smooth plane curves. International Journal of Computer Vision, 19(3):211–236, 1996. 2 [19] F. Chazal, D. Cohen-Steiner, L. Guibas, F. M´emoli, and S. Oudot. Gromov-Hausdorff stable signatures for shapes using persistence. Computer Graphics Forum, 28(5):1393–1403, July 2009. 2 [20] T. Cohignac, C. Lopez, and J. Morel. Integral and local affine invariant parameter and application to shape recognition. In Proceedings of the 12th IAPR International Conference on Pattern Recognition (ICPR), volume 1, pages 164–168, October 1994. 1 [21] R. R. Coifman, S. Lafon, A. B. Lee, M. Maggioni, B. Nadler, F. Warner, and S. W. Zucker. Geometric diffusions as a tool for harmonic analysis and structure definition of data: Diffusion maps. PNAS, 102(21):7426– 7431, 2005. 2, 6 [22] M. P. do Carmo. Differential geometry of curves and surfaces. Prentice-Hall, 1976. 5 [23] G. Dziuk. Finite elements for the Beltrami operator on arbitrary surfaces. In S. Hildebrandt and R. Leis, editors, Partial differential equations and calculus of variations, pages 142–155. 1988. 6 [24] A. Elad and R. Kimmel. Bending invariant representations for surfaces. In Proc. Computer Vision and Pattern Recognition (CVPR), pages 168–174, 2001. 2 [25] A. Gray, E. Abbena, and S. Salamon. Modern Differential Geometry of Curves and Surfaces with Mathematica. 3rd edition, 2006. 4 [26] A. Hamza and H. Krim. Geodesic matching of triangulated surfaces. IEEE Transactions on Image Processing, 15(8):2249–2258, 2006. 2 [27] R. Kimmel. Affine differential signatures for gray level images of planar shapes. In Proceedings of the 13th International Conference on Pattern Recognition, pages 45–49, vol. 1, Vienna, Austria, 25-30 August 1996. IEEE. 2 [28] H. Ling and D. Jacobs. Using the inner-distance for classification of articulated shapes. In Proc. Computer Vision and Pattern Recognition (CVPR), volume 2, pages 719–726, San Diego, USA, 20-26 June 2005. 2 [29] Y. Lipman and T. Funkhouser. M¨obius voting for surface correspondence. In Proc. ACM Transactions on Graphics (SIGGRAPH), volume 28, 2009. 2 [30] D. Lowe. Distinctive image features from scale-invariant keypoint. International Journal of Computer Vision (IJCV), 60(2):91–110, 2004. 2 [31] F. M´emoli and G. Sapiro. A theoretical and computational framework for isometry invariant recognition of point cloud data. Foundations of Computational Mathematics, 5(3):313–347, 2005. 2 [32] M. Meyer, M. Desbrun, P. Schroder, and A. H. Barr. Discrete differential-geometry operators for triangulated 2-manifolds. Visualization and Mathematics III, pages 35–57, 2003. 5 [33] T. Moons, E. Pauwels, L. Van Gool, and A. Oosterlinck. Foundations of semi-differential invariants. International Journal of Computer Vision (IJCV), 14(1):25–48, 1995. 2 [34] J. M. Morel and G.Yu. ASIFT: A new framework for fully affine invariant image comparison. Journal on Imaging Sciences (SIAM), 2:438–469, 2009. 2
12
Technion - Computer Science Department - Tehnical Report CIS-2012-01 - 2012
[35] P. Olver. A survey of moving frames. In H. Li, P. Olver, and G. Sommer, editors, Computer Algebra and Geometric Algebra with Applications, LNCS 3519, pages 105–138, New York, 2005. Springer-Verlag. 2 [36] P. J. Olver. Joint invariant signatures. Foundations of Computational Mathematics, 1:3–67, 1999. 2 [37] M. Ovsjanikov, A. M. Bronstein, M. M. Bronstein, and L. J. Guibas. Shape Google: a computer vision approach to invariant shape retrieval. In Proc. Non-Rigid Shape Analysis and Deformable Image Alignment (NORDIA), 2009. 2, 7 [38] M. Ovsjanikov, Q. M´erigot, F. M´emoli, and L. J. Guibas. One point isometric matching with the heat kernel. Proc. Symposium on Geometry Processing (SGP), 29(5):1555–1564, Jul 2010. 2 [39] E. Pauwels, T. Moons, L. Van Gool, P. Kempenaers, and A. Oosterlinck. Recognition of planar shapes under affine distortion. International Journal of Computer Vision (IJCV), 14(1):49–65, 1995. 2 [40] K. Polthier and M. Schmies. Straightest geodesics on polyhedral surfaces. Mathematical Visualization, 1998. 5 [41] H. Qiu and E. Hancock. Clustering and embedding using commute times. IEEE Transactions on Pattern Analysis and Machine Intelligence, 29(11):1873–1890, November 2007. 7 [42] D. Raviv, A. M. Bronstein, M. M. Bronstein, R. Kimmel, and N. Sochen. Affine-invariant diffusion geometry of deformable 3D shapes. In Proc. Computer Vision and Pattern Recognition (CVPR), 2011. 2, 3, 6 [43] R. Rustamov. Laplace-Beltrami eigenfunctions for deformation invariant shape representation. In Proc. Symposium on Geometry Processing (SGP), pages 225–233, 2007. 2 [44] G. Sapiro. Affine Invariant Shape Evolutions. PhD thesis, Technion - IIT, 1993. 2 [45] J. Sun, M. Ovsjanikov, and L. J. Guibas. A concise and provably informative multi-scale signature based on heat diffusion. In Proc. Symposium on Geometry Processing (SGP), 2009. 2, 6 [46] L. Van Gool, M. Brill, E. Barrett, T. Moons, and E. Pauwels. Semi-differential invariants for nonplanar curves. In J. Mundy and A. Zisserman, editors, Geometric Invariance in Computer Vision, pages Chapter 11: 293–309, Massachusetts / London, Englad, 1992. MIT Press: Cambridge. 2 [47] L. Van Gool, T. Moons, E. Pauwels, and A. Oosterlinck. Semi-differential invariants. In J. Mundy and A. Zisserman, editors, Geometric Invariance in Computer Vision, pages Chapter 8: 157–192, Massachusetts / London, England, 1992. MIT Press: Cambridge. 2 [48] Y. Wang, M. Gupta, S. Zhang, S. Wang, X. Gu, D. Samaras, and P. Huang. High resolution tracking of nonrigid motion of densely sampled 3D data using harmonic maps. International Journal of Computer Vision (IJCV), 76(3):283–300, 2008. 2 [49] I. Weiss. Projective invariants of shapes. Technical Report CAR-TR-339, University of Maryland - Center for Automation, January 1988. 1
13
