PDF (365K) - Rutgers CS
Blended Deformable Models (appeared in Proceedings CVPR ’94, pp. 566–572)
Douglas DeCarlo and Dimitri Metaxas Department of Computer & Information Science University of Pennsylvania Philadelphia PA 19104-6389 Abstract We develop a new class of parameterized models based on the linear interpolation of two parameterized shapes using a blending function. Using a small number of additional parameters, blending extends the coverage of shape primitives. In particular, it offers the ability to construct shapes whose genus can change. Blended models are incorporated into our physics-based shape estimation framework which uses dynamic deformable models with global and local deformations. We present experiments involving the extraction of complex shapes including an example of dynamic genus change.
1
Introduction
Developing shape models for shape estimation that cover a wide variety of shapes using a small number of intuitive parameters is a difficult and important open problem. Even though there exist models which have flexible and intuitive representations such as those used in solid modeling [4, 8], they were not designed for shape estimation. The systems described in [5, 10] provide surface connectivity information and can model surfaces of arbitrary topology, but do not provide an abstraction of shape. In fact, no existing shape model with a compact representation can represent objects of varying genus in a unified model. Recently, systems which are applicable for both shape reconstruction and shape recognition were presented [6, 7, 12]. In [7] the shape was specified by its deformation modes and extracted by using a closed-form solution of modal analysis. Shape was represented in [12] using a wavelet basis and estimated by embedding it in a probabalistic framework. Both of these methods provide a collection of parameters ordered by level of detail. The models in [6, 11] incorporate global deformations, which represent prominent shape features, and local deformations, which capture surface detail. For these models, correct configurations are extracted using a physics-based framework where forces are exerted by data. The second author was supported by NSF grant IRI-9309917.
We propose an extension to the shape representation of [6, 11] which we call blended deformable models. Given two shapes that can be defined parametrically on a common material coordinate space, blended shapes are constructed by linear interpolation using a blending function that specifies the relative weight of corresponding points on the two shapes. This new class of models produces a compact and intuitive representation of an object. For example, a sphere and a cylinder blended together could produce a bullet shaped object (see Fig. 2). In addition, it is able to represent shapes of genus 0 and 1: a sphere and torus blended together would produce an object in which a hole will appear depending on the blending function. Fig. 3 shows a variety of shapes that we can create using blending. In this paper, we show how blended models can be incorporated into the previously developed physics-based framework in [6, 11]. We then present modifications to this framework that improve the robustness and stability of the estimation process. We demonstrate our technique through a series of experiments involving incomplete range data from various objects.
2
Geometry of deformable models
As in [6, 11], the position of a point on the model at time t and at position in material coordinate space with respect to an inertial frame of reference is
u
c
x(u; t) = c(t) + R(t)p(u; t);
R
(1)
where is the origin of the inertial frame, and is a rotation matrix which specifies the relative orientation of the inertial frame to a fixed reference frame. In this reference frame, the position of model points , is the sum of a reference shape and a local displacement so that ( ; t) = ( ; t) + ( ; t): (2 )
d
p
s
pu
su du These local displacements, d, allow the representation of fine detail, while the reference shape s captures salient shape features. The reference shape of the model, s, is constructed by applying a global deformation T (such as bending and twisting) with parameters b to a shape primitive e as in s(u) = T(e; b0; b1; : : :): (3 ) i
e
y h(x) = f (x) (x)+ g(x)(1 , (x)) f (x)
The material coordinates of the shape primitive are specified over a domain , and are given by = (u; v) . For a 3-D shape, we have ( ) : ! 3 . These primitives have global shape parameters which specify the shape. Including these parameters, we represent the geometric primitive as ( ; a1; a2; : : :) (4 )
eu
eu
which is defined parametrically in shape parameters ai .
2.1
R
u
(x) 1
g(x) x
u over and has global
0
Figure 1: Blending of two functions blending function (x) u
Supertoroid definition
= 2
f (x), g(x)
x using
α(u) 1
The definition for the superquadric toroid primitive (with 7 shape parameters) is given by
0 @
1 A
a1 2 1 a4 +1 (a4 + C2u )Cv a2 1 2 torus (u; v) = a5 +1 (a5 + C2u )Sv 1 a3 S2u u ( =2; =2] C = sgn(cos ) cos v ( ; ] S = sgn(sin ) sin
e
u
(a)
(5)
(c)
π/2
u
(d)
the “top” of s looks like s2 since ( 2 ) = 0, and how the “bottom” of s looks like s1 since (, 2 ) = 1. The global parameters of s will include the global parameters of s1 and s2 , those that specify (see section 3.1), and the global deformation parameters in T. A single T is applied separately to each shape primitive. The resulting deformed shapes are blended together. While it would be possible to have vary with u and v, we restrict so it does not vary with v. This restriction on limits the coverage of representable shapes. To improve the shape coverage, it is possible to perform blending on blended shapes to produce a more flexible and hierarchical shape structure. When blending shapes, not all combinations of primitives will achieve desirable results. For example, if two primitives are incompatible, the blended result may interpenetrate. There must be a correspondence between the two shapes in space for two shapes to be compatible. Section 5 discusses how this correspondence can be improved without altering the shape. By using different blending functions, we can create the shapes shown in Fig. 3. Notice how some of the objects have both “square” and “round” areas, which is not easily representable using tapering or other global parameterized deformations. The linear interpolation of two curves where a homotopy of curves is used to construct an extruded surface was presented in [8]. This construction is quite different from the blending presented here, since only curves were considered, not shapes.
This is a similar definition of a supertoroid given by Barr [1]. The addition of a5 , a second inner radius parameter allows asymmetric holes. The presence of the scaling factors 1=(a4 + 1) and 1=(a5 + 1) separate the effects of the global size parameters (a1 , a2 and a3 ) from the inner radii parameters (a4 and a5 ) to allow hole size changes that do not affect the global torus size. Now that the model geometry has been described, the shape blending technique can be presented.
Blended models
In a method analogous to the linear interpolation of two points, it is possible to blend two functions. Given two functions, f (x) and g(x), we can blend them using a third function, (x) (with range [0; 1]), so that
h(x) = f (x) (x) + g(x)(1 , (x)):
(6) An example of this is shown in Fig. 1. Notice how h(x) = f (x) where (x) = 1, h(x) = g(x) where (x) = 0, and how h(x) is between f (x) and g(x) everywhere.
Using this idea, we can blend parameterized shapes (such as superquadrics) by the following: s(u; v) = s1 (u; v) (u) + s2 (u; v)(1 , (u))
(b)
0
Figure 2: (a) Shape s1 (b) Shape s2 (c) Blended shape s (d) Blending function (u)
, j j , j j where a1 ; a2 ; a3 ; 1 ; 2 > 0 and a4 ; a5 1. a1 , a2 and a3 are size parameters in the x, y and z directions respectively. 1 and 2 are squareness parameters as in a superquadric. a4 and a5 are hole radius parameters in the x and y directions.
3
0 −π/2
= , 2
(7)
where s1 and s2 are two shape primitives parameterized over , as in Fig. 2(a) and (b). Fig. 2(c) shows the result of blending the shapes shown in Fig. 2(a) and (b). The blending function used to blend the shapes is shown in Fig. 2(d). The blending is performed along u, which corresponds to the z -axis in these shapes (from pole to pole). Notice how
3.1
Blending function parameterization
The blending function, , is implemented as a nonuniform quadratic B-spline function [2]. Given different 2
(a)
Figure 3: Examples of blended shapes types of shape primitives, the domain of may vary. For a superquadric, maps [, 2 ; 2 ] to [0; 1]. The B-spline function is specified using L + 1 control values, fci j i 0 : : :Lg and L knots fui j i 1 : : :Lg, with u1 and uL fixed to be the upper and lower bounds of the domain of . The function has the values (u1 ) = c0 and (uL) = cL and has a continuous first derivative except where two knot values are equal.
3.2
(b)
(c)
(d)
(e)
Figure 4: Changing from a sphere (a) to a torus (e) can be added are limited.
Genus changing
It is also possible to blend objects of genus 0 (a sphere) with objects of genus 1 (a torus). A hole will appear in the blended object as changes. There is no smooth transition between these two shapes because they are not homeomorphic. Yet it is possible to have a transition between the two where there is a single discontinuous event – when the object changes genus. This event affects only the topology of the object, not the shape. An example transition is shown in Fig. 4. In Fig. 4, s1 is a torus, and s2 is a sphere. Initially, (u) = 0, and the genus of the shape is 0. We compute the blended result s using (7). Fig. 4(a) shows the initial sphere. After this, the nodes slowly dimple inward (b) until they touch (c). The values of ( 2 ) and (, 2 ) change from 0 to 1 to produce this dimpling. The object is topologically a sphere, but in (c), has the same shape as a torus with a zero size hole (with a4 = a5 = 1). The connectivity of the surface near this point is changed to be that of a torus. A discussion of how the node interconnections change is given in section 3.3. The hole can now be opened by increasing the torus inner radius parameters, shown in (d) and (e) (which are shown from a slightly different viewpoint to make the hole visible). The topology should be changed from sphere to torus when the shape has the configuration shown in (c). This can be detected by examination of the blending function. There are two constraints on the parameters of a torussphere blend that must be enforced to insure the blended shape remains closed. The torus hole must remain closed when the object is genus 0. When the object is genus 1, ( 2 ) and (, 2 ) must have the value 1 so that the blended object is completely toroidal at the location of the hole. The ideas presented here for changing the genus of shapes can be applied to any shape primitives. The actual steps involved will vary for different primitives. In this case, since varies only with u, the types of holes that
3.3
Node interconnections
When altering the genus of an object, the mesh of nodes must be reconnected to conform to the new topology. This is a straightforward but necessary part of the genus conversion process. Fig. 5 shows how is “folded up” to produce a sphere or torus. The arrows in these diagrams indicate two nodes being “merged” together, since the material coordinates of the nodes map to the same model coordinates. For both the sphere and the torus, a tube is made first (the dotted lines). For a sphere (a), the north and south poles are created by closing each end of the tube. For a torus (b), the ends of the tube are connected together. When the genus changes, the node mesh must be unfolded, and re-folded to have the proper configuration.
u v (a)
(b)
Figure 5: Node interconnection differences between a sphere (a) and torus (b)
4
Dynamics and generalized forces
The dynamics framework given in [6] can be used with few alterations. This framework requires the computation of the global shape model Jacobian, J = @ s=@ qs . > > >> q = (q> c ; q ; qs ; qd ) is the vector of the model’s generalized coordinates, where qc = c(t), q is the quaternion used to specify R(t), qd is the vector used to specify lo> > > > (qs1 and cal deformations, and qs = (q> s1 ; qs2 ; qb ; qT ) qs2 are the parameters of each of the blended shapes, qb is described in section 4.1, and qT are the parameters of the parameterized global deformations). 3
These generalized coordinates are integrated into the dynamic framework, which computes an updated set of generalized coordinates that more closely represent the desired shape. This framework combines the applied forces from the data with stiffness forces of the model to enhance model smoothness, and includes damping for stability. An adaptive Euler integration method is used to implement the dynamics.
4.1
(a)
Figure 6: Example of a superquadric (1 unscaled (a) and scaled space (b)
Blending function dynamics
= (c0 ; : : :cL; u2; : : :; uL,1)> :
(8)
To determine how these parameters change due to applied forces, the Jacobian is needed. Given (7) above,
@ s(u; v) , = s1(u; v) , s2 (u; v) @ @ q(u) (9) Jb = @ qb b To compute @ (u)=@ qb , we apply the product rule to
s1
s2
b
with
sphere (cos
1
u; sin u)
(cos
u
Ss (u); sin Ss (u)) 1
x (a) sphere
z
(10)
where Js1 is the Jacobian for the first shape (with respect to qs1 ), and Js2 is the Jacobian for the second shape (with respect to qs2 ). Intuitively, this means the Jacobians for the components of a blended shape have a greater or lesser effect at a particular location depending on the function .
5
1 4)
superquadric (1 = 14 )
z
the de Boor algorithm [2]. To make sure the components of qb have reasonable values, the constraints ci [0; 1] and ui uj for all 1 i j L are enforced. The addition of blending also changes the Jacobian for the global shape. The Jacobian for the blended shape can be computed using (7) as J = ( (u)J> ; (1 , (u))J> ; J> )>
= 2 =
Franklin [3] uses explicit sampling to generate a more uniform sampling of the surface. This method was designed for static superquadrics, and loses its advantages of efficiency for dynamic shapes. Instead of this, we use an approximation by projecting a unit superquadric onto the unit sphere when 1 < 1 or 2 < 1, and place nodes on the superquadric where they project down to nodes on the sphere, as in Fig. 7(a). This approximation yields fast and adequate results.
The parameters used to construct the blending function are the L + 1 control values and the L , 2 movable knots (u1 and uL are fixed), which yields 2L , 1 total parameters to specify . We concatenate all these parameters into the vector qb , so that qb
(b)
x = cos u
torus (a4 = a5 = 1)
x=
u
1+cos St (2u) 2
=
cos2
St (u)
x (b)
Material space scaling
Figure 7: Material space scaling for a superquadric (a) and torus (b) (first quadrant, y = 0)
For a shape e(u) with u , we can scale (or warp) the domain so that we have e(S (u)), where S : ! is invertible. This operation does not change the shape, but instead changes how the material parameters are mapped onto the object (e.g.– for a discretized object, the nodes will change position, but the shape will remain the same). By introducing this material space scaling, we improve the distribution of nodes (such as in a “square” superquadric), and also improve the compatibility of shape parameterizations when blending different shapes. When discretizing a superquadric, the nodes tend to collect near the “squared” edges if either 1 or 2 is near zero. Fig. 6(a) shows an example of this uneven node spacing.
In order to find the scaling function Ss (u), we solve tan u = tan1 Ss(u). When we generalize to all quadrants, and do a similar calculation for v and 2 , we obtain
Ss (u) = Ss (u; v) = (tan,1 Tu ; tan,1 Tv ) (11) where T = sgn(tan )j tan j . This scaling is performed 1 1
1 2
only when 1 < 1 or 2 < 1. Fig. 8 shows the behavior of (11) on u-v space with decreasing values of . The example where = 0:25 shows the space scale used to correct the node distribution for the superquadric in Fig. 6(b). The scaling in (11) can also be applied to a supertoroid to correctly rescale any 4
ε = 1.0
π/2
π/2
ε = 0.5
π/2
u
u
u
−π/2 −π
−π/2 −π
−π/2 −π
v
π
v
π
ε = 0.25
The fitting examples shown in section 8 show the fit after each step in the fitting schedule. The schedules used in the experiments are given in Fig. 11.
7 v
π
Range data is often incomplete, and usually contains information from only one view of an object. The shape of such an object is underdetermined. As a result, it would be possible for forces to deform this object into an excessively large shape, but still have a reasonable fit on the data. An example of this problem is exhibited in Fig. 9(b).
Figure 8: Example of scaled space (superquadric) with varying = 1 = 2 uneven discretization problems caused by torus squareness when 1 < 1 or 2 < 1. The scaling function for a torus (St ) is used to minimize the effect of the difference in parameterizations of the supertoroid and superquadric when blended. Without scaling, this difference would cause the Jacobians of the blending parameters near the torus hole to have unreasonable values. To minimize this difference when a torus and superquadric are blended, we project the unit torus (with a zero size hole) onto the unit sphere, in a direction parallel to the z -axis (along the hole) as shown in Fig. 7(b). This projection is performed assuming the torus has a zero sized hole (a4 = a5 = 1). It is for this case that the Jacobians of the blending parameters near the hole are affected most. Solving for St, and generalizing for all quadrants, we obtain
St (u) = St (u; v) = (sgn u cos,1 pcos u; v)
(a)
(b)
(c)
Figure 9: Shape with range data (a) covering part of the object fitted without (b) and with volume minimizing force (c)
p
Solina [9] added a minimization term a1 a2 a3 to minimize the volume of the solution. This can be accomplished by a more physics-based approach with similar results. By applying forces to the portions of an object where there are no data forces, the volume of an object can be controlled. Given node i with no data forces applied, the force would be 1 d ,x k ci i (13) fi = Nf v kdci , xi k
(12)
When applying more than one space scaling function, those that make parameterizations compatible should be applied first, followed by those that improve tessellations. For the supertoroid case, we would have etorus (Ss (St(u)))).
6
Volume minimization forces
Parameter scheduling
As the number of parameters increases, having all parameters change at the same time can cause the fitted object to become “stuck” in an undesirable solution. To prevent this, a general schedule for fitting the parameters must be developed. When fitting a blended object, the rough shape should be found first, followed by global deformations, followed by blending parameters. After this, the solution will be close enough to the final solution that allowing all parameters to be changed at once will improve the solution. For objects which are torus blends, there is an additional step added after the blending parameter step, and that is a step to attempt to open the hole. Since varies only with u, we must make sure that the object is aligned with the data correctly so that blending can occur. In general, this alignment can be difficult to determine in advance (especially for torus data). If the fit is not satisfactory, or blending does not occur to any reasonable extent, the fit is restarted to try the other two axes.
where dci is the closest data point to node i, xi is the reference frame position of node i, and Nf is the number of nodes without data forces applied. This force amounts to constant magnitude force in the direction of the nearest data point. kv represents the magnitude of the total force used to control the volume. The value of kv should be highest when the fitting starts, and should be reduced as the deviation from the data decreases. The volume minimization forces should have no effect on local deformations. The results of using the force given in (13) are shown in Fig. 9(c).
8
Experiments
In the following fitting experiments, we show the results of using blended shapes in our system. Fig. 10 shows information on each of the experiments including the number of data points, the resulting mean squared error (MSE), the size of the parameter set, the dimensions of the node mesh, and the number iterations taken for the fit. 5
of nodes, d is the number of data points) due to initial nearest node computations (for force assignment). Once the shape acquires its rough general shape, the complexity approaches O(n + d) since nearest node information can often be carried across iterations. Since less range data can be used in the beginning, this offers an additional constant factor speed increase.
In each of the examples, the initial model configuration is shown. Initially, the model has all global shape parameters equal to 1, and is centered at the center of mass of the data. The blending function is initialized to (u) = 0. At the start of the fit, only 1=10 of the data is used (it is selected randomly). Data sphere/cyl chess rook torus
Source Pts MSE L # Parm MSU 2015 3.64% 9 27 NRCC 1325 1.26% 15 39 CAD 1503 0.62% 2 15
MSU data from Michigan State University PRIP database NRCC data from National Research Council Canada
Mesh 17 17 17 17 16 12
# Iter 223 119 147
9
We have developed and presented a new approach to shape modeling based on shape blending. These models we created can compactly and intuitively represent a large class of shapes in a single model, including shapes of varying genus. This extension is likely to be useful for recognition because blended shapes can be parameterized using a small number of intuitive global parameters. We demonstrated the close to real-time performance of our technique in a variety of shape estimation experiments involving the extraction of complex shapes with incomplete range data.
Figure 10: Experiment data and statistics The schedules (see section 6) used for these fitting examples are shown in Fig. 11. Each stage in these schedules are completed once the average change of parameters per iteration drops below a specified value. Fig. 12 through 14 show the fitting results obtained for the three experiments. 1
2
3 4
Superquadric-Superquadric s2 : a1 , a2 , a3 global deformations s1 : a1 , a2 , a3 ; s2 : a1 , a2 , a3 blending parameters – all parameters
Conclusion
Superquadric-Torus a1 , a2 , a3 global deformations blending parameters
s2 :
References [1] A. Barr. “Superquadrics and Angle-Preserving Transformations”, IEEE Computer Graphics and Applications, 1(1), pp. 11–23, 1981.
s1 : a4 , a5 (hole) all parameters
[2] G. Farin. “Curves and Surfaces for Computer Aided Geometric Design”, Academic Press, Third edition, 1993. [3] W.R. Franklin and A. Barr. “Faster Calculation of Superquadric Shapes”, IEEE Computer Graphics and Applications, 1(3), pp. 41– 47, 1981.
Figure 11: Parameter schedules for fitting experiments Fig. 12 shows the fitting of a sphere/cylinder object. A blend of two superquadrics is used as the model. The initial model and range data is shown in (a), and the rough fit after step 1 is shown in (b). The fitting after step 2 is shown in (c), and (d) shows the fit after step 4 using all the range data. Since this object has a corner where the sphere and cylinder meet, the blending function (e) has developed a C 0 point. Fig. 13 shows the fitting of a rook. This example is fit in a similar way as the previous example, except in (d), the number of nodes was tripled to allow a greater accuracy of the fit. In (e), local deformations were added to allow fitting of some of the detail. Using local deformations, the MSE dropped to 0.59%, and 30 more iterations were performed. Fig. 14 shows the fitting of torus data. The initial range data is shown in (a), and the initialization is shown in (b). The rough fit after step 1 is shown in (c). The hole is “pinched” closed in (d) after step 2, and the genus changes to 1. The hole is opened (step 3) in (e) and (f) (which are the same object from different viewpoints). A final fit using all data (step 4) is shown in (g). Each iteration takes approximately 1 second on a 33 MHz SGI R3000 using data sets of this size. An adaptive Euler method is used to update the object state. Initially, iterations have O(nd) behavior (where n is the number
[4] C.M. Hoffmann and J. Hopcroft. “The Geometry of Projective Blending Surfaces”, Artificial Intelligence, 37, pp. 357–376, 1988. [5] R. Malladi, J.A. Sethian, and B.C. Vemuri, “A Fast Level Set based Algorithm for Topology-Independent Shape Modeling,” to appear in Journal of Mathematical Imaging and Vision, special issue on Topology and Geometry in Computer Vision. [6] D. Metaxas and D. Terzopoulos. “Shape and Nonrigid Motion Estimation Through Physics-Based Synthesis”, IEEE Trans. Pattern Analysis and Machine Intelligence,15(6), pp. 569–579,June, 1993. [7] A. Pentland and S. Sclaroff. “Closed-Form Solutions for Physically Based Shape Modeling and Recognition”, IEEE Trans. Pattern Analysis and Machine Intelligence, 13(7), pp. 715–729, 1991. [8] J. Snyder “Generative Modeling for Computer Graphics and CAD”, Academic Press, 1992. [9] F. Solina and R. Bajcsy. “Recovery of Parametric Models from Range Images: The Case for Superquadrics with Global Deformations”, IEEE Trans. Pattern Analysis and Machine Intelligence, 12(2), pp 131–146, 1990. [10] R. Szeliski, D. Tonnesen and D. Terzopoulos. “Modeling Surfaces of Arbitrary Topology with Dynamic Particles”, Proc. CVPR ’93, pp. 82–87, New York, June 1993. [11] D. Terzopoulos and D. Metaxas. “Dynamic 3D Models with Local and Global Deformations: Deformable Superquadrics”, IEEE Trans. Pattern Analysis and Machine Intelligence, 13(7), pp. 703– 714, 1991. [12] B.C. Vemuri and A. Radisavljevic. “From Global to Local, a Continuum of Shape Models with Fractal Priors”, Proc. CVPR ’93, pp. 307–313, New York, June 1993.
6
α(u) 1
0 −π/2
(a)
(b)
(c)
0
(d)
π/2
u
(e)
Figure 12: Fitting of sphere/cylinder data and blending function (e)
α(u) 1
0 −π/2
(a)
(b)
(c)
(d)
(e)
0
(f)
Figure 13: Fitting of chess rook data with global deformations only (d) and with local deformations (e)
(a)
(b)
(c)
(d) α(u) 1
0 −π/2
(e)
(f)
(g)
Figure 14: Fitting of torus data with genus change in (d)
7
0
(h)
π/2
u
π/2
u
Douglas DeCarlo and Dimitri Metaxas Department of Computer & Information Science University of Pennsylvania Philadelphia PA 19104-6389 Abstract We develop a new class of parameterized models based on the linear interpolation of two parameterized shapes using a blending function. Using a small number of additional parameters, blending extends the coverage of shape primitives. In particular, it offers the ability to construct shapes whose genus can change. Blended models are incorporated into our physics-based shape estimation framework which uses dynamic deformable models with global and local deformations. We present experiments involving the extraction of complex shapes including an example of dynamic genus change.
1
Introduction
Developing shape models for shape estimation that cover a wide variety of shapes using a small number of intuitive parameters is a difficult and important open problem. Even though there exist models which have flexible and intuitive representations such as those used in solid modeling [4, 8], they were not designed for shape estimation. The systems described in [5, 10] provide surface connectivity information and can model surfaces of arbitrary topology, but do not provide an abstraction of shape. In fact, no existing shape model with a compact representation can represent objects of varying genus in a unified model. Recently, systems which are applicable for both shape reconstruction and shape recognition were presented [6, 7, 12]. In [7] the shape was specified by its deformation modes and extracted by using a closed-form solution of modal analysis. Shape was represented in [12] using a wavelet basis and estimated by embedding it in a probabalistic framework. Both of these methods provide a collection of parameters ordered by level of detail. The models in [6, 11] incorporate global deformations, which represent prominent shape features, and local deformations, which capture surface detail. For these models, correct configurations are extracted using a physics-based framework where forces are exerted by data. The second author was supported by NSF grant IRI-9309917.
We propose an extension to the shape representation of [6, 11] which we call blended deformable models. Given two shapes that can be defined parametrically on a common material coordinate space, blended shapes are constructed by linear interpolation using a blending function that specifies the relative weight of corresponding points on the two shapes. This new class of models produces a compact and intuitive representation of an object. For example, a sphere and a cylinder blended together could produce a bullet shaped object (see Fig. 2). In addition, it is able to represent shapes of genus 0 and 1: a sphere and torus blended together would produce an object in which a hole will appear depending on the blending function. Fig. 3 shows a variety of shapes that we can create using blending. In this paper, we show how blended models can be incorporated into the previously developed physics-based framework in [6, 11]. We then present modifications to this framework that improve the robustness and stability of the estimation process. We demonstrate our technique through a series of experiments involving incomplete range data from various objects.
2
Geometry of deformable models
As in [6, 11], the position of a point on the model at time t and at position in material coordinate space with respect to an inertial frame of reference is
u
c
x(u; t) = c(t) + R(t)p(u; t);
R
(1)
where is the origin of the inertial frame, and is a rotation matrix which specifies the relative orientation of the inertial frame to a fixed reference frame. In this reference frame, the position of model points , is the sum of a reference shape and a local displacement so that ( ; t) = ( ; t) + ( ; t): (2 )
d
p
s
pu
su du These local displacements, d, allow the representation of fine detail, while the reference shape s captures salient shape features. The reference shape of the model, s, is constructed by applying a global deformation T (such as bending and twisting) with parameters b to a shape primitive e as in s(u) = T(e; b0; b1; : : :): (3 ) i
e
y h(x) = f (x) (x)+ g(x)(1 , (x)) f (x)
The material coordinates of the shape primitive are specified over a domain , and are given by = (u; v) . For a 3-D shape, we have ( ) : ! 3 . These primitives have global shape parameters which specify the shape. Including these parameters, we represent the geometric primitive as ( ; a1; a2; : : :) (4 )
eu
eu
which is defined parametrically in shape parameters ai .
2.1
R
u
(x) 1
g(x) x
u over and has global
0
Figure 1: Blending of two functions blending function (x) u
Supertoroid definition
= 2
f (x), g(x)
x using
α(u) 1
The definition for the superquadric toroid primitive (with 7 shape parameters) is given by
0 @
1 A
a1 2 1 a4 +1 (a4 + C2u )Cv a2 1 2 torus (u; v) = a5 +1 (a5 + C2u )Sv 1 a3 S2u u ( =2; =2] C = sgn(cos ) cos v ( ; ] S = sgn(sin ) sin
e
u
(a)
(5)
(c)
π/2
u
(d)
the “top” of s looks like s2 since ( 2 ) = 0, and how the “bottom” of s looks like s1 since (, 2 ) = 1. The global parameters of s will include the global parameters of s1 and s2 , those that specify (see section 3.1), and the global deformation parameters in T. A single T is applied separately to each shape primitive. The resulting deformed shapes are blended together. While it would be possible to have vary with u and v, we restrict so it does not vary with v. This restriction on limits the coverage of representable shapes. To improve the shape coverage, it is possible to perform blending on blended shapes to produce a more flexible and hierarchical shape structure. When blending shapes, not all combinations of primitives will achieve desirable results. For example, if two primitives are incompatible, the blended result may interpenetrate. There must be a correspondence between the two shapes in space for two shapes to be compatible. Section 5 discusses how this correspondence can be improved without altering the shape. By using different blending functions, we can create the shapes shown in Fig. 3. Notice how some of the objects have both “square” and “round” areas, which is not easily representable using tapering or other global parameterized deformations. The linear interpolation of two curves where a homotopy of curves is used to construct an extruded surface was presented in [8]. This construction is quite different from the blending presented here, since only curves were considered, not shapes.
This is a similar definition of a supertoroid given by Barr [1]. The addition of a5 , a second inner radius parameter allows asymmetric holes. The presence of the scaling factors 1=(a4 + 1) and 1=(a5 + 1) separate the effects of the global size parameters (a1 , a2 and a3 ) from the inner radii parameters (a4 and a5 ) to allow hole size changes that do not affect the global torus size. Now that the model geometry has been described, the shape blending technique can be presented.
Blended models
In a method analogous to the linear interpolation of two points, it is possible to blend two functions. Given two functions, f (x) and g(x), we can blend them using a third function, (x) (with range [0; 1]), so that
h(x) = f (x) (x) + g(x)(1 , (x)):
(6) An example of this is shown in Fig. 1. Notice how h(x) = f (x) where (x) = 1, h(x) = g(x) where (x) = 0, and how h(x) is between f (x) and g(x) everywhere.
Using this idea, we can blend parameterized shapes (such as superquadrics) by the following: s(u; v) = s1 (u; v) (u) + s2 (u; v)(1 , (u))
(b)
0
Figure 2: (a) Shape s1 (b) Shape s2 (c) Blended shape s (d) Blending function (u)
, j j , j j where a1 ; a2 ; a3 ; 1 ; 2 > 0 and a4 ; a5 1. a1 , a2 and a3 are size parameters in the x, y and z directions respectively. 1 and 2 are squareness parameters as in a superquadric. a4 and a5 are hole radius parameters in the x and y directions.
3
0 −π/2
= , 2
(7)
where s1 and s2 are two shape primitives parameterized over , as in Fig. 2(a) and (b). Fig. 2(c) shows the result of blending the shapes shown in Fig. 2(a) and (b). The blending function used to blend the shapes is shown in Fig. 2(d). The blending is performed along u, which corresponds to the z -axis in these shapes (from pole to pole). Notice how
3.1
Blending function parameterization
The blending function, , is implemented as a nonuniform quadratic B-spline function [2]. Given different 2
(a)
Figure 3: Examples of blended shapes types of shape primitives, the domain of may vary. For a superquadric, maps [, 2 ; 2 ] to [0; 1]. The B-spline function is specified using L + 1 control values, fci j i 0 : : :Lg and L knots fui j i 1 : : :Lg, with u1 and uL fixed to be the upper and lower bounds of the domain of . The function has the values (u1 ) = c0 and (uL) = cL and has a continuous first derivative except where two knot values are equal.
3.2
(b)
(c)
(d)
(e)
Figure 4: Changing from a sphere (a) to a torus (e) can be added are limited.
Genus changing
It is also possible to blend objects of genus 0 (a sphere) with objects of genus 1 (a torus). A hole will appear in the blended object as changes. There is no smooth transition between these two shapes because they are not homeomorphic. Yet it is possible to have a transition between the two where there is a single discontinuous event – when the object changes genus. This event affects only the topology of the object, not the shape. An example transition is shown in Fig. 4. In Fig. 4, s1 is a torus, and s2 is a sphere. Initially, (u) = 0, and the genus of the shape is 0. We compute the blended result s using (7). Fig. 4(a) shows the initial sphere. After this, the nodes slowly dimple inward (b) until they touch (c). The values of ( 2 ) and (, 2 ) change from 0 to 1 to produce this dimpling. The object is topologically a sphere, but in (c), has the same shape as a torus with a zero size hole (with a4 = a5 = 1). The connectivity of the surface near this point is changed to be that of a torus. A discussion of how the node interconnections change is given in section 3.3. The hole can now be opened by increasing the torus inner radius parameters, shown in (d) and (e) (which are shown from a slightly different viewpoint to make the hole visible). The topology should be changed from sphere to torus when the shape has the configuration shown in (c). This can be detected by examination of the blending function. There are two constraints on the parameters of a torussphere blend that must be enforced to insure the blended shape remains closed. The torus hole must remain closed when the object is genus 0. When the object is genus 1, ( 2 ) and (, 2 ) must have the value 1 so that the blended object is completely toroidal at the location of the hole. The ideas presented here for changing the genus of shapes can be applied to any shape primitives. The actual steps involved will vary for different primitives. In this case, since varies only with u, the types of holes that
3.3
Node interconnections
When altering the genus of an object, the mesh of nodes must be reconnected to conform to the new topology. This is a straightforward but necessary part of the genus conversion process. Fig. 5 shows how is “folded up” to produce a sphere or torus. The arrows in these diagrams indicate two nodes being “merged” together, since the material coordinates of the nodes map to the same model coordinates. For both the sphere and the torus, a tube is made first (the dotted lines). For a sphere (a), the north and south poles are created by closing each end of the tube. For a torus (b), the ends of the tube are connected together. When the genus changes, the node mesh must be unfolded, and re-folded to have the proper configuration.
u v (a)
(b)
Figure 5: Node interconnection differences between a sphere (a) and torus (b)
4
Dynamics and generalized forces
The dynamics framework given in [6] can be used with few alterations. This framework requires the computation of the global shape model Jacobian, J = @ s=@ qs . > > >> q = (q> c ; q ; qs ; qd ) is the vector of the model’s generalized coordinates, where qc = c(t), q is the quaternion used to specify R(t), qd is the vector used to specify lo> > > > (qs1 and cal deformations, and qs = (q> s1 ; qs2 ; qb ; qT ) qs2 are the parameters of each of the blended shapes, qb is described in section 4.1, and qT are the parameters of the parameterized global deformations). 3
These generalized coordinates are integrated into the dynamic framework, which computes an updated set of generalized coordinates that more closely represent the desired shape. This framework combines the applied forces from the data with stiffness forces of the model to enhance model smoothness, and includes damping for stability. An adaptive Euler integration method is used to implement the dynamics.
4.1
(a)
Figure 6: Example of a superquadric (1 unscaled (a) and scaled space (b)
Blending function dynamics
= (c0 ; : : :cL; u2; : : :; uL,1)> :
(8)
To determine how these parameters change due to applied forces, the Jacobian is needed. Given (7) above,
@ s(u; v) , = s1(u; v) , s2 (u; v) @ @ q(u) (9) Jb = @ qb b To compute @ (u)=@ qb , we apply the product rule to
s1
s2
b
with
sphere (cos
1
u; sin u)
(cos
u
Ss (u); sin Ss (u)) 1
x (a) sphere
z
(10)
where Js1 is the Jacobian for the first shape (with respect to qs1 ), and Js2 is the Jacobian for the second shape (with respect to qs2 ). Intuitively, this means the Jacobians for the components of a blended shape have a greater or lesser effect at a particular location depending on the function .
5
1 4)
superquadric (1 = 14 )
z
the de Boor algorithm [2]. To make sure the components of qb have reasonable values, the constraints ci [0; 1] and ui uj for all 1 i j L are enforced. The addition of blending also changes the Jacobian for the global shape. The Jacobian for the blended shape can be computed using (7) as J = ( (u)J> ; (1 , (u))J> ; J> )>
= 2 =
Franklin [3] uses explicit sampling to generate a more uniform sampling of the surface. This method was designed for static superquadrics, and loses its advantages of efficiency for dynamic shapes. Instead of this, we use an approximation by projecting a unit superquadric onto the unit sphere when 1 < 1 or 2 < 1, and place nodes on the superquadric where they project down to nodes on the sphere, as in Fig. 7(a). This approximation yields fast and adequate results.
The parameters used to construct the blending function are the L + 1 control values and the L , 2 movable knots (u1 and uL are fixed), which yields 2L , 1 total parameters to specify . We concatenate all these parameters into the vector qb , so that qb
(b)
x = cos u
torus (a4 = a5 = 1)
x=
u
1+cos St (2u) 2
=
cos2
St (u)
x (b)
Material space scaling
Figure 7: Material space scaling for a superquadric (a) and torus (b) (first quadrant, y = 0)
For a shape e(u) with u , we can scale (or warp) the domain so that we have e(S (u)), where S : ! is invertible. This operation does not change the shape, but instead changes how the material parameters are mapped onto the object (e.g.– for a discretized object, the nodes will change position, but the shape will remain the same). By introducing this material space scaling, we improve the distribution of nodes (such as in a “square” superquadric), and also improve the compatibility of shape parameterizations when blending different shapes. When discretizing a superquadric, the nodes tend to collect near the “squared” edges if either 1 or 2 is near zero. Fig. 6(a) shows an example of this uneven node spacing.
In order to find the scaling function Ss (u), we solve tan u = tan1 Ss(u). When we generalize to all quadrants, and do a similar calculation for v and 2 , we obtain
Ss (u) = Ss (u; v) = (tan,1 Tu ; tan,1 Tv ) (11) where T = sgn(tan )j tan j . This scaling is performed 1 1
1 2
only when 1 < 1 or 2 < 1. Fig. 8 shows the behavior of (11) on u-v space with decreasing values of . The example where = 0:25 shows the space scale used to correct the node distribution for the superquadric in Fig. 6(b). The scaling in (11) can also be applied to a supertoroid to correctly rescale any 4
ε = 1.0
π/2
π/2
ε = 0.5
π/2
u
u
u
−π/2 −π
−π/2 −π
−π/2 −π
v
π
v
π
ε = 0.25
The fitting examples shown in section 8 show the fit after each step in the fitting schedule. The schedules used in the experiments are given in Fig. 11.
7 v
π
Range data is often incomplete, and usually contains information from only one view of an object. The shape of such an object is underdetermined. As a result, it would be possible for forces to deform this object into an excessively large shape, but still have a reasonable fit on the data. An example of this problem is exhibited in Fig. 9(b).
Figure 8: Example of scaled space (superquadric) with varying = 1 = 2 uneven discretization problems caused by torus squareness when 1 < 1 or 2 < 1. The scaling function for a torus (St ) is used to minimize the effect of the difference in parameterizations of the supertoroid and superquadric when blended. Without scaling, this difference would cause the Jacobians of the blending parameters near the torus hole to have unreasonable values. To minimize this difference when a torus and superquadric are blended, we project the unit torus (with a zero size hole) onto the unit sphere, in a direction parallel to the z -axis (along the hole) as shown in Fig. 7(b). This projection is performed assuming the torus has a zero sized hole (a4 = a5 = 1). It is for this case that the Jacobians of the blending parameters near the hole are affected most. Solving for St, and generalizing for all quadrants, we obtain
St (u) = St (u; v) = (sgn u cos,1 pcos u; v)
(a)
(b)
(c)
Figure 9: Shape with range data (a) covering part of the object fitted without (b) and with volume minimizing force (c)
p
Solina [9] added a minimization term a1 a2 a3 to minimize the volume of the solution. This can be accomplished by a more physics-based approach with similar results. By applying forces to the portions of an object where there are no data forces, the volume of an object can be controlled. Given node i with no data forces applied, the force would be 1 d ,x k ci i (13) fi = Nf v kdci , xi k
(12)
When applying more than one space scaling function, those that make parameterizations compatible should be applied first, followed by those that improve tessellations. For the supertoroid case, we would have etorus (Ss (St(u)))).
6
Volume minimization forces
Parameter scheduling
As the number of parameters increases, having all parameters change at the same time can cause the fitted object to become “stuck” in an undesirable solution. To prevent this, a general schedule for fitting the parameters must be developed. When fitting a blended object, the rough shape should be found first, followed by global deformations, followed by blending parameters. After this, the solution will be close enough to the final solution that allowing all parameters to be changed at once will improve the solution. For objects which are torus blends, there is an additional step added after the blending parameter step, and that is a step to attempt to open the hole. Since varies only with u, we must make sure that the object is aligned with the data correctly so that blending can occur. In general, this alignment can be difficult to determine in advance (especially for torus data). If the fit is not satisfactory, or blending does not occur to any reasonable extent, the fit is restarted to try the other two axes.
where dci is the closest data point to node i, xi is the reference frame position of node i, and Nf is the number of nodes without data forces applied. This force amounts to constant magnitude force in the direction of the nearest data point. kv represents the magnitude of the total force used to control the volume. The value of kv should be highest when the fitting starts, and should be reduced as the deviation from the data decreases. The volume minimization forces should have no effect on local deformations. The results of using the force given in (13) are shown in Fig. 9(c).
8
Experiments
In the following fitting experiments, we show the results of using blended shapes in our system. Fig. 10 shows information on each of the experiments including the number of data points, the resulting mean squared error (MSE), the size of the parameter set, the dimensions of the node mesh, and the number iterations taken for the fit. 5
of nodes, d is the number of data points) due to initial nearest node computations (for force assignment). Once the shape acquires its rough general shape, the complexity approaches O(n + d) since nearest node information can often be carried across iterations. Since less range data can be used in the beginning, this offers an additional constant factor speed increase.
In each of the examples, the initial model configuration is shown. Initially, the model has all global shape parameters equal to 1, and is centered at the center of mass of the data. The blending function is initialized to (u) = 0. At the start of the fit, only 1=10 of the data is used (it is selected randomly). Data sphere/cyl chess rook torus
Source Pts MSE L # Parm MSU 2015 3.64% 9 27 NRCC 1325 1.26% 15 39 CAD 1503 0.62% 2 15
MSU data from Michigan State University PRIP database NRCC data from National Research Council Canada
Mesh 17 17 17 17 16 12
# Iter 223 119 147
9
We have developed and presented a new approach to shape modeling based on shape blending. These models we created can compactly and intuitively represent a large class of shapes in a single model, including shapes of varying genus. This extension is likely to be useful for recognition because blended shapes can be parameterized using a small number of intuitive global parameters. We demonstrated the close to real-time performance of our technique in a variety of shape estimation experiments involving the extraction of complex shapes with incomplete range data.
Figure 10: Experiment data and statistics The schedules (see section 6) used for these fitting examples are shown in Fig. 11. Each stage in these schedules are completed once the average change of parameters per iteration drops below a specified value. Fig. 12 through 14 show the fitting results obtained for the three experiments. 1
2
3 4
Superquadric-Superquadric s2 : a1 , a2 , a3 global deformations s1 : a1 , a2 , a3 ; s2 : a1 , a2 , a3 blending parameters – all parameters
Conclusion
Superquadric-Torus a1 , a2 , a3 global deformations blending parameters
s2 :
References [1] A. Barr. “Superquadrics and Angle-Preserving Transformations”, IEEE Computer Graphics and Applications, 1(1), pp. 11–23, 1981.
s1 : a4 , a5 (hole) all parameters
[2] G. Farin. “Curves and Surfaces for Computer Aided Geometric Design”, Academic Press, Third edition, 1993. [3] W.R. Franklin and A. Barr. “Faster Calculation of Superquadric Shapes”, IEEE Computer Graphics and Applications, 1(3), pp. 41– 47, 1981.
Figure 11: Parameter schedules for fitting experiments Fig. 12 shows the fitting of a sphere/cylinder object. A blend of two superquadrics is used as the model. The initial model and range data is shown in (a), and the rough fit after step 1 is shown in (b). The fitting after step 2 is shown in (c), and (d) shows the fit after step 4 using all the range data. Since this object has a corner where the sphere and cylinder meet, the blending function (e) has developed a C 0 point. Fig. 13 shows the fitting of a rook. This example is fit in a similar way as the previous example, except in (d), the number of nodes was tripled to allow a greater accuracy of the fit. In (e), local deformations were added to allow fitting of some of the detail. Using local deformations, the MSE dropped to 0.59%, and 30 more iterations were performed. Fig. 14 shows the fitting of torus data. The initial range data is shown in (a), and the initialization is shown in (b). The rough fit after step 1 is shown in (c). The hole is “pinched” closed in (d) after step 2, and the genus changes to 1. The hole is opened (step 3) in (e) and (f) (which are the same object from different viewpoints). A final fit using all data (step 4) is shown in (g). Each iteration takes approximately 1 second on a 33 MHz SGI R3000 using data sets of this size. An adaptive Euler method is used to update the object state. Initially, iterations have O(nd) behavior (where n is the number
[4] C.M. Hoffmann and J. Hopcroft. “The Geometry of Projective Blending Surfaces”, Artificial Intelligence, 37, pp. 357–376, 1988. [5] R. Malladi, J.A. Sethian, and B.C. Vemuri, “A Fast Level Set based Algorithm for Topology-Independent Shape Modeling,” to appear in Journal of Mathematical Imaging and Vision, special issue on Topology and Geometry in Computer Vision. [6] D. Metaxas and D. Terzopoulos. “Shape and Nonrigid Motion Estimation Through Physics-Based Synthesis”, IEEE Trans. Pattern Analysis and Machine Intelligence,15(6), pp. 569–579,June, 1993. [7] A. Pentland and S. Sclaroff. “Closed-Form Solutions for Physically Based Shape Modeling and Recognition”, IEEE Trans. Pattern Analysis and Machine Intelligence, 13(7), pp. 715–729, 1991. [8] J. Snyder “Generative Modeling for Computer Graphics and CAD”, Academic Press, 1992. [9] F. Solina and R. Bajcsy. “Recovery of Parametric Models from Range Images: The Case for Superquadrics with Global Deformations”, IEEE Trans. Pattern Analysis and Machine Intelligence, 12(2), pp 131–146, 1990. [10] R. Szeliski, D. Tonnesen and D. Terzopoulos. “Modeling Surfaces of Arbitrary Topology with Dynamic Particles”, Proc. CVPR ’93, pp. 82–87, New York, June 1993. [11] D. Terzopoulos and D. Metaxas. “Dynamic 3D Models with Local and Global Deformations: Deformable Superquadrics”, IEEE Trans. Pattern Analysis and Machine Intelligence, 13(7), pp. 703– 714, 1991. [12] B.C. Vemuri and A. Radisavljevic. “From Global to Local, a Continuum of Shape Models with Fractal Priors”, Proc. CVPR ’93, pp. 307–313, New York, June 1993.
6
α(u) 1
0 −π/2
(a)
(b)
(c)
0
(d)
π/2
u
(e)
Figure 12: Fitting of sphere/cylinder data and blending function (e)
α(u) 1
0 −π/2
(a)
(b)
(c)
(d)
(e)
0
(f)
Figure 13: Fitting of chess rook data with global deformations only (d) and with local deformations (e)
(a)
(b)
(c)
(d) α(u) 1
0 −π/2
(e)
(f)
(g)
Figure 14: Fitting of torus data with genus change in (d)
7
0
(h)
π/2
u
π/2
u
