Folds and cuts: how shading flows into edges ... - Semantic Scholar
Folds and Cuts: How Shading Flows Into Edges Patrick S. Huggins Steven W. Zucker” Center for Computational Vision and Control Yale University New Haven, CT, U.S.A. (huggins,zucker} @cs.yale.edu
Abstract We consider the interactions between edges and intensity distributions in semi-open image neighborhoods surrounding them. Locally this amounts to a kind ofBgure-ground problem, and we analyze the case of smooth surfaces occluding one another: Techniquesfrom differential topology permit a classification of edges based on what we call folds and cuts. Intuitively, folds arise when a surface ‘yolds” out of sight, which in turn nzay “cut” another surface from view. The classijication depends on tangency between an edge tangent map and a shading jlow field. Examples are included.
1. Introduction Edges are crucial to the interpretation of images, as evidenced by the numerous attempts at their detection. Perhaps as important as this detection is the intepretation of the edges themselves. Understanding the geometry of a scene often depends solely on the edges which signal occlusion [9][ 14],[6], while other types of edges may be irrelevant or distracting. However, the identification of occlusion edges is an unresolved problem in vision. Indeed, it is thought to be locally undecidable, as expressed by the classical Gestalt ownership question: does the edge belong to the figure, or to the ground? Viewed as a global problem, it is known to be combinatorially difficult: perfect line drawing interpretation is NP-complete for the simple blocks world [ 161. Various heuristics, such as contour closure or convexity, have been suggested [7], but these do not clarify the connection between edges and scene geometry. Are there circumstances in which the inverse image of an edge onto a scene can be characterized? An examination of natural images suggests that the intensity distribution in the neighborhood of edges contains relevant information, and our goal in this ‘Supported by AFOSR
0-7695-1143-0/01 $10.00 0 2001 IEEE
paper is to show one basic way to exploit it. In an image, occlusion edges arise when the tangent plane to the object “folds” out of sight; this naturally suggests a type of “figure”. In particular, it enjoys a stable pattern of shading with respect to the edge. Fig. 1 shows how shading can be lifted into (orientation, position)-space. Notice in particular that the lift shows a jump in (position, orientation)-space, between the cylinder and the background. Furthermore, notice that this lift osculates the edge lift on the fold side, thereby answering the Gestalt question of edge ownership. At the same time, the fold side of the edge “cuts” the background scene, which implies that the background cannot in general exhibit this relationship to the edge. Our main contribution in this paper is to develop this difference between folds and cuts in a technical sense and to propose a specific mechanism for classifying edges in terms of folds and cuts based on the interaction between edges and the shading flow field.
2 Folds and Cuts In order to understand occlusion edges, we consider the image formation process which gives rise to them. This enables us to formally define folds and cuts, and also provides intuition as to how we can distinguish between them in images where shading is present.
2.1 Definitions Consider an image I of a smooth ( C 2 )surface C; where I is defined over the image domain 2 C R2 ( I : 2 + Rf) and C is a mapping from the surface parameter space, X C R2, to portion of ‘the world’, Y c R3, (E : X -+ Y). For a given viewing direction V E S 2 (the unit sphere), the surface is projected onto the image domain by I’I, : Y + 2.For simplicity, we assume that II is orthographic projection, although this particular choice is not crucial to
153
on C, by Pfold
= {Yp E YI
xp
E
x7v E TYP[C(X)I,Yp = C(%))
Since the singularities of IIv O Clead to discontinuities if we take 2 as the domain, they naturally translate into edges in the image corresponding to the occluding contour and its end points (although due to occlusion and opacity not all of these points, as defined, are visible). Now suppose C is piecewise smooth, i.e. we permit discontinuities of all orders in E. We now have two additional sources of discontinuity in the image mapping: points where the surface itself is discontinuous,
4
Fboundary
= {Yp E YI 3 E
s17lim E(% + 4 # EbC,), E+O
YP
= C(ZP))
and points where the surface normal, N E S2,is discontinuous,
rcreaSe = {yP E YI 36 E S1, lim N ( z p+ €6) # N(x,), E+O
Figure 1. (a) An image of a typical edge due to occlusion. Does the right half of the image appear as a vertically oriented cylinder? (b) The shading flow field of the image; note its orientation relative to that of the edge. (c) The shading flow field as it appears in 2-y-8 space. Here the shading in the righthand side of the image is clearly tangent to the edge, while the shading in the lefthand side is transverse. We denote these configurationsas FOLD and CUT, respectively.
YP
= C(ZP))
As a result of occlusion, the edges present in the image have two pre-images in the scene: the edge of the occluder, and the curve that this projects to, along the view direction, on the occluded background. We denote this second locus of points as I?-shadow, rr-shadow = {yp E YI 3t E
Yq E
R+,y P
rfoid U rboundary
+ tV, U rcrease)
= yq
Fig. 2 summarizes the points defined so far. our reasoning. Thus the mapping from the surface domain to the image (domain) takes R2 to R2. Points in the resulting image are either regular or singular, depending on whether the Jacobian of the surface to image mapping, d(IIv o C) is of full rank or not. An important result in differential topology is the Whitney Theorem for mappings from R2 to R2 [4][ 1 11, which states that such mappings generically have only two types of singularities, folds and cusps. Let T, [A]denote the tangent space of the manifold A at the point 2.
Definition 2 The CUT is the set of points in the image domain where IIv o C is discontinuous due to occlusion, surface discontinuities, or surface normal discontinuities, and points where N is discontinuous. 7Cut
= { z p E ZI
v E TYP[W)I7Y p = W ZP
n(rr-shadozuurboundaryUrcrease))
Note that ? f o l d f l 7cut may be non-empty, while their respective pre-images are disjoint, except at special points such as corners and cusps.
2.2 The appearance of folds and cuts
Definition 1 The FOLD is the singularity locus of the surface to image mapping, IIv 0 C, where C is smooth. In the case of orthographic projection the fold is the image of those points on the surface whose tangent plane contains the view direction. Yfold
= {+ E 21 z p E
As a result of the projective singularity which gives rise to folds, folds and cuts have distinct appearances in images. Consider the simple case of a surface viewed such that its image has a fold, with a curve on the surface which runs through the fold. In general, the curve in the image osculates the fold (Fig. 3). Let (Y be a smooth curve on the surface (C) parameterized by U C R; a : U Xi . If C o (Y passes through
P ) ,
= nv(YP>)
We denote the fold generator, i.e. the pre-image of 7 f o l d
154
See [5] for a proof. This result appears in different forms in several domains in vision, e.g. work on line drawing interpretation [15], shadows [ 12][8], and silhouettes [18], the main difference being the physical nature of the curve considered. A similar result applies to a family of curves lying on a surface. Shading on a surface can be treated as such a family of curves, so the tangency condition applies, as noted in [ 17][ IO]; see also [2]. For a discontinuity in the image due to a cut, the situation is reversed: a curve will generically appear to be transverse to the edge locus, and similarly for a family of curves.
Figure 2. The categories of points under a mapping from R2to R2: (1) a regular point, (2) a fold point, (3) a cusp, (4) a I?-shadow point, (5) a crease point, (6) a boundary point. The viewpoint is taken to be at the upper left. From this position the fold and the J?-shadow appear aligned.
3 Shading at an edge Now consider a shaded image of a surface. Assuming point source illumination from infinity in the direction L and a Lambertian surface with constant albedo, then the shading at a point p is s(p) = N . L where N is the normal to the surface at p ; this is the standard model assumed by most shape-from-shading algorithms. We define the shadingjowfield to be the unit vector field tangent to the level sets of the shading field:
Figure 3. (a) A curve, p, passing through a point on the fold generator, r j o l d ; as defined by the viewpoint at the lower left. The tangent to the curve T[P]at the point of intersection lies in the tangent plane to the surface at that point, T, as does the tangent to the fold generator, T [ r f o l d ] . (b) In the image, the tangent plane to the surface at the fold projects to a line, and so the curve, 6 = II(p), is tangent to the fold, y j o l d .
The structure of the shading flow field can be used to distinguish between several types of edges, e.g. cast shadows and albedo changes [I]. Applying the intuition developed in the previous section, the shading flow field can be used to categorize edge neighborhoods as fold or cut. Since C is smooth (except possibly at and at isolated singular points), N varies smoothly, and as a result so does s. Thus S is the tangent field to a family of smooth curves. Consider S at an edge point p. If p is a fold point, then in the image S ( p ) = Tp[yfOld].If p is a cut point, See Fig. 4.This is the basis for our then S ( p ) # Tp[7cut]. proposed categorization of edge points.
rCut
point yp on the surface then Typ[C 0 a ( U ) ]C T y p [ C ( X ) ] . An immediate consequence of this for images is that, if we choose V such that I I v ( g P ) E y f o l d , then the image of cy is tangent to the fold, i.e. TLp[II 0 C 0 a ( U ) ]= Tz,[ Y j o l d ( Y ) ] , where z p = nv(yp). There is only one choice of V for which this does not o a ( U ) ] . At such a point the image of cy hold: V E Tvp[C has a cusp and is transverse (i.e. non-tangent) to “ f o l d . Intuitively, this is not a “generic viewpoint” [ 3 ] ;in fact, using the concept of transversaliv we can prove the following:
Proposition 1 At an edge point p E y in an image we can define two semi-open neighborhoods, N; and N F , where the sueace to image mapping is continuous in each neighborhood. We can then classify p as follows:
1. FOLD-FOLD: The shadingflow is tangent to y in N; and in Np”,with exception at isolated points. 2. FOLD-CUT: The shadingjow is tangent to y at p in N: and the shadingjow is transverse to at p in
Theorem 1 lj in an image of an arbitrary surj+iace,with an arbitrary curve lying on the surface, viewed from an arbitrary viewpoint, the curve on the surface crosses the fold generator; then the curve in the image will typically appear tangent to the fold at the corresponding point in the image.
r
N:, with exception at isolated points. 3. CUT-CUT: The shading flow is transverse to y ut p in N: and in NF, with exception at isolated points.
155
Figure 5. The effects of blur on edge classification. The shading flow field of Fig. l b with (a) twice as much blur and (b) four times as much blur. Observe how within the blur window centered on the edge the shading flow field always appear to be tangent to the edge. that it is readily generalized to more realistic shading distributions. For example, shading that results from diffuse lighting can be expressed in terms of an aperture function that smoothly varies over the surface [ 131, meeting the conditions we described in Section 3, thus enabling us to make the fold-cut distinction. The same analysis could be applied to texture or range data (Fig. 9).
Figure 4. Shaded surfaces with a fold (a) and a cut (c); the viewpoint is from the lower left. Their respective shading flow fields approach tangency near the fold (b), but remain transverse at the cut (d).
References
In the next section we discuss the computation of this categorization and demonstrate its applicability.
[I] P. Breton and S. W. Zucker. Shadows and shading flow fields. In Computer Vision and Pattern Recognition, pages 782-789, 1996. [2] J. P. Dufour. Familles de courbes planes differentiables. Topology, 4:449-474, 1983. [3] W. T. Freeman. The generic viewpoint assumption in a framework for visual perception. Nature, 368:542-545, 1994. [4] M. Golubitsky and M. Guillemin. Stable Mappings and Their Singulurities. Springer-Verlag, 1973. [ 5 ] P. S. Huggins and S. W. Zucker. How folds cut a scene. In International Workshop on Visual Form 4,2001. [6] K. Ikeuchi and B. K. P. Horn. Numerical shape from shading and occluding boundaries. Artificial Intelligence, 17:141184, 1981. [7] G. Kanizsa. Organization in Vision. Praeger, 1979. [8] D. C. Knill, P. Mamassian, and D. Kersten. The geometry of shadows. J Opt SOCAm A , 14, 1997. [9] J. J. Koenderink. What does the occluding contour tell us about solid shape? Perception, 13:321-330, 1976. [IO] J . J. Koenderink. Solid Shape. MIT Press, 1990. [ 1 I ] J. J. Koenderink and A. J. van Doorn. The singularitiesof the visual mapping. Biological Cybemetics, 245 1-59, 1976. [12] D. L. and S. N. Singularities of illuminated surfaces. International Journal Computer Vision, 23:207-216, 1997. [I31 M. S. Langer and S. W. Zucker. Shape from shading on a cloudy day. J Opt Soc Am A , 1 11467478, 1994. [ 141 J. Malik. Interpreting line drawings of curved objects. International Journal of Computer Vision, 1:73-107, 1987. [ 151 V. S. Nalwa. Line-drawing interpretation: a mathematical framework. International Journal of Cornputer Vision, 21103-124, 1988.
4 Computation The condition we have described to classify edges requires a measurement of the shading flow field at an edge point. However, this is not straightforward because derivative operators that straddle an edge will produce bogus results. As an approximation, we therefore measure the shading flow at a point offset some distance from the edge in the normal direction, however we need to be careful that our measurements are not influenced by the other side of the edge (see Fig. 5). To acheive this we use an offset distance greater than the radius of our derivative filter. By taking shading flow measurements away from the edge, we necessarily deviate from ideal tangency measurements. To combat this we extrapolate the shading flow to the edge point using second order derivative measurements (the Hessian of the image) at our offset point. Figures 6 through 10 demonstrate the results of our computations.
5 Conclusions We have shown how occlusion gives rise to folds and cuts, and how these labels can be applied to edge classifi-
cation. Rough categorizations are computable locally. Furthermore, the advantage of our analysis for this problem is
156
Figure 6. A Klein bottle (a) and its shading flow field at a fold (b) and a cusp (c) (the highlighted regions of (a)). On the fold side of the edge, the shading flow field is tangent to edge, while on the cut side it is transverse. In the vicinity of a cusp, the transition is evident as the shading flow field swings around the cusp point and becomes discontinuous. (d) and (e) depict these flow fields in 2-y-8 space. [I61 P. Parodi. The complexity of understanding line-drawings of origami scenes. International Journal of Computer Ksion, 18: 139-1 70, 1996. [17] 3. H. Rieger. The geometry of view space of opaque objects bounded by smooth surfaces. Artijicial Intelligence, 44: 140, 1990. [ 181 P. U. Tse and M. K. Albert. Amodal completion in the absence of image tangent discontinuities. Perception, 27:455464, 1998.
Figure 7. (a) The Klein bottle. (b) The results of our classification using a first order approximation to the shading flow field at an edge. Each edge is classified on both sides. WHITE denotes a FOLD, BLACK denotes a CUT. (c) The results of our classification using a second order approximation.
157
Figure 8. (a) A checkered sphere. (b) The results of our classification; WHITEZFOLD, BLACKZCUT. (c) The shading flow field of the lower right region of the sphere. Note how the shading flow field is interpreted as transverse to the albedo edge, resulting in a CUTCUT classification.
Figure 10. (a) The nose cone of an F16 and (b) its shading flow field. Despite noise in the measurements, the tangency of the shading flow field to the occluding contour is clearly visible on the fold side of the edge, while it is transverse on the background side.
Figure 9. (a) Range image of a toy. Here the “shading flow field” is actually the tangent field to the isodepth contours. However, our classification (b) still applies.
158
Abstract We consider the interactions between edges and intensity distributions in semi-open image neighborhoods surrounding them. Locally this amounts to a kind ofBgure-ground problem, and we analyze the case of smooth surfaces occluding one another: Techniquesfrom differential topology permit a classification of edges based on what we call folds and cuts. Intuitively, folds arise when a surface ‘yolds” out of sight, which in turn nzay “cut” another surface from view. The classijication depends on tangency between an edge tangent map and a shading jlow field. Examples are included.
1. Introduction Edges are crucial to the interpretation of images, as evidenced by the numerous attempts at their detection. Perhaps as important as this detection is the intepretation of the edges themselves. Understanding the geometry of a scene often depends solely on the edges which signal occlusion [9][ 14],[6], while other types of edges may be irrelevant or distracting. However, the identification of occlusion edges is an unresolved problem in vision. Indeed, it is thought to be locally undecidable, as expressed by the classical Gestalt ownership question: does the edge belong to the figure, or to the ground? Viewed as a global problem, it is known to be combinatorially difficult: perfect line drawing interpretation is NP-complete for the simple blocks world [ 161. Various heuristics, such as contour closure or convexity, have been suggested [7], but these do not clarify the connection between edges and scene geometry. Are there circumstances in which the inverse image of an edge onto a scene can be characterized? An examination of natural images suggests that the intensity distribution in the neighborhood of edges contains relevant information, and our goal in this ‘Supported by AFOSR
0-7695-1143-0/01 $10.00 0 2001 IEEE
paper is to show one basic way to exploit it. In an image, occlusion edges arise when the tangent plane to the object “folds” out of sight; this naturally suggests a type of “figure”. In particular, it enjoys a stable pattern of shading with respect to the edge. Fig. 1 shows how shading can be lifted into (orientation, position)-space. Notice in particular that the lift shows a jump in (position, orientation)-space, between the cylinder and the background. Furthermore, notice that this lift osculates the edge lift on the fold side, thereby answering the Gestalt question of edge ownership. At the same time, the fold side of the edge “cuts” the background scene, which implies that the background cannot in general exhibit this relationship to the edge. Our main contribution in this paper is to develop this difference between folds and cuts in a technical sense and to propose a specific mechanism for classifying edges in terms of folds and cuts based on the interaction between edges and the shading flow field.
2 Folds and Cuts In order to understand occlusion edges, we consider the image formation process which gives rise to them. This enables us to formally define folds and cuts, and also provides intuition as to how we can distinguish between them in images where shading is present.
2.1 Definitions Consider an image I of a smooth ( C 2 )surface C; where I is defined over the image domain 2 C R2 ( I : 2 + Rf) and C is a mapping from the surface parameter space, X C R2, to portion of ‘the world’, Y c R3, (E : X -+ Y). For a given viewing direction V E S 2 (the unit sphere), the surface is projected onto the image domain by I’I, : Y + 2.For simplicity, we assume that II is orthographic projection, although this particular choice is not crucial to
153
on C, by Pfold
= {Yp E YI
xp
E
x7v E TYP[C(X)I,Yp = C(%))
Since the singularities of IIv O Clead to discontinuities if we take 2 as the domain, they naturally translate into edges in the image corresponding to the occluding contour and its end points (although due to occlusion and opacity not all of these points, as defined, are visible). Now suppose C is piecewise smooth, i.e. we permit discontinuities of all orders in E. We now have two additional sources of discontinuity in the image mapping: points where the surface itself is discontinuous,
4
Fboundary
= {Yp E YI 3 E
s17lim E(% + 4 # EbC,), E+O
YP
= C(ZP))
and points where the surface normal, N E S2,is discontinuous,
rcreaSe = {yP E YI 36 E S1, lim N ( z p+ €6) # N(x,), E+O
Figure 1. (a) An image of a typical edge due to occlusion. Does the right half of the image appear as a vertically oriented cylinder? (b) The shading flow field of the image; note its orientation relative to that of the edge. (c) The shading flow field as it appears in 2-y-8 space. Here the shading in the righthand side of the image is clearly tangent to the edge, while the shading in the lefthand side is transverse. We denote these configurationsas FOLD and CUT, respectively.
YP
= C(ZP))
As a result of occlusion, the edges present in the image have two pre-images in the scene: the edge of the occluder, and the curve that this projects to, along the view direction, on the occluded background. We denote this second locus of points as I?-shadow, rr-shadow = {yp E YI 3t E
Yq E
R+,y P
rfoid U rboundary
+ tV, U rcrease)
= yq
Fig. 2 summarizes the points defined so far. our reasoning. Thus the mapping from the surface domain to the image (domain) takes R2 to R2. Points in the resulting image are either regular or singular, depending on whether the Jacobian of the surface to image mapping, d(IIv o C) is of full rank or not. An important result in differential topology is the Whitney Theorem for mappings from R2 to R2 [4][ 1 11, which states that such mappings generically have only two types of singularities, folds and cusps. Let T, [A]denote the tangent space of the manifold A at the point 2.
Definition 2 The CUT is the set of points in the image domain where IIv o C is discontinuous due to occlusion, surface discontinuities, or surface normal discontinuities, and points where N is discontinuous. 7Cut
= { z p E ZI
v E TYP[W)I7Y p = W ZP
n(rr-shadozuurboundaryUrcrease))
Note that ? f o l d f l 7cut may be non-empty, while their respective pre-images are disjoint, except at special points such as corners and cusps.
2.2 The appearance of folds and cuts
Definition 1 The FOLD is the singularity locus of the surface to image mapping, IIv 0 C, where C is smooth. In the case of orthographic projection the fold is the image of those points on the surface whose tangent plane contains the view direction. Yfold
= {+ E 21 z p E
As a result of the projective singularity which gives rise to folds, folds and cuts have distinct appearances in images. Consider the simple case of a surface viewed such that its image has a fold, with a curve on the surface which runs through the fold. In general, the curve in the image osculates the fold (Fig. 3). Let (Y be a smooth curve on the surface (C) parameterized by U C R; a : U Xi . If C o (Y passes through
P ) ,
= nv(YP>)
We denote the fold generator, i.e. the pre-image of 7 f o l d
154
See [5] for a proof. This result appears in different forms in several domains in vision, e.g. work on line drawing interpretation [15], shadows [ 12][8], and silhouettes [18], the main difference being the physical nature of the curve considered. A similar result applies to a family of curves lying on a surface. Shading on a surface can be treated as such a family of curves, so the tangency condition applies, as noted in [ 17][ IO]; see also [2]. For a discontinuity in the image due to a cut, the situation is reversed: a curve will generically appear to be transverse to the edge locus, and similarly for a family of curves.
Figure 2. The categories of points under a mapping from R2to R2: (1) a regular point, (2) a fold point, (3) a cusp, (4) a I?-shadow point, (5) a crease point, (6) a boundary point. The viewpoint is taken to be at the upper left. From this position the fold and the J?-shadow appear aligned.
3 Shading at an edge Now consider a shaded image of a surface. Assuming point source illumination from infinity in the direction L and a Lambertian surface with constant albedo, then the shading at a point p is s(p) = N . L where N is the normal to the surface at p ; this is the standard model assumed by most shape-from-shading algorithms. We define the shadingjowfield to be the unit vector field tangent to the level sets of the shading field:
Figure 3. (a) A curve, p, passing through a point on the fold generator, r j o l d ; as defined by the viewpoint at the lower left. The tangent to the curve T[P]at the point of intersection lies in the tangent plane to the surface at that point, T, as does the tangent to the fold generator, T [ r f o l d ] . (b) In the image, the tangent plane to the surface at the fold projects to a line, and so the curve, 6 = II(p), is tangent to the fold, y j o l d .
The structure of the shading flow field can be used to distinguish between several types of edges, e.g. cast shadows and albedo changes [I]. Applying the intuition developed in the previous section, the shading flow field can be used to categorize edge neighborhoods as fold or cut. Since C is smooth (except possibly at and at isolated singular points), N varies smoothly, and as a result so does s. Thus S is the tangent field to a family of smooth curves. Consider S at an edge point p. If p is a fold point, then in the image S ( p ) = Tp[yfOld].If p is a cut point, See Fig. 4.This is the basis for our then S ( p ) # Tp[7cut]. proposed categorization of edge points.
rCut
point yp on the surface then Typ[C 0 a ( U ) ]C T y p [ C ( X ) ] . An immediate consequence of this for images is that, if we choose V such that I I v ( g P ) E y f o l d , then the image of cy is tangent to the fold, i.e. TLp[II 0 C 0 a ( U ) ]= Tz,[ Y j o l d ( Y ) ] , where z p = nv(yp). There is only one choice of V for which this does not o a ( U ) ] . At such a point the image of cy hold: V E Tvp[C has a cusp and is transverse (i.e. non-tangent) to “ f o l d . Intuitively, this is not a “generic viewpoint” [ 3 ] ;in fact, using the concept of transversaliv we can prove the following:
Proposition 1 At an edge point p E y in an image we can define two semi-open neighborhoods, N; and N F , where the sueace to image mapping is continuous in each neighborhood. We can then classify p as follows:
1. FOLD-FOLD: The shadingflow is tangent to y in N; and in Np”,with exception at isolated points. 2. FOLD-CUT: The shadingjow is tangent to y at p in N: and the shadingjow is transverse to at p in
Theorem 1 lj in an image of an arbitrary surj+iace,with an arbitrary curve lying on the surface, viewed from an arbitrary viewpoint, the curve on the surface crosses the fold generator; then the curve in the image will typically appear tangent to the fold at the corresponding point in the image.
r
N:, with exception at isolated points. 3. CUT-CUT: The shading flow is transverse to y ut p in N: and in NF, with exception at isolated points.
155
Figure 5. The effects of blur on edge classification. The shading flow field of Fig. l b with (a) twice as much blur and (b) four times as much blur. Observe how within the blur window centered on the edge the shading flow field always appear to be tangent to the edge. that it is readily generalized to more realistic shading distributions. For example, shading that results from diffuse lighting can be expressed in terms of an aperture function that smoothly varies over the surface [ 131, meeting the conditions we described in Section 3, thus enabling us to make the fold-cut distinction. The same analysis could be applied to texture or range data (Fig. 9).
Figure 4. Shaded surfaces with a fold (a) and a cut (c); the viewpoint is from the lower left. Their respective shading flow fields approach tangency near the fold (b), but remain transverse at the cut (d).
References
In the next section we discuss the computation of this categorization and demonstrate its applicability.
[I] P. Breton and S. W. Zucker. Shadows and shading flow fields. In Computer Vision and Pattern Recognition, pages 782-789, 1996. [2] J. P. Dufour. Familles de courbes planes differentiables. Topology, 4:449-474, 1983. [3] W. T. Freeman. The generic viewpoint assumption in a framework for visual perception. Nature, 368:542-545, 1994. [4] M. Golubitsky and M. Guillemin. Stable Mappings and Their Singulurities. Springer-Verlag, 1973. [ 5 ] P. S. Huggins and S. W. Zucker. How folds cut a scene. In International Workshop on Visual Form 4,2001. [6] K. Ikeuchi and B. K. P. Horn. Numerical shape from shading and occluding boundaries. Artificial Intelligence, 17:141184, 1981. [7] G. Kanizsa. Organization in Vision. Praeger, 1979. [8] D. C. Knill, P. Mamassian, and D. Kersten. The geometry of shadows. J Opt SOCAm A , 14, 1997. [9] J. J. Koenderink. What does the occluding contour tell us about solid shape? Perception, 13:321-330, 1976. [IO] J . J. Koenderink. Solid Shape. MIT Press, 1990. [ 1 I ] J. J. Koenderink and A. J. van Doorn. The singularitiesof the visual mapping. Biological Cybemetics, 245 1-59, 1976. [12] D. L. and S. N. Singularities of illuminated surfaces. International Journal Computer Vision, 23:207-216, 1997. [I31 M. S. Langer and S. W. Zucker. Shape from shading on a cloudy day. J Opt Soc Am A , 1 11467478, 1994. [ 141 J. Malik. Interpreting line drawings of curved objects. International Journal of Computer Vision, 1:73-107, 1987. [ 151 V. S. Nalwa. Line-drawing interpretation: a mathematical framework. International Journal of Cornputer Vision, 21103-124, 1988.
4 Computation The condition we have described to classify edges requires a measurement of the shading flow field at an edge point. However, this is not straightforward because derivative operators that straddle an edge will produce bogus results. As an approximation, we therefore measure the shading flow at a point offset some distance from the edge in the normal direction, however we need to be careful that our measurements are not influenced by the other side of the edge (see Fig. 5). To acheive this we use an offset distance greater than the radius of our derivative filter. By taking shading flow measurements away from the edge, we necessarily deviate from ideal tangency measurements. To combat this we extrapolate the shading flow to the edge point using second order derivative measurements (the Hessian of the image) at our offset point. Figures 6 through 10 demonstrate the results of our computations.
5 Conclusions We have shown how occlusion gives rise to folds and cuts, and how these labels can be applied to edge classifi-
cation. Rough categorizations are computable locally. Furthermore, the advantage of our analysis for this problem is
156
Figure 6. A Klein bottle (a) and its shading flow field at a fold (b) and a cusp (c) (the highlighted regions of (a)). On the fold side of the edge, the shading flow field is tangent to edge, while on the cut side it is transverse. In the vicinity of a cusp, the transition is evident as the shading flow field swings around the cusp point and becomes discontinuous. (d) and (e) depict these flow fields in 2-y-8 space. [I61 P. Parodi. The complexity of understanding line-drawings of origami scenes. International Journal of Computer Ksion, 18: 139-1 70, 1996. [17] 3. H. Rieger. The geometry of view space of opaque objects bounded by smooth surfaces. Artijicial Intelligence, 44: 140, 1990. [ 181 P. U. Tse and M. K. Albert. Amodal completion in the absence of image tangent discontinuities. Perception, 27:455464, 1998.
Figure 7. (a) The Klein bottle. (b) The results of our classification using a first order approximation to the shading flow field at an edge. Each edge is classified on both sides. WHITE denotes a FOLD, BLACK denotes a CUT. (c) The results of our classification using a second order approximation.
157
Figure 8. (a) A checkered sphere. (b) The results of our classification; WHITEZFOLD, BLACKZCUT. (c) The shading flow field of the lower right region of the sphere. Note how the shading flow field is interpreted as transverse to the albedo edge, resulting in a CUTCUT classification.
Figure 10. (a) The nose cone of an F16 and (b) its shading flow field. Despite noise in the measurements, the tangency of the shading flow field to the occluding contour is clearly visible on the fold side of the edge, while it is transverse on the background side.
Figure 9. (a) Range image of a toy. Here the “shading flow field” is actually the tangent field to the isodepth contours. However, our classification (b) still applies.
158
