Acquiring 3d Models From Rotation And Highlights - Semantic Scholar
Acquiring 3D Models from Rotation and Highlights Jiang Y U Zheng, Y oshihiro Fukagawa, Tetsuo Ohtsuka and Norihiro Abe Faculty of Computer Science and Systems Engineering Kyushu Institute of Technology 680-4 Kawazu, Iizuka, Fukuoka 820, Japan
Abstract
those necessary information from continuous images. Finally, experimental results are &scussed.
Thts paper proposes an approach to acquire 3D models of objects with specular reflectance for graphcs use. Highlight and rotation informations are employed in the model recovery. We control object rotation and extract motion of a hghlight stripe, from which the object shape can be qualitatively inferred and quantitatively reconstructed by solving a first-order linear differential equation. We have experimented on simulated and real objects to obtain their models.
2 Motion Characteristics of Features 2.1 Motion of Fixed Points The system setting is &splayed in Fig. 1. When an object rotates around a known axis, its continuous images are taken and a spatial-temporal volume is piled. The y axis of the image frame is set parallel to the rotation axis using a simple calibration and the optical axis of camera goes through the rotation axis. An object point Po(,Y, Z) described in the object centered coordinates system is projected as p(x,y,8) in the volume. The surface normal at the point is n=[nx, ny, n d . The rotation angle 8 is known and is clockwise in the analysis. A linear illuminant set in the vertical hrection is long enough to produce highlights for surface normals with different n, components. It locates at distant position so that the horizontal components of incident lights are written as an approximate vector L=[L,,L.J. The components L, and L, can be further denoted by its angle $0 from the camera direction, whch can be written as L=[-sincpo, coscpo] and $oE(-zi2, ~ 1 2 ) . Obviously, the highlight stripes move on object surface during the rotation. We therefore denote the hghlight point as H(X(B), Y@), Z(8)) and its image position as h(x(B),
1. Introduction The objective of t h s work is to establish a 3D graphcs model of an object when it is rotated. For an object with rich texture, one can use shape from motion in model recovering [3,4]. For objects with only convex surfaces, shape from contour method is applicable [l]. In this paper, we deal with smooth surfaces with specular reflectance which may yield hghlights. A hghlight has been considered as a noise of surface features and some effort has been made to eliminate it from the object surface. However, in the case where few texture appears on a smooth surface, highlights plays an important rule in perceiving a shape. In order to recover a complete model, we put the object on a turn table and rotate it so that it reveals all of its surfaces to camera. The rotation is readable. The direction of a static illuminant is known by a simple calibration. The issue becomes shape from shadmg and known rotation. To simplify the problem, we use orthogonal projection and parallel illumination. At present, we assume objects have smooth surfaces. The motion information is taken from Epipolar-Plane Images parallel to the rotation plane. Two lunds of visual features ----- fixed features either from comer points or texture points, and hghlights are considered to use. Among them, the hghlight gradually shift on object surface during the rotation; the motion of it can be tracked in EPIs at different heights. From obtained trajectories, object surfaces can be calculated. It is not difficult to realize that normal can be determined from hghlights shifting on the surfaces during the rotation. As to the surface itself, we found that a smooth surface can be described by a first-order differential equation which has a unique solution. Combining those fixed features as boundary condition, we can finally reconstruct the model.
Axis
Light
Orthogonal projection
Fig. 1 Image formation geometry of shape from rotation.
As an object rotates, its surface points have their traces in the correspondmg EPIs as sinusoidal curves of half visible period, even they are not distinct enough to be tracked. If the component of a surface normal in the rotation plane is discontinuous at a point, the shadings on
In the following, w e will introduce some qualitative
characteristics of highlight trajectory and the principle of shape recovery scheme. Then we describe how to extract
33 1
1051-4651/94$04.00 0 1994 IEEE
Authorized licensed use limited to: IUPUI. Downloaded on June 03,2010 at 18:05:15 UTC from IEEE Xplore. Restrictions apply.
its both sides are different and its trace in EPI appears as an edge (typically a segment of sinusoidal curve). If the albedo has a discontinuity at a point, the point also draws a sinusoidal trace of edge in the EPI. These two kinds of points are called fixed points since, for such a point, multiple lines of sight through its projections in dfferent images cross at the same 3D position.
I
/
Rotational axis
connecting their hghlight trajectories in EPI. Figure 4 shows two types of shapes, for instance, whch are convexconcave-convex and comer-linear-convex combinations, respectively. In the first case, hghlight A moves on the first convex surface. In the meanwhile, point B with zero curvature (where shape changes from concave to convex) becomes a highlight point. It splits into two highlight points C and D that move on convex and concave shapes separately. Point D then merges with hghlight point A at another zero curvature point E and disappears. Thls splitting and merging process can be observed from trajectories of hghlights in the correspondmg EPI. We can assert that trajectories of concave and convex shapes should have a smooth connection. The tangent of trajectory is horizontal at the connecting points. T h s is because splitting and merging points are zero curvature as a short linear segment which has a horizontal trajectory in EPI. Similarly, we can qualitatively know the trajectory of highlight for the comer-linear-convex combination (Fig. 4(b)). It is a sinusoidal curve A followed by a horizontal segment AB, and then a trace of convex shape. At the corner point (an extreme case of convex shape), the hghlight stays constant.
illumination direction
Image fame
/
Fig. 2 One cross section of an object parallel to the
shape & highlight
rotation plane.
2.2 Qualitative Motion of Highlights Let us first qualitatively look at the motion types of hghlights according to shapes such as comer, convex, linear, concave, etc. Based on analysis of highlight's trajectories, we can qualitatively infer the shapes. If the surface normal and surface albedo are continuous at a point, matching its projections in continuous images is no longer possible as what has been done for fixed points [1,2,6]. An alternative way is to look at the shading. As an object rotates, its surface elements, in turn, face to the illuminant. A highlight, determined by the surface normal, shlft on surfaces and its trace is possible to be located with respect to the rotation angle. According to curvatures along object boundary on a horizontal cross section, shapes are categorized as either of convex corner, convex, linear, concave. We hence find an interesting effect that hghlights have some basic types of trajectories over the traces of surface points in the correspondmg EpipolarPlane Image. Figure 3 shows hghlight moves on the surfaces and correspondlng trajectories over traces of surface points. On a convex shape, for example, the highlight moves relatively in the inverse direction of the rotation, which yields its image velocity lower than that of surface points (see its trace at right). A linear boundary, however, has its points face the light direction at the same time, which generates a horizontal stripe of highlights in the EPI. Further, a concave shape has its highlight moving in the same direction of rotation; its image velocity is hgher than that of surface points. At a corner, hghlight does not appear if the corner is strictly a zero curvature. Combining dlfferent surfaces together corresponds to
t r a c e of highlight ir !PI
comer C=oO
I convex c>o
fixed c)
$J I o \
plane
c=o
:oncave C d
0
I
U \
% o \
position of highlight i ~ u " a t i o n direction
0
direction
h trace of hiphdight
s trace of surface points
Fig. 3 Trajectories of highhghts over traces of surface points of Mferent shapes.
As a result, a highlight shifts on object surfaces and passes all surface points at least one time if no serious 332
Authorized licensed use limited to: IUPUI. Downloaded on June 03,2010 at 18:05:15 UTC from IEEE Xplore. Restrictions apply.
occlusion of light occurs. Generated from either fixed or shifting points on object surfaces, a queue of connected trajectories in EPI acrosses trajectories of all surface points in one period of rotation. We hence attempt to compute positions of all surface points from this queue. Curvaturesof shapes r>n -. .
equation from the camera geometry. ~ ( 0=) x H = X(0) cos0 + Z(0) sin0 (4) Assuming the shape is not linear, the corresponding highlight trace is then not horizontal and we can take derivative of (4) with respect to 0 to obtain
3
x'(0) = Xe'cosO+Ze'sinO - Xsine +Zcose (9 The first two terms express a possible shift of H on the surface and (X'e, Z'e) gwes the tangent direction of the boundary. In fact, the tangent direction at the hghlight point dlvides half the angle between camera viewing &ration and the drection of illumination in the rotational plane (incident angle equals to reflecting angle). Therefore, the surface normal dlrects to the angle 0+(p0/2-~/2 in the object centered coordinate system and we obtain equation
Rotyc
Abstract
those necessary information from continuous images. Finally, experimental results are &scussed.
Thts paper proposes an approach to acquire 3D models of objects with specular reflectance for graphcs use. Highlight and rotation informations are employed in the model recovery. We control object rotation and extract motion of a hghlight stripe, from which the object shape can be qualitatively inferred and quantitatively reconstructed by solving a first-order linear differential equation. We have experimented on simulated and real objects to obtain their models.
2 Motion Characteristics of Features 2.1 Motion of Fixed Points The system setting is &splayed in Fig. 1. When an object rotates around a known axis, its continuous images are taken and a spatial-temporal volume is piled. The y axis of the image frame is set parallel to the rotation axis using a simple calibration and the optical axis of camera goes through the rotation axis. An object point Po(,Y, Z) described in the object centered coordinates system is projected as p(x,y,8) in the volume. The surface normal at the point is n=[nx, ny, n d . The rotation angle 8 is known and is clockwise in the analysis. A linear illuminant set in the vertical hrection is long enough to produce highlights for surface normals with different n, components. It locates at distant position so that the horizontal components of incident lights are written as an approximate vector L=[L,,L.J. The components L, and L, can be further denoted by its angle $0 from the camera direction, whch can be written as L=[-sincpo, coscpo] and $oE(-zi2, ~ 1 2 ) . Obviously, the highlight stripes move on object surface during the rotation. We therefore denote the hghlight point as H(X(B), Y@), Z(8)) and its image position as h(x(B),
1. Introduction The objective of t h s work is to establish a 3D graphcs model of an object when it is rotated. For an object with rich texture, one can use shape from motion in model recovering [3,4]. For objects with only convex surfaces, shape from contour method is applicable [l]. In this paper, we deal with smooth surfaces with specular reflectance which may yield hghlights. A hghlight has been considered as a noise of surface features and some effort has been made to eliminate it from the object surface. However, in the case where few texture appears on a smooth surface, highlights plays an important rule in perceiving a shape. In order to recover a complete model, we put the object on a turn table and rotate it so that it reveals all of its surfaces to camera. The rotation is readable. The direction of a static illuminant is known by a simple calibration. The issue becomes shape from shadmg and known rotation. To simplify the problem, we use orthogonal projection and parallel illumination. At present, we assume objects have smooth surfaces. The motion information is taken from Epipolar-Plane Images parallel to the rotation plane. Two lunds of visual features ----- fixed features either from comer points or texture points, and hghlights are considered to use. Among them, the hghlight gradually shift on object surface during the rotation; the motion of it can be tracked in EPIs at different heights. From obtained trajectories, object surfaces can be calculated. It is not difficult to realize that normal can be determined from hghlights shifting on the surfaces during the rotation. As to the surface itself, we found that a smooth surface can be described by a first-order differential equation which has a unique solution. Combining those fixed features as boundary condition, we can finally reconstruct the model.
Axis
Light
Orthogonal projection
Fig. 1 Image formation geometry of shape from rotation.
As an object rotates, its surface points have their traces in the correspondmg EPIs as sinusoidal curves of half visible period, even they are not distinct enough to be tracked. If the component of a surface normal in the rotation plane is discontinuous at a point, the shadings on
In the following, w e will introduce some qualitative
characteristics of highlight trajectory and the principle of shape recovery scheme. Then we describe how to extract
33 1
1051-4651/94$04.00 0 1994 IEEE
Authorized licensed use limited to: IUPUI. Downloaded on June 03,2010 at 18:05:15 UTC from IEEE Xplore. Restrictions apply.
its both sides are different and its trace in EPI appears as an edge (typically a segment of sinusoidal curve). If the albedo has a discontinuity at a point, the point also draws a sinusoidal trace of edge in the EPI. These two kinds of points are called fixed points since, for such a point, multiple lines of sight through its projections in dfferent images cross at the same 3D position.
I
/
Rotational axis
connecting their hghlight trajectories in EPI. Figure 4 shows two types of shapes, for instance, whch are convexconcave-convex and comer-linear-convex combinations, respectively. In the first case, hghlight A moves on the first convex surface. In the meanwhile, point B with zero curvature (where shape changes from concave to convex) becomes a highlight point. It splits into two highlight points C and D that move on convex and concave shapes separately. Point D then merges with hghlight point A at another zero curvature point E and disappears. Thls splitting and merging process can be observed from trajectories of hghlights in the correspondmg EPI. We can assert that trajectories of concave and convex shapes should have a smooth connection. The tangent of trajectory is horizontal at the connecting points. T h s is because splitting and merging points are zero curvature as a short linear segment which has a horizontal trajectory in EPI. Similarly, we can qualitatively know the trajectory of highlight for the comer-linear-convex combination (Fig. 4(b)). It is a sinusoidal curve A followed by a horizontal segment AB, and then a trace of convex shape. At the corner point (an extreme case of convex shape), the hghlight stays constant.
illumination direction
Image fame
/
Fig. 2 One cross section of an object parallel to the
shape & highlight
rotation plane.
2.2 Qualitative Motion of Highlights Let us first qualitatively look at the motion types of hghlights according to shapes such as comer, convex, linear, concave, etc. Based on analysis of highlight's trajectories, we can qualitatively infer the shapes. If the surface normal and surface albedo are continuous at a point, matching its projections in continuous images is no longer possible as what has been done for fixed points [1,2,6]. An alternative way is to look at the shading. As an object rotates, its surface elements, in turn, face to the illuminant. A highlight, determined by the surface normal, shlft on surfaces and its trace is possible to be located with respect to the rotation angle. According to curvatures along object boundary on a horizontal cross section, shapes are categorized as either of convex corner, convex, linear, concave. We hence find an interesting effect that hghlights have some basic types of trajectories over the traces of surface points in the correspondmg EpipolarPlane Image. Figure 3 shows hghlight moves on the surfaces and correspondlng trajectories over traces of surface points. On a convex shape, for example, the highlight moves relatively in the inverse direction of the rotation, which yields its image velocity lower than that of surface points (see its trace at right). A linear boundary, however, has its points face the light direction at the same time, which generates a horizontal stripe of highlights in the EPI. Further, a concave shape has its highlight moving in the same direction of rotation; its image velocity is hgher than that of surface points. At a corner, hghlight does not appear if the corner is strictly a zero curvature. Combining dlfferent surfaces together corresponds to
t r a c e of highlight ir !PI
comer C=oO
I convex c>o
fixed c)
$J I o \
plane
c=o
:oncave C d
0
I
U \
% o \
position of highlight i ~ u " a t i o n direction
0
direction
h trace of hiphdight
s trace of surface points
Fig. 3 Trajectories of highhghts over traces of surface points of Mferent shapes.
As a result, a highlight shifts on object surfaces and passes all surface points at least one time if no serious 332
Authorized licensed use limited to: IUPUI. Downloaded on June 03,2010 at 18:05:15 UTC from IEEE Xplore. Restrictions apply.
occlusion of light occurs. Generated from either fixed or shifting points on object surfaces, a queue of connected trajectories in EPI acrosses trajectories of all surface points in one period of rotation. We hence attempt to compute positions of all surface points from this queue. Curvaturesof shapes r>n -. .
equation from the camera geometry. ~ ( 0=) x H = X(0) cos0 + Z(0) sin0 (4) Assuming the shape is not linear, the corresponding highlight trace is then not horizontal and we can take derivative of (4) with respect to 0 to obtain
3
x'(0) = Xe'cosO+Ze'sinO - Xsine +Zcose (9 The first two terms express a possible shift of H on the surface and (X'e, Z'e) gwes the tangent direction of the boundary. In fact, the tangent direction at the hghlight point dlvides half the angle between camera viewing &ration and the drection of illumination in the rotational plane (incident angle equals to reflecting angle). Therefore, the surface normal dlrects to the angle 0+(p0/2-~/2 in the object centered coordinate system and we obtain equation
Rotyc
