Measurement of Shape and Refractive Index of Transparent Object
Measurement of Shape and Refractive Index of Transparent Object Yoshitsugu Manabe, Miki Tsujita, and Kunihiro Chihara Nara Institute of Science and Technology 8916-5 Takayama, Ikoma, Nara, JAPAN fmanabe, [email protected] Abstract When digitalizing characteristics of objects, shapes of objects is very important information. However the measurement methods of transparent objects are few. This paper proposes a new measurement method of a transparent object by changing background patterns. A silhouette of the transparent object is extracted using the change of the background patterns according to refraction, and the shape is measured with the use of the silhouette. Furthermore, the refractive index of the transparent object is estimated by changing background patterns and the measured shape.
1. Introduction Recently, the representation methods of the transparent objects based on image-based rendering are proposed for higher reality [6]. The method based can represent the transparent objects like a photograph, because this method uses acquired images from many view points while changing lighting condition. However this method only reproduces the transparent object at determined view points based on acquired images. As the solution of this problem, the characteristics of the transparent objects like a shape or refractive index should be measured. In general, there are two measurement methods of the shape of object. They are contact-type and non-contact-type methods. The contact-type method uses a contact probe to measures the shape of the object [2]. On the other hand, the non-contact-type method almost measures the shape of the object using diffuse reflection of the object surface, for example, shape-from-shading [1], Photometric stereo [3], Moire topography [5], and spot , slit or pattern projection method. These methods, however, cannot measure transparent objects like a crystal, because transparent objects cannot observe diffuse reflection. Currently, as a method of shape measurement of the transparent objects, Miyazaki et. al. have proposed a measurement method based on analyzing degree of polarization
of surface reflection in visible and far infrared wavelengths, respectively [7]. However this method cannot measure a shape of object if its refractive index is unknown. This paper proposes a measurement method of shapes and refractive index of transparent objects with convex shape and uniform refractive index. This method measures the shape of transparent objects using the silhouette method, and it estimates the reflective index of the transparent objects using changing background patterns by refraction.
2. Approaches Existing non-contact-type measuring methods cannot almost measure the shape of transparent objects without diffuse reflection. Therefore our method uses a silhouette of the transparent object. The silhouette method using difference from background can measure the shape of transparent objects. As difference from background, background patterns by means of refraction of transparent objects are used, so this method can measure a transparent object that is unknown material. The refractive index of transparent object is determined using light passes and the shape of transparent object. The light pass in transparent object is extracted by means of investigating an appeared point on the surface that move from a background point by passing transparent object.
2.1. Measurement of Shape Shape from silhouette is a method of shape measurement based on a sequence of silhouette taken the object from multiple viewpoints [9]. The image of the object is acquired under a viewpoint, and a silhouette of the object is extracted from the acquired image. The silhouettes are extracted from the acquired image with different viewpoints, then the shape is reconstructed from these silhouettes of the object. However for transparent object, it is difficult to extract the silhouette of the object for uniform background. Therefore our proposed method uses patterns on background.
0-7695-2128-2/04 $20.00 (C) 2004 IEEE
A
n1
Background
θ1i p1
P1 −n1
θ1o θ 2o
p2
n2
θ 2i
−n2
p3
A’
P2 Image Plane
Figure 1. Corresponding Points When setting a transparent object in front of patterns, patterns change along silhouette of the transparent object by refraction. The silhouettes of transparent object are extracted by observing the difference between changing patterns and background patterns. While rotating the transparent object, each silhouette of the transparent object is extracted, and the shape of the transparent object is measured by integrating these silhouettes.
Figure 2. Light Path in Transparent Object
2.2. Estimation of Refractive Index Kazama has proposed an estimation method of the refractive index of transparent object by extracting light passes using corresponding points between background and measured image (figure 1) [4]. This method assumed the shape of the transparent object was known, and the refractive index was estimated recursively. If the light is parallel projection, the refractive index can be determined uniquely by light passes and the reconstructed shape. Figure 2 shows schematic of light pass from background to plane of camera. A is the corresponding point of background image, P1 is the intersection point of the transparent object’s surface and starting light ray from A, and P2 is the point passing light ray through the transparent object. Then the normal vector at P1 is denoted as n1 , and the incident angle at P1 is denoted as 1i . Moreover p1 and p2 are unit vetor from P1 to A and from P1 to P2 , respectively. The incident angle 1i is represented as
1 = cos01 (n1 p1 ) and the refractive angle 1 is represented as
(1)
1
i
o
1 = cos01 ( n1 ) p2 : (2) The refractive index n is calculated following Snell’s law using the incident angle 1 and refractive angle 1 as sin 1 n = sin (3) 1 Moreover the refractive index can be calculated using 2 and 2 same as above process. f 0
o
1
i
g
o
i
Figure 3. Flow of Shape Measurement
3. Measurement Method of Shape and Refractive Index Our proposed method uses the movement of a point to a position in measured image from a position in background image by refraction. To measure the shape and the refractive index of the transparent object effectively, patterns on a background is very important. This method uses twodimensional coded images mesured by horizontal and vertical 8bit graycode patterns[8] as the background patterns. The background images and the measured images are acquired in front of the graycode patterns, and the measurement space is encoded. The moving point is found by the two-dimensional coded images using vertical and horizontal graycode patterns.
3.1. Shape
o
i
o
Figure 3 shows process of the shape measurement from silhouettes. The silhouette is extracted using the difference
0-7695-2128-2/04 $20.00 (C) 2004 IEEE
Figure 4. Estimation of Refractive Index from Pattern Change
of the background and the measured images. The transparent object is rotated with the use of the turntable in order to measure whole object. When the transparent object turns around, the shape of the transparent object is measured from integrating these extracted silhouettes.
Figure 5. Extracted Silhouette using Space Coded Images
3.2. Refractive Index Figure 4 shows the estimation process of the refractive index. A point of the background appears a different position from the position of the background by the refraction. These points are measured as points with same horizontal and vertical code in the gackground and the measured images. The refractive index is estimated using positions calculated by these corresponding points and the measured shape.
4. Experiment This paper has measured a spherical and a torso object of acrylic to show effectiveness of our proposed method. The experiment system consists of a turntable, a liquid crystal display and a CCD camera. The turntable is set in front of a liquid crystal display. The liquid crystal display is used to display the graycode patterns as the background. Furthermore the CCD camera is set in front of the turntable. The transparent object is set on the turntable. The rotating step is 10 degrees, so the silhouettes of the transparent object are measured at 36 positions.
4.1. Results Figure 5 (a) shows background images, and (b) shows measured images on a rotating angle. (c) and (d) are space
Figure 6. 3D Reconstracted Image of Sphere
coded images created by (a) and (b), respectively. Then (e) is a silhouette image of the transparent object using the difference between (c) and (d). Figure 6 is a reconstructed shape of the spherical transparent object from the extracted silhouette. This result represents shape of the spherical transparent object. Next, the refractive index of transparent object is calculated by our method. Figure 7 shows corresponding points that were found in space coded background and measured image. The refractive indexes were calculated using the measured shape and the corresponding points (table 1). Code numbers in table 1 are position of corresponding points found from two-dimensional coded images in figure 7, and refractive indexs at P1 and P2 are caluculated . Our proposed method can measure the shape and etimate the refractive index of the transparent object.
0-7695-2128-2/04 $20.00 (C) 2004 IEEE
Figure 8. Result for Acrylic Torso
5. Conclusion
Figure 7. Corresponding Points
Table 1. Estimation Results of Sphere Code No. 1 2 3 4
(141; 83) (193; 91) (171; 79) (165; 81)
at P1
2:13 1:45 1:75 2:24
at P2
2:12 2:61 2:60 2:28
Figure 8 and table 2 are results of a acrylic torso. The results of shape measurement for sphere and torso become rough because the rotating step of turntable was 10 degree. However the reconstructed results are correctly. And the results of estimation of refractive indexes don’t become constant for the each target. The refractive index of general acrylic is about 1:5. Almost estimation results are large a littel form this value. This is considered that the estimated accuracy of normal vectors of the surface depend on.
Table 2. Estimation Results of Torso Code No.
1 2 3
(193; 168) (206; 182) (182; 167)
at P1
1:82 0:15 1:64
at P2
1:82 1:20 1:64
In this paper, we proposed a method measuring the shape and the refractive index of transparent objects by changing background patterns. Space coded images of background and measured images were created using 8bit graycode patterns, respectively. And a silhouette of a transparent object was extracted from space coded images. Then the shape of a spherical transparent object was reconstructed using extracted object silhouette, and the refractive index was estimated .
References [1] B. K. P. Horn and M. Brooks. Shape from Shading. MIT Press, 1989. [2] http://www.rolanddg.com/products/pix30and4.html. [3] K. Ikeuchi. Determining surface orientations of specular surfaces by using the photometric stereo method. IEEE Trans. Pattern Analysis and Machine Intelligence, PAMI-3(6):661– 669, 1981. [4] N. Kazama. Mesurement of optical properties and method of rendering for transparent objects. Master’s thesis, NAIST Master’s Thesis, 1999. (in Japanese). [5] C. Lu and S. Inokuchi. Intensity-modulated moir´e topography. APPLIED OPTICS, 38(19):4019–4029, 1999. [6] W. Matusik, H. Pfister, R. Ziegler, A. Ngan, and L. McMillan. Acquisition and rendering of transparent and refractive objects. Thirteen Eurographics Workshop on Rendering, pages 267–277, 2002. [7] D. Miyazaki, M. Saito, Y. Sato, and K. Ikeuchi. Determining surface orientations of transparent objects by use of polarization degrees in visible and infrared wavelengths. Journal of the Optical Society of America A, 19(4):687–694, 2002. [8] K. Sato and S. Inokuchi. Three-dimensional surface measurement by space encoding range imaging. Journal of Robotic Systems, 2(1):27–39, 1985. [9] S. Tosovic, R. Sablatnig, and M.Kampel. On combining shape from silhouette and shape from structured light. 1st International Symposium on 3D Data Processing Visualization and Transmission, pages 108–118, 2002.
0-7695-2128-2/04 $20.00 (C) 2004 IEEE
1. Introduction Recently, the representation methods of the transparent objects based on image-based rendering are proposed for higher reality [6]. The method based can represent the transparent objects like a photograph, because this method uses acquired images from many view points while changing lighting condition. However this method only reproduces the transparent object at determined view points based on acquired images. As the solution of this problem, the characteristics of the transparent objects like a shape or refractive index should be measured. In general, there are two measurement methods of the shape of object. They are contact-type and non-contact-type methods. The contact-type method uses a contact probe to measures the shape of the object [2]. On the other hand, the non-contact-type method almost measures the shape of the object using diffuse reflection of the object surface, for example, shape-from-shading [1], Photometric stereo [3], Moire topography [5], and spot , slit or pattern projection method. These methods, however, cannot measure transparent objects like a crystal, because transparent objects cannot observe diffuse reflection. Currently, as a method of shape measurement of the transparent objects, Miyazaki et. al. have proposed a measurement method based on analyzing degree of polarization
of surface reflection in visible and far infrared wavelengths, respectively [7]. However this method cannot measure a shape of object if its refractive index is unknown. This paper proposes a measurement method of shapes and refractive index of transparent objects with convex shape and uniform refractive index. This method measures the shape of transparent objects using the silhouette method, and it estimates the reflective index of the transparent objects using changing background patterns by refraction.
2. Approaches Existing non-contact-type measuring methods cannot almost measure the shape of transparent objects without diffuse reflection. Therefore our method uses a silhouette of the transparent object. The silhouette method using difference from background can measure the shape of transparent objects. As difference from background, background patterns by means of refraction of transparent objects are used, so this method can measure a transparent object that is unknown material. The refractive index of transparent object is determined using light passes and the shape of transparent object. The light pass in transparent object is extracted by means of investigating an appeared point on the surface that move from a background point by passing transparent object.
2.1. Measurement of Shape Shape from silhouette is a method of shape measurement based on a sequence of silhouette taken the object from multiple viewpoints [9]. The image of the object is acquired under a viewpoint, and a silhouette of the object is extracted from the acquired image. The silhouettes are extracted from the acquired image with different viewpoints, then the shape is reconstructed from these silhouettes of the object. However for transparent object, it is difficult to extract the silhouette of the object for uniform background. Therefore our proposed method uses patterns on background.
0-7695-2128-2/04 $20.00 (C) 2004 IEEE
A
n1
Background
θ1i p1
P1 −n1
θ1o θ 2o
p2
n2
θ 2i
−n2
p3
A’
P2 Image Plane
Figure 1. Corresponding Points When setting a transparent object in front of patterns, patterns change along silhouette of the transparent object by refraction. The silhouettes of transparent object are extracted by observing the difference between changing patterns and background patterns. While rotating the transparent object, each silhouette of the transparent object is extracted, and the shape of the transparent object is measured by integrating these silhouettes.
Figure 2. Light Path in Transparent Object
2.2. Estimation of Refractive Index Kazama has proposed an estimation method of the refractive index of transparent object by extracting light passes using corresponding points between background and measured image (figure 1) [4]. This method assumed the shape of the transparent object was known, and the refractive index was estimated recursively. If the light is parallel projection, the refractive index can be determined uniquely by light passes and the reconstructed shape. Figure 2 shows schematic of light pass from background to plane of camera. A is the corresponding point of background image, P1 is the intersection point of the transparent object’s surface and starting light ray from A, and P2 is the point passing light ray through the transparent object. Then the normal vector at P1 is denoted as n1 , and the incident angle at P1 is denoted as 1i . Moreover p1 and p2 are unit vetor from P1 to A and from P1 to P2 , respectively. The incident angle 1i is represented as
1 = cos01 (n1 p1 ) and the refractive angle 1 is represented as
(1)
1
i
o
1 = cos01 ( n1 ) p2 : (2) The refractive index n is calculated following Snell’s law using the incident angle 1 and refractive angle 1 as sin 1 n = sin (3) 1 Moreover the refractive index can be calculated using 2 and 2 same as above process. f 0
o
1
i
g
o
i
Figure 3. Flow of Shape Measurement
3. Measurement Method of Shape and Refractive Index Our proposed method uses the movement of a point to a position in measured image from a position in background image by refraction. To measure the shape and the refractive index of the transparent object effectively, patterns on a background is very important. This method uses twodimensional coded images mesured by horizontal and vertical 8bit graycode patterns[8] as the background patterns. The background images and the measured images are acquired in front of the graycode patterns, and the measurement space is encoded. The moving point is found by the two-dimensional coded images using vertical and horizontal graycode patterns.
3.1. Shape
o
i
o
Figure 3 shows process of the shape measurement from silhouettes. The silhouette is extracted using the difference
0-7695-2128-2/04 $20.00 (C) 2004 IEEE
Figure 4. Estimation of Refractive Index from Pattern Change
of the background and the measured images. The transparent object is rotated with the use of the turntable in order to measure whole object. When the transparent object turns around, the shape of the transparent object is measured from integrating these extracted silhouettes.
Figure 5. Extracted Silhouette using Space Coded Images
3.2. Refractive Index Figure 4 shows the estimation process of the refractive index. A point of the background appears a different position from the position of the background by the refraction. These points are measured as points with same horizontal and vertical code in the gackground and the measured images. The refractive index is estimated using positions calculated by these corresponding points and the measured shape.
4. Experiment This paper has measured a spherical and a torso object of acrylic to show effectiveness of our proposed method. The experiment system consists of a turntable, a liquid crystal display and a CCD camera. The turntable is set in front of a liquid crystal display. The liquid crystal display is used to display the graycode patterns as the background. Furthermore the CCD camera is set in front of the turntable. The transparent object is set on the turntable. The rotating step is 10 degrees, so the silhouettes of the transparent object are measured at 36 positions.
4.1. Results Figure 5 (a) shows background images, and (b) shows measured images on a rotating angle. (c) and (d) are space
Figure 6. 3D Reconstracted Image of Sphere
coded images created by (a) and (b), respectively. Then (e) is a silhouette image of the transparent object using the difference between (c) and (d). Figure 6 is a reconstructed shape of the spherical transparent object from the extracted silhouette. This result represents shape of the spherical transparent object. Next, the refractive index of transparent object is calculated by our method. Figure 7 shows corresponding points that were found in space coded background and measured image. The refractive indexes were calculated using the measured shape and the corresponding points (table 1). Code numbers in table 1 are position of corresponding points found from two-dimensional coded images in figure 7, and refractive indexs at P1 and P2 are caluculated . Our proposed method can measure the shape and etimate the refractive index of the transparent object.
0-7695-2128-2/04 $20.00 (C) 2004 IEEE
Figure 8. Result for Acrylic Torso
5. Conclusion
Figure 7. Corresponding Points
Table 1. Estimation Results of Sphere Code No. 1 2 3 4
(141; 83) (193; 91) (171; 79) (165; 81)
at P1
2:13 1:45 1:75 2:24
at P2
2:12 2:61 2:60 2:28
Figure 8 and table 2 are results of a acrylic torso. The results of shape measurement for sphere and torso become rough because the rotating step of turntable was 10 degree. However the reconstructed results are correctly. And the results of estimation of refractive indexes don’t become constant for the each target. The refractive index of general acrylic is about 1:5. Almost estimation results are large a littel form this value. This is considered that the estimated accuracy of normal vectors of the surface depend on.
Table 2. Estimation Results of Torso Code No.
1 2 3
(193; 168) (206; 182) (182; 167)
at P1
1:82 0:15 1:64
at P2
1:82 1:20 1:64
In this paper, we proposed a method measuring the shape and the refractive index of transparent objects by changing background patterns. Space coded images of background and measured images were created using 8bit graycode patterns, respectively. And a silhouette of a transparent object was extracted from space coded images. Then the shape of a spherical transparent object was reconstructed using extracted object silhouette, and the refractive index was estimated .
References [1] B. K. P. Horn and M. Brooks. Shape from Shading. MIT Press, 1989. [2] http://www.rolanddg.com/products/pix30and4.html. [3] K. Ikeuchi. Determining surface orientations of specular surfaces by using the photometric stereo method. IEEE Trans. Pattern Analysis and Machine Intelligence, PAMI-3(6):661– 669, 1981. [4] N. Kazama. Mesurement of optical properties and method of rendering for transparent objects. Master’s thesis, NAIST Master’s Thesis, 1999. (in Japanese). [5] C. Lu and S. Inokuchi. Intensity-modulated moir´e topography. APPLIED OPTICS, 38(19):4019–4029, 1999. [6] W. Matusik, H. Pfister, R. Ziegler, A. Ngan, and L. McMillan. Acquisition and rendering of transparent and refractive objects. Thirteen Eurographics Workshop on Rendering, pages 267–277, 2002. [7] D. Miyazaki, M. Saito, Y. Sato, and K. Ikeuchi. Determining surface orientations of transparent objects by use of polarization degrees in visible and infrared wavelengths. Journal of the Optical Society of America A, 19(4):687–694, 2002. [8] K. Sato and S. Inokuchi. Three-dimensional surface measurement by space encoding range imaging. Journal of Robotic Systems, 2(1):27–39, 1985. [9] S. Tosovic, R. Sablatnig, and M.Kampel. On combining shape from silhouette and shape from structured light. 1st International Symposium on 3D Data Processing Visualization and Transmission, pages 108–118, 2002.
0-7695-2128-2/04 $20.00 (C) 2004 IEEE
