Towards Accurate and Efficient Representation of Image ... - CVIP Lab
Towards Accurate and Efficient Representation of Image Irradiance of Convex-Lambertian Objects Under Unknown Near Lighting Shireen Y. Elhabian, Ham Rara and Aly A. Farag University of Louisville, CVIP Lab Louisville, KY, USA syelha01,[email protected], [email protected]
Abstract
Belhumer and Kriegman [2] for Lambertian objects under distant illumination. On the other hand, Yuille et al. [3] used a human face and other objects to construct principal component analysis (PCA) on images acquired by a distant point light source moving on the locus of a sphere surrounding the object. They empirically illustrated that image variations due to illumination of Lambertian objects can be explained by the first five principal components. Thus linear subspaces can be used representing a certain class of images, capturing variations due to different imaging conditions [1][4][5]. As outlined by Lee et al. [6], this subspace can be obtained using three different ways; (1) performing PCA on a large set of images of object(s) of interest under different imaging conditions, (2) using 3D models, image formation process can be simulated to render synthetic images under assumed imaging conditions, while PCA is again used to compute the required subspace, or (3) assuming certain surface reflectance distribution function, the harmonic expansion of the lighting function can be used to derive an analytic subspace to approximate images under fixed pose but different illumination conditions. Basri and Jacobs [5] and Ramamoorthi [7] formulated the image irradiance equation [8] in a convolution framework where the lighting function acts as a signal filtered by the Lambertian kernel. They provided an analytical expression of an image of a convex-Lambertian object illuminated by distant lighting. They proved that the Lambertian kernel acts as a low pass filter, presenting the image irradiance as a band-limited signal being represented by a finite number of basis functions. Frolova et al. [9] extended the formulation presented in [5] to a simplified near light model. Both [5] and [7] used spherical harmonics (SH) to represent the surface reflectance function in the frequency domain. While Basri and Jacobs [5] formulated this process in global coordinates with respect to global reference frame, Ramamoorthi [7] made the distinction between such a formalization in global and local (with respect to surface points) coordinates. In this paper, we consider the image irradiance in global coordinates. Harmonic basis, in general, are known to provide opti-
Surface irradiance signals are turned into outgoing radiance through the surface reflectance function, which can be significantly perturbed by the illumination conditions. Due to their low-frequency nature, irradiance signals can be represented using low-order basis functions, where spherical harmonics (SH) have been extensively used to provide such basis. When capturing image irradiance from a single viewpoint, the visible part of the object’s surface constructs the upper hemisphere of the surface normals where the SH are no longer orthonormal. This reduced domain paves the way for even lower-dimensional approximation since full spherical representation is not needed. While harmonic basis are known to be optimal under distant light, light coming from near-by objects and indoor environments are common near light scenarios; it is essential to relax distant light assumption. Considering light source(s) distributed uniformly over the upper hemisphere, we propose the use of hemispherical harmonics (HSH) to model image irradiance of convex Lambertian objects perceived from single viewpoint under unknown near illumination. We prove analytically, and experimentally validated, that the Lambertian kernel has a more compact harmonic expansion in the hemispherical domain when compared to its spherical counterpart. We illustrate that HSH provide an efficient and accurate lowdimensional representation of image irradiance of Lambertian objects under near lighting conditions in contrast to SH.
1. Introduction The appearance of an object under fixed pose depends primarily on its geometrical structure (shape), reflectance properties (material) and illumination. Low-dimensional representation of image irradiance under unknown arbitrary lighting is a fundamental process for many computer vision tasks including shape recovery and object recognition. Theoretically, the space of all possible images under all illumination conditions is infinite dimensional since the lighting function is an arbitrary function [1], this was validated by 1
Figure 1. (Right) Considering a single viewpoint, the upper hemisphere of surface normals are visible instead of the whole sphere of surface orientations. The spherical harmonics basis functions are no longer orthonormal on this reduced domain, introducing the need for hemispherical basis for image irradiance representation. (Left) Visualization of up to 3rd order hemispherical harmonic basis. We use the mean shape and albedo from USF HumanID 3D face database [17]. Note that the zeroth order basis shows the DC component which describes the appearance of the object under ambient lighting. Explicit formulas for such basis are listed in (3).
mal basis for images of Lambertian objects under distant light [9]. Nonetheless, light coming from close by objects, indoor environments and reflections from surrounding surfaces are examples of common near light scenarios, thus it is crucial to relax the distant light assumption. On the other hand, in a single viewpoint, the visible part of the object’s surface constructs the upper hemisphere of the surface normals where the SH are no longer orthonormal (see Figure 1 for illustration). This reduced domain paves the way for even lower-dimensional approximation since full spherical representation is not needed. Ramamoorthi [1], under the assumption of distant light, linearly combined the SH basis to create a new set of orthonormal eigenfunctions over this restricted domain. Nonetheless, the question remains; how to efficiently and accurately represent image irradiance in this hemispherical domain, where hemispherical functions present discontinuities at the boundary of the hemisphere when represented in the spherical domain [10], demanding more coefficients for accurate representation. In this paper, considering light source(s) distributed uniformly over the upper hemisphere, we propose the use of hemispherical harmonics (HSH) to model the image irradiance of convex Lambertian objects perceived from a single viewpoint under unknown near illumination. We formulate and prove the addition theorem for HSH in order to provide an analytical expression of the image irradiance equation in the hemispherical domain. We present both analytical and empirical justification of lowdimensional representation of near lighting conditions in
this reduced domain compared to the spherical one. We provide a closed form for the HSH expansion of the Lambertian kernel under near lighting, in addition to that of the total squared energy of such a kernel as a function of the distance to the light source, this allows the computation of the approximation accuracy of HSH expansion as a function of the harmonic order. Our experiments illustrate that, despite of having poor approximation accuracy under very close lights, such behavior improves exponentially with little increase in the distance to the light source relative to the object size. The rest of the paper is organized as follows: Section 2 discusses the hemispherical harmonics, presenting the function approximation accuracy. Section 3 introduces the hemispherical harmonics representation for image irradiance equation. Section 4 address the near light illumination model. Later sections deal with the experimental results and conclusions.
2. Hemispherical Harmonics Several hemispherical basis have been proposed in literature to represent hemispherical functions. Sloan et al. [11] used SH to represent an even-reflected (about xy -plane) version of a hemispherical function. Coefficients were found using least squares SH, however this leads to non-zero values in the lower hemisphere. Koenderink et al. [12] used Zernike polynomials [13], which are basis functions defined on a disk, to build hemispherical basis. Yet, such polynomials have high computational cost. Makhotkin [14] and
Gautron et al. [10] proposed hemispherically orthonomal basis through mapping the negative pole of the sphere to the border of the hemisphere. Such contraction was achieved through shifting the adjoint Jacobi polynomials [14] and the associated Legendre polynomials [10] without affecting the orthogonality relationship. Recently, Habel and Wimmer [15] used the SH as an intermediate basis to define polynomial-based hemispherical basis denoted by H-basis. They used the SH basis functions which are symmetric to the z = 0 plane since they are orthogonal over the upper hemisphere. While other basis functions are shifted the same way proposed by [10]. Although such basis definition leads to polynomial basis, this inhibit us from deriving an analytical expression of harmonic expansion of the Lambertian kernel. In this paper, we adopt the hemispherical basis defined by Gautron et al. [10]. Hemispherical basis functions {Hnm (θ, φ)} are defined from the shifted associated Legendre polynomials as follows [10], √ e m em 2Nn Pn (cos θ) cos(mφ) en0 Pen0 (cos θ) Hnm (θ, φ) = N √ m −m en Pen (cos θ) sin(−mφ) 2N
m>0 m=0
(2n + 1)(n − |m|)! 2π(n + |m|)!
H10 =
√1 2π
q
3 (2z 2π
(2)
3 y√ z(1−z) 2π 2 2 √x +y q 3 x√ z(1−z) 2 2π x2 +y 2
o =2 H11
− 1)
e = H11
PN
n=0 R
√ q z(1−z) 5 xyz(1−z) 5 y(2z−1) o = 12 o =6 √ H21 H22 6π x2 +y 2 6π 2 x√ +y 2 q q z(1−z) 5 5 x(2z−1) e =6 √ (6z 2 − 6z + 1) H21 H20 = 2π 6π x2 +y 2 q 2 2 5 z(1−z)(x −y ) e =6 H22 x2 +y 2 q6π y (3x2 −y 2 ) 7 o = 10 H33 (z(1 − z))3/2 2 2 3/2 5π (x +y ) q 7 o = 60 H32 z(2z − 1)(1 − z) x2xy 30π +y 2 q p 14 o =3 2 − 5z + 1) z(1 − z) √ y H31 (5z 3π x2 +y 2 q (3) 7 2 H30 = (2z − 1)(10z − 10z + 1) q2π p 14 e =3 H31 (5z 2 − 5z + 1) z(1 − z) √ 2x 2 3π x +y q x2 −y 2 7 e = 60 H32 z(2z − 1)(1 − z) x2 +y 2 q 30π 2 2 3/2 x(x −3y ) 7 e = 10 H33 (z(1 − z)) 3/2 5π 2 2 (x +y ) q
The hemispherical basis functions form a complete set of basis for hemispherical function approximation over the [0, π/2] × [0, 2π] domain.
|f (u)| du
Pn
Ω+
Using sine and cosine of multiple angles and spherical to cartesian coordinates conversion, up to the 3rd order hemispherical basis functions represented in cartesian coordinates can be written as follows where the superscripts e and o denote the even and odd components of the basis functions(refer to Figure 1 for their visualization). √ q H00 =
Ω+
Considering the N th order function approximation, we are only interested in the first (N + 1)-harmonic terms, thus such approximation captures a certain amount of the function’s total energy defined as the cumulative sum of energies maintained by individual harmonic terms, (this cumulative energy can also be referred to as function approximation accuracy since the more energy the harmonic terms capture from the function to be approximated, the more accurate the harmonic expansion is). Thus the approximation accuracy can be defined as,
m 1). Emphasizing that, for near light and asymptotically distant light, higher order components have smaller contribution to the Lambertian kernel which indicate that the reflectance function resulted from single directional light source can be approximated by lower order harmonics. (2) Comparing the zeroth and first order terms of SH versus that of HSH, it can be observed that as the light source becomes nearer to the object, these terms capture more signal energy in the HSH domain in contrast to that of SH domain. (3) Harmonic terms of higher orders (n > 1) in the hemispherical domain maintain lower energy than that of the spherical domain, reflecting the capability of HSH of presenting a more compact representation to the Lambertian kernel under near and asymptotically distant lighting conditions.
(a)
(b)
Figure 3. Spherical harmonics (a) versus hemispherical harmonics (b) Lambertian kernel approximation accuracy under near light as functions of relative distance to the light source.
Figure 4 shows the behavior of the approximation accuracy (cumulative energy) eN captured by the N th order function approximation (up to the sixth order). It can be inferred that: (1) For a light source at a specific distant to the object, the approximation accuracy increases proportionally with the harmonic order. (2) In case of near and asymptotic distant light source, there is a significant increase in the approximation accuracy of HSH when compared to that of SH, especially when comparing the 1st order HSH approximation accuracy which reaches to 100 % at R/r ≥ 4
(a)
(b)
Figure 4. Spherical harmonics (a) versus hemispherical harmonics (b) Lambertian kernel N th order approximation accuracy under near light as functions of relative distance to the light source.
Figure 5 shows the approximating harmonic order to reach to 99.22% accuracy. It is evident that the order needed to achieve such an accuracy level drops to 1st order in case of HSH, starting at (R/r) = 4. For the SH case, it drops only up to the 4th order, starting at (R/r) = 5.
5. Experimental Results We perform near light simulations with the USF 3D human face database [17] which contains 100 subjects of diverse gender and ethnicity. Each 3D facial surface is represented as a 2-manifold triangular mesh which is rerepresented in terms of Monge patches using orthographic projection. This provides a bijective mapping between surface points and image coordinates. We use forward finite difference to approximate surface derivatives from which surface normals are obtained. In order to quantitatively analyze the performance of the proposed image irradiance representation, we simulate the image irradiance equation by fixing the camera and the surface in position, and illuminating the facial surface using (1) a single light source and (2) multiple light sources located at finite distances. The light directions are randomly sampled using Monte Carlo sampling, where lights are cast with equal probabilities. The light sources are moved from close by to very far ones relative to the size of the object at hand. Experiments are repeated 50 times with randomly sampled light directions to guarantee the statistical significance of our results. For each synthetic irradiance E , the approximated irradiance Eb is compared with the groundtruth E and the mean of the absolute error is computed. Figs. 6(a) and 6(b) show the image irradiance approx-
Figure 5. Approximating order to reach to 99.22% accuracy, as function of relative distance to the light source.
imation performance in terms of mean absolute error, for single and multiple light sources scenarios, as functions of the relative distance to the light source. Experiments are conducted for up to the 3rd order for HSH and SH approximations. Regardless of the harmonic order and the number of light sources, the approximation error decreases rapidly with the distance to the light source and approaches an asymptotic value at very far distances. Comparing the results of single versus multiple light sources, the latter scenario provide lower approximation errors, which agrees with the conclusion presented by Frolova et al. [9] that a single distant light source provide the worst case illumintation for SH approximation. Our results provide an empirical validation for this conclusion which also holds for HSH approximation and near lighting. In all cases, HSH approximation performance provides a lower bound for SH approximation, which emphasizes its power in image irradiance modeling under near lighting conditions.
(a)
(b)
Figure 6. (a) Single, (b) Multiple, light source scenario: The mean of the mean absolute error measured between the ground-truth image irradiance and the approximated ones (using up to 3rd order HSH compared to these of SH) as a function of the relative distance to the light source
6. Conclusion In this paper, we presented an efficient and accurate representation for image irradiance equation in the case of a single viewpoint, where spherical harmonics are no longer orthonormal. We derived the hemispherical harmonic (HSH) basis to model near illumination conditions under the assumption of convex-Lambertian surfaces. We presented analytical and experimental evaluation of HSH performance. Our experiments illustrated the capability of HSH of presenting a more compact representation to the Lambertian kernel under near and asymptotically distant lighting conditions. In terms of harmonic order, it was evident that the order needed to achieve 99.22% accuracy level drops to 1st order in case of HSH as the light distance increases; in comparison, the SH case falls only up to the 4th order, leading to the conclusion that HSH handles close-by lights as distant ones when compared to SH. HSH also proved to have the capability of modeling image irradiance under near lighting better than SH. Ongoing efforts are directed towards modeling more realistic surface BRDF’s where HSH is known to outperform SH in terms of approximation accuracy and efficiency.
References [1] R. Ramamoorthi, Analytic PCA construction for theoretical analysis of lighting variability in images of a Lambertian object, IEEE Trans. Pattern Analysis and Machine Intelligence (PAMI), Vol. 24, No. 10, pp. 1322-1333, Oct. 2002. 1, 2 [2] P. N. Belhumeur, D. J. Kriegman, What is the set of images of an object under all possible lighting conditions?, Int’l Journal of Computer Vision, vol. 28, no. 3, pp. 245-260, 1998. 1 [3] A. Yuille, D. Snow, R. Epstein, and P. Belhumeur, Determining Generative Models of Objects under Varying Illumination: Shape and Albedo from Multiple Images Using SVD and Integrability, Intl J. Computer Vision, vol. 35, no. 3, pp. 203-222, 1999. 1 [4] A. Georghiades, P. Belhumeur, and D. Kriegman, From Few to Many: Generative Models for Recognition under Variable Pose and Illumination, Proc. Fourth Intl Conf. Automatic Face and Guesture Recogniton, pp. 277-284, 2000. 1 [5] R. Basri, D. W. Jacobs, Lambertian reflectance and linear subspaces, Pattern Analysis and Machine Intelligence, IEEE Transactions on, vol.25, no.2, pp. 218- 233, Feb 2003. 1, 3 [6] K-C. Lee, J. Ho, D. Kriegman, Acquiring Linear Subspaces for Face Recognition under Variable Lighting, IEEE Trans. Pattern Analysis and Machine Intelligence (PAMI), Vol. 27, No. 5, pp. 684-698, May 2005. 1 [7] R. Ramamoorthi, A Signal-Processing Framework for Forward and Inverse Rendering, Ph.D. Dissertation. Stanford University, Stanford, CA, USA, Advisor(s) P. Hanrahan. 1 [8] B.K.P. Horn. Shape from Shading, A Method for Obtaining the Shape of a Smooth Opaque Object from One View. PhD thesis, Massachusetts Inst. of Technology, Cambridge, Massachusetts, 1970. 1 [9] D. Frolova, D. Simakov, and R. Basri, Accuracy of spherical harmonic approximations for images of Lambertian objects under far and near lighting, European Conf. on Computer Vision (ECCV-04), Prague, 574-587, 2004. 1, 2, 4, 6 [10] P. Gautron, J. Krivanek, S.N. Pattanaik, and K. Bouatouch, A Novel Hemispherical Basis for Accurate and Efficient Rendering, Proc. Eurographics Symp. Rendering, 2004. 2, 3, 4 [11] P. P. Sloan, J. Hall, J. Hart, J. Snyder, Clustered principal components for precomputed radiance transfer, ACM Trans. Graph, vol.22, no.3, pp. 382-391, 2003. 2 [12] J. Koenderink, A. Van Doorn, M. Stavridi, Bidirectional reflection distribution function expressed in terms of surface scattering modes, in ECCV,pp. 28-39, 1996. 2 [13] J. C. Wyant and K. Creath, Basic wavefront aberration theory for optical metrology, In Applied optics and Optical Engineering, vol XI , Academic Press, Inc., pp. 27.39, 1992. 2 [14] O. A. Makhotkin, Analysis of radiative transfer between surfaces by hemispherical harmonics, Journal of Quantitative Spectroscopy and Radiative Transfer, Vol. 56, no. 6, 869-879, 1996. 2, 3 [15] R. Habel and M. Wimmer, Efficient Irradiance Normal Mapping, Proceedings of the 2010 ACM SIGGRAPH symposium on Interactive 3D Graphics and Games, pp. 189-195, 2010. 3 [16] P. K. Kythe and M. R. Schaferkotter, Handbook of Computational Methods for Integration, Chapman & Hall/CRC, 2004. 4 [17] S. Sarkar, USF HumanID 3D Face Database,University of South Florida, Tampa, FL, USA. 2, 5
Abstract
Belhumer and Kriegman [2] for Lambertian objects under distant illumination. On the other hand, Yuille et al. [3] used a human face and other objects to construct principal component analysis (PCA) on images acquired by a distant point light source moving on the locus of a sphere surrounding the object. They empirically illustrated that image variations due to illumination of Lambertian objects can be explained by the first five principal components. Thus linear subspaces can be used representing a certain class of images, capturing variations due to different imaging conditions [1][4][5]. As outlined by Lee et al. [6], this subspace can be obtained using three different ways; (1) performing PCA on a large set of images of object(s) of interest under different imaging conditions, (2) using 3D models, image formation process can be simulated to render synthetic images under assumed imaging conditions, while PCA is again used to compute the required subspace, or (3) assuming certain surface reflectance distribution function, the harmonic expansion of the lighting function can be used to derive an analytic subspace to approximate images under fixed pose but different illumination conditions. Basri and Jacobs [5] and Ramamoorthi [7] formulated the image irradiance equation [8] in a convolution framework where the lighting function acts as a signal filtered by the Lambertian kernel. They provided an analytical expression of an image of a convex-Lambertian object illuminated by distant lighting. They proved that the Lambertian kernel acts as a low pass filter, presenting the image irradiance as a band-limited signal being represented by a finite number of basis functions. Frolova et al. [9] extended the formulation presented in [5] to a simplified near light model. Both [5] and [7] used spherical harmonics (SH) to represent the surface reflectance function in the frequency domain. While Basri and Jacobs [5] formulated this process in global coordinates with respect to global reference frame, Ramamoorthi [7] made the distinction between such a formalization in global and local (with respect to surface points) coordinates. In this paper, we consider the image irradiance in global coordinates. Harmonic basis, in general, are known to provide opti-
Surface irradiance signals are turned into outgoing radiance through the surface reflectance function, which can be significantly perturbed by the illumination conditions. Due to their low-frequency nature, irradiance signals can be represented using low-order basis functions, where spherical harmonics (SH) have been extensively used to provide such basis. When capturing image irradiance from a single viewpoint, the visible part of the object’s surface constructs the upper hemisphere of the surface normals where the SH are no longer orthonormal. This reduced domain paves the way for even lower-dimensional approximation since full spherical representation is not needed. While harmonic basis are known to be optimal under distant light, light coming from near-by objects and indoor environments are common near light scenarios; it is essential to relax distant light assumption. Considering light source(s) distributed uniformly over the upper hemisphere, we propose the use of hemispherical harmonics (HSH) to model image irradiance of convex Lambertian objects perceived from single viewpoint under unknown near illumination. We prove analytically, and experimentally validated, that the Lambertian kernel has a more compact harmonic expansion in the hemispherical domain when compared to its spherical counterpart. We illustrate that HSH provide an efficient and accurate lowdimensional representation of image irradiance of Lambertian objects under near lighting conditions in contrast to SH.
1. Introduction The appearance of an object under fixed pose depends primarily on its geometrical structure (shape), reflectance properties (material) and illumination. Low-dimensional representation of image irradiance under unknown arbitrary lighting is a fundamental process for many computer vision tasks including shape recovery and object recognition. Theoretically, the space of all possible images under all illumination conditions is infinite dimensional since the lighting function is an arbitrary function [1], this was validated by 1
Figure 1. (Right) Considering a single viewpoint, the upper hemisphere of surface normals are visible instead of the whole sphere of surface orientations. The spherical harmonics basis functions are no longer orthonormal on this reduced domain, introducing the need for hemispherical basis for image irradiance representation. (Left) Visualization of up to 3rd order hemispherical harmonic basis. We use the mean shape and albedo from USF HumanID 3D face database [17]. Note that the zeroth order basis shows the DC component which describes the appearance of the object under ambient lighting. Explicit formulas for such basis are listed in (3).
mal basis for images of Lambertian objects under distant light [9]. Nonetheless, light coming from close by objects, indoor environments and reflections from surrounding surfaces are examples of common near light scenarios, thus it is crucial to relax the distant light assumption. On the other hand, in a single viewpoint, the visible part of the object’s surface constructs the upper hemisphere of the surface normals where the SH are no longer orthonormal (see Figure 1 for illustration). This reduced domain paves the way for even lower-dimensional approximation since full spherical representation is not needed. Ramamoorthi [1], under the assumption of distant light, linearly combined the SH basis to create a new set of orthonormal eigenfunctions over this restricted domain. Nonetheless, the question remains; how to efficiently and accurately represent image irradiance in this hemispherical domain, where hemispherical functions present discontinuities at the boundary of the hemisphere when represented in the spherical domain [10], demanding more coefficients for accurate representation. In this paper, considering light source(s) distributed uniformly over the upper hemisphere, we propose the use of hemispherical harmonics (HSH) to model the image irradiance of convex Lambertian objects perceived from a single viewpoint under unknown near illumination. We formulate and prove the addition theorem for HSH in order to provide an analytical expression of the image irradiance equation in the hemispherical domain. We present both analytical and empirical justification of lowdimensional representation of near lighting conditions in
this reduced domain compared to the spherical one. We provide a closed form for the HSH expansion of the Lambertian kernel under near lighting, in addition to that of the total squared energy of such a kernel as a function of the distance to the light source, this allows the computation of the approximation accuracy of HSH expansion as a function of the harmonic order. Our experiments illustrate that, despite of having poor approximation accuracy under very close lights, such behavior improves exponentially with little increase in the distance to the light source relative to the object size. The rest of the paper is organized as follows: Section 2 discusses the hemispherical harmonics, presenting the function approximation accuracy. Section 3 introduces the hemispherical harmonics representation for image irradiance equation. Section 4 address the near light illumination model. Later sections deal with the experimental results and conclusions.
2. Hemispherical Harmonics Several hemispherical basis have been proposed in literature to represent hemispherical functions. Sloan et al. [11] used SH to represent an even-reflected (about xy -plane) version of a hemispherical function. Coefficients were found using least squares SH, however this leads to non-zero values in the lower hemisphere. Koenderink et al. [12] used Zernike polynomials [13], which are basis functions defined on a disk, to build hemispherical basis. Yet, such polynomials have high computational cost. Makhotkin [14] and
Gautron et al. [10] proposed hemispherically orthonomal basis through mapping the negative pole of the sphere to the border of the hemisphere. Such contraction was achieved through shifting the adjoint Jacobi polynomials [14] and the associated Legendre polynomials [10] without affecting the orthogonality relationship. Recently, Habel and Wimmer [15] used the SH as an intermediate basis to define polynomial-based hemispherical basis denoted by H-basis. They used the SH basis functions which are symmetric to the z = 0 plane since they are orthogonal over the upper hemisphere. While other basis functions are shifted the same way proposed by [10]. Although such basis definition leads to polynomial basis, this inhibit us from deriving an analytical expression of harmonic expansion of the Lambertian kernel. In this paper, we adopt the hemispherical basis defined by Gautron et al. [10]. Hemispherical basis functions {Hnm (θ, φ)} are defined from the shifted associated Legendre polynomials as follows [10], √ e m em 2Nn Pn (cos θ) cos(mφ) en0 Pen0 (cos θ) Hnm (θ, φ) = N √ m −m en Pen (cos θ) sin(−mφ) 2N
m>0 m=0
(2n + 1)(n − |m|)! 2π(n + |m|)!
H10 =
√1 2π
q
3 (2z 2π
(2)
3 y√ z(1−z) 2π 2 2 √x +y q 3 x√ z(1−z) 2 2π x2 +y 2
o =2 H11
− 1)
e = H11
PN
n=0 R
√ q z(1−z) 5 xyz(1−z) 5 y(2z−1) o = 12 o =6 √ H21 H22 6π x2 +y 2 6π 2 x√ +y 2 q q z(1−z) 5 5 x(2z−1) e =6 √ (6z 2 − 6z + 1) H21 H20 = 2π 6π x2 +y 2 q 2 2 5 z(1−z)(x −y ) e =6 H22 x2 +y 2 q6π y (3x2 −y 2 ) 7 o = 10 H33 (z(1 − z))3/2 2 2 3/2 5π (x +y ) q 7 o = 60 H32 z(2z − 1)(1 − z) x2xy 30π +y 2 q p 14 o =3 2 − 5z + 1) z(1 − z) √ y H31 (5z 3π x2 +y 2 q (3) 7 2 H30 = (2z − 1)(10z − 10z + 1) q2π p 14 e =3 H31 (5z 2 − 5z + 1) z(1 − z) √ 2x 2 3π x +y q x2 −y 2 7 e = 60 H32 z(2z − 1)(1 − z) x2 +y 2 q 30π 2 2 3/2 x(x −3y ) 7 e = 10 H33 (z(1 − z)) 3/2 5π 2 2 (x +y ) q
The hemispherical basis functions form a complete set of basis for hemispherical function approximation over the [0, π/2] × [0, 2π] domain.
|f (u)| du
Pn
Ω+
Using sine and cosine of multiple angles and spherical to cartesian coordinates conversion, up to the 3rd order hemispherical basis functions represented in cartesian coordinates can be written as follows where the superscripts e and o denote the even and odd components of the basis functions(refer to Figure 1 for their visualization). √ q H00 =
Ω+
Considering the N th order function approximation, we are only interested in the first (N + 1)-harmonic terms, thus such approximation captures a certain amount of the function’s total energy defined as the cumulative sum of energies maintained by individual harmonic terms, (this cumulative energy can also be referred to as function approximation accuracy since the more energy the harmonic terms capture from the function to be approximated, the more accurate the harmonic expansion is). Thus the approximation accuracy can be defined as,
m 1). Emphasizing that, for near light and asymptotically distant light, higher order components have smaller contribution to the Lambertian kernel which indicate that the reflectance function resulted from single directional light source can be approximated by lower order harmonics. (2) Comparing the zeroth and first order terms of SH versus that of HSH, it can be observed that as the light source becomes nearer to the object, these terms capture more signal energy in the HSH domain in contrast to that of SH domain. (3) Harmonic terms of higher orders (n > 1) in the hemispherical domain maintain lower energy than that of the spherical domain, reflecting the capability of HSH of presenting a more compact representation to the Lambertian kernel under near and asymptotically distant lighting conditions.
(a)
(b)
Figure 3. Spherical harmonics (a) versus hemispherical harmonics (b) Lambertian kernel approximation accuracy under near light as functions of relative distance to the light source.
Figure 4 shows the behavior of the approximation accuracy (cumulative energy) eN captured by the N th order function approximation (up to the sixth order). It can be inferred that: (1) For a light source at a specific distant to the object, the approximation accuracy increases proportionally with the harmonic order. (2) In case of near and asymptotic distant light source, there is a significant increase in the approximation accuracy of HSH when compared to that of SH, especially when comparing the 1st order HSH approximation accuracy which reaches to 100 % at R/r ≥ 4
(a)
(b)
Figure 4. Spherical harmonics (a) versus hemispherical harmonics (b) Lambertian kernel N th order approximation accuracy under near light as functions of relative distance to the light source.
Figure 5 shows the approximating harmonic order to reach to 99.22% accuracy. It is evident that the order needed to achieve such an accuracy level drops to 1st order in case of HSH, starting at (R/r) = 4. For the SH case, it drops only up to the 4th order, starting at (R/r) = 5.
5. Experimental Results We perform near light simulations with the USF 3D human face database [17] which contains 100 subjects of diverse gender and ethnicity. Each 3D facial surface is represented as a 2-manifold triangular mesh which is rerepresented in terms of Monge patches using orthographic projection. This provides a bijective mapping between surface points and image coordinates. We use forward finite difference to approximate surface derivatives from which surface normals are obtained. In order to quantitatively analyze the performance of the proposed image irradiance representation, we simulate the image irradiance equation by fixing the camera and the surface in position, and illuminating the facial surface using (1) a single light source and (2) multiple light sources located at finite distances. The light directions are randomly sampled using Monte Carlo sampling, where lights are cast with equal probabilities. The light sources are moved from close by to very far ones relative to the size of the object at hand. Experiments are repeated 50 times with randomly sampled light directions to guarantee the statistical significance of our results. For each synthetic irradiance E , the approximated irradiance Eb is compared with the groundtruth E and the mean of the absolute error is computed. Figs. 6(a) and 6(b) show the image irradiance approx-
Figure 5. Approximating order to reach to 99.22% accuracy, as function of relative distance to the light source.
imation performance in terms of mean absolute error, for single and multiple light sources scenarios, as functions of the relative distance to the light source. Experiments are conducted for up to the 3rd order for HSH and SH approximations. Regardless of the harmonic order and the number of light sources, the approximation error decreases rapidly with the distance to the light source and approaches an asymptotic value at very far distances. Comparing the results of single versus multiple light sources, the latter scenario provide lower approximation errors, which agrees with the conclusion presented by Frolova et al. [9] that a single distant light source provide the worst case illumintation for SH approximation. Our results provide an empirical validation for this conclusion which also holds for HSH approximation and near lighting. In all cases, HSH approximation performance provides a lower bound for SH approximation, which emphasizes its power in image irradiance modeling under near lighting conditions.
(a)
(b)
Figure 6. (a) Single, (b) Multiple, light source scenario: The mean of the mean absolute error measured between the ground-truth image irradiance and the approximated ones (using up to 3rd order HSH compared to these of SH) as a function of the relative distance to the light source
6. Conclusion In this paper, we presented an efficient and accurate representation for image irradiance equation in the case of a single viewpoint, where spherical harmonics are no longer orthonormal. We derived the hemispherical harmonic (HSH) basis to model near illumination conditions under the assumption of convex-Lambertian surfaces. We presented analytical and experimental evaluation of HSH performance. Our experiments illustrated the capability of HSH of presenting a more compact representation to the Lambertian kernel under near and asymptotically distant lighting conditions. In terms of harmonic order, it was evident that the order needed to achieve 99.22% accuracy level drops to 1st order in case of HSH as the light distance increases; in comparison, the SH case falls only up to the 4th order, leading to the conclusion that HSH handles close-by lights as distant ones when compared to SH. HSH also proved to have the capability of modeling image irradiance under near lighting better than SH. Ongoing efforts are directed towards modeling more realistic surface BRDF’s where HSH is known to outperform SH in terms of approximation accuracy and efficiency.
References [1] R. Ramamoorthi, Analytic PCA construction for theoretical analysis of lighting variability in images of a Lambertian object, IEEE Trans. Pattern Analysis and Machine Intelligence (PAMI), Vol. 24, No. 10, pp. 1322-1333, Oct. 2002. 1, 2 [2] P. N. Belhumeur, D. J. Kriegman, What is the set of images of an object under all possible lighting conditions?, Int’l Journal of Computer Vision, vol. 28, no. 3, pp. 245-260, 1998. 1 [3] A. Yuille, D. Snow, R. Epstein, and P. Belhumeur, Determining Generative Models of Objects under Varying Illumination: Shape and Albedo from Multiple Images Using SVD and Integrability, Intl J. Computer Vision, vol. 35, no. 3, pp. 203-222, 1999. 1 [4] A. Georghiades, P. Belhumeur, and D. Kriegman, From Few to Many: Generative Models for Recognition under Variable Pose and Illumination, Proc. Fourth Intl Conf. Automatic Face and Guesture Recogniton, pp. 277-284, 2000. 1 [5] R. Basri, D. W. Jacobs, Lambertian reflectance and linear subspaces, Pattern Analysis and Machine Intelligence, IEEE Transactions on, vol.25, no.2, pp. 218- 233, Feb 2003. 1, 3 [6] K-C. Lee, J. Ho, D. Kriegman, Acquiring Linear Subspaces for Face Recognition under Variable Lighting, IEEE Trans. Pattern Analysis and Machine Intelligence (PAMI), Vol. 27, No. 5, pp. 684-698, May 2005. 1 [7] R. Ramamoorthi, A Signal-Processing Framework for Forward and Inverse Rendering, Ph.D. Dissertation. Stanford University, Stanford, CA, USA, Advisor(s) P. Hanrahan. 1 [8] B.K.P. Horn. Shape from Shading, A Method for Obtaining the Shape of a Smooth Opaque Object from One View. PhD thesis, Massachusetts Inst. of Technology, Cambridge, Massachusetts, 1970. 1 [9] D. Frolova, D. Simakov, and R. Basri, Accuracy of spherical harmonic approximations for images of Lambertian objects under far and near lighting, European Conf. on Computer Vision (ECCV-04), Prague, 574-587, 2004. 1, 2, 4, 6 [10] P. Gautron, J. Krivanek, S.N. Pattanaik, and K. Bouatouch, A Novel Hemispherical Basis for Accurate and Efficient Rendering, Proc. Eurographics Symp. Rendering, 2004. 2, 3, 4 [11] P. P. Sloan, J. Hall, J. Hart, J. Snyder, Clustered principal components for precomputed radiance transfer, ACM Trans. Graph, vol.22, no.3, pp. 382-391, 2003. 2 [12] J. Koenderink, A. Van Doorn, M. Stavridi, Bidirectional reflection distribution function expressed in terms of surface scattering modes, in ECCV,pp. 28-39, 1996. 2 [13] J. C. Wyant and K. Creath, Basic wavefront aberration theory for optical metrology, In Applied optics and Optical Engineering, vol XI , Academic Press, Inc., pp. 27.39, 1992. 2 [14] O. A. Makhotkin, Analysis of radiative transfer between surfaces by hemispherical harmonics, Journal of Quantitative Spectroscopy and Radiative Transfer, Vol. 56, no. 6, 869-879, 1996. 2, 3 [15] R. Habel and M. Wimmer, Efficient Irradiance Normal Mapping, Proceedings of the 2010 ACM SIGGRAPH symposium on Interactive 3D Graphics and Games, pp. 189-195, 2010. 3 [16] P. K. Kythe and M. R. Schaferkotter, Handbook of Computational Methods for Integration, Chapman & Hall/CRC, 2004. 4 [17] S. Sarkar, USF HumanID 3D Face Database,University of South Florida, Tampa, FL, USA. 2, 5
