A Gabor Quotient Image for Face Recognition under Varying Illumination
A Gabor Quotient Image for Face Recognition under Varying Illumination Sanun Srisuk and Amnart Petpon Department of Computer Engineering, Mahanakorn University of Technology 51 Cheum-Sampan Rd., Nong Chok, Bangkok, THAILAND 10530 [email protected], [email protected]
Abstract. In this paper, we introduce a novel concept of illumination normalization for robust face recognition under different illumination conditions. The concept is extended from the Self Quotient Image (SQI) by which the 2D Gabor filter is applied instead of weighted Gaussian filter in order to increase more efficiency of the face recognition. Our experimental result, which is conducted on Yale face database B, has shown that our proposed method reached a very high recognition rate even in the case of extreme varying illumination. Keyword: Illumination Normalization, Self Quotient Image, Gabor Quotient Image.
1 Introduction Face recognition is one of the major issues in biometric technology. It identifies and/or verifies a person by using a 2D/3D physical characteristics of the face images. The baseline method of face recognition system is the eigenface [1] by which the goal of the eigenface method is to project linearly the image space onto the feature space which has less dimensionality. One can reconstruct a face image by using only a few eigenvectors which correspond to the largest eigenvalues, known as eigenpicture, eigenface, Karhunen-Loeve transform and principal component analysis [1], [2]. Several techniques have been proposed for solving a major problem in face recognition such as fisherface [2], elastic bunch graph matching [3] and support vector machine [4]. However, there are still many challenge problems in face recognition system such as facial expressions, pose variations and illumination changes. Those variations degrade the performance of face recognition system. It is known that illumination variation is the most impact of the changes in appearance of the face images because of its fluctuation by increasing or decreasing the intensities of face images due to shadow cast given by different light source direction. Belhumeur et. al. [2] suggested that discarding the three most significant principal components can reduce the illumination variation in the face images. Nevertheless, the three most significant principal components not only contain illumination variations but also some useful information, therefore, the system was also degraded as well. It has been proved that the difference due to illumination changes is more significant than the difference between individuals in face recognition system [5], [6]. Shashua et. al. [7] proposed a quotient image model in which all face images were defined as the same shape but differ in their surface albedo. The quotient image is a G. Bebis et al. (Eds.): ISVC 2008, Part II, LNCS 5359, pp. 511–520, 2008. c Springer-Verlag Berlin Heidelberg 2008
512
S. Srisuk and A. Petpon
ratio between a test image and any linear combination of three different illumination images, hence, the quotient image is illumination free. Wang et. al. [8], [9] proposed a self quotient image by using only single image. The self quotient image was obtained by using the Gaussian function as a smoothing kernel function. The total variation quotient image and logarithmic quotient image [10], [11] have been proposed by which the face image was decomposed into a small scale (texture) and large scale (cartoon) images. The normalized image was obtained by dividing the original image with the large scale one. The TVQI has a very high computational complexity due to the second order cone programming [12] as their kernel function. More recently, Wang et. al. [13] proposed a morphological quotient image and dynamic morphological quotient image models in which the illumination variations can be estimated easily by utilizing closing operation (dilation and then erosion) rather than opening operation (erosion and then dilation). In this paper, we propose a new illumination normalization technique using Gabor function. The Gabor quotient image is extended from the self quotient image in which the Gaussian function is substituted by the even Gabor function. We will show that our proposed method has a very high recognition rate. The rest of this paper is organized as follows. In section 2, we give a brief overview of the reflectance model, then, the Gabor filter is described in details. Finally, we introduce our new proposed method, Gabor quotient image, for face recognition under varying illumination. Experimental results on Yale face database B will be given in section 3. Section 4 gives a conclusion.
2 Illumination Normalization Method In this section, we give an overview of reflectance model for image representation. We then describe the Gabor filter in which it can be used to normalize the face image. Finally, we introduce our new concept, the Gabor Quotient Image (GQI), for face recognition under varying illumination. The proposed method is based on the extension of Self-Quotient Image (SQI). 2.1 Reflectance Model A face may be classified as 3D objects that have the same shape but differ in the surface albedo function. In general, an image f (x, y) is characterized by two components: the amount of source illumination incident on the scene being viewed and the amount of illumination reflected by the objects in the scene. In both Retinex and Lambertian theories, they are based on the physical imaging model by which the intensity can be represented as the product of reflectance and illumination. In Retinex theory, the image f (x, y) is formed as: f (x, y) = r(x, y)i(x, y), (1) where r(x, y) ∈ (0, 1) is the reflectance determining by the characteristics of the imaged objects and i(x, y) ∈ (0, ∞) is the illumination source. In Lambertian reflectance function, the image can be described by the product of the albedo (texture) and the cosine angle between a point light source and the surface normal f (x, y) = ρ(x, y)n(x, y)T s,
(2)
A Gabor Quotient Image for Face Recognition under Varying Illumination
513
where ρ(x, y) is the albedo (surface texture) associated with point x, y in the face object, n(x, y) is the surface normal (shape) of the face object, and s is the point light source direction whose magnitude is the light source intensity. In this paper, the surface normal of the object is assumed to be the same for all objects of the class. In fact, the reflectance and albedo are illumination free in quotient image. The SQI [8] has been proposed for synthesizing an illumination normalization using only single image as a quotient between face image f (x, y) and a smoothed image S(x, y) Q(x, y) =
f (x, y) S(x, y)
(3)
The image S(x, y) in [8] is an isotropic Gaussian smoothing which is equivalent to the center/surround retinex transform [14]. 2.2 2D Gabor Filter The Gabor filter is one of the popular techniques in vision partly because Daugman [15] showed that receptive fields of most orientation receptive neurons in the cat’s brain looked very much like Gabor functions. The Gabor filters offer optimal trade-off between spectral bandwidth and spatial localization. In addition, it is closely related to wavelet decomposition in which the analysis of image using various orientation and frequency bandwidths can be obtained. Gabor filters have been applied in many applications such texture segmentation, image representation, edge detection and face recognition. Mathematically, the 2D Gabor filter is formed by modulating a complex sinusoid by a Gaussian function G(x, y) = s(x, y)h(x, y), (4) where s(x, y) is a complex sinusoid (known as a carrier) and g(x, y) is a 2-D Gaussian shaped function (known as envelope). The complex sinusoid is defined as follows: s(x, y) = e−
j2π λ
xr
(5)
,
where λ is the wave-length of the complex sinusoid function. The 2-D Gaussian function with different scale in x- and y-axis is
h(x, y) = e
− 12
x2 r 2 σx
y2
+ σr2
y
(6)
,
where σx and σy are the standard deviation or scale of Gaussian function, and xr and yr are a rotation operation in which xr = (x − x0 ) cos(θ) + (y − y0 ) sin(θ),
(7)
yr = −(x − x0 ) sin(θ) + (y − y0 ) cos(θ),
(8)
and where θ is the orientation, (x0 , y0 ) is the center of the receptive field in the spatial domain so that Gaussian function is peak at the location (x0 , y0 ). Daugman [15] generalized the Gabor function to the 2D form of receptive fields of the orientation selective simple cells: G(x, y) = e
− j2π λ xr
e
− 12
x2 r 2 σx
y2
+ σr2
y
.
(9)
514
S. Srisuk and A. Petpon
Therefore, the 2D Gabor function is a product of an elliptical Gaussian (adjusted by σx and σy ) and a complex plane wave. It is known that the complex valued 2D Gabor function contains in quadrature projection an even-symmetric cosine component and an odd-symmetric sine component. In order to obtain the Gabor response separately, the 2D Gabor function can be rewritten as an even and odd Gabor function (shown in Fig. 1) 1 x2r yr2 − 2 σ2 + σ2 2π x y xr e Geven (x, y) = cos , (10) λ Godd (x, y) = sin
1 x2r yr2 − 2 σ2 + σ2 2π x y . xr e λ
(11)
Hence, an even-symmetric cosine is a real part of Gabor function.
(b)
(a)
(c)
(d)
Fig. 1. Example of Gabor response, (a) 2D Gaussian function (b) 2D sinusoid function (c) Gabor response given by the even (its real part) of Gabor function, and (d) Gabor response given by the odd (its imaginary part) of Gabor function (λ = 2, σx = 2, σy = 2, θ = 0)
2.3 Gabor Quotient Image Model Numerous algorithms have been proposed for solving the problems of varying illumination in face recognition system. It is however known that the variations between the images of the same face due to illumination and viewing direction are almost always larger than image variations due the change in face identity. Therefore, the recognition accuracy must be improved for face recognition robust under varying illumination. Our proposed model is inspired by the self quotient image [9]. The SQI is based on the quotient image proposed by Shashua et. al. [7] with the following advantages: the alignment procedure is not required, the training set for estimating lighting direction
A Gabor Quotient Image for Face Recognition under Varying Illumination
515
is not needed, illumination source can be any types, and its computational cost is not expensive. In this paper, a Gabor quotient image is defined as Q(x, y) =
f (x, y) f (x, y) = , SG (x, y) G(x, y) ∗ f (x, y)
(12)
where * is the convolution operator. We called this model as a Gabor Quotient Image (GQI). We empirically found that G(x, y) can only be Geven (x, y). The subsequent task of our proposed method is to normalize Q(x, y) to have pixel intensity between 0 and 1, and to increase contrast of image by applying linear transformation function Q (x, y) =
Q(x, y) − Qmin , Qmax − Qmin
Qnorm (x, y) = 1 − e
Q (x,y) − E(Q (x,y))
(13)
,
(14)
where Qmax and Qmin are maximum and minimum values of Q respectively, and E(.) is a mean value. Therefore, Qnorm is a normalized Gabor quotient image and is used as an image for face recognition as shown in Fig. 2.
(a)
(b)
(c)
Fig. 2. Normalized face image examples, (a) the original image, (b) the quotient image obtained by the real part of Gabor function, and (c) the normalized Gabor quotient image Qnorm
3 Experimental Results For the testing of our algorithm we evaluated the proposed method on Yale face database B [16]. The database contains 5,760 single light source images of 10 subjects each seen under 576 viewing conditions (9 poses x 64 illumination conditions). For every subject in a particular pose, an image with ambient (background) illumination was also captured. The total number of images is thus in fact 5,760+90=5,850. The database was divided into 5 subsets according to the angle of the light source directions. The details for each subset are as follows: – Subset 1 (0◦ to 12◦ ) - 7 face images under different illumination conditions, 70 images in total,
516
S. Srisuk and A. Petpon
(a)
(b)
(c)
(d)
(e)
Fig. 3. Examples of 5 subsets in Yale Face Database B. (a) subset 1 (b) subset 2 (c) subset 3 (d) subset 4 (e) subset 5.
– Subset 2 (13◦ to 25◦ ) - 12 face images under different illumination conditions, 1 corrupted image was discarded, 119 images in total, – Subset 3 (26◦ to 50◦ ) - 12 face images under different illumination conditions, 1 corrupted image was discarded, 119 images in total, – Subset 4 (51◦ to 77◦ ) - 14 face images under different illumination conditions, 2 corrupted image was discarded, 138 images in total, – Subset 5 (above 78◦ ) - 18 face images under different illumination conditions, 1 corrupted image was discarded, 179 images in total. The examples of face database corresponding to each subset were shown in Fig. 3. It can be seen that the image quality was degraded from subset 1 to subset 5. In the following experiments, all face images (subset 1 to 5, 625 face images in total) were manually rotated, resized and cropped to 100 × 100 pixels with 256 gray levels according to the coordinates of two eyes. They were cropped so that the only face regions are considered. The recognition was performed by applying our proposed method, Gabor quotient image, to all face images. Then, the GQI face image was transformed to the principal component analysis (PCA) [1] for dimensionality reduction. In this experiment, 60 principal components were chosen and then classified with nearest neighbor by L2 distance DL 2 = (x − y)2 , (15) where x and y are test and training images, respectively. In the first experiment, all face images of subset 1 were used as a training set and subset 2, 3, 4 and 5 were used separately as a test set. The result was shown in Fig 4(a). The Fig 4(b) to (e) were results on using subset 2 to 5 as a training set and the other as a test set, separately. The parameters for this experiment were as follows: λ = 1, σx = 1, σy = 7 and θ = 0. It is clear that our GQI method outperformed all other methods: Histogram Equalization (HEQ), Gamma Intensity Correction (GIC), Log Transform (LOG), Self Quotient Image (SQI) [9], Morphological Quotient Image (MQI) and Dynamic Morphological Quotient Image (DMQI). In the second experiment, we selected one of the subset to be a test set and all other subsets be the training set. For example, when we used subsets 2, 3, 4 and 5 as a training set, the subset 1 was used as a test set, as shown in Table 1. The other was the same scheme for subsets 2, 3, 4 and 5 as a test set. Compared to the other methods in Table 1, our proposed method still has a highest recognition rate.
A Gabor Quotient Image for Face Recognition under Varying Illumination 100
HEQ GIC LOG SQI MQI DMQI GQI
90 80 70
100
80 70 60
50
50
40
40
30
30
20
20
10 0
HEQ GIC LOG SQI MQI DMQI GQI
90
60
10 1
2
3
4
0
5
1
2
(a)
3
4
5
(b)
100
HEQ GIC LOG SQI MQI DMQI GQI
90 80 70
100
80 70 60
50
50
40
40
30
30
20
20
10
HEQ GIC LOG SQI MQI DMQI GQI
90
60
0
517
10 1
2
3
4
0
5
1
2
(c)
3
4
5
(d)
100
HEQ GIC LOG SQI MQI DMQI GQI
90 80 70 60 50 40 30 20 10 0
1
2
3
4
5
(e)
Fig. 4. The recognition rate (%) of first experiment by taken (a) subset 1, (b) subset 2, (c) subset 3, (d) subset 4, and (e) subset 5 as the training set Table 1. Recognition performance on using subset 1 to 5 as a test set Methods/Test Set subset 1 Raw Image 100.00 97.14 HE [17] 100.00 Adaptive HE [18] 100.00 GIC [19] 100.00 Log Transform [17] 90.00 SQI [9] (3 × 3 Gaussian Filter) SQI [9] (3 × 3 Weighted Gaussian Filter) 94.29 100.00 3 × 3 MQI [13] 100.00 DMQI [13] 100.00 GQI [λ = 1, σx = 2, σy = 3, θ = 0] 100.00 GQI [λ = 1, σx = 1, σy = 7, θ = 0] GQI [λ = 50, σx = 2, σy = 3, θ = 0] 100.00 GQI [λ = 50, σx = 1, σy = 7, θ = 0] 100.00
subset 2 97.48 100.00 100.00 98.32 98.32 94.12 100.00 100.00 100.00 100.00 100.00 100.00 100.00
subset 3 74.79 72.27 73.95 43.70 89.08 95.80 100.00 100.00 99.16 100.00 100.00 100.00 100.00
subset 4 73.91 86.23 89.86 60.14 81.16 97.83 100.00 97.83 99.28 100.00 100.00 100.00 100.00
subset 5 31.28 50.84 68.72 22.91 36.31 98.88 99.44 93.85 94.41 99.44 100.00 99.44 100.00
518
S. Srisuk and A. Petpon
Table 2. Recognition performance on using subset 1 to 5 as a training set and the others were grouped as a test set Methods/Test Set All w/o 1 All w/o 2 All w/o 3 All w/o 4 All w/o 5 Raw Image 22.34 28.66 30.04 19.71 19.06 20.36 27.08 31.62 29.57 26.01 HE [17] 38.74 42.09 35.77 33.68 26.01 Adaptive HE [18] 20.90 21.34 16.40 12.94 14.57 GIC [19] 32.07 35.38 45.45 23.82 22.42 Log Transform [17] 61.26 67.59 77.87 54.62 47.31 SQI [9] (3 × 3 Gaussian Filter) 95.26 96.44 79.67 74.44 SQI [9] (3 × 3 Weighted Gaussian Filter) 92.79 99.10 99.80 97.23 78.44 81.39 3 × 3 MQI [13] 94.41 84.78 78.85 87.06 88.34 DMQI [13] 99.28 99.80 99.60 96.30 93.05 GQI [λ = 1, σx = 2, σy = 3, θ = 0] 99.82 99.80 98.22 96.92 93.27 GQI [λ = 1, σx = 1, σy = 7, θ = 0] 99.28 99.80 99.60 96.30 92.83 GQI [λ = 50, σx = 2, σy = 3, θ = 0] 99.82 99.80 98.22 96.92 93.27 GQI [λ = 50, σx = 1, σy = 7, θ = 0]
Table 3. The average computation time of Table 1 and 2 (in second) Methods Table 1 Raw Image 16.42 17.04 HE [17] 58.47 Adaptive HE [18] 17.23 GIC [19] 16.06 Log Transform [17] 17.39 SQI [9] (3 × 3 Gaussian Filter) 220.75 SQI [9] (3 × 3 Weighted Gaussian Filter) 21.19 3 × 3 MQI [13] 33.16 DMQI [13] 26.94 Gabor-Quotient Image [λ = 1, σx = 2, σy = 3, θ = 0] 21.22 Gabor-Quotient Image [λ = 1, σx = 1, σy = 7, θ = 0] Gabor-Quotient Image [λ = 50, σx = 2, σy = 3, θ = 0] 25.68 Gabor-Quotient Image [λ = 50, σx = 1, σy = 7, θ = 0] 21.10
Table 2 6.46 7.54 50.24 7.99 6.66 7.77 213.98 10.85 23.32 16.44 11.77 16.40 11.95
In the final experiment, we collected several subsets to be a test set as one subset. For example, if subset 1 was used as a training set, subsets 2, 3, 4 and 5 were grouped together to be a test set that is ‘All w/o 1’ as shown in Table 2. The other was the same scheme for subsets 2, 3, 4 and 5 as a training set. It confirms that our approach outperformed the other methods while they are facing with varying illumination. Our proposed method still has a very high recognition rate (93.27%) with ‘All w/o 5’ in which subset 5 (most extreme case of varying illumination in Yale face database B) was used as a training set and subsets 1, 2, 3 and 4 were used as a test set. The average computation time was shown in Table 3.
A Gabor Quotient Image for Face Recognition under Varying Illumination
519
4 Conclusions We have presented a new approach, Gabor quotient image, for face recognition which was robust under varying illumination. Our new approach based on Gabor filtering in which the even function was used to generate Gabor feature of the face image. The original image was normalized by the Gabor feature image which is then become the Gabor quotient image. Our proposed method required only single image, no need a prior alignment and less computational time. The Gabor quotient image has been compared against several methods. It was confirmed by the experimental results that our proposed method outperformed all other methods in case of extreme varying illumination. In the future work, we will improve the performance of the Gabor quotient image by minimizing the illumination variation while maintaining the discrimination ability of the facial feature.
References 1. Turk, M., Pentland, A.: Eigenfaces for recognition. J. Cognitive Neuroscience 3, 71–86 (1991) 2. Belhumeur, P., Hespanha, J., Kriegman, D.: Eigenfaces vs. fisherfaces: Recognition using class specific linear projection. IEEE Trans. Pattern Analysis and Machine Intelligence 20, 71–86 (1997) 3. Wiskott, L., Fellous, J., Kruger, N., Mals-burg, C.: Face recognition by elastic bunch graph matching. IEEE Trans. Pattern Analysis and Machine Intelligence 19, 775–779 (1997) 4. Guo, G., Li, S., Chan, K.: Face recognition by support vector machines. In: Proc. of IEEE Automatic Face and Gesture Recognition, pp. 196–201 (2000) 5. Adini, Y., Moses, Y., Ullman, S.: Face recognition: The problem of compensating for changes in illumination direction. IEEE Trans. Pattern Analysis and Machine Intelligence 19, 721– 732 (1997) 6. Zhao, W., Chellappa, R.: Robust face recognition using symmetric shape from shading. In: Technical report, Center for Automation Research, University of Maryland (1999) 7. Shashua, A., Riklin-Raviv, T.: The quotient image: Class-based re-rendering and recognition with varying illuminations. IEEE Trans. Pattern Analysis and Machine Intelligence 23, 129– 139 (2001) 8. Wang, H., Li, S., Wang, Y.: Generalized quotient image. In: Proc. of IEEE Computer Vision and Pattern Recognition (2004) 9. Wang, H., Li, S., Wang, Y.: Face recognition under varying lighting conditions using self quotient image. In: Proc. of IEEE Automatic Face and Gesture Recognition (2004) 10. Chen, T., Yin, W., Zhou, X.S., Comaniciu, D., Huang, T.S.: Illumination normalization for face recognition and uneven background correction using total variation based image models. In: Proc. of IEEE Computer Vision and Pattern Recognition (2005) 11. Chen, T., Yin, W., Zhou, X.S., Comaniciu, D., Huang, T.S.: Total variation models for variable lighting face recognition. IEEE Trans. Pattern Analysis and Machine Intelligence 28, 1519–1524 (2006) 12. Alizadeh, F., Goldfarb, D.: Second order cone programming. Math. Programming 95, 3–51 (2003) 13. Wang, J., Wu, L., He, X., Tian, J.: A new method of illumination invariant face recognition. In: Proc. of International Conference on Innovative Computing (2007)
520
S. Srisuk and A. Petpon
14. Jobson, D.J., Rahman, Z.: Properties and performance of a center/surround retinex. IEEE Trans. on Image Processing 6, 451–462 (1997) 15. Daugman, J.: Complete discrete 2-d gabor transforms by neural networks for image analysis and compression. IEEE Trans. Acoustics, Speech, and Signal Processing 36, 1169–1179 (1988) 16. Georghiades, A., Belhumeur, P., Kriegman, D.: From few to many: Illumination cone models for face recognition under variable lighting and pose. IEEE Trans. Pattern Analysis and Machine Intelligence 23, 643–660 (2001) 17. Gonzalez, R.C., Woods, R.E.: Digital Image Processing. Addison-Wesley Longman Publishing Co., Inc., Boston (2001) 18. Reza, A.M.: Realization of the contrast limited adaptive histogram equalization (clahe) for real-time image enhancement. J. VLSI Signal Processing System 38, 35–44 (2004) 19. Shan, S., Gao, W., Cao, B., Zhao, D.: Illumination normalization for robust face recognition against varying lighting conditions. In: Proc. of IEEE International Workshop on Analysis and Modeling of Faces and Gestures, Washington, DC, USA, p. 157 (2003)
Abstract. In this paper, we introduce a novel concept of illumination normalization for robust face recognition under different illumination conditions. The concept is extended from the Self Quotient Image (SQI) by which the 2D Gabor filter is applied instead of weighted Gaussian filter in order to increase more efficiency of the face recognition. Our experimental result, which is conducted on Yale face database B, has shown that our proposed method reached a very high recognition rate even in the case of extreme varying illumination. Keyword: Illumination Normalization, Self Quotient Image, Gabor Quotient Image.
1 Introduction Face recognition is one of the major issues in biometric technology. It identifies and/or verifies a person by using a 2D/3D physical characteristics of the face images. The baseline method of face recognition system is the eigenface [1] by which the goal of the eigenface method is to project linearly the image space onto the feature space which has less dimensionality. One can reconstruct a face image by using only a few eigenvectors which correspond to the largest eigenvalues, known as eigenpicture, eigenface, Karhunen-Loeve transform and principal component analysis [1], [2]. Several techniques have been proposed for solving a major problem in face recognition such as fisherface [2], elastic bunch graph matching [3] and support vector machine [4]. However, there are still many challenge problems in face recognition system such as facial expressions, pose variations and illumination changes. Those variations degrade the performance of face recognition system. It is known that illumination variation is the most impact of the changes in appearance of the face images because of its fluctuation by increasing or decreasing the intensities of face images due to shadow cast given by different light source direction. Belhumeur et. al. [2] suggested that discarding the three most significant principal components can reduce the illumination variation in the face images. Nevertheless, the three most significant principal components not only contain illumination variations but also some useful information, therefore, the system was also degraded as well. It has been proved that the difference due to illumination changes is more significant than the difference between individuals in face recognition system [5], [6]. Shashua et. al. [7] proposed a quotient image model in which all face images were defined as the same shape but differ in their surface albedo. The quotient image is a G. Bebis et al. (Eds.): ISVC 2008, Part II, LNCS 5359, pp. 511–520, 2008. c Springer-Verlag Berlin Heidelberg 2008
512
S. Srisuk and A. Petpon
ratio between a test image and any linear combination of three different illumination images, hence, the quotient image is illumination free. Wang et. al. [8], [9] proposed a self quotient image by using only single image. The self quotient image was obtained by using the Gaussian function as a smoothing kernel function. The total variation quotient image and logarithmic quotient image [10], [11] have been proposed by which the face image was decomposed into a small scale (texture) and large scale (cartoon) images. The normalized image was obtained by dividing the original image with the large scale one. The TVQI has a very high computational complexity due to the second order cone programming [12] as their kernel function. More recently, Wang et. al. [13] proposed a morphological quotient image and dynamic morphological quotient image models in which the illumination variations can be estimated easily by utilizing closing operation (dilation and then erosion) rather than opening operation (erosion and then dilation). In this paper, we propose a new illumination normalization technique using Gabor function. The Gabor quotient image is extended from the self quotient image in which the Gaussian function is substituted by the even Gabor function. We will show that our proposed method has a very high recognition rate. The rest of this paper is organized as follows. In section 2, we give a brief overview of the reflectance model, then, the Gabor filter is described in details. Finally, we introduce our new proposed method, Gabor quotient image, for face recognition under varying illumination. Experimental results on Yale face database B will be given in section 3. Section 4 gives a conclusion.
2 Illumination Normalization Method In this section, we give an overview of reflectance model for image representation. We then describe the Gabor filter in which it can be used to normalize the face image. Finally, we introduce our new concept, the Gabor Quotient Image (GQI), for face recognition under varying illumination. The proposed method is based on the extension of Self-Quotient Image (SQI). 2.1 Reflectance Model A face may be classified as 3D objects that have the same shape but differ in the surface albedo function. In general, an image f (x, y) is characterized by two components: the amount of source illumination incident on the scene being viewed and the amount of illumination reflected by the objects in the scene. In both Retinex and Lambertian theories, they are based on the physical imaging model by which the intensity can be represented as the product of reflectance and illumination. In Retinex theory, the image f (x, y) is formed as: f (x, y) = r(x, y)i(x, y), (1) where r(x, y) ∈ (0, 1) is the reflectance determining by the characteristics of the imaged objects and i(x, y) ∈ (0, ∞) is the illumination source. In Lambertian reflectance function, the image can be described by the product of the albedo (texture) and the cosine angle between a point light source and the surface normal f (x, y) = ρ(x, y)n(x, y)T s,
(2)
A Gabor Quotient Image for Face Recognition under Varying Illumination
513
where ρ(x, y) is the albedo (surface texture) associated with point x, y in the face object, n(x, y) is the surface normal (shape) of the face object, and s is the point light source direction whose magnitude is the light source intensity. In this paper, the surface normal of the object is assumed to be the same for all objects of the class. In fact, the reflectance and albedo are illumination free in quotient image. The SQI [8] has been proposed for synthesizing an illumination normalization using only single image as a quotient between face image f (x, y) and a smoothed image S(x, y) Q(x, y) =
f (x, y) S(x, y)
(3)
The image S(x, y) in [8] is an isotropic Gaussian smoothing which is equivalent to the center/surround retinex transform [14]. 2.2 2D Gabor Filter The Gabor filter is one of the popular techniques in vision partly because Daugman [15] showed that receptive fields of most orientation receptive neurons in the cat’s brain looked very much like Gabor functions. The Gabor filters offer optimal trade-off between spectral bandwidth and spatial localization. In addition, it is closely related to wavelet decomposition in which the analysis of image using various orientation and frequency bandwidths can be obtained. Gabor filters have been applied in many applications such texture segmentation, image representation, edge detection and face recognition. Mathematically, the 2D Gabor filter is formed by modulating a complex sinusoid by a Gaussian function G(x, y) = s(x, y)h(x, y), (4) where s(x, y) is a complex sinusoid (known as a carrier) and g(x, y) is a 2-D Gaussian shaped function (known as envelope). The complex sinusoid is defined as follows: s(x, y) = e−
j2π λ
xr
(5)
,
where λ is the wave-length of the complex sinusoid function. The 2-D Gaussian function with different scale in x- and y-axis is
h(x, y) = e
− 12
x2 r 2 σx
y2
+ σr2
y
(6)
,
where σx and σy are the standard deviation or scale of Gaussian function, and xr and yr are a rotation operation in which xr = (x − x0 ) cos(θ) + (y − y0 ) sin(θ),
(7)
yr = −(x − x0 ) sin(θ) + (y − y0 ) cos(θ),
(8)
and where θ is the orientation, (x0 , y0 ) is the center of the receptive field in the spatial domain so that Gaussian function is peak at the location (x0 , y0 ). Daugman [15] generalized the Gabor function to the 2D form of receptive fields of the orientation selective simple cells: G(x, y) = e
− j2π λ xr
e
− 12
x2 r 2 σx
y2
+ σr2
y
.
(9)
514
S. Srisuk and A. Petpon
Therefore, the 2D Gabor function is a product of an elliptical Gaussian (adjusted by σx and σy ) and a complex plane wave. It is known that the complex valued 2D Gabor function contains in quadrature projection an even-symmetric cosine component and an odd-symmetric sine component. In order to obtain the Gabor response separately, the 2D Gabor function can be rewritten as an even and odd Gabor function (shown in Fig. 1) 1 x2r yr2 − 2 σ2 + σ2 2π x y xr e Geven (x, y) = cos , (10) λ Godd (x, y) = sin
1 x2r yr2 − 2 σ2 + σ2 2π x y . xr e λ
(11)
Hence, an even-symmetric cosine is a real part of Gabor function.
(b)
(a)
(c)
(d)
Fig. 1. Example of Gabor response, (a) 2D Gaussian function (b) 2D sinusoid function (c) Gabor response given by the even (its real part) of Gabor function, and (d) Gabor response given by the odd (its imaginary part) of Gabor function (λ = 2, σx = 2, σy = 2, θ = 0)
2.3 Gabor Quotient Image Model Numerous algorithms have been proposed for solving the problems of varying illumination in face recognition system. It is however known that the variations between the images of the same face due to illumination and viewing direction are almost always larger than image variations due the change in face identity. Therefore, the recognition accuracy must be improved for face recognition robust under varying illumination. Our proposed model is inspired by the self quotient image [9]. The SQI is based on the quotient image proposed by Shashua et. al. [7] with the following advantages: the alignment procedure is not required, the training set for estimating lighting direction
A Gabor Quotient Image for Face Recognition under Varying Illumination
515
is not needed, illumination source can be any types, and its computational cost is not expensive. In this paper, a Gabor quotient image is defined as Q(x, y) =
f (x, y) f (x, y) = , SG (x, y) G(x, y) ∗ f (x, y)
(12)
where * is the convolution operator. We called this model as a Gabor Quotient Image (GQI). We empirically found that G(x, y) can only be Geven (x, y). The subsequent task of our proposed method is to normalize Q(x, y) to have pixel intensity between 0 and 1, and to increase contrast of image by applying linear transformation function Q (x, y) =
Q(x, y) − Qmin , Qmax − Qmin
Qnorm (x, y) = 1 − e
Q (x,y) − E(Q (x,y))
(13)
,
(14)
where Qmax and Qmin are maximum and minimum values of Q respectively, and E(.) is a mean value. Therefore, Qnorm is a normalized Gabor quotient image and is used as an image for face recognition as shown in Fig. 2.
(a)
(b)
(c)
Fig. 2. Normalized face image examples, (a) the original image, (b) the quotient image obtained by the real part of Gabor function, and (c) the normalized Gabor quotient image Qnorm
3 Experimental Results For the testing of our algorithm we evaluated the proposed method on Yale face database B [16]. The database contains 5,760 single light source images of 10 subjects each seen under 576 viewing conditions (9 poses x 64 illumination conditions). For every subject in a particular pose, an image with ambient (background) illumination was also captured. The total number of images is thus in fact 5,760+90=5,850. The database was divided into 5 subsets according to the angle of the light source directions. The details for each subset are as follows: – Subset 1 (0◦ to 12◦ ) - 7 face images under different illumination conditions, 70 images in total,
516
S. Srisuk and A. Petpon
(a)
(b)
(c)
(d)
(e)
Fig. 3. Examples of 5 subsets in Yale Face Database B. (a) subset 1 (b) subset 2 (c) subset 3 (d) subset 4 (e) subset 5.
– Subset 2 (13◦ to 25◦ ) - 12 face images under different illumination conditions, 1 corrupted image was discarded, 119 images in total, – Subset 3 (26◦ to 50◦ ) - 12 face images under different illumination conditions, 1 corrupted image was discarded, 119 images in total, – Subset 4 (51◦ to 77◦ ) - 14 face images under different illumination conditions, 2 corrupted image was discarded, 138 images in total, – Subset 5 (above 78◦ ) - 18 face images under different illumination conditions, 1 corrupted image was discarded, 179 images in total. The examples of face database corresponding to each subset were shown in Fig. 3. It can be seen that the image quality was degraded from subset 1 to subset 5. In the following experiments, all face images (subset 1 to 5, 625 face images in total) were manually rotated, resized and cropped to 100 × 100 pixels with 256 gray levels according to the coordinates of two eyes. They were cropped so that the only face regions are considered. The recognition was performed by applying our proposed method, Gabor quotient image, to all face images. Then, the GQI face image was transformed to the principal component analysis (PCA) [1] for dimensionality reduction. In this experiment, 60 principal components were chosen and then classified with nearest neighbor by L2 distance DL 2 = (x − y)2 , (15) where x and y are test and training images, respectively. In the first experiment, all face images of subset 1 were used as a training set and subset 2, 3, 4 and 5 were used separately as a test set. The result was shown in Fig 4(a). The Fig 4(b) to (e) were results on using subset 2 to 5 as a training set and the other as a test set, separately. The parameters for this experiment were as follows: λ = 1, σx = 1, σy = 7 and θ = 0. It is clear that our GQI method outperformed all other methods: Histogram Equalization (HEQ), Gamma Intensity Correction (GIC), Log Transform (LOG), Self Quotient Image (SQI) [9], Morphological Quotient Image (MQI) and Dynamic Morphological Quotient Image (DMQI). In the second experiment, we selected one of the subset to be a test set and all other subsets be the training set. For example, when we used subsets 2, 3, 4 and 5 as a training set, the subset 1 was used as a test set, as shown in Table 1. The other was the same scheme for subsets 2, 3, 4 and 5 as a test set. Compared to the other methods in Table 1, our proposed method still has a highest recognition rate.
A Gabor Quotient Image for Face Recognition under Varying Illumination 100
HEQ GIC LOG SQI MQI DMQI GQI
90 80 70
100
80 70 60
50
50
40
40
30
30
20
20
10 0
HEQ GIC LOG SQI MQI DMQI GQI
90
60
10 1
2
3
4
0
5
1
2
(a)
3
4
5
(b)
100
HEQ GIC LOG SQI MQI DMQI GQI
90 80 70
100
80 70 60
50
50
40
40
30
30
20
20
10
HEQ GIC LOG SQI MQI DMQI GQI
90
60
0
517
10 1
2
3
4
0
5
1
2
(c)
3
4
5
(d)
100
HEQ GIC LOG SQI MQI DMQI GQI
90 80 70 60 50 40 30 20 10 0
1
2
3
4
5
(e)
Fig. 4. The recognition rate (%) of first experiment by taken (a) subset 1, (b) subset 2, (c) subset 3, (d) subset 4, and (e) subset 5 as the training set Table 1. Recognition performance on using subset 1 to 5 as a test set Methods/Test Set subset 1 Raw Image 100.00 97.14 HE [17] 100.00 Adaptive HE [18] 100.00 GIC [19] 100.00 Log Transform [17] 90.00 SQI [9] (3 × 3 Gaussian Filter) SQI [9] (3 × 3 Weighted Gaussian Filter) 94.29 100.00 3 × 3 MQI [13] 100.00 DMQI [13] 100.00 GQI [λ = 1, σx = 2, σy = 3, θ = 0] 100.00 GQI [λ = 1, σx = 1, σy = 7, θ = 0] GQI [λ = 50, σx = 2, σy = 3, θ = 0] 100.00 GQI [λ = 50, σx = 1, σy = 7, θ = 0] 100.00
subset 2 97.48 100.00 100.00 98.32 98.32 94.12 100.00 100.00 100.00 100.00 100.00 100.00 100.00
subset 3 74.79 72.27 73.95 43.70 89.08 95.80 100.00 100.00 99.16 100.00 100.00 100.00 100.00
subset 4 73.91 86.23 89.86 60.14 81.16 97.83 100.00 97.83 99.28 100.00 100.00 100.00 100.00
subset 5 31.28 50.84 68.72 22.91 36.31 98.88 99.44 93.85 94.41 99.44 100.00 99.44 100.00
518
S. Srisuk and A. Petpon
Table 2. Recognition performance on using subset 1 to 5 as a training set and the others were grouped as a test set Methods/Test Set All w/o 1 All w/o 2 All w/o 3 All w/o 4 All w/o 5 Raw Image 22.34 28.66 30.04 19.71 19.06 20.36 27.08 31.62 29.57 26.01 HE [17] 38.74 42.09 35.77 33.68 26.01 Adaptive HE [18] 20.90 21.34 16.40 12.94 14.57 GIC [19] 32.07 35.38 45.45 23.82 22.42 Log Transform [17] 61.26 67.59 77.87 54.62 47.31 SQI [9] (3 × 3 Gaussian Filter) 95.26 96.44 79.67 74.44 SQI [9] (3 × 3 Weighted Gaussian Filter) 92.79 99.10 99.80 97.23 78.44 81.39 3 × 3 MQI [13] 94.41 84.78 78.85 87.06 88.34 DMQI [13] 99.28 99.80 99.60 96.30 93.05 GQI [λ = 1, σx = 2, σy = 3, θ = 0] 99.82 99.80 98.22 96.92 93.27 GQI [λ = 1, σx = 1, σy = 7, θ = 0] 99.28 99.80 99.60 96.30 92.83 GQI [λ = 50, σx = 2, σy = 3, θ = 0] 99.82 99.80 98.22 96.92 93.27 GQI [λ = 50, σx = 1, σy = 7, θ = 0]
Table 3. The average computation time of Table 1 and 2 (in second) Methods Table 1 Raw Image 16.42 17.04 HE [17] 58.47 Adaptive HE [18] 17.23 GIC [19] 16.06 Log Transform [17] 17.39 SQI [9] (3 × 3 Gaussian Filter) 220.75 SQI [9] (3 × 3 Weighted Gaussian Filter) 21.19 3 × 3 MQI [13] 33.16 DMQI [13] 26.94 Gabor-Quotient Image [λ = 1, σx = 2, σy = 3, θ = 0] 21.22 Gabor-Quotient Image [λ = 1, σx = 1, σy = 7, θ = 0] Gabor-Quotient Image [λ = 50, σx = 2, σy = 3, θ = 0] 25.68 Gabor-Quotient Image [λ = 50, σx = 1, σy = 7, θ = 0] 21.10
Table 2 6.46 7.54 50.24 7.99 6.66 7.77 213.98 10.85 23.32 16.44 11.77 16.40 11.95
In the final experiment, we collected several subsets to be a test set as one subset. For example, if subset 1 was used as a training set, subsets 2, 3, 4 and 5 were grouped together to be a test set that is ‘All w/o 1’ as shown in Table 2. The other was the same scheme for subsets 2, 3, 4 and 5 as a training set. It confirms that our approach outperformed the other methods while they are facing with varying illumination. Our proposed method still has a very high recognition rate (93.27%) with ‘All w/o 5’ in which subset 5 (most extreme case of varying illumination in Yale face database B) was used as a training set and subsets 1, 2, 3 and 4 were used as a test set. The average computation time was shown in Table 3.
A Gabor Quotient Image for Face Recognition under Varying Illumination
519
4 Conclusions We have presented a new approach, Gabor quotient image, for face recognition which was robust under varying illumination. Our new approach based on Gabor filtering in which the even function was used to generate Gabor feature of the face image. The original image was normalized by the Gabor feature image which is then become the Gabor quotient image. Our proposed method required only single image, no need a prior alignment and less computational time. The Gabor quotient image has been compared against several methods. It was confirmed by the experimental results that our proposed method outperformed all other methods in case of extreme varying illumination. In the future work, we will improve the performance of the Gabor quotient image by minimizing the illumination variation while maintaining the discrimination ability of the facial feature.
References 1. Turk, M., Pentland, A.: Eigenfaces for recognition. J. Cognitive Neuroscience 3, 71–86 (1991) 2. Belhumeur, P., Hespanha, J., Kriegman, D.: Eigenfaces vs. fisherfaces: Recognition using class specific linear projection. IEEE Trans. Pattern Analysis and Machine Intelligence 20, 71–86 (1997) 3. Wiskott, L., Fellous, J., Kruger, N., Mals-burg, C.: Face recognition by elastic bunch graph matching. IEEE Trans. Pattern Analysis and Machine Intelligence 19, 775–779 (1997) 4. Guo, G., Li, S., Chan, K.: Face recognition by support vector machines. In: Proc. of IEEE Automatic Face and Gesture Recognition, pp. 196–201 (2000) 5. Adini, Y., Moses, Y., Ullman, S.: Face recognition: The problem of compensating for changes in illumination direction. IEEE Trans. Pattern Analysis and Machine Intelligence 19, 721– 732 (1997) 6. Zhao, W., Chellappa, R.: Robust face recognition using symmetric shape from shading. In: Technical report, Center for Automation Research, University of Maryland (1999) 7. Shashua, A., Riklin-Raviv, T.: The quotient image: Class-based re-rendering and recognition with varying illuminations. IEEE Trans. Pattern Analysis and Machine Intelligence 23, 129– 139 (2001) 8. Wang, H., Li, S., Wang, Y.: Generalized quotient image. In: Proc. of IEEE Computer Vision and Pattern Recognition (2004) 9. Wang, H., Li, S., Wang, Y.: Face recognition under varying lighting conditions using self quotient image. In: Proc. of IEEE Automatic Face and Gesture Recognition (2004) 10. Chen, T., Yin, W., Zhou, X.S., Comaniciu, D., Huang, T.S.: Illumination normalization for face recognition and uneven background correction using total variation based image models. In: Proc. of IEEE Computer Vision and Pattern Recognition (2005) 11. Chen, T., Yin, W., Zhou, X.S., Comaniciu, D., Huang, T.S.: Total variation models for variable lighting face recognition. IEEE Trans. Pattern Analysis and Machine Intelligence 28, 1519–1524 (2006) 12. Alizadeh, F., Goldfarb, D.: Second order cone programming. Math. Programming 95, 3–51 (2003) 13. Wang, J., Wu, L., He, X., Tian, J.: A new method of illumination invariant face recognition. In: Proc. of International Conference on Innovative Computing (2007)
520
S. Srisuk and A. Petpon
14. Jobson, D.J., Rahman, Z.: Properties and performance of a center/surround retinex. IEEE Trans. on Image Processing 6, 451–462 (1997) 15. Daugman, J.: Complete discrete 2-d gabor transforms by neural networks for image analysis and compression. IEEE Trans. Acoustics, Speech, and Signal Processing 36, 1169–1179 (1988) 16. Georghiades, A., Belhumeur, P., Kriegman, D.: From few to many: Illumination cone models for face recognition under variable lighting and pose. IEEE Trans. Pattern Analysis and Machine Intelligence 23, 643–660 (2001) 17. Gonzalez, R.C., Woods, R.E.: Digital Image Processing. Addison-Wesley Longman Publishing Co., Inc., Boston (2001) 18. Reza, A.M.: Realization of the contrast limited adaptive histogram equalization (clahe) for real-time image enhancement. J. VLSI Signal Processing System 38, 35–44 (2004) 19. Shan, S., Gao, W., Cao, B., Zhao, D.: Illumination normalization for robust face recognition against varying lighting conditions. In: Proc. of IEEE International Workshop on Analysis and Modeling of Faces and Gestures, Washington, DC, USA, p. 157 (2003)
