Illumination Invariant Face Recognition Using ... - Semantic Scholar
manuscript No. (will be inserted by the editor)
Dayron Rizo-Rodr´ıguez · Heydi M´ endez-V´ azquez · Edel Garc´ıa-Reyes
Illumination Invariant Face Recognition Using Quaternion-Based Correlation Filters Received: date / Revised: date
Abstract Existing face recognition systems decrease their performance when face images are affected by lighting variations. Recently, several quaternionic representations of face image features and a quaternion-based correlation filter have been combined in order to cope with the effects of having nonproperly illuminated face images. In this paper, a generalized scheme for face recognition using quaternion-based correlation filters is described. Furthermore, a comparison by using different quaternion-based correlation filters in this scheme is presented. Three different quaternion-based correlation filters are designed and conjugated with four face feature extraction methods aiming at obtaining the best combination. Verification and identification experiments confirms that when combining a quaternionic representation with a quaternion-based correlation filter, both with good discriminative power and illumination invariant properties, an improvement in face recognition accuracy is obtained. Keywords quaternions · correlation filters · illumination invariant face recognition Dayron Rizo-Rodr´ıguez 7a. #218 b/ 218 and 222, Rpto. Siboney, Playa, Havana, Cuba. Tel.: +537-272-1670 E-mail: [email protected] Heydi M´endez-V´ azquez 7a. #218 b/ 218 and 222, Rpto. Siboney, Playa, Havana, Cuba. Tel.: +537-272-1670 E-mail: [email protected] Edel Garc´ıa-Reyes 7a. #218 b/ 218 and 222, Rpto. Siboney, Playa, Havana, Cuba. Tel.: +537-272-1670 E-mail: [email protected]
2
Dayron Rizo-Rodr´ıguez et al.
1 Introduction Face recognition is an area of image processing and pattern recognition that has been widely extended in a lot of applications because of its social impact. Several sources causing the degradation of the performance of face recognition approaches have been identified. Among them, much research has been devoted to illumination factor since algorithms have shown to be extremely sensitives to lighting variations on face images [1]. One of the ways used by researchers to deal with illumination problems has been the extraction of illumination invariant features from face images. Many illumination invariant face recognition methods have been proposed in the literature. Different mathematical and computational tools have been used in the development of these methods such as statistical analysis [2], transformation into frequency domain [3–5] and the extraction of features derived from the morphological features from the face [6,7]. Most of existing illumination invariant feature extraction algorithms need more than one image per subject to carry out a training phase. In realapplications, it is unlikely to have this requirement, so the effectiveness of applying those methods decreases dramatically. Aiming at using only one training image, a quaternion-based correlation filter for face recognition was proposed in [8]. First, a two-level discrete wavelet decomposition (DWT) is applied to obtain a quaternionic representation of face image features. Then, a quaternion-based correlation filter is designed from this representation. Very good results were achieved in face identification experiments [8]. A comparison between the employment of complex and quaternionic representations on face recognition was presented in [9]. For that purpose, four different feature extraction methods were used. The benefits of using a quaternionic representation were validated in [9], a point that was not well clarified in [8]. Moreover, a comparison of a quaternion-based correlation filter designed from the same features extraction approaches is presented in [10]. In face recognition experiments, the one designed from local binary patterns (LBP) achieved the best results. Therefore, the use of a feature extraction method with a well-known behavior in front of illumination variations in face recognition can improve the performance of quaternion-based correlation filters [10]. The unconstrained optimal trade-off quaternion filter (UOTQF) is used in both [8] and [10]. Based on the traditional unconstrained optimal tradeoff filter (UOTF), the UOTQF is used to perform the cross-correlation of two quaternionic representations. Good results are achieved using it, but on the other hand, it arises the possibility of evaluating other correlation filters. Precisely, this is the aim of this work, keeping the condition of using only one training face image per person, to evaluate other correlation filters in quaternionic domain. A phase only quaternion filter (POQF) and a separable trade-off quaternion filter (STOQF) were selected for this purpose. In order to compare their illumination invariant and discriminative properties, verification and identification experiments were conducted on XM2VTS and Extended Yale B databases respectively. The POQF constructed from
Face Recognition Using Quaternion-Based Correlation Filters
3
LBP features showed the best performance in both face recognition experiments. The rest of this paper is organized as follows. In Section 2, the generalization of the face recognition process using quaternion-based correlation filters is explained. In Section 3, the quaternion-based correlation filters incorporated to face recognition process are described. Experimental results are shown in Section 4. Finally, Section 5 gives the conclusions of the paper.
2 Generalization of Face Recognition Process In this section, a brief summary of the quaternion theory is initially presented. Due to the fact that quaternion numbers have been widely addressed, only needed properties are described. Next, a generalization of the face recognition process using quaternion-based correlation filters is given.
2.1 Quaternions Quaternion algebra was discovered by Sir William Rowan Hamilton in 1843 [11]. A quaternion is composed by a real part and an imaginary part consisting of three orthogonal components. The cartesian representation of quaternion numbers would be as follows: q = a + bi + cj + dk
(1)
where a, b, c, d are real numbers and i, j, k are imaginary operators subject to the following restrictions: i2 = j 2 = k 2 = −1 ij = k, jk = i, ki = j, ji = −k, kj = −i, ik = −j. Since the imaginary operators are mutually orthogonal, they define a 3-D imaginary space. Thus, they can be considered hypercomplex numbers [12]. Quaternion conjugate (q), modulus (|q|) and inverse (q −1 ), are given by: q = a − bi − cj − dk √ |q| = a2 + b2 + c2 + d2 and q −1 =
q . |q 2 |
(2) (3) (4)
A pure quaternion has a zero real part (a = 0), a f ull quaternion has nonzero real part and a unit quaternion has a unit modulus. Another way of expressing a quaternion number is considering it as a composition of a scalar(S) and a pure quaternion part(V ), which is represented as [12]: q = S(q) + V (q) where S(q) = a and V (q) = bi + cj + dk by analogy with Eq.(1).
(5)
4
Dayron Rizo-Rodr´ıguez et al.
Being q1 = a1 + b1i + c1j + d1k and q2 = a2 + b2i + c2j + d2k two quaternion numbers represented throughout the cartesian expression defined in Eq.(1), addition of q1 and q2 is a component-wise operation computed as [13]: q1 + q2 = (a1 + a2) + (b1 + b2)i + (c1 + c2)j + (d1 + d2)k.
(6)
The product of q1 and q2 would be expressed as follows [14]: q1q2 = (a1a2 − b1b2 − c1c2 − d1d2) + (a1b2 + b1a2 + c1d2 − d1c2)i +(a1c2 + c1a2 + d1b2 − b1d2)j + (a1d2 + d1a2 + b1c2 − c1b2)k. (7) It can be appreciated from Eq.(7) that the product of quaternions is not commutative because reversing the order when multiplying the imaginary components inverts the sign of the product of those components. For this reason, in case of multiplication of pure quaternions the product is anti-commutative [13]. Therefore, the quaternion discrete fourier transform (QDFT) has been defined in two ways, left-side QDFT (QFDTL ) and rightside QDFT (QFDTR ). Both of them are used to transform a quaternion number into frequency domain. QFDTL and QFDTR are defined as [14]: QF DT L (p, s) = √
M −1 N −1 ∑ ∑ 1 e−2µπ((pm/M )+(sn/N )) q(m, n) M N m=0 n=0
(8)
M −1 N −1 ∑ ∑ 1 q(m, n)e−2µπ((pm/M )+(sn/N )) M N m=0 n=0
(9)
QF DT R (p, s) = √
where µ is any unit pure quaternion and q is a quaternion number. To obtain the inverse quaternion discrete fourier transform (IQDFT) of QFDTL and QFDTR , a change of sign from negative to positive is performed in the exponential expressions of Eq.(8) and Eq.(9) [14]. IQFDTL and IQFDTR would be then expressed as: IQF DT L (m, n) = √
M −1 N −1 ∑ ∑ 1 e2µπ((pm/M )+(sn/N )) Q(p, s) M N m=0 n=0
(10)
M −1 N −1 ∑ ∑ 1 Q(p, s)e2µπ((pm/M )+(sn/N )) M N m=0 n=0
(11)
IQF DT R (m, n) = √
where Q is a quaternion number in frequency domain. Given two pure quaternions p1 and p2, p1 can be decomposed into parallel and perpendicular components to p2 throughout the following expression [12]: p1 = p1∥ + p1⊥
(12)
where p1∥ and p1⊥ are expressed by: p1⊥ =
1 (p1 + p1p2p1), p1⊥ ⊥p2 2
(13)
Face Recognition Using Quaternion-Based Correlation Filters
p1∥ =
1 (p1 − p1p2p1), p1∥ ∥p2 2
5
(14)
A f ull quaternion q, represented using Eq.(5), can be also decomposed into parallel and perpendicular components to a pure one, p, as follows [12]: q⊥ = V⊥ (q)
(15)
q∥ = S(q) + V∥ (q)
(16)
where V⊥ (q) is the component of V (q) perpendicular to p, computed following Eq.(13) and V∥ (q) is the component of V (q) parallel to p, obtained from Eq.(14). Some equivalent hypercomplex cross-correlation forms are presented in [12]. According to the authors, all those forms yielded identical spectra for the same image set in experimental tests. In this paper, one of them is assumed, which is defined as follows: C(f, g) = IQDF T L (QDF T∥L (f )QDF T R (g) +QDF T⊥L (f )IQDF T R (g))
(17)
where f and g are f ull quaternion numbers and the unit pure quaternion µ = (1, 1, 1) is used to obtain parallel and perpendicular components.
2.2 Generalized Schemes Based on Eq.(1), the general expression used to represent face images throughout the structure of quaternion numbers is defined as [10] : q(x, y) = Q1 (x, y) + Q2 (x, y)i + Q3 (x, y)j + Q4 (x, y)k
(18)
where Q1 (x, y), Q2 (x, y), Q3 (x, y) and Q4 (x, y) would be four descriptions of the image at (x, y) coordinate. Hence, it is necessary to decompose the image into four bands of information in order to construct the quaternionic representation in Eq.(18). These descriptions can be obtained by applying any face feature extraction method. We are then including the generalization of the feature extraction method [10]. However, the same quaternion-based correlation filter was used in [8] and [10] to carry out face recognition task. Due to the fact that a lot of correlation filters have been defined in the literature, the possibility of integrating some of them is noticed. Therefore, in this work the generalization of the quaternionbased correlation filter to be used is proposed. In Figure 1, a generalized face recognition process using quaternion-based correlation filters is presented in two schemes. First scheme, a), presents the enrollment of face images from the use of quaternion-based correlation filters. The second one, b), shows the recognition stage of a new target face image using this approach.
6
Dayron Rizo-Rodr´ıguez et al.
Fig. 1 Schemes of face recognition phases: a) enrollment and b) recognition.
3 Quaternion-Based Correlation Filters As it was stated in the introductory section of this paper, most of existing illumination invariant face recognition methods have a practical drawback: they need more than one training image per subject, captured under different lighting conditions, in order to achieve a higher performance. The design of correlation filters in quaternionic domain has been proposed as one of the ways to follow in order to overcome this problem. Correlation filters have probed to be a useful tool for object recognition in image processing [15]. They have been largely used in this area because of their profitable contributions. First, correlation filters can be designed to obtain noise tolerance and good discriminative power. Moreover, they inherently provide graceful degradation facing occlusions, noise, etc. Furthermore, their most important property is the built-in shift invariance in the filtering operation [17]. In object recognition tasks, including face recognition, a well-designed correlation filter should yield large peak values in correlation output plane from the true identity and no discernible peaks for other people [8]. Depending on the method used to design a correlation filter, its capacity to identify objets in an image can be affected [15]. In [15], correlation filters are divided into two groups with respect to the size of the training set to be employed. The first group is composed by filters that are designed only from one training object. In the second one, those filters designed from more than one training object are assembled. Following the ideas above, correlation filters of the first group would be more suitable for our purpose. Among them, the phase only correlation filter (POF) is selected because it has been widely applied in object recognition [18]. On the the other hand, in spite of belonging to the second group, the separable trade-off filter (STOF) is also evaluated since it shows a very good performance when recognizing face images [15].
Face Recognition Using Quaternion-Based Correlation Filters
7
In this section, POF and STOF are described and expressed in quaternionic domain. In order to establish comparison with the experimental results presented in [10], the unconstrained optimal trade-off filter (UOTF) is analyzed as well.
3.1 Phase Only Quaternion Correlation Filter Phase only correlation filter (POF) was proposed in [19]. This method is based on the fact that the phase contains information relating to the displacement between two images of the same object [18]. Thus, the phase difference of both images transformed into frequency domain using discrete fourier transform is computed. Then, a sharp peak is generated at the position corresponding to the spatial shift when executing the inverse discrete fourier transform operation to this phase difference [20]. Some advantages of phase correlation over other correlation methods are presented in [18]. First, it is relatively scene-independent and it is insensitive to narrow bandwidth noise and convolutional image degradations. Moreover, it is invariant to changes in the intensity of the image caused by scaling or level shifting. So, it is suitable in environments where the light level is not easily controlled [18], which is an important property for the purpose of this work. Furthermore, phase correlation produces a sharper correlation peak than cross-correlation [22]. The hypercomplex generalization of POF was proposed in [21]. In that work quaternion numbers were used and the generalization of POF to hypercomplex numbers was named as phase only quaternion correlation filter (POQF). Mathematically, POQF is defined as [20]: ( ) R(f, g) P (f, g) = IQDF T L (19) |R(f, g)| where R(f, g) = QDF T L (f )QDF T∥R (g) +IQDF T L (f )QDF T⊥R (g)
(20)
where f and g are f ull quaternion numbers and the unit pure quaternion µ = (1, 1, 1) is used to obtain parallel and perpendicular components.
3.2 Separable Trade-Off Quaternion Filter A separable trade-off filter (STOF) was proposed in [15]. It is based on a design methodology of separable filters but with the same structure of tradeoff filters. Therefore, this filter can be considered as a hybrid between a separable and a trade-off filter. The general case of STOF is calculated by [15]: h = T −1 X(X + T −1 X)−1 X (21)
8
Dayron Rizo-Rodr´ıguez et al.
where X is a column vector formed with the column of the discrete fourier transform of each training image, + indicates the conjugated transpose , T is a diagonal matrix with the sum of the columns of the power spectral density of the training images plus the columns of the matrix C, which is the power spectral density of noise with zero mean. T is defined as: } { N ∑ 2 Tj,j = (1 − β) |Xi (j)| + βC(j) (22) i=1
where β is the balance parameter of STOF. To obtain a separable trade-off quaternion filter (STOQF), all vector and matrix terms of Eq.(21) and Eq.(22) are quaternion number arrays. Moreover, mathematical operations in both equations are executed in quaternionic domain. Results obtained in the experimental section of [15] showed the effectiveness of STOF when recognizing images captured under different angles of the face.
3.3 Unconstrained Optimal Trade-Off Quaternion Filter The unconstrained optimal trade-off quaternion filter (UOTQF), was proposed in [8]. The derivation of this filter is similar to the one of the traditional unconstrained optimal trade-off filter (UOTF) [23], and has the following closed form solution: h = γ(αD + C)−1 m (23) where h is the designed frequency domain filter represented in the vector form, m represents the frequency domain training image in the vector form, α and γ are trade-off parameters, which can be tuned to obtain the optimal trade-off between maximizing the discrimination ability and minimizing the output noise variance of the filter h. D is a diagonal matrix, where the main diagonal is the average power spectrum of the training images and C is also a diagonal matrix, representing the noise power spectral density. Typically a white noise model is assumed, thus C takes the form of an identity matrix. In this formulation of the UOTQF, all vector and matrix terms are quaternion number arrays, while for the UOTF they are all complex number arrays. In spite of the fact that trade-off filters belong to the group that need more than one training image per subject, in [8] excellent face recognition results were achieved when designing a UOTQF from only one training image with good lighting conditions. Moreover, it was used in [10] in order to determine the best feature extraction method to obtain a quaternionic representation for face recognition.
4 Experimental Evaluation Face recognition results achieved when using each quaternion-based correlation filter described in Section 3 are presented in this section. Every filter is evaluated considering the face recognition scheme presented in Figure
Face Recognition Using Quaternion-Based Correlation Filters
9
1. To construct the quaternionic representation in Eq.(18), the four image decompositions based on different face images representations, described in [10], were used: discrete wavelet decomposition (DWT), image differentiation (DIF), discrete cosine transform (DCT) and local binary patterns (LBP). For example, the LBP operator at four different radii, from one to four pixels, is computed to obtain LBP quaternionic representation. The filters STOQF and UOTQF are stored as features in the enrollment phase and then in the recognition phase, those features and target image are cross-correlated using Eq.(17). In the case of POQF, in the enrollment phase the QDFT of the training image is stored and then, in the recognition phase, it is correlated with target image using Eq.(19). Moreover, in order to conduct a deeper analysis, results of raw quaternion cross-correlation of face images (RQCC), i.e., without using a quaternion-based correlation filter in enrollment phase, are also given. Verification and identification experiments were conducted in XM2VTS [24] and Extended Yale B [25] databases respectively. In both experiments, quaternion-based correlation filters were designed using only one training image per subject.
4.1 Verification experiment Designed for experiments on XM2VTS database, Configuration I of the Lausanne protocol [24] was used to compare the performance of the different quaternion-based correlation filters on a face verification setting. Under this configuration, the 2360 face images of 295 subject on the database, are divided into a Training, an Evaluation and a Test sets, composed of images under controlled illumination conditions used as clients and imposters. An additional set (Dark) which contains images of every subject under non regular lighting conditions is used to test the behavior of the methods in the presence of this kind of variations. In a verification setup, the probability of a false match is estimated by the observed False Acceptance Rate (FAR). For an experiment of N attempted impostor matches, FAR is computed as the ratio of accepted matches with respect to the total number of trials, F AR = naccept /N , and it is often expressed in percent [13]. Nevertheless, the opposite error is also important. The False Rejection Rate (FRR) for an experiment of N attempted client matches is then F RR = nreject /N [13]. FRR may be important when we want that the system incorrectly rejects as few authorized users as possible. The Equal Error Rate(ERR) is often used as a one-dimensional measure of the accuracy of a verification system. This is the ratio at which the FAR and FRR are the same. In practice, biometric systems do not operate at this point, but is supposed that the lower the EER the lower the error rates. Under the selected protocol, the similarity value obtained by the classification method at this point in the Evaluation set is used as a threshold for the decision of acceptance or rejection in the Test and Dark sets. The Total Error Rate (TER), which is the sum of FRR and FAR, is computed for each set of the database when applying the quaternion-based
10
Dayron Rizo-Rodr´ıguez et al.
Table 1 Verification Results in terms of TER (%). a) DWT DIF DCT LBP c) DWT DIF DCT LBP
Eval. 42.67 34.00 58.00 36.33 Eval. 33.00 23.65 59.33 21.85
RQCC Test 34.55 32.24 59.33 31.17 STOQF Test 27.96 20.15 57.34 18.56
b) Dark 51.22 49.08 76.27 58.06 Dark 50.19 41.11 76.84 40.08
DWT DIF DCT LBP d) DWT DIF DCT LBP
Eval. 32.60 23.55 58.33 22.00 Eval. 31.92 23.17 57.67 21.28
UTOQF Test 27.05 20.64 55.68 18.56 POQF Test 27.73 19.76 54.99 16.86
Dark 49.87 41.54 75.90 40.68 Dark 50.38 39.36 75.57 37.88
correlation filters. Obtained results are shown in Table 1. This table is divided into four portions in order to provide a better understanding of our experimental results: a) RQCC, b) UTOQF, c) STOQF and d) POQF. It can be appreciated in Table 1, that face verification is more accurate when using any quaternion-based correlation filter than when it is performed with RQCC. This fact supports the inclusion of quaternion-based correlation filters in our face recognition scheme, in Figure 1. Moreover, the superiority of POQF can be seen in this table. POQF outperforms the other two quaternion-based correlation filters in all evaluated subsets. In fact, the difference is noticeable in Dark set, so it is confirmed the good performance of this filter when dealing with image captured under uncontrolled illumination conditions. Additionally, it can be seen that, since UTOQF and STOQF belong to the group of filters designed from more than one training image and POQF is member of the group of filters that need only one training image, POQF is better when having the restriction of using only one training image. Furthermore, as it was stated in [10], it can be verified in Table 1 that LBP was the best feature extraction method in all cases. Finally, it can be concluded that the conjugation of POQF as quaternion-based correlation filter and LBP as feature extraction approach presented the best performance overall.
4.2 Identification experiment The Extended Yale B [25] database was used for face identification experiments. It contains images of 38 subjects seen under 64 different illumination conditions, in which the angle between the light source direction and the camera axis was changed each time, in a way that the larger the angle, the more unfavorable the lighting conditions are. Based on the angle of the incident illumination, this database is usually divided into 6 subsets. Images with frontal angles were used as gallery and the others, were divided into 5 subsets according to the angle of the incident light in the following way: S1 contains 225 images with angles between 0 - 120 , S2 is composed by 456 images with 130 - 250 angles, S3 have 525 images with
Face Recognition Using Quaternion-Based Correlation Filters
11
Table 2 Recognition Rates (%) obtained in Identification Experiment. a) DWT DIF DCT LBP b) DWT DIF DCT LBP c) DWT DIF DCT LBP d) DWT DIF DCT LBP
S1 98.22 100.0 59.11 100
S2 100.0 100.0 58.33 100.0
S1 98.22 100.0 59.11 99.56
S2 100.0 100.0 58.33 100.0
S1 98.22 100.0 56.89 100.0
S2 100.0 100.0 59.65 100.0
S1 98.22 100.0 59.11 100.0
S2 100.0 100.0 58.33 100.0
RQCC S3 73.71 99.24 19.43 97.71 UTOQF S3 73.71 99.24 19.42 97.71 STOQF S3 73.33 99.05 19.05 97.71 POQF S3 73.71 99.05 19.43 97.71
S4 43.64 96.59 16.66 94.49
S5 10.10 56.01 30.78 81.27
S4 44.73 97.15 16.66 95.83
S5 11.92 59.07 30.60 82.92
S4 42.32 96.73 13.82 96.49
S5 11.57 58.70 27.94 83.10
S4 42.76 97.59 16.89 96.50
S5 11.92 59.43 30.60 83.74
angles between 260 - 500 , 456 images with angles between 510 - 700 are in S4 and S5 contains 562 images with angles between 710 - 1300 . Therefore, S1 contains images with minor illumination variations and S5 the most affected ones. Since we are less concerned with the classification step, in all cases a nearest neighbourhood classifier is employed in the case of identification. Recognition rates obtained in each subset of the database using the quaternion-based correlation filters described in previous sections and the RQCC are shown in Table 2, for the first place in the candidate list. Moreover, the cumulative match score vs. rank curve for S5, is presented in Figure 2. It illustrates the behavior of the four quaternion-based correlation filters in the most difficult subset. Some of the conclusions drawn when analyzing Table 1 can be also extracted from Table 2. First, the results of RQCC are worse than those achieved when using quaternion-based correlation filters. Furthermore, POQF presents better identification rates than UTOQF and STOQF in general. However, in all cases the use of DIF as feature extraction method presents better results than the other three methods in the subsets S1, S2, S3 and S4. In these subsets the difference between DIF and LBP is very small. Nevertheless, in S5 subset the LBP method performs much better, achieving more than 30% of difference in correct classification. In Figure 2, it is also confirmed above conclusion. In the four graphics, LBP-based approach out-
12
Dayron Rizo-Rodr´ıguez et al.
Fig. 2 Cumulative match score vs. rank curve for S5.
performs the others in all ranks. In addition, it was the only method that achieved more than 90% of correct classification in this subset. According to this analysis, it can be reaffirmed that POQF and LBP constitute the best combination.
5 Conclusions and Future Works In this paper a comparison of quaternion-based correlation filters for illumination invariant face recognition is presented. As it was proposed in [9], quaternionic representations of face image features, based on DIF, DWT, DCT and LBP, are constructed. On the other hand, following the face recognition process used in [8], a quaternion-based correlation filter is designed from only one training image per subject and it is cross-correlated with a target image represented in quaternion frequency domain. As a result, a generalized face recognition process using quaternion-based correlation filters is proposed. Three quaternion-based correlation filters were compared in our work. The first one was the POQF, which belongs to those filters designed from only one training image per subject and has demonstrated a good behavior facing illumination variation problems in image processing. The STOQF was the second one selected because, in spite of being a member of the group of filters that need more than one training image, it showed a good performance when recognizing face object images [15]. The UOTQF was the third filter evaluated aiming at establishing a comparison with experimental results presented in [10] and because of excellent results were achieved when a UTOQF was used for face recognition in quaternionic domain [8]. Results
Face Recognition Using Quaternion-Based Correlation Filters
13
obtained when face recognition process is executed without applying any quaternion-based correlation filter, RQCC, were also analyzed. Face verification and identification experiments were conducted. In both cases an improvement of the performance was obtained by including correlation filters in quaternionic domain instead of using RQCC. The best discriminative power of the POQF for face recognition, in front of images affected by uncontrolled lighting conditions under the requirement of being designed from only one training image per subject, was also presented. On the other hand, LBP was confirmed to be the best feature extraction method for our purpose. Finally, it can be concluded that the best face recognition scheme is the one that uses POQF as quaternion-based correlation filter and LBP as feature extraction method. Surprisingly, the quaternionic representation constructed using DCT features, another well-known method for illumination invariant face recognition [26], achieved the worst performance. Probably, the multiresolution analysis based on this method and the way in which the DCT coefficients are discarded to form the quaternionic representation need to be improved. For the continuity of this work, a better quaternion representation based on DCT will be considered, in order to improve face recognition results. References 1. Adini, Y., Moses, Y., Ullman, S.: Face Recognition: The Problem of Compensating for Changes in Illumination Direction. IEEE Trans. Pattern Anal. Machine Intell 19, Issue 7 (1997) 2. Belhumeur, P., Pentland, J., Kriegman, D.: Eigenfaces vs. fisherfaces: Recognition using class specific linear projection. IEEE Trans. Pattern Anal. Machine Intell 19, 711–720 (1997) 3. Hafed, Z.M., Levine, M.D.: Face recognition using the discrete cosine transform. Int. J. Comput. Vision 43, 167–188 (2001) 4. Savvides, M., Vijayakumar, B., Khosla, P.: Corefaces-robust shift invariant pca based correlation filter for illumination tolerant face recognition. In: IEEE CVPR04, June 26-July 2nd, 834–841. Washington DC, USA (2004) 5. Qing, L., Shan, S., Chen, X., Gao, W.: Face recognition under varying lighting based on the probabilistic model of gabor phase. In: IEEE ICPR2006, Aug 20-24, 1139–1142. San Antonio, Texas, USA (2006) 6. Pujol, A.: Contributions to shape and texture face similarity measurement. PhD thesis, Universitat Autnoma de Barcelona (2001) 7. Ahonen, T., Hadid, A., Pietikainen, M.: Face recognition with local binary patterns. In: 8th European Conference on Computer Vision, May 11-14, 469–481. Prague, Czech Republic (2004) 8. Chunyan, X., Savvides, M., Vijayakumar, B.: Quaternion correlation filters for face recognition in wavelet domain. In: IEEE ICASSP2005, March 18-23, 85–88. Philadelphia, PA, USA (2005) 9. Rizo-Rodriguez, D., Mendez-Vazquez, H., Garcia-Reyes, E.: Illumination invariant face image representation using quaternions. In: CIARP 2010, November 8-11, 434–441. Sao Paulo, Brazil (2010) 10. Rizo, D., Mendez, H., Garcia, E., San Martin, C., Meza, P.: Quaternion Correlation Filters for Illumination Invariant Face Recognition. In: CIARP 2011, November 15-18, 467–474. Pucon, Chile (2011) 11. Hamilton, W.R.: Elements of Quaternions. Longmans, Green, London, UK (1866) 12. Moxey, C., Sangwine, S., Ell, S.: Hypercomplex correlation techniques for vector images. IEEE Transaction on Signal Processing 51, 1941–1953 (2003)
14
Dayron Rizo-Rodr´ıguez et al.
13. Jones III, C. F.: Color Face Recognition using Quaternionic Gabor Filters. PhD thesis, Virginia Polytechnic Institute and State University (2004) 14. Jones III, C. F.: Quaternions et Algbres Gomtriques, de nouveaux outils pour les images. PhD thesis, Universit de Poitiers (2006) 15. San Martin, C., Vargas, A., Campos, J., Torres, S.: Image pattern recognition with separable trade-off correlation filters. In: 7th ACIVS, September 20-23, 162–169. Antwerp, Belgium (2005) 16. Refregier, Ph.: Optimal trade-off filters for noise robustness, sharpness of the correlation peak, and Horner efficiency. Optics Letters 16, 829–831 (1991) 17. Vijaya Kumar, B. V. K., Savvides, M., Chunyan, X., Venkataramani, K., Thornton, J., Mahalanobis, A.: Biometric Verification with Correlation Filters. Applied Optics 43, 391–402 (2004) 18. Thornton, A.: Colour object recognition using a complex colour representation and the frequency domain. PhD thesis, University of Reading, UK (1998) 19. Kuglin, C. D., Hines, D. C.: The phase correlation image alignment method. In: Int. Conf Cybernetics and Society, September 1975, 163–165. NY, USA (1975) 20. Moxey, C. E., Sangwine, S. J., Ell, T. A.: Color-grayscale image registration using hypercomplex phase correlation. In: IEEE ICIP, September 22-25, 385– 388. Rochester, NY, USA (2002) 21. Sangwine, S. J., Ell, T. A., Moxey, C. E.: Vector phase correlation. Electronics Letters 37, 1513–1515 (2001) 22. Horner, J. L., Gianino, P. D.: Phase-Only Matched Filtering. Applied Optics 23, 812–816 (1984) 23. Vijayakumar, B., Savvides, K., Venkataramani, K., Chunyan, X.: Spatial frequency domain image processing for biometric recognition. In: IEEE ICIP, September 22-25, 53–56. Rochester, NY, USA (2002) 24. Messer, K., Matas, J., Kittler, J., Jonsson, K.: Xm2vtsdb: The extended m2vts database. In: AVBPA 1999, March 2223, 72–77. Washington, D.C., USA (1999) 25. Lee, K.C., Ho, J., Kriegman, D.J.: Acquiring linear subspaces for face recognition under variable lighting. IEEE Trans. Pattern Anal. Machine Intell 27, 684–698 (2005) 26. Chen, W., Er, M.J., Wu, S.: Illumination compensation and normalization for robust face recognition using discrete cosine transform in logarithm domain. Systems, Man and Cybernetics, Part B, IEEE Transactions 36, 458–466 (2006)
Dayron Rizo-Rodr´ıguez · Heydi M´ endez-V´ azquez · Edel Garc´ıa-Reyes
Illumination Invariant Face Recognition Using Quaternion-Based Correlation Filters Received: date / Revised: date
Abstract Existing face recognition systems decrease their performance when face images are affected by lighting variations. Recently, several quaternionic representations of face image features and a quaternion-based correlation filter have been combined in order to cope with the effects of having nonproperly illuminated face images. In this paper, a generalized scheme for face recognition using quaternion-based correlation filters is described. Furthermore, a comparison by using different quaternion-based correlation filters in this scheme is presented. Three different quaternion-based correlation filters are designed and conjugated with four face feature extraction methods aiming at obtaining the best combination. Verification and identification experiments confirms that when combining a quaternionic representation with a quaternion-based correlation filter, both with good discriminative power and illumination invariant properties, an improvement in face recognition accuracy is obtained. Keywords quaternions · correlation filters · illumination invariant face recognition Dayron Rizo-Rodr´ıguez 7a. #218 b/ 218 and 222, Rpto. Siboney, Playa, Havana, Cuba. Tel.: +537-272-1670 E-mail: [email protected] Heydi M´endez-V´ azquez 7a. #218 b/ 218 and 222, Rpto. Siboney, Playa, Havana, Cuba. Tel.: +537-272-1670 E-mail: [email protected] Edel Garc´ıa-Reyes 7a. #218 b/ 218 and 222, Rpto. Siboney, Playa, Havana, Cuba. Tel.: +537-272-1670 E-mail: [email protected]
2
Dayron Rizo-Rodr´ıguez et al.
1 Introduction Face recognition is an area of image processing and pattern recognition that has been widely extended in a lot of applications because of its social impact. Several sources causing the degradation of the performance of face recognition approaches have been identified. Among them, much research has been devoted to illumination factor since algorithms have shown to be extremely sensitives to lighting variations on face images [1]. One of the ways used by researchers to deal with illumination problems has been the extraction of illumination invariant features from face images. Many illumination invariant face recognition methods have been proposed in the literature. Different mathematical and computational tools have been used in the development of these methods such as statistical analysis [2], transformation into frequency domain [3–5] and the extraction of features derived from the morphological features from the face [6,7]. Most of existing illumination invariant feature extraction algorithms need more than one image per subject to carry out a training phase. In realapplications, it is unlikely to have this requirement, so the effectiveness of applying those methods decreases dramatically. Aiming at using only one training image, a quaternion-based correlation filter for face recognition was proposed in [8]. First, a two-level discrete wavelet decomposition (DWT) is applied to obtain a quaternionic representation of face image features. Then, a quaternion-based correlation filter is designed from this representation. Very good results were achieved in face identification experiments [8]. A comparison between the employment of complex and quaternionic representations on face recognition was presented in [9]. For that purpose, four different feature extraction methods were used. The benefits of using a quaternionic representation were validated in [9], a point that was not well clarified in [8]. Moreover, a comparison of a quaternion-based correlation filter designed from the same features extraction approaches is presented in [10]. In face recognition experiments, the one designed from local binary patterns (LBP) achieved the best results. Therefore, the use of a feature extraction method with a well-known behavior in front of illumination variations in face recognition can improve the performance of quaternion-based correlation filters [10]. The unconstrained optimal trade-off quaternion filter (UOTQF) is used in both [8] and [10]. Based on the traditional unconstrained optimal tradeoff filter (UOTF), the UOTQF is used to perform the cross-correlation of two quaternionic representations. Good results are achieved using it, but on the other hand, it arises the possibility of evaluating other correlation filters. Precisely, this is the aim of this work, keeping the condition of using only one training face image per person, to evaluate other correlation filters in quaternionic domain. A phase only quaternion filter (POQF) and a separable trade-off quaternion filter (STOQF) were selected for this purpose. In order to compare their illumination invariant and discriminative properties, verification and identification experiments were conducted on XM2VTS and Extended Yale B databases respectively. The POQF constructed from
Face Recognition Using Quaternion-Based Correlation Filters
3
LBP features showed the best performance in both face recognition experiments. The rest of this paper is organized as follows. In Section 2, the generalization of the face recognition process using quaternion-based correlation filters is explained. In Section 3, the quaternion-based correlation filters incorporated to face recognition process are described. Experimental results are shown in Section 4. Finally, Section 5 gives the conclusions of the paper.
2 Generalization of Face Recognition Process In this section, a brief summary of the quaternion theory is initially presented. Due to the fact that quaternion numbers have been widely addressed, only needed properties are described. Next, a generalization of the face recognition process using quaternion-based correlation filters is given.
2.1 Quaternions Quaternion algebra was discovered by Sir William Rowan Hamilton in 1843 [11]. A quaternion is composed by a real part and an imaginary part consisting of three orthogonal components. The cartesian representation of quaternion numbers would be as follows: q = a + bi + cj + dk
(1)
where a, b, c, d are real numbers and i, j, k are imaginary operators subject to the following restrictions: i2 = j 2 = k 2 = −1 ij = k, jk = i, ki = j, ji = −k, kj = −i, ik = −j. Since the imaginary operators are mutually orthogonal, they define a 3-D imaginary space. Thus, they can be considered hypercomplex numbers [12]. Quaternion conjugate (q), modulus (|q|) and inverse (q −1 ), are given by: q = a − bi − cj − dk √ |q| = a2 + b2 + c2 + d2 and q −1 =
q . |q 2 |
(2) (3) (4)
A pure quaternion has a zero real part (a = 0), a f ull quaternion has nonzero real part and a unit quaternion has a unit modulus. Another way of expressing a quaternion number is considering it as a composition of a scalar(S) and a pure quaternion part(V ), which is represented as [12]: q = S(q) + V (q) where S(q) = a and V (q) = bi + cj + dk by analogy with Eq.(1).
(5)
4
Dayron Rizo-Rodr´ıguez et al.
Being q1 = a1 + b1i + c1j + d1k and q2 = a2 + b2i + c2j + d2k two quaternion numbers represented throughout the cartesian expression defined in Eq.(1), addition of q1 and q2 is a component-wise operation computed as [13]: q1 + q2 = (a1 + a2) + (b1 + b2)i + (c1 + c2)j + (d1 + d2)k.
(6)
The product of q1 and q2 would be expressed as follows [14]: q1q2 = (a1a2 − b1b2 − c1c2 − d1d2) + (a1b2 + b1a2 + c1d2 − d1c2)i +(a1c2 + c1a2 + d1b2 − b1d2)j + (a1d2 + d1a2 + b1c2 − c1b2)k. (7) It can be appreciated from Eq.(7) that the product of quaternions is not commutative because reversing the order when multiplying the imaginary components inverts the sign of the product of those components. For this reason, in case of multiplication of pure quaternions the product is anti-commutative [13]. Therefore, the quaternion discrete fourier transform (QDFT) has been defined in two ways, left-side QDFT (QFDTL ) and rightside QDFT (QFDTR ). Both of them are used to transform a quaternion number into frequency domain. QFDTL and QFDTR are defined as [14]: QF DT L (p, s) = √
M −1 N −1 ∑ ∑ 1 e−2µπ((pm/M )+(sn/N )) q(m, n) M N m=0 n=0
(8)
M −1 N −1 ∑ ∑ 1 q(m, n)e−2µπ((pm/M )+(sn/N )) M N m=0 n=0
(9)
QF DT R (p, s) = √
where µ is any unit pure quaternion and q is a quaternion number. To obtain the inverse quaternion discrete fourier transform (IQDFT) of QFDTL and QFDTR , a change of sign from negative to positive is performed in the exponential expressions of Eq.(8) and Eq.(9) [14]. IQFDTL and IQFDTR would be then expressed as: IQF DT L (m, n) = √
M −1 N −1 ∑ ∑ 1 e2µπ((pm/M )+(sn/N )) Q(p, s) M N m=0 n=0
(10)
M −1 N −1 ∑ ∑ 1 Q(p, s)e2µπ((pm/M )+(sn/N )) M N m=0 n=0
(11)
IQF DT R (m, n) = √
where Q is a quaternion number in frequency domain. Given two pure quaternions p1 and p2, p1 can be decomposed into parallel and perpendicular components to p2 throughout the following expression [12]: p1 = p1∥ + p1⊥
(12)
where p1∥ and p1⊥ are expressed by: p1⊥ =
1 (p1 + p1p2p1), p1⊥ ⊥p2 2
(13)
Face Recognition Using Quaternion-Based Correlation Filters
p1∥ =
1 (p1 − p1p2p1), p1∥ ∥p2 2
5
(14)
A f ull quaternion q, represented using Eq.(5), can be also decomposed into parallel and perpendicular components to a pure one, p, as follows [12]: q⊥ = V⊥ (q)
(15)
q∥ = S(q) + V∥ (q)
(16)
where V⊥ (q) is the component of V (q) perpendicular to p, computed following Eq.(13) and V∥ (q) is the component of V (q) parallel to p, obtained from Eq.(14). Some equivalent hypercomplex cross-correlation forms are presented in [12]. According to the authors, all those forms yielded identical spectra for the same image set in experimental tests. In this paper, one of them is assumed, which is defined as follows: C(f, g) = IQDF T L (QDF T∥L (f )QDF T R (g) +QDF T⊥L (f )IQDF T R (g))
(17)
where f and g are f ull quaternion numbers and the unit pure quaternion µ = (1, 1, 1) is used to obtain parallel and perpendicular components.
2.2 Generalized Schemes Based on Eq.(1), the general expression used to represent face images throughout the structure of quaternion numbers is defined as [10] : q(x, y) = Q1 (x, y) + Q2 (x, y)i + Q3 (x, y)j + Q4 (x, y)k
(18)
where Q1 (x, y), Q2 (x, y), Q3 (x, y) and Q4 (x, y) would be four descriptions of the image at (x, y) coordinate. Hence, it is necessary to decompose the image into four bands of information in order to construct the quaternionic representation in Eq.(18). These descriptions can be obtained by applying any face feature extraction method. We are then including the generalization of the feature extraction method [10]. However, the same quaternion-based correlation filter was used in [8] and [10] to carry out face recognition task. Due to the fact that a lot of correlation filters have been defined in the literature, the possibility of integrating some of them is noticed. Therefore, in this work the generalization of the quaternionbased correlation filter to be used is proposed. In Figure 1, a generalized face recognition process using quaternion-based correlation filters is presented in two schemes. First scheme, a), presents the enrollment of face images from the use of quaternion-based correlation filters. The second one, b), shows the recognition stage of a new target face image using this approach.
6
Dayron Rizo-Rodr´ıguez et al.
Fig. 1 Schemes of face recognition phases: a) enrollment and b) recognition.
3 Quaternion-Based Correlation Filters As it was stated in the introductory section of this paper, most of existing illumination invariant face recognition methods have a practical drawback: they need more than one training image per subject, captured under different lighting conditions, in order to achieve a higher performance. The design of correlation filters in quaternionic domain has been proposed as one of the ways to follow in order to overcome this problem. Correlation filters have probed to be a useful tool for object recognition in image processing [15]. They have been largely used in this area because of their profitable contributions. First, correlation filters can be designed to obtain noise tolerance and good discriminative power. Moreover, they inherently provide graceful degradation facing occlusions, noise, etc. Furthermore, their most important property is the built-in shift invariance in the filtering operation [17]. In object recognition tasks, including face recognition, a well-designed correlation filter should yield large peak values in correlation output plane from the true identity and no discernible peaks for other people [8]. Depending on the method used to design a correlation filter, its capacity to identify objets in an image can be affected [15]. In [15], correlation filters are divided into two groups with respect to the size of the training set to be employed. The first group is composed by filters that are designed only from one training object. In the second one, those filters designed from more than one training object are assembled. Following the ideas above, correlation filters of the first group would be more suitable for our purpose. Among them, the phase only correlation filter (POF) is selected because it has been widely applied in object recognition [18]. On the the other hand, in spite of belonging to the second group, the separable trade-off filter (STOF) is also evaluated since it shows a very good performance when recognizing face images [15].
Face Recognition Using Quaternion-Based Correlation Filters
7
In this section, POF and STOF are described and expressed in quaternionic domain. In order to establish comparison with the experimental results presented in [10], the unconstrained optimal trade-off filter (UOTF) is analyzed as well.
3.1 Phase Only Quaternion Correlation Filter Phase only correlation filter (POF) was proposed in [19]. This method is based on the fact that the phase contains information relating to the displacement between two images of the same object [18]. Thus, the phase difference of both images transformed into frequency domain using discrete fourier transform is computed. Then, a sharp peak is generated at the position corresponding to the spatial shift when executing the inverse discrete fourier transform operation to this phase difference [20]. Some advantages of phase correlation over other correlation methods are presented in [18]. First, it is relatively scene-independent and it is insensitive to narrow bandwidth noise and convolutional image degradations. Moreover, it is invariant to changes in the intensity of the image caused by scaling or level shifting. So, it is suitable in environments where the light level is not easily controlled [18], which is an important property for the purpose of this work. Furthermore, phase correlation produces a sharper correlation peak than cross-correlation [22]. The hypercomplex generalization of POF was proposed in [21]. In that work quaternion numbers were used and the generalization of POF to hypercomplex numbers was named as phase only quaternion correlation filter (POQF). Mathematically, POQF is defined as [20]: ( ) R(f, g) P (f, g) = IQDF T L (19) |R(f, g)| where R(f, g) = QDF T L (f )QDF T∥R (g) +IQDF T L (f )QDF T⊥R (g)
(20)
where f and g are f ull quaternion numbers and the unit pure quaternion µ = (1, 1, 1) is used to obtain parallel and perpendicular components.
3.2 Separable Trade-Off Quaternion Filter A separable trade-off filter (STOF) was proposed in [15]. It is based on a design methodology of separable filters but with the same structure of tradeoff filters. Therefore, this filter can be considered as a hybrid between a separable and a trade-off filter. The general case of STOF is calculated by [15]: h = T −1 X(X + T −1 X)−1 X (21)
8
Dayron Rizo-Rodr´ıguez et al.
where X is a column vector formed with the column of the discrete fourier transform of each training image, + indicates the conjugated transpose , T is a diagonal matrix with the sum of the columns of the power spectral density of the training images plus the columns of the matrix C, which is the power spectral density of noise with zero mean. T is defined as: } { N ∑ 2 Tj,j = (1 − β) |Xi (j)| + βC(j) (22) i=1
where β is the balance parameter of STOF. To obtain a separable trade-off quaternion filter (STOQF), all vector and matrix terms of Eq.(21) and Eq.(22) are quaternion number arrays. Moreover, mathematical operations in both equations are executed in quaternionic domain. Results obtained in the experimental section of [15] showed the effectiveness of STOF when recognizing images captured under different angles of the face.
3.3 Unconstrained Optimal Trade-Off Quaternion Filter The unconstrained optimal trade-off quaternion filter (UOTQF), was proposed in [8]. The derivation of this filter is similar to the one of the traditional unconstrained optimal trade-off filter (UOTF) [23], and has the following closed form solution: h = γ(αD + C)−1 m (23) where h is the designed frequency domain filter represented in the vector form, m represents the frequency domain training image in the vector form, α and γ are trade-off parameters, which can be tuned to obtain the optimal trade-off between maximizing the discrimination ability and minimizing the output noise variance of the filter h. D is a diagonal matrix, where the main diagonal is the average power spectrum of the training images and C is also a diagonal matrix, representing the noise power spectral density. Typically a white noise model is assumed, thus C takes the form of an identity matrix. In this formulation of the UOTQF, all vector and matrix terms are quaternion number arrays, while for the UOTF they are all complex number arrays. In spite of the fact that trade-off filters belong to the group that need more than one training image per subject, in [8] excellent face recognition results were achieved when designing a UOTQF from only one training image with good lighting conditions. Moreover, it was used in [10] in order to determine the best feature extraction method to obtain a quaternionic representation for face recognition.
4 Experimental Evaluation Face recognition results achieved when using each quaternion-based correlation filter described in Section 3 are presented in this section. Every filter is evaluated considering the face recognition scheme presented in Figure
Face Recognition Using Quaternion-Based Correlation Filters
9
1. To construct the quaternionic representation in Eq.(18), the four image decompositions based on different face images representations, described in [10], were used: discrete wavelet decomposition (DWT), image differentiation (DIF), discrete cosine transform (DCT) and local binary patterns (LBP). For example, the LBP operator at four different radii, from one to four pixels, is computed to obtain LBP quaternionic representation. The filters STOQF and UOTQF are stored as features in the enrollment phase and then in the recognition phase, those features and target image are cross-correlated using Eq.(17). In the case of POQF, in the enrollment phase the QDFT of the training image is stored and then, in the recognition phase, it is correlated with target image using Eq.(19). Moreover, in order to conduct a deeper analysis, results of raw quaternion cross-correlation of face images (RQCC), i.e., without using a quaternion-based correlation filter in enrollment phase, are also given. Verification and identification experiments were conducted in XM2VTS [24] and Extended Yale B [25] databases respectively. In both experiments, quaternion-based correlation filters were designed using only one training image per subject.
4.1 Verification experiment Designed for experiments on XM2VTS database, Configuration I of the Lausanne protocol [24] was used to compare the performance of the different quaternion-based correlation filters on a face verification setting. Under this configuration, the 2360 face images of 295 subject on the database, are divided into a Training, an Evaluation and a Test sets, composed of images under controlled illumination conditions used as clients and imposters. An additional set (Dark) which contains images of every subject under non regular lighting conditions is used to test the behavior of the methods in the presence of this kind of variations. In a verification setup, the probability of a false match is estimated by the observed False Acceptance Rate (FAR). For an experiment of N attempted impostor matches, FAR is computed as the ratio of accepted matches with respect to the total number of trials, F AR = naccept /N , and it is often expressed in percent [13]. Nevertheless, the opposite error is also important. The False Rejection Rate (FRR) for an experiment of N attempted client matches is then F RR = nreject /N [13]. FRR may be important when we want that the system incorrectly rejects as few authorized users as possible. The Equal Error Rate(ERR) is often used as a one-dimensional measure of the accuracy of a verification system. This is the ratio at which the FAR and FRR are the same. In practice, biometric systems do not operate at this point, but is supposed that the lower the EER the lower the error rates. Under the selected protocol, the similarity value obtained by the classification method at this point in the Evaluation set is used as a threshold for the decision of acceptance or rejection in the Test and Dark sets. The Total Error Rate (TER), which is the sum of FRR and FAR, is computed for each set of the database when applying the quaternion-based
10
Dayron Rizo-Rodr´ıguez et al.
Table 1 Verification Results in terms of TER (%). a) DWT DIF DCT LBP c) DWT DIF DCT LBP
Eval. 42.67 34.00 58.00 36.33 Eval. 33.00 23.65 59.33 21.85
RQCC Test 34.55 32.24 59.33 31.17 STOQF Test 27.96 20.15 57.34 18.56
b) Dark 51.22 49.08 76.27 58.06 Dark 50.19 41.11 76.84 40.08
DWT DIF DCT LBP d) DWT DIF DCT LBP
Eval. 32.60 23.55 58.33 22.00 Eval. 31.92 23.17 57.67 21.28
UTOQF Test 27.05 20.64 55.68 18.56 POQF Test 27.73 19.76 54.99 16.86
Dark 49.87 41.54 75.90 40.68 Dark 50.38 39.36 75.57 37.88
correlation filters. Obtained results are shown in Table 1. This table is divided into four portions in order to provide a better understanding of our experimental results: a) RQCC, b) UTOQF, c) STOQF and d) POQF. It can be appreciated in Table 1, that face verification is more accurate when using any quaternion-based correlation filter than when it is performed with RQCC. This fact supports the inclusion of quaternion-based correlation filters in our face recognition scheme, in Figure 1. Moreover, the superiority of POQF can be seen in this table. POQF outperforms the other two quaternion-based correlation filters in all evaluated subsets. In fact, the difference is noticeable in Dark set, so it is confirmed the good performance of this filter when dealing with image captured under uncontrolled illumination conditions. Additionally, it can be seen that, since UTOQF and STOQF belong to the group of filters designed from more than one training image and POQF is member of the group of filters that need only one training image, POQF is better when having the restriction of using only one training image. Furthermore, as it was stated in [10], it can be verified in Table 1 that LBP was the best feature extraction method in all cases. Finally, it can be concluded that the conjugation of POQF as quaternion-based correlation filter and LBP as feature extraction approach presented the best performance overall.
4.2 Identification experiment The Extended Yale B [25] database was used for face identification experiments. It contains images of 38 subjects seen under 64 different illumination conditions, in which the angle between the light source direction and the camera axis was changed each time, in a way that the larger the angle, the more unfavorable the lighting conditions are. Based on the angle of the incident illumination, this database is usually divided into 6 subsets. Images with frontal angles were used as gallery and the others, were divided into 5 subsets according to the angle of the incident light in the following way: S1 contains 225 images with angles between 0 - 120 , S2 is composed by 456 images with 130 - 250 angles, S3 have 525 images with
Face Recognition Using Quaternion-Based Correlation Filters
11
Table 2 Recognition Rates (%) obtained in Identification Experiment. a) DWT DIF DCT LBP b) DWT DIF DCT LBP c) DWT DIF DCT LBP d) DWT DIF DCT LBP
S1 98.22 100.0 59.11 100
S2 100.0 100.0 58.33 100.0
S1 98.22 100.0 59.11 99.56
S2 100.0 100.0 58.33 100.0
S1 98.22 100.0 56.89 100.0
S2 100.0 100.0 59.65 100.0
S1 98.22 100.0 59.11 100.0
S2 100.0 100.0 58.33 100.0
RQCC S3 73.71 99.24 19.43 97.71 UTOQF S3 73.71 99.24 19.42 97.71 STOQF S3 73.33 99.05 19.05 97.71 POQF S3 73.71 99.05 19.43 97.71
S4 43.64 96.59 16.66 94.49
S5 10.10 56.01 30.78 81.27
S4 44.73 97.15 16.66 95.83
S5 11.92 59.07 30.60 82.92
S4 42.32 96.73 13.82 96.49
S5 11.57 58.70 27.94 83.10
S4 42.76 97.59 16.89 96.50
S5 11.92 59.43 30.60 83.74
angles between 260 - 500 , 456 images with angles between 510 - 700 are in S4 and S5 contains 562 images with angles between 710 - 1300 . Therefore, S1 contains images with minor illumination variations and S5 the most affected ones. Since we are less concerned with the classification step, in all cases a nearest neighbourhood classifier is employed in the case of identification. Recognition rates obtained in each subset of the database using the quaternion-based correlation filters described in previous sections and the RQCC are shown in Table 2, for the first place in the candidate list. Moreover, the cumulative match score vs. rank curve for S5, is presented in Figure 2. It illustrates the behavior of the four quaternion-based correlation filters in the most difficult subset. Some of the conclusions drawn when analyzing Table 1 can be also extracted from Table 2. First, the results of RQCC are worse than those achieved when using quaternion-based correlation filters. Furthermore, POQF presents better identification rates than UTOQF and STOQF in general. However, in all cases the use of DIF as feature extraction method presents better results than the other three methods in the subsets S1, S2, S3 and S4. In these subsets the difference between DIF and LBP is very small. Nevertheless, in S5 subset the LBP method performs much better, achieving more than 30% of difference in correct classification. In Figure 2, it is also confirmed above conclusion. In the four graphics, LBP-based approach out-
12
Dayron Rizo-Rodr´ıguez et al.
Fig. 2 Cumulative match score vs. rank curve for S5.
performs the others in all ranks. In addition, it was the only method that achieved more than 90% of correct classification in this subset. According to this analysis, it can be reaffirmed that POQF and LBP constitute the best combination.
5 Conclusions and Future Works In this paper a comparison of quaternion-based correlation filters for illumination invariant face recognition is presented. As it was proposed in [9], quaternionic representations of face image features, based on DIF, DWT, DCT and LBP, are constructed. On the other hand, following the face recognition process used in [8], a quaternion-based correlation filter is designed from only one training image per subject and it is cross-correlated with a target image represented in quaternion frequency domain. As a result, a generalized face recognition process using quaternion-based correlation filters is proposed. Three quaternion-based correlation filters were compared in our work. The first one was the POQF, which belongs to those filters designed from only one training image per subject and has demonstrated a good behavior facing illumination variation problems in image processing. The STOQF was the second one selected because, in spite of being a member of the group of filters that need more than one training image, it showed a good performance when recognizing face object images [15]. The UOTQF was the third filter evaluated aiming at establishing a comparison with experimental results presented in [10] and because of excellent results were achieved when a UTOQF was used for face recognition in quaternionic domain [8]. Results
Face Recognition Using Quaternion-Based Correlation Filters
13
obtained when face recognition process is executed without applying any quaternion-based correlation filter, RQCC, were also analyzed. Face verification and identification experiments were conducted. In both cases an improvement of the performance was obtained by including correlation filters in quaternionic domain instead of using RQCC. The best discriminative power of the POQF for face recognition, in front of images affected by uncontrolled lighting conditions under the requirement of being designed from only one training image per subject, was also presented. On the other hand, LBP was confirmed to be the best feature extraction method for our purpose. Finally, it can be concluded that the best face recognition scheme is the one that uses POQF as quaternion-based correlation filter and LBP as feature extraction method. Surprisingly, the quaternionic representation constructed using DCT features, another well-known method for illumination invariant face recognition [26], achieved the worst performance. Probably, the multiresolution analysis based on this method and the way in which the DCT coefficients are discarded to form the quaternionic representation need to be improved. For the continuity of this work, a better quaternion representation based on DCT will be considered, in order to improve face recognition results. References 1. Adini, Y., Moses, Y., Ullman, S.: Face Recognition: The Problem of Compensating for Changes in Illumination Direction. IEEE Trans. Pattern Anal. Machine Intell 19, Issue 7 (1997) 2. Belhumeur, P., Pentland, J., Kriegman, D.: Eigenfaces vs. fisherfaces: Recognition using class specific linear projection. IEEE Trans. Pattern Anal. Machine Intell 19, 711–720 (1997) 3. Hafed, Z.M., Levine, M.D.: Face recognition using the discrete cosine transform. Int. J. Comput. Vision 43, 167–188 (2001) 4. Savvides, M., Vijayakumar, B., Khosla, P.: Corefaces-robust shift invariant pca based correlation filter for illumination tolerant face recognition. In: IEEE CVPR04, June 26-July 2nd, 834–841. Washington DC, USA (2004) 5. Qing, L., Shan, S., Chen, X., Gao, W.: Face recognition under varying lighting based on the probabilistic model of gabor phase. In: IEEE ICPR2006, Aug 20-24, 1139–1142. San Antonio, Texas, USA (2006) 6. Pujol, A.: Contributions to shape and texture face similarity measurement. PhD thesis, Universitat Autnoma de Barcelona (2001) 7. Ahonen, T., Hadid, A., Pietikainen, M.: Face recognition with local binary patterns. In: 8th European Conference on Computer Vision, May 11-14, 469–481. Prague, Czech Republic (2004) 8. Chunyan, X., Savvides, M., Vijayakumar, B.: Quaternion correlation filters for face recognition in wavelet domain. In: IEEE ICASSP2005, March 18-23, 85–88. Philadelphia, PA, USA (2005) 9. Rizo-Rodriguez, D., Mendez-Vazquez, H., Garcia-Reyes, E.: Illumination invariant face image representation using quaternions. In: CIARP 2010, November 8-11, 434–441. Sao Paulo, Brazil (2010) 10. Rizo, D., Mendez, H., Garcia, E., San Martin, C., Meza, P.: Quaternion Correlation Filters for Illumination Invariant Face Recognition. In: CIARP 2011, November 15-18, 467–474. Pucon, Chile (2011) 11. Hamilton, W.R.: Elements of Quaternions. Longmans, Green, London, UK (1866) 12. Moxey, C., Sangwine, S., Ell, S.: Hypercomplex correlation techniques for vector images. IEEE Transaction on Signal Processing 51, 1941–1953 (2003)
14
Dayron Rizo-Rodr´ıguez et al.
13. Jones III, C. F.: Color Face Recognition using Quaternionic Gabor Filters. PhD thesis, Virginia Polytechnic Institute and State University (2004) 14. Jones III, C. F.: Quaternions et Algbres Gomtriques, de nouveaux outils pour les images. PhD thesis, Universit de Poitiers (2006) 15. San Martin, C., Vargas, A., Campos, J., Torres, S.: Image pattern recognition with separable trade-off correlation filters. In: 7th ACIVS, September 20-23, 162–169. Antwerp, Belgium (2005) 16. Refregier, Ph.: Optimal trade-off filters for noise robustness, sharpness of the correlation peak, and Horner efficiency. Optics Letters 16, 829–831 (1991) 17. Vijaya Kumar, B. V. K., Savvides, M., Chunyan, X., Venkataramani, K., Thornton, J., Mahalanobis, A.: Biometric Verification with Correlation Filters. Applied Optics 43, 391–402 (2004) 18. Thornton, A.: Colour object recognition using a complex colour representation and the frequency domain. PhD thesis, University of Reading, UK (1998) 19. Kuglin, C. D., Hines, D. C.: The phase correlation image alignment method. In: Int. Conf Cybernetics and Society, September 1975, 163–165. NY, USA (1975) 20. Moxey, C. E., Sangwine, S. J., Ell, T. A.: Color-grayscale image registration using hypercomplex phase correlation. In: IEEE ICIP, September 22-25, 385– 388. Rochester, NY, USA (2002) 21. Sangwine, S. J., Ell, T. A., Moxey, C. E.: Vector phase correlation. Electronics Letters 37, 1513–1515 (2001) 22. Horner, J. L., Gianino, P. D.: Phase-Only Matched Filtering. Applied Optics 23, 812–816 (1984) 23. Vijayakumar, B., Savvides, K., Venkataramani, K., Chunyan, X.: Spatial frequency domain image processing for biometric recognition. In: IEEE ICIP, September 22-25, 53–56. Rochester, NY, USA (2002) 24. Messer, K., Matas, J., Kittler, J., Jonsson, K.: Xm2vtsdb: The extended m2vts database. In: AVBPA 1999, March 2223, 72–77. Washington, D.C., USA (1999) 25. Lee, K.C., Ho, J., Kriegman, D.J.: Acquiring linear subspaces for face recognition under variable lighting. IEEE Trans. Pattern Anal. Machine Intell 27, 684–698 (2005) 26. Chen, W., Er, M.J., Wu, S.: Illumination compensation and normalization for robust face recognition using discrete cosine transform in logarithm domain. Systems, Man and Cybernetics, Part B, IEEE Transactions 36, 458–466 (2006)
