Spectral Face Recognition Using Orthogonal Subspace Bases
Spectral Face Recognition Using Orthogonal Subspace Bases Andrew Wimberlya, Stefan A. Robilab*, Tansy Peplauc a Dept. Of computer Science, Oberlin College, Oberlin OH, USA, 44074 b Dept. of Computer Science, Center of Imaging and Optics, Montclair State University, Montclair, NJ USA 07043 c Dept. of Computer Science, Saint Lawrence University, Canton, NY USA 13617 ABSTRACT We present an efficient method for facial recognition using hyperspectral imaging and orthogonal subspaces. Projecting the data into orthogonal subspaces has the advantage of compactness and reduction of redundancy. We focus on two approaches: Principal Component Analysis and Orthogonal Subspace Projection. Our work is separated in three stages. First, we designed an experimental setup that allowed us to create a hyperspectral image database of 17 subjects under different facial expressions and viewing angles. Second, we investigated approaches to employ spectral information for the generation of fused grayscale images. Third, we designed and tested a recognition system based on the methods described above. The experimental results show that spectral fusion leads to improvement of recognition accuracy when compared to regular imaging. The work expands on previous band extraction research and has the distinct advantage of being one of the first that combines spatial information (i.e. face characteristics) with spectral information. In addition, the techniques are general enough to accommodate differences in skin spectra. Keywords: face recognition, hyperspectral data, Principal Component Analysis, orthogonal subspaces.
1. INTRODUCTION Accurate individual identification is an essential part of any authentication protocol and is receiving increased interest in any area where security is important [1]. Often, such identification is done using biometrics, technologies that measure and analyze human body characteristics (such as fingerprint, iris, eye retina, gait, palm print, etc.) for authentication purposes [2]. Among biometrics, face recognition provides an unobtrusive and if needed discrete method of classifying individuals, a task that has become increasingly important in many fields swamped by the deluge of data available (but lack of human analysts) such as surveillance, law enforcement, and access control [3]. Face based recognition also has the distinct advantage of being readily accepted by individuals since it does not require any contact with the sensor (as it is the case for finger and palm prints), it does not require focus to delicate parts of the body (as perceived in iris and retina scans) and corresponds to a traditional human way of social interaction (details of the face being a traditional human way to differentiate individuals)[4]. Nevertheless, applications that focus on face recognition often fall behind human operators especially in the match accuracy [5]. A major problem in automating face recognition is due to the fact that many such applications focus on a two dimensional aspect of the face and and are unreliable difficulties when the subject is not directly oriented towards the camera (a problem easily solved by human visual perception). Additional efforts that include three dimensional volume morphing improve performance [6, 7]. Given the extraordinary progress in digital imaging sensors, correlated to a decrease in production costs, one has to ask if face recognition would benefit from expanding the image collected beyond the normal human visible range. Various studies have showed that infrared imaging yields comparable or slightly better results than visible imaging [8-10]. Of particular interest is the survey done in [11] where various large studies were compared. The authors concluded that, PCA-based recognition (explain in the section below) using visible-light images outperforms PCA-based recognition using infrared images, and more importantly, the combination of PCA-based recognition using visible-light and infrared imagery substantially outperforms either one individually, especially when the images would be collected at various times.
*[email protected], phone 1 973 655-4230, fax 1 973 655-4164, csam.montclair.edu/~robila
Hyperspectral imaging constitutes a relatively new approach in image based data collection. Unlike grayscale or color imagers that focus on averaging the visible spectrum (or components of the visible spectrum such as Red Green and Blue), a hyperspectral sensor aims to divide the light spectrum in narrow wavelength intervals and collect a separate image for each interval. In addition, current sensors are capable of expanding the wavelength range beyond the visible one and up to the Near and Mid Infrared. A single hyperspectral image (also known as hyperspectral cube) is thus formed of tens to hundreds of grayscale images, leading to a level of detail unmatched by any other two dimensional imaging technology. Given such richness of information, using hyperspectral sensors for face recognition is a natural choice. Previous work has focused mainly on the spectral aspects of the skin, i.e. whether the skin spectra (pixel values collected in the same location across the stack of images in the cube) for various subjects can be used for individual authentication [12, 13]. In our own research we investigated spectral differences among a small number of subjects and concluded (similar with previous work) that discrimination based on spectral information by itself offers promises for human identification [14]. However, a spectral centered approach constitutes a deviation from the traditional image centered face recognition problem where the face shape or other features (position and shape of eyes, nose, etc) were employed. Among many such techniques, of particular interest is the development of the PCA based recognition developed by Turk and Pentland shown to achieve over 90% accuracy within relatively well centered and oriented image faces [15]. In a recently published paper, we introduced the use of PCA based recognition expanded to hyperspectral images by creating eigenfaces from grayscale averages of hyperspectral cube [16]. Our results suggested that spectral imagery use improves PCA based classification, however, more work needs to be performed in understanding how the hyperspectral cube can be efficiently fused prior to face recognition. In this paper, we investigate an efficient method for facial recognition by using hyperspectral imaging and orthogonal subspaces. Projecting the data into orthogonal subspaces provides the unique advantage of compactness and reduction of data redundancy. In our case we focus on two approaches: Principal Component Analysis (PCA), and Orthogonal Subspace Projection (OSP). Our work can be separated in three main stages. In the first we have designed an experimental setup that allowed us to create a hyperspectral image database of 17 subjects each with five different facial expressions and viewing angles. Second, we investigated approaches to employ spectral information for the generation of fused grayscale images. Here we have looked at fusion of visible and infrared spectral images. Third, we designed and tested a recognition system based on the methods described above (PCA and OSP).
2. BACKGROUND 2.1 Principal Component Analysis (PCA) In PCA, given a random vector x, the goal is to find a linear transform W such that : Y = Wx the components of y are uncorrelated, i.e.:
∑ y = E{( y − E{y})( y − E{y})T }
(1) (2)
is diagonal and (when sorting according to their variance) the first y component accounts for as much of the variability in the data as possible, and each succeeding component accounts for as much of the remaining variability as possible. To solve PCA, if Ax is the matrix denoting the normalized eigenvectors for the covariance matrix Σx, we have: (3) Cx = ATx ∑x Ax where Cx is the corresponding diagonal eigenvalue matrix. Since in Eq. (5), Σy is required to be diagonal, we note that W= AxT leads to the PCA solution: (4) y = ATx x The components of y are called principal components and the transform W= AxT is called principal component transform. While over a century old, PCA continues to be used in a variety of applications, as the resulting components are considered to provide a good representation from the point of view of second order statistics of the internal structure of the data [17]. Within the face recognition problem, PCA was introduced by Turk and Pentland in [15]. Given a collection of grayscale face images {x1, x2,…, xM} the eigenfaces {y1, y2, …, yM’} are generated as the first few principal components resulting
from the collection. Next, we compute the coefficient vector Ωi = (ωi,1, ωi,2, …, ωi,M’) for each image xi and store these M’ weight coefficients for each image in database by computing the dot product of the normalized image with each selected component face: (5) ϖ i , j = y Ti * ( x j − x) where x is the expected value for all the images. Next, for the person p in our database, we compute the average ഥ ଶ , … , ߱ ഥெᇱ ቁ by averaging the weights of the coefficient vectors for all the images coefficient vector ቀ߱ ഥଵ , ߱ corresponding to the person stored in the face database. To classify a new image xTest, we first we compute the coefficient vector (ω1 Test, ω2 Test, …. , ωM’ Test) for the image in a similar fashion as in eq. 2. Next, we compute the Euclidean distance between its coefficient vector (ω1Test ….ωM’Test) and average coefficient vector ቀ߱ ഥଵ , ߱ ഥ ଶ , … , ߱ ഥெᇱ ቁ: M'
D ( p , x Test ) =
∑ (ϖ
Test i
−ϖ i p
)
2
(6)
i =1
We label the new image with the class that minimizes the distance above. Variations of this approach exist, with eigenfaces being replaced by icafaces, i.e. components produced through Independent Component Analysis (ICA). ICA is a technique that aims for statistical independence and not just uncorellation. Both eigenfaces and icafaces were successfully shown to be efficient in recognition [18]. In our previous work focused we compared the two approaches and we concluded that ICA based recognition is not providing any improvement over the traditional eigenface approach. However, as the number of icafaces used increases, the classification accuracy also increased [16]. 2.2 Orthogonal Subspace Projection (OSP) In OSP, the linear transform focuses on elimination of undesired information while maximizing the desired information content. Consider a desired vector d and a collection of undesired vectors U = {u1, …,uM}, and assume that a new vector r can be seen as a linear combination of both the desired and undesired vectors: ∝ ܌ = ܚ+ ߛ܃+ ܖ
(7)
where α and γ are abundances and n is random noise. The goal of OSP is to transform r onto a subspace that is orthogonal to the columns of U. The resulting component will only contain energy associated with the r and n. The transform is obtained as [19, 20]: ( = ۾۷ − ܃܃# ) (8) where: ܃# = (ି)܃ ܂ ܃ ܂ ܃ (9) is the pseudoinverse of U. Furthermore, the transformed data are transformed such that the signal to noise ratio is maximized. Both steps are summarized in one equation, with the combined transform being: ࢊࢀ ࡼ
ࢊࢀ ࡼࢊ
(10)
While OSP was introduced for hyperspectral data processing (such as target detection or classification), recently, Zhou and collaborators presented an OSP based algorithm, aimed at grayscale face recognition [21]. Given a training collection of grayscale face images,{x1, x2,…, xM} the ospfaces {y1, y2, …, yM} are generated as the transforms in equation 10: ࢞ࢀ ࡼ
࢟ = ࢞ࢀࡼ ࢞
(11)
and Pi obtained based on Ui={x1,.., xi-1, xi+1,…, xM} i.e. of all the images excluding xi. To classify a new image xTest, we project the test image based on the various OSP transforms by computing the dot product between xTest and each yi and selecting as match the training image xmatch whose ospface produces the largest value. Intuitively, this would mean that projecting the test image in the space that eliminated all other training images safe xmatch would in fact show us how similar the test image is from xmatch.
Coupled with preprocessing steps that improved the original image quality, the OSP approach was shown to be superior to PCA. In their research, Zhou and collaborators showed that use of OSP faces outperform PCA when applying the algorithms to a subset of the Yale database [21, 22]. However, the use of OSP requires larger storage than PCA. Whereas, for a set of training images, the eigenface approach will store only a limited number of eigenfaces and a number of omega coefficients for each image, OSP based detection requires that for each image we store a similarly sized one (the ospface). Such tradeoffs need to be considered especially in the case of processing large databases. Finally we note that PCA can also be considered as an orthogonal subspace projection, however, the goal of this projection is different.
3. PROPOSED APPROACH 3.1 Spectral Image Fusion Our primary goal is the design of PCA and OSP based strategies for face recognition using hyperspectral data. While both techniques work quite well with grayscale images they cannot be applied directly to hyperspectral cubes since each such cube contains tens to hundreds of image bands. In a previous publication we investigated various band fusion approaches such as averaging all bands, selecting the first principal component, or an average of the first three principal components [16]. Our results suggested that no significant difference exists between the use of image averaging or PC averaging. We believe that this can be explained by the large number of visible bands in image cube. To understand better the impact of infrared information we suggest the use of the following three different fusion approaches: AVGVIS: Average three visible bands in one grayscale image corresponding to the Red, Green and Blue spectrum ranges. AVGIR: Average the same visible bands as well as a infrared band. PC-AVG: Perform PCA on each image separately. Average the first three principal components [16]. 3.2 Image Enhancement Given that the various approaches described above yield different ranges for the pixel values we have first transformed the data such that each image’s pixel values are within the same interval. Next, we used histogram equalization in a similar approach as described in [21, 23]. This method enhances the images contrast by having the intensities better distributed on the histogram (see Fig. 1). 3500
1400
3000
1200
2500
1000
2000
800
1500
600
1000
400
500
200
0
0 0
200
400
600
800
1000
1200
1400
0
0.1
0.2
a) Figure 1. a) Original image histogram, b) histogram after histogram equalization.
0.3
0.4
0.5
0.6
b)
a) b) Figure 2. a) Hyperspectral bands fused as grayscale images, b) histogram equalized images
0.7
0.8
0.9
1
This allows for low contrast portions of the image to increase in contrast while maintaining the previously high contrast areas. By using histogram equalization we also reduce the illumination and other environmental factors’ impact on the image. The difference between regular and equalized images is exemplified in fig. 2. In both images we chose the illumination angle rendered a side of the face to be darker. After the histogram equalization both side show with high contrast. 3.3 Group Based Orthogonal Subspace Projection (GOSP) As described in [21], the OSP face detection algorithm is based on the understanding that the training database contains a single image for each subject. Such approach may not be viable in a real life situation where multiple images of the same person (representing various angles, makeups, etc.) are being stored. A possible extension in this case would be to continue creating an osp face for each image in the database. However, such approach will not work. In the event that two images of the same subject are significantly similar, the OSP face for one would try to eliminate the information of the other, basically producing incorrect results. Instead, we suggest that the OSP projections be modified to accommodate groups of images in the following Group Orthogonal Space Projection (GOSP) algorithm: Step 1. Compute Group Based Orthogonal projections Given a training collection of grayscale face images X select from this group all images that correspond to the same subject i. Let us name this subgroup Xi={xi1, xi2,…, xik}. In this, case, the orthogonal projection is computed as in equation 8 with: ( = ܑ ۾۷ − ܑ ܃ ܑ ܃# ) ܑ܂
ܑ ܃# = ቀ ܑ ܃ ܃ቁ
ି
ࢁ = ࢄ\ࢄ
(11) ܃
ܑ܂
(12) (13)
i.e. the undesired information is composed of all the images that do not match the subject i: Step 2. Compute the GOSP faces for each image Given an image xij, its GOSP face is obtained as:
࢟ =
ࢀ
࢞ ࡼ
(14)
ࢀ
࢞ ࡼ ࢞
Step 3. Match new images Given a test image xTest we say that it matches subject k if: ࢀ
ࢀ
࢟ ࢞ࢀࢋ࢙࢚ = ܠ܉ܕ, ࢟ ࢞ࢀࢋ࢙࢚
(15)
For one of the xkl images in the database associated with subject k.
4. DATA COLLECTION In a continuation of our work presented in [14, 16] we designed and created a hyperspectral face database. In July 2009 using new and previously created data we created HYPDB3.0, a database of 85 images covering 17 different subjects. Fig. 3 provides the generic experimental collection setup. A Surface Optics 700 hyperspectral camera was placed approximately six feet from the subjects. The camera is a scanner able to collect data between 400 and 1000nm with a dynamic binning approach. In our case we used 120 bins resulting in the same number of spectral bands, each 5nm wide. Given the static positioning of the instrument, the camera is equipped with an internal movement system that allows the collection of a 3D set by sensing 2D spectral slices in sequential fashion, in an approach similar to a pushbroom sensor.
Figure 3. Experimental setup for HYPDB3.0 collection.
Figure 4. Grayscale representation of the representative images forming HYPDB3.0.
Since each image consisted of 640 slices, combined with a sizeable exposure time, the pushbroom approach required the subjects to stay still for a relatively long period of time (10 (10-15 15 seconds) in order to avoid elongation of the faces. The same considerations ions explain why some images present artifacts such as straight lines that occurred when the sensor malfunctioned. Lacking full spectrum lighting we decided to collect the face images outside benefiting from high intensity solar light. The collection was ddone one in a single session in an early afternoon. For each person, five different images were collected. These included various face expressions and postures characterized as: “normal”, “smiling”, “angry”, “head tilted”, and “subject’s choice”. Each image was initially 640x640 pixels and was manually cropped to 200x200 pixels. Samples from the 2009 and 2008 campaigns are presented in fig. 4. Compared with the previous year where the background was formed of a brick building, the images expose a uniform cloth bbackground. ackground.
5. EXPERIMENTAL RESULTS To validate our work we have performed a series of experiments that varied the number of test images , the number of eigenfaces,, as well as the type of grayscale image images used. Fig. 5 provides a qualitative assessment of the data through the five highest variance eigenfaces for AVGVIS and AVGIR.. This supports our belief that introduction of infrared in the data provides difference in the results.
a)
b) Figure 5. First five eigenfaces faces for the AVGVIS and AVGIR
Next, for each of the three grayscale databases described in section 3.1. we measured the overall eigenface classification accuracy using runs based on the following steps: - for each subject randomly select one image as test image - vary the train data from two to four images randomly chosen for each subject - vary the number of eigenfaces used from 5 to 50 For consistency, each variant was run twenty times and results averaged. Where the number of eigenvectors was larger than the number of images, the minimum between the numbers was used. 0.6
0.6
0.55
0.55
0.5
0.5
0.45
0.45
0.4
0.4
0.35
0.35
0.3
0.3
0.25
0.25
2 3 4
0.2
0.2 5
10
15
20
25
30
35
40
45
5
50
10
15
20
25
30
35
40
45
50
a)
b)
0.6
0.6
0.55
0.55
0.5
0.5
0.45
0.45
0.4
0.4
0.35
0.35
0.3
0.3
0.25
0.25
2 3 4
0.2
0.2 5
10
15
20
25
30
35
40
45
5
50
10
15
20
25
30
35
40
45
50
c)
d)
0.6
0.6
0.55
0.55
0.5
0.5
0.45
0.45
0.4
0.4
0.35
0.35
0.3
0.3
0.25
0.25
2 3 4
0.2
0.2
5
10
15
20
25
30
35
40
45
50
5
10
15
20
25
30
35
40
45
50
e)
Figure 6. Classification accuracy vs number of eigenvalues used for eigeneigenface based face recognition. a) AVGVIS, b) AVGVIS histogram equalized, c) AVGIR, d) AVGIR histogram equalized, e) PC-AVG, f) PC-AVG histogram equalized, AVGIR(b), and PC-AVG(c). The number of training images used for subject is set to two (blue dotted), three (red dashed), and four (green solid).
f)
Fig 6 display the average accuracy for AVGVIS, AVGIR, and PC-AVG and the counterparts where the histogram equalization was performed. The horizontal axis corresponds to the number of eigenfaces used. The three lines correspond to using two (blue dotted), three (red dashed), and four (green solid) images as training for each subject. The figure allows us to investigate four aspects: Impact of the number of training images available for each person. Due to the small number of subjects (17) that would have limited the number of possible eigenfaces, we did not run the experiments with one training image per person. We note that overall, accuracy improves when more than two images are used as training for each person. If for AVGVIS and AVGIR there is no clear advantage of using three or four images, the use of PC-AVG seems to indicate that an increase in available training images will improve accuracy. A possible explanation for this is that, as it can be seen in fig. 4, the five images for each subject are significantly varied in terms of both facial expression as well as face angle. As such, an increase in the number of faces for each subject would allow for the differences to be included and for the classifier to have its performance reduced. When the data are histogram equalized, the accuracy differences between three and four training images are slightly reduced. Impact of the number of eigenfaces used. Across all six cases it is clear that having only five or ten eigenfaces is not sufficient. No significant increase in accuracy is noticed when the number of eigenfaces used is 20 or higher. However, beyond such value, the accuracy is not always increasing as the number of eigenfaces is increasing. No differences are noticed between the regular images and their histogram equalized versions. Impact of Histogram Equalization. As indicated above, equalization of the histogram does not seem to have any significant impact on the number of eigenfaces or training images that should be used for increased accuracy. However, the results show that the equalization has improved accuracy on average by 5%, supporting the claim that such preprocessing is a valuable one. In a second experiment we investigated on the use of the ospfaces. Fig. 7 displays the average accuracy for the three grayscale databases. The experiments have focused only on the study of the impact of the number of training images and the histogram equalization and the difference between OSP and GOSP. 0.8
0.8
0.7
0.7
0.6
0.6
0.5
GOSP
0.5
0.4
OSP
0.4
OSP
0.3
GOSP-EQ
0.3
GOSP-EQ
OSP-EQ
0.2
GOSP
OSP-EQ
0.2
0.1
0.1
0
0
1
2
3
4
1
2
3
4
a)
b)
0.8 0.7 0.6 0.5
GOSP
0.4
OSP
0.3
GOSP-EQ OSP-EQ
0.2 0.1 0 1
2
3
4
c) Figure 7. Classification accuracy vs number of training images used for OSP based face recognition. a) AVGVIS, b) AVGIR histogram equalized, c) PC-AVG ..
The graphs include regular OSP based detection applied to regular (dotted red) and histogram equalized (dotted brown). The GOSP was applied also on both variants (regular – dashed blue, histogram equalized – solid green). As in the previous experiment, each point in the graph reflects the average accuracy computed based on 20 runs. The experiment shows a remarkable difference between OSP and GOSP with OSP providing almost no correct answer (even when we used only one train image per subject). The results are different than the ones reported in [21]. This can be due to the larger variety of orientation of faces as well as the lack of the additional preprocessing steps suggested in the above mentioned publication. Nevertheless, GOSP provides significantly accurate results, with AVGIR reaching 70%.. Furthermore, the accuracy of the classification is correlated to the number of training images used and is improved when histogram equalization is performed. Fig 8. Provides a summary of our experiments, focused only on histogram equalized data and on GOSP and eigenface based recognition . In case of eigenfaces we used the first 50 bases. The figure shows that GOSP outperforms PCA by close to 20%. In addition, the use of infrared information contributes to improvements in accuracy.
6. CONCLUSIONS We presented a novel approach for face recognition that combines orthogonal subspaces and hyperspectral imagery. Several important contributions were made through our work. First, we provided a rigorous discussion of orthogonal subspace projections and their relationship with the face detection problem. Second we introduced a modification for OSP that allows the ospfaces to include information related to multiple faces of the same subject. Third, we considered various fusion approaches for hyperspectral data aiming for a representation that is valuable for face recognition. Finally, we have continued the assembly of a hyperspectral face dataset that will be shortly be offered for use to the scientific community. Following extensive experiments we conclude that Group OSP outperforms previous OSP and PCA based technique and results in significantly high accuracy levels. In addition, the use of infrared bands and the histogram based preprocessing increase the accuracy. Our work supports the claim that face recognition can benefit from the use of multi and hyperspectral data.
ACKNOWLEDGEMENT This material is based in part upon work supported by the National Science Foundation under Grant Number IIS0648814. Any opinions, findings, and conclusions or recommendations expressed in this material are those of the author(s) and do not necessarily reflect the views of the National Science Foundation. Stefan Robila was also supported through Montclair State University Faculty Scholarship Program. 0.8 0.7 0.6
GOSP-AVGVIS
0.5
GOSP-AVGIR GOSP-PC-AVG
0.4
EIG-AVGVIS 0.3
EIG-AVGIR EIG-PC-AVG
0.2 0.1 0 1
2
3
4
Figure 8. Overall classification accuracy vs number of training images
REFERENCES [1] [2] [3] [4] [5] [6] [7] [8] [9] [10] [11] [12]
[13] [14] [15] [16]
[17] [18] [19] [20] [21] [22] [23]
L. O’Gorman, “Seven issues with human authentication technologies,” in Proc. Workshop Automatic Identification Advanced Technologies (AutoID) (2002). A. K. Jain, A. Ross, and S. Prabhakar, “An introduction to biometric recognition,” Biometrics 14 (2004). M. H. Yang, D. J. Kriegman, and N. Ahuja, “Detecting faces in images: A survey,” IEEE Transactions on Pattern analysis and Machine intelligence, 34-58 (2002). A. A. Ross, K. Nandakumar, and A. K. Jain, Handbook of multibiometrics, Springer (2006). P. J. Phillips, A. Martin, C. L. Wilson, and M. Przybocki, “An introduction to evaluating biometric systems,” Computer, 56-63 (2000). R. Chellappa and C. L. Wilson, “Human and machine recognition of faces: A survey,” Proceedings of the IEEE 83, 705-740 (1995). A. F. Abate, M. Nappi, D. Riccio, and G. Sabatino, “2D and 3D face recognition: A survey,” Pattern Recognition Letters 28, 1885-1906 (2007). D. A. Socolinsky, A. Selinger, and J. D. Neuheisel, “Face recognition with visible and thermal infrared imagery,” Computer Vision and Image Understanding 91, 72-114 (2003). D. A. Socolinsky, C. K. Eveland, L. B. Wolff, and J. D. Neuheisel, Illumination invariant face recognition using thermal infrared imagery, EQUINOX CORP NEW YORK NY (2006). G. Bebis, A. Gyaourova, S. Singh, and I. Pavlidis, “Face recognition by fusing thermal infrared and visible imagery,” Image and Vision Computing 24, 727-742 (2006). X. Chen, P. J. Flynn, and K. W. Bowyer, “IR and visible light face recognition,” Computer Vision and Image Understanding 99, 332-358 (2005). Z. Pan, G. E. Healey, M. Prasad, and B. J. Tromberg, “Hyperspectral face recognition under variable outdoor illumination,” in Algorithms and Technologies for Multispectral, Hyperspectral, and Ultraspectral Imagery X 5425, pp. 520-529, SPIE, Orlando, FL, USA (2004). Z. Pan, G. Healey, M. Prasad, and B. Tromberg, “Face recognition in hyperspectral images,” IEEE Transactions on Pattern Analysis and Machine Intelligence 25, 1552-1560 (2003). S. A. Robila, “Toward hyperspectral face recognition,” in Image Processing: Algorithms and Systems VI 6812, pp. 68120X-9, SPIE, San Jose, CA, USA (2008). M. Turk and A. Pentland, “Eigenfaces for recognition,” Journal of cognitive neuroscience 3, 71-86 (1991). S. A. Robila, A. LaChance, and S. Ruff, “Investigating face recognition from hyperspectral data: impact of band extraction,” in Algorithms and Technologies for Multispectral, Hyperspectral, and Ultraspectral Imagery XV 7334, pp. 73341Y-10, SPIE, Orlando, FL, USA (2009). I. T. Jolliffe, Principal component analysis, Springer verlag (2002). M. S. Bartlett, J. R. Movellan, and T. J. Sejnowski, “Face recognition by independent component analysis,” IEEE Transactions on neural networks 13, 1450-1464 (2002). J. C. Harsanyi and C. I. Chang, “Hyperspectral image classification and dimensionality reduction: an orthogonal subspace projection approach,” IEEE Transactions on Geoscience and Remote Sensing 32, 779-785 (1994). H. Ren and C. I. Chang, “A generalized orthogonal subspace projection approach to unsupervised multispectral image classification,” IEEE Transactions on Geoscience and Remote Sensing 38, 2515-2528 (2000). Z. Zhou, Y. Du, and C. Chang, “An orthogonal subspace projection approach for face recognition,” in Mobile Multimedia/Image Processing, Security, and Applications 2009 7351, pp. 73510E-8, SPIE, Orlando, FL, USA (2009). P. N. Belhumeur, J. P. Hespanha, and D. J. Kriegman, “Eigenfaces vs. fisherfaces: Recognition using class specific linear projection,” IEEE Transactions on pattern analysis and machine intelligence 19, 711-720 (1997). K. W. Bowyer, K. Chang, and P. Flynn, “A survey of approaches to three-dimensional face recognition,” in IEEE ICPR, pp. 358–361 (2004).
1. INTRODUCTION Accurate individual identification is an essential part of any authentication protocol and is receiving increased interest in any area where security is important [1]. Often, such identification is done using biometrics, technologies that measure and analyze human body characteristics (such as fingerprint, iris, eye retina, gait, palm print, etc.) for authentication purposes [2]. Among biometrics, face recognition provides an unobtrusive and if needed discrete method of classifying individuals, a task that has become increasingly important in many fields swamped by the deluge of data available (but lack of human analysts) such as surveillance, law enforcement, and access control [3]. Face based recognition also has the distinct advantage of being readily accepted by individuals since it does not require any contact with the sensor (as it is the case for finger and palm prints), it does not require focus to delicate parts of the body (as perceived in iris and retina scans) and corresponds to a traditional human way of social interaction (details of the face being a traditional human way to differentiate individuals)[4]. Nevertheless, applications that focus on face recognition often fall behind human operators especially in the match accuracy [5]. A major problem in automating face recognition is due to the fact that many such applications focus on a two dimensional aspect of the face and and are unreliable difficulties when the subject is not directly oriented towards the camera (a problem easily solved by human visual perception). Additional efforts that include three dimensional volume morphing improve performance [6, 7]. Given the extraordinary progress in digital imaging sensors, correlated to a decrease in production costs, one has to ask if face recognition would benefit from expanding the image collected beyond the normal human visible range. Various studies have showed that infrared imaging yields comparable or slightly better results than visible imaging [8-10]. Of particular interest is the survey done in [11] where various large studies were compared. The authors concluded that, PCA-based recognition (explain in the section below) using visible-light images outperforms PCA-based recognition using infrared images, and more importantly, the combination of PCA-based recognition using visible-light and infrared imagery substantially outperforms either one individually, especially when the images would be collected at various times.
*[email protected], phone 1 973 655-4230, fax 1 973 655-4164, csam.montclair.edu/~robila
Hyperspectral imaging constitutes a relatively new approach in image based data collection. Unlike grayscale or color imagers that focus on averaging the visible spectrum (or components of the visible spectrum such as Red Green and Blue), a hyperspectral sensor aims to divide the light spectrum in narrow wavelength intervals and collect a separate image for each interval. In addition, current sensors are capable of expanding the wavelength range beyond the visible one and up to the Near and Mid Infrared. A single hyperspectral image (also known as hyperspectral cube) is thus formed of tens to hundreds of grayscale images, leading to a level of detail unmatched by any other two dimensional imaging technology. Given such richness of information, using hyperspectral sensors for face recognition is a natural choice. Previous work has focused mainly on the spectral aspects of the skin, i.e. whether the skin spectra (pixel values collected in the same location across the stack of images in the cube) for various subjects can be used for individual authentication [12, 13]. In our own research we investigated spectral differences among a small number of subjects and concluded (similar with previous work) that discrimination based on spectral information by itself offers promises for human identification [14]. However, a spectral centered approach constitutes a deviation from the traditional image centered face recognition problem where the face shape or other features (position and shape of eyes, nose, etc) were employed. Among many such techniques, of particular interest is the development of the PCA based recognition developed by Turk and Pentland shown to achieve over 90% accuracy within relatively well centered and oriented image faces [15]. In a recently published paper, we introduced the use of PCA based recognition expanded to hyperspectral images by creating eigenfaces from grayscale averages of hyperspectral cube [16]. Our results suggested that spectral imagery use improves PCA based classification, however, more work needs to be performed in understanding how the hyperspectral cube can be efficiently fused prior to face recognition. In this paper, we investigate an efficient method for facial recognition by using hyperspectral imaging and orthogonal subspaces. Projecting the data into orthogonal subspaces provides the unique advantage of compactness and reduction of data redundancy. In our case we focus on two approaches: Principal Component Analysis (PCA), and Orthogonal Subspace Projection (OSP). Our work can be separated in three main stages. In the first we have designed an experimental setup that allowed us to create a hyperspectral image database of 17 subjects each with five different facial expressions and viewing angles. Second, we investigated approaches to employ spectral information for the generation of fused grayscale images. Here we have looked at fusion of visible and infrared spectral images. Third, we designed and tested a recognition system based on the methods described above (PCA and OSP).
2. BACKGROUND 2.1 Principal Component Analysis (PCA) In PCA, given a random vector x, the goal is to find a linear transform W such that : Y = Wx the components of y are uncorrelated, i.e.:
∑ y = E{( y − E{y})( y − E{y})T }
(1) (2)
is diagonal and (when sorting according to their variance) the first y component accounts for as much of the variability in the data as possible, and each succeeding component accounts for as much of the remaining variability as possible. To solve PCA, if Ax is the matrix denoting the normalized eigenvectors for the covariance matrix Σx, we have: (3) Cx = ATx ∑x Ax where Cx is the corresponding diagonal eigenvalue matrix. Since in Eq. (5), Σy is required to be diagonal, we note that W= AxT leads to the PCA solution: (4) y = ATx x The components of y are called principal components and the transform W= AxT is called principal component transform. While over a century old, PCA continues to be used in a variety of applications, as the resulting components are considered to provide a good representation from the point of view of second order statistics of the internal structure of the data [17]. Within the face recognition problem, PCA was introduced by Turk and Pentland in [15]. Given a collection of grayscale face images {x1, x2,…, xM} the eigenfaces {y1, y2, …, yM’} are generated as the first few principal components resulting
from the collection. Next, we compute the coefficient vector Ωi = (ωi,1, ωi,2, …, ωi,M’) for each image xi and store these M’ weight coefficients for each image in database by computing the dot product of the normalized image with each selected component face: (5) ϖ i , j = y Ti * ( x j − x) where x is the expected value for all the images. Next, for the person p in our database, we compute the average ഥ ଶ , … , ߱ ഥெᇱ ቁ by averaging the weights of the coefficient vectors for all the images coefficient vector ቀ߱ ഥଵ , ߱ corresponding to the person stored in the face database. To classify a new image xTest, we first we compute the coefficient vector (ω1 Test, ω2 Test, …. , ωM’ Test) for the image in a similar fashion as in eq. 2. Next, we compute the Euclidean distance between its coefficient vector (ω1Test ….ωM’Test) and average coefficient vector ቀ߱ ഥଵ , ߱ ഥ ଶ , … , ߱ ഥெᇱ ቁ: M'
D ( p , x Test ) =
∑ (ϖ
Test i
−ϖ i p
)
2
(6)
i =1
We label the new image with the class that minimizes the distance above. Variations of this approach exist, with eigenfaces being replaced by icafaces, i.e. components produced through Independent Component Analysis (ICA). ICA is a technique that aims for statistical independence and not just uncorellation. Both eigenfaces and icafaces were successfully shown to be efficient in recognition [18]. In our previous work focused we compared the two approaches and we concluded that ICA based recognition is not providing any improvement over the traditional eigenface approach. However, as the number of icafaces used increases, the classification accuracy also increased [16]. 2.2 Orthogonal Subspace Projection (OSP) In OSP, the linear transform focuses on elimination of undesired information while maximizing the desired information content. Consider a desired vector d and a collection of undesired vectors U = {u1, …,uM}, and assume that a new vector r can be seen as a linear combination of both the desired and undesired vectors: ∝ ܌ = ܚ+ ߛ܃+ ܖ
(7)
where α and γ are abundances and n is random noise. The goal of OSP is to transform r onto a subspace that is orthogonal to the columns of U. The resulting component will only contain energy associated with the r and n. The transform is obtained as [19, 20]: ( = ۾۷ − ܃܃# ) (8) where: ܃# = (ି)܃ ܂ ܃ ܂ ܃ (9) is the pseudoinverse of U. Furthermore, the transformed data are transformed such that the signal to noise ratio is maximized. Both steps are summarized in one equation, with the combined transform being: ࢊࢀ ࡼ
ࢊࢀ ࡼࢊ
(10)
While OSP was introduced for hyperspectral data processing (such as target detection or classification), recently, Zhou and collaborators presented an OSP based algorithm, aimed at grayscale face recognition [21]. Given a training collection of grayscale face images,{x1, x2,…, xM} the ospfaces {y1, y2, …, yM} are generated as the transforms in equation 10: ࢞ࢀ ࡼ
࢟ = ࢞ࢀࡼ ࢞
(11)
and Pi obtained based on Ui={x1,.., xi-1, xi+1,…, xM} i.e. of all the images excluding xi. To classify a new image xTest, we project the test image based on the various OSP transforms by computing the dot product between xTest and each yi and selecting as match the training image xmatch whose ospface produces the largest value. Intuitively, this would mean that projecting the test image in the space that eliminated all other training images safe xmatch would in fact show us how similar the test image is from xmatch.
Coupled with preprocessing steps that improved the original image quality, the OSP approach was shown to be superior to PCA. In their research, Zhou and collaborators showed that use of OSP faces outperform PCA when applying the algorithms to a subset of the Yale database [21, 22]. However, the use of OSP requires larger storage than PCA. Whereas, for a set of training images, the eigenface approach will store only a limited number of eigenfaces and a number of omega coefficients for each image, OSP based detection requires that for each image we store a similarly sized one (the ospface). Such tradeoffs need to be considered especially in the case of processing large databases. Finally we note that PCA can also be considered as an orthogonal subspace projection, however, the goal of this projection is different.
3. PROPOSED APPROACH 3.1 Spectral Image Fusion Our primary goal is the design of PCA and OSP based strategies for face recognition using hyperspectral data. While both techniques work quite well with grayscale images they cannot be applied directly to hyperspectral cubes since each such cube contains tens to hundreds of image bands. In a previous publication we investigated various band fusion approaches such as averaging all bands, selecting the first principal component, or an average of the first three principal components [16]. Our results suggested that no significant difference exists between the use of image averaging or PC averaging. We believe that this can be explained by the large number of visible bands in image cube. To understand better the impact of infrared information we suggest the use of the following three different fusion approaches: AVGVIS: Average three visible bands in one grayscale image corresponding to the Red, Green and Blue spectrum ranges. AVGIR: Average the same visible bands as well as a infrared band. PC-AVG: Perform PCA on each image separately. Average the first three principal components [16]. 3.2 Image Enhancement Given that the various approaches described above yield different ranges for the pixel values we have first transformed the data such that each image’s pixel values are within the same interval. Next, we used histogram equalization in a similar approach as described in [21, 23]. This method enhances the images contrast by having the intensities better distributed on the histogram (see Fig. 1). 3500
1400
3000
1200
2500
1000
2000
800
1500
600
1000
400
500
200
0
0 0
200
400
600
800
1000
1200
1400
0
0.1
0.2
a) Figure 1. a) Original image histogram, b) histogram after histogram equalization.
0.3
0.4
0.5
0.6
b)
a) b) Figure 2. a) Hyperspectral bands fused as grayscale images, b) histogram equalized images
0.7
0.8
0.9
1
This allows for low contrast portions of the image to increase in contrast while maintaining the previously high contrast areas. By using histogram equalization we also reduce the illumination and other environmental factors’ impact on the image. The difference between regular and equalized images is exemplified in fig. 2. In both images we chose the illumination angle rendered a side of the face to be darker. After the histogram equalization both side show with high contrast. 3.3 Group Based Orthogonal Subspace Projection (GOSP) As described in [21], the OSP face detection algorithm is based on the understanding that the training database contains a single image for each subject. Such approach may not be viable in a real life situation where multiple images of the same person (representing various angles, makeups, etc.) are being stored. A possible extension in this case would be to continue creating an osp face for each image in the database. However, such approach will not work. In the event that two images of the same subject are significantly similar, the OSP face for one would try to eliminate the information of the other, basically producing incorrect results. Instead, we suggest that the OSP projections be modified to accommodate groups of images in the following Group Orthogonal Space Projection (GOSP) algorithm: Step 1. Compute Group Based Orthogonal projections Given a training collection of grayscale face images X select from this group all images that correspond to the same subject i. Let us name this subgroup Xi={xi1, xi2,…, xik}. In this, case, the orthogonal projection is computed as in equation 8 with: ( = ܑ ۾۷ − ܑ ܃ ܑ ܃# ) ܑ܂
ܑ ܃# = ቀ ܑ ܃ ܃ቁ
ି
ࢁ = ࢄ\ࢄ
(11) ܃
ܑ܂
(12) (13)
i.e. the undesired information is composed of all the images that do not match the subject i: Step 2. Compute the GOSP faces for each image Given an image xij, its GOSP face is obtained as:
࢟ =
ࢀ
࢞ ࡼ
(14)
ࢀ
࢞ ࡼ ࢞
Step 3. Match new images Given a test image xTest we say that it matches subject k if: ࢀ
ࢀ
࢟ ࢞ࢀࢋ࢙࢚ = ܠ܉ܕ, ࢟ ࢞ࢀࢋ࢙࢚
(15)
For one of the xkl images in the database associated with subject k.
4. DATA COLLECTION In a continuation of our work presented in [14, 16] we designed and created a hyperspectral face database. In July 2009 using new and previously created data we created HYPDB3.0, a database of 85 images covering 17 different subjects. Fig. 3 provides the generic experimental collection setup. A Surface Optics 700 hyperspectral camera was placed approximately six feet from the subjects. The camera is a scanner able to collect data between 400 and 1000nm with a dynamic binning approach. In our case we used 120 bins resulting in the same number of spectral bands, each 5nm wide. Given the static positioning of the instrument, the camera is equipped with an internal movement system that allows the collection of a 3D set by sensing 2D spectral slices in sequential fashion, in an approach similar to a pushbroom sensor.
Figure 3. Experimental setup for HYPDB3.0 collection.
Figure 4. Grayscale representation of the representative images forming HYPDB3.0.
Since each image consisted of 640 slices, combined with a sizeable exposure time, the pushbroom approach required the subjects to stay still for a relatively long period of time (10 (10-15 15 seconds) in order to avoid elongation of the faces. The same considerations ions explain why some images present artifacts such as straight lines that occurred when the sensor malfunctioned. Lacking full spectrum lighting we decided to collect the face images outside benefiting from high intensity solar light. The collection was ddone one in a single session in an early afternoon. For each person, five different images were collected. These included various face expressions and postures characterized as: “normal”, “smiling”, “angry”, “head tilted”, and “subject’s choice”. Each image was initially 640x640 pixels and was manually cropped to 200x200 pixels. Samples from the 2009 and 2008 campaigns are presented in fig. 4. Compared with the previous year where the background was formed of a brick building, the images expose a uniform cloth bbackground. ackground.
5. EXPERIMENTAL RESULTS To validate our work we have performed a series of experiments that varied the number of test images , the number of eigenfaces,, as well as the type of grayscale image images used. Fig. 5 provides a qualitative assessment of the data through the five highest variance eigenfaces for AVGVIS and AVGIR.. This supports our belief that introduction of infrared in the data provides difference in the results.
a)
b) Figure 5. First five eigenfaces faces for the AVGVIS and AVGIR
Next, for each of the three grayscale databases described in section 3.1. we measured the overall eigenface classification accuracy using runs based on the following steps: - for each subject randomly select one image as test image - vary the train data from two to four images randomly chosen for each subject - vary the number of eigenfaces used from 5 to 50 For consistency, each variant was run twenty times and results averaged. Where the number of eigenvectors was larger than the number of images, the minimum between the numbers was used. 0.6
0.6
0.55
0.55
0.5
0.5
0.45
0.45
0.4
0.4
0.35
0.35
0.3
0.3
0.25
0.25
2 3 4
0.2
0.2 5
10
15
20
25
30
35
40
45
5
50
10
15
20
25
30
35
40
45
50
a)
b)
0.6
0.6
0.55
0.55
0.5
0.5
0.45
0.45
0.4
0.4
0.35
0.35
0.3
0.3
0.25
0.25
2 3 4
0.2
0.2 5
10
15
20
25
30
35
40
45
5
50
10
15
20
25
30
35
40
45
50
c)
d)
0.6
0.6
0.55
0.55
0.5
0.5
0.45
0.45
0.4
0.4
0.35
0.35
0.3
0.3
0.25
0.25
2 3 4
0.2
0.2
5
10
15
20
25
30
35
40
45
50
5
10
15
20
25
30
35
40
45
50
e)
Figure 6. Classification accuracy vs number of eigenvalues used for eigeneigenface based face recognition. a) AVGVIS, b) AVGVIS histogram equalized, c) AVGIR, d) AVGIR histogram equalized, e) PC-AVG, f) PC-AVG histogram equalized, AVGIR(b), and PC-AVG(c). The number of training images used for subject is set to two (blue dotted), three (red dashed), and four (green solid).
f)
Fig 6 display the average accuracy for AVGVIS, AVGIR, and PC-AVG and the counterparts where the histogram equalization was performed. The horizontal axis corresponds to the number of eigenfaces used. The three lines correspond to using two (blue dotted), three (red dashed), and four (green solid) images as training for each subject. The figure allows us to investigate four aspects: Impact of the number of training images available for each person. Due to the small number of subjects (17) that would have limited the number of possible eigenfaces, we did not run the experiments with one training image per person. We note that overall, accuracy improves when more than two images are used as training for each person. If for AVGVIS and AVGIR there is no clear advantage of using three or four images, the use of PC-AVG seems to indicate that an increase in available training images will improve accuracy. A possible explanation for this is that, as it can be seen in fig. 4, the five images for each subject are significantly varied in terms of both facial expression as well as face angle. As such, an increase in the number of faces for each subject would allow for the differences to be included and for the classifier to have its performance reduced. When the data are histogram equalized, the accuracy differences between three and four training images are slightly reduced. Impact of the number of eigenfaces used. Across all six cases it is clear that having only five or ten eigenfaces is not sufficient. No significant increase in accuracy is noticed when the number of eigenfaces used is 20 or higher. However, beyond such value, the accuracy is not always increasing as the number of eigenfaces is increasing. No differences are noticed between the regular images and their histogram equalized versions. Impact of Histogram Equalization. As indicated above, equalization of the histogram does not seem to have any significant impact on the number of eigenfaces or training images that should be used for increased accuracy. However, the results show that the equalization has improved accuracy on average by 5%, supporting the claim that such preprocessing is a valuable one. In a second experiment we investigated on the use of the ospfaces. Fig. 7 displays the average accuracy for the three grayscale databases. The experiments have focused only on the study of the impact of the number of training images and the histogram equalization and the difference between OSP and GOSP. 0.8
0.8
0.7
0.7
0.6
0.6
0.5
GOSP
0.5
0.4
OSP
0.4
OSP
0.3
GOSP-EQ
0.3
GOSP-EQ
OSP-EQ
0.2
GOSP
OSP-EQ
0.2
0.1
0.1
0
0
1
2
3
4
1
2
3
4
a)
b)
0.8 0.7 0.6 0.5
GOSP
0.4
OSP
0.3
GOSP-EQ OSP-EQ
0.2 0.1 0 1
2
3
4
c) Figure 7. Classification accuracy vs number of training images used for OSP based face recognition. a) AVGVIS, b) AVGIR histogram equalized, c) PC-AVG ..
The graphs include regular OSP based detection applied to regular (dotted red) and histogram equalized (dotted brown). The GOSP was applied also on both variants (regular – dashed blue, histogram equalized – solid green). As in the previous experiment, each point in the graph reflects the average accuracy computed based on 20 runs. The experiment shows a remarkable difference between OSP and GOSP with OSP providing almost no correct answer (even when we used only one train image per subject). The results are different than the ones reported in [21]. This can be due to the larger variety of orientation of faces as well as the lack of the additional preprocessing steps suggested in the above mentioned publication. Nevertheless, GOSP provides significantly accurate results, with AVGIR reaching 70%.. Furthermore, the accuracy of the classification is correlated to the number of training images used and is improved when histogram equalization is performed. Fig 8. Provides a summary of our experiments, focused only on histogram equalized data and on GOSP and eigenface based recognition . In case of eigenfaces we used the first 50 bases. The figure shows that GOSP outperforms PCA by close to 20%. In addition, the use of infrared information contributes to improvements in accuracy.
6. CONCLUSIONS We presented a novel approach for face recognition that combines orthogonal subspaces and hyperspectral imagery. Several important contributions were made through our work. First, we provided a rigorous discussion of orthogonal subspace projections and their relationship with the face detection problem. Second we introduced a modification for OSP that allows the ospfaces to include information related to multiple faces of the same subject. Third, we considered various fusion approaches for hyperspectral data aiming for a representation that is valuable for face recognition. Finally, we have continued the assembly of a hyperspectral face dataset that will be shortly be offered for use to the scientific community. Following extensive experiments we conclude that Group OSP outperforms previous OSP and PCA based technique and results in significantly high accuracy levels. In addition, the use of infrared bands and the histogram based preprocessing increase the accuracy. Our work supports the claim that face recognition can benefit from the use of multi and hyperspectral data.
ACKNOWLEDGEMENT This material is based in part upon work supported by the National Science Foundation under Grant Number IIS0648814. Any opinions, findings, and conclusions or recommendations expressed in this material are those of the author(s) and do not necessarily reflect the views of the National Science Foundation. Stefan Robila was also supported through Montclair State University Faculty Scholarship Program. 0.8 0.7 0.6
GOSP-AVGVIS
0.5
GOSP-AVGIR GOSP-PC-AVG
0.4
EIG-AVGVIS 0.3
EIG-AVGIR EIG-PC-AVG
0.2 0.1 0 1
2
3
4
Figure 8. Overall classification accuracy vs number of training images
REFERENCES [1] [2] [3] [4] [5] [6] [7] [8] [9] [10] [11] [12]
[13] [14] [15] [16]
[17] [18] [19] [20] [21] [22] [23]
L. O’Gorman, “Seven issues with human authentication technologies,” in Proc. Workshop Automatic Identification Advanced Technologies (AutoID) (2002). A. K. Jain, A. Ross, and S. Prabhakar, “An introduction to biometric recognition,” Biometrics 14 (2004). M. H. Yang, D. J. Kriegman, and N. Ahuja, “Detecting faces in images: A survey,” IEEE Transactions on Pattern analysis and Machine intelligence, 34-58 (2002). A. A. Ross, K. Nandakumar, and A. K. Jain, Handbook of multibiometrics, Springer (2006). P. J. Phillips, A. Martin, C. L. Wilson, and M. Przybocki, “An introduction to evaluating biometric systems,” Computer, 56-63 (2000). R. Chellappa and C. L. Wilson, “Human and machine recognition of faces: A survey,” Proceedings of the IEEE 83, 705-740 (1995). A. F. Abate, M. Nappi, D. Riccio, and G. Sabatino, “2D and 3D face recognition: A survey,” Pattern Recognition Letters 28, 1885-1906 (2007). D. A. Socolinsky, A. Selinger, and J. D. Neuheisel, “Face recognition with visible and thermal infrared imagery,” Computer Vision and Image Understanding 91, 72-114 (2003). D. A. Socolinsky, C. K. Eveland, L. B. Wolff, and J. D. Neuheisel, Illumination invariant face recognition using thermal infrared imagery, EQUINOX CORP NEW YORK NY (2006). G. Bebis, A. Gyaourova, S. Singh, and I. Pavlidis, “Face recognition by fusing thermal infrared and visible imagery,” Image and Vision Computing 24, 727-742 (2006). X. Chen, P. J. Flynn, and K. W. Bowyer, “IR and visible light face recognition,” Computer Vision and Image Understanding 99, 332-358 (2005). Z. Pan, G. E. Healey, M. Prasad, and B. J. Tromberg, “Hyperspectral face recognition under variable outdoor illumination,” in Algorithms and Technologies for Multispectral, Hyperspectral, and Ultraspectral Imagery X 5425, pp. 520-529, SPIE, Orlando, FL, USA (2004). Z. Pan, G. Healey, M. Prasad, and B. Tromberg, “Face recognition in hyperspectral images,” IEEE Transactions on Pattern Analysis and Machine Intelligence 25, 1552-1560 (2003). S. A. Robila, “Toward hyperspectral face recognition,” in Image Processing: Algorithms and Systems VI 6812, pp. 68120X-9, SPIE, San Jose, CA, USA (2008). M. Turk and A. Pentland, “Eigenfaces for recognition,” Journal of cognitive neuroscience 3, 71-86 (1991). S. A. Robila, A. LaChance, and S. Ruff, “Investigating face recognition from hyperspectral data: impact of band extraction,” in Algorithms and Technologies for Multispectral, Hyperspectral, and Ultraspectral Imagery XV 7334, pp. 73341Y-10, SPIE, Orlando, FL, USA (2009). I. T. Jolliffe, Principal component analysis, Springer verlag (2002). M. S. Bartlett, J. R. Movellan, and T. J. Sejnowski, “Face recognition by independent component analysis,” IEEE Transactions on neural networks 13, 1450-1464 (2002). J. C. Harsanyi and C. I. Chang, “Hyperspectral image classification and dimensionality reduction: an orthogonal subspace projection approach,” IEEE Transactions on Geoscience and Remote Sensing 32, 779-785 (1994). H. Ren and C. I. Chang, “A generalized orthogonal subspace projection approach to unsupervised multispectral image classification,” IEEE Transactions on Geoscience and Remote Sensing 38, 2515-2528 (2000). Z. Zhou, Y. Du, and C. Chang, “An orthogonal subspace projection approach for face recognition,” in Mobile Multimedia/Image Processing, Security, and Applications 2009 7351, pp. 73510E-8, SPIE, Orlando, FL, USA (2009). P. N. Belhumeur, J. P. Hespanha, and D. J. Kriegman, “Eigenfaces vs. fisherfaces: Recognition using class specific linear projection,” IEEE Transactions on pattern analysis and machine intelligence 19, 711-720 (1997). K. W. Bowyer, K. Chang, and P. Flynn, “A survey of approaches to three-dimensional face recognition,” in IEEE ICPR, pp. 358–361 (2004).
