An Efficient Feature Selection Method for Object Detection
An Efficient Feature Selection Method for Object Detection Duy-Dinh Le1 and Shin’ichi Satoh1,2 1
The Graduate University for Advanced Studies, Shonan Village, Hayama, Kanagawa, Japan 240-0193 [email protected] 2 National Institute of Informatics, 2-1-2 Hitotsubashi, Chiyoda-ku, Tokyo, Japan 101-8430 [email protected]
Abstract. We propose a simple yet efficient feature-selection method — based on principle component analysis (PCA) — for SVM-based classifiers. The idea is to select features whose corresponding axes are closest to the principle components computed from a data distribution by PCA. Experimental results show that our proposed method reduces dimensionality similar to PCA, but maintains the original measurement meanings while decreasing the computation time significantly.
1
Introduction
In many object-detection systems, feature selection — which is generally considered as the selection of a smaller subset of features from a large set of features — is one of the critical issues for the following three reasons. First, there are many ways to represent a target object, leading to a huge input feature set. For example, Haar wavelet features used in [1] are in the order of thousands. However, only small and incomplete training sets are available. As a result, systems will suffer from the curse of dimensionality and overfitting. Second, a large feature set includes many irrelevant and correlated features that can degrade the generalization performance of classifiers [2,3]. Third, selecting an optimal feature subset from a large input feature set can improve the performance and speed of classifiers. In face detection, the success of systems such as those in [1,4] comes mainly from efficient feature-selection methods. Most work, however, only focuses on feature-extraction methods, such as principle-component analysis (PCA), linear discriminant analysis (LDA), and independent-component analysis (ICA) [5,6,7], which try to map data from highdimensional space to lower-dimensional space. This might be because featureselection methods, such as sequential forward selection (SFS), sequential backward selection (SBS), and sequential forward floating search (SFFS) [8,9], incur very high computational cost. In this paper, to address these problems, we propose a simple yet efficient feature-selection method for object detection. The main idea is to select features S. Singh et al. (Eds.): ICAPR 2005, LNCS 3686, pp. 461–468, 2005. c Springer-Verlag Berlin Heidelberg 2005
462
D.-D. Le and S. Satoh
whose corresponding axes are closest to principle components computed by PCA from the data distribution. This is a very naive feature-selection method, but experimental results on different kinds of features show that when working with support vector machine (SVM)-based classifiers, our proposed method has comparable performance, but faster speed, compared to a feature-selection method based on PCA directly. The rest of the paper is organized as follows: In section 2, feature extraction by PCA is presented. Our feature selection method is introduced in section 3. Experimental results are showed in section 4. Finally, section 5 concludes the paper.
2
Feature Extraction Using PCA
The main steps to extract features using PCA are summarized in the following. The details are given in [5]. Each face image I(x, y) is represented as an N × N vector Γi . 1 M The average face Ψ is computed as: Ψ = M Σi=1 Γi where M is the number of face images in the training set. The difference between each face and the average face is given as: Φi = Γi −Ψ . 1 M A covariance matrix is then estimated as: C = M Σi=1 Φi ΦTi = AAT where A = [Φ1 Φ2 ...ΦM ]. Eigenvectors ui and corresponding eigenvalues λi of the covariance matrix C can be evaluated by using a Singular Value Decomposition (SVD) method [5]: Cui = λi ui . Because matrix C is usually very large (N 2 × N 2 ), evaluating eigenvectors and eigenvalues is very expensive. Instead, eigenvectors vi and corresponding eigen values µi of matrix AT A (M × M ) can be computed. After that, ui can be computed from vi as follows: ui = Avi , j = 1, ..., M . To reduce dimensionality, only a smaller number of eigenvectors K(K
The Graduate University for Advanced Studies, Shonan Village, Hayama, Kanagawa, Japan 240-0193 [email protected] 2 National Institute of Informatics, 2-1-2 Hitotsubashi, Chiyoda-ku, Tokyo, Japan 101-8430 [email protected]
Abstract. We propose a simple yet efficient feature-selection method — based on principle component analysis (PCA) — for SVM-based classifiers. The idea is to select features whose corresponding axes are closest to the principle components computed from a data distribution by PCA. Experimental results show that our proposed method reduces dimensionality similar to PCA, but maintains the original measurement meanings while decreasing the computation time significantly.
1
Introduction
In many object-detection systems, feature selection — which is generally considered as the selection of a smaller subset of features from a large set of features — is one of the critical issues for the following three reasons. First, there are many ways to represent a target object, leading to a huge input feature set. For example, Haar wavelet features used in [1] are in the order of thousands. However, only small and incomplete training sets are available. As a result, systems will suffer from the curse of dimensionality and overfitting. Second, a large feature set includes many irrelevant and correlated features that can degrade the generalization performance of classifiers [2,3]. Third, selecting an optimal feature subset from a large input feature set can improve the performance and speed of classifiers. In face detection, the success of systems such as those in [1,4] comes mainly from efficient feature-selection methods. Most work, however, only focuses on feature-extraction methods, such as principle-component analysis (PCA), linear discriminant analysis (LDA), and independent-component analysis (ICA) [5,6,7], which try to map data from highdimensional space to lower-dimensional space. This might be because featureselection methods, such as sequential forward selection (SFS), sequential backward selection (SBS), and sequential forward floating search (SFFS) [8,9], incur very high computational cost. In this paper, to address these problems, we propose a simple yet efficient feature-selection method for object detection. The main idea is to select features S. Singh et al. (Eds.): ICAPR 2005, LNCS 3686, pp. 461–468, 2005. c Springer-Verlag Berlin Heidelberg 2005
462
D.-D. Le and S. Satoh
whose corresponding axes are closest to principle components computed by PCA from the data distribution. This is a very naive feature-selection method, but experimental results on different kinds of features show that when working with support vector machine (SVM)-based classifiers, our proposed method has comparable performance, but faster speed, compared to a feature-selection method based on PCA directly. The rest of the paper is organized as follows: In section 2, feature extraction by PCA is presented. Our feature selection method is introduced in section 3. Experimental results are showed in section 4. Finally, section 5 concludes the paper.
2
Feature Extraction Using PCA
The main steps to extract features using PCA are summarized in the following. The details are given in [5]. Each face image I(x, y) is represented as an N × N vector Γi . 1 M The average face Ψ is computed as: Ψ = M Σi=1 Γi where M is the number of face images in the training set. The difference between each face and the average face is given as: Φi = Γi −Ψ . 1 M A covariance matrix is then estimated as: C = M Σi=1 Φi ΦTi = AAT where A = [Φ1 Φ2 ...ΦM ]. Eigenvectors ui and corresponding eigenvalues λi of the covariance matrix C can be evaluated by using a Singular Value Decomposition (SVD) method [5]: Cui = λi ui . Because matrix C is usually very large (N 2 × N 2 ), evaluating eigenvectors and eigenvalues is very expensive. Instead, eigenvectors vi and corresponding eigen values µi of matrix AT A (M × M ) can be computed. After that, ui can be computed from vi as follows: ui = Avi , j = 1, ..., M . To reduce dimensionality, only a smaller number of eigenvectors K(K
