A New Sparsity-Based Band Selection Method for ... - Semantic Scholar
IEEE GEOSCIENCE AND REMOTE SENSING LETTERS, VOL. 12, NO. 2, FEBRUARY 2015
329
A New Sparsity-Based Band Selection Method for Target Detection of Hyperspectral Image Kang Sun, Xiurui Geng, and Luyan Ji
Abstract—Band selection (BS) plays an important role in the dimensionality reduction of hyperspectral data. However, as to the existing BS methods, few are specially designed for target detection. In this letter, we combine the target detection and BS process together and put forward a new BS method for target detection, named least absolute shrinkage and selection operator (LASSO)-based BS (LBS). Interestingly, by using a linear regression model with L1 regularization (LASSO model), LBS transforms the discrete BS problem into the continuous optimization problem, which cannot only avoid the complicated subset selection process but also evaluate the importance of all the bands simultaneously. The experiments on real hyperspectral data demonstrate that LBS is a very effective BS method for target detection. Index Terms—Band selection (BS), constrained energy minimization (CEM), hyperspectral data, least absolute shrinkage and selection operator (LASSO), target detection.
I. I NTRODUCTION
A
S AN effective dimensionality reduction approach, band selection (BS) has become one of the most important steps in hyperspectral data preprocessing in recent years. By selecting a subset of bands from a larger/full set of bands, BS has two main advantages. First, BS reduces the large redundancy in hyperspectral data, which therefore mitigates the “dimensionality curse” (also known as “Hughes effect”) [1]. Second, BS alleviates the computational burden derived from the large hyperspectral data volume. In addition, compared to the other alternative practice for dimensionality reduction, i.e., feature extraction, BS is more physically meaningful since it keeps the original bands rather than the transformed bands. BS receives increasing interest recently. Many BS methods have been reported in literatures. As to these methods, they can be roughly grouped into two types, namely, supervised and unsupervised. The supervised methods require some prior information such as training samples and target signatures, for instance, [2]–[4]. In general, these methods first define some criteria, such as Jeffreys–Matusita distance [5], classes divergence, and signature angle [6]. Then the subset of bands that maximize (or minimize) the criteria for training samples is Manuscript received May 5, 2014; revised June 16, 2014; accepted July 7, 2014. Date of publication August 5, 2014; date of current version August 21, 2014. K. Sun and X. Geng are with the Key Laboratory of Technology in Geo-spatial Information Processing and Application System, Institute of Electronics, Chinese Academy of Sciences, Beijing 100190, China (e-mail: [email protected]; [email protected]). L. Ji is with the Centre for Earth System Science, Tsinghua University, Beijing 100084, China (e-mail: [email protected]). Color versions of one or more of the figures in this paper are available online at http://ieeexplore.ieee.org. Digital Object Identifier 10.1109/LGRS.2014.2337957
selected. The unsupervised BS techniques are generally based on information evaluation methods and try to select the subset that has the maximum information [7]–[9], minimum similarity [10], [11], maximum information divergence [12], or the other information measurements [13]–[15]. However, these methods, no matter supervised or unsupervised, are mostly proposed for classification, and few of them are specially designed for target detection, although target detection is also one of the most important applications for hyperspectral remote sensing. For instance, supervised methods directly use the classification-related criteria to guide the BS process [2], [6], [16]–[18], whereas most of the unsupervised techniques evaluate the selection results by classification accuracy [11], [13], [14], [19]. In this letter, we propose a new BS method, particularly for small target detection, called least absolute shrinkage and selection operator (LASSO)-based BS (LBS). LBS exploits the fact that, for constrained energy minimization (CEM), the more bands involved, the better performance it has [20]. That is to say, the CEM detection result from the full band set is the best. On the other hand, we know that the detection map of CEM can be regarded as the linear combination of the bands. Interpreting the CEM result as the linear regression, LBS describes the BS problem by a well-studied model for LASSO. As a result, LBS converts the discrete BS problem into a continuous optimization problem, which avoids the complicated subset search operation. LASSO is the expression describing the problem of linear regression with L1 regularizations [21], which already has some sophisticated solving methods. According to the authors’ knowledge, LASSO has not yet been used for BS. We do not intend to explore any new solution for LASSO, but to use the sophisticated LASSO method to accomplish BS tasks for hyperspectral image in context of small target detection. II. M ETHODOLOGY A. Model Constructing CEM [22] is one of the most widely used target detection methods for hyperspectral data. In [20], Geng et al. proved that, the more bands involved in CEM, the better performance of target detection is (with lower objective function). This is a very important conclusion that plays a significant role in the presentation of LBS. It means that the CEM detection result from the full band set is the best, and thus, the removal of any bands will lead to the decrease of the detection performance. In spite of this, we still think that some bands play a crucial role in target detection, and we can obtain a quite acceptable detection performance using only these bands.
1545-598X © 2014 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.
330
IEEE GEOSCIENCE AND REMOTE SENSING LETTERS, VOL. 12, NO. 2, FEBRUARY 2015
Assume that the L-band N -pixel hyperspectral data are denoted by XN ×L = [x1 , x2 , . . . , xL ], where xi = [x1i , x2i , . . . , xN i ]T is the vector constructed by the ith band, and the target signature is dL×1 = [d1 , d2 , . . . , dL ]T . The purpose of CEM is to design a finite-impulse response linear filter w = [w1 , w2 , . . . , wL ]T that keeps the output energy of the target to be unity while suppresses the output energy of background at the same time. If all the bands are involved in CEM, the detection result is R−1 d yN ×1 = Xw = X (1) dT R−1 d where w = R−1 d/dT R−1 d is the CEM operator, and R = (1/N )XT X is the autocorrelation matrix of the data. Suppose we want to select n bands from a total of L bands, without loss of generality, denote the index of selected bands by {a1 , a2 , . . . , an }. Then the CEM detection result using the selected subset is
y=
n w i x ai
(2)
i=1
where w1 denotes the L1 norm of w, corresponding to the sum of absolute values of the elements in w, and t is a parameter that controls the sparsity of w. By now, we have transformed the discrete BS problem for target detection into the continuous optimization problem, which can avoid the time-consuming subset search. In the following, we discuss the solving method for (5).
B. Model Solving In fact, (5) is the famous LASSO problem, i.e., linear regression with L1 regularization, which has been deeply researched. Many influential works that provide different approaches to solve LASSO have been reported [23]–[26]. This letter aims at presenting a new BS method for target detection and, thus, does not explore any other new approaches to solve (5). As to these various solving methods, the method presented in [21] is one of the most influential techniques that use a quadratic programming with nonnegative constraints. We decide to adopt this approach to solve (5), and the specific process is introduced as follows. This method represents each element wi as the difference between two nonnegative variables by
where y is the CEM detection map, and w = T w1 , w2 , · · · , wn is the corresponding CEM operator. Considering the CEM detection map from the full set is the best one, we naturally hope that the detection map from the subset is as close to that from the full set as possible, which can be measured by the following equation: 2 n 2 w i x ai . (3) f w = y − y = y − 2 i=1
2
It can be seen from (1) and (2) that the CEM detection map is the linear combination of the bands with weight wi . Interestingly, although y in (2) is the combination of subset {a1 , a2 , . . . , an }, we can still interpret it as the combination of the full band set by setting the weights of the other (L − n) bands to be zero. As a result, (3) can be converted into 2 L 2 w i xi , f (w) = y − y = y − 2
i=1
s.t. w = n 0
2
(4)
where w0 is the L0 norm for w, measuring the number of nonzero elements in w. However, it is an NP-hard problem to solving L0 regularized optimization. According to the studies reported in recent years, L1 regularization is roughly equivalent to L0 regularization under some conditions. In other words, L1 regularization also makes the solution sparse, but is more easily to be solved. Therefore, we transform (4) into the following optimization problem with L1 regularization:
2 L w i xi , f (w) = y −
i=1
2
s.t. w ≤ t 1
(5)
+
−
wi = wi − wi .
(6) +
−
We can see that if wi is positive, then wi = wi and wi = 0; + − on the contrary, if wi is negative, then wi = 0 and wi = −wi . Thus, the absolute value of wi can be expressed as +
−
|w i | = w i + w i .
(7)
As a result, the intractable constraint w1 ≤ t can be converted in to the following 2L linear inequalities: ⎧ L + − ⎨ w ≤t i + wi (8) ⎩ i=1 + − wi , wi ≥ 0.
Equation (7) can be understood as+ to augment w into a new vector with 2L elements w ˜ = w . Correspondingly, − w 2L×1 ¯ = [X | the data X in the objective function are augmented as X −X]N ×2L . As a result, (5) can be converted into the following formulation: ⎧ 2L ⎨ 2 ˜i ≤ t ¯ ˜ 2 , s.t. i=1 w (9) min y − Xw ⎩ w ˜i ≥ 0. Although (9) doubles the number of involved variables in the problem, it is a standard quadratic programming and can be solved easily (for instance, by quadprog() function in MATLAB). Up to now, we have transformed the BS for target detection into quadratic programming that avoids the complex subset selection. It should be noted that parameter t controls the sparsity of the results and thus determines the number of selected bands.
SUN et al.: NEW SPARSITY-BASED BS METHOD FOR TARGET DETECTION OF HYPERSPECTRAL IMAGE
Fig. 1. target.
331
Xi’an data. (a) Band 15 of the data. (b) CEM detection map of the
We will investigate the relation between the number of selected bands n and the value of t in Section III.
Fig. 2. (a) The solid line denotes the relationship between rn and tn, and the dotted line is the reference line y = x. (b) LBS weight plot for the bands.
III. E XPERIMENTS Here, we will evaluate the performance of LBS with real hyperspectral data. An unsupervised BS method, namely, MEV [8], and two supervised BS methods, namely, BAO [6] and VNVBS [18], are also compared with LBS. In addition, the mean vector of the image is adopted as the reference signature required in BAO and VNVBS. A. Xi’an Data The data set used in this experiment is a real hyperspectral image acquired by Operational Modular Imaging Spectrometer II, which is a hyperspectral imaging system developed by Shanghai Institute of Technical Physics, Chinese Academy of Sciences. These data consist of a total of 64 bands from visible to thermal infrared with a spatial resolution of 3.6 m. We cropped a subset with 200 × 200 pixels for this experiment. Small man-made targets are distributed at two locations around the middle bottom of the image, occupying about 30 pixels. From the raw image [see Fig. 1(a)], the targets are difficult to find visually due to their small size. Fig. 1(b) shows the CEM detection result using full bands. Test 1: Relationship Between Parameter t and Number of Selected Bands n: As aforementioned, parameter t plays a very important role in LBS, controlling the number of selected bands. Let t0 be the L1 norm of the CEM operator derived from the full set. Values of t < t0 will cause the shrinkage of the solutions toward 0; for instance, if t = t0 /2, the effect will be roughly similar to finding the best subset with size of L/2 [21]. Therefore, it can be understood that the relationship between t and t0 roughly corresponds to the relationship between n and L. In this test, we will investigate the relation between t and the number of selected bands n. We first chose various values for t and selected different numbers of bands (real number, rn). To investigate the relation between t and rn, we use L ∗ t/t0 as the “theoretical number” (tn). The result is shown in Fig. 2(a). From Fig. 2(a), we can see that rn and tn are highly correlated (with a correlation coefficient (CC) of 0.9925), although they do not strictly equal to each other. This provides a very reliable reference for the determination of number bands by parameter t. For instance, if the number of bands we wanted is n, the value of t is about t = t0 ∗ n/L. Although the number of the obtained bands is not necessarily n when t = t0 ∗ n/L, it should be close to n.
Fig. 3. CEM results using different numbers of bands (from left to right: 6, 9, 14, 21, and 32) selected by different methods.
As for these data, t0 is 0.0453, and we chose t = 0.009 and obtained the following weights [shown in Fig. 2(b)]. From Fig. 2(b), it can be seen that LBS can shrink the weight coefficient effectively. Most of the weights are 0, and only a few of them are nonzero. The bands corresponding to these nonzero weights are the exact bands we need. Test 2: Accuracy Comparison: We have assigned different values to t and actually selected 6, 9, 14, 21, and 32 bands by LBS, respectively. For fair comparison, we also selected the same number of bands by MEV, VNVBS, and BAO, respectively. Then the selected subsets are used in CEM for target detection. The detection maps are shown in Fig. 3. We can see from Fig. 3 that the CEM detection performance for all the methods is improving with the increase in the number of selected bands. This is consistent with the conclusion in [20]. Bands selected by LBS always have the best performance in terms of target detection, particularly when the number of selected bands is relatively small. For instance, when the number of selected bands is 6, only LBS-selected subset can separate the target from background successfully. On the other hand, when the number of selected bands is large (for instance ≥14),
332
IEEE GEOSCIENCE AND REMOTE SENSING LETTERS, VOL. 12, NO. 2, FEBRUARY 2015
Fig. 5. Subscene used in this experiment. (a) False color map. (b) Spectrum of Buddingtonite. (c) Manually made ground truth for Buddingtonite. Fig. 4. Quantitative comparison of the methods. (a) CCs for CEM results from selected bands and full bands. (b) Output energy.
all the selected subsets can distinguish the target in acceptable levels, although with different suppression to the background. The detection maps for LBS and MEV are difficult to be visually evaluated when 32 bands are selected. Therefore, we also compared the performance of these methods quantitatively. Since the detection map from the full band set is the best, we used CCs between the CEM detection maps from selected bands (see Fig. 3) and that from full bands [see Fig. 1(b)] as the evaluation measure. The corresponding CCs for different methods are shown in Fig. 4(a). Moreover, we also compared the value of the CEM objective function (output energy) for the subsets selected by different methods [see Fig. 4(b)]. The subset with lower output energy is more effective since the goal of CEM is to minimize the output energy. From Fig. 4(a), we can see that the CEM detection results from subsets selected by LBS always have the highest CCs, with the most noticeable when the number of selected bands is small. Fig. 4(b) demonstrates that the output energy of CEM is the lowest when all the bands are involved. As to these BS methods, the CEM output energies of subsets selected LBS are always the lowest, corresponding to the highest CCs shown in Fig. 4(a). The highest CCs and the lowest output energies of LBS demonstrate that LBS is the most effective method in terms of small target detection. This can be attributed to the fact that LBS is designed specially for target detection and is guided by the CEM detection map from the full set during the selection procedure. Thus, it takes full advantage of the target and background information. The other methods, namely, MEV, BAO, and VNVBS, although can be used as BS methods for target detection, do not take the target and background into account simultaneously. Therefore, they are less effective in terms of target detection. B. Cuprite Data In order to further verify the effectiveness of our method, we conduct another experiment using the widely used hyperspectral data, namely, Cuprite, which were acquired by Airborne Visible Infrared Imaging Spectrometer over Nevada in 1997. We cropped a subscene [see Fig. 5(a)] with a size of 350 × 350 pixels and 224 bands. The noisy and low signal-to-noise-ratio bands, numbered 1–3, 105–115, 150–170, and 221–224, are manually removed, and therefore, the remainder 185 bands participate in the following experiment. The Buddingtonite mineral is selected as the small target of interest due to its relatively low probability distribution. The spectrum of Bud-
Fig. 6. CEM detection map for Buddingtonite using the subset selected by (a) LBS, (b) MEV, (c) BAO, and (d) VNVBS.
dingtonite [see Fig. 5(b)] can be extracted by some endmember extraction methods, such as [27] and [28]. It is noteworthy that the ground truth [see Fig. 5(c)] for Buddingtonite is manually made according to the fully constrained unmixing results and ground investigation [29], since no pixel-level ground truth is available. We selected 15 bands by these BS methods from the involved 185 bands. Then the selected subsets are used in CEM detection. The detection maps are shown in Fig. 6. It can be seen from Fig. 6 that the subsets selected by LBS and BAO can separate the target from the background successfully. The background is suppressed more heavily by the subset selected by LBS compared to BAO. The other two BS methods, namely, MEV and VNVBS, failed to select the subsets that can separate the target from the background. Combining the results from Xi’an data, we can see that although the three compared BS methods, namely, MEV, BAO, and VNVBS, can sometimes select the right subset to separate the target from background, they are not robust. For instance, MEV and VBVBS seem to perform well for Xi’an data, but they have poor performance for Cuprite data. Contrarily, BAO has a much better performance for Cuprite data than Xi’an data. This can be attributed to the fact that these three methods have taken only part of target and background information into account. The LBS considers the target and background simultaneously and therefore has robust performance for different data sets. To quantitatively compare these methods, we use receiver operating characteristic (ROC) for the detection maps (see Fig. 7). The ROC curves show that the subset selected by LBS has the best detection performance since it has the highest detection probability at the same false alarm ratio. Compared with LBS, BAO has a lower detection probability, which can be ascribed to the insufficient suppression to background. The other two methods, namely, MEV and VNVBS, have very poor performance in terms of target detection. IV. C ONCLUSION In this letter, we have proposed a new BS method for target detection called LBS. LBS exploits the fact that the
SUN et al.: NEW SPARSITY-BASED BS METHOD FOR TARGET DETECTION OF HYPERSPECTRAL IMAGE
Fig. 7.
ROC curves for CEM detection maps from different BS methods.
performance of CEM is increasing with the increase in the number of bands. Interestingly, LBS formulates the discrete BS problem by a continuous optimization model (LASSO) and therefore can evaluate all the bands at the same time. The experimental results verify the effectiveness of LBS in terms of target detection. The validity of LBS can be attributed to the simultaneous consideration of target and background information. We deem it necessary to take both the target and background information into account for BS in the context of target detection. Some BS techniques try to select the informative bands (for instance, MEV and some information-theory-based methods). Although these methods have good performance in terms of classification, they may ignore the small targets. Thus, they may be not suitable BS methods for target detection. There are also supervised BS methods based on spectra discrimination, for instance, BAO and VNVBS. They do use target signature, but their limitation is that they use a single reference signature to represent background information. A single mean spectrum cannot sufficiently reflect the background information since the background generally contains many ground objects. For LBS, one of the problems is the determination of the number of bands n. LBS can control the number of bands roughly by adjusting parameter t, but the control is not exact. How to determine the number of bands more exactly and adaptively needs to be further studied. R EFERENCES [1] B. Kim and D. A. Landgrebe, “Hierarchical classifier design in highdimensional numerous class cases,” IEEE Trans. Geosci. Remote Sens., vol. 29, no. 4, pp. 518–528, Jul. 1991. [2] B.-C. Kuo, H.-H. Ho, C.-H. Li, C.-C. Hung, and J.-S. Taur, “A kernelbased feature selection method for SVM with RBF kernel for hyperspectral image classification,” IEEE J. Sel. Topics Appl. Earth Observ. Remote Sens., vol. 7, no. 2, pp. 317–326, Jan. 2014. [3] C. Yang, S. Liu, L. Bruzzone, R. Guan, and P. Du, “A feature-metric-based affinity propagation technique for feature selection in hyperspectral image classification,” IEEE Geosci. Remote Sens. Lett., vol. 10, no. 5, pp. 1152– 1156, Sep. 2013. [4] H. Yang, Q. Du, H. Su, and Y. Sheng, “An efficient method for supervised hyperspectral band selection,” IEEE Geosci. Remote Sens. Lett., vol. 8, no. 1, pp. 138–142, Jan. 2011. [5] L. Bruzzone, F. Roli, and S. B. Serpico, “An extension of the Jeffreys–Matusita distance to multiclass cases for feature selection,” IEEE Trans. Geosci. Remote Sens., vol. 33, no. 6, pp. 1318–1321, Nov. 1995.
333
[6] N. Keshava, “Distance metrics and band selection in hyperspectral processing with applications to material identification and spectral libraries,” IEEE Trans. Geosci. Remote Sens., vol. 42, no. 7, pp. 1552–1565, Jul. 2004. [7] P. S. Chacvez, G. L. Berlin, and B. Sowers, “Statistical method for selecting Landsat MSS ratios,” J. Appl. Photograph. Eng., vol. 1, no. 8, pp. 23–30, 1982. [8] C. Sheffield, “Selecting band combinations from multispectral data,” Photogramm. Eng. Remote Sens., vol. 51, no. 6, pp. 681–687, 1985. [9] C.-I. Chang, Q. Du, T.-L. Sun, and T.-L. Sun, “A joint band prioritization and band-decorrelation approach to band selection for hyperspectral image classification,” IEEE Trans. Geosci. Remote Sens., vol. 37, no. 6, pp. 2631–2641, Nov. 1999. [10] P. Mitra, C. Murthy, and S. K. Pal, “Unsupervised feature selection using feature similarity,” IEEE Trans. Pattern Anal. Mach. Intell., vol. 24, no. 3, pp. 301–312, Mar. 2002. [11] Q. Du and H. Yang, “Similarity-based unsupervised band selection for hyperspectral image analysis,” IEEE Geosci. Remote Sens. Lett., vol. 5, no. 4, pp. 564–568, Oct. 2008. [12] C.-I. Chang and S. Wang, “Constrained band selection for hyperspectral imagery,” IEEE Trans. Geosci. Remote Sens., vol. 44, no. 6, pp. 1575– 1585, Jun. 2006. [13] S. Jia, Z. Ji, Y. Qian, and L. Shen, “Unsupervised band selection for hyperspectral imagery classification without manual band removal,” IEEE J. Sel. Topics Appl. Earth Observ. Remote Sens., vol. 5, no. 2, pp. 531–543, Apr. 2012. [14] A. Martínez-Usó, F. Pla, J. M. Sotoca et al., “Clustering-based hyperspectral band selection using information measures,” IEEE Trans. Geosci. Remote Sens., vol. 45, no. 12, pp. 4158–4171, Dec. 2007. [15] B. Mojaradi, H. Emami, M. Varshosaz, and S. Varshosaz, “A novel band selection method for hyperspectral data analysis,” Int. Arch. Photogramm. Remote Sens. Spatial Inf. Sci., vol. 37, pp. 447–454, 2008. [16] L. Bruzzone and S. B. Serpico, “A technique for feature selection in multiclass problems,” Int. J. Remote Sens., vol. 21, no. 3, pp. 549–563, 2000. [17] H. Su and Q. Du, “Hyperspectral band clustering and band selection for urban land cover classification,” Geocarto Int., vol. 27, no. 5, pp. 395– 411, 2012. [18] S. Wang and C.-I. Chang, “Variable-number variable-band selection for feature characterization in hyperspectral signatures,” IEEE Trans. Geosci. Remote Sens., vol. 45, no. 9, pp. 2979–2992, Sep. 2007. [19] M. G. Asl, M. R. Mobasheri, and B. Mojaradi, “Unsupervised feature selection using geometrical measures in prototype space for hyperspectral imagery,” IEEE Trans. Geosci. Remote Sens., vol. 52, no. 7, pp. 3774– 3787, Jul. 2014. [20] X. Geng, L. Ji, K. Sun, and Y. Zhao, “CEM: More bands, better performance,” IEEE Geosci. Remote Sens. Lett., vol. 11, no. 11, pp. 1876–1880, Nov. 2014. [21] R. Tibshirani, “Regression shrinkage and selection via the LASSO,” J. R. Statist. Soc. Series B (Methodological), vol. 73, no. 3, pp. 267–288, Jun. 1996. [22] C.-I. Chang, Hyperspectral Imaging: Techniques for Spectral Detection and Classification. New York, NY, USA: Plenum, 2003. [23] S. S. Chen, D. L. Donoho, and M. A. Saunders, “Atomic decomposition by basis pursuit,” SIAM J. Sci. Comput., vol. 20, no. 1, pp. 33–61, 1999. [24] M. Osborne, B. Presnell, and B. Turlach, “On the LASSO and its dual,” J. Comput. Graph. Statist., vol. 9, no. 2, pp. 319–337, 2000. [25] K. S. Shevade and S. S. Keerthi, “A simple and efficient method for gene selection using sparse logistic regression,” Bioinformatics, vol. 19, no. 17, pp. 2246–2253, 2003. [26] B. Efron, T. Hastie, I. Johnstone, and R. Tibshirani, “Least angle regression,” Ann. Statist., vol. 32, no. 2, pp. 407–451, 2004. [27] K. Sun, X. Geng, P. Wang, and Y. Zhao, “A fast endmember extraction algorithm based on Gram determinant,” IEEE Geosci. Remote Sens. Lett., vol. 11, no. 6, pp. 1124–1128, Jun. 2014. [28] X. Geng, Z. Xiao, L. Ji, Y. Zhao, and F. Wang, “A Gaussian elimination based fast endmember extraction algorithm for hyperspectral imagery,” ISPRS J. Photogramm. Remote Sens., vol. 79, pp. 211–218, May 2013. [29] G. Swayze, R. Clark, F. Kruse, S. Sutley, and A. Gallagher, “Groundtruthing AVIRIS mineral mapping at Cuprite, Nevada,” in Proc. Summaries 3rd Annu. JPL Airborne Geosci. Workshop, 1992, pp. 47–49.
329
A New Sparsity-Based Band Selection Method for Target Detection of Hyperspectral Image Kang Sun, Xiurui Geng, and Luyan Ji
Abstract—Band selection (BS) plays an important role in the dimensionality reduction of hyperspectral data. However, as to the existing BS methods, few are specially designed for target detection. In this letter, we combine the target detection and BS process together and put forward a new BS method for target detection, named least absolute shrinkage and selection operator (LASSO)-based BS (LBS). Interestingly, by using a linear regression model with L1 regularization (LASSO model), LBS transforms the discrete BS problem into the continuous optimization problem, which cannot only avoid the complicated subset selection process but also evaluate the importance of all the bands simultaneously. The experiments on real hyperspectral data demonstrate that LBS is a very effective BS method for target detection. Index Terms—Band selection (BS), constrained energy minimization (CEM), hyperspectral data, least absolute shrinkage and selection operator (LASSO), target detection.
I. I NTRODUCTION
A
S AN effective dimensionality reduction approach, band selection (BS) has become one of the most important steps in hyperspectral data preprocessing in recent years. By selecting a subset of bands from a larger/full set of bands, BS has two main advantages. First, BS reduces the large redundancy in hyperspectral data, which therefore mitigates the “dimensionality curse” (also known as “Hughes effect”) [1]. Second, BS alleviates the computational burden derived from the large hyperspectral data volume. In addition, compared to the other alternative practice for dimensionality reduction, i.e., feature extraction, BS is more physically meaningful since it keeps the original bands rather than the transformed bands. BS receives increasing interest recently. Many BS methods have been reported in literatures. As to these methods, they can be roughly grouped into two types, namely, supervised and unsupervised. The supervised methods require some prior information such as training samples and target signatures, for instance, [2]–[4]. In general, these methods first define some criteria, such as Jeffreys–Matusita distance [5], classes divergence, and signature angle [6]. Then the subset of bands that maximize (or minimize) the criteria for training samples is Manuscript received May 5, 2014; revised June 16, 2014; accepted July 7, 2014. Date of publication August 5, 2014; date of current version August 21, 2014. K. Sun and X. Geng are with the Key Laboratory of Technology in Geo-spatial Information Processing and Application System, Institute of Electronics, Chinese Academy of Sciences, Beijing 100190, China (e-mail: [email protected]; [email protected]). L. Ji is with the Centre for Earth System Science, Tsinghua University, Beijing 100084, China (e-mail: [email protected]). Color versions of one or more of the figures in this paper are available online at http://ieeexplore.ieee.org. Digital Object Identifier 10.1109/LGRS.2014.2337957
selected. The unsupervised BS techniques are generally based on information evaluation methods and try to select the subset that has the maximum information [7]–[9], minimum similarity [10], [11], maximum information divergence [12], or the other information measurements [13]–[15]. However, these methods, no matter supervised or unsupervised, are mostly proposed for classification, and few of them are specially designed for target detection, although target detection is also one of the most important applications for hyperspectral remote sensing. For instance, supervised methods directly use the classification-related criteria to guide the BS process [2], [6], [16]–[18], whereas most of the unsupervised techniques evaluate the selection results by classification accuracy [11], [13], [14], [19]. In this letter, we propose a new BS method, particularly for small target detection, called least absolute shrinkage and selection operator (LASSO)-based BS (LBS). LBS exploits the fact that, for constrained energy minimization (CEM), the more bands involved, the better performance it has [20]. That is to say, the CEM detection result from the full band set is the best. On the other hand, we know that the detection map of CEM can be regarded as the linear combination of the bands. Interpreting the CEM result as the linear regression, LBS describes the BS problem by a well-studied model for LASSO. As a result, LBS converts the discrete BS problem into a continuous optimization problem, which avoids the complicated subset search operation. LASSO is the expression describing the problem of linear regression with L1 regularizations [21], which already has some sophisticated solving methods. According to the authors’ knowledge, LASSO has not yet been used for BS. We do not intend to explore any new solution for LASSO, but to use the sophisticated LASSO method to accomplish BS tasks for hyperspectral image in context of small target detection. II. M ETHODOLOGY A. Model Constructing CEM [22] is one of the most widely used target detection methods for hyperspectral data. In [20], Geng et al. proved that, the more bands involved in CEM, the better performance of target detection is (with lower objective function). This is a very important conclusion that plays a significant role in the presentation of LBS. It means that the CEM detection result from the full band set is the best, and thus, the removal of any bands will lead to the decrease of the detection performance. In spite of this, we still think that some bands play a crucial role in target detection, and we can obtain a quite acceptable detection performance using only these bands.
1545-598X © 2014 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.
330
IEEE GEOSCIENCE AND REMOTE SENSING LETTERS, VOL. 12, NO. 2, FEBRUARY 2015
Assume that the L-band N -pixel hyperspectral data are denoted by XN ×L = [x1 , x2 , . . . , xL ], where xi = [x1i , x2i , . . . , xN i ]T is the vector constructed by the ith band, and the target signature is dL×1 = [d1 , d2 , . . . , dL ]T . The purpose of CEM is to design a finite-impulse response linear filter w = [w1 , w2 , . . . , wL ]T that keeps the output energy of the target to be unity while suppresses the output energy of background at the same time. If all the bands are involved in CEM, the detection result is R−1 d yN ×1 = Xw = X (1) dT R−1 d where w = R−1 d/dT R−1 d is the CEM operator, and R = (1/N )XT X is the autocorrelation matrix of the data. Suppose we want to select n bands from a total of L bands, without loss of generality, denote the index of selected bands by {a1 , a2 , . . . , an }. Then the CEM detection result using the selected subset is
y=
n w i x ai
(2)
i=1
where w1 denotes the L1 norm of w, corresponding to the sum of absolute values of the elements in w, and t is a parameter that controls the sparsity of w. By now, we have transformed the discrete BS problem for target detection into the continuous optimization problem, which can avoid the time-consuming subset search. In the following, we discuss the solving method for (5).
B. Model Solving In fact, (5) is the famous LASSO problem, i.e., linear regression with L1 regularization, which has been deeply researched. Many influential works that provide different approaches to solve LASSO have been reported [23]–[26]. This letter aims at presenting a new BS method for target detection and, thus, does not explore any other new approaches to solve (5). As to these various solving methods, the method presented in [21] is one of the most influential techniques that use a quadratic programming with nonnegative constraints. We decide to adopt this approach to solve (5), and the specific process is introduced as follows. This method represents each element wi as the difference between two nonnegative variables by
where y is the CEM detection map, and w = T w1 , w2 , · · · , wn is the corresponding CEM operator. Considering the CEM detection map from the full set is the best one, we naturally hope that the detection map from the subset is as close to that from the full set as possible, which can be measured by the following equation: 2 n 2 w i x ai . (3) f w = y − y = y − 2 i=1
2
It can be seen from (1) and (2) that the CEM detection map is the linear combination of the bands with weight wi . Interestingly, although y in (2) is the combination of subset {a1 , a2 , . . . , an }, we can still interpret it as the combination of the full band set by setting the weights of the other (L − n) bands to be zero. As a result, (3) can be converted into 2 L 2 w i xi , f (w) = y − y = y − 2
i=1
s.t. w = n 0
2
(4)
where w0 is the L0 norm for w, measuring the number of nonzero elements in w. However, it is an NP-hard problem to solving L0 regularized optimization. According to the studies reported in recent years, L1 regularization is roughly equivalent to L0 regularization under some conditions. In other words, L1 regularization also makes the solution sparse, but is more easily to be solved. Therefore, we transform (4) into the following optimization problem with L1 regularization:
2 L w i xi , f (w) = y −
i=1
2
s.t. w ≤ t 1
(5)
+
−
wi = wi − wi .
(6) +
−
We can see that if wi is positive, then wi = wi and wi = 0; + − on the contrary, if wi is negative, then wi = 0 and wi = −wi . Thus, the absolute value of wi can be expressed as +
−
|w i | = w i + w i .
(7)
As a result, the intractable constraint w1 ≤ t can be converted in to the following 2L linear inequalities: ⎧ L + − ⎨ w ≤t i + wi (8) ⎩ i=1 + − wi , wi ≥ 0.
Equation (7) can be understood as+ to augment w into a new vector with 2L elements w ˜ = w . Correspondingly, − w 2L×1 ¯ = [X | the data X in the objective function are augmented as X −X]N ×2L . As a result, (5) can be converted into the following formulation: ⎧ 2L ⎨ 2 ˜i ≤ t ¯ ˜ 2 , s.t. i=1 w (9) min y − Xw ⎩ w ˜i ≥ 0. Although (9) doubles the number of involved variables in the problem, it is a standard quadratic programming and can be solved easily (for instance, by quadprog() function in MATLAB). Up to now, we have transformed the BS for target detection into quadratic programming that avoids the complex subset selection. It should be noted that parameter t controls the sparsity of the results and thus determines the number of selected bands.
SUN et al.: NEW SPARSITY-BASED BS METHOD FOR TARGET DETECTION OF HYPERSPECTRAL IMAGE
Fig. 1. target.
331
Xi’an data. (a) Band 15 of the data. (b) CEM detection map of the
We will investigate the relation between the number of selected bands n and the value of t in Section III.
Fig. 2. (a) The solid line denotes the relationship between rn and tn, and the dotted line is the reference line y = x. (b) LBS weight plot for the bands.
III. E XPERIMENTS Here, we will evaluate the performance of LBS with real hyperspectral data. An unsupervised BS method, namely, MEV [8], and two supervised BS methods, namely, BAO [6] and VNVBS [18], are also compared with LBS. In addition, the mean vector of the image is adopted as the reference signature required in BAO and VNVBS. A. Xi’an Data The data set used in this experiment is a real hyperspectral image acquired by Operational Modular Imaging Spectrometer II, which is a hyperspectral imaging system developed by Shanghai Institute of Technical Physics, Chinese Academy of Sciences. These data consist of a total of 64 bands from visible to thermal infrared with a spatial resolution of 3.6 m. We cropped a subset with 200 × 200 pixels for this experiment. Small man-made targets are distributed at two locations around the middle bottom of the image, occupying about 30 pixels. From the raw image [see Fig. 1(a)], the targets are difficult to find visually due to their small size. Fig. 1(b) shows the CEM detection result using full bands. Test 1: Relationship Between Parameter t and Number of Selected Bands n: As aforementioned, parameter t plays a very important role in LBS, controlling the number of selected bands. Let t0 be the L1 norm of the CEM operator derived from the full set. Values of t < t0 will cause the shrinkage of the solutions toward 0; for instance, if t = t0 /2, the effect will be roughly similar to finding the best subset with size of L/2 [21]. Therefore, it can be understood that the relationship between t and t0 roughly corresponds to the relationship between n and L. In this test, we will investigate the relation between t and the number of selected bands n. We first chose various values for t and selected different numbers of bands (real number, rn). To investigate the relation between t and rn, we use L ∗ t/t0 as the “theoretical number” (tn). The result is shown in Fig. 2(a). From Fig. 2(a), we can see that rn and tn are highly correlated (with a correlation coefficient (CC) of 0.9925), although they do not strictly equal to each other. This provides a very reliable reference for the determination of number bands by parameter t. For instance, if the number of bands we wanted is n, the value of t is about t = t0 ∗ n/L. Although the number of the obtained bands is not necessarily n when t = t0 ∗ n/L, it should be close to n.
Fig. 3. CEM results using different numbers of bands (from left to right: 6, 9, 14, 21, and 32) selected by different methods.
As for these data, t0 is 0.0453, and we chose t = 0.009 and obtained the following weights [shown in Fig. 2(b)]. From Fig. 2(b), it can be seen that LBS can shrink the weight coefficient effectively. Most of the weights are 0, and only a few of them are nonzero. The bands corresponding to these nonzero weights are the exact bands we need. Test 2: Accuracy Comparison: We have assigned different values to t and actually selected 6, 9, 14, 21, and 32 bands by LBS, respectively. For fair comparison, we also selected the same number of bands by MEV, VNVBS, and BAO, respectively. Then the selected subsets are used in CEM for target detection. The detection maps are shown in Fig. 3. We can see from Fig. 3 that the CEM detection performance for all the methods is improving with the increase in the number of selected bands. This is consistent with the conclusion in [20]. Bands selected by LBS always have the best performance in terms of target detection, particularly when the number of selected bands is relatively small. For instance, when the number of selected bands is 6, only LBS-selected subset can separate the target from background successfully. On the other hand, when the number of selected bands is large (for instance ≥14),
332
IEEE GEOSCIENCE AND REMOTE SENSING LETTERS, VOL. 12, NO. 2, FEBRUARY 2015
Fig. 5. Subscene used in this experiment. (a) False color map. (b) Spectrum of Buddingtonite. (c) Manually made ground truth for Buddingtonite. Fig. 4. Quantitative comparison of the methods. (a) CCs for CEM results from selected bands and full bands. (b) Output energy.
all the selected subsets can distinguish the target in acceptable levels, although with different suppression to the background. The detection maps for LBS and MEV are difficult to be visually evaluated when 32 bands are selected. Therefore, we also compared the performance of these methods quantitatively. Since the detection map from the full band set is the best, we used CCs between the CEM detection maps from selected bands (see Fig. 3) and that from full bands [see Fig. 1(b)] as the evaluation measure. The corresponding CCs for different methods are shown in Fig. 4(a). Moreover, we also compared the value of the CEM objective function (output energy) for the subsets selected by different methods [see Fig. 4(b)]. The subset with lower output energy is more effective since the goal of CEM is to minimize the output energy. From Fig. 4(a), we can see that the CEM detection results from subsets selected by LBS always have the highest CCs, with the most noticeable when the number of selected bands is small. Fig. 4(b) demonstrates that the output energy of CEM is the lowest when all the bands are involved. As to these BS methods, the CEM output energies of subsets selected LBS are always the lowest, corresponding to the highest CCs shown in Fig. 4(a). The highest CCs and the lowest output energies of LBS demonstrate that LBS is the most effective method in terms of small target detection. This can be attributed to the fact that LBS is designed specially for target detection and is guided by the CEM detection map from the full set during the selection procedure. Thus, it takes full advantage of the target and background information. The other methods, namely, MEV, BAO, and VNVBS, although can be used as BS methods for target detection, do not take the target and background into account simultaneously. Therefore, they are less effective in terms of target detection. B. Cuprite Data In order to further verify the effectiveness of our method, we conduct another experiment using the widely used hyperspectral data, namely, Cuprite, which were acquired by Airborne Visible Infrared Imaging Spectrometer over Nevada in 1997. We cropped a subscene [see Fig. 5(a)] with a size of 350 × 350 pixels and 224 bands. The noisy and low signal-to-noise-ratio bands, numbered 1–3, 105–115, 150–170, and 221–224, are manually removed, and therefore, the remainder 185 bands participate in the following experiment. The Buddingtonite mineral is selected as the small target of interest due to its relatively low probability distribution. The spectrum of Bud-
Fig. 6. CEM detection map for Buddingtonite using the subset selected by (a) LBS, (b) MEV, (c) BAO, and (d) VNVBS.
dingtonite [see Fig. 5(b)] can be extracted by some endmember extraction methods, such as [27] and [28]. It is noteworthy that the ground truth [see Fig. 5(c)] for Buddingtonite is manually made according to the fully constrained unmixing results and ground investigation [29], since no pixel-level ground truth is available. We selected 15 bands by these BS methods from the involved 185 bands. Then the selected subsets are used in CEM detection. The detection maps are shown in Fig. 6. It can be seen from Fig. 6 that the subsets selected by LBS and BAO can separate the target from the background successfully. The background is suppressed more heavily by the subset selected by LBS compared to BAO. The other two BS methods, namely, MEV and VNVBS, failed to select the subsets that can separate the target from the background. Combining the results from Xi’an data, we can see that although the three compared BS methods, namely, MEV, BAO, and VNVBS, can sometimes select the right subset to separate the target from background, they are not robust. For instance, MEV and VBVBS seem to perform well for Xi’an data, but they have poor performance for Cuprite data. Contrarily, BAO has a much better performance for Cuprite data than Xi’an data. This can be attributed to the fact that these three methods have taken only part of target and background information into account. The LBS considers the target and background simultaneously and therefore has robust performance for different data sets. To quantitatively compare these methods, we use receiver operating characteristic (ROC) for the detection maps (see Fig. 7). The ROC curves show that the subset selected by LBS has the best detection performance since it has the highest detection probability at the same false alarm ratio. Compared with LBS, BAO has a lower detection probability, which can be ascribed to the insufficient suppression to background. The other two methods, namely, MEV and VNVBS, have very poor performance in terms of target detection. IV. C ONCLUSION In this letter, we have proposed a new BS method for target detection called LBS. LBS exploits the fact that the
SUN et al.: NEW SPARSITY-BASED BS METHOD FOR TARGET DETECTION OF HYPERSPECTRAL IMAGE
Fig. 7.
ROC curves for CEM detection maps from different BS methods.
performance of CEM is increasing with the increase in the number of bands. Interestingly, LBS formulates the discrete BS problem by a continuous optimization model (LASSO) and therefore can evaluate all the bands at the same time. The experimental results verify the effectiveness of LBS in terms of target detection. The validity of LBS can be attributed to the simultaneous consideration of target and background information. We deem it necessary to take both the target and background information into account for BS in the context of target detection. Some BS techniques try to select the informative bands (for instance, MEV and some information-theory-based methods). Although these methods have good performance in terms of classification, they may ignore the small targets. Thus, they may be not suitable BS methods for target detection. There are also supervised BS methods based on spectra discrimination, for instance, BAO and VNVBS. They do use target signature, but their limitation is that they use a single reference signature to represent background information. A single mean spectrum cannot sufficiently reflect the background information since the background generally contains many ground objects. For LBS, one of the problems is the determination of the number of bands n. LBS can control the number of bands roughly by adjusting parameter t, but the control is not exact. How to determine the number of bands more exactly and adaptively needs to be further studied. R EFERENCES [1] B. Kim and D. A. Landgrebe, “Hierarchical classifier design in highdimensional numerous class cases,” IEEE Trans. Geosci. Remote Sens., vol. 29, no. 4, pp. 518–528, Jul. 1991. [2] B.-C. Kuo, H.-H. Ho, C.-H. Li, C.-C. Hung, and J.-S. Taur, “A kernelbased feature selection method for SVM with RBF kernel for hyperspectral image classification,” IEEE J. Sel. Topics Appl. Earth Observ. Remote Sens., vol. 7, no. 2, pp. 317–326, Jan. 2014. [3] C. Yang, S. Liu, L. Bruzzone, R. Guan, and P. Du, “A feature-metric-based affinity propagation technique for feature selection in hyperspectral image classification,” IEEE Geosci. Remote Sens. Lett., vol. 10, no. 5, pp. 1152– 1156, Sep. 2013. [4] H. Yang, Q. Du, H. Su, and Y. Sheng, “An efficient method for supervised hyperspectral band selection,” IEEE Geosci. Remote Sens. Lett., vol. 8, no. 1, pp. 138–142, Jan. 2011. [5] L. Bruzzone, F. Roli, and S. B. Serpico, “An extension of the Jeffreys–Matusita distance to multiclass cases for feature selection,” IEEE Trans. Geosci. Remote Sens., vol. 33, no. 6, pp. 1318–1321, Nov. 1995.
333
[6] N. Keshava, “Distance metrics and band selection in hyperspectral processing with applications to material identification and spectral libraries,” IEEE Trans. Geosci. Remote Sens., vol. 42, no. 7, pp. 1552–1565, Jul. 2004. [7] P. S. Chacvez, G. L. Berlin, and B. Sowers, “Statistical method for selecting Landsat MSS ratios,” J. Appl. Photograph. Eng., vol. 1, no. 8, pp. 23–30, 1982. [8] C. Sheffield, “Selecting band combinations from multispectral data,” Photogramm. Eng. Remote Sens., vol. 51, no. 6, pp. 681–687, 1985. [9] C.-I. Chang, Q. Du, T.-L. Sun, and T.-L. Sun, “A joint band prioritization and band-decorrelation approach to band selection for hyperspectral image classification,” IEEE Trans. Geosci. Remote Sens., vol. 37, no. 6, pp. 2631–2641, Nov. 1999. [10] P. Mitra, C. Murthy, and S. K. Pal, “Unsupervised feature selection using feature similarity,” IEEE Trans. Pattern Anal. Mach. Intell., vol. 24, no. 3, pp. 301–312, Mar. 2002. [11] Q. Du and H. Yang, “Similarity-based unsupervised band selection for hyperspectral image analysis,” IEEE Geosci. Remote Sens. Lett., vol. 5, no. 4, pp. 564–568, Oct. 2008. [12] C.-I. Chang and S. Wang, “Constrained band selection for hyperspectral imagery,” IEEE Trans. Geosci. Remote Sens., vol. 44, no. 6, pp. 1575– 1585, Jun. 2006. [13] S. Jia, Z. Ji, Y. Qian, and L. Shen, “Unsupervised band selection for hyperspectral imagery classification without manual band removal,” IEEE J. Sel. Topics Appl. Earth Observ. Remote Sens., vol. 5, no. 2, pp. 531–543, Apr. 2012. [14] A. Martínez-Usó, F. Pla, J. M. Sotoca et al., “Clustering-based hyperspectral band selection using information measures,” IEEE Trans. Geosci. Remote Sens., vol. 45, no. 12, pp. 4158–4171, Dec. 2007. [15] B. Mojaradi, H. Emami, M. Varshosaz, and S. Varshosaz, “A novel band selection method for hyperspectral data analysis,” Int. Arch. Photogramm. Remote Sens. Spatial Inf. Sci., vol. 37, pp. 447–454, 2008. [16] L. Bruzzone and S. B. Serpico, “A technique for feature selection in multiclass problems,” Int. J. Remote Sens., vol. 21, no. 3, pp. 549–563, 2000. [17] H. Su and Q. Du, “Hyperspectral band clustering and band selection for urban land cover classification,” Geocarto Int., vol. 27, no. 5, pp. 395– 411, 2012. [18] S. Wang and C.-I. Chang, “Variable-number variable-band selection for feature characterization in hyperspectral signatures,” IEEE Trans. Geosci. Remote Sens., vol. 45, no. 9, pp. 2979–2992, Sep. 2007. [19] M. G. Asl, M. R. Mobasheri, and B. Mojaradi, “Unsupervised feature selection using geometrical measures in prototype space for hyperspectral imagery,” IEEE Trans. Geosci. Remote Sens., vol. 52, no. 7, pp. 3774– 3787, Jul. 2014. [20] X. Geng, L. Ji, K. Sun, and Y. Zhao, “CEM: More bands, better performance,” IEEE Geosci. Remote Sens. Lett., vol. 11, no. 11, pp. 1876–1880, Nov. 2014. [21] R. Tibshirani, “Regression shrinkage and selection via the LASSO,” J. R. Statist. Soc. Series B (Methodological), vol. 73, no. 3, pp. 267–288, Jun. 1996. [22] C.-I. Chang, Hyperspectral Imaging: Techniques for Spectral Detection and Classification. New York, NY, USA: Plenum, 2003. [23] S. S. Chen, D. L. Donoho, and M. A. Saunders, “Atomic decomposition by basis pursuit,” SIAM J. Sci. Comput., vol. 20, no. 1, pp. 33–61, 1999. [24] M. Osborne, B. Presnell, and B. Turlach, “On the LASSO and its dual,” J. Comput. Graph. Statist., vol. 9, no. 2, pp. 319–337, 2000. [25] K. S. Shevade and S. S. Keerthi, “A simple and efficient method for gene selection using sparse logistic regression,” Bioinformatics, vol. 19, no. 17, pp. 2246–2253, 2003. [26] B. Efron, T. Hastie, I. Johnstone, and R. Tibshirani, “Least angle regression,” Ann. Statist., vol. 32, no. 2, pp. 407–451, 2004. [27] K. Sun, X. Geng, P. Wang, and Y. Zhao, “A fast endmember extraction algorithm based on Gram determinant,” IEEE Geosci. Remote Sens. Lett., vol. 11, no. 6, pp. 1124–1128, Jun. 2014. [28] X. Geng, Z. Xiao, L. Ji, Y. Zhao, and F. Wang, “A Gaussian elimination based fast endmember extraction algorithm for hyperspectral imagery,” ISPRS J. Photogramm. Remote Sens., vol. 79, pp. 211–218, May 2013. [29] G. Swayze, R. Clark, F. Kruse, S. Sutley, and A. Gallagher, “Groundtruthing AVIRIS mineral mapping at Cuprite, Nevada,” in Proc. Summaries 3rd Annu. JPL Airborne Geosci. Workshop, 1992, pp. 47–49.
