HYPERSPECTRAL IMAGING PHENOMENOLOGY FOR THE ... - RIT CIS

Report 2 Downloads 203 Views
HYPERSPECTRAL IMAGING PHENOMENOLOGY FOR THE DETECTION AND TRACKING OF PEDESTRIANS John Kerekes†

Jared Herweg*,† *Air Force Institute of Technology Dayton, OH 45433, USA



Rochester Institute of Technology Rochester, NY 14623, USA

Michael Eismann‡ ‡

Air Force Research Laboratory Dayton, OH 45433, USA

ABSTRACT The popularity of hyperspectral imaging in remote sensing continues to to be adapted in novel ways to overcome challenging imaging problems. This paper reports on some of the latest research efforts exploring the phenomenology of using hyperspectral imaging as an aid in detecting and tracking human pedestrians. An assessment of the likelihood of distinguishing between pedestrians given observable materials and based on signal-to-noise level is presented. Initial results indicate favorable separability can be achieved with signal-tonoise ratios as low as 13 for certain materials. Additionally, an overview of a real-world urban hyperspectral imaging data collection effort is presented.

complex target whose spectral signature is made up of their subregion characteristics, where subregion refers to items such as hair, skin, and clothing. Typically, it can be observed that individual recognition among pedestrians is largely due to these subregions. Note that in many cultures a pedestrian’s shirt and trousers are made of different materials or are distinguishable by different colors. Thus the pedestrian can be thought of as having four contiguous regions, which could be used to differentiate between pedestrians. One step in this research effort looked at the spectral separability between pedestrians based on their subregions. There may be further distinguishing aspects associated with these subregions such as texture, but they will not be considered here.

Index Terms— Hyperspectral imaging, Pedestrian tracking, Target detection, Spectral separability

Despite the potential for recognition of individual pedestrians based on the characteristics of the subregions, illumination variations across the pedestrian poses a unique challenge [6]. Even when a pedestrian is in direct illumination, portions of them may be self shadowed. This is particularly evident when a pedestrian is moving. Such shadowing across subregions complicates pixel association since adjacent pixels of the same material may exhibit different signal levels. Some approaches to illumination invariant classification of remote sensed imagery utilize spatial information associated with the spectral information [7, 8].

1. INTRODUCTION Hyperspectral imaging (HSI) continues to mature and progress as a technology for improving target detection and discrimination in remote sensing applications. Traditionally, HSI has been used in remote sensing for land use and land cover classification research [1]. HSI has also been used for smaller targets such as vehicles where spectral features serve as additional discriminants between targets for improved tracking [2, 3]. More recently, research has shown how hyperspectral imaging can be used independently for skin detection as well as clothing classification [4, 5]. Our work builds upon these studies related to the constituent materials found on human pedestrians. This paper presents an overview of some of the challenges experienced and the results of the phenomenology of HSI when applied to detecting pedestrians in a complex urban environment. 2. PEDESTRIAN DETECTION CHALLENGES One of the primary challenges to a tracking algorithm is distinguishing between moving objects within a scene; in this case between pedestrians. This becomes even more of a challenge when the pedestrians come within close proximity of each other. Intuitively a pedestrian can be thought of as a

Another challenge is associating the overall spectral signature of a pedestrian when they enter total shadow. There has been some work looking at the challenge of detecting the same object from full illumination to full shadow when the target spectral signature was based on the first detection of the target in an image set [9]. Note that atmospheric compensation methods could be utilized to overcome some of these challenges by converting from the sensor reaching radiance to reflectance [1]. However, each technique has its benefits and limitations. Typically the approach chosen for real-world imaging is contingent on the type of information available and if in-scene compensation is possible. The focus of work presented here is to study the various levels of signal-to noise (SNR) required to produce separability between pedestrians based on the spectral profile of their individual subregions.

3. ESTIMATING PEDESTRIAN SUBREGION SEPARABILITY To analyze the separability between subregions, spectral reflectance measurements of the hair, skin, shirt, and trousers or shorts (if worn) for 28 unique pedestrians were collected. For the skin measurements, a measurement of each person’s right cheek, right forearm, and their right calf (if exposed) were measured. These measurements were collected using an Analytical Spectral Devices, Inc., Full Range Field Spectrometer [10]. The spectrometer was a three detector instrument which could collect spectra from 350 - 2500 nm and had a full-width half-max spectral resolution of 3 - 12 nm, depending on the detector. A contact probe with an integrated illumination source was used to collect the relative reflectance samples [11]. The material reflectance was measured at 1 nm intervals and subsequently resampled from 2151 spectral bands down to 216 spectral bands. The resampled reflectance samples were commensurate with the hyperspectral imaging data collected with these same pedestrians (see Section 4). Examples of the data collected for the hair and trousers subregions are shown in Figures 1 and 2. Note that there are visible spectral differences for the hair samples below 1500 nm, shown in Figure 1. For the trousers, the predominant material was cotton, but there was still significant variations among the samples shown in Figure 2.

Fig. 1. Spectral profiles for the several spectral relative reflectance measurements of pedestrian’s hair. One of the things that we wanted to assess was the likelihood of separability at different levels of SNR. A distribution of noisy samples were generated for each signature such that x ˜ = ~x +

~x ∗ ℵ(0, 1) SN R

(1)

where ~x was the p-dimensional sample spectral vector for a pedestrian’s subregion, ℵ(0, 1) was a p-dimensional vector of

Fig. 2. Spectral profiles for the several spectral relative reflectance measurements of pedestrian’s trousers or shorts. random variables from the Normal Distribution, the sample vector ~x divided by the SNR was substituted for the standard deviation [1], and ∗ signifies an element by element multiply operation. By applying a flat SNR level across all bands, this assessment was independent of any particular sensor characteristic. The separability was assessed by computing the spectral distance from each non-noise modulated subregion spectral sample to all noisy samples within the same subregion. The adjusted spectral Euclidean distance metric was used such that [12] v u p u1 X de (~x, ~y ) = t (xi − yi )2 (2) p i=1 where ~x represents the p-dimensional spectral vector of a material measurement from the pedestrian of interest (POI) and ~y represents the spectral reflectance vector of one of the POI’s noisy samples or any of the nosiy or non-noise added samples of the same subregion from other pedestrians. To assess how likely the subregion samples from one person could be distinguished from the same subregion samples of all other pedestrians, two classes were defined. The first class represented the distribution of distances between a particular pedestrian’s subregion sample and the noise modulated versions of that sample. This was called the POI class. The second class represented the distribution of distances between the non-noise modulated POI subregion sample and all the remaining noisy samples of the other pedestrians. This constituted a background class. With this one-versus-all construct, the probability of error per SNR for the POI class, p(error|ωP OI ), was calculated according to the Bayes minimum error threshold [13]. The probability of error for the POI class was chosen instead of the total probability of error since in real-world imaging it was expected each subregion would

only be covered by very few pixels. As such, we were mainly concerned with at what SNR levels would the distributions begin to overlap and lead to missed detections. The Bayes rule for minimum cost could be used to overcome this limitation, but the costs in our case were assumed to be equal with no particular application presently considered. An example of the two class distributions from the hair subregion data at an SNR level of 8 is shown in Figure 3. The distributions were estimated empirically using kernel based density estimation with a Gaussian kernel [13] and 100,000 samples generated using equation 2. It can be shown that the probability density function p(de |ωP OI ) follows a χ2 distribution with p degrees of freedom [14]. However, the densities of both classes were empirically estimated using the kernel based density method since p(de |ωbkgnd ) did not follow a standard distribution. Fig. 4. Plot of the probability of error for the POI class versus SNR for each of the subregions. Note that fairly low probabilities of error were achieved with relatively low SNR, but in typical imagery there are very few pixels on each subregion. 4. REAL-WORLD IMAGING OF PEDESTRIANS USING HYPERSPECTRAL IMAGER

Fig. 3. Showing an example of the two class distance distributions for a pedestrian’s face skin reflectance sample. The probability of the POI class distribution, denoted by the solid line, is seen on the left while the probability density function of the background class distribution, denoted by the dashed line, is seen on the right. The SNR level for these two distributions was 8.

The probability of error was computed for each subregion of each pedestrian. Figure 4 shows the probability of error versus SNR for the four subregions: hair, face, shirt, and trousers. Each of the four curves are the average probability of error for all 28 pedestrians of the respective subregion. From the results in Figure 4, SNR’s as low as 14 achieved good separability for the subregions hair, shirt, and trousers, with hair being the most separable. The face samples required a much higher SNR to achieve the same separability as the other regions.

In addition to the individual reflectance samples that were collected for the pedestrians, a hyperspectral imager was used to collect imagery data of the pedestrians while posing in an urban scene. This imagery was collected as part of the Hyperspectral Measurements of Natural Signatures for Pedestrian (HYMNS-P) Experiment [15]. The imager was placed on a roof overlooking an urban scene. The imager collected 220 spectral bands from 450 - 2450 nm with a 1 mrad resolution per pixel [16]. The pedestrians whose spectral reflectance samples shown in Figures 1 and 2 were placed around the scene in natural poses at known locations during the image capture. An example true color image of the scene is shown in Figure 5. During the experiment, several frames of imagery were captured with the pedestrians moved between each frame. In addition to the spectral reflectance measurements from the pedestrians, other aspects of ground truth were collected as well, which included spectral reflectance measurements of in-scene true background materials (not to be confused with the background class discussed in Section 3). Calibration panels were also placed in the scene both propped up, facing the imager, as well as laying flat on the ground with direct illumination. Portions of this unique data set will become publicly accessible for future studies looking into the pedestrian detection and tracking problem. 5. CONCLUSION In this paper we have discussed some of the challenges associated with the detecting and tracking pedestrians in remote

feature spacing for hyperspectral data,” in Hyperspectral Image and Signal Processing: Evolution in Remote Sensing (WHISPERS), 2010 2nd Workshop on, June 2010, Proc. WHISPERS, pp. 1–4.

Fig. 5. True color radiance image of the Hyperspectral Measurements of Natural Signatures for Pedestrians (HYMNSP) scene with pedestrians present. Several variations of this static scene were used for this research effort. sensed imagery. Analysis of data collected so far suggest discrimination between subregions, when set as a two class problem, can be achieved with moderately reasonable SNR levels. Using the real-world data collected, additional analysis will be performed looking at the variability among the samples within the subregions and between subregions due to illumination and other geometric effects.

DISCLAIMER The views expressed in this article are those of the authors and do not reflect the official policy or position of the United States Air Force, Department of Defense, or the United States Government. 6. REFERENCES [1] John R. Schott, Remote Sensing, Oxford University Press, Oxford, NY, second edition, 2007. [2] J. Blackburn, M. Mendenhall, A. Rice, P. Shelnutt, N. Soliman, and J. Vasquez, “Feature aided tracking with hyperspectral imagery,” Aug. 2007, vol. 6699 of Proc. SPIE. [3] Andrew Rice, Juan Vasquez, Michael Mendenhall, and John Kerekes, “Feature-aided tracking via synthetic hyperspectral imagery,” in Hyperspectral Image and Signal Processing: Evolution in Remote Sensing, 2009. WHISPERS ’09. First Workshop on, 2009, pp. 1 –4.

[6] Jared A. Herweg, John P. Kerekes, Emmitt J. Ientilucci, and Michael T. Eismann, “Spectral variations in HSI signatures of thin fabrics for detecting and tracking of pedestrians,” 2011, Proc. SPIE 8040, 80400, pp. 1–11. [7] S. M. Adler-Golden, et. al., “Shadow-insensitive material detection/classification with atmospherically corrected hyperspectral imagery,” Bellingham, WA, 2001, vol. 4381 of Proc. SPIE, pp. 460–469. [8] Edward A. Ashton, Brian D. Wemett, Robert A. Leathers, and Trijntje V. Downes, “A novel method for illumination suppression in hyperspectral images,” Bellingham, WA, 2008, vol. 6966 of Proc. SPIE, pp. 1– 8. [9] Emmitt J. Ientilucci, “Legeraging lidar data to aid in hyperspectral image target detection in the radiance domain,” 2012, Proc. SPIE 8390, 839007, pp. 1–12. [10] Analytical Spectral Devices, Inc., “FieldSpec Pro User’s Guide,” www.asdi.com, 2002, Web, 15 Oct 2010. [11] Danielle Simmons, “Performance characterization of an innovative illumination source for the analytical spectral device spectroradiometer, FieldSpec Pro FR,” Rochester Institute of Technology Digital Imaging and Remote Sensing Laboratory Probe Development Status Report, 2006. [12] Peter Bajorski, Statistics for Imaging, Optics, and Photonics, Wiley, Hoboken, NJ, 2011. [13] Andrew Webb, Statistical Pattern Recognition, John Wiley & Sons, West Sussex, England, second edition, 2005. [14] Alberto Leon-Garcia, Probability and Random Processes for Electrical Engineering, Addison-Wesley, Reading, Mass., second edition, 1994.

[4] Abel S. Nunez, A Physical Model of Human Skin and Its Application for Search and Rescue, Ph. D. dissertation, Air Force Institute of Technology, Wright-Patterson Air Force Base, OH, 2010.

[15] Jared A. Herweg, John P. Kerekes, and Michael Eismann, “Hyperspectral imaging of natural signatures for pedestrians,” 2012, Proc. SPIE 8390, 83901C, pp. 1–10.

[5] Jeffrey D. Clark, Michael J. Mendenhall, and Gilbert L. Peterson, “Stochastic feature selection with distributed

[16] Christopher M. Jengo and Joe LaVeigne, “Sensor performance comparison of hyperspectir instruments 1 and 2,” 2004, vol. 3, pp. 1799–1805.