Haze Removal on Superpixel Domain
2013 10th International Conference on Ubiquitous Robots and Ambient Intelligence (URAI) October 31-November 2, 2013 / Ramada Plaza Jeju Hotel, Jeju, Korea
Haze Removal on Superpixel Domain Sang-Woo Noh1, Byungtae Ahn2 and In So Kweon3* 1,2,3,4
Department of Electrical Engineering, KAIST, Daejeon 305-701, Korea 1 2 3 (E-mail: [email protected], btahn@ rcv.kaist.ac.kr, [email protected]) Abstract –Haze and fog cause poor visibility for vision sensor, and make computer vision problems challenging. In this paper, we propose a new de-hazing method for single image on super-pixel domain. One of widely used approaches for de-hazing is to introduce dark channel prior. However, in the previous researches, dark channel prior map is estimated with the fixed size and rectangle patch, and its size affects the performance of de-hazing algorithm. By replacing the conventional fixed size patch to the super-pixel, transmission map can be estimated with more specific details, which allow obtaining non-saturated haze free results. We demonstrate our preliminary results by comparisons between our method and one of the state-of-the-art algorithm with varying patch size on transmission map and restored images. Keywords – Dark channel prior, Super-pixel, Haze removal
Fig. 1. [Left] The initial transmission map with 15x15 patch. [Right] The initial transmission map with super-pixel. where ܫሺݔሻ is the observed haze image, ܬሺݔሻ is scene irradiance(the clear haze-free image), ܣis the overall atmosphere light, and ݐሺݔሻ is the transmission of the light reflected by the object. The goal of haze removal is to recover ܬሺݔሻ, A and ݐሺݔሻ from ܫሺݔሻ. ܬሺݔሻݐሺݔሻ is the direct attenuation.
1. Introduction In various outdoor computer vision systems such as surveillance camera systems, intelligent vehicles and remote sensing, foggy weather condition is a challenging problem, because many vision algorithms, such as feature detection and photometric analysis, are affected by the biased and low-contrast scene radiance resulted from haze artifacts. In the computer vision community, there have been numerous efforts to eliminate the haze in an image. Tan [1] removed the haze effects by the contrast maximization technique with a single image. Fattal [2] estimated the scene radiance and the transmission image to compensate haze regions. Recently, He et al. [3] proposed a novel approach for haze removal using dark channel prior, which is based on the statistics of outdoor haze-free images. They used soft matting method [4] to refine the transmission map. To use dark channel prior and matting scheme produced impressive results, but it is computational intensive. For low complexity, several methods (e.g. guided filter [5]) have been proposed to accelerate the refinement step in the de-hazing algorithm. In this paper, we propose new dark channel estimation method using super-pixel segmentation to obtain the refined transmission map including fine details of scene structure.
2.2 The proposed method and transmission map estimation Estimating the atmosphere light and transmission map is the most important process in haze removal. We estimate atmosphere light based on dark channel prior. He et al. [3] proposed (2), which describes dark channel prior. ܬௗ ሺݔሻ ൌ ൬ ൫ܬ ሺݕሻ൯൰ ሺʹሻ אሼǡǡሽ ௬אπሺ௫ሻ
where ܬ is a color channel of ;ܬπሺݔሻ is a patch around x. Conventional approach generally used 11x11 or 15x15 patch size to estimate dark channel and the rough transmission map. As mentioned in [6], the patch size (x) in (2) affects haze removal results. If the patch size is too small, the recovered scene radiance could be oversaturated. If the patch size is too large, halo effects on depth edges become stronger, and haze removal effect is degraded. The proposed method estimates dark channel and the rough transmission map on super-pixel, each πሺݔሻ means super-pixel. So we re-define πሺݔሻ as a segment of super-pixel including x. We use quick shift algorithm [7] to segment super-pixel. Quick shift uses a mode-seeking segmentation. It moves each point in the feature space to the nearest neighbor that increases the Parzen density estimated by
2. Haze removal
ܲ െ ܲ ǡሺ͵ሻ ǣௗ൫௫ೕ ǡ௫ ൯ழఛ ܦ
ݕ ሺͳሻ ൌ
2.1 Haze modeling The atmosphere attenuation model can be defined as the following equation: ܫሺݔሻ ൌ ܬሺݔሻݐሺݔሻ ܣ൫ͳ െ ݐሺݔሻ൯ǡሺͳሻ
978-1-4799-1197-4/13/$31.00 ©2013 IEEE
ٻ
where ܦ is the length of the shifts ݕ ሺͲሻ ݕٻ ሺͳሻ ܲٻڇis the Parzen density.
597
Fig. 3. [Top] the recovered images, [Bottom] the transmission map, [Right] using super-pixel, [Midlle] using 15x15 patch, [Left] using 3x3 patch.
Fig. 2. [Left] the recovered images, [Right] the transmission map, [Top] using super-pixel, [Midlle] using 1515 patch, [Bottom] using 3X3 patch. We pick the top 0.1% brightest pixels in ܬௗ . These pixels are most haze-opaque. The air light ܣis estimated from average of these pixels. The transmission map can be estimated by: ݐሺݔሻ ൌ ͳ െ ቆ ቆ אሼǡǡሽ
௬אπሺ௫ሻ
ܬ ሺݕሻ ቇቇǤሺͶሻ ܣ
4. Conclusion In this paper, we exploited quick shift algorithm to estimate the transmission map. In the estimated transmission map, no oversaturated region exists with less halo effects. The proposed method is especially good performance for nature scene. Our limitation is some depth errors for artificial structure.
2.3 Refine the transmission map
5. Acknowledgement
Through (4), we obtain the rough transmission map. To refine transmission map, He et al. [3] used soft matting method. In recent researches, there are several approaches using guided filter [5] to refine the transmission map. We also use guided filter. Since atmosphere light ܣand the refined transmission map are known, we can get the haze-free image ܬfrom (1).
This research was supported by the MOTIE (The Ministry of Trade, Industry and Energy), Korea, under the Human Resources Development Program for Convergence Robot Specialists support program supervised by the NIPA(National IT Industry Promotion Agency) (H1502-13-1001).
3. Experimental Results
References
We compare our method with the conventional approach [3] which use the fixed size patch to estimate dark channel prior map. Instead of the fixed size patch, we use super-pixel obtained by over-segmentation algorithm [7].ٻIn experiments, we set our parameters, such as ratio to 0.5, kernel size to 2 and maximum distance to 10, we compare the results obtained by [3] with the patch size 3x3 and 15x15.ٻIn figure 1 and 2, halos effects on depth edges appear in the transmission map results obtained by [3], when using the 15x15 patch. There is the oversaturated regions in the haze-free results obtained by [3] with 3x3 patch in Fig 3.ٻIn contrast to the conventional approach, our method produces more satisfying results in the transmission maps with preserving fine details, and there are no over-saturated regions in the estimated haze-free images by our method.
[1] R. Tan, “Visibility in bad weather from a single image,” CVPR, 2008. [2] R. Fattal, “Single Image Dehazing,” ACM SIGGRAP H, 2008. [3] K. He, J. Sun, and X. Tang, “Single image haze removal using dark channel prior,” CVPR, 2009. [4] A. Levin, D. Lischinski, and Y. Weiss, “A closed form solution to natural image matting,” CVPR, 2006. [5] K. He, J. Sun and X. Tang, “Guided image filtering,” ECCV, 2010 [6] K. He, J. Sun, and X. Tang,"Single image haze removal using dark channel prior." Pattern Analysis and Machine Intelligence, 2011. [7] A. Vedaldi and S. Soatto.,“Quick shift and kernel methods for mode seeking,” ECCV, 2008.
598
Haze Removal on Superpixel Domain Sang-Woo Noh1, Byungtae Ahn2 and In So Kweon3* 1,2,3,4
Department of Electrical Engineering, KAIST, Daejeon 305-701, Korea 1 2 3 (E-mail: [email protected], btahn@ rcv.kaist.ac.kr, [email protected]) Abstract –Haze and fog cause poor visibility for vision sensor, and make computer vision problems challenging. In this paper, we propose a new de-hazing method for single image on super-pixel domain. One of widely used approaches for de-hazing is to introduce dark channel prior. However, in the previous researches, dark channel prior map is estimated with the fixed size and rectangle patch, and its size affects the performance of de-hazing algorithm. By replacing the conventional fixed size patch to the super-pixel, transmission map can be estimated with more specific details, which allow obtaining non-saturated haze free results. We demonstrate our preliminary results by comparisons between our method and one of the state-of-the-art algorithm with varying patch size on transmission map and restored images. Keywords – Dark channel prior, Super-pixel, Haze removal
Fig. 1. [Left] The initial transmission map with 15x15 patch. [Right] The initial transmission map with super-pixel. where ܫሺݔሻ is the observed haze image, ܬሺݔሻ is scene irradiance(the clear haze-free image), ܣis the overall atmosphere light, and ݐሺݔሻ is the transmission of the light reflected by the object. The goal of haze removal is to recover ܬሺݔሻ, A and ݐሺݔሻ from ܫሺݔሻ. ܬሺݔሻݐሺݔሻ is the direct attenuation.
1. Introduction In various outdoor computer vision systems such as surveillance camera systems, intelligent vehicles and remote sensing, foggy weather condition is a challenging problem, because many vision algorithms, such as feature detection and photometric analysis, are affected by the biased and low-contrast scene radiance resulted from haze artifacts. In the computer vision community, there have been numerous efforts to eliminate the haze in an image. Tan [1] removed the haze effects by the contrast maximization technique with a single image. Fattal [2] estimated the scene radiance and the transmission image to compensate haze regions. Recently, He et al. [3] proposed a novel approach for haze removal using dark channel prior, which is based on the statistics of outdoor haze-free images. They used soft matting method [4] to refine the transmission map. To use dark channel prior and matting scheme produced impressive results, but it is computational intensive. For low complexity, several methods (e.g. guided filter [5]) have been proposed to accelerate the refinement step in the de-hazing algorithm. In this paper, we propose new dark channel estimation method using super-pixel segmentation to obtain the refined transmission map including fine details of scene structure.
2.2 The proposed method and transmission map estimation Estimating the atmosphere light and transmission map is the most important process in haze removal. We estimate atmosphere light based on dark channel prior. He et al. [3] proposed (2), which describes dark channel prior. ܬௗ ሺݔሻ ൌ ൬ ൫ܬ ሺݕሻ൯൰ ሺʹሻ אሼǡǡሽ ௬אπሺ௫ሻ
where ܬ is a color channel of ;ܬπሺݔሻ is a patch around x. Conventional approach generally used 11x11 or 15x15 patch size to estimate dark channel and the rough transmission map. As mentioned in [6], the patch size (x) in (2) affects haze removal results. If the patch size is too small, the recovered scene radiance could be oversaturated. If the patch size is too large, halo effects on depth edges become stronger, and haze removal effect is degraded. The proposed method estimates dark channel and the rough transmission map on super-pixel, each πሺݔሻ means super-pixel. So we re-define πሺݔሻ as a segment of super-pixel including x. We use quick shift algorithm [7] to segment super-pixel. Quick shift uses a mode-seeking segmentation. It moves each point in the feature space to the nearest neighbor that increases the Parzen density estimated by
2. Haze removal
ܲ െ ܲ ǡሺ͵ሻ ǣௗ൫௫ೕ ǡ௫ ൯ழఛ ܦ
ݕ ሺͳሻ ൌ
2.1 Haze modeling The atmosphere attenuation model can be defined as the following equation: ܫሺݔሻ ൌ ܬሺݔሻݐሺݔሻ ܣ൫ͳ െ ݐሺݔሻ൯ǡሺͳሻ
978-1-4799-1197-4/13/$31.00 ©2013 IEEE
ٻ
where ܦ is the length of the shifts ݕ ሺͲሻ ݕٻ ሺͳሻ ܲٻڇis the Parzen density.
597
Fig. 3. [Top] the recovered images, [Bottom] the transmission map, [Right] using super-pixel, [Midlle] using 15x15 patch, [Left] using 3x3 patch.
Fig. 2. [Left] the recovered images, [Right] the transmission map, [Top] using super-pixel, [Midlle] using 1515 patch, [Bottom] using 3X3 patch. We pick the top 0.1% brightest pixels in ܬௗ . These pixels are most haze-opaque. The air light ܣis estimated from average of these pixels. The transmission map can be estimated by: ݐሺݔሻ ൌ ͳ െ ቆ ቆ אሼǡǡሽ
௬אπሺ௫ሻ
ܬ ሺݕሻ ቇቇǤሺͶሻ ܣ
4. Conclusion In this paper, we exploited quick shift algorithm to estimate the transmission map. In the estimated transmission map, no oversaturated region exists with less halo effects. The proposed method is especially good performance for nature scene. Our limitation is some depth errors for artificial structure.
2.3 Refine the transmission map
5. Acknowledgement
Through (4), we obtain the rough transmission map. To refine transmission map, He et al. [3] used soft matting method. In recent researches, there are several approaches using guided filter [5] to refine the transmission map. We also use guided filter. Since atmosphere light ܣand the refined transmission map are known, we can get the haze-free image ܬfrom (1).
This research was supported by the MOTIE (The Ministry of Trade, Industry and Energy), Korea, under the Human Resources Development Program for Convergence Robot Specialists support program supervised by the NIPA(National IT Industry Promotion Agency) (H1502-13-1001).
3. Experimental Results
References
We compare our method with the conventional approach [3] which use the fixed size patch to estimate dark channel prior map. Instead of the fixed size patch, we use super-pixel obtained by over-segmentation algorithm [7].ٻIn experiments, we set our parameters, such as ratio to 0.5, kernel size to 2 and maximum distance to 10, we compare the results obtained by [3] with the patch size 3x3 and 15x15.ٻIn figure 1 and 2, halos effects on depth edges appear in the transmission map results obtained by [3], when using the 15x15 patch. There is the oversaturated regions in the haze-free results obtained by [3] with 3x3 patch in Fig 3.ٻIn contrast to the conventional approach, our method produces more satisfying results in the transmission maps with preserving fine details, and there are no over-saturated regions in the estimated haze-free images by our method.
[1] R. Tan, “Visibility in bad weather from a single image,” CVPR, 2008. [2] R. Fattal, “Single Image Dehazing,” ACM SIGGRAP H, 2008. [3] K. He, J. Sun, and X. Tang, “Single image haze removal using dark channel prior,” CVPR, 2009. [4] A. Levin, D. Lischinski, and Y. Weiss, “A closed form solution to natural image matting,” CVPR, 2006. [5] K. He, J. Sun and X. Tang, “Guided image filtering,” ECCV, 2010 [6] K. He, J. Sun, and X. Tang,"Single image haze removal using dark channel prior." Pattern Analysis and Machine Intelligence, 2011. [7] A. Vedaldi and S. Soatto.,“Quick shift and kernel methods for mode seeking,” ECCV, 2008.
598
