21st International Conference on Pattern Recognition (ICPR 2012) November 11-15, 2012. Tsukuba, Japan
Blind Image Deblurring Based on Sparse Prior of Dictionary Pair Haisen Li† , Yanning Zhang†∗, Haichao Zhang† , Yu Zhu† and Jinqiu Sun‡ † Shaanxi Key Laboratory of Speech and Image Information Processing School of Computer Science and Technology, Northwestern Polytechnical University ‡ Institude of Precision Guidance and Control School of Astronautics, Northwestern Polytechnical University
Abstract
Traditional methods to remove blur are always done by employing deconvoltion method. However, the deconvolution is an ill-posed problem, and many priors and regularities are introduced to weaken the influence of artifacts. Recently, sparse prior has been used in image deblurring, which can get satisfactory performance via constraining the coefficients in some transform domains [3, 2]. Dilip et al. proposed a non-blind deblurring method assuming that the image’s gradient follows a hyper-Laplacian distribution [3]. Cai et al. proposed a blind motion deblurring method using the sparsity of the blur kernel and the clear image under certain overcomplete frame systems (such as curvelet system) to estimate the blur kernel and image alternatively [2]. After the over-complete dictionary was introduced in image denoising by Aharon and Elad [1], the sparse prior under over-completed dictionary has also been used in image deblurring [8, 9, 5], Zhang et al. proposed a blind image deblurring method based on image patch’s sparsity [8, 9], and Jia et al. proposed a new non-blind image deblurring method based on joint model of natural images by combining sparse representation of image patches and sparse gradient priors [5].
Blind image deblurring, aiming at obtaining the sharp image from blurred one, is a widely existing problem in image processing. Traditional image deblurring methods always use the deconvolution method to remove the blur kernel’s effect, however, deconvolution is so sensitive to noise that inevitable artifacts always exist in the deblurring results, even though regularity terms are introduced as constraints. In this paper, we propose a novel blind image deblurring method based on the sparse prior of dictionary pair, estimating the sparse coefficient, sharp image and blur kernel alternately. The proposed method could avoid the deconvolution problem which is an ill-posed problem, and obtain the result with fewer artifacts. Compared with the state-of-the-art method, experimental results demonstrate that the proposed method could obtain better performance.
1. Introduction
However, the deblur result from deconvolution method always generates some artifacts even though the regularity is introduced. Yang et al. proposed a super-resolution method based on dictionary pair, which assumes that the high resolution and low resolution patches have the same coefficients under the highresolution and the low resolution dictionary respectively [7]. It inspired us to solve the deblurring problem using the sharp and blur dictionary pair, instead of the traditional deconvolution methods, to restore the image. This could not only avoid the ill-posed deconvolution problem, but also bring more high-frequency information from the dictionary pair. Based on this idea, Lou et al. proposed a non-blind image deblurring method named direct sparse deblurring using two dictionaries to deblur the image [6].
Image deblurring aims at restoring a high-quality image from the blurred version which may be taken by atmospheric turbulence, defocusing or the relative motion between the scene and the camera during exposure time. The blurring effect is usually modeled as a linear degradation system as follows[2] Y =K ⊗X +η
(1)
where Y is the blurred image, X is the desired sharp image, η is the additive noise usually assumed as following Gaussian distribution, ⊗ is the convolution operator, and K is the blur kernel that is assumed to be linear shift-invariant. ∗ denotes
the corresponding author
978-4-9906441-1-6 ©2012 IAPR
3054
In this paper, we propose a blind image deblurring method by combining the dictionary pair and the sparse gradient prior, and estimating the blur kernel, sharp image and sparse coefficients respectively through minimizing the energy function. The rest of the paper is organized as follows: In section 2 and 3, we describe the formulation of the image deblurring method and how to optimize it. Experimental results are shown in the Section 4, and we conclude this paper in Section 5.
Algorithm 1: Blind image deblurring using dictionary pair Input: blurred image Y , sharp dictionary D, Iteration number T and kernel size Initialization: K For: t = 1, 2, . . . , T Sparse representation: update the α ˆ i via minimizing the equation (6) ˆ via Latent image estimation: update the X minimizing the equation (8) ˆ via Blur kernel estimation: update the K minimizing the equation (10) End ˆ blur kernel K ˆ Output: estimated latent image X,
2. Problem formulation Let D ∈ Rn×k be an over-complete dictionary with k atoms. An image X ∈ Rn could be sparsely represented by D as follows [1]: X = Dα
3. Optimization
(2)
where the representation coefficient α ∈ Rk is sparse. Because the convolution operator is a linear operator, the blurred image could represent as follow ignoring the noise term:
As a non-convex problem, the proposed model could be generally solved by alternately optimization method designed to handle multivariate optimization. The method is described in Algorithm 1.
Y = K ⊗ X = K ⊗ (Dα) = (K ⊗ D)α = Db α (3)
3.1. Sparse representation sub-problem
Therefore, we could assume the blur image and sharp image have the same coefficient under corresponding dictionary, and restore the sharp image via sparse reconstruction using the blur image sparse coefficients on sharp dictionary. From the dictionary pair consisting of sharp dictionary and blur dictionary, the deblurring method could bring more details than the ones employing deconvolution method. Based on the dictionary pair, our method obtains the latent image by optimizing the equation (4) : ˆ K ˆ = arg min {αˆi }, X,
X
{αi },X,K i=1
ˆ and estimated image Given the estimating kernel K ˆ X, the problem is then transformed to: X ||Ri Y − Db αi ||22 + λ||αi ||1 {ˆ αi } = argmin {αi }
i
(6) The blur dictionary could be obtained via Eq. (5). It is obvious that we could decompose the Eq. (6) for each patch separately, and solve each problem as follows: α ˆ i = argmin{||Ri Y − Db αi ||22 + λ||αi ||1 }
[kRi Y −Db αi k22 +λkαi k1 ]
(7)
αi
+τ kG ⊗ Xkω + βkKk22
(4)
3.2. X sub-problem: latent image estimation
Db = K ⊗ D
(5)
In this sub-problem, we reconstruct the laten image from the coefficient of blurry image based on the dictionary pair. When the current estimation α ˆ i is known, minimization of model (4) reduces to the following problem: X ˆ = argmin{ X kRi X − Dα ˆ i ||22 +τ ||G⊗X||ω } (8)
where Where Y ∈ RN is the blurry image, the Ri ∈ Rn×N denotes a matrix that extracts the i-th patch from the image, and λ, τ , β are the regularity weights. Equation (4) has three terms, the first term is the blurred image sparse representation under the blur dictionary Db , αi is the sparse coefficient of the i-th image patch; the second term is the sparse prior of the gradient image assumed to follow the hyper-Laplacian distribution [3], while G is the gradient extraction filters and ω ∈ [0.5 0.8] ; the third term is a L2 norm regularization to the blur kernel estimation.
X
i
The solution to (8) can be obtained quickly according to [3]. In practice, we set the G as first-order gradiT ent filters G1 = [1, −1], G2 = [1, −1] and set ω = 0.5 the same as configurations in [3].
3055
Table 2. SSIM of the deblurring image SSIM File Blurtype Dilip’s[4] Proposed motion 0.7896 0.8939 Barbara gaussian 0.7455 0.8066 camshake 0.7727 0.8758 motion 0.7398 0.8116 Boat gaussian 0.7187 0.7506 camshake 0.7386 0.8499 motion 0.7054 0.8289 Cameraman gaussian 0.6631 0.8265 camshake 0.7140 0.8722 motion 0.7085 0.8437 House gaussian 0.6946 0.8232 camshake 0.6047 0.8560
Table 1. PSNR of the deblurring image PSNR File Blurtype Dilip’s[4] Proposed motion 27.6603 30.0180 Barbara gaussian 27.1537 27.2266 camshake 26.8942 29.2629 motion 25.6146 26.7938 Boat gaussian 25.6239 25.6440 camshake 25.0594 28.2141 motion 21.4108 22.8291 Cameraman gaussian 21.0861 22.9643 camshake 21.2851 24.1115 motion 28.7757 30.7956 House gaussian 28.6392 30.1041 camshake 24.6928 31.4624
The experimental results of deblurred image in terms of PSNR and SSIM are presented in Table 1 and Table 2 respectively. From the two tables, it is shown the proposed method has higher PSNR and SSIM than Dilip’s method [4]. Experimental results in Fig. 1 shows that the proposed method could get more accurate estimation of the blur kernel. From the deblurred image, there exists noise amplification in Dilip’s method, however, the proposed method could get more acceptable result without noise amplification. The reason is that the deconvolution method is an ill-posed problem and is sensitive to noise, and the proposed method, using the dictionary pair to avoid the ill-posed problem, could obtain the result with fewer artifacts.
3.3. K sub-problem: blur kernel estimation In this sub-problem, we fix all other variables except K, and estimate the blur kernel K. The minimization of model (4) reduces to the following problem: ˆ = argmin ||K ⊗ X ˆ − Y ||2 + β||K||2 K 2 2
(9)
K
This is a least square problem with Tikhonov regularization, which leads to a closed form solution for K : ˆ = F −1 ( K
ˆ ◦ F (Y ) F (X) ) ˆ ◦ F (X) ˆ + βI F (X)
(10)
where F (·) and F −1 (·) are 2D FFT operator and 2D IFFT operator respectively, F (·) is the conjugate of F (·), and ◦ denotes element-wise multiplication.
5. Conclusion In this paper, an effective blind image deblurring method has been proposed, which is based on the assumption that the sparse coefficients of the blurry image under blur dictionary are the same as the sparse coefficients of the sharp image under the sharp dictionary. Moreover, the proposed method combines the dictionary pair and the sparse gradient prior. By utilizing the dictionary pair, ill-posed inverse problem and artifacts are avoided, and a gradient sparse prior sharpens the results further. Experiment shows that the proposed method has better performance than the state-of-the-art method. Acknowledgement: This work is supported by the National Natural Science Foundation of China (No.60903126), China Postdoctoral Special Science Foundation (No.201003685, No.20090451397).
4. Experimental Result The proposed algorithm is implemented and corresponding experiments to verify its effectiveness are carried out in MATLAB. First, blurred images for experiment are synthesized with three different kinds of blur kernel, including motion blur kernel(direction 45 degree and length 5 pixel), Gaussian blur kernel(standard deviation 2 pixels) and camera-shake blur kernel(shown in Figure 1), and then additive Gaussian noise following standard deviation of 0.01 are added to the blurred image. In all our experiments, we set λ = 0.01, τ = 2000, β = 0.01, T = 5 empirically. The sharp dictionary D is learned from the sharp images using KSVD [1]. We compared our result with the state-of-the-art algorithm [4] which is a blind image deblurring method based on deconvolution method. The final results are evaluated in terms of PSNR(peak signal to noise ratio) and SSIM(structural similarity index).
References [1] M. Aharon, M. Elad, and A. Bruckstein. K-svd: An algorithm for designing overcomplete dictionaries for sparse
3056
(a) sharp image
[2]
[3]
[4]
[5]
(b) blurry image with noise (c) result from [4] Figure 1. Deblur result.
representation. IEEE Transactions on Signal Processing, 54(11):4311–4322, 2006. J. Cai, H. Ji, C. Liu, and Z. Shen. Blind motion deblurring from a single image using sparse approximation. In IEEE Conference on Computer Vision and Pattern Recognition, 2009, pages 104–111, 2009. K. Dilip and F. Rob. Fast image deconvolution using hyper-laplacian priors. In Annual Conference on Neural Information Processing Systems, pages 1033–1041, 2009. K. Dilip and F. Tay, T.and Rob. Blind deconvolution using a normalized sparsity measure. In IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pages 233–240, 2011. C. Jia and L. E. Brain. Patch-based image deconvolution via joint modeling of sparse priors. In 18th IEEE Inter-
(d) Proposed method
national Conference on Image Processing (ICIP), 2011. [6] Y. Lou, A. Bertozzi, and S. Soatto. Direct sparse deblurring. Journal of Mathematical Imaging and Vision, 39(1):1–12, 2011. [7] J. Yang, J. Wright, H. Thomas, and Y. Ma. Image superresolution via sparse representation. IEEE Transactions on Image Processing, 19(11), 2010. [8] H. Zhang, J. Yang, Y. Zhang, and T. S. Huang. Close the loop: Joint blind image restoration and recognition with sparse representation prior. In IEEE 12th International Conference on Computer Vision, 2011. [9] H. Zhang, J. Yang, Y. Zhang, and T. S. Huang. Sparse representation based blind image deblurring. In IEEE International Conference on Multimedia and Expo (ICME), pages 1–6, 2011.
3057