New Regularization Scheme for Blind Color Image deconvolution
New Regularization Scheme for Blind
Color Image deconvolution Journal of Electronic Imaging Vol. 20, No. 1, 2011
Li Chen, Yu He and Kim-Hui Yap Presented by Il-Su Park
School of Electrical Engineering and Computer Science Kyungpook National Univ.
Abstract Proposed
method
– New regularization scheme • Addressing blind color image deconvolution
– Unified regularization scheme for image • Recovering edges of color images • Reducing color artifact
– Reinforcement regularization framework • Integrating a soft parametric learning term in addressing blind color image deconvolution
2 / 34
Introduction Blind
color image deconvolution(restoration)
– Captured image • Blurred due to factors – Out-of-focus optical system » Diffraction degradation – Relative motion between the camera and imaging scenes
– Objective • Estimating the original color image from the observed blurred color image, given limited or no prior knowledge of the blurring function
3 / 34
– Classical color image restoration • Hunt and Kubler – Minimum mean square error multichannel filter – Assumption
» Separable multichannel correlation matrix
• Kalman filter – Assumption » Knowing the blurs prior to image restoration
4 / 34
– Blind color image deconvolution • Stochastic method – Modeling images as random fields – Estimating the original color image as the most probable realization of a certain random process – Assumption » Gaussian-distributed image and noise
• Deterministic method – Finding an estimate of the original color image by minimizing the norm of a certain residuum – Employ regularization theory – Do not take the characteristics of the blurring function into consideration 5 / 34
– Proposed method • Using the color image properties • Unified regularization scheme for image – Recovering edges of color images
– Reducing color artifacts
• New regularization framework – Integrating a soft parametric learning term in addressing blind color image deconvolution
• Reinforcement-learning blur modeling scheme – Evaluating the relevance of manifold parametric blur structures
6 / 34
Problem formulation Blurred
color image
– Representing the observed blurred color image zj
i r , g ,b
hij fi n j ,
j r, g, b
(1)
where denotes two-dimensional convolution operation, z j , fi , and n j are the j th blurred color channel, ith original color channel, and j th channel noise, and hii and hij i j are intrachannel and interchannel PSFs or blurs.
– Ignoring the interchannel degradation • Much smaller than the coefficients of intrachanel degradation
7 / 34
– Expressing the image degradation model in the vector-matrix form z Hf n Fh n
(2)
zr fr hrr nr where z z g , f fg , h h gg , and n n g are vectors representing z b fb hbb nb the discrete, concatenated and lexicographically-ordered z j , fi , hii , and ni , respectively.
H rr H 0 0
0 H gg 0
0 0 , H rr
Fr F 0 0
0 Fg 0
0 0 Fr
(3)
where H ii and Fi are the corresponding matrices constructed from hii and fi.
8 / 34
– Blind deconvolution problem • Ill-posed with respect to unknown variables f and h • Finding the solution of Eq. (2) • Cost function
1 ˆ ˆ Q J f , hˆ z Hf 2 2 2 1 ˆ ˆ Q z Fh 2 2 2
fˆ S hˆ fˆ S hˆ
(4)
where the first term in Eq. (4) represents the least-square data fidelity of the estimated image fˆ and PSF hˆ , and Q fˆ and S hˆ denote the regularization functions in the image- and blur-domains, respectively.
9 / 34
Regularization issues Image-domain
regularization function Q fˆ
– Unified regularization scheme • Use of the TV technique(L1 norm) – Preserving the edge information for the intrachannel
• Adoption of a vector-based regularization term – Imposing color smoothness on the pixels within a small neighborhood
Q fˆ Qep fˆ Qcr fˆ
(5)
where Qep fˆ impose the smoothness on each color image channel while preserving the edge information, Qcr fˆ plays the role of reducing the color artifacts, and and are regularization parameters that control the relative contribution of the constraint terms in the cost function. 10 / 34
– Qep fˆ
• Using the TV technique
1 Qep fˆ fˆi dx 2 i r , g ,b
(6)
where x x, y and f i denote gradient of each image channel.
– Qcr fˆ
• Suppressing the color artifact • For example – Outer product of the vectors for adjacent pixels 1 2 2 1 2 sin 2
2
f g x1 , y1 f b x2 , y2 f b x1 , y1 f g x2 , y2 f b x1 , y1 f r x2 , y2 f r x1 , y1 f b x2 , y2 f r x1 , y1 f g x2 , y2 f g x1 , y1 f r x2 , y2
2
2
2
2
2 2
where is the angle between the vectors 1 and 2 .
11 / 34
• Extending example to the problem for the whole image
• Adjacent pixels for each pixel in the channel fˆi
S pq fˆi
where S pq represents the translational integer-shift operator of an image by translational vector (p,q).
fˆ *S fˆ fˆ *S fˆ 2 fˆ *S fˆ fˆ *S fˆ pq b b pq g b pq r r pq b 1 1 1 g 2 2 p 1 q 1 fˆ *S fˆ fˆ *S fˆ 2 r pq g g pq r 2 1 1 2 1 ˆ ˆ ˆ ˆ fi *S pq fi 1 fi 1 *S pq fi 2 2 i r , g ,b p 1 q 1
2 2
(7)
where * is an element–by-element product operator and we define that r 1 g , g 1 b, b 1 r.
12 / 34
Blur-domain
regularization function S fˆ
– Channel blurs • Two hypotheses H 0 :The blurs in each channel are all the same H1 :The blurs in each channel are different
(8)
• First, propose a new reinforcement regularization scheme for blur estimation corresponding to different hypothesis
• Later, propose a decision scheme to make a decision between the two hypotheses
13 / 34
– Proposed regularization function • Consisting of a fundamental regularization part and a reinforcement soft parametric blur learning term
S hˆ U k hˆ Rk hˆ
•
(9)
where k 0, 1 correspond to the two hypotheses H 0 and H1 , and and are regularization parameters that control the relative contribution among the constraint terms in the cost function. Fundamental regularization part U hˆ k
– U 0 hˆ
L O O 1 U 0 hˆ E0hˆ , where E0 O L O 2 2 O O L
2
(10)
where L is conventional 2-D Laplacian operator, and E0 imposes the smoothing constraint on the PSFs in each blur channel. 14 / 34
– U1 hˆ
» Performing high pass filtering on both sides of Eq. (1) ziH a x, y zi x, y hii f iH
(11)
where ziH and f iH are high-frequency information of the i th blurred channel image and i th original channel image, respectively.
» Developing spectral-based regularization operator ziH x, y hˆ jj x, y z jH x, y hˆii x, y O i, j r , g , b
(12)
» Transforming the regularization term into the minimization problem ziH x, y hˆ jj x, y Minimize ˆ x , y i j z jH x, y hii x, y i , j r , g ,b
2
(13)
15 / 34
» Expressing the regularization term in the vector-matrix form z gH 1 U1 hˆ E1hˆ , where E1 z bH 2 2 O
2
-z rH O z bH
O -z rH -z gH
(14)
where the matrix z iH i r , g , b is the convolution matrix formed by the ziH x, y .
16 / 34
• Soft parametric blur-learning term Rk hˆ – Integrating the parametric information of blurring function into the algorithm – Finding the best-fit parametric blur model with respect to each image channel hˆ rP 1 ˆ ˆ 2 ˆ ˆ Rk h h h P , where h P hˆ gP 2 2 ˆ hbP
(15)
where hˆ iP are the vectors formed by the reinforcement-learning blur models with respect to each blur channel.
17 / 34
Reinforcement-learning blur modeling for color image Schematic
procedure of the reinforcementlearning blur modeling
Fig. 1. Reinforcement-learning blur modeling.
Predefined
parametric model set
1 2 2 Out-of-focus blurs : P r if x y r 1 2 r Gaussian blurs : P2 exp x 2 y 2 / 2 2 Linear blurs : P3 a, b b a x 2 y 2
where a, b, r , and are model parameters of blurs.
(16)
18 / 34
Fuzzy
membership estimation
– Procedure • Find the best-fit parametric arguments i for the kth blur type in the ith blurred channel 2 i arg min hˆii pk , i r , g , b
2
(17)
• Compute the fuzzy membership of hˆii belonging to the kth blur type 2 ik hii exp hˆii pk i , i r , g , b
2
(18)
• Determine the best-fit blur model m and compute the corresponding m 1 m arg max ik , m im , i r , g , b 3 i i
(19)
• Output pm i 19 / 34
Weighted
mean filtering
– Obtaining the reinforcement-learning blur hˆiP based on m and pm i
– Derivation of the reinforcement-learning blurs hˆ iP 0hˆ ii m pm i ˆ i r , g ,b ˆ ˆ , when we decide the hypothesis H 0 (20) h rP h gP hbP 3 when we decide the hypothesis H1 hˆ iP 0hˆ ii m pm i ,
20 / 34
Optimization procedure Cost
function to be minimized from Eq. (4)
1 ˆ ˆ fˆ T T fˆ J fˆr , fˆg , fˆb , hˆ z Hf i 2 2 2
2
fˆ *S fˆ fˆ *S fˆ 2 E hˆ 2 hˆ hˆ i pq i 1 i 1 pq i 2 2 k 2 2 p 2 i r , g ,b p 1 q 1 1
Procedure
1
(21)
of AM
– A. Initialization • Initialize fˆi 0 x, y zi x, y and hˆii0 x, y to delta functions
21 / 34
– B. At the mth iteration • Minimize the image domain cost function to estimate the image
2 1 ˆ m 1fˆ fˆ T fˆ z H r r r r r 2 r 2 2 m fˆr arg min 1 1 fr DbS pq fˆr S pq Dbfˆr 2 p 1 q 1
(22) 2 2 S pq D g fˆr D g S pq fˆr 2 2
where Di is given as Di diag f i 1,1 , f i 1, 2 ,
, fi M f , N f
.
– Solving for fˆrm, using the linear equation
T H ˆ m 1 H ˆ m 1 T r r r ˆm T 1 1 DbS pq S pq Db DbS pq S pq Db fr Hˆ m1 T p 1 q 1 S pq D g D g S pq S pq D g D g S pq
T
zr
(23)
– Impose the display constraint on the r color image channel 0 fˆr x, y 255
(24)
22 / 34
• Minimize the blur-domain cost function to estimate the blur 2 2 2 1 hˆ m arg min z Fˆ mhˆ E k hˆ hˆ hˆ p 2 2 2 2 2 2
(25)
– Solving for hˆ using the linear equation m
Fˆ m
T
Fˆ m ETk Ek I hˆ m Fˆ m
T
y hˆ mp
(26)
– Impose the blur constraint on i=r,g,b blur channels
hˆ x, y 1; ii
hii x, y 0
(27)
23 / 34
– C. Stop if convergence is reached; otherwise, go to (B) SM
1 hˆ i hˆ i 1 3 i r , g ,b
2 2
Fig. 2. Schematic diagram of the proposed algorithm.
(28)
24 / 34
Experimental results Two
different sets of data
– Simulated and real-life image Use
of the well-known performance measures
– Evaluating the performance
NMSE hˆ 100
hˆ h
PSNR fˆ 10 log10
h
2
2
(29)
2552 1 fˆ f M f Nf
2 2
where a, b, r , and are model parameters of blurs.
25 / 34
Blind
color deconvolution on simulated images
– Woman image • Intrachannel PSFs – Gaussian blurs with standard deviation of 2.0, 2.5, 3.0 and with support of 3 3, 5 5, 7 7
• Additive noise of 30 dB
Fig. 3. Blind color deconvolution of woman image. (a) Original image, (b) noisy blurred image (different Gaussian blurs with standard deviation of 2.0, 2.5, 3.0), (c) restored image using the DR 26 / 34 method (Ref. 6), (d) restored image using the proposed method.
• PSNR and NMSE measures – The lowest NMSE and the highest PSNR values Table 1 Performance (women) of blind color image deconvolution (DR method and the methods by different combinations of the regularization terms).
27 / 34
• Estimated blurs and the ground truth
Fig. 4. Ground truth and estimated PSFs for woman image 28 / 34
– Tilehouse image • PSFs – Three linear motion blurs by five pixels, with an angle of 15 deg in a counterclockwise direction
Fig. 5. Blind color deconvolution of tilehouse image for motion blur. (a) Original image, (b) noisy blurred image (Motion blur an angle of 15 deg in a counterclockwise direction), (c) restored image using the DR method (Ref. 6), (d) restored image using the proposed method. 29 / 34
• Estimated blurs and the ground truth
Fig. 6. Ground truth and estimated PSFs for tilehouse image. 30 / 34
• PSNR and NMSE measures Table 2 Performance (tilehouse) of blind color image deconvolution (DR method and the methods by different combinations of the regularization terms).
31 / 34
– Tilehouse image • PSFs – Not belonging to the predefined classes – 5x5 nonstandard exponential blur
hii x, y exp x 2 y 2 , where 0.4
Fig. 7. Blind color deconvolution of tilehouse image blurred by nonstandard blur. (a) Original image, (b) noisy nonstandard blurred image, (c) restored image using the DR method (Ref. 6), (d) restored image using the proposed method. 32 / 34
Blind
color deconvolution on real-time images
– Bookcase image • Using a Cannon IXUS V3 camera under the wide-angle setting
Fig. 8. Blind color deconvolution of real-life image. (a) Noisy blurred image using cannon camera under wide-angle setting, (b) restored image using the DR method (Ref. 6), (c) restored image using the proposed method. 33 / 34
Conclusion Proposed
method
– New reinforcement regularization framework • Integrate a soft parametric learning term
– Reinforcement blur-modeling scheme • Evaluate the relevance of manifold parametric blur structures
– Unified regularization scheme for image • Recover edges of color mages and reduce color artifacts
34 / 34
Appendix
A: Substituted quadratic regularization
scheme – Alleviating the difficulty of the nonlinear partial differential equations(PDEs) – Reformulating fi Qep
1 fˆ i x 2 i r , g ,b x , y
2
, where
2
1 f i
fˆi x 1, y fˆi x, y i y
1 ˆT x f T i T y i fˆ 2 1 fˆ T T i fˆ 2
f i
2 ˆf x, y 1 fˆ x, y i i
(30)
Trk r O O k k Tg g O , k x, y . where T i O k O O T b b
r x x, y
f r x 1, y f r x, y frT Trx r fr 2
r x a, b, c
(31)
d , e
f r x 1, y f r x, y Vfr 2
2 2
f rT V T Vf r
x, y
1 1 0 1 where v 0 0
x, y
0
f x 1, y f x, y
r x
r
r
0 1 1
0
2
(32)
f rT V T WVf r
where W is given as W diag a, b, c
Trx r VT WV
d , e
(33)
Appendix
B: Proof and motivation of Eq. (12) ziH x, y hˆ jj x, y
(34)
fi x, y hˆii x, y a x, y hˆ jj x, y
ziH x, y hˆ jj x, y fi x, y hˆii x, y a x, y hˆ jj x, y fi x, y a x, y hˆii x, y hˆ jj x, y
(35)
f j x, y a x, y hˆii x, y hˆ jj x, y f j x, y hˆii x, y a x, y hˆ jj x, y z jH x, y hˆii x, y
x , y i j
ziH x, y hˆ jj x, y z jH x, y hˆ x, y
2
i , j r , g ,b
E1hˆ
2 2
where E1 is the regularization operator defined by Eq. (14).
(36)
Color Image deconvolution Journal of Electronic Imaging Vol. 20, No. 1, 2011
Li Chen, Yu He and Kim-Hui Yap Presented by Il-Su Park
School of Electrical Engineering and Computer Science Kyungpook National Univ.
Abstract Proposed
method
– New regularization scheme • Addressing blind color image deconvolution
– Unified regularization scheme for image • Recovering edges of color images • Reducing color artifact
– Reinforcement regularization framework • Integrating a soft parametric learning term in addressing blind color image deconvolution
2 / 34
Introduction Blind
color image deconvolution(restoration)
– Captured image • Blurred due to factors – Out-of-focus optical system » Diffraction degradation – Relative motion between the camera and imaging scenes
– Objective • Estimating the original color image from the observed blurred color image, given limited or no prior knowledge of the blurring function
3 / 34
– Classical color image restoration • Hunt and Kubler – Minimum mean square error multichannel filter – Assumption
» Separable multichannel correlation matrix
• Kalman filter – Assumption » Knowing the blurs prior to image restoration
4 / 34
– Blind color image deconvolution • Stochastic method – Modeling images as random fields – Estimating the original color image as the most probable realization of a certain random process – Assumption » Gaussian-distributed image and noise
• Deterministic method – Finding an estimate of the original color image by minimizing the norm of a certain residuum – Employ regularization theory – Do not take the characteristics of the blurring function into consideration 5 / 34
– Proposed method • Using the color image properties • Unified regularization scheme for image – Recovering edges of color images
– Reducing color artifacts
• New regularization framework – Integrating a soft parametric learning term in addressing blind color image deconvolution
• Reinforcement-learning blur modeling scheme – Evaluating the relevance of manifold parametric blur structures
6 / 34
Problem formulation Blurred
color image
– Representing the observed blurred color image zj
i r , g ,b
hij fi n j ,
j r, g, b
(1)
where denotes two-dimensional convolution operation, z j , fi , and n j are the j th blurred color channel, ith original color channel, and j th channel noise, and hii and hij i j are intrachannel and interchannel PSFs or blurs.
– Ignoring the interchannel degradation • Much smaller than the coefficients of intrachanel degradation
7 / 34
– Expressing the image degradation model in the vector-matrix form z Hf n Fh n
(2)
zr fr hrr nr where z z g , f fg , h h gg , and n n g are vectors representing z b fb hbb nb the discrete, concatenated and lexicographically-ordered z j , fi , hii , and ni , respectively.
H rr H 0 0
0 H gg 0
0 0 , H rr
Fr F 0 0
0 Fg 0
0 0 Fr
(3)
where H ii and Fi are the corresponding matrices constructed from hii and fi.
8 / 34
– Blind deconvolution problem • Ill-posed with respect to unknown variables f and h • Finding the solution of Eq. (2) • Cost function
1 ˆ ˆ Q J f , hˆ z Hf 2 2 2 1 ˆ ˆ Q z Fh 2 2 2
fˆ S hˆ fˆ S hˆ
(4)
where the first term in Eq. (4) represents the least-square data fidelity of the estimated image fˆ and PSF hˆ , and Q fˆ and S hˆ denote the regularization functions in the image- and blur-domains, respectively.
9 / 34
Regularization issues Image-domain
regularization function Q fˆ
– Unified regularization scheme • Use of the TV technique(L1 norm) – Preserving the edge information for the intrachannel
• Adoption of a vector-based regularization term – Imposing color smoothness on the pixels within a small neighborhood
Q fˆ Qep fˆ Qcr fˆ
(5)
where Qep fˆ impose the smoothness on each color image channel while preserving the edge information, Qcr fˆ plays the role of reducing the color artifacts, and and are regularization parameters that control the relative contribution of the constraint terms in the cost function. 10 / 34
– Qep fˆ
• Using the TV technique
1 Qep fˆ fˆi dx 2 i r , g ,b
(6)
where x x, y and f i denote gradient of each image channel.
– Qcr fˆ
• Suppressing the color artifact • For example – Outer product of the vectors for adjacent pixels 1 2 2 1 2 sin 2
2
f g x1 , y1 f b x2 , y2 f b x1 , y1 f g x2 , y2 f b x1 , y1 f r x2 , y2 f r x1 , y1 f b x2 , y2 f r x1 , y1 f g x2 , y2 f g x1 , y1 f r x2 , y2
2
2
2
2
2 2
where is the angle between the vectors 1 and 2 .
11 / 34
• Extending example to the problem for the whole image
• Adjacent pixels for each pixel in the channel fˆi
S pq fˆi
where S pq represents the translational integer-shift operator of an image by translational vector (p,q).
fˆ *S fˆ fˆ *S fˆ 2 fˆ *S fˆ fˆ *S fˆ pq b b pq g b pq r r pq b 1 1 1 g 2 2 p 1 q 1 fˆ *S fˆ fˆ *S fˆ 2 r pq g g pq r 2 1 1 2 1 ˆ ˆ ˆ ˆ fi *S pq fi 1 fi 1 *S pq fi 2 2 i r , g ,b p 1 q 1
2 2
(7)
where * is an element–by-element product operator and we define that r 1 g , g 1 b, b 1 r.
12 / 34
Blur-domain
regularization function S fˆ
– Channel blurs • Two hypotheses H 0 :The blurs in each channel are all the same H1 :The blurs in each channel are different
(8)
• First, propose a new reinforcement regularization scheme for blur estimation corresponding to different hypothesis
• Later, propose a decision scheme to make a decision between the two hypotheses
13 / 34
– Proposed regularization function • Consisting of a fundamental regularization part and a reinforcement soft parametric blur learning term
S hˆ U k hˆ Rk hˆ
•
(9)
where k 0, 1 correspond to the two hypotheses H 0 and H1 , and and are regularization parameters that control the relative contribution among the constraint terms in the cost function. Fundamental regularization part U hˆ k
– U 0 hˆ
L O O 1 U 0 hˆ E0hˆ , where E0 O L O 2 2 O O L
2
(10)
where L is conventional 2-D Laplacian operator, and E0 imposes the smoothing constraint on the PSFs in each blur channel. 14 / 34
– U1 hˆ
» Performing high pass filtering on both sides of Eq. (1) ziH a x, y zi x, y hii f iH
(11)
where ziH and f iH are high-frequency information of the i th blurred channel image and i th original channel image, respectively.
» Developing spectral-based regularization operator ziH x, y hˆ jj x, y z jH x, y hˆii x, y O i, j r , g , b
(12)
» Transforming the regularization term into the minimization problem ziH x, y hˆ jj x, y Minimize ˆ x , y i j z jH x, y hii x, y i , j r , g ,b
2
(13)
15 / 34
» Expressing the regularization term in the vector-matrix form z gH 1 U1 hˆ E1hˆ , where E1 z bH 2 2 O
2
-z rH O z bH
O -z rH -z gH
(14)
where the matrix z iH i r , g , b is the convolution matrix formed by the ziH x, y .
16 / 34
• Soft parametric blur-learning term Rk hˆ – Integrating the parametric information of blurring function into the algorithm – Finding the best-fit parametric blur model with respect to each image channel hˆ rP 1 ˆ ˆ 2 ˆ ˆ Rk h h h P , where h P hˆ gP 2 2 ˆ hbP
(15)
where hˆ iP are the vectors formed by the reinforcement-learning blur models with respect to each blur channel.
17 / 34
Reinforcement-learning blur modeling for color image Schematic
procedure of the reinforcementlearning blur modeling
Fig. 1. Reinforcement-learning blur modeling.
Predefined
parametric model set
1 2 2 Out-of-focus blurs : P r if x y r 1 2 r Gaussian blurs : P2 exp x 2 y 2 / 2 2 Linear blurs : P3 a, b b a x 2 y 2
where a, b, r , and are model parameters of blurs.
(16)
18 / 34
Fuzzy
membership estimation
– Procedure • Find the best-fit parametric arguments i for the kth blur type in the ith blurred channel 2 i arg min hˆii pk , i r , g , b
2
(17)
• Compute the fuzzy membership of hˆii belonging to the kth blur type 2 ik hii exp hˆii pk i , i r , g , b
2
(18)
• Determine the best-fit blur model m and compute the corresponding m 1 m arg max ik , m im , i r , g , b 3 i i
(19)
• Output pm i 19 / 34
Weighted
mean filtering
– Obtaining the reinforcement-learning blur hˆiP based on m and pm i
– Derivation of the reinforcement-learning blurs hˆ iP 0hˆ ii m pm i ˆ i r , g ,b ˆ ˆ , when we decide the hypothesis H 0 (20) h rP h gP hbP 3 when we decide the hypothesis H1 hˆ iP 0hˆ ii m pm i ,
20 / 34
Optimization procedure Cost
function to be minimized from Eq. (4)
1 ˆ ˆ fˆ T T fˆ J fˆr , fˆg , fˆb , hˆ z Hf i 2 2 2
2
fˆ *S fˆ fˆ *S fˆ 2 E hˆ 2 hˆ hˆ i pq i 1 i 1 pq i 2 2 k 2 2 p 2 i r , g ,b p 1 q 1 1
Procedure
1
(21)
of AM
– A. Initialization • Initialize fˆi 0 x, y zi x, y and hˆii0 x, y to delta functions
21 / 34
– B. At the mth iteration • Minimize the image domain cost function to estimate the image
2 1 ˆ m 1fˆ fˆ T fˆ z H r r r r r 2 r 2 2 m fˆr arg min 1 1 fr DbS pq fˆr S pq Dbfˆr 2 p 1 q 1
(22) 2 2 S pq D g fˆr D g S pq fˆr 2 2
where Di is given as Di diag f i 1,1 , f i 1, 2 ,
, fi M f , N f
.
– Solving for fˆrm, using the linear equation
T H ˆ m 1 H ˆ m 1 T r r r ˆm T 1 1 DbS pq S pq Db DbS pq S pq Db fr Hˆ m1 T p 1 q 1 S pq D g D g S pq S pq D g D g S pq
T
zr
(23)
– Impose the display constraint on the r color image channel 0 fˆr x, y 255
(24)
22 / 34
• Minimize the blur-domain cost function to estimate the blur 2 2 2 1 hˆ m arg min z Fˆ mhˆ E k hˆ hˆ hˆ p 2 2 2 2 2 2
(25)
– Solving for hˆ using the linear equation m
Fˆ m
T
Fˆ m ETk Ek I hˆ m Fˆ m
T
y hˆ mp
(26)
– Impose the blur constraint on i=r,g,b blur channels
hˆ x, y 1; ii
hii x, y 0
(27)
23 / 34
– C. Stop if convergence is reached; otherwise, go to (B) SM
1 hˆ i hˆ i 1 3 i r , g ,b
2 2
Fig. 2. Schematic diagram of the proposed algorithm.
(28)
24 / 34
Experimental results Two
different sets of data
– Simulated and real-life image Use
of the well-known performance measures
– Evaluating the performance
NMSE hˆ 100
hˆ h
PSNR fˆ 10 log10
h
2
2
(29)
2552 1 fˆ f M f Nf
2 2
where a, b, r , and are model parameters of blurs.
25 / 34
Blind
color deconvolution on simulated images
– Woman image • Intrachannel PSFs – Gaussian blurs with standard deviation of 2.0, 2.5, 3.0 and with support of 3 3, 5 5, 7 7
• Additive noise of 30 dB
Fig. 3. Blind color deconvolution of woman image. (a) Original image, (b) noisy blurred image (different Gaussian blurs with standard deviation of 2.0, 2.5, 3.0), (c) restored image using the DR 26 / 34 method (Ref. 6), (d) restored image using the proposed method.
• PSNR and NMSE measures – The lowest NMSE and the highest PSNR values Table 1 Performance (women) of blind color image deconvolution (DR method and the methods by different combinations of the regularization terms).
27 / 34
• Estimated blurs and the ground truth
Fig. 4. Ground truth and estimated PSFs for woman image 28 / 34
– Tilehouse image • PSFs – Three linear motion blurs by five pixels, with an angle of 15 deg in a counterclockwise direction
Fig. 5. Blind color deconvolution of tilehouse image for motion blur. (a) Original image, (b) noisy blurred image (Motion blur an angle of 15 deg in a counterclockwise direction), (c) restored image using the DR method (Ref. 6), (d) restored image using the proposed method. 29 / 34
• Estimated blurs and the ground truth
Fig. 6. Ground truth and estimated PSFs for tilehouse image. 30 / 34
• PSNR and NMSE measures Table 2 Performance (tilehouse) of blind color image deconvolution (DR method and the methods by different combinations of the regularization terms).
31 / 34
– Tilehouse image • PSFs – Not belonging to the predefined classes – 5x5 nonstandard exponential blur
hii x, y exp x 2 y 2 , where 0.4
Fig. 7. Blind color deconvolution of tilehouse image blurred by nonstandard blur. (a) Original image, (b) noisy nonstandard blurred image, (c) restored image using the DR method (Ref. 6), (d) restored image using the proposed method. 32 / 34
Blind
color deconvolution on real-time images
– Bookcase image • Using a Cannon IXUS V3 camera under the wide-angle setting
Fig. 8. Blind color deconvolution of real-life image. (a) Noisy blurred image using cannon camera under wide-angle setting, (b) restored image using the DR method (Ref. 6), (c) restored image using the proposed method. 33 / 34
Conclusion Proposed
method
– New reinforcement regularization framework • Integrate a soft parametric learning term
– Reinforcement blur-modeling scheme • Evaluate the relevance of manifold parametric blur structures
– Unified regularization scheme for image • Recover edges of color mages and reduce color artifacts
34 / 34
Appendix
A: Substituted quadratic regularization
scheme – Alleviating the difficulty of the nonlinear partial differential equations(PDEs) – Reformulating fi Qep
1 fˆ i x 2 i r , g ,b x , y
2
, where
2
1 f i
fˆi x 1, y fˆi x, y i y
1 ˆT x f T i T y i fˆ 2 1 fˆ T T i fˆ 2
f i
2 ˆf x, y 1 fˆ x, y i i
(30)
Trk r O O k k Tg g O , k x, y . where T i O k O O T b b
r x x, y
f r x 1, y f r x, y frT Trx r fr 2
r x a, b, c
(31)
d , e
f r x 1, y f r x, y Vfr 2
2 2
f rT V T Vf r
x, y
1 1 0 1 where v 0 0
x, y
0
f x 1, y f x, y
r x
r
r
0 1 1
0
2
(32)
f rT V T WVf r
where W is given as W diag a, b, c
Trx r VT WV
d , e
(33)
Appendix
B: Proof and motivation of Eq. (12) ziH x, y hˆ jj x, y
(34)
fi x, y hˆii x, y a x, y hˆ jj x, y
ziH x, y hˆ jj x, y fi x, y hˆii x, y a x, y hˆ jj x, y fi x, y a x, y hˆii x, y hˆ jj x, y
(35)
f j x, y a x, y hˆii x, y hˆ jj x, y f j x, y hˆii x, y a x, y hˆ jj x, y z jH x, y hˆii x, y
x , y i j
ziH x, y hˆ jj x, y z jH x, y hˆ x, y
2
i , j r , g ,b
E1hˆ
2 2
where E1 is the regularization operator defined by Eq. (14).
(36)
