Template matching with noisy patches: a contrast-invariant GLR test
Template matching with noisy patches: a contrast-invariant GLR test Charles Deledalle1, Lo¨ıc Denis2 and Florence Tupin3 IMB, CNRS-Universit´e Bordeaux 1 — 2Laboratoire Hubert Curien, CNRS-Univ. Saint-Etienne — 3Institut Mines-T´el´ecom, T´el´ecom ParisTech, CNRS-LTCI
Contrast-invariant template matching with noisy patches a noise-free patch of N pixel values, a noisy patch realization of X ∼ p(·|θ) where the pdf/pmf p is known, a template taken from a dictionary D of noise-free patches (with N pixels),
Using the Generalized Likelihood Ratio (GLR): I Replaces the unknowns ρ and θ by their maximum likelihood estimates (MLE) under each hypothesis: supρ p(x|θ = Tρ(a), H0) p(x|θ = Tρˆ(a)) G(x, a) = = ˆ supt p(x|θ = t, H1) p(x|θ = t)
D as small as possible ⇒ represents classes of patches identical up to a radiometric transformation I I
x=
+
=
I I
|
{z θ
}
≡
a=
|
{z σn
, |
where ρˆ and tˆ are the MLE of the unknown ρ and θ, Satisfies the contrast invariance property by construction, Asymptotically (wrt the SNR) optimal with constant false alarm rate (CFAR). Invariant upon changes of variable [Kay and Gabriel, 2003], May fail in low SNR conditions, where the MLE is known to behave poorly.
{z
equivalence class
}
SC (x, a) = C(s(x), s(a)) I
Problem statement
I
Contrast invariance I Define a template matching criterion c : (x, a) 7→ c(x, a) > 0, I The larger c(x, a), the more relevant the match between x and a, I Consider invariance up to a family of transformation Tρ parametrized by ρ ∀X, a, ρ, c(X, Tρ(a)) = c(X, a) . Example (affine contrast change): Tρ(a) = Tα,β (a) = αa + β1, where 1k = 1 for all 1 ≤ k ≤ N .
Robustness to noise I Template matching formulation: ∃ρ
I
θ = Tρ(a) Hypotheses test (parameter test): H0 : ∃ρ H1 : ∀ρ
I
⇒
θ = Tρ(a) θ 6= Tρ(a)
0.8
0.8
0.8
0.6
0.4
where it is assumed that s(X) ∼ N (s(θ), σ 2I). Usually simpler to evaluate in closed-form, and then, leads to faster algorithms. Limited to the existence of a stabilization function s.
I
I
0.2 0.4 0.6 0.8 Probability of false alarm
1
0
0
0.2 0.4 0.6 0.8 Probability of false alarm
0.4
G SG C SC
0.2
1
0
0
0.2 0.4 0.6 0.8 Probability of false alarm
(b)
1
(c)
Dictionary of 196 templates of size N = 8 × 8 (extracted from the image “Barbara” with k-means), Several noisy realizations x are generated for several radiometric transformations θ = Tρ(a), GLR provides the best performance followed by Gaussian GLR after variance stabilization, Correlation acts poorly in all situations.
Proposition (GLR for Gaussian noise) Consider that X follows an uncorrelated Gaussian distribution: ! (xk − θk )2 1 , exp − p(xk |θk ) = √ 2 2σ 2πσ and consider the class of affine contrast transformations Tα,β (x) = αx + β1. In this case, we have kx − x¯1k2 2 . − log G(x, a) = 1 − C(x, a)2 2σ 2 (a)
I
Correlation does not take into account the noise while GLR does, ex.: − log C
>
,
while
− log LG
,
− log LG
,
(c)
(d)
− log C
(b)
Figure: (a) Noisy input image damaged by gamma noise (with L = 10, P SN R = 21.14). (b) Denoised image using the GLR after variance stabilization followed by a debiasing step following [Xie et al., 2002] (P SN R = 27.42). (c) Denoised image using the GLR adapted to gamma noise (P SN R = 27.53). (d) Image composed of the atoms of the dictionary.
,
Template-matching based denoising: I The dictionary D provides a generative model of the patches x of the noisy image, I Each patch of the image can then be estimated as: X X 1 ? ˆ θ(x) = G(x, a)a with Z = G(x, a) , Z a∈D
a∈D
where a? = Tρˆ(a) and ρˆ is the MLE of ρ used in the calculation of G(x, a).
How to define a suitable template matching criterion? I
In fact 0 ×
+ β “explains” better
than
.
Typical contrast-invariant template matching Proposition (GLR for Gamma noise) Consider that X follows a gamma distribution such that L−1 L L xk Lxk p(xk |θk ) = exp − L θk Γ(L)θk and consider the class of log-affine transformations Tα,β (x) = βxα where (.)α is the element-wise power function. In this case, we have − log G(x, a) = L
N X k=1
ˆ and βˆ can be obtained iteratively as where α P k 1 − rk,i log ak ˆ i+1 = α ˆi − P α 2 k rk,i(log ak )
log
ˆ αˆ βa k xk
!
Xx 1 k and βˆi+1 = α ˆi N a k k
ˆ α with rk,i = xk /(βˆiak i ), whatever the initialization.
Is correlation a robust template matching criterion wrt different noise statistics?
0
0.6
Figure: (a) ROC curve obtained under Gaussian noise, (b) ROC curve obtained under gamma noise and (c) ROC curve obtained under Poisson noise. In all experiments, the SNR is about −3dB.
I
Correlation vs GLR under Gaussian noise
Normalized correlation: I Most usual way to measure similarity up to an affine change of contrast: P ¯ ¯ (x − x )(a − a ) k k k C(x, a) = qP P 2 2 ¯ ¯ (x − x ) (a − a ) k k k k P P 1 1 where x¯ = N xk and a¯ = N ak . I such that +1 X 0 = 0.97 ⇒ decide “similar” θ = αa + β : C , = −1 +1 X 0 = 0.07 ⇒ decide “dissimilar” θ 6= αa + β : C , = −1
G SG C SC
0.2
(a)
I
“ x matches with a ” (null hypothesis), (alternative hypothesis).
0.4
G C
SG (x, a) = G(s(x), s(a))
and
0.6
Application to dictionary-based denoising
Optimal Neyman-Pearson criterion (maximizing PD for any given PF A) is the likelihood ratio test: p(x|θ = Tρ(a), H0) . L(x, a) = p(x|θ, H1) ⇒ cannot be evaluated since ρ and θ are unknown.
http://www.math.u-bordeaux1.fr/~cdeledal/
1
0.2
Using variance stabilization: I Transform patches such that the noise component be approximately Gaussian (with constant variance), I Example: homomorphic transform for multiplicative noise; Anscombe transform for Poisson noise. I Given an application s which stabilizes the variance for a specific noise distribution, stabilization-based criteria can be obtained on the output of s as:
Poisson
1
0
and
Gamma
1
}
How to match noisy patches x with templates a?
I
Gaussian
Probability of detection
Consider I θ I x I a ∈ D
Detection performance
Probability of detection
Introduction
Probability of detection
1
Similar result for Poisson noise.
“Multi-scale” shift-invariant dictionary: I D is composed of the set of all atoms extracted from a 128 × 128 image (a.k.a., an epitome) built following the transparent dead leaves model of [Galerne and Gousseau, 2012], I The dictionary is then shift invariant and denoising can be performed in the Fourier domain (see [Jost et al., 2006, Benoˆıt et al., 2011]) while representing information of different scales, I GLR for the gamma law or for the Gaussian law after stabilizing the variance are both satisfactory visually and in term of PSNR. References Benoˆıt, L., Mairal, J., Bach, F., and Ponce, J. (2011). Sparse image representation with epitomes. In Proceedings of CVPR, 2011. IEEE. Galerne, B. and Gousseau, Y. (2012). The transparent dead leaves model. Advances in Applied Probability, 44(1):1–20. Jost, P., Vandergheynst, P., Lesage, S., and Gribonval, R. (2006). Motif: an efficient algorithm for learning translation invariant dictionaries. In Proceedings of ICASSP 2006. IEEE. Kay, S. and Gabriel, J. (2003). An invariance property of the generalized likelihood ratio test. IEEE Signal Processing Letters, 10(12):352–355. Xie, H., Pierce, L., and Ulaby, F. (2002). SAR speckle reduction using wavelet denoising and Markov random field modeling. IEEE Transactions on Geoscience and Remote Sensing, 40(10):2196–2212.
[email protected]
Contrast-invariant template matching with noisy patches a noise-free patch of N pixel values, a noisy patch realization of X ∼ p(·|θ) where the pdf/pmf p is known, a template taken from a dictionary D of noise-free patches (with N pixels),
Using the Generalized Likelihood Ratio (GLR): I Replaces the unknowns ρ and θ by their maximum likelihood estimates (MLE) under each hypothesis: supρ p(x|θ = Tρ(a), H0) p(x|θ = Tρˆ(a)) G(x, a) = = ˆ supt p(x|θ = t, H1) p(x|θ = t)
D as small as possible ⇒ represents classes of patches identical up to a radiometric transformation I I
x=
+
=
I I
|
{z θ
}
≡
a=
|
{z σn
, |
where ρˆ and tˆ are the MLE of the unknown ρ and θ, Satisfies the contrast invariance property by construction, Asymptotically (wrt the SNR) optimal with constant false alarm rate (CFAR). Invariant upon changes of variable [Kay and Gabriel, 2003], May fail in low SNR conditions, where the MLE is known to behave poorly.
{z
equivalence class
}
SC (x, a) = C(s(x), s(a)) I
Problem statement
I
Contrast invariance I Define a template matching criterion c : (x, a) 7→ c(x, a) > 0, I The larger c(x, a), the more relevant the match between x and a, I Consider invariance up to a family of transformation Tρ parametrized by ρ ∀X, a, ρ, c(X, Tρ(a)) = c(X, a) . Example (affine contrast change): Tρ(a) = Tα,β (a) = αa + β1, where 1k = 1 for all 1 ≤ k ≤ N .
Robustness to noise I Template matching formulation: ∃ρ
I
θ = Tρ(a) Hypotheses test (parameter test): H0 : ∃ρ H1 : ∀ρ
I
⇒
θ = Tρ(a) θ 6= Tρ(a)
0.8
0.8
0.8
0.6
0.4
where it is assumed that s(X) ∼ N (s(θ), σ 2I). Usually simpler to evaluate in closed-form, and then, leads to faster algorithms. Limited to the existence of a stabilization function s.
I
I
0.2 0.4 0.6 0.8 Probability of false alarm
1
0
0
0.2 0.4 0.6 0.8 Probability of false alarm
0.4
G SG C SC
0.2
1
0
0
0.2 0.4 0.6 0.8 Probability of false alarm
(b)
1
(c)
Dictionary of 196 templates of size N = 8 × 8 (extracted from the image “Barbara” with k-means), Several noisy realizations x are generated for several radiometric transformations θ = Tρ(a), GLR provides the best performance followed by Gaussian GLR after variance stabilization, Correlation acts poorly in all situations.
Proposition (GLR for Gaussian noise) Consider that X follows an uncorrelated Gaussian distribution: ! (xk − θk )2 1 , exp − p(xk |θk ) = √ 2 2σ 2πσ and consider the class of affine contrast transformations Tα,β (x) = αx + β1. In this case, we have kx − x¯1k2 2 . − log G(x, a) = 1 − C(x, a)2 2σ 2 (a)
I
Correlation does not take into account the noise while GLR does, ex.: − log C
>
,
while
− log LG
,
− log LG
,
(c)
(d)
− log C
(b)
Figure: (a) Noisy input image damaged by gamma noise (with L = 10, P SN R = 21.14). (b) Denoised image using the GLR after variance stabilization followed by a debiasing step following [Xie et al., 2002] (P SN R = 27.42). (c) Denoised image using the GLR adapted to gamma noise (P SN R = 27.53). (d) Image composed of the atoms of the dictionary.
,
Template-matching based denoising: I The dictionary D provides a generative model of the patches x of the noisy image, I Each patch of the image can then be estimated as: X X 1 ? ˆ θ(x) = G(x, a)a with Z = G(x, a) , Z a∈D
a∈D
where a? = Tρˆ(a) and ρˆ is the MLE of ρ used in the calculation of G(x, a).
How to define a suitable template matching criterion? I
In fact 0 ×
+ β “explains” better
than
.
Typical contrast-invariant template matching Proposition (GLR for Gamma noise) Consider that X follows a gamma distribution such that L−1 L L xk Lxk p(xk |θk ) = exp − L θk Γ(L)θk and consider the class of log-affine transformations Tα,β (x) = βxα where (.)α is the element-wise power function. In this case, we have − log G(x, a) = L
N X k=1
ˆ and βˆ can be obtained iteratively as where α P k 1 − rk,i log ak ˆ i+1 = α ˆi − P α 2 k rk,i(log ak )
log
ˆ αˆ βa k xk
!
Xx 1 k and βˆi+1 = α ˆi N a k k
ˆ α with rk,i = xk /(βˆiak i ), whatever the initialization.
Is correlation a robust template matching criterion wrt different noise statistics?
0
0.6
Figure: (a) ROC curve obtained under Gaussian noise, (b) ROC curve obtained under gamma noise and (c) ROC curve obtained under Poisson noise. In all experiments, the SNR is about −3dB.
I
Correlation vs GLR under Gaussian noise
Normalized correlation: I Most usual way to measure similarity up to an affine change of contrast: P ¯ ¯ (x − x )(a − a ) k k k C(x, a) = qP P 2 2 ¯ ¯ (x − x ) (a − a ) k k k k P P 1 1 where x¯ = N xk and a¯ = N ak . I such that +1 X 0 = 0.97 ⇒ decide “similar” θ = αa + β : C , = −1 +1 X 0 = 0.07 ⇒ decide “dissimilar” θ 6= αa + β : C , = −1
G SG C SC
0.2
(a)
I
“ x matches with a ” (null hypothesis), (alternative hypothesis).
0.4
G C
SG (x, a) = G(s(x), s(a))
and
0.6
Application to dictionary-based denoising
Optimal Neyman-Pearson criterion (maximizing PD for any given PF A) is the likelihood ratio test: p(x|θ = Tρ(a), H0) . L(x, a) = p(x|θ, H1) ⇒ cannot be evaluated since ρ and θ are unknown.
http://www.math.u-bordeaux1.fr/~cdeledal/
1
0.2
Using variance stabilization: I Transform patches such that the noise component be approximately Gaussian (with constant variance), I Example: homomorphic transform for multiplicative noise; Anscombe transform for Poisson noise. I Given an application s which stabilizes the variance for a specific noise distribution, stabilization-based criteria can be obtained on the output of s as:
Poisson
1
0
and
Gamma
1
}
How to match noisy patches x with templates a?
I
Gaussian
Probability of detection
Consider I θ I x I a ∈ D
Detection performance
Probability of detection
Introduction
Probability of detection
1
Similar result for Poisson noise.
“Multi-scale” shift-invariant dictionary: I D is composed of the set of all atoms extracted from a 128 × 128 image (a.k.a., an epitome) built following the transparent dead leaves model of [Galerne and Gousseau, 2012], I The dictionary is then shift invariant and denoising can be performed in the Fourier domain (see [Jost et al., 2006, Benoˆıt et al., 2011]) while representing information of different scales, I GLR for the gamma law or for the Gaussian law after stabilizing the variance are both satisfactory visually and in term of PSNR. References Benoˆıt, L., Mairal, J., Bach, F., and Ponce, J. (2011). Sparse image representation with epitomes. In Proceedings of CVPR, 2011. IEEE. Galerne, B. and Gousseau, Y. (2012). The transparent dead leaves model. Advances in Applied Probability, 44(1):1–20. Jost, P., Vandergheynst, P., Lesage, S., and Gribonval, R. (2006). Motif: an efficient algorithm for learning translation invariant dictionaries. In Proceedings of ICASSP 2006. IEEE. Kay, S. and Gabriel, J. (2003). An invariance property of the generalized likelihood ratio test. IEEE Signal Processing Letters, 10(12):352–355. Xie, H., Pierce, L., and Ulaby, F. (2002). SAR speckle reduction using wavelet denoising and Markov random field modeling. IEEE Transactions on Geoscience and Remote Sensing, 40(10):2196–2212.
[email protected]
