Linear model - Semantic Scholar
Mean Shift-based Bayesian Image Reconstruction into Visual Subspace Torbjørn Vik1 , Fabrice Heitz1 and Pierre Charbonnier2
1 Laboratoire
des Sciences de l’Image, de l’Informatique et de la Télédétection UMR-7005 CNRS/Strasbourg I University, Illkirch, France 2 Laboratoire Régional des Ponts et Chaussées Strasbourg, France
ICIP 2003 – p. 1
Plan Appearance-based modeling Linear model, Probabilistic modeling Reconstruction algorithm Example Summary
ICIP 2003 – p. 2
Appearance-based models
Statistical analysis extracts characteristic features of an object class from raw training images Applications: detection, recognition
ICIP 2003 – p. 3
Appearance-based models
Statistical analysis extracts characteristic features of an object class from raw training images Applications: detection, recognition Training Training images
Estimated model
Recognition
Class
Image to recognize
ICIP 2003 – p. 3
Visual subspace
Image with d pixels ⇐⇒ vector Rd Training images ⇐⇒ cloud of points in Rd Probabilistic modeling Dimension reduction (PCA, ICA, FA, PP,...) Model: Deterministic part (dependent on a subspace variable) and noise
ICIP 2003 – p. 4
Visual subspace: illustration
Training images
3D observation space
ICIP 2003 – p. 5
Visual subspace: illustration
PCA
3D observation space
ICIP 2003 – p. 5
Visual subspace: illustration
y x
y = g(x) +
ICIP 2003 – p. 5
Visual subspace: illustration
y x
y = g(x) + Error Deterministic relation
ICIP 2003 – p. 5
Plan Appearance-based modeling Linear model, Probabilistic modeling Reconstruction algorithm Example Summary
ICIP 2003 – p. 6
Linear model Linear model (from factor analysis):
y = Wx + µ +
y:
observed image
W:
generation matrix
x:
subspace variable
µ:
mean image
:
pixel/observation noise
ICIP 2003 – p. 7
Linear model Linear model (from factor analysis): Training
y = Wx + µ +
y:
observed image
W:
generation matrix
x:
subspace variable
µ:
mean image
:
pixel/observation noise
ICIP 2003 – p. 7
Linear model Linear model (from factor analysis):
y = Wx + µ +
y:
observed image
W:
generation matrix
x:
subspace variable
µ:
mean image
:
pixel/observation noise
Random variables
p(x)
p()
ICIP 2003 – p. 7
Linear model Linear model (from factor analysis):
y = Wx + µ +
y:
observed image
W:
generation matrix
x:
subspace variable
µ:
mean image
:
pixel/observation noise
ICIP 2003 – p. 7
Linear model Linear model (from factor analysis):
y = Wx + µ +
y:
observed image
W:
generation matrix
x:
subspace variable
µ:
mean image
:
pixel/observation noise
?
Reconstruction problem
ICIP 2003 – p. 7
Linear model Linear model (from factor analysis):
y = Wx + µ +
y:
observed image
W:
generation matrix
x:
subspace variable
µ:
mean image
:
pixel/observation noise
?
Reconstruction problem ˆ = arg maxx p(x|y) MAP estimation: x ICIP 2003 – p. 7
Probabilistic modeling: standard
p(x) = constant: uniform p() = N (0, σ 2 ), Gaussian ˆ GM L =⇒ standard ML/LS estimate: x
ICIP 2003 – p. 8
Probabilistic modeling: standard
p(x) = constant: uniform p() = N (0, σ 2 ), Gaussian ˆ GM L =⇒ standard ML/LS estimate: x y
xGT Class distribution
ˆ GM L x ICIP 2003 – p. 8
Probabilistic modeling: robust noise
p(x) = constant: uniform Pd 1 p() ∝ exp(− 2 i=1 ρ(i ))
ˆ RM L =⇒ iterated reweighted LS (IRLS): x
ICIP 2003 – p. 9
Probabilistic modeling: robust noise
p(x) = constant: uniform Pd 1 p() ∝ exp(− 2 i=1 ρ(i ))
ˆ RM L =⇒ iterated reweighted LS (IRLS): x y
xGT ˆ RM L x p(x)
ˆ GM L x ICIP 2003 – p. 9
Probabilistic modeling: proposed model 1 N
PN
Γi (x), Γi (x) = N (xi , Σ): Non-Gaussian Pd 1 p() ∝ exp(− 2 i=1 ρ(i )) p(x) =
i=1
ˆ RM AP ? =⇒ how to find x
y
xGT p(x)
ˆ RM AP x
ˆ RM L x
ˆ GM L x ICIP 2003 – p. 10
Plan Appearance-based modeling Linear model, Probabilistic modeling Reconstruction algorithm Example Summary
ICIP 2003 – p. 11
Standard Mean Shift D. Comaniciu and P. Meer, 2002 Gradient ascent: ∇p(x) = p(x)Σ−1 ms(x) Mean Shift : ms(x) = Adaptive Step-size
"P
N i=1 Γi (x)xi PN i=1 Γi (x)
−x
#
ICIP 2003 – p. 12
Standard Mean Shift D. Comaniciu and P. Meer, 2002 Gradient ascent:
Mean Shift :
∇p(x) = p(x)Σ−1 ms(x) | {z } step size
ms(x) = Adaptive Step-size
"P
N i=1 Γi (x)xi PN i=1 Γi (x)
−x
#
ICIP 2003 – p. 12
Modified Mean Shift for visual subspace
p(x): Kernel density function p() = N (0, σ 2 ), Gaussian Gradient ascent: ∇p(x|y) = p(x|y)Σ−1 mms(x) mms(x) : Modified Mean Shift : # "P N i=1 ci Γi (x)µi −x mms(x) = PN i=1 ci Γi (x) ICIP 2003 – p. 13
Half-quadratic theory
Noise: p() ∝
exp(− 12
Pd
i=1
ρ(i ))
Rewrite by introducing an auxiliary variable b: p() = max p˜(, b) b
1 T p˜(, b) ∝ exp − ( B + Ξ(b)) 2
B = diag(b) = y − µ − Wx
ICIP 2003 – p. 14
Half-quadratic theory
Noise: p() ∝
exp(− 12
Pd
i=1
ρ(i ))
Rewrite by introducing an auxiliary variable b: p() = max p˜(, b) b
1 p˜(, b) ∝ exp − ( | T{z B} 2
b fixed⇒mms
+Ξ(b))
!
B = diag(b)
= y − µ − Wx
ICIP 2003 – p. 14
Half-quadratic theory
Noise: p() ∝
exp(− 12
Pd
i=1
ρ(i ))
Rewrite by introducing an auxiliary variable b: p() = max p˜(, b) b
1 p˜(, b) ∝ exp − 2
B = diag(b)
(T B + Ξ(b)) | {z }
x fixed⇒arg maxb analytic
= y − µ − Wx ICIP 2003 – p. 14
Complete algorithm
Hypotheses: p(x): Kernel density function Pd 1 p() ∝ exp(− 2 i=1 ρ(i ))
Algorithm sketch: repeat repeat x(k+1) ← x(k) + mms(x(k) ) until inner loop convergence ρ0 (j ) bj ← j until outer loop convergence
ICIP 2003 – p. 15
Plan Appearance-based modeling Linear model, Probabilistic modeling Reconstruction algorithm Example Summary
ICIP 2003 – p. 16
Example COIL
Var 3
xGT Var 2 Var 1
y
ICIP 2003 – p. 17
Example COIL
3D
x ˆ GM L
x ˆ RM L
xGT x ˆ RM AP
x ˆ RM L
x ˆ GM L
x ˆ RM AP
ICIP 2003 – p. 18
Example COIL
All images Euclidian distance in visual subspace from ground truth Average results:
µd ± σ d med ± M AD
GML 34.1 ± 6.8 35.8 ± 4.1
RML 15.7 ± 19.0 7.1 ± 3.1
RMAP 11.9 ± 14.8 4.4 ± 2.5
ICIP 2003 – p. 19
Summary
Contributions: =⇒ A general probabilistic model for visual appearance modeling was presented: robust noise and non-parametric subspace distribution =⇒ An algorithm for MAP reconstruction of an observed image, based on the Mean Shift and HQ-theory
ICIP 2003 – p. 20
Mean Shift: the term
ms(x) =
"P
N i=1 Γi (x)xi PN i=1 Γi (x)
x1
x
−x
#
xN
ICIP 2003 – p. 21
Linear model Decompose an image into a linear basis
+x1
=
y
+...
µ
w1
. . . + xd
+
w
ICIP 2003 – p. 22
Appearance-based models: application
Object/face recognition Separate training and recognition Training Training images
Estimated model
Recognition
Class
Image to recognize
ICIP 2003 – p. 23
Appearance-based models: recognition
Image reconstruction: given an observation y, find x This x is then classified in subspace Standard estimation: Least squares (LS) y
xGT Class distribution
ˆ GM L x ICIP 2003 – p. 24
Estimation of x Maximum likelihood (ML): ˆ = arg max p(y|x) x x
Maximum a postieriori (MAP): 1 ˆ = arg max p(x|y) = arg max x p(y|x)p(x) x x p(y)
ICIP 2003 – p. 25
Visual subspace: notion
g(x)
ICIP 2003 – p. 26
Visual subspace: notion
g(x)
ICIP 2003 – p. 26
Visual subspace: notion
y = g(x) + g(x)
ICIP 2003 – p. 26
Visual subspace: notion
y = g(x) + g(x)
Error Deterministic relation
ICIP 2003 – p. 26
References [CM02] D. Comaniciu and P. Meer. Mean Shift: A robust approach toward feature space analysis. IEEE Trans. Pattern Anal. Machine Intell., 24(5):603–619, May 2002.
ICIP 2003 – p. 27
1 Laboratoire
des Sciences de l’Image, de l’Informatique et de la Télédétection UMR-7005 CNRS/Strasbourg I University, Illkirch, France 2 Laboratoire Régional des Ponts et Chaussées Strasbourg, France
ICIP 2003 – p. 1
Plan Appearance-based modeling Linear model, Probabilistic modeling Reconstruction algorithm Example Summary
ICIP 2003 – p. 2
Appearance-based models
Statistical analysis extracts characteristic features of an object class from raw training images Applications: detection, recognition
ICIP 2003 – p. 3
Appearance-based models
Statistical analysis extracts characteristic features of an object class from raw training images Applications: detection, recognition Training Training images
Estimated model
Recognition
Class
Image to recognize
ICIP 2003 – p. 3
Visual subspace
Image with d pixels ⇐⇒ vector Rd Training images ⇐⇒ cloud of points in Rd Probabilistic modeling Dimension reduction (PCA, ICA, FA, PP,...) Model: Deterministic part (dependent on a subspace variable) and noise
ICIP 2003 – p. 4
Visual subspace: illustration
Training images
3D observation space
ICIP 2003 – p. 5
Visual subspace: illustration
PCA
3D observation space
ICIP 2003 – p. 5
Visual subspace: illustration
y x
y = g(x) +
ICIP 2003 – p. 5
Visual subspace: illustration
y x
y = g(x) + Error Deterministic relation
ICIP 2003 – p. 5
Plan Appearance-based modeling Linear model, Probabilistic modeling Reconstruction algorithm Example Summary
ICIP 2003 – p. 6
Linear model Linear model (from factor analysis):
y = Wx + µ +
y:
observed image
W:
generation matrix
x:
subspace variable
µ:
mean image
:
pixel/observation noise
ICIP 2003 – p. 7
Linear model Linear model (from factor analysis): Training
y = Wx + µ +
y:
observed image
W:
generation matrix
x:
subspace variable
µ:
mean image
:
pixel/observation noise
ICIP 2003 – p. 7
Linear model Linear model (from factor analysis):
y = Wx + µ +
y:
observed image
W:
generation matrix
x:
subspace variable
µ:
mean image
:
pixel/observation noise
Random variables
p(x)
p()
ICIP 2003 – p. 7
Linear model Linear model (from factor analysis):
y = Wx + µ +
y:
observed image
W:
generation matrix
x:
subspace variable
µ:
mean image
:
pixel/observation noise
ICIP 2003 – p. 7
Linear model Linear model (from factor analysis):
y = Wx + µ +
y:
observed image
W:
generation matrix
x:
subspace variable
µ:
mean image
:
pixel/observation noise
?
Reconstruction problem
ICIP 2003 – p. 7
Linear model Linear model (from factor analysis):
y = Wx + µ +
y:
observed image
W:
generation matrix
x:
subspace variable
µ:
mean image
:
pixel/observation noise
?
Reconstruction problem ˆ = arg maxx p(x|y) MAP estimation: x ICIP 2003 – p. 7
Probabilistic modeling: standard
p(x) = constant: uniform p() = N (0, σ 2 ), Gaussian ˆ GM L =⇒ standard ML/LS estimate: x
ICIP 2003 – p. 8
Probabilistic modeling: standard
p(x) = constant: uniform p() = N (0, σ 2 ), Gaussian ˆ GM L =⇒ standard ML/LS estimate: x y
xGT Class distribution
ˆ GM L x ICIP 2003 – p. 8
Probabilistic modeling: robust noise
p(x) = constant: uniform Pd 1 p() ∝ exp(− 2 i=1 ρ(i ))
ˆ RM L =⇒ iterated reweighted LS (IRLS): x
ICIP 2003 – p. 9
Probabilistic modeling: robust noise
p(x) = constant: uniform Pd 1 p() ∝ exp(− 2 i=1 ρ(i ))
ˆ RM L =⇒ iterated reweighted LS (IRLS): x y
xGT ˆ RM L x p(x)
ˆ GM L x ICIP 2003 – p. 9
Probabilistic modeling: proposed model 1 N
PN
Γi (x), Γi (x) = N (xi , Σ): Non-Gaussian Pd 1 p() ∝ exp(− 2 i=1 ρ(i )) p(x) =
i=1
ˆ RM AP ? =⇒ how to find x
y
xGT p(x)
ˆ RM AP x
ˆ RM L x
ˆ GM L x ICIP 2003 – p. 10
Plan Appearance-based modeling Linear model, Probabilistic modeling Reconstruction algorithm Example Summary
ICIP 2003 – p. 11
Standard Mean Shift D. Comaniciu and P. Meer, 2002 Gradient ascent: ∇p(x) = p(x)Σ−1 ms(x) Mean Shift : ms(x) = Adaptive Step-size
"P
N i=1 Γi (x)xi PN i=1 Γi (x)
−x
#
ICIP 2003 – p. 12
Standard Mean Shift D. Comaniciu and P. Meer, 2002 Gradient ascent:
Mean Shift :
∇p(x) = p(x)Σ−1 ms(x) | {z } step size
ms(x) = Adaptive Step-size
"P
N i=1 Γi (x)xi PN i=1 Γi (x)
−x
#
ICIP 2003 – p. 12
Modified Mean Shift for visual subspace
p(x): Kernel density function p() = N (0, σ 2 ), Gaussian Gradient ascent: ∇p(x|y) = p(x|y)Σ−1 mms(x) mms(x) : Modified Mean Shift : # "P N i=1 ci Γi (x)µi −x mms(x) = PN i=1 ci Γi (x) ICIP 2003 – p. 13
Half-quadratic theory
Noise: p() ∝
exp(− 12
Pd
i=1
ρ(i ))
Rewrite by introducing an auxiliary variable b: p() = max p˜(, b) b
1 T p˜(, b) ∝ exp − ( B + Ξ(b)) 2
B = diag(b) = y − µ − Wx
ICIP 2003 – p. 14
Half-quadratic theory
Noise: p() ∝
exp(− 12
Pd
i=1
ρ(i ))
Rewrite by introducing an auxiliary variable b: p() = max p˜(, b) b
1 p˜(, b) ∝ exp − ( | T{z B} 2
b fixed⇒mms
+Ξ(b))
!
B = diag(b)
= y − µ − Wx
ICIP 2003 – p. 14
Half-quadratic theory
Noise: p() ∝
exp(− 12
Pd
i=1
ρ(i ))
Rewrite by introducing an auxiliary variable b: p() = max p˜(, b) b
1 p˜(, b) ∝ exp − 2
B = diag(b)
(T B + Ξ(b)) | {z }
x fixed⇒arg maxb analytic
= y − µ − Wx ICIP 2003 – p. 14
Complete algorithm
Hypotheses: p(x): Kernel density function Pd 1 p() ∝ exp(− 2 i=1 ρ(i ))
Algorithm sketch: repeat repeat x(k+1) ← x(k) + mms(x(k) ) until inner loop convergence ρ0 (j ) bj ← j until outer loop convergence
ICIP 2003 – p. 15
Plan Appearance-based modeling Linear model, Probabilistic modeling Reconstruction algorithm Example Summary
ICIP 2003 – p. 16
Example COIL
Var 3
xGT Var 2 Var 1
y
ICIP 2003 – p. 17
Example COIL
3D
x ˆ GM L
x ˆ RM L
xGT x ˆ RM AP
x ˆ RM L
x ˆ GM L
x ˆ RM AP
ICIP 2003 – p. 18
Example COIL
All images Euclidian distance in visual subspace from ground truth Average results:
µd ± σ d med ± M AD
GML 34.1 ± 6.8 35.8 ± 4.1
RML 15.7 ± 19.0 7.1 ± 3.1
RMAP 11.9 ± 14.8 4.4 ± 2.5
ICIP 2003 – p. 19
Summary
Contributions: =⇒ A general probabilistic model for visual appearance modeling was presented: robust noise and non-parametric subspace distribution =⇒ An algorithm for MAP reconstruction of an observed image, based on the Mean Shift and HQ-theory
ICIP 2003 – p. 20
Mean Shift: the term
ms(x) =
"P
N i=1 Γi (x)xi PN i=1 Γi (x)
x1
x
−x
#
xN
ICIP 2003 – p. 21
Linear model Decompose an image into a linear basis
+x1
=
y
+...
µ
w1
. . . + xd
+
w
ICIP 2003 – p. 22
Appearance-based models: application
Object/face recognition Separate training and recognition Training Training images
Estimated model
Recognition
Class
Image to recognize
ICIP 2003 – p. 23
Appearance-based models: recognition
Image reconstruction: given an observation y, find x This x is then classified in subspace Standard estimation: Least squares (LS) y
xGT Class distribution
ˆ GM L x ICIP 2003 – p. 24
Estimation of x Maximum likelihood (ML): ˆ = arg max p(y|x) x x
Maximum a postieriori (MAP): 1 ˆ = arg max p(x|y) = arg max x p(y|x)p(x) x x p(y)
ICIP 2003 – p. 25
Visual subspace: notion
g(x)
ICIP 2003 – p. 26
Visual subspace: notion
g(x)
ICIP 2003 – p. 26
Visual subspace: notion
y = g(x) + g(x)
ICIP 2003 – p. 26
Visual subspace: notion
y = g(x) + g(x)
Error Deterministic relation
ICIP 2003 – p. 26
References [CM02] D. Comaniciu and P. Meer. Mean Shift: A robust approach toward feature space analysis. IEEE Trans. Pattern Anal. Machine Intell., 24(5):603–619, May 2002.
ICIP 2003 – p. 27
