Synthesis of Color Filter Array Pattern in Digital Images
Faculty of Computer Science Institute of Systems Architecture, Privacy and Data Security Research Group
Synthesis of Color Filter Array Pattern in Digital Images M atthias Kirchner and Rainer Böhme {matthias.kirchner,rainer.boehme}@inf.tu-dresden.de
M edia Forensics and Security XI San Jose, CA · 2009/ 01/ 20
Digital image forensics and tamper hiding ◮ variety of different forensic tools can be found in the literature ◮ existing schemes work well under laboratory conditions resampling artifacts
sensor dust
copy & paste
double compression
Kirchner & Böhme
How reliable are forensic results if the presumed counterfeiter is aware of the forensic tools?
sensor noise lens distortions
Image Forensics
CFA pattern
Synthesis of CFA Pattern in Digital Images
slide 1 of 13
Digital image forensics and tamper hiding ◮ variety of different forensic tools can be found in the literature ◮ existing schemes work well under laboratory conditions resampling artifacts
sensor dust
copy & paste
double compression
Kirchner & Böhme
How reliable are forensic results if the presumed counterfeiter is aware of the forensic tools?
Tamper hiding sensor noise
◮ mislead forensic tools such that they produce false negatives lens distortions
Image Forensics
CFA pattern
Synthesis of CFA Pattern in Digital Images
slide 1 of 13
1 CFA Synthesis
Problem statement ◮ typical digital cameras use a color filter array (CFA) to capture full color images
◮ color filter interpolation introduces periodic correlation pattern between neighboring pixels Kirchner & Böhme
Synthesis of CFA Pattern in Digital Images
slide 2 of 13
Problem statement ◮ typical digital cameras use a color filter array (CFA) to capture full color images
◮ CFA pattern has to be restored to conceal traces of manipulation
◮ color filter interpolation introduces periodic correlation pattern between neighboring pixels Kirchner & Böhme
Synthesis of CFA Pattern in Digital Images
slide 2 of 13
Problem statement ◮ typical digital cameras use a color filter array (CFA) to capture full color images
◮ CFA pattern has to be restored to conceal traces of manipulation ◮ straight-forward: re-interpolation
◮ color filter interpolation introduces periodic correlation pattern between neighboring pixels
◮ overw rites two thirds of all pixels w ith new (interpolated) values
Kirchner & Böhme
Synthesis of CFA Pattern in Digital Images
slide 2 of 13
A minimal distortion approach Linear model ◮ CFA interpolation follow s a linear equation ◮ image w ith incomplete/missing CFA pattern is corrupted by an additive residual ǫ
ˆ = Hx y y = Hx + ǫ
CFA synthesis ◮ find a possible sensor signal x such that ◮ least squares solution
ˆ k → min kǫk = ky − y x = (H′ H)−1 H′ y
CFA re-interpolation not from the signal itself, but from a pre-filtered version
Kirchner & Böhme
Synthesis of CFA Pattern in Digital Images
slide 3 of 13
Structure of H ◮ for N pixels per channel and M ≤ N/2 genuine sensor samples, a direct implementation of the LS solution is impossible for typical image sizes ◮ matrix H has dimension N × M ◮ cubic complexity: x = (H′ H)−1 H′ y
N
H
inversion O(M 3 ) multiplication O(M 2 N )
M Kirchner & Böhme
Synthesis of CFA Pattern in Digital Images
slide 4 of 13
Structure of H ◮ for N pixels per channel and M ≤ N/2 genuine sensor samples, a direct implementation of the LS solution is impossible for typical image sizes ◮ matrix H has dimension N × M ◮ cubic complexity: x = (H′ H)−1 H′ y
N
H
inversion O(M 3 ) multiplication O(M 2 N )
Efficiency improvements ◮ matrix H is typically sparse (interpolation kernels have finite support) and has a regular structure (Bayer pattern) M Kirchner & Böhme
Synthesis of CFA Pattern in Digital Images
slide 4 of 13
2 Red Channel
Partitioning H ◮ columns partition H into repeating blocks A, B w ith B = 1/2A ◮ H= A⊗A √ √ ◮ A has only dimension N × N /2 + 1
1 1/2 1/2
A
1 1/2 1/2
1
=
1/2
1/2
1/4 1/4
1/4 1/4
1/2
1/2
1/4 1/4
1/4 1/4
1/2
1/2
1 1/2 1/2 b b
Kirchner & Böhme
Synthesis of CFA Pattern in Digital Images
slide 5 of 13
Partitioning H ◮ columns partition H into repeating blocks A, B w ith B = 1/2A ◮ H= A⊗A √ √ ◮ A has only dimension N × N /2 + 1
1 1/2 1/2
√
1
N
1/2 1/2
1 1/2
1/2
1/4 1/4
1/4 1/4
1/2
1/2
1/4 1/4
1/4 1/4
1/2
Kronecker tweaks 1/2
1 1/2 1/2 b b
x = (H′ H)−1 H′ y w ith H× = H′ H : ◮ (H× )−1 = (A× )−1 ⊗ (A× )−1 w ith H+ = (H× )−1 H′ : ◮ H+ = A+ ⊗ A+
Kirchner & Böhme
Synthesis of CFA Pattern in Digital Images
(pseudo-inverse)
slide 5 of 13
Analytical inversion Φ = (A× )−1 ◮ method by Huang & M cColl (1997)
5/4 1/4 1/4 3/2 1/4
0
1/4 3/2 1/4
ζi = 3/2 ζi−1 − (1/4)2 ζi−2
1/4 3/2 1/4
A×
second order linear recurrences: υj = 3/2 υj+1 − (1/4)2 υj+2
=
and ratios: ζi ξi = ζi−1
0
and
γi =
υi υi+1
1/4 3/2 1/4 1/4 3/2 1/4 1/4 5/4
A× is tridiagonal symmetric
Kirchner & Böhme
Synthesis of CFA Pattern in Digital Images
slide 6 of 13
Analytical inversion Φ = (A× )−1 ◮ method by Huang & M cColl (1997) −1/4 γi
Φi+1,j
second order linear recurrences:
,j
Φj
Φi,j =
×
“ A j,j
=
ζi = 3/2 ζi−1 − (1/4)2 ζi−2
−
υj = 3/2 υj+1 − (1/4)2 υj+2
16 1/
and ratios: ζi ξi = ζi−1
−
− (ξ j
Φ=
+
1 1
and
−
γ j+
γi =
”−
1 1)
υi υi+1
−1/4 ξi
1
Φi,j =
Φi−1,j
◮ inversion has complexity O(N/4)
Kirchner & Böhme
Synthesis of CFA Pattern in Digital Images
slide 6 of 13
Red channel approximate solution Infinite image ◮ Φ is asymptotically symmetric Toeplitz Φj,j → ΦD ”|i−j| “ 1 ΦD Φi,j → − /q4
Kirchner & Böhme
Synthesis of CFA Pattern in Digital Images
slide 7 of 13
Red channel approximate solution Asymptotic kernel
Infinite image ◮ Φ is asymptotically symmetric Toeplitz
0.5
Φj,j → ΦD ”|i−j| “ 1 ΦD Φi,j → − /q4
Synthesis of Color Filter Array Pattern in Digital Images M atthias Kirchner and Rainer Böhme {matthias.kirchner,rainer.boehme}@inf.tu-dresden.de
M edia Forensics and Security XI San Jose, CA · 2009/ 01/ 20
Digital image forensics and tamper hiding ◮ variety of different forensic tools can be found in the literature ◮ existing schemes work well under laboratory conditions resampling artifacts
sensor dust
copy & paste
double compression
Kirchner & Böhme
How reliable are forensic results if the presumed counterfeiter is aware of the forensic tools?
sensor noise lens distortions
Image Forensics
CFA pattern
Synthesis of CFA Pattern in Digital Images
slide 1 of 13
Digital image forensics and tamper hiding ◮ variety of different forensic tools can be found in the literature ◮ existing schemes work well under laboratory conditions resampling artifacts
sensor dust
copy & paste
double compression
Kirchner & Böhme
How reliable are forensic results if the presumed counterfeiter is aware of the forensic tools?
Tamper hiding sensor noise
◮ mislead forensic tools such that they produce false negatives lens distortions
Image Forensics
CFA pattern
Synthesis of CFA Pattern in Digital Images
slide 1 of 13
1 CFA Synthesis
Problem statement ◮ typical digital cameras use a color filter array (CFA) to capture full color images
◮ color filter interpolation introduces periodic correlation pattern between neighboring pixels Kirchner & Böhme
Synthesis of CFA Pattern in Digital Images
slide 2 of 13
Problem statement ◮ typical digital cameras use a color filter array (CFA) to capture full color images
◮ CFA pattern has to be restored to conceal traces of manipulation
◮ color filter interpolation introduces periodic correlation pattern between neighboring pixels Kirchner & Böhme
Synthesis of CFA Pattern in Digital Images
slide 2 of 13
Problem statement ◮ typical digital cameras use a color filter array (CFA) to capture full color images
◮ CFA pattern has to be restored to conceal traces of manipulation ◮ straight-forward: re-interpolation
◮ color filter interpolation introduces periodic correlation pattern between neighboring pixels
◮ overw rites two thirds of all pixels w ith new (interpolated) values
Kirchner & Böhme
Synthesis of CFA Pattern in Digital Images
slide 2 of 13
A minimal distortion approach Linear model ◮ CFA interpolation follow s a linear equation ◮ image w ith incomplete/missing CFA pattern is corrupted by an additive residual ǫ
ˆ = Hx y y = Hx + ǫ
CFA synthesis ◮ find a possible sensor signal x such that ◮ least squares solution
ˆ k → min kǫk = ky − y x = (H′ H)−1 H′ y
CFA re-interpolation not from the signal itself, but from a pre-filtered version
Kirchner & Böhme
Synthesis of CFA Pattern in Digital Images
slide 3 of 13
Structure of H ◮ for N pixels per channel and M ≤ N/2 genuine sensor samples, a direct implementation of the LS solution is impossible for typical image sizes ◮ matrix H has dimension N × M ◮ cubic complexity: x = (H′ H)−1 H′ y
N
H
inversion O(M 3 ) multiplication O(M 2 N )
M Kirchner & Böhme
Synthesis of CFA Pattern in Digital Images
slide 4 of 13
Structure of H ◮ for N pixels per channel and M ≤ N/2 genuine sensor samples, a direct implementation of the LS solution is impossible for typical image sizes ◮ matrix H has dimension N × M ◮ cubic complexity: x = (H′ H)−1 H′ y
N
H
inversion O(M 3 ) multiplication O(M 2 N )
Efficiency improvements ◮ matrix H is typically sparse (interpolation kernels have finite support) and has a regular structure (Bayer pattern) M Kirchner & Böhme
Synthesis of CFA Pattern in Digital Images
slide 4 of 13
2 Red Channel
Partitioning H ◮ columns partition H into repeating blocks A, B w ith B = 1/2A ◮ H= A⊗A √ √ ◮ A has only dimension N × N /2 + 1
1 1/2 1/2
A
1 1/2 1/2
1
=
1/2
1/2
1/4 1/4
1/4 1/4
1/2
1/2
1/4 1/4
1/4 1/4
1/2
1/2
1 1/2 1/2 b b
Kirchner & Böhme
Synthesis of CFA Pattern in Digital Images
slide 5 of 13
Partitioning H ◮ columns partition H into repeating blocks A, B w ith B = 1/2A ◮ H= A⊗A √ √ ◮ A has only dimension N × N /2 + 1
1 1/2 1/2
√
1
N
1/2 1/2
1 1/2
1/2
1/4 1/4
1/4 1/4
1/2
1/2
1/4 1/4
1/4 1/4
1/2
Kronecker tweaks 1/2
1 1/2 1/2 b b
x = (H′ H)−1 H′ y w ith H× = H′ H : ◮ (H× )−1 = (A× )−1 ⊗ (A× )−1 w ith H+ = (H× )−1 H′ : ◮ H+ = A+ ⊗ A+
Kirchner & Böhme
Synthesis of CFA Pattern in Digital Images
(pseudo-inverse)
slide 5 of 13
Analytical inversion Φ = (A× )−1 ◮ method by Huang & M cColl (1997)
5/4 1/4 1/4 3/2 1/4
0
1/4 3/2 1/4
ζi = 3/2 ζi−1 − (1/4)2 ζi−2
1/4 3/2 1/4
A×
second order linear recurrences: υj = 3/2 υj+1 − (1/4)2 υj+2
=
and ratios: ζi ξi = ζi−1
0
and
γi =
υi υi+1
1/4 3/2 1/4 1/4 3/2 1/4 1/4 5/4
A× is tridiagonal symmetric
Kirchner & Böhme
Synthesis of CFA Pattern in Digital Images
slide 6 of 13
Analytical inversion Φ = (A× )−1 ◮ method by Huang & M cColl (1997) −1/4 γi
Φi+1,j
second order linear recurrences:
,j
Φj
Φi,j =
×
“ A j,j
=
ζi = 3/2 ζi−1 − (1/4)2 ζi−2
−
υj = 3/2 υj+1 − (1/4)2 υj+2
16 1/
and ratios: ζi ξi = ζi−1
−
− (ξ j
Φ=
+
1 1
and
−
γ j+
γi =
”−
1 1)
υi υi+1
−1/4 ξi
1
Φi,j =
Φi−1,j
◮ inversion has complexity O(N/4)
Kirchner & Böhme
Synthesis of CFA Pattern in Digital Images
slide 6 of 13
Red channel approximate solution Infinite image ◮ Φ is asymptotically symmetric Toeplitz Φj,j → ΦD ”|i−j| “ 1 ΦD Φi,j → − /q4
Kirchner & Böhme
Synthesis of CFA Pattern in Digital Images
slide 7 of 13
Red channel approximate solution Asymptotic kernel
Infinite image ◮ Φ is asymptotically symmetric Toeplitz
0.5
Φj,j → ΦD ”|i−j| “ 1 ΦD Φi,j → − /q4
