Reflectance Reconstruction for Multispectral ... - Color & Imaging Lab.
Reflectance reconstruction for multispectral imaging by adaptive Wiener estimation Optics Express, Vol. 15, No. 23, 2007 Hui-Liang Shen, Pu-Qing Cai, Si-Jie Shao, and John H. Xin Presented by Ji-Hoon Yoo School of Electrical Engineering and Computer Science Kyungpook National Univ.
Abstract Multispectral imaging method – Reconstruction of spectral reflectance • Application of Wiener estimation
Proposed method – Improved reflectance reconstruction method • Use of autocorrelation matrix calculation in Wiener estimation − Selection of training samples
Comparison between Wiener estimation and proposed method – Application in case of different channel numbers and noise levels
2/22
Introduction Several reflectance reconstruction techniques – Wiener estimation – Pseudoinverse method – Finite-dimensional modeling
Proposed method – Reconstruction of spectral reflectance • Use of modified Wiener estimation method − Training sample selection − Autocorrelation matrix construction
3/22
Formulation of multispectral imaging Components of multispectral imaging system – Monochrome digital camera – Several narrow filters – Response of camera
Expression of response vc of the cth channel vc l ( )r ( ) f c ( ) s ( )d bc nc mc ( )r ( )d bc nc
(1)
where l ( ) is the spectral power distribution fo the imaging illuminant, r ( ) is the spectral reflectance of the sample, f c ( ) is the spectral transmittance fo the cth channel, s ( ) is the spectral sensitivity of the monochrome camera, bc ( ) is the bias reponse by dark current, nc ( ) is the zero-mean imaging noise, and mc ( ) is the spectral responsivity.
4/22
– Expression of matrix notation v = Mr + b + n
(2)
where v is the C 1 vector of response, M is the C N matrix of spectral responsivity, r is the N 1 vector of reflectance, b is the C 1 vector of dark current response, and n is the C 1 vector of imaging noise.
• Expression of spectral responsivity M mij 0
(3)
where 1 i C and 1 j N .
• Modification of Eq. (2) − Acquiring detailed solution of M » Removal of dark current b
u = Mr + n
(4)
5/22
Wiener estimation Acquisition of estimated reflectance – Finding matrix W • Transforming response u into estimated reflectance rˆ rˆ = Wu
(5)
• Application of transform matrix WWE K r MT (MK r MT K n ) 1
(6)
where K r and K n are autocorrelation matrices of reflectance and noise.
K r E{rrT }
(7)
K n diag{ 12 , 22 , , C2 }
(8)
6/22
− Estimation of noise variance of cth channel in actual imaging system
ˆ C2 E uc m c r
2
(9)
where x is Frobenius norm of vector x, uc is the response of the cth channel and m c is the spectral responsivity of the cth channel.
7/22
The proposed method Adaptive Wiener estimation – Measurement of spectral similarity • Application of mean spectral distance and maximum spectral distance r ri rˆ rˆ di mean i (1 )max ˆ ˆ r r r r i i
(9)
where 0.5 is a scaling factor, x is the absolute values the elements of vector x.
8/22
– Making set of training samples Ω r1 r1 , , ri ri , rL k1 times ki times 1 times
(11)
where repeat time ki for the ith(1 i L) selected training sample is calculated as
ki (d L / di ) 0.5 where the operator x rounds x to the nearest integer towards zero, 1 is the exponential factor.
(12)
– Modification of Wiener estimation WAWE K r , MT (MK r , MT K n ) 1
(13)
where K r , is the autocorrelation matrix of the reflectance in the training set and K n is the same as Eq. (8).
9/22
Experimental results Components of experiment for multispectral imaging system – Monochrome digital camera model • Retiga-Exi
– Liquid crystal tunable filter(LCTF) – Color target • GretagMacbeth Color Checker
– Reflectance of color patch on Color Checker • Using GretagMacbeth spectrophotometer 7000A − Applying interval of 10nm
– Multispectral images of Color Checker • Acquisition under approximate D65 lighting condition
10/22
Visualization of 31-channel spectral responsivity M
Fig. 1. The spectral responsivities of the 31 channels of the multispectral imaging system. 11/22
Visualization of whole Munsell reflectances
Fig. 2. Reflectances of the 1296 Munsell color chips. 12/22
Evaluation of color accuracy for reflectance reconstruction methods Table 1. The corresponding channel sequences of different channel number C.
13/22
– Evaluation of spectral error • Calculation between actual reflectance and its estimate − Application of rms 1/2
(r rˆ)T (r rˆ) rms N
(14)
– Evaluation of colormetric error E00* • Use of several standard illuminants − D65 − F2
14/22
Synthetic data – Synthesizing response usyn of multispectral imaging system usyn Mr + n σ
(15)
where n is additive Gaussian noise with zero-mean and variance 2 .
• Relationship between noise variance and signal-to-noise ratio(SNR) Tr(MK r MT ) SNR 10 log 2
(16)
where term Tr(MK r MT ) is the average signal power captured by the. multispectral imaging system.
15/22
– Distribution of average spectral error and colormetric error • Channel number C=6 and SNR=50
Fig. 3. The distribution of average reflectance rms error (left) and average color difference error (right) with respect to the training sample number L. 16/22
– Visualization of selected training sets for given candidates
Fig. 4. Two typical examples of the training sets for the given candidate samples.
17/22
– Comparison of several methods Table 2. Comparison of the spectral and colorimetric errors.
18/22
19/22
Real data – Using actual responses of imaging system – Comparison between AWE and WE Table 3. Comparison of the spectral and colorimetric errors.
20/22
– Visualization of reconstructed reflectance using AWE and WE • Channel number C=7
Fig. 5. Reflectance reconstruction of the proposed adaptive Wiener estimation and traditional Wiener estimation. 21/22
Conclusion Proposed method – Reconstruction of spectral reflectances • Using adaptive selection of training samples
– Performance • Investigation for different channel numbers and SNR levels
– Experimental results • Better than Wiener estimation − Overcoming both spectral and color metric prediction errors
22/22
Abstract Multispectral imaging method – Reconstruction of spectral reflectance • Application of Wiener estimation
Proposed method – Improved reflectance reconstruction method • Use of autocorrelation matrix calculation in Wiener estimation − Selection of training samples
Comparison between Wiener estimation and proposed method – Application in case of different channel numbers and noise levels
2/22
Introduction Several reflectance reconstruction techniques – Wiener estimation – Pseudoinverse method – Finite-dimensional modeling
Proposed method – Reconstruction of spectral reflectance • Use of modified Wiener estimation method − Training sample selection − Autocorrelation matrix construction
3/22
Formulation of multispectral imaging Components of multispectral imaging system – Monochrome digital camera – Several narrow filters – Response of camera
Expression of response vc of the cth channel vc l ( )r ( ) f c ( ) s ( )d bc nc mc ( )r ( )d bc nc
(1)
where l ( ) is the spectral power distribution fo the imaging illuminant, r ( ) is the spectral reflectance of the sample, f c ( ) is the spectral transmittance fo the cth channel, s ( ) is the spectral sensitivity of the monochrome camera, bc ( ) is the bias reponse by dark current, nc ( ) is the zero-mean imaging noise, and mc ( ) is the spectral responsivity.
4/22
– Expression of matrix notation v = Mr + b + n
(2)
where v is the C 1 vector of response, M is the C N matrix of spectral responsivity, r is the N 1 vector of reflectance, b is the C 1 vector of dark current response, and n is the C 1 vector of imaging noise.
• Expression of spectral responsivity M mij 0
(3)
where 1 i C and 1 j N .
• Modification of Eq. (2) − Acquiring detailed solution of M » Removal of dark current b
u = Mr + n
(4)
5/22
Wiener estimation Acquisition of estimated reflectance – Finding matrix W • Transforming response u into estimated reflectance rˆ rˆ = Wu
(5)
• Application of transform matrix WWE K r MT (MK r MT K n ) 1
(6)
where K r and K n are autocorrelation matrices of reflectance and noise.
K r E{rrT }
(7)
K n diag{ 12 , 22 , , C2 }
(8)
6/22
− Estimation of noise variance of cth channel in actual imaging system
ˆ C2 E uc m c r
2
(9)
where x is Frobenius norm of vector x, uc is the response of the cth channel and m c is the spectral responsivity of the cth channel.
7/22
The proposed method Adaptive Wiener estimation – Measurement of spectral similarity • Application of mean spectral distance and maximum spectral distance r ri rˆ rˆ di mean i (1 )max ˆ ˆ r r r r i i
(9)
where 0.5 is a scaling factor, x is the absolute values the elements of vector x.
8/22
– Making set of training samples Ω r1 r1 , , ri ri , rL k1 times ki times 1 times
(11)
where repeat time ki for the ith(1 i L) selected training sample is calculated as
ki (d L / di ) 0.5 where the operator x rounds x to the nearest integer towards zero, 1 is the exponential factor.
(12)
– Modification of Wiener estimation WAWE K r , MT (MK r , MT K n ) 1
(13)
where K r , is the autocorrelation matrix of the reflectance in the training set and K n is the same as Eq. (8).
9/22
Experimental results Components of experiment for multispectral imaging system – Monochrome digital camera model • Retiga-Exi
– Liquid crystal tunable filter(LCTF) – Color target • GretagMacbeth Color Checker
– Reflectance of color patch on Color Checker • Using GretagMacbeth spectrophotometer 7000A − Applying interval of 10nm
– Multispectral images of Color Checker • Acquisition under approximate D65 lighting condition
10/22
Visualization of 31-channel spectral responsivity M
Fig. 1. The spectral responsivities of the 31 channels of the multispectral imaging system. 11/22
Visualization of whole Munsell reflectances
Fig. 2. Reflectances of the 1296 Munsell color chips. 12/22
Evaluation of color accuracy for reflectance reconstruction methods Table 1. The corresponding channel sequences of different channel number C.
13/22
– Evaluation of spectral error • Calculation between actual reflectance and its estimate − Application of rms 1/2
(r rˆ)T (r rˆ) rms N
(14)
– Evaluation of colormetric error E00* • Use of several standard illuminants − D65 − F2
14/22
Synthetic data – Synthesizing response usyn of multispectral imaging system usyn Mr + n σ
(15)
where n is additive Gaussian noise with zero-mean and variance 2 .
• Relationship between noise variance and signal-to-noise ratio(SNR) Tr(MK r MT ) SNR 10 log 2
(16)
where term Tr(MK r MT ) is the average signal power captured by the. multispectral imaging system.
15/22
– Distribution of average spectral error and colormetric error • Channel number C=6 and SNR=50
Fig. 3. The distribution of average reflectance rms error (left) and average color difference error (right) with respect to the training sample number L. 16/22
– Visualization of selected training sets for given candidates
Fig. 4. Two typical examples of the training sets for the given candidate samples.
17/22
– Comparison of several methods Table 2. Comparison of the spectral and colorimetric errors.
18/22
19/22
Real data – Using actual responses of imaging system – Comparison between AWE and WE Table 3. Comparison of the spectral and colorimetric errors.
20/22
– Visualization of reconstructed reflectance using AWE and WE • Channel number C=7
Fig. 5. Reflectance reconstruction of the proposed adaptive Wiener estimation and traditional Wiener estimation. 21/22
Conclusion Proposed method – Reconstruction of spectral reflectances • Using adaptive selection of training samples
– Performance • Investigation for different channel numbers and SNR levels
– Experimental results • Better than Wiener estimation − Overcoming both spectral and color metric prediction errors
22/22
