Automatic Grayscale Image Colorization Using Histogram Regression

Automatic Grayscale Image Colorization Using Histogram Regression Pattern Recognition Letters Vol. 33, No. 13, 2012 Shiguang Liu and Xiang Zhang Presented by Ji-Hoon Yoo

School of Electrical Engineering and Computer Science Kyungpook National Univ.

Abstract  Colorizing grayscale images – Previous colorization algorithms • User intervention − Hard work for making colorization

– Proposed method • Automatic grayscale image colorization − Locally weighted regression − Aligning features for matching

» Finding and adjusting zero-point − Weighted colorization

2/17

Introduction  Grayscale image colorization – Increasing visual appeal of images – Source image • Reference color image

– Target image • Grayscale image to be colorized

 Previous method – User interactions for satisfactory result • Labor intensive and time consuming − Tedious handmade work

3/17

 Proposed method – Automatic grayscale image colorization • Locally weighted regression(LWR) − Analyzing feature distributions of source and target images

• Matching features for achieving luminance-color correspondence − By Finding and adjusting zero-points

• Transferring colors from source image to grayscale image

4/17

Related Work  Semi-automatic methods – Transfer color to grayscale • • • • •

Luminance key and matching area using swatches Segmentation and local correlation Chrominance blending and kernel Hilbert space multiple images on web Morphological distance and histogram matching

– Processing manually • User interaction for accurate processing − Labor intensive and time consuming

• Unseemly color to different region

5/17

Overview of the histogram regression based automatic colorization method  Overview of automatic colorization – Flow chart of algorithm

Fig. 1. overview of our algorithm

6/17

Locally weighted regression on histograms  Locally weighted regression – Image and training data on histogram

Fig. 2. Results of performing LWR on the image histogram: (a) grayscale image, (b) histogram and fitting curve. (c) slopes and (d) y-intersects.

7/17

– Minimization for fitting data m

  arg min  w j ( T X j  Y j )

(1)

j 1

where      0 ,  1 

x  Xj= 0   x1 

(2)

where ( X , Y ) and  represent training data and fitting data, ( X j , Y j ) denote jth sampling of training data, m is size of training data. Vector  is regression results for particular j,  0 is slope at j, and 1 is y-intersect at j. w j is weight.

– Weight term

 X X j w j  exp    2 2 

2

   

(3)

where  is constant in our experiments.

– Histogram of image 1 y j ( j  N , 0  j  255) and x j     j

(4) 8/17

Zero-points adjustment and matching  Generation of zero-points – Extrema alternation sequence 𝑇 • Maxima and minima of zero-points − Local maximum on 𝑇 𝑖 = 1 − Local minimum on 𝑇 𝑖 = −1 » Segmentation set as 𝑆

• Case of 𝑇𝑆 = 𝑇𝑇 − Similar contents of source image and grayscale image − Matched segmentation 𝑆𝑆 and 𝑆𝑇

9/17

 Adjustment of zero-points – Front-rear adjustment • Making difference of two sequence even − Elimination of front or rear for odd » Without alternation property

– Matching zero-points • Pair adjustment algorithm − Elimination of adjacent pair of points

(a) (b) Fig. 3. Pairs of points in trend: (a) zero-points in ascent trend and (b) zero-points in descent trend. 10/17

Weighted colorization  Luminance-color correspondence – Weight function  x w( x,  , )  exp   2  2  

2

  (SS )  (SS )  ,  =(j-T )  S ,     ( ST )  ( ST ) 

(5)

where  and  are the central value and the bandwidth.  S and T are the minimal value in S S and ST .

– Weighted colorization formula   (SS )  (SS )  w i , ( j   )   ,  i1    cs (i ) T S  ( S )  ( S )  T T  cT ( j )   255  (SS )  (SS )  w i , ( j   )   ,  i1   T  ( ST ) S  ( S S )   255

(6)

where CT ( j ) is average color of gray image. Central value and bandwidth are  and  . 11/17

Confidence  Confidence measure – Measuring size of δ q1  e



  ( ZT )

(7)

where ZT is sequence set of grayscale image after processing by zero-points adjustment algorithm.

– Definition of mean 𝜆 and standard deviation 𝜎 of histogram value 1 255  yj  256 j 0

(8)

1 255  ( y j   )2  256 j 0

(9)

12/17

Implementation and results  Experiment result – Confidence and processing time Table 1. parameters and their values in our experiments

13/17

– Results of our algorithm

(a) source image

(b) grayscale image

(f) histogram of (a)

(g) histogram of (b)

(c) matching result of (a) (d) matching result of (b)

(e) colorization result

(h) adjusted results of (f) (i) adjusted results of (g)

(j) histogram of (e)

Fig. 4. Colorization result of an animal scene. 14/17

– Contrast of colorization result

(a)

(b)

(c)

Fig. 5. Contrast between our colorization result and other methods. (a) and (b) From left to right: source image: grayscale image: result of Welsh et al. (2002): result of Li and Hao (2008): result of our method and ground truth. (c) PSNR contrast between our method and other methods. 15/17

– colorization results

(a)

(b)

(c)

(d)

(e)

(f)

Fig. 6. Colorization results, from left to right: source, target, histogram of source, histogram of gray 16/17 image, adjusted histogram of source. adjusted histogram of gray image, colorization result.

Conclusions and future work  Automatic grayscale image colorization – Locally weighted regression • Histograms of source image and grayscale image

– Zero-points adjustment algorithm • Pairing procedures − between histograms of source image and grayscale image

– Weighted colorization • Colorizing grayscale image

17/17