Separate Color Correction for Tone Compression in HDR Image ...
IEICE TRANS. FUNDAMENTALS, VOL.E96–A, NO.8 AUGUST 2013
1752
PAPER
Separate Color Correction for Tone Compression in HDR Image Rendering Hwi-Gang KIM† , Nonmember and Sung-Hak LEE††a) , Member
SUMMARY Many High-Dynamic-Range (HDR) rendering techniques have been developed. Of these, the image color appearance model, iCAM, is a typical HDR image rendering algorithm. HDR rendering methods normally require a tone compression process and include many color space transformations from the RGB signal of an input image to the RGB signal of output devices for the realistic depiction of a captured image. The iCAM06, which is a refined iCAM, also contains a tone compression step and several color space conversions for HDR image reproduction. On the other hand, the tone compression and frequent color space changes in the iCAM06 cause color distortion, such as a hue shift and saturation reduction of the output image. To solve these problems, this paper proposes a separate color correction method that has no effect on the output luminance values by controlling only the saturation and hue of the color attributes. The color saturation of the output image was compensated for using the compensation gain and the hue shift was corrected using the rotation matrix. The separate color correction method reduces the existing color changes in iCAM06. The compensation gain and rotation matrix for the color correction were formulated based on the relationship between the input and output tristimulus values through the tone compression. The experimental results show that the revised iCAM06 with the proposed method has better performance than the default iCAM06. key words: HDR image, iCAM06, tone compression, Hue, saturation
1.
Introduction
People are living in a luminance environment with the extensive dynamic range, and a wide range of absolute luminance levels is sometimes encountered. In addition, the human visual system (HVS) handles a wide luminance dynamic range easily. On the other hand, most available digital cameras and color display devices are limited in their dynamic range and cannot show a real scene in a single image [1]. Therefore, it is essential to render HDR images on the display devices with a lower dynamic range [2]. Consequently, HDR imaging techniques have been introduced and developed to take an image of a real wide luminance dynamic range in a single image. An HDR image can represent an image visually close to dynamic range of the visible world. It is created by merging several images of the same scene with different exposure values. The iCAM (image Color Appearance Model) makes it possible to represent the HDR images onto practical output devices. The iCAM06 is a recent refined model, in which several modules have been affected by preManuscript received December 14, 2012. Manuscript revised April 17, 2013. † The author is with the Division of Mathematical Models, National Institute for Mathematical Sciences, Korea. †† The author is with the School of Electronics Engineering, Kyungpook National University, Korea. a) E-mail: [email protected] DOI: 10.1587/transfun.E96.A.1752
vious algorithms, such as the local white adaptation, chromatic adaptation and IPT uniform color space conversion from the iCAM framework [3], [4]. In addition, iCAM06 incorporates some modified steps in that the single-scale Gaussian filtering is replaced with a two-layer image decomposition using the edge preserving bilateral filter and a simple non-linear local gamma correction is replaced with photoreceptor response functions. The iCAM06 also predicts a range of visual effects, such as a contrast change by the Stevens effect, colorfulness change by the Hunt effect, and an image contrast change in a complex image with surround luminance by the Bartleson-Breneman effect. Moreover, the iCAM06 extends to a wide range of luminance levels through the sum of scotopic and photopic vision signals [4]. In general, the dynamic range of an HDR image needs to be mapped to output devices, which is called tone mapping or tone compression, which is characterized on the photoreceptor curves of the human eyes. After tone compression, the color space conversions are normally necessary to adjust the color attributes. The iCAM06 causes saturation reduction by color clipping in the tone compression process and color signals do not maintain the hue linearity and exhibit a lack of perceptual color uniformity in some color space transformations [5]–[7]. In particular, in the case of an image including low chromatic regions, such as a snow and clouds scene, the rendered result image by iCAM06 is reddish all over the original image. This reddish phenomenon occurs near the white point in the results of iCAM06 [8]. The goal of iCAM06 is realistic image rendering that is adaptive to visual adaptation for a wide luminance level. In this regard, the color change is a weakness of the iCAM06. This paper proposes a color correction method that is separate from tone compression process by combining each saturation and hue correction. The separate color correction method preserves the compressed luminance condition of the output image and compensates for the color distortion in the rendered result images by iCAM06. First, the saturation compensation gain was obtained based on the ratio of the calculated saturation values in the tone-compressed image and the original image for each and the new tristimulus values for saturation correction was calculated. Second, the hue angle shifted by frequent color space conversions was calculated, and the hue angle was set to the previous original hue angle using the rotation matrix. Finally, some objective and subjective experiments were performed to confirm the performance of the proposed method. The experimen-
c 2013 The Institute of Electronics, Information and Communication Engineers Copyright
KIM and LEE: SEPARATE COLOR CORRECTION FOR TONE COMPRESSION
1753
tal results show that the modified iCAM06 with the separate color correction method has better performance in terms of the output color accuracy and individual preference for test images. In addition, this method can be applied easily to other algorithms for color correction. Section 2 describes the framework of iCAM06 and explains briefly the several processes in iCAM06 including the color space transformations. The proposed method and experimental results is discussed in Sects. 3 and 4, respectively. Section 5 reports the conclusions. 2.
Previous Works
2.1 Tone Compression in the iCAM06 The purposes of iCAM06 were to predict accurately the human visual attributes of complex images in a wide range of luminance levels and to display the same human visual perception across media [4]. The iCAM06 represents a wide luminance range using tone compression functions based on the scotopic and photopic visual responses. The nonlinear compression is designed according to the photoreceptor responses including cones and rods. The tone compression functions, as shown in Eqs. (1)–(5), are similar to those of CIECAM02 [9]. On the other hand, they include a slightly modified power value p. The user controllable variable p accounts for the steepness of the photoreceptor response curves. 400(F L R /YW ) p + 0.1 27.13 + (F L R /YW ) p 400(F L G /YW ) p Ga = + 0.1 27.13 + (F L G /YW ) p 400(F L B /YW ) p Ba = + 0.1 27.13 + (F L B /YW ) p F L = 0.2k4 (5LA ) + 0.1(1 − k4 )2 (5LA )1/3 k = 1/(5LA + 1) Ra =
(1) (2) (3) (4) (5)
where RG B and RG Ba are the cone response before and after tone compression under the chromatic adaptation of D65 illumination, respectively. LA is the adapting luminance taken to be 20% of the luminance of a white object in the scene. YW is the luminance of a local adapted white image. The non-linear sigmoid functions are used in the iCAM06 for tone compression. The above functions are based on a generalized Michaelis-Menten Equation and are consistent with the Valeton and van Norren’s experimental data [10]. Hunt [11] discussed the specifics and advantages of the equations. The F L function is used to predict a variety of the luminance-dependent appearance effects. Finally, the tone compression response is the sum of the cones and rods response, but the rods response has very little impact [4]. 2.2 Color Space Transformation for Image Adjustment The tone compressed signals were converted into IPT color space. The IPT uniform opponent color space is useful
for color attribute adjustments without unwanted image artifacts. Three visual attributes, such as lightness, hue and chroma, were adjusted to predict the effective color appearance effects. The details adjustment in the detail-layer processing was applied to predict the Stevens effect that increases the perceived lightness contrast with increasing luminance. In the IPT color space, P and T were adjusted to predict the Hunt effect that increases the colorfulness with increasing luminance for an accurate image appearance. To display the processed image on the output devices, The IPT values were converted back to XYZ values by the inverted chromatic adaptation transformation from the CIE standard illuminant, D65 , to the output device white point [4]. The inverse output characterization model was used to transform the XYZ values to the device dependent RGB values. For clear visibility, any extremely dark or bright pixel data were cut down by the clipping function [2]. The clipped data were limited to the range of the 1st and 99th percentile of the processed data as follows: ⎧ ⎪ RGB, for RGB1% ≤ RGB ≤ RGB99% ⎪ ⎪ ⎨ RGB , for RGB99% < RGB clip : RGBclip = ⎪ 99% ⎪ ⎪ ⎩RGB , for RGB < RGB 1%
1%
(6) The clipped RGB data were normalized to Eq. (7), and modulated with the power function. The output values, RGBout , were rescaled according to the output values from 0 to 255, as shown Eq. (8). RGBn =
RGBclip − (RGBclip )min (RGBclip )max − (RGBclip )min
RGBout = 255(RGBn )1/1.7
(7) (8)
The HDR rendering process in iCAM06 includes color conversions for image adjustment. These are the main reason for color distortion. That is, the color shift of the tone compression step can enlarge the color error in a display terminal [8]. 2.3 Saturation Compensation in Tone Compression The color saturation is a unique perceptual experience separated from chroma. The definition of color saturation is the colorfulness of a judged area in proportion to its brightness. Remarkably, under viewing conditions with the luminance level within photopic vision, the color stimulus of the chromaticity shows similar saturation for all luminance levels, except when the brightness is quite high [12]. However, the iCAM06 causes a decrease in saturation by the color clipping during its tone compression process. The default iCAM06 does not consider saturation compensation. Therefore, it is important to add a color saturation compensation step to solve the under-saturation problem [13]. Generally the CIE 1976 u v saturation is calculated from the following equation: Satuv = 13 (u − un )2 + (v − vn )2 (9)
IEICE TRANS. FUNDAMENTALS, VOL.E96–A, NO.8 AUGUST 2013
1754
where un and vn are the chromaticity coordinates of the reference white point. The attenuation factor, α, was obtained from the ratio of the calculated saturation values in the tone compressed image and the original image for each [13]. The factor, α, and the compensated saturation value, Sat sc , were calculated as follows: Satc (10) Sat sc = α Satc α= , Satin > 0 (11) Satin where Satc and Satin were calculated from the CIE 1976 u v saturation, which are the color saturation values of the tone compressed signal, RG B a , and the non- compressed input signal, RG B , respectively. The attenuation factor, α, was multiplied inversely by the compressed color saturation value. The compensated chromaticity coordinates (usc , vsc ) were calculated from Sat sc as follows: usc = Sat sc · cos θ + un vsc = Sat sc · sin θ + vn vc − vn θ = arctan uc − un
(12) (13) (14)
The calculated usc , vsc were transformed to the corrected tristimulus XYZ sc and the new tristimulus were transformed to the IPT uniform opponent color space, as shown in Sect. 2.2. 3.
Separate Color Correction for Color Preserving
Despite the additional saturation compensation step as described in Sect. 2.3, iCAM06 still has a hue shift problem. Therefore, this paper proposes the completed color correction method including the hue compensation and previous saturation compensation. This method, first, calculated the saturation compensation gain similar to Eq. (10), which was obtained from the ratio of the calculated saturation values in the tone compressed image and the original image. The saturation compensation gain, S g , and the saturationcompensated chromaticity coordinates (usc , vsc ) were calculated as follows: Satin (15) Sg = Satc usc = S g (uc − un ) + un (16) (17) v sc = S g (vc − vn ) + vn In these equations, unlike the equations in Sect. 2.3, the computation was simplified for the saturation compensation without trigonometric functions, which require their polarity conditions to select positive or negative values depending on the angle range. In addition, the modified compensation equations are more suitable for the gain control of colors around a reference white point compared to the previous equations, which can change colors incorrectly when
changing polarity conditions. Then the hue shift correction procedure was added, which could return the shifted hue angle to the original hue angle using a rotation matrix. The hue angle was calculated using Eq. (18) in the CIE 1976 L∗ u∗ v∗ color space [14]. huv = arctan
v∗ u∗
(18)
The CIELUV∗ is defined by following equations: L∗ = 116(Y/Yn )1/3 − 16 u∗ = 13L∗ (u − un )1/3 v∗ = 13L∗ (v − vn )1/3
(19) (20) (21)
where L∗ is the lightness, u∗ is the redness-greenness, and v∗ is the yellowness-blueness. Y is the white stimulus of the tristimulus XYZ value and Yn is the reference white [14]. This process handles the u∗ v∗ coordinate points for the hue shift correction. Therefore, this method does not affect the luminance of the output images. The hue angle difference was obtained by comparing the two hue angles from the (u , v ) of the non-shifted input signal and the (usc , vsc ) of the shifted output signal at the previous work and the difference angle φd between the two chromaticity coordinate points can be found by subtracting the two angles as follows: φ sc = h sc = arctan(v∗sc /u∗sc ) φin = hin = arctan(v∗in /u∗in ) φd = φ sc − φin
(22) (23) (24)
where φ sc (h sc ) and φin (hin ) mean the hue angles of the shifted output signal and the non-shifted input signal respectively. The shifted coordinates (usc , vsc ) in the previous compensation step are revolved around the white point by the difference angle, φd , between the two points using the rotation matrix as follows:
uhc u sc = (φd ) (25) vhc vsc
cos φd sin φd (φd ) = (26) − sin φd cos φd where (usc , vsc ) means the final correction coordinates. The complete color correction incorporates the saturation compensation and hue correction method. The corrected (uhc , vhc ) were transformed to the corrected tristimulus XYZ hc . In subsequent processes, there is no need for any additional color adjustments in iCAM06 because the lower colorfulness effect due to under saturation is compensated for by proposed color correction process. The remaining procedure is the same as the existing iCAM06 from the inverse chromatic adaptation transformation to the output RGB. Figure 1(a) presents the overall flowchart of the modified iCAM06 including the completed separate color correction step. The separate color correction makes it possible to skip the post-color adjustments after the tone compression
KIM and LEE: SEPARATE COLOR CORRECTION FOR TONE COMPRESSION
1755
Fig. 1 Flowchart of the modified iCAM06 by the separate color correction: (a) Modified iCAM06 including the proposed method, (b) separate color correction.
in iCAM06. Figure 1(b) shows the magnified block diagram of the proposed color correction step. The proposed method maintains the image luminance with the same Yhc and Yc . 4.
Simulation Results
4.1 Objective Assessment The proposed method was simulated to confirm the performance and compare the results with the previous algorithms. The change in the hue angle with the variation of color intensity was confirmed. Here, the color patches with the same hue and different saturation were used, in which the color intensity changes at regular intervals. The used 7 colors are red, green, blue, cyan, magenta, yellow, and purple by the RGB tristimulus combination. The multiple scatter and lines for the selected colors are drawn on the coordinates CIE 1976 u v , as shown in Fig. 2. The gray points and lines mean the original input placed on the same hue trajectory for the change in color intensity. The color points and lines mean the output values. Figure 2 confirmed that the hue shift of each color was much less than that of iCAM06. This result suggests that the altered hue angles were restored close to the original hue angles using the proposed method. The modified iCAM06 maintain the perceptual color uniformity of the rendered output image. In addition, the output color gamut is also considerably wider than the existing iCAM06. Table 1 lists the calculated color differences for each color sample from both diagrams shown in Fig. 2.
Fig. 2 Comparison between the input and rendered output of samples on the CIE 1976 u v chromaticity diagrams: (a) The iCAM06, (b) the modified. Table 1
Color difference errors for the selected colors.
4.2 Subjective Assessment Subjective experiments were conducted to confirm the visual performance of the color reproduction. Pair comparison tests were conducted to evaluate the rendering performance in two ways; the individual preference evaluation and color accuracy scoring to original scenes. The preference evaluation was conducted just by comparing the rendered results between the iCAM06 and the proposed, and the accuracy scoring method counts the color matching for the originals before merging an HDR image. Figure 3 provides examples of the images for the comparison tests. Thumbnail images
IEICE TRANS. FUNDAMENTALS, VOL.E96–A, NO.8 AUGUST 2013
1756
Fig. 3 Examples of the images used for the comparison test; CraterLake, 507, LasVegas Store, LabBooth in a clockwise direction.
are placed in the left column and red circled parts on the pictures are for the effective color accuracy comparison. The rendering results were placed top and bottom in the right columns. The experiments were carried out with 24 people with normal color vision. The rendered images were displayed on a 27 DELL U2711 Monitor. First, in the individual preference test, the participants were asked to select the image they prefer by a subjective judgment between a pair of rendered images. In this time, especially, the easiest picture was selected to examine the hue correction effect among the various rendered result images. The top-left image, named CraterLake, includes a wide low chromatic field like snow and sky. The top image by iCAM06 shows an overall reddish colored image. The bottom image by the proposed color correction shows that the color of snow and sky is closer to the actual color. Next, in the color accuracy test, the participants were asked to select a more accurate image by a subjective judgment in comparison with the left thumbnail images. In this case, a range of colorful rendering images were used (Fig. 3) to observe the performance when the images have more nat-
ural colors in real-world scenes. Figure 4 shows the preference and accuracy score chart. In both score charts, the interval scale along with the 99% confidence limits was generated using Thurston’s Law of Comparative Judgments, Case V [15]. In Fig. 4, other HDR rendering methods were used to evaluate the performance of the proposed method. The HDR images contain luminance levels that far exceed the luminance data that can be stored in 8 or 16 bpc (bits per channel) image files in Photoshop CS3. In addition, it is important to adjust the exposure and contrast when converting a 32 bpc HDR image to 8 or 16 bpc to produce an image with the proper dynamic range [16]. The EH (equalize histogram) compresses the dynamic range of the HDR image while trying to preserve some contrast, and the LA (local adaptation) adjusts the tonality in the HDR image by calculating the amount of correction necessary for local brightness regions throughout the image. The modified iCAM06 performed significantly better than the other algorithms. Overall, the rendering result images of the modified iCAM06 with the proposed separate color correction showed better performance for color accuracy than the other
KIM and LEE: SEPARATE COLOR CORRECTION FOR TONE COMPRESSION
1757
References
Fig. 4 Performance score charts; (a) preference score chart for CraterLake, (b) accuracy score chart for 6 test images by iCAM06, the proposed, EH (equalize histogram), and LA (local adaptation) in Photoshop CS3.
algorithms. 5.
Conclusions
The iCAM06, which is a typical HDR rendering algorithm, contains tone compression and many color space conversions during the stage for reproducing the HDR images. On the other hand, saturation reduction and hue shift occur through its compression and color conversion processes. The proposed color correction method does not need to regulate the hue and saturation separated from the luminance processing. The proposed method showed better performance than the default iCAM06, in terms of the rendering preference and color rendering accuracy. In addition, the color correction was achieved from color distortion of the output images during HDR rendering. The separate color correction method complements the tone compression using the photoreceptor responses. Moreover, this method is beneficial for applications without a post-color enhanced adjustment and the hue correction using the rotation matrix in the u∗ v∗ color space can be applied to the other tonal processes.
[1] E. Reinhard, G. Ward, S. Pattanaik, and P. Devevec, High Dynamic Range Imaging: Acquisition, Display and Image-Based Lighting, Morgan Kaufmann, San Francisco, 2005. [2] G.M. Johnson and M.D. Fairchild, “Rendering HDR images,” IS&T/SID 11th Color Imaging Conference, pp.36–41, Scottsdale, Arizona, 2003. [3] M.D. Fairchild and G.M. Johnson, “The iCAM framework for image appearance, image differences, and image quality,” J. Electronic Imaging, vol.13, pp.126–138, 2004. [4] J. Kuang, G.M. Johnson, and M.D. Fairchild, “iCAM06: A refined image appearance model for HDR image rendering,” J. Visual Communication and Image Representation, vol.18, no.5, pp.406–414, 2007. [5] J. Marguier and S. S¨usstrunk, “Color matching functions for a perceptually uniform RGB space,” ISCC/CIE Expert Symposium, Ottawa, Canada, 2006. [6] I. Lissner and P. Urban, “How perceptually uniform can a hue linear color space be?,” 18th Color and Imaging Conference: Color Science and Engineering Systems, Technologies, and Applications, vol.18, pp.97–102, Texas, San Antonio, 2010. [7] R. Mantiuk, A. Tomaszerska, and W. Heidrich, “Color correction for tone mapping,” EUROGRAPHICS 2009, vol.28, no.2, 2009. [8] S.M. Chae, S.H. Lee, H.J. Kwon, and K.I. Sohng, “A tone compression model for the compensation of white point shift generated from HDR rendering,” IEICE Trans. Fundamentals, vol.E95-A, no.8, pp.1297–1301, Aug. 2012. [9] N. Moroney, M.D. Fairchild, R.W. G. Hunt, C.J. LI, M.R. Luo, and T. Newman, “The CIECAM02 color appearance model,” IS&T/SID 10th Color Imaging Conference, pp.23–27, Scottsdale, Arizona, 2002. [10] J.M. Valeton and D.V. Norren, “Light adaptation of primate cones: An analysis based on extracellular data,” Vision Research, vol.23, pp.1539–1547, 1983. [11] R.W. G. Hunt, C.J. Li, and M.R. Luo, “Dynamic cone response functions for models of color appearance,” Color Research & Application, vol.28, pp.82–88, 2003. [12] M.D. Fairchild, Color Appearance Model, 2nd ed., Wiley–IS&T, Chichester, UK, 2005. [13] H.G. Kim, S.H. Lee, T.W. Bae, and K.I. Sohng, “Color saturation compensation in iCAM06 for high-chroma HDR imaging,” IEICE Trans. Fundamentals, vol.E94-A, no.11, pp.2353–2357, Nov. 2011. [14] A. Ford and A. Roberts, Colour Space Conversions, Technical report, www.poynton.com/PDFs/coloureq.pdf, 1998. [15] J. Morovic, Color Gamut Mapping, Wiley-IS&T, Hewlett-Packard Company, Barcelona, Spain, 2008. [16] Adobe Help Resource Center, Adobe Photoshop CS3 User Guide, http://help.adobe.com/archive/en US/photoshop/cs3/photoshop cs3 help.pdf
Hwi-Gang Kim received his B.S. and M.S. degrees in Electrical Engineering from Kyungpook National University in 2008 and 2010, respectively. Since 2011, he has been working at the National Institute for Mathematical Sciences as a research fellow. His research field includes color image processing, color appearance model, object tracking, and pattern recognition.
IEICE TRANS. FUNDAMENTALS, VOL.E96–A, NO.8 AUGUST 2013
1758
Sung-Hak Lee received his B.S., M.S., and Ph.D. degrees in Electrical Engineering from Kyungpook National University in 1997, 1999, and 2008 respectively. He worked at LG electronics from 1999 to 2004 as a senior research engineer. He currently works at the school of Electrical Engineering and Computer Science of Kyungpook National University as a research professor. His research field includes color management, color appearance model, color image processing, and display applications for human visual system.
1752
PAPER
Separate Color Correction for Tone Compression in HDR Image Rendering Hwi-Gang KIM† , Nonmember and Sung-Hak LEE††a) , Member
SUMMARY Many High-Dynamic-Range (HDR) rendering techniques have been developed. Of these, the image color appearance model, iCAM, is a typical HDR image rendering algorithm. HDR rendering methods normally require a tone compression process and include many color space transformations from the RGB signal of an input image to the RGB signal of output devices for the realistic depiction of a captured image. The iCAM06, which is a refined iCAM, also contains a tone compression step and several color space conversions for HDR image reproduction. On the other hand, the tone compression and frequent color space changes in the iCAM06 cause color distortion, such as a hue shift and saturation reduction of the output image. To solve these problems, this paper proposes a separate color correction method that has no effect on the output luminance values by controlling only the saturation and hue of the color attributes. The color saturation of the output image was compensated for using the compensation gain and the hue shift was corrected using the rotation matrix. The separate color correction method reduces the existing color changes in iCAM06. The compensation gain and rotation matrix for the color correction were formulated based on the relationship between the input and output tristimulus values through the tone compression. The experimental results show that the revised iCAM06 with the proposed method has better performance than the default iCAM06. key words: HDR image, iCAM06, tone compression, Hue, saturation
1.
Introduction
People are living in a luminance environment with the extensive dynamic range, and a wide range of absolute luminance levels is sometimes encountered. In addition, the human visual system (HVS) handles a wide luminance dynamic range easily. On the other hand, most available digital cameras and color display devices are limited in their dynamic range and cannot show a real scene in a single image [1]. Therefore, it is essential to render HDR images on the display devices with a lower dynamic range [2]. Consequently, HDR imaging techniques have been introduced and developed to take an image of a real wide luminance dynamic range in a single image. An HDR image can represent an image visually close to dynamic range of the visible world. It is created by merging several images of the same scene with different exposure values. The iCAM (image Color Appearance Model) makes it possible to represent the HDR images onto practical output devices. The iCAM06 is a recent refined model, in which several modules have been affected by preManuscript received December 14, 2012. Manuscript revised April 17, 2013. † The author is with the Division of Mathematical Models, National Institute for Mathematical Sciences, Korea. †† The author is with the School of Electronics Engineering, Kyungpook National University, Korea. a) E-mail: [email protected] DOI: 10.1587/transfun.E96.A.1752
vious algorithms, such as the local white adaptation, chromatic adaptation and IPT uniform color space conversion from the iCAM framework [3], [4]. In addition, iCAM06 incorporates some modified steps in that the single-scale Gaussian filtering is replaced with a two-layer image decomposition using the edge preserving bilateral filter and a simple non-linear local gamma correction is replaced with photoreceptor response functions. The iCAM06 also predicts a range of visual effects, such as a contrast change by the Stevens effect, colorfulness change by the Hunt effect, and an image contrast change in a complex image with surround luminance by the Bartleson-Breneman effect. Moreover, the iCAM06 extends to a wide range of luminance levels through the sum of scotopic and photopic vision signals [4]. In general, the dynamic range of an HDR image needs to be mapped to output devices, which is called tone mapping or tone compression, which is characterized on the photoreceptor curves of the human eyes. After tone compression, the color space conversions are normally necessary to adjust the color attributes. The iCAM06 causes saturation reduction by color clipping in the tone compression process and color signals do not maintain the hue linearity and exhibit a lack of perceptual color uniformity in some color space transformations [5]–[7]. In particular, in the case of an image including low chromatic regions, such as a snow and clouds scene, the rendered result image by iCAM06 is reddish all over the original image. This reddish phenomenon occurs near the white point in the results of iCAM06 [8]. The goal of iCAM06 is realistic image rendering that is adaptive to visual adaptation for a wide luminance level. In this regard, the color change is a weakness of the iCAM06. This paper proposes a color correction method that is separate from tone compression process by combining each saturation and hue correction. The separate color correction method preserves the compressed luminance condition of the output image and compensates for the color distortion in the rendered result images by iCAM06. First, the saturation compensation gain was obtained based on the ratio of the calculated saturation values in the tone-compressed image and the original image for each and the new tristimulus values for saturation correction was calculated. Second, the hue angle shifted by frequent color space conversions was calculated, and the hue angle was set to the previous original hue angle using the rotation matrix. Finally, some objective and subjective experiments were performed to confirm the performance of the proposed method. The experimen-
c 2013 The Institute of Electronics, Information and Communication Engineers Copyright
KIM and LEE: SEPARATE COLOR CORRECTION FOR TONE COMPRESSION
1753
tal results show that the modified iCAM06 with the separate color correction method has better performance in terms of the output color accuracy and individual preference for test images. In addition, this method can be applied easily to other algorithms for color correction. Section 2 describes the framework of iCAM06 and explains briefly the several processes in iCAM06 including the color space transformations. The proposed method and experimental results is discussed in Sects. 3 and 4, respectively. Section 5 reports the conclusions. 2.
Previous Works
2.1 Tone Compression in the iCAM06 The purposes of iCAM06 were to predict accurately the human visual attributes of complex images in a wide range of luminance levels and to display the same human visual perception across media [4]. The iCAM06 represents a wide luminance range using tone compression functions based on the scotopic and photopic visual responses. The nonlinear compression is designed according to the photoreceptor responses including cones and rods. The tone compression functions, as shown in Eqs. (1)–(5), are similar to those of CIECAM02 [9]. On the other hand, they include a slightly modified power value p. The user controllable variable p accounts for the steepness of the photoreceptor response curves. 400(F L R /YW ) p + 0.1 27.13 + (F L R /YW ) p 400(F L G /YW ) p Ga = + 0.1 27.13 + (F L G /YW ) p 400(F L B /YW ) p Ba = + 0.1 27.13 + (F L B /YW ) p F L = 0.2k4 (5LA ) + 0.1(1 − k4 )2 (5LA )1/3 k = 1/(5LA + 1) Ra =
(1) (2) (3) (4) (5)
where RG B and RG Ba are the cone response before and after tone compression under the chromatic adaptation of D65 illumination, respectively. LA is the adapting luminance taken to be 20% of the luminance of a white object in the scene. YW is the luminance of a local adapted white image. The non-linear sigmoid functions are used in the iCAM06 for tone compression. The above functions are based on a generalized Michaelis-Menten Equation and are consistent with the Valeton and van Norren’s experimental data [10]. Hunt [11] discussed the specifics and advantages of the equations. The F L function is used to predict a variety of the luminance-dependent appearance effects. Finally, the tone compression response is the sum of the cones and rods response, but the rods response has very little impact [4]. 2.2 Color Space Transformation for Image Adjustment The tone compressed signals were converted into IPT color space. The IPT uniform opponent color space is useful
for color attribute adjustments without unwanted image artifacts. Three visual attributes, such as lightness, hue and chroma, were adjusted to predict the effective color appearance effects. The details adjustment in the detail-layer processing was applied to predict the Stevens effect that increases the perceived lightness contrast with increasing luminance. In the IPT color space, P and T were adjusted to predict the Hunt effect that increases the colorfulness with increasing luminance for an accurate image appearance. To display the processed image on the output devices, The IPT values were converted back to XYZ values by the inverted chromatic adaptation transformation from the CIE standard illuminant, D65 , to the output device white point [4]. The inverse output characterization model was used to transform the XYZ values to the device dependent RGB values. For clear visibility, any extremely dark or bright pixel data were cut down by the clipping function [2]. The clipped data were limited to the range of the 1st and 99th percentile of the processed data as follows: ⎧ ⎪ RGB, for RGB1% ≤ RGB ≤ RGB99% ⎪ ⎪ ⎨ RGB , for RGB99% < RGB clip : RGBclip = ⎪ 99% ⎪ ⎪ ⎩RGB , for RGB < RGB 1%
1%
(6) The clipped RGB data were normalized to Eq. (7), and modulated with the power function. The output values, RGBout , were rescaled according to the output values from 0 to 255, as shown Eq. (8). RGBn =
RGBclip − (RGBclip )min (RGBclip )max − (RGBclip )min
RGBout = 255(RGBn )1/1.7
(7) (8)
The HDR rendering process in iCAM06 includes color conversions for image adjustment. These are the main reason for color distortion. That is, the color shift of the tone compression step can enlarge the color error in a display terminal [8]. 2.3 Saturation Compensation in Tone Compression The color saturation is a unique perceptual experience separated from chroma. The definition of color saturation is the colorfulness of a judged area in proportion to its brightness. Remarkably, under viewing conditions with the luminance level within photopic vision, the color stimulus of the chromaticity shows similar saturation for all luminance levels, except when the brightness is quite high [12]. However, the iCAM06 causes a decrease in saturation by the color clipping during its tone compression process. The default iCAM06 does not consider saturation compensation. Therefore, it is important to add a color saturation compensation step to solve the under-saturation problem [13]. Generally the CIE 1976 u v saturation is calculated from the following equation: Satuv = 13 (u − un )2 + (v − vn )2 (9)
IEICE TRANS. FUNDAMENTALS, VOL.E96–A, NO.8 AUGUST 2013
1754
where un and vn are the chromaticity coordinates of the reference white point. The attenuation factor, α, was obtained from the ratio of the calculated saturation values in the tone compressed image and the original image for each [13]. The factor, α, and the compensated saturation value, Sat sc , were calculated as follows: Satc (10) Sat sc = α Satc α= , Satin > 0 (11) Satin where Satc and Satin were calculated from the CIE 1976 u v saturation, which are the color saturation values of the tone compressed signal, RG B a , and the non- compressed input signal, RG B , respectively. The attenuation factor, α, was multiplied inversely by the compressed color saturation value. The compensated chromaticity coordinates (usc , vsc ) were calculated from Sat sc as follows: usc = Sat sc · cos θ + un vsc = Sat sc · sin θ + vn vc − vn θ = arctan uc − un
(12) (13) (14)
The calculated usc , vsc were transformed to the corrected tristimulus XYZ sc and the new tristimulus were transformed to the IPT uniform opponent color space, as shown in Sect. 2.2. 3.
Separate Color Correction for Color Preserving
Despite the additional saturation compensation step as described in Sect. 2.3, iCAM06 still has a hue shift problem. Therefore, this paper proposes the completed color correction method including the hue compensation and previous saturation compensation. This method, first, calculated the saturation compensation gain similar to Eq. (10), which was obtained from the ratio of the calculated saturation values in the tone compressed image and the original image. The saturation compensation gain, S g , and the saturationcompensated chromaticity coordinates (usc , vsc ) were calculated as follows: Satin (15) Sg = Satc usc = S g (uc − un ) + un (16) (17) v sc = S g (vc − vn ) + vn In these equations, unlike the equations in Sect. 2.3, the computation was simplified for the saturation compensation without trigonometric functions, which require their polarity conditions to select positive or negative values depending on the angle range. In addition, the modified compensation equations are more suitable for the gain control of colors around a reference white point compared to the previous equations, which can change colors incorrectly when
changing polarity conditions. Then the hue shift correction procedure was added, which could return the shifted hue angle to the original hue angle using a rotation matrix. The hue angle was calculated using Eq. (18) in the CIE 1976 L∗ u∗ v∗ color space [14]. huv = arctan
v∗ u∗
(18)
The CIELUV∗ is defined by following equations: L∗ = 116(Y/Yn )1/3 − 16 u∗ = 13L∗ (u − un )1/3 v∗ = 13L∗ (v − vn )1/3
(19) (20) (21)
where L∗ is the lightness, u∗ is the redness-greenness, and v∗ is the yellowness-blueness. Y is the white stimulus of the tristimulus XYZ value and Yn is the reference white [14]. This process handles the u∗ v∗ coordinate points for the hue shift correction. Therefore, this method does not affect the luminance of the output images. The hue angle difference was obtained by comparing the two hue angles from the (u , v ) of the non-shifted input signal and the (usc , vsc ) of the shifted output signal at the previous work and the difference angle φd between the two chromaticity coordinate points can be found by subtracting the two angles as follows: φ sc = h sc = arctan(v∗sc /u∗sc ) φin = hin = arctan(v∗in /u∗in ) φd = φ sc − φin
(22) (23) (24)
where φ sc (h sc ) and φin (hin ) mean the hue angles of the shifted output signal and the non-shifted input signal respectively. The shifted coordinates (usc , vsc ) in the previous compensation step are revolved around the white point by the difference angle, φd , between the two points using the rotation matrix as follows:
uhc u sc = (φd ) (25) vhc vsc
cos φd sin φd (φd ) = (26) − sin φd cos φd where (usc , vsc ) means the final correction coordinates. The complete color correction incorporates the saturation compensation and hue correction method. The corrected (uhc , vhc ) were transformed to the corrected tristimulus XYZ hc . In subsequent processes, there is no need for any additional color adjustments in iCAM06 because the lower colorfulness effect due to under saturation is compensated for by proposed color correction process. The remaining procedure is the same as the existing iCAM06 from the inverse chromatic adaptation transformation to the output RGB. Figure 1(a) presents the overall flowchart of the modified iCAM06 including the completed separate color correction step. The separate color correction makes it possible to skip the post-color adjustments after the tone compression
KIM and LEE: SEPARATE COLOR CORRECTION FOR TONE COMPRESSION
1755
Fig. 1 Flowchart of the modified iCAM06 by the separate color correction: (a) Modified iCAM06 including the proposed method, (b) separate color correction.
in iCAM06. Figure 1(b) shows the magnified block diagram of the proposed color correction step. The proposed method maintains the image luminance with the same Yhc and Yc . 4.
Simulation Results
4.1 Objective Assessment The proposed method was simulated to confirm the performance and compare the results with the previous algorithms. The change in the hue angle with the variation of color intensity was confirmed. Here, the color patches with the same hue and different saturation were used, in which the color intensity changes at regular intervals. The used 7 colors are red, green, blue, cyan, magenta, yellow, and purple by the RGB tristimulus combination. The multiple scatter and lines for the selected colors are drawn on the coordinates CIE 1976 u v , as shown in Fig. 2. The gray points and lines mean the original input placed on the same hue trajectory for the change in color intensity. The color points and lines mean the output values. Figure 2 confirmed that the hue shift of each color was much less than that of iCAM06. This result suggests that the altered hue angles were restored close to the original hue angles using the proposed method. The modified iCAM06 maintain the perceptual color uniformity of the rendered output image. In addition, the output color gamut is also considerably wider than the existing iCAM06. Table 1 lists the calculated color differences for each color sample from both diagrams shown in Fig. 2.
Fig. 2 Comparison between the input and rendered output of samples on the CIE 1976 u v chromaticity diagrams: (a) The iCAM06, (b) the modified. Table 1
Color difference errors for the selected colors.
4.2 Subjective Assessment Subjective experiments were conducted to confirm the visual performance of the color reproduction. Pair comparison tests were conducted to evaluate the rendering performance in two ways; the individual preference evaluation and color accuracy scoring to original scenes. The preference evaluation was conducted just by comparing the rendered results between the iCAM06 and the proposed, and the accuracy scoring method counts the color matching for the originals before merging an HDR image. Figure 3 provides examples of the images for the comparison tests. Thumbnail images
IEICE TRANS. FUNDAMENTALS, VOL.E96–A, NO.8 AUGUST 2013
1756
Fig. 3 Examples of the images used for the comparison test; CraterLake, 507, LasVegas Store, LabBooth in a clockwise direction.
are placed in the left column and red circled parts on the pictures are for the effective color accuracy comparison. The rendering results were placed top and bottom in the right columns. The experiments were carried out with 24 people with normal color vision. The rendered images were displayed on a 27 DELL U2711 Monitor. First, in the individual preference test, the participants were asked to select the image they prefer by a subjective judgment between a pair of rendered images. In this time, especially, the easiest picture was selected to examine the hue correction effect among the various rendered result images. The top-left image, named CraterLake, includes a wide low chromatic field like snow and sky. The top image by iCAM06 shows an overall reddish colored image. The bottom image by the proposed color correction shows that the color of snow and sky is closer to the actual color. Next, in the color accuracy test, the participants were asked to select a more accurate image by a subjective judgment in comparison with the left thumbnail images. In this case, a range of colorful rendering images were used (Fig. 3) to observe the performance when the images have more nat-
ural colors in real-world scenes. Figure 4 shows the preference and accuracy score chart. In both score charts, the interval scale along with the 99% confidence limits was generated using Thurston’s Law of Comparative Judgments, Case V [15]. In Fig. 4, other HDR rendering methods were used to evaluate the performance of the proposed method. The HDR images contain luminance levels that far exceed the luminance data that can be stored in 8 or 16 bpc (bits per channel) image files in Photoshop CS3. In addition, it is important to adjust the exposure and contrast when converting a 32 bpc HDR image to 8 or 16 bpc to produce an image with the proper dynamic range [16]. The EH (equalize histogram) compresses the dynamic range of the HDR image while trying to preserve some contrast, and the LA (local adaptation) adjusts the tonality in the HDR image by calculating the amount of correction necessary for local brightness regions throughout the image. The modified iCAM06 performed significantly better than the other algorithms. Overall, the rendering result images of the modified iCAM06 with the proposed separate color correction showed better performance for color accuracy than the other
KIM and LEE: SEPARATE COLOR CORRECTION FOR TONE COMPRESSION
1757
References
Fig. 4 Performance score charts; (a) preference score chart for CraterLake, (b) accuracy score chart for 6 test images by iCAM06, the proposed, EH (equalize histogram), and LA (local adaptation) in Photoshop CS3.
algorithms. 5.
Conclusions
The iCAM06, which is a typical HDR rendering algorithm, contains tone compression and many color space conversions during the stage for reproducing the HDR images. On the other hand, saturation reduction and hue shift occur through its compression and color conversion processes. The proposed color correction method does not need to regulate the hue and saturation separated from the luminance processing. The proposed method showed better performance than the default iCAM06, in terms of the rendering preference and color rendering accuracy. In addition, the color correction was achieved from color distortion of the output images during HDR rendering. The separate color correction method complements the tone compression using the photoreceptor responses. Moreover, this method is beneficial for applications without a post-color enhanced adjustment and the hue correction using the rotation matrix in the u∗ v∗ color space can be applied to the other tonal processes.
[1] E. Reinhard, G. Ward, S. Pattanaik, and P. Devevec, High Dynamic Range Imaging: Acquisition, Display and Image-Based Lighting, Morgan Kaufmann, San Francisco, 2005. [2] G.M. Johnson and M.D. Fairchild, “Rendering HDR images,” IS&T/SID 11th Color Imaging Conference, pp.36–41, Scottsdale, Arizona, 2003. [3] M.D. Fairchild and G.M. Johnson, “The iCAM framework for image appearance, image differences, and image quality,” J. Electronic Imaging, vol.13, pp.126–138, 2004. [4] J. Kuang, G.M. Johnson, and M.D. Fairchild, “iCAM06: A refined image appearance model for HDR image rendering,” J. Visual Communication and Image Representation, vol.18, no.5, pp.406–414, 2007. [5] J. Marguier and S. S¨usstrunk, “Color matching functions for a perceptually uniform RGB space,” ISCC/CIE Expert Symposium, Ottawa, Canada, 2006. [6] I. Lissner and P. Urban, “How perceptually uniform can a hue linear color space be?,” 18th Color and Imaging Conference: Color Science and Engineering Systems, Technologies, and Applications, vol.18, pp.97–102, Texas, San Antonio, 2010. [7] R. Mantiuk, A. Tomaszerska, and W. Heidrich, “Color correction for tone mapping,” EUROGRAPHICS 2009, vol.28, no.2, 2009. [8] S.M. Chae, S.H. Lee, H.J. Kwon, and K.I. Sohng, “A tone compression model for the compensation of white point shift generated from HDR rendering,” IEICE Trans. Fundamentals, vol.E95-A, no.8, pp.1297–1301, Aug. 2012. [9] N. Moroney, M.D. Fairchild, R.W. G. Hunt, C.J. LI, M.R. Luo, and T. Newman, “The CIECAM02 color appearance model,” IS&T/SID 10th Color Imaging Conference, pp.23–27, Scottsdale, Arizona, 2002. [10] J.M. Valeton and D.V. Norren, “Light adaptation of primate cones: An analysis based on extracellular data,” Vision Research, vol.23, pp.1539–1547, 1983. [11] R.W. G. Hunt, C.J. Li, and M.R. Luo, “Dynamic cone response functions for models of color appearance,” Color Research & Application, vol.28, pp.82–88, 2003. [12] M.D. Fairchild, Color Appearance Model, 2nd ed., Wiley–IS&T, Chichester, UK, 2005. [13] H.G. Kim, S.H. Lee, T.W. Bae, and K.I. Sohng, “Color saturation compensation in iCAM06 for high-chroma HDR imaging,” IEICE Trans. Fundamentals, vol.E94-A, no.11, pp.2353–2357, Nov. 2011. [14] A. Ford and A. Roberts, Colour Space Conversions, Technical report, www.poynton.com/PDFs/coloureq.pdf, 1998. [15] J. Morovic, Color Gamut Mapping, Wiley-IS&T, Hewlett-Packard Company, Barcelona, Spain, 2008. [16] Adobe Help Resource Center, Adobe Photoshop CS3 User Guide, http://help.adobe.com/archive/en US/photoshop/cs3/photoshop cs3 help.pdf
Hwi-Gang Kim received his B.S. and M.S. degrees in Electrical Engineering from Kyungpook National University in 2008 and 2010, respectively. Since 2011, he has been working at the National Institute for Mathematical Sciences as a research fellow. His research field includes color image processing, color appearance model, object tracking, and pattern recognition.
IEICE TRANS. FUNDAMENTALS, VOL.E96–A, NO.8 AUGUST 2013
1758
Sung-Hak Lee received his B.S., M.S., and Ph.D. degrees in Electrical Engineering from Kyungpook National University in 1997, 1999, and 2008 respectively. He worked at LG electronics from 1999 to 2004 as a senior research engineer. He currently works at the school of Electrical Engineering and Computer Science of Kyungpook National University as a research professor. His research field includes color management, color appearance model, color image processing, and display applications for human visual system.
