hsv-domain enhancement of high contrast images - Universidad de ...
HSV-DOMAIN ENHANCEMENT OF HIGH CONTRAST IMAGES power laws and unsharp masking for bounded and circular signals
Alfredo Restrepo (Palacios), Stefano Marsi and Giovanni Ramponi DEEI, University of Trieste, Via Valerio 10, Trieste, Italy [email protected], [email protected], [email protected]
Keywords:
Tone mappings, gamma correction, unsharp masking, circular processing, color enhancement.
Abstract:
We present techniques for performing luminance mapping and unsharp-masking-like processing on digital images; they are specifically designed for their application on bounded and on circular signals. We study the effect of gamma correction on contrast and develop an adaptive technique for luminance and contrast correction. We use the proposed tools for the enhancement of high contrast color images.
1
INTRODUCTION
The luminance component of an image is normally coded in the linearly-ordered and bounded interval [0, 1]. For static scenes, the method of (Debevek and Malik, 1997) allows the capture of high luminance range scenes in the form of High Dynamic Range Radiance Maps (HDR images, for short); the radiance is unbounded in principle and needs to be mapped to a luminance component. Given the relatively low dynamic range of visualization devices, it is not possible to literally display HDR images; nevertheless, the display of high contrast (HC) images does benefit from the use of local contrast amplification, as it improves their quality and readability; image contrast results both from luminance contrast and from hue contrast. By an HC image it is meant an image (that typically corresponds to a high radiance range scene) with a luminance histogram having modes near 0 and near 1, corresponding to very light and very dark regions in the image; such regions normally contain (perhaps unnoticeable) subregions at small contrast. Large contrast is likely to be observable in any type of image, also in HC images. Small contrast, even though present and probably observable in the original scene, may not be noticeable. The luminance component of an HC image usually can be meaningfully segmented into light and dark regions and we base a correction of luminance and of color saturation on such a segmentation. If
realism is a goal, the increase of color saturation at light regions should be somewhat smaller than that at regions of intermediate luminance; similarly, increments of luminance should be smaller at light regions. Depending on the scene, other things could be taken into account; e.g. the illumination in shadows under a sunny sky tends to be bluer than direct sun light which is relatively yellower; also, the Bezold-Brucke effect predicts a shift towards yellow of oranges and cetrines, and a shift towards blue of cyans and purples, when the intensity of the illumination increases. In addition to the topics of logarithms and saturating nonlinearities, a recurrent topic in the subject of tone mapping is power laws; consider for example the bias function used by Drago (Drago and Chiba, 1997) and the use of power laws made in (Tumblin and Hodgins, 1999). We consider here a variant approach in the use of power laws; also, we use power laws for color saturation enhancement. The enhancement of contrast for the visualization of HC images is a topic of actuality. Meylan and Susstrunk (Meylan and Susstrunk, 2006) locally enhance contrast within a retinex framework by computing a local luminance using a mask resulting from the convolution of the image and a Gaussian surround function. (Drago and Chiba, 1997) use a logarithmic law where the base of the logaithm is made dependant on a power law of the pixelwise (rather than local) luminance. In Section 2, we propose a method for mapping
the luminance of an HDR image to the interval [0,1]. In Sections 3 and 4, we present techniques for the enhancement of small local contrast of the (bounded) luminance component. In Section 5.1, we present a technique for the enhancement of local contrast for the (circular) hue variable and in Section 5.2 we present an adaptive technique that works both on the luminance and saturation components and that improves the readability of HC images.
2
ON THE RENDITION OF HDR IMAGES
Even though there is no consensus regarding a formula for the conversion of radiance into luminance, there is an agreement that it should be approximately logarithmic. As (Drago and Chiba, 1997) point out, both the logarithmic Weber-Fechner law and Stevens’ power law, are relatively close in practice. The following formula, maps the radiance r of a given image onto the [0, 1]-coded luminance X by computing Y=log(r) and the min (which typically is negative) and range (i.e. the max minus the min) of Y. X = (Y − min)/range
(1)
In this way, the minimal radiance is mapped to zero (“black”) and the largest one to one (“white”). We assume that the minimal radiance is positive, even if small; this is a valid assumption so long as there are no “black body radiators at 0 K” in the scene and the image is registered at a varied enough set of speeds. As an example, consider the result of the application of the method to the HDR image memorial.hdr (r was calculated as 0.3R+0.6G+0.1B), shown in Figure 1. Tumblin and Hodgins (Tumblin and Hodgins, 1999) proposed a set of desired properties that a compression method should meet, namely, progressiveness (large contrast is more compressed than small contrast), monotonicity (small luminances remain smaller than large luminances), symmetry (very large and very light regions are affected equally), asymptoticity (an infinite range is compressed into a finite one), minimality (scene contrast is compressed only just enough to fit display contrast) and adjustabbility (suits viewer preferences); of these, for the [0, 1] rendition of HDR images, symmetry is the hardest to meet because even though the minimal radiance is in practice near zero, the largest one can vary widely. To add adjustability to the method in Equation 1, the radiance r can be raised to a power before computing Y.
Figure 1: (logR+7.2)/13, applied to the radiance of memorial.hdr
2.1
Naka-Rushton saturating nonlinearities
In addition to logarithmic nonlinearities, other types of nonlinearity have been considered for the rendering of HDR images; in particular, those suspected to play a role in the retina of vertebrates (Meylan et al., 2007). (Naka and Rushton, 1966) proposed the nonlinearity V = 0.5(1 + tanh(x − x0 ))
(2)
(where we have added the normalizing factor 0.5) to model the S-potential response to flashes of monochromatic light of (probably bipolar) retinal cells, where the variable x represents the log(I) of the intensity I; see Fig. 3. In terms of I, the nonlinearity 2 becomes: V = 0.5(1 + tanh(log( II0 ))) = I 2I+I 2 which 0
is closely related to the other nonlinearity they proJ pose: V0 = J+1 , see Figure 2, better known in the form x y= (3) x + x0
3
ON GAMMA CORRECTION AND CONTRAST ENHANCEMENT
The difference between the luminances of two neighboring pixels measures local contrast. In relation to the [0, 1]-coded luminance space, the (unsigned) contrast can be large or small; by small local contrast we
Figure 2: The Naka-Rushton saturating formula for parameter values x0 = 0.2, 0.5, 1, 2, 3.
Figure 3: The Naka-Rushton saturating formula of the second type, 0.5(1 +tanh(x − x0 )), for parameter values x0 = 3, 4, 5, 6, 7.
mean window-neighbor pixels (i.e. pixels that fit in a single window) differing in luminance by a small amount, e.g. less than 0.1; by large local contrast we mean neighbor pixels differing in luminance by a large amount, e.g. more than 0.3; we are concerned here with small local contrast. The derivative of a luminance map measures its effect on small contrast at different luminances. We define the small contrast amplification (SCA) of a tone map as the value of the derivative of the mapping evaluated at the local luminance (which is measured using a location statistic such as the average, the median, the midrange (i.e. the average of the min and the max) or a weighted combination of these) of the window (called also the receptive field) in question. In general, a [0, 1] → [0, 1] luminance map has different effects on small local contrast at different luminances; for the mapping technique known as gamma correction, the luminance of each pixel is raised to a positive power. Such powerlaw techniques are usually applied using the same exponent for each pixel, and also in a pointwise fashion in the sense that the resulting luminance of the pixel depends on the luminance of the pixel alone. By using an exponent g larger (resp. lower) than one an image becomes darker (resp. lightes); this effect has been
exploited in the restoration of faded prints (Restrepo and Ramponi, 2008b); see Figure 4; by looking at the derivative of the mapping it is seen that small local contrast is improved where it matters, i.e. in the luminance range where the image is originally heavily represented; see Figure 5. Thus, it is conceivable to use an adaptive power law approach to improve the contrast of images, regardless of their luminance histogram; there is a caveat though as we show below. Regarding the use of exponent laws that depend locally on the luminance, we consider two approaches. In the first approach, we keep the SCA constant and equal to one. It is seen that the resulting effect on the luminance is monotonically nonincreasing: a nonaesthetic effect since it reverses nonlocal large contrast. In the second approach the SCA is maximized at each luminance; it will be seen that the output luminance becomes constant, rendering the method, as such, useless. A variant approach that renders the second approach useful is obtained by making the map a particular case of a parametrized family of mappings where the parameter can be made to depend on local contrast, which may be measured with e.g with the range (the max minus the min), as done e.g. in (Restrepo and Ramponi, 2008a).
Figure 4: The effect of gamma correction Lg at different luminances L in[0, 1], at different exponents g = 0.2, 0.5, 1, 2, 4: compare with the curve g=1. Lg > L if g < 1, and viceversa
3.1
An adaptive power law for constant SCA: a negative results
Constant SCA using a variant power law tone map is achieved as follows. The derivative of the function G(L) := Lg is given by G0 (L) = gLg−1 ; solving for L 1
the equation G0 (L) = 1 we get L1 (g) = ( g1 ) g−1 , see Figure 6. Call the inverse of this function g1 (L); we believe it has no closed form expression. The resulting adaptive power law is given by AG1 (L) = Lg1 (l) , where l is the local luminance l measured with an alpha blend of the pointwise luminance and the median
3.2
Maximal SCA
Consider now the second approach; it uses an exponent that maximizes the SCA at each possible local d2 luminance. From dgdL Lg = 0 one gets the desired ex1 ponent g2 (L) = − log(L) . Again, for a pointwise local
Figure 5: Derivatives of Lg for g = 0.2. 0.5, 0.7, 1, 2, 3, 4, 8. Note the coresponding regions where the derivative has high values.
luminance, AG2 (L) given by Lg2 (L) is not useful: it is the constant function 1e , see in Figure 8 the curve with parameter K=1. The maps shown in the figure darken pixels for luminances above 1e , and viceversa. We consider this a desirable property of tone maps for HC images since in such images, both dark and light gray levels tend to get clustered together.
of the luminances in the receptive field. In an extreme case (K=1 in Figure 7) where the local luminance is pointwise, l = L, AG1 (L) is given by Lg1 (L) is a decreasing function; this case does not have aesthetic applications as it reverses the luminances in the image; nevertheless, it may be of interest, e.g. in product inspection, to have an image where the contrast has been uniformized.
Figure 8: The function L1+K(g2 (L)−1) ; K = 0.1, 0.3, 0.5, 0.7 and 1.0 (indicated on the right). AG2 (L) > L if L < 1e and viceversa; 1e is a fixed point for each K.
3.3 Figure 6: L1 (1) =
1
Unitary-slope luminance L1 (g) = ( g1 ) g−1 ;
1 e
Figure 7: The function L1+K(g1 (L)−1) ; K = 0.1, ... 1.0. Luminance intervals of negative slope get reversed: monotonicity is lost.
Modulation of the technique using local contrast and local luminance
In addition to the use of a local luminance l, besides the pointwise luminance L, local contrast can be used to obtain useful versions of the methods described in the two previous subsections. As already suggested in Figure 8, consider the use of the map AGB2 (L) = L1+K(g2 (l)−1) where the parameter K ∈ [0, 1] is a function of the local contrast c. At one extreme, K=1, we have AGB2 (L) = AG2 (L) = Lg2 (l) already considered; at the other extreme, K=0, we have the do-nothing case AGB2 (L) = L1 . From Figure 7 we conclude that g1 probably needs a special dependency of K on the local contrast; the use of g2 gives more straightforward results. The parameter K is made dependant on the local contrast c as measured e.g. by the range of the luminances in a window centered at each pixel; in Figure 9, K(c)=c was used. The use local contrast and local luminance renders the method of the variant type (Devlin, 2002).
functions L−l ), L ∈ [0, l] (4) l L−l ), L ∈ [l, 1] (5) fl (L) = L + h2 ( 1−l where the functions h have infinite slope at x=0 and are given by 1 −c l h1 (x) = (6) (−x) 2n (1 + x), x ∈ [−1, 0] MAX 3 1 c 1−l h2 (x) = (x) 2n (1 − x), x ∈ [0, 1] (7) MAX 3 where the positive integer n controls the behaviour 1 1 2n ( 2n+1 ) 2n near the point of infinite slope, MAX = 2n+1 and c ∈ [0, 1] is a constant that can be used for tuning the desired amount of masking. Clearly, the funcfl (L) = L + h1 (
Figure 9: Application of AGB2 , K(c) = c; l = 0.5L + 0.5median; c=0.5rng+0.5qsrng to image in Figure 1
.
4
UNSHARP MASKING FOR BOUNDED SIGNALS
Taking inspiration from the maps shown in Figures 8 and 7, we decided to directly design one such map family, parametrized by local luminance, without assuming any underlying map, such as that of gamma correction. Each tone map in the family of maps fl : [0, 1] → [0, 1] should be increasing and meet f(0)=0, f(1)=1. Then, for each pixel luminance L, depending on the local luminance, one fl of the family of tone maps is applied, and the corrected luminance fl (L) results. The chosen map fl (L) has a slope larger than one for pixel luminances L near the local luminance l; in this way, small contrast is amplified, that is, the variations of luminance near the luminance of the receptive field are enlarged. Each map fl : [0, 1] → [0, 1] in the family is a continuous function with fL (L) = L, so that it fixes local luminance; also, it has a convex derivative fl0 and fl0 (L) has a maximum at l = L, locally increasing small contrast for luminances L near the local luminance l. We also impose the requirement that the derivative be positive (i.e. that the function be strictly increasing); this ensures that luminances (even if far off the local luminance of the field) do not become saturated near 0, 1 or any intermediate level. The methodology can be catalogued as one of unsharp masking, even if of a special type, specifically designed to be applied to signals that live in the interval [0, 1]. In particular, consider the following family of
Figure 10: c=0.5, n=5, l=0.6
tions h go to 0 in a finite interval. The technique is of the unsharp masking type whenever the computation of the local luminance l involves an operation of the smoothing type, as it is usually the case. The difference between the signal and the smoothed version is amplified accordingly. Figure 11 shows an example of the application of the technique to the image in Figure 1.
5
PROCESSING OF THE CHROMA COMPONENTS
We process the chroma components of the image as well; in the HSV color system, the saturation is a linear bounded variable and the hue is a circular variable (Restrepo et al., 2008b). We define a class of circular unsharp masking maps and apply them to the hue component for enhancing fine detail of color images; the proposed technique compares favorably with that in (Restrepo et al., 2008a). The saturation component of HC images is enhanced using gamma correction; saturation, being a bounded variable, is amenable to gamma-correction enhancement.
contrast is enhanced, i.e. the variations of hue near the local hue of the field are enlarged. First, define a continuous, increasing function f : [0, 2P] → [0, 2P] with f (0) = 0, f (2P) = 2P, f (P) = P such that the derivative of f is maximal at P. For example, consider the map f : [0, 2P] → [0, 2P] with infinite slope at P, given by 1
f (x) = x − cx(P − x) 2n , x ∈ [0, P] 1
f (x) = x + c(2P − x)(x − P) 2n , x ∈ [P, 2P]
(8) (9)
where the constant c ensures that f’ is positive everywhere. With one such function f already defined,
Figure 11: The image in Figure 1, after the application of bounded unsharp masking; c=0.3, n=1
5.1
Unsharp masking for circular signals
The angular hue variable of the HSV color system measures 0 degrees for R (red), 120 degrees for G (green) and 240 degrees for and B (blue); nevertheless, it can be argued that, yellow being a nonbinary color, the perceptual difference between red and yellow and that between yellow and green, are the same as those between green and blue, and blue and red. Thus, we transform the hue variable h of the HSV system to a modified hue H, also circular, as follows. Denoting the angle pi with the letter P, for angles h between 0 and (2/3)P, put H=(3/2)h while for angles h between (2/3)P and 2P, put H=(3/4)h + (1/4)2P. It is also convenient to denote each angle x with the complex number e jx ; et S1 = {z ∈ C : |z| = 1} be the unit circle of the complex plane, centered at the origin, 1 1 3 where e j0 , e j2P 4 , e j2P 2 , e j2P 4 represent the basic hues red, yellow, green and blue, respectively, and where the binary hues are correspondingly in between, e.g. the oranges are of the form e jw with 0 < w < P2 . With the purpose of enhancing the hue contrast of images, the hue H of each pixel is mapped to a hue mt (H), where the function m depends on the local hue t, which is computed using e.g. the circular average or a circular median, (when they exist, see (Mardia and Jupp, 2000) and (Restrepo et al., 2008b)) of the hues in the receptive field. For the examples given here, the circular mean was used. The map mt (H) has a “slope” larger than one for values of H near the local hue t of the receptive field; in this way, small
Figure 12: n=1, c=.3; consider the rectangle [0, 2P]×[0, 2P] as a torus split along a meridian and a longitude.
for each t, let mt : S1 → S1 be the function given by j f (∠ e
jH
)
−e jt where ∠z stands for the angle mt (H) = −e jt e of the complex number z. This effectively implements a circular map, analogous to the identity i : S1 → S1 , (its graph lives on the torus S1 × S1 ) but with derivative larger than one near t. It fixes local hue: mt (t) = t; also, it has a convex derivative mt0 ; mt0 (H), maximal at H = t, and minimal and smaller than one, at the opposed hue, it locally increases small contrast, for hues H near t. A requirement that the derivative be positive avoids the clustering of hues. As in the previous section, the methodology presented unsharp masking type.
5.2
Saturation enhancement using power laws
The enhancement of the saturation is made using a power law. Consider the enhancement of the HC image shown in Figure 17, resulting from a picture being taken inside a room with a window, at daylight. Since it can be argued that the corrections needed depend on the level of radiance coming from each part in the 3D scene, rather than making corrections to the image on the basis of pointwise luminance, we segment the image using a standard region-growing routine (Kroon,
Figure 16: Chroma processing of the image in Figure 15; saturation processing with g = 0.7 and hue processing with c=0.3, n=1. Figure 13: Market Image.
Figure 17: An image of high contrast, resulting from a high dynamic radiance scene.
2008) into light and dark regions and apply a correction that depends on the region. See Figure 18. We
Figure 14: The hue component of Market image has been processed using circular unsharp masking with n = 1, c = 0.3. Even though only the hue component has been processed, the image is crispier.
Figure 18: The image is segmented into light and dark regions.
Figure 15: Image of a faded tapestry.
use power laws on the saturation and value conmponents of the image. Regarding the saturation component S, we used the exponents 0.95 and 0.7 for the clear and dark regions, respectively; regarding the luminance component V, we applied no correction and the exponent 0.5 in the light and dark regions, respectively. See the result in Figure 19. There seems to be no reason to decrease the saturation at any part of the image.
ACKNOWLEDGEMENTS This work was partially supported by the FIRB project no. RBNE039LLC and by a grant of the University of Trieste. The Ancient tapestry of Figure 15 belongs to the Museo Civico Sartorium of Trieste. A. Restrepo is on leave of absence from the dpt. de Ing. Electrica y Electronica, Universidad de los Andes, Bogota, Colombia, ([email protected]). Figure 19: The image in Figure 17, after adaptive saturation and value enhancement.
6
CONCLUSION
Just as it is useful to increase the contrast of signals of a (linear) bounded range, it is of value to have a tool for increasing the contrast of signals with a circular range. In addition to presenting a method for the rendering of (unbounded) HDR images to the [0, 1] luminance interval, a set of tools for luminance correction, luminance unsharp masking, hue unsharp masking and saturation enhancement, has been presented. Applications regarding their use in HC image enhancement have been presented. Since the effects on contrast of gamma corrrection depend on the local luminance, we have studied in some detail this dependency; also, we have presented a technique for the adaptive application of exponentiation. We consider power laws only as applied to variables that range in the interval [0, 1]. Regarding unsharp masking, if both the V and the H components are sharpened the image may become too crispy. The readability of an HC image is usually improved manipulating the luminance of the image; this nevertheless usually also leads to a loss of depth (in the perceived 3D scene): a compromise must be made. Broadly speaking, the continuous magnitudes in the physical world are unbounded and linearly ordered; they are typically modelled on the real line or on the positive real line. The circle is a mathematical model e.g. of the phase of a complex number and of the perceptual magnitude of hue. Transducers give bounded electrical readings normally using some type (smooth or not) of saturating nonlinearity. Both bounded and circular magnitudes play an important role in image processing. It is usually a fruitful strategy to simulate the known mechanisms present in biological vision systems for their implementation in cameras and in image processing software; nevertheless, it must not be forgotten that, when seen, the image will be again processed by the Human Visual System and there is a risk of overdoing things.
REFERENCES Debevek, P. and Malik, J. (1997). Recovering high dynamic range radiance maps from photographs. SIGGRAPH’97, pages 369–378. Devlin, K. (2002). A review of tone reproduction techniques. Technical Rep. Dep. of CS, U. of Bristol, CSTR-02(-005):1–13. Drago, F., M. K. A. T. and Chiba, N. (1997). Adaptive logarithmic mapping for displaying high contrast scenes. EUROGRAPHICS 2003, 22(3):419–427. Kroon, D. (2008). http://www.mathworks.com/ matlabcentral/fileexchange/. Mardia, K. and Jupp, P. (2000). Directional Statistics. Wiley, Chichester. Meylan, L., Alleysson, D., and Susstrunk, S. (2007). Model of retinal local adaptation for the tone mapping of color filter array images. J. Opt. Soc. Am. A, 24(9):2807–2816. Meylan, L. and Susstrunk, S. (2006). High dynamic range image rendering with a retinex-based adaptive filter. IEEE trans. on Image Processing, 15(9):2820–2830. Naka, K. and Rushton, W. (1966). S-potentials from colour units in the retina of fish (cyprinidae). J. Physiol, 185:536–555. Restrepo, A., Marsi, S., and Ramponi, G. (2008a). Chromatic enhancement for virtual restoration of art works. EVA 08, Florence, April 2008, V. Cappellini and J. Hemsley, eds.(Pitagora, Bologna):94–99. Restrepo, A. and Ramponi, G. (2008a). Filtering and luminance correction of aged photographs. IST/SPIE E.I. Sci. and Techn. San Jose, California, USA, Jan. 27-31, 2008, 6812(02):1 – 11. Restrepo, A. and Ramponi, G. (”2008”b). Word descriptors of image quality based on local dispersionversus-location distributions. EUSIPCO 08, Lausanne, Switzerland, August 25-29, 2008, pages –. Restrepo, A., Rodriguez, C., and Vejarano, C. (2008b). Circular processing of the hue variable: a particular trait of image processing. Procs. VISSAP -Second Int. Conf. on Computer Vision Theory and Applications. Tumblin and Hodgins (1999). Two methods for display of high contrast images. ACM trans. on Graphics, 18(1):56–94.
Alfredo Restrepo (Palacios), Stefano Marsi and Giovanni Ramponi DEEI, University of Trieste, Via Valerio 10, Trieste, Italy [email protected], [email protected], [email protected]
Keywords:
Tone mappings, gamma correction, unsharp masking, circular processing, color enhancement.
Abstract:
We present techniques for performing luminance mapping and unsharp-masking-like processing on digital images; they are specifically designed for their application on bounded and on circular signals. We study the effect of gamma correction on contrast and develop an adaptive technique for luminance and contrast correction. We use the proposed tools for the enhancement of high contrast color images.
1
INTRODUCTION
The luminance component of an image is normally coded in the linearly-ordered and bounded interval [0, 1]. For static scenes, the method of (Debevek and Malik, 1997) allows the capture of high luminance range scenes in the form of High Dynamic Range Radiance Maps (HDR images, for short); the radiance is unbounded in principle and needs to be mapped to a luminance component. Given the relatively low dynamic range of visualization devices, it is not possible to literally display HDR images; nevertheless, the display of high contrast (HC) images does benefit from the use of local contrast amplification, as it improves their quality and readability; image contrast results both from luminance contrast and from hue contrast. By an HC image it is meant an image (that typically corresponds to a high radiance range scene) with a luminance histogram having modes near 0 and near 1, corresponding to very light and very dark regions in the image; such regions normally contain (perhaps unnoticeable) subregions at small contrast. Large contrast is likely to be observable in any type of image, also in HC images. Small contrast, even though present and probably observable in the original scene, may not be noticeable. The luminance component of an HC image usually can be meaningfully segmented into light and dark regions and we base a correction of luminance and of color saturation on such a segmentation. If
realism is a goal, the increase of color saturation at light regions should be somewhat smaller than that at regions of intermediate luminance; similarly, increments of luminance should be smaller at light regions. Depending on the scene, other things could be taken into account; e.g. the illumination in shadows under a sunny sky tends to be bluer than direct sun light which is relatively yellower; also, the Bezold-Brucke effect predicts a shift towards yellow of oranges and cetrines, and a shift towards blue of cyans and purples, when the intensity of the illumination increases. In addition to the topics of logarithms and saturating nonlinearities, a recurrent topic in the subject of tone mapping is power laws; consider for example the bias function used by Drago (Drago and Chiba, 1997) and the use of power laws made in (Tumblin and Hodgins, 1999). We consider here a variant approach in the use of power laws; also, we use power laws for color saturation enhancement. The enhancement of contrast for the visualization of HC images is a topic of actuality. Meylan and Susstrunk (Meylan and Susstrunk, 2006) locally enhance contrast within a retinex framework by computing a local luminance using a mask resulting from the convolution of the image and a Gaussian surround function. (Drago and Chiba, 1997) use a logarithmic law where the base of the logaithm is made dependant on a power law of the pixelwise (rather than local) luminance. In Section 2, we propose a method for mapping
the luminance of an HDR image to the interval [0,1]. In Sections 3 and 4, we present techniques for the enhancement of small local contrast of the (bounded) luminance component. In Section 5.1, we present a technique for the enhancement of local contrast for the (circular) hue variable and in Section 5.2 we present an adaptive technique that works both on the luminance and saturation components and that improves the readability of HC images.
2
ON THE RENDITION OF HDR IMAGES
Even though there is no consensus regarding a formula for the conversion of radiance into luminance, there is an agreement that it should be approximately logarithmic. As (Drago and Chiba, 1997) point out, both the logarithmic Weber-Fechner law and Stevens’ power law, are relatively close in practice. The following formula, maps the radiance r of a given image onto the [0, 1]-coded luminance X by computing Y=log(r) and the min (which typically is negative) and range (i.e. the max minus the min) of Y. X = (Y − min)/range
(1)
In this way, the minimal radiance is mapped to zero (“black”) and the largest one to one (“white”). We assume that the minimal radiance is positive, even if small; this is a valid assumption so long as there are no “black body radiators at 0 K” in the scene and the image is registered at a varied enough set of speeds. As an example, consider the result of the application of the method to the HDR image memorial.hdr (r was calculated as 0.3R+0.6G+0.1B), shown in Figure 1. Tumblin and Hodgins (Tumblin and Hodgins, 1999) proposed a set of desired properties that a compression method should meet, namely, progressiveness (large contrast is more compressed than small contrast), monotonicity (small luminances remain smaller than large luminances), symmetry (very large and very light regions are affected equally), asymptoticity (an infinite range is compressed into a finite one), minimality (scene contrast is compressed only just enough to fit display contrast) and adjustabbility (suits viewer preferences); of these, for the [0, 1] rendition of HDR images, symmetry is the hardest to meet because even though the minimal radiance is in practice near zero, the largest one can vary widely. To add adjustability to the method in Equation 1, the radiance r can be raised to a power before computing Y.
Figure 1: (logR+7.2)/13, applied to the radiance of memorial.hdr
2.1
Naka-Rushton saturating nonlinearities
In addition to logarithmic nonlinearities, other types of nonlinearity have been considered for the rendering of HDR images; in particular, those suspected to play a role in the retina of vertebrates (Meylan et al., 2007). (Naka and Rushton, 1966) proposed the nonlinearity V = 0.5(1 + tanh(x − x0 ))
(2)
(where we have added the normalizing factor 0.5) to model the S-potential response to flashes of monochromatic light of (probably bipolar) retinal cells, where the variable x represents the log(I) of the intensity I; see Fig. 3. In terms of I, the nonlinearity 2 becomes: V = 0.5(1 + tanh(log( II0 ))) = I 2I+I 2 which 0
is closely related to the other nonlinearity they proJ pose: V0 = J+1 , see Figure 2, better known in the form x y= (3) x + x0
3
ON GAMMA CORRECTION AND CONTRAST ENHANCEMENT
The difference between the luminances of two neighboring pixels measures local contrast. In relation to the [0, 1]-coded luminance space, the (unsigned) contrast can be large or small; by small local contrast we
Figure 2: The Naka-Rushton saturating formula for parameter values x0 = 0.2, 0.5, 1, 2, 3.
Figure 3: The Naka-Rushton saturating formula of the second type, 0.5(1 +tanh(x − x0 )), for parameter values x0 = 3, 4, 5, 6, 7.
mean window-neighbor pixels (i.e. pixels that fit in a single window) differing in luminance by a small amount, e.g. less than 0.1; by large local contrast we mean neighbor pixels differing in luminance by a large amount, e.g. more than 0.3; we are concerned here with small local contrast. The derivative of a luminance map measures its effect on small contrast at different luminances. We define the small contrast amplification (SCA) of a tone map as the value of the derivative of the mapping evaluated at the local luminance (which is measured using a location statistic such as the average, the median, the midrange (i.e. the average of the min and the max) or a weighted combination of these) of the window (called also the receptive field) in question. In general, a [0, 1] → [0, 1] luminance map has different effects on small local contrast at different luminances; for the mapping technique known as gamma correction, the luminance of each pixel is raised to a positive power. Such powerlaw techniques are usually applied using the same exponent for each pixel, and also in a pointwise fashion in the sense that the resulting luminance of the pixel depends on the luminance of the pixel alone. By using an exponent g larger (resp. lower) than one an image becomes darker (resp. lightes); this effect has been
exploited in the restoration of faded prints (Restrepo and Ramponi, 2008b); see Figure 4; by looking at the derivative of the mapping it is seen that small local contrast is improved where it matters, i.e. in the luminance range where the image is originally heavily represented; see Figure 5. Thus, it is conceivable to use an adaptive power law approach to improve the contrast of images, regardless of their luminance histogram; there is a caveat though as we show below. Regarding the use of exponent laws that depend locally on the luminance, we consider two approaches. In the first approach, we keep the SCA constant and equal to one. It is seen that the resulting effect on the luminance is monotonically nonincreasing: a nonaesthetic effect since it reverses nonlocal large contrast. In the second approach the SCA is maximized at each luminance; it will be seen that the output luminance becomes constant, rendering the method, as such, useless. A variant approach that renders the second approach useful is obtained by making the map a particular case of a parametrized family of mappings where the parameter can be made to depend on local contrast, which may be measured with e.g with the range (the max minus the min), as done e.g. in (Restrepo and Ramponi, 2008a).
Figure 4: The effect of gamma correction Lg at different luminances L in[0, 1], at different exponents g = 0.2, 0.5, 1, 2, 4: compare with the curve g=1. Lg > L if g < 1, and viceversa
3.1
An adaptive power law for constant SCA: a negative results
Constant SCA using a variant power law tone map is achieved as follows. The derivative of the function G(L) := Lg is given by G0 (L) = gLg−1 ; solving for L 1
the equation G0 (L) = 1 we get L1 (g) = ( g1 ) g−1 , see Figure 6. Call the inverse of this function g1 (L); we believe it has no closed form expression. The resulting adaptive power law is given by AG1 (L) = Lg1 (l) , where l is the local luminance l measured with an alpha blend of the pointwise luminance and the median
3.2
Maximal SCA
Consider now the second approach; it uses an exponent that maximizes the SCA at each possible local d2 luminance. From dgdL Lg = 0 one gets the desired ex1 ponent g2 (L) = − log(L) . Again, for a pointwise local
Figure 5: Derivatives of Lg for g = 0.2. 0.5, 0.7, 1, 2, 3, 4, 8. Note the coresponding regions where the derivative has high values.
luminance, AG2 (L) given by Lg2 (L) is not useful: it is the constant function 1e , see in Figure 8 the curve with parameter K=1. The maps shown in the figure darken pixels for luminances above 1e , and viceversa. We consider this a desirable property of tone maps for HC images since in such images, both dark and light gray levels tend to get clustered together.
of the luminances in the receptive field. In an extreme case (K=1 in Figure 7) where the local luminance is pointwise, l = L, AG1 (L) is given by Lg1 (L) is a decreasing function; this case does not have aesthetic applications as it reverses the luminances in the image; nevertheless, it may be of interest, e.g. in product inspection, to have an image where the contrast has been uniformized.
Figure 8: The function L1+K(g2 (L)−1) ; K = 0.1, 0.3, 0.5, 0.7 and 1.0 (indicated on the right). AG2 (L) > L if L < 1e and viceversa; 1e is a fixed point for each K.
3.3 Figure 6: L1 (1) =
1
Unitary-slope luminance L1 (g) = ( g1 ) g−1 ;
1 e
Figure 7: The function L1+K(g1 (L)−1) ; K = 0.1, ... 1.0. Luminance intervals of negative slope get reversed: monotonicity is lost.
Modulation of the technique using local contrast and local luminance
In addition to the use of a local luminance l, besides the pointwise luminance L, local contrast can be used to obtain useful versions of the methods described in the two previous subsections. As already suggested in Figure 8, consider the use of the map AGB2 (L) = L1+K(g2 (l)−1) where the parameter K ∈ [0, 1] is a function of the local contrast c. At one extreme, K=1, we have AGB2 (L) = AG2 (L) = Lg2 (l) already considered; at the other extreme, K=0, we have the do-nothing case AGB2 (L) = L1 . From Figure 7 we conclude that g1 probably needs a special dependency of K on the local contrast; the use of g2 gives more straightforward results. The parameter K is made dependant on the local contrast c as measured e.g. by the range of the luminances in a window centered at each pixel; in Figure 9, K(c)=c was used. The use local contrast and local luminance renders the method of the variant type (Devlin, 2002).
functions L−l ), L ∈ [0, l] (4) l L−l ), L ∈ [l, 1] (5) fl (L) = L + h2 ( 1−l where the functions h have infinite slope at x=0 and are given by 1 −c l h1 (x) = (6) (−x) 2n (1 + x), x ∈ [−1, 0] MAX 3 1 c 1−l h2 (x) = (x) 2n (1 − x), x ∈ [0, 1] (7) MAX 3 where the positive integer n controls the behaviour 1 1 2n ( 2n+1 ) 2n near the point of infinite slope, MAX = 2n+1 and c ∈ [0, 1] is a constant that can be used for tuning the desired amount of masking. Clearly, the funcfl (L) = L + h1 (
Figure 9: Application of AGB2 , K(c) = c; l = 0.5L + 0.5median; c=0.5rng+0.5qsrng to image in Figure 1
.
4
UNSHARP MASKING FOR BOUNDED SIGNALS
Taking inspiration from the maps shown in Figures 8 and 7, we decided to directly design one such map family, parametrized by local luminance, without assuming any underlying map, such as that of gamma correction. Each tone map in the family of maps fl : [0, 1] → [0, 1] should be increasing and meet f(0)=0, f(1)=1. Then, for each pixel luminance L, depending on the local luminance, one fl of the family of tone maps is applied, and the corrected luminance fl (L) results. The chosen map fl (L) has a slope larger than one for pixel luminances L near the local luminance l; in this way, small contrast is amplified, that is, the variations of luminance near the luminance of the receptive field are enlarged. Each map fl : [0, 1] → [0, 1] in the family is a continuous function with fL (L) = L, so that it fixes local luminance; also, it has a convex derivative fl0 and fl0 (L) has a maximum at l = L, locally increasing small contrast for luminances L near the local luminance l. We also impose the requirement that the derivative be positive (i.e. that the function be strictly increasing); this ensures that luminances (even if far off the local luminance of the field) do not become saturated near 0, 1 or any intermediate level. The methodology can be catalogued as one of unsharp masking, even if of a special type, specifically designed to be applied to signals that live in the interval [0, 1]. In particular, consider the following family of
Figure 10: c=0.5, n=5, l=0.6
tions h go to 0 in a finite interval. The technique is of the unsharp masking type whenever the computation of the local luminance l involves an operation of the smoothing type, as it is usually the case. The difference between the signal and the smoothed version is amplified accordingly. Figure 11 shows an example of the application of the technique to the image in Figure 1.
5
PROCESSING OF THE CHROMA COMPONENTS
We process the chroma components of the image as well; in the HSV color system, the saturation is a linear bounded variable and the hue is a circular variable (Restrepo et al., 2008b). We define a class of circular unsharp masking maps and apply them to the hue component for enhancing fine detail of color images; the proposed technique compares favorably with that in (Restrepo et al., 2008a). The saturation component of HC images is enhanced using gamma correction; saturation, being a bounded variable, is amenable to gamma-correction enhancement.
contrast is enhanced, i.e. the variations of hue near the local hue of the field are enlarged. First, define a continuous, increasing function f : [0, 2P] → [0, 2P] with f (0) = 0, f (2P) = 2P, f (P) = P such that the derivative of f is maximal at P. For example, consider the map f : [0, 2P] → [0, 2P] with infinite slope at P, given by 1
f (x) = x − cx(P − x) 2n , x ∈ [0, P] 1
f (x) = x + c(2P − x)(x − P) 2n , x ∈ [P, 2P]
(8) (9)
where the constant c ensures that f’ is positive everywhere. With one such function f already defined,
Figure 11: The image in Figure 1, after the application of bounded unsharp masking; c=0.3, n=1
5.1
Unsharp masking for circular signals
The angular hue variable of the HSV color system measures 0 degrees for R (red), 120 degrees for G (green) and 240 degrees for and B (blue); nevertheless, it can be argued that, yellow being a nonbinary color, the perceptual difference between red and yellow and that between yellow and green, are the same as those between green and blue, and blue and red. Thus, we transform the hue variable h of the HSV system to a modified hue H, also circular, as follows. Denoting the angle pi with the letter P, for angles h between 0 and (2/3)P, put H=(3/2)h while for angles h between (2/3)P and 2P, put H=(3/4)h + (1/4)2P. It is also convenient to denote each angle x with the complex number e jx ; et S1 = {z ∈ C : |z| = 1} be the unit circle of the complex plane, centered at the origin, 1 1 3 where e j0 , e j2P 4 , e j2P 2 , e j2P 4 represent the basic hues red, yellow, green and blue, respectively, and where the binary hues are correspondingly in between, e.g. the oranges are of the form e jw with 0 < w < P2 . With the purpose of enhancing the hue contrast of images, the hue H of each pixel is mapped to a hue mt (H), where the function m depends on the local hue t, which is computed using e.g. the circular average or a circular median, (when they exist, see (Mardia and Jupp, 2000) and (Restrepo et al., 2008b)) of the hues in the receptive field. For the examples given here, the circular mean was used. The map mt (H) has a “slope” larger than one for values of H near the local hue t of the receptive field; in this way, small
Figure 12: n=1, c=.3; consider the rectangle [0, 2P]×[0, 2P] as a torus split along a meridian and a longitude.
for each t, let mt : S1 → S1 be the function given by j f (∠ e
jH
)
−e jt where ∠z stands for the angle mt (H) = −e jt e of the complex number z. This effectively implements a circular map, analogous to the identity i : S1 → S1 , (its graph lives on the torus S1 × S1 ) but with derivative larger than one near t. It fixes local hue: mt (t) = t; also, it has a convex derivative mt0 ; mt0 (H), maximal at H = t, and minimal and smaller than one, at the opposed hue, it locally increases small contrast, for hues H near t. A requirement that the derivative be positive avoids the clustering of hues. As in the previous section, the methodology presented unsharp masking type.
5.2
Saturation enhancement using power laws
The enhancement of the saturation is made using a power law. Consider the enhancement of the HC image shown in Figure 17, resulting from a picture being taken inside a room with a window, at daylight. Since it can be argued that the corrections needed depend on the level of radiance coming from each part in the 3D scene, rather than making corrections to the image on the basis of pointwise luminance, we segment the image using a standard region-growing routine (Kroon,
Figure 16: Chroma processing of the image in Figure 15; saturation processing with g = 0.7 and hue processing with c=0.3, n=1. Figure 13: Market Image.
Figure 17: An image of high contrast, resulting from a high dynamic radiance scene.
2008) into light and dark regions and apply a correction that depends on the region. See Figure 18. We
Figure 14: The hue component of Market image has been processed using circular unsharp masking with n = 1, c = 0.3. Even though only the hue component has been processed, the image is crispier.
Figure 18: The image is segmented into light and dark regions.
Figure 15: Image of a faded tapestry.
use power laws on the saturation and value conmponents of the image. Regarding the saturation component S, we used the exponents 0.95 and 0.7 for the clear and dark regions, respectively; regarding the luminance component V, we applied no correction and the exponent 0.5 in the light and dark regions, respectively. See the result in Figure 19. There seems to be no reason to decrease the saturation at any part of the image.
ACKNOWLEDGEMENTS This work was partially supported by the FIRB project no. RBNE039LLC and by a grant of the University of Trieste. The Ancient tapestry of Figure 15 belongs to the Museo Civico Sartorium of Trieste. A. Restrepo is on leave of absence from the dpt. de Ing. Electrica y Electronica, Universidad de los Andes, Bogota, Colombia, ([email protected]). Figure 19: The image in Figure 17, after adaptive saturation and value enhancement.
6
CONCLUSION
Just as it is useful to increase the contrast of signals of a (linear) bounded range, it is of value to have a tool for increasing the contrast of signals with a circular range. In addition to presenting a method for the rendering of (unbounded) HDR images to the [0, 1] luminance interval, a set of tools for luminance correction, luminance unsharp masking, hue unsharp masking and saturation enhancement, has been presented. Applications regarding their use in HC image enhancement have been presented. Since the effects on contrast of gamma corrrection depend on the local luminance, we have studied in some detail this dependency; also, we have presented a technique for the adaptive application of exponentiation. We consider power laws only as applied to variables that range in the interval [0, 1]. Regarding unsharp masking, if both the V and the H components are sharpened the image may become too crispy. The readability of an HC image is usually improved manipulating the luminance of the image; this nevertheless usually also leads to a loss of depth (in the perceived 3D scene): a compromise must be made. Broadly speaking, the continuous magnitudes in the physical world are unbounded and linearly ordered; they are typically modelled on the real line or on the positive real line. The circle is a mathematical model e.g. of the phase of a complex number and of the perceptual magnitude of hue. Transducers give bounded electrical readings normally using some type (smooth or not) of saturating nonlinearity. Both bounded and circular magnitudes play an important role in image processing. It is usually a fruitful strategy to simulate the known mechanisms present in biological vision systems for their implementation in cameras and in image processing software; nevertheless, it must not be forgotten that, when seen, the image will be again processed by the Human Visual System and there is a risk of overdoing things.
REFERENCES Debevek, P. and Malik, J. (1997). Recovering high dynamic range radiance maps from photographs. SIGGRAPH’97, pages 369–378. Devlin, K. (2002). A review of tone reproduction techniques. Technical Rep. Dep. of CS, U. of Bristol, CSTR-02(-005):1–13. Drago, F., M. K. A. T. and Chiba, N. (1997). Adaptive logarithmic mapping for displaying high contrast scenes. EUROGRAPHICS 2003, 22(3):419–427. Kroon, D. (2008). http://www.mathworks.com/ matlabcentral/fileexchange/. Mardia, K. and Jupp, P. (2000). Directional Statistics. Wiley, Chichester. Meylan, L., Alleysson, D., and Susstrunk, S. (2007). Model of retinal local adaptation for the tone mapping of color filter array images. J. Opt. Soc. Am. A, 24(9):2807–2816. Meylan, L. and Susstrunk, S. (2006). High dynamic range image rendering with a retinex-based adaptive filter. IEEE trans. on Image Processing, 15(9):2820–2830. Naka, K. and Rushton, W. (1966). S-potentials from colour units in the retina of fish (cyprinidae). J. Physiol, 185:536–555. Restrepo, A., Marsi, S., and Ramponi, G. (2008a). Chromatic enhancement for virtual restoration of art works. EVA 08, Florence, April 2008, V. Cappellini and J. Hemsley, eds.(Pitagora, Bologna):94–99. Restrepo, A. and Ramponi, G. (2008a). Filtering and luminance correction of aged photographs. IST/SPIE E.I. Sci. and Techn. San Jose, California, USA, Jan. 27-31, 2008, 6812(02):1 – 11. Restrepo, A. and Ramponi, G. (”2008”b). Word descriptors of image quality based on local dispersionversus-location distributions. EUSIPCO 08, Lausanne, Switzerland, August 25-29, 2008, pages –. Restrepo, A., Rodriguez, C., and Vejarano, C. (2008b). Circular processing of the hue variable: a particular trait of image processing. Procs. VISSAP -Second Int. Conf. on Computer Vision Theory and Applications. Tumblin and Hodgins (1999). Two methods for display of high contrast images. ACM trans. on Graphics, 18(1):56–94.
