Inkjet Printer Model-Based Halftoning - Semantic Scholar
674
IEEE TRANSACTIONS ON IMAGE PROCESSING, VOL. 14, NO. 5, MAY 2005
Inkjet Printer Model-Based Halftoning Je-Ho Lee, Member, IEEE, and Jan P. Allebach, Fellow, IEEE
Abstract—The quality of halftone prints produced by inkjet (IJ) printers can be limited by random dot-placement errors. While a large literature addresses model-based halftoning for electrophotographic printers, little work has been done on model-based halftoning for IJ printers. In this paper, we propose model-based approaches to both iterative least-squares halftoning and tone-dependent error diffusion (TDED). The particular approach to iterative least-squares halftoning that we use is direct binary search (DBS). For DBS, we use a stochastic model for the equivalent gray-scale image, based on measured dot statistics of printed IJ halftone patterns. For TDED, we train the tone-dependent weights and thresholds to mimic the spectrum of halftone textures generated by model-based DBS. We do this under a metric that enforces both the correct radially averaged spectral profile and angular symmetry at each radial frequency. Experimental results generated with simulated printers and a real printer show that both IJ model-based DBS and IJ model-based TDED very effectively suppress IJ printer-induced artifacts. Index Terms—Digital halftoning, direct binary search (DBS), error diffusion, inkjet (IJ) printer models.
I. INTRODUCTION
D
IGITAL halftoning is the process of creating a pattern of binary pixels that the eye perceives as a continuoustone image. Digital halftoning is necessary for devices, such as printers, that can only produce a small set of output levels. Methods for digital halftoning may be categorized into three groups according to the computational complexity required to actually render the continuous-tone image in halftone form, independent of the computation required to design the halftoning algorithm [1]. The simplest group consists of point processes, in which each pixel of the halftone image is a function of only one pixel from the continuous-tone image. Screening or dithering, the digital analog of the contact screening process from the graphic arts and printing industry, is the predominant algorithm of this type. The second class includes algorithms that require processing of pixels from a local neighborhood in the continuous-tone image. The dominant example of this type of algorithm is error diffusion. The last and most complex class of algorithms are those that are iterative in nature, requiring several passes through the continuous-tone image to determine the halftone image. Iterative methods for halftoning come in two Manuscript received April 25, 2003; revised May 11, 2004. This work was supported by the Hewlett-Packard Company. The associate editor coordinating the review of the manuscript and approving it for publication was Dr. Reiner Eschbach. J.-H. Lee was with the School of Electrical and Computer Engineering, Purdue University, West Lafayette, IN 47907-1285 USA. He is now with Hewlett-Packard Company, Vancouver, WA 98683 USA (e-mail: [email protected]). J. P. Allebach is with the School of Electrical and Computer Engineering, Purdue University, West Lafayette, IN 47907-1285 USA (e-mail: [email protected]). Digital Object Identifier 10.1109/TIP.2005.843787
flavors—those that find the minimum or maximum of some sort of metric and those that satisfy constraints. In general, implicit in the development of the halftone algorithms is the assumption that the printer is an ideal device, i.e., it always puts ideal square dots at exactly the correct locations. Unfortunately, however, most printers do not obey these assumptions. When halftoning is done without accounting for this fact, the printed textures may not have the appearance predicted by the halftoning algorithm. In order to reproduce the original grayscale image correctly, it is necessary to incorporate within the halftone algorithm a model for the printing process. There has been a great deal of research considering how to model printers and how printer models may be incorporated within a halftoning algorithm. Depending on the approach taken to the modeling, printer models can be classified into two categories, print characterization models and process characterization models [2]. The print characterization models are based entirely on a characterization of the halftone image as it exists on the printed page. The process characterization models reflect the actual process whereby the digital halftone is transformed to colorant on the page. The most widely used print characterization model is the hard circular dot (HCD) model, which was first proposed by Roetling and Holladay [3] to account for spreading of colorant on the medium, which results in increased absorptance of the print. This phenomenon is commonly referred to as mechanical dot gain. In the HCD model, it is assumed that the printer generates a circular spot with constant absorptance at each printer addressable location and multiple dot overlap is resolved by a logical OR at each point in the print. Stucki [4] suggested an extension of error diffusion that incorporates an HCD model. Stevenson and Arce [5] extended error diffusion to the hexagonal lattice and used an HCD model to compensate for neighborhood dot interaction. Pappas et al. [6] popularized the HCD model and generalized the HCD model to a parametric method [7]. Knox et al. [8] developed a similar parametric model. Baqai and Allebach [9] incorporated the HCD model into the direct binary search (DBS) halftoning algorithm, which seeks to minimize the mean-squared error between the perceived rendered halftone and the continuous-tone original image [10], [11]. Lin and Wiseman [12] proposed a stochastic dot model based on modeling the probability of a toner particle occurring at a particular location as being Gaussian distributed around the center of the dot. Flohr et al. [13] also used this model to improve the quality of textures produced by DBS. Baqai and Allebach [9] proposed a stochastic model based on the tabular model, and incorporated it into the DBS algorithm. The origin of process characterization models can be traced from the work by Ruckdeschel and Hauser [14]. They analyzed optical dot gain, also known as the Yule–Nielsen effect, which is the apparent darkening of the print caused by trapping of
1057-7149/$20.00 © 2005 IEEE
LEE AND ALLEBACH: INKJET PRINTER MODEL-BASED HALFTONING
scattered light under colorant dots. Loce et al. [15] proposed a model for the electrophotographic (EP) printing process to analyze banding induced by mechanical vibration. Kacker and Allebach [16] developed a detailed model for the EP process within the DBS halftoning algorithm, focusing on the exposure step, in which the optical photoconductor drum is written with a Gaussian laser beam. The advantage of process characterization models compared to print characterization is that fewer measurements may be required to parameterize the model. However, these models are quite complex in general. Despite these advances, virtually all printer model-based halftoning work has focused on EP printers. The lack of comparable work for inkjet (IJ) printers may be due to the fact that IJ printers do render dots that have a nearly hard, ideal profile, and are much more stable than dots rendered by EP printers. However, as we shall see, IJ printers do exhibit dot-displacement errors that are caused by misaligned nozzles in the printhead, and which appear random in nature. The goal of this paper is to model the IJ printer and then to incorporate this model into halftoning algorithms so that high print quality can be achieved. Here, we develop a stochastic printer model based on the dot statistics of the IJ printer. Our IJ printer model is primarily based on print characterization. To model the IJ printer, we take an approach similar to the one used in [9]: We print a specially designed test pattern, scan the print with a high-resolution scanner, and estimate the dot statistics from the print. However, whereas [9] directly measures the statistics of pixels such as their mean and variance conditioned on a certain neighborhood configuration, we measure the displacement and profile of each individual dot and then infer the conditional pixel statistics based on this information. This enables us to reduce the number of measurements required to model the printer. We incorporate an IJ model based on these statistics into the DBS halftoning algorithm. We also develop a printer model-based version of tone-dependent error diffusion (TDED) [17], which uses tone-dependent error weights and thresholds obtained through a process of training with DBS halftone textures. Experimental results demonstrate the effectiveness of these model-based halftoning algorithms. The remainder of this paper is organized as follows. In Section II, we describe the printer characterization process and the underlying model for the printer based on that characterization. In Sections III and IV, we describe model-based DBS and model-based TDED, respectively, and present experimental results. In Section V, we summarize the key findings of the paper.
II. PRINTER MODEL In this section, we will discuss a stochastic model which is based on a print characterization process. Our paper is focused on high-resolution IJ printers. We use an HP DeskJet 970Cx printer as our test vehicle. As its contemporaries do, this printer supports various print options such as different print resolutions, speeds, directions (uni/bi), number of printing passes, and number of ink drops at each pixel. In fact, the printer reveals various artifacts depending on its print mode. The source of artifacts are typically twofold: dot irregularity and dot-placement error.
675
Fig. 1. Illustration of the two passes used to print the pixels in the shaded region.
The cause of dot irregularity may be either ink coalescence or creation of satellites. Ink coalescence can take place when adjacent nozzles are fired simultaneously. Firing the nozzles at a higher frequency than they can handle may cause puddling around the nozzle opening that can create the satellites. Multiple-pass print modes are usually employed to avoid these printing artifacts [18]. In a multiple-pass mode, the pen visits each pixel more than once and puts a drop there during a certain pass. Therefore, extra logic control is required to determine whether the pen should fire a drop at a given location on a given pass when the pixel at that location is to be marked. Print masks are applied to provide this extra logic. They are usually coded as an array of 0s and 1s. A 1 indicates firing the nozzle, if the halftone pixel at that point has value 1. A 0 means no firing regardless of the value of the corresponding halftone pixel. Fig. 1 [18] shows an example for a two-pass print mode, with print-mask width 4 and height 8. As this mask array is tiled over the media, the pen sweeps across the media and marks every other pixel location if the halftone pixel value there is 1, in a checkerboard pattern. Afterward, the media is advanced half the pen height and the printing resumes. All entries that were missed by M1 the first time are picked up by M2 during the second time when the pen visits. The print masks reflect hardware limitations on firing frequency and avoid consecutive firing of the same nozzle. Dot-placement errors are caused by misaligned nozzles in a print head. Fig. 2 illustrates the nozzle geometry. The print head contains nozzles for three colorants: cyan, magenta, and yellow; each colorant is assigned two columns of the nozzle. From the top view, we can see that the nozzle columns of each colorant are at the edge of the ink feed slot, and the nozzle plate bows down into the ink feed slot as it spans the void. The drop will travel roughly perpendicular to the nozzle plate at the nozzle. However, since the nozzle columns are located on the curved part of the plate, the normals to the nozzle plate at each nozzle column have a slight inward tilt, causing the drop to travel slightly inward. This phenomenon is sometimes referred as a toe-in or dimple effect. In addition to this systematic bias, the dot displacement fluctuates randomly, which suggests the use of a stochastic model. In this paper, we have chosen the 600-dpi, 10 in/s, unidirectional, multiple-pass print mode as our target print mode. Even though the discussion here will be restricted to this print mode,
676
IEEE TRANSACTIONS ON IMAGE PROCESSING, VOL. 14, NO. 5, MAY 2005
Fig. 2.
Structure of print head.
Fig. 3. The test pattern. (a) A portion of the scanned test pattern. (b) A portion of the segmented pattern plus segmentation mask (marked by yellow circles).
our framework for model-based halftoning can be applied to other print modes or other IJ printers with minor modifications. As the first step toward the printer modeling, we characterize the HP DeskJet 970Cx printer. We print a test pattern with the target printer/mode using magenta ink which is the darkest of the three colorants. The printed image is then scanned at 4000-dpi resolution with a Howtek D4000 drum scanner, and the resulting scanned image is transferred to a computer to obtain dot statistics such as the mean and variance of the dot profile, and the horizontal and vertical dot displacements. The test pattern consists of 600 600 pixels where only every fifth column and fifth row contains colored pixels. Hence, there is a total of 14, 400 sample dots in the printed test pattern. Fig. 3(a) shows a portion of the scanned test pattern, and Fig. 3(b) shows a portion of the segmented pattern, plus the segmentation mask (marked by yellow circles). After segmentation, we measure dot profiles and compute the centroid of each dot based on its absorptance. The horizontal (or vertical) displacement of a dot is estimated by computing the difference between its centroid and the average value of the centroids of all the dots in the same row (or column). We measure dot displacements separately for even and odd raster lines, since even and odd lines are printed with different nozzle columns, i.e., each raster will only be printed by the left column or the right column, not by both. Fig. 4 shows the measurement data. As can be seen from the mean and standard deviation of the dot profiles, the rendered dots are reasonably hard. The mean dot
Fig. 4. Measurement data for the HP DeskJet 970Cx printer used in the 600-dpi, 10 in/s, unidirectional print mode. Dot profiles: (a) the mean dot profile and (b) the standard deviation of dot profiles. Histogram of the dot displacements: (c) horizontal dot displacements for even lines, (d) horizontal dot displacements for odd lines, (e) vertical dot displacements for even lines, and (f) vertical dot displacements for odd lines. The vertical dashed lines indicate the mean dot displacements.
size is close to , where in is the spacing between printer addressable spots with this print mode. We can also see from Fig. 4(c)–(f) that dots are significantly displaced horizontally with different mean shifts for even and odd lines, for even lines and for as expected (approximately odd lines). The figures indicate that there also exists nonzero vertical dot displacement, the distribution of which can be approximated as Gaussian. However, we have observed that these vertical placement errors do not cause significant visual artifacts in the rendered image. Hence, we will ignore them in this paper. To ensure consistency of the dot statistics as well as reliability of the measurements, we collected data from five different ink cartridges with three measurements for each pen. The final measurement data shown in Fig. 4 is the average of all 15 of those data sets. We found that all the measurements were quite consistent. In particular, the standard deviation in the mean shift over the 15 data sets was less than 10% for both even and odd lines. Fig. 5 illustrates the artifacts caused by the printer. The most prominent artifacts shown in Fig. 5(b) are vertical white wormy patterns which are introduced by horizontal dot-placement errors.
LEE AND ALLEBACH: INKJET PRINTER MODEL-BASED HALFTONING
677
Fig. 5. Illustration of the artifacts introduced by the random dot-placement error. (a) Halftone image generated by DBS with ideal printer model and rendered with an ideal printer. (b) The scanned HP DeskJet 970Cx printer output for (a). Note the vertical white veins to the left of the propeller and nose cone.
Based on these measured dot statistics, we define a stochastic printer model for the target printer as follows: dot with flat dot profile; • square • random horizontal dot displacement with first-order statistics approximated by the measured data; • no vertical dot displacement; • displacements mutually independent from pixel to pixel. The rationale for the mutual independence assumption is coupled with the print mask for the multiple-pass print mode. It is most likely that adjacent dots will be inked by different nozzles in different passes. Next, we will develop frameworks for IJ printer model-based halftoning in the context of DBS and TDED in Sections III and IV, respectively. III. IJ PRINTER MODEL-BASED DBS A. Overview of DBS DBS is a recursive search heuristic which uses a human visual system (HVS) model and a printer model to minimize a measure of the perceived difference between the rendered halftone and the continuous-tone image [10], [11]. By modeling the low-pass characteristic of the HVS as a linear shift-invariant filtering operation, DBS seeks a halftone image that minimizes
or 1 denote the digital halftone image. The value. Let halftone image rendered by the ideal printer is then given by (2) where is the spot function of the ideal printer is the periodicity matrix whose columns comprise the and basis for the lattice of printer addressable dots. We can express and as the relation between (3) where embodies the effect of cascading the ideal printer rendering and HVS models. With a high-resolution output device such as a 600-dpi printer, the printer spot profile is sufficiently narrow compared to so that . Throughout this paper, we will assume that this relationship holds and refer to the point spread function of the HVS . In addition, we will assume that the continuous-tone as can be also expressed in terms of its samples image according to (2). Then, the error measure in (1) can be written [10], [11] as
(1) and . Here, where denotes the convolution operation, represents the point is the continuous-parameter spread function of the HVS, is the continuous-tone original rendered halftone, and image, which is assumed to lie between 0 (white) and 1 and as the perceived rendered (black). We refer to halftone and the perceived continuous-tone images, respectively. Throughout the remainder of this paper, we represent continuous and discrete spatial coordinates, respectively, with and . vectors In general, the DBS algorithm is implicitly based on an ideal printer model. An ideal printer is one that can reproduce an ideal square dot with absorptance equal to that of the input pixel
(4) where
, , and . Starting with an arbitrary initial halftone, DBS scans through the halftone image in raster order, from left to right and from top to bottom. At each pixel of the halftone image, DBS considers toggling its state, or swapping it with any of its neighbors that differ in state. If any one of these changes reduces the error measure , DBS keeps that change which reduces it the most. One iteration of the algorithm consists of a visit to every pixel in the halftone image. When no changes are retained throughout an entire iteration, the algorithm has converged. This typically requires ten or so iterations.
678
IEEE TRANSACTIONS ON IMAGE PROCESSING, VOL. 14, NO. 5, MAY 2005
^[m] FOR THE DESKJET HP970Cx PRINTER WHERE m = [ 0 1] TABLE I
STATISTICS OF g
{
^[m ]
Note that g is a function of g and their respective displacements.
i; j
,
m = [ ] , m = [ + 1] i; j
i; j
, AND
m = [ + 2] { i; j
[m ], [m ], and [m ], and their respective displacements, while ^[m ] is a function of [m ], [m ], and [m ], g
g
g
The frequency response of the HVS model that we use for DBS is given by [19]
the average absorptance of at [20]; so
(6)
B. Stochastic Printer Model Even though we assume a hard square dot with size for will not be constant within our printer model, the value of cell due to random dot displacements. To account each which is for this, we use an equivalent grayscale image
within the
g
g
cell centered (7)
(5) is the spatial frequency coordinate in cywhere cles/radian subtended at the retina, is the average luminance , and . of the light from the print in cd/m , Taking the inverse two-dimensional (2-D) continuous-space Fourier transform yields the radially symmetric point spread . Here, the spatial coordinate has function units of radians subtended at the retina. To convert to units measured on the printed page, we note that a length when viewed at a distance will subtend an angle radians for . Thus
g
where . The equivalent grayscale image summarizes the microstructure of the halftone dots by a printer addresssingle value that is constant over each able cell. Under the equivalent grayscale model, the rendered halftone image on the page is assumed to be given by (8) The validity of (8) rests with the notion that the human viewer cannot resolve the microstructure of on a scale less than and perceives only an average value. In our printer is a function of neighboring halftone pixel values model, as well as their respective random displacements. Let dedenote the set of pixels in the neighborhood of , and let note the displacement vector at pixel . Then, this dependency can be expressed as (9) to only Since we assume that dots can be misplaced from is a 1 3 neighborhood of in the horizontal direction,
LEE AND ALLEBACH: INKJET PRINTER MODEL-BASED HALFTONING
679
Fig. 6. DBS results with the simulated never-centered printer (printed at 100 dpi). Halftone image generated by DBS with no printer model and rendered with (a) an ideal printer and (b) the never-centered printer. (c) Halftone image generated by DBS with the never-centered printer model and rendered with the never-centered printer.
. The dot-displacement is a stochastic process; so, is also a random 2-D sequence. Whether refers to a random process or one realization of that random process will be clear from the context.
a random quantity, we will take its expectation with respect to . Then, DBS seeks the digital halftone that minimizes (10)
C. Error Metric for Printer Model-Based DBS To put the equivalent grayscale model in the context of DBS, we first substitute into (4). Since is now
The error measure is a statistical average over the ensemble of all possible realizations of the sample mean-squared perceived
680
IEEE TRANSACTIONS ON IMAGE PROCESSING, VOL. 14, NO. 5, MAY 2005
Fig. 7. DBS halftone images rendered with the simulated bi-displacement printer (printed at 100 dpi) (a) generated by DBS with no printer model, (b) generated by DBS with the mean dot model for the bi-displacement printer, and (c) generated by DBS with the stochastic model for the bi-displacement printer.
rendered error for a given digital halftone. Substituting (4) into (10), we obtain, yet, another expression for , which is (11)
(12) Equation (12) has a nice interpretation. The first term on the right hand side is the error metric for a deterministic model, , which measures the perceived difference based on
between the ensemble average of the rendered halftone and the original image for a given digital halftone. The second term accounts for random fluctuations between adjacent dots. This term will penalize the use of local halftone textures that are vulnerable to dot-placement errors. If the range of dot displacements is relatively small, there would exist only limited dot interaction between neighboring dots. In this case, we may approximate by the the first term in (12) and use it as an error metric denoted by (13)
LEE AND ALLEBACH: INKJET PRINTER MODEL-BASED HALFTONING
681
We will refer to DBS algorithms based on error measures and as DBS with a stochastic dot model and DBS with a mean dot model, respectively. is computationally The direct evaluation of either or prohibitive. We will, therefore, describe an efficient method to evaluate the effect of trial halftone changes, which is similar to that reported for conventional DBS [10], [11]. Our derivation of the method for efficient evaluation of halftone changes will be restricted to the stochastic dot model only, since the mean dot model may be viewed as a special case of this more general model. Let us, then, consider the effect of a trial change on . We denote everything after the change by a prime. From (11), the new error can be expressed as
as well as its neighbors , for generally alter . Therefore, the change in due to the toggle at can be written as for (19) for Substituting (19) into (18), the change in error can be expressed as
(20) Similarly, we have the following equation for the change in error and due to a trial swap between pixels
(14) Then, we have
(21) (15) . If the spatial because distance between and is sufficiently large, there will be no interaction between the pixels and , and we can assume that and are independent, which implies that the same is true for and . More specifically, for the displacement and (or and ) models that we shall consider, are only dependent if and are neighbors of each other. Thus, we have if if where fine
(16) . Now, let us deto be the cross correlation between and
Table I summarizes the statistics of based on the measurements in Section II. The values of , , and will be precomputed based on Table I, and stored in a LUT. We also can precompute and store it in a LUT, since it is independent of the halftone. However, depends on the halftone; so, we have to update it whenever we accept a trial change. The extent . The update of the changes is limited by the support of equation is given by
if accept toggle if accept
(17) By substituting (16) and (17) into (15), the change in error can be expressed as
(18) Consider the effect on the error metric of a trial toggle at in the halftone. The change in would any location
swap. (22) D. Experimental Results for IJ Printer Model-Based DBS In this section, we evaluate the efficacy of using the printer model with DBS. We will show results for DBS with the mean dot model and the stochastic dot model and compare them with DBS with an ideal printer model. With any type of printer model, there are two important, and somewhat distinct questions to be addressed. The first one is how well DBS can exploit the printer model to create textures that are visually pleasing yet robust to the limitations of the printing technology, as they are represented by the model. The second question is how well
682
IEEE TRANSACTIONS ON IMAGE PROCESSING, VOL. 14, NO. 5, MAY 2005
Fig. 8. DBS halftone images rendered with the simulated correlated-displacement printer (printed at 100 dpi) (a) generated by DBS with no printer model, (b) generated by DBS with the mean dot model for the correlated-displacement printer, and (c) generated by DBS with the stochastic model for the correlated-displacement printer.
the model actually characterizes the printer. To address the first question without complication from the second, we use simulated printers. 1) Simulated Printer Results: Our first simulated printer model is a “never-centered” printer. As its name implies, this synthetic printer always prints dots with a fixed nonzero displacement, such that every dot on even raster lines is shifted and every dot on odd lines is shifted by horizontally by . This is a deterministic printer. Therefore, a stochastic dot model and a mean dot model for this printer will produce identical halftone images. Fig. 6 compares images rendered by the never-centered printer using DBS halftones generated
with no printer model and with the never-centered printer model. For comparison, we also show the result of printing on an ideal printer the DBS halftone generated with no printer model. These images were all printed at 200-dpi resolution simulated printer using 2 2 real printer dots for each addressable cell. Hence, the print is actually 100 dpi. The DBS halftone generated with no printer model and rendered with the never-centered printer [Fig. 6(b)] shows white worm artifacts throughout the sky behind the aircraft, noisy texture on the fuselage of the aircraft, and loss of detail in the word “SIX SHOOTER” and the image of Yosemite Sam. The DBS halftone generated with the printer model and rendered with
LEE AND ALLEBACH: INKJET PRINTER MODEL-BASED HALFTONING
683
Fig. 9. HP DeskJet 970Cx printer outputs (scanned at 4000 dpi). (a) DBS with no printer model. (b) DBS with the mean dot model for this printer. (c) DBS with the stochastic dot model for this printer.
the never-centered printer [Fig. 6(c)] shows none of these problems, and achieves almost the same level of quality as a DBS halftone generated with no printer model and rendered with an ideal printer [Fig. 6(a)]. We developed a second simulated printer model which has characteristics similar to those of our target printer, the HP 970Cx. This simulated printer behaves in a stochastic way such that dots on even/odd raster lines are shifted horizontally by (0 )/(0 or ) with equal probability. The dot-placement or errors are mutually independent. The mean dot shifts for even and , respectively, which and odd lines are, thus, matches our measurement for the target printer. We will call this printer the “bi-displacement” printer. Simulation results are shown in Fig. 7. Both the mean [Fig. 7(b)] and stochastic [Fig. 7(c)] dot models significantly reduce the white worm artifacts in the highlight tone patches and the veining artifacts and graininess in the mid tone patches that can be seen in the halftone [Fig. 7(a)] generated with no printer model. The halftones generated by the mean and stochastic dot models are
very similar in appearance. This is due to the very limited interaction between dots, which makes the impact of the covariance term in (12) minimal. In order to show the merit of the stochastic dot model over the mean dot model, we developed a third simulated stochastic printer. This simulated printer has a larger range of horizontal for odd lines) than dot shift (0 or for even lines and 0 or the previous printer does. Also, the dot-displacement errors are now strongly correlated for those dots on the same line. For the first dot on each line, the dot displacement is equally likely to for odd lines. For the rebe 0 or for even lines and 0 or maining dots on the line, the displacements are governed by a sequence of independent, identically distributed random variables , that take on value 1 with probability 0.8 and value 1 with probability 0.2. If , then the displacement for the th dot is the same as that for the th dot. Otherwise, it has the opposite value. For example, on an even-numbered th dot is 0 and , line, if the displacement of the then the displacement for the th dot will be . We will call
684
IEEE TRANSACTIONS ON IMAGE PROCESSING, VOL. 14, NO. 5, MAY 2005
Fig. 10. Comparison of two error measures. (a) A bitmap for absorptance 57/255. (b) The power spectrum of (a). (c) A second bitmap for the same absorptance. (d) The power spectrum of (c).
this printer the “correlated-displacement” printer. Simulation results for this printer are shown in Fig. 8. As before the halftone generated with no printer model [Fig. 8(a)] shows objectionable white worm artifacts in light tone patches and severe graininess in the mid-tone patches. Both the mean [Fig. 8(b)] and stochastic [Fig. 8(c)] dot models reduce those artifacts significantly. However, the stochastic model clearly shows its benefit over the deterministic model for this printer, especially in the mid-tone patches. Here, we still see horizontal banding and graininess in the halftone generated with the mean dot model. These artifacts are much less noticeable with the correlated-displacement model. 2) Real Printer Results: The images printed by the actual HP DeskJet 970Cx printer are shown in Fig. 9. In order to demonstrate the performance of each halftoning algorithm more clearly, we scan each print at 4000 dpi. As with the simulated printers, lots of objectionable white wormy patterns can be observed in the rendered halftone image generated by
DBS with no printer model [Fig. 9(a)]. Both DBS with the mean and stochastic dot models get rid of this artifact and yield more homogeneous patterns as seen in Fig. 9(b) and (c). IV. IJ PRINTER MODEL-BASED TONE-DEPENDENT ERROR DIFFUSION A. Overview of TDED Despite its capability to produce the highest level of print quality, DBS is not practical for most desktop printing applications because of its computational complexity, which requires several passes through the continuous-tone image to determine the final halftone image. For this reason, error diffusion, which was first proposed by Floyd and Steinberg [21], has been widely used in desktop printing applications. Since Floyd and Steinberg’s seminal work, much effort has been expended toward improving the quality and extending the capabilities of error diffusion. This includes using different weight sets [22], threshold
LEE AND ALLEBACH: INKJET PRINTER MODEL-BASED HALFTONING
Fig. 11.
685
Block diagram for training IJ printer model-based TDED.
modulation [23], different scan paths [22], [24], and an adaptive filter [25]. While these methods have been shown to be very effective for improving halftone quality, the resulting quality is still not comparable to that produced by search methods such as DBS. In recent years, tone-dependent methods that use different weight sets for different input pixel values have been developed. Eschbach used a larger weight set for extreme gray levels and a smaller set for midtones [26]. Ostromoukhov designed a method that uses a two-term weight set for extreme gray levels and a four-term weight set for midtones [27]. He later [28] optimized the weight set by minimizing the difference between the Fourier spectra of certain gray levels and the corresponding blue noise spectra [22]. Li and Allebach [17] used tone-dependent weight and threshold sets in their TDED algorithm, where the weight and threshold sets are chosen through an off-line training process. In the TDED algorithm, the error weights and threshand , respectively, are functions of olds, denoted by is determined the input pixel value . The binary output according to if otherwise.
(23)
Here, is the modified continuous-tone image, and the threshold function is given by if otherwise where
and
the input continuous-tone image prior to thresholding, or turn . off this modulation by setting is thresholded, the quantization error is computed Once as (27) and distributed to future pixels according to the tone-dependent weights (28) where denotes the region of support of the weight set. In order to obtain the optimal weights and thresholds, Li and Allebach employed the frequency weighted mean-squared error measure (1). However, they found that although this metric can generally indicate the smoothness of the binary texture, it sometimes fails to measure the texture variety and homogeneity. It works well for DBS because DBS only finds a local minimum of the metric starting from a random initial halftone that is directionally unbiased and has good texture variety. When optimizing the weights and thresholds for TDED, the initial halftone texture is generated by error diffusion and, thus, may be inhomogeneous and lacking in variety, especially in the midtones. For this reason, Li and Allebach used the error measure (1) only for the extreme levels (from 1 to 20 and from 235 to 254). For the remaining levels, they employed the measure
(24) (29)
are tone-dependent parameters satisfying (25) (26)
with period and the doubly periodic function is a halftone image for absorptance 0.5 generated by DBS. The thresholding operation given in (24) can be interpreted as a conditional threshold modulation process, i.e., with proper values of and , TDED can add the optimal amount of dither signal to
based on matching the magnitude of the spectrum of the DBS texture for the corresponding gray level with that for a given weight set of the TDED spectrum and threshold . The training process then seeks the parameter sets and that minimize this error measure. The search strategy used in [17] was based on a simple multigrid search method, which requires intensive computation. However, once the optimal sets are obtained from this
686
IEEE TRANSACTIONS ON IMAGE PROCESSING, VOL. 14, NO. 5, MAY 2005
Fig. 12. TDED results with the simulated never-centered printer (printed at 100 dpi). Halftone image generated by TDED with no printer model and rendered with (a) an ideal printer and (b) the never-centered printer. (c) Halftone image generated by TDED with the never-centered printer model and rendered with the never-centered printer.
off-line process, TDED requires exactly same order of computation as does conventional error diffusion.
B. Error Metric and Training Strategy for IJ Printer Model-Based TDED The success of TDED has demonstrated the power of training-based methods for halftoning algorithm design. With
this capability, it would be advantageous to be able to train the algorithm to avoid textures that exhibit noise artifacts which are intrinsic to the IJ process. Thus, we train the TDED parameters so that the spectrum of the TDED halftone image rendered under a model for the IJ process will be similar to that of the ideal DBS halftone image. It should be noted that with IJ model-based DBS, we actually embed the IJ printer model in the halftoning algorithm. With TDED, however, the IJ printer model is only used as part of the training process.
LEE AND ALLEBACH: INKJET PRINTER MODEL-BASED HALFTONING
687
Fig. 13. TDED results with the simulated bi-displacement printer (printed at 200 dpi). Halftone image generated by TDED with no printer model and rendered with (a) an ideal printer and (b) the bi-displacement printer. (c) Halftone image generated by TDED with the bi-displacement printer model and rendered with the bi-displacement printer.
The architecture of the core TDED algorithm remains the same as before. The training process for the IJ printer model-based TDED seeks the parameter sets and that minimize
(30)
grayscale image with the parameter sets and for the cor, we first responding gray level . To estimate , generate 50 128 128 TDED halftone images , and then compute the autocorrelation functions based on the printer model. The discrete Fourier transform (DFT) of the autocorrelation , is given by function, which is denoted by
where and are, respectively, the power spectra of the DBS halftone and the TDED equivalent
(31)
688
for . Here, and mutually independent, unless other. Thus, (31) can be written as
IEEE TRANSACTIONS ON IMAGE PROCESSING, VOL. 14, NO. 5, MAY 2005
and are are neighbors of each
(32) for
. Then, can be approximated by , which is the sample average of over . Similarly, we approximate the power spectrum , the sample average of the of the DBS texture by DFTs of the 50 DBS halftone images as in [17]. However, we have observed that globally minimizing the spectral difference based on the total mean squared error often leads to an undesirable rendered TDED halftone image. The power spectrum of the DBS pattern has a characteristically blue noise nature. This suggests a new error metric which somehow accounts more explicitly for this spectral property. We adopt two metrics proposed by Ulichney in [22] and define a new error metric based on the combination of these two metrics. We use the radially averaged power spectrum of and its anisotropy, which are denoted by and , respectively, where . They are defined as (33) (34) Here, is the number of samples at a given radial frequency and is the sample variance of those samples of . In our new training strategy, we compute the difand , and measure the ference between anisotropy of . The resulting error metric is given by (35) where is a parameter that controls the relative emphasis of the two terms in the error metric. We determined empirically that yields good results. The first term in matches the radially averaged power spectrum of the DBS texture with that of the TDED halftone. The second term explicitly encourages radial symmetry in the TDED power spectrum. Fig. 10 illustrates the efficacy of the error measure over the measure . Although the halftone in Fig. 10(c) has a larger value than the halftone in Fig. 10(a), it has more homogeneous texture and a more radially symmetric spectrum. On the other hand, the resulting values of agree with these observations. To determine the optimal parameter set, we use the derivative-free downhill simplex optimization method due to Nelder and Mead [29]. We
Fig. 14. HP DeskJet 970Cx printer outputs (scanned at 4000 dpi). (a) TDED with no model for this printer and (b) TDED with printer model for this printer.
have found that this optimization algorithm reduces the training time by a factor of ten compared to the one used in [17]. Fig. 11 summarizes the training algorithm for IJ printer model-based TDED based on the error metric defined in (35). C. Experimental Results for IJ Printer Model-Based TDED We evaluate the IJ printer model-based TDED halftoning algorithm by applying it to two of the same simulated printers and the same real printer as in Section III with the same test images (the mustang and step wedge images). The results are shown in Figs. 12–14. The simulation results for TDED shown in Figs. 12 and 13 parallel those for DBS very closely. The TDED halftones generated with no printer model and rendered with the nonideal simulated printers [Figs. 12(b) and 13(b)] suffer from artifacts similar to those seen earlier with DBS. On the other hand, TDED trained with the IJ printer model [Figs. 12(c) and 13(c)] achieves almost the same level of quality as a TDED halftone generated with no printer model and rendered with an ideal printer [Figs. 12(a) and 13(a)]. The images printed by the HP DeskJet 970Cx printer are shown in Fig. 14. Again, TDED trained with the IJ printer model yields improved print quality by suppressing artifacts such as white worms and veins, which are easily observed in the rendered halftone generated by TDED with no printer model.
LEE AND ALLEBACH: INKJET PRINTER MODEL-BASED HALFTONING
V. CONCLUSION In this paper, we introduced IJ printer model-based halftoning algorithms within the frameworks of DBS and TDED. We determined the printer model based on a printer characterization process, and we successfully incorporated it into the existing halftoning algorithms. For model-based DBS, we proposed two different error metrics to handle the random dot-placement errors in the printer and showed that both work well. We also showed that the algorithm can be efficiently implemented by recursively evaluating the effect of trial changes. For model-based TDED, we proposed a training method that encourages TDED to mimic the spectral characteristics of DBS. For this purpose, we used a new error metric based on the radially averaged power spectrum and anisotropy. We also employed a new optimization method, the downhill simplex method, which reduces the training complexity dramatically. The printer model is incorporated within the halftoning algorithm only through the training of the error diffusion parameters. The structure of the TDED algorithm itself is not modified. Both the model-based DBS and TDED halftoning algorithms described here remove most of the visual artifacts caused by the printer. Although this paper has focused on a specific inkjet printer—the HP DeskJet 970cx—the modeling approach that we presented is quite general and could be applied to other printers, as well. REFERENCES [1] J. P. Allebach, Ed., Selected Papers on Digital Halftoning. Bellingham, WA: SPIE, 1999. [2] J. P. Allebach, “DBS: retrospective and future directions,” in Proc. SPIE Color Imaging: Device-Independent Color, Color Hardcopy, and Graphic Arts VI, 2001, pp. 358–376. [3] P. G. Roetling and T. M. Holladay, “Tone reproduction and screen design for pictorial electrographic printing,” J. Appl. Photo. Eng., vol. 15, no. 4, pp. 179–182, 1979. [4] P. Stucki, “MECCA—A multiple-error correcting computation algorithm for bilevel image hardcopy reproduction,” IBM Res. Lab., Zurich, Switzerland, Res. Rep. RZ1060, 1981. [5] R. L. Stevenson and G. R. Arce, “Binary display of hexagonally sampled continuous-tone images,” J. Opt. Soc. Amer. A, vol. 2, no. 7, pp. 1009–1013, Jul. 1985. [6] T. N. Pappas and D. L. Neuhoff, “Model-based halftoning,” in Proc. SPIE Human Vision, Visual Processing, and Digital Display II, 1991, pp. 244–255. [7] T. N. Pappas, C. Dong, and D. L. Neuhoff, “Measurement of printer parameters for model-based halftoning,” J. Imag. Technol., vol. 2, no. 3, pp. 193–204, Jul. 1993. [8] K. Knox, C. Hains, and G. Sharma, “Halftone-independent calibratin of black & white printers,” in Proc. SPIE Human Vision and Electronic Imaging, 1996, pp. 432–436. [9] F. A. Baqai and J. P. Allebach, “Halftoning via direct binary search using analytical and stochastic printer models,” IEEE Trans. Image Process., vol. 12, pp. 1–5, Jan. 2003. [10] M. Analoui and J. P. Allebach, “Model based halftoning using direct binary search,” in Proc. SPIE Human Vision, Visual Processing, and Digital Display III, 1992, pp. 96–108. [11] D. J. Lieberman and J. P. Allebach, “A dual interpretation for direct binary search and its implications for tone reproduction and texture quality,” IEEE Trans. Image Process., vol. 9, no. 11, pp. 1950–1963, Nov. 2000. [12] Q. Lin and J. Wiseman, “Impact of electrophotographic printer dot modeling on halftone image quality,” in SID Int. Symp. Tech. Papers, Seattle, WA, May 1993, pp. 147–150. [13] T. J. Flohr and J. P. Allebach, “Halftoning via direct binary search with a stochastic dot model,” presented at the IS&T’s 3rd Tech. Symp. Prepress, Proofing and Printing, Chicago, IL, Nov. 1993. [14] F. R. Ruckdeschel and O. G. Hauser, “Yule-Nielsen effect in printing: a physical analysis,” Appl. Opt., vol. 17, no. 21, pp. 3376–3383, Nov. 1978.
689
[15] R. P. Loce, W. L. Lama, and M. S. Maltz, “Modeling vibration-induced halftone banding in a xerographic laser printer,” Electron. Imag., vol. 4, no. 1, pp. 48–61, Jan. 1995. [16] D. Kacker, T. Camis, and J. P. Allebach, “Electrophographic process embedded in direct binary search,” IEEE Trans. Image Process., vol. 11, no. 3, pp. 234–257, Mar. 2002. [17] P. Li and J. P. Allebach, “Tone-dependent error diffusion,” IEEE Trans. Image Process., vol. 13, no. 2, pp. 201–215, Feb. 2004. [18] J. Yen, M. Carlsson, M. Chang, J. Carcia, and H. Nguyen, “Constraint solving for inkjet print mask design,” J. Imag. Sci. Technol., vol. 44, no. 5, pp. 391–397, Sep./Oct. 2000. [19] R. Näsänen, “Visibility of halftone dot textures,” IEEE Trans. Syst., Man, Cybern., vol. 14, no. 6, pp. 920–924, Dec. 1984. [20] T. N. Pappas and D. L. Neuhoff, “Printer models and error diffusion,” IEEE Trans. Image Process., vol. 4, no. 1, pp. 66–80, Jan. 1995. [21] R. W. Floyd and L. Steinberg, “An adaptive algorithm for spatial grayscale,” in Proc. SID, 1976, pp. 75–77. [22] R. A. Ulichney, Digital Halftoning. Cambridge, MA: MIT Press, 1987. [23] K. T. Knox, “Error diffusion: a theoretical view,” in Proc. SPIE Human Vision, Visual Processing, and Digital Display IV, 1993, pp. 326–331. [24] I. H. Witten and R. M. Neal, “Using Peano curves for bilevel display of continous-tone images,” IEEE Comput. Graph. Appl., vol. CGA-2, no. 3, pp. 47–52, May 1982. [25] J. Lee, Y. Kwon, and H. Kim, “A color halftoning algorithm for low-bit flat panel displays,” in Proc. IEEE Int. Conf. Image Process., vol. II, Washington, DC, Oct. 1995, pp. 331–334. [26] R. Eschbach, “Reduction of artifacts in error diffusion by means of input-dependent weights,” J. Electron. Imag., vol. 2, no. 4, pp. 352–358, Oct. 1993. [27] V. Ostromoukhov, “Enhanced error-diffusion method for color or blackand-white reproduction,” U.S. Patent 5 737 453, 1998. , “A simple and efficient error-diffusion algorithm,” presented at [28] the SIGGRAPH, Los Angeles, CA, Aug.. 2001. [29] W. H. Press, S. A. Teukolsky, W. T. Vetterling, and B. P. Flannery, Eds., Numerical Recipes in C, 2nd ed. New York: Cambridge Univ. Press, 1992. Je-Ho Lee (M’00) received the B.S. degree in electrical engineering from the Hanyang University, Seoul, Korea, the M.S. degree in electrical engineering from the Pennsylvania State University, Philadelphia, and the Ph.D. degree in electrical engineering from Purdue University, West Lafayette, IN, in 1990, 1992, and 2002, respectively. From 1992 to 1997, he was with the Korea Institute of Science and Technology as a Research Staffer. He is currently an Imaging Scientist with HewlettPackard Company, Vancouver, WA. His research interests include digital halftoning, image enhancement, color image processing, and image compression.
Jan P. Allebach (F’91) received the B.S.E.E. from the University of Delaware, Newark, and the Ph.D. degree from Princeton University, Princeton, NJ, in 1972 and 1976, respectively. He was on the faculty at the University of Delaware from 1976 to 1983. Since 1983, he has been with Purdue University West Lafayette, IN, where he is the Michael J. and Katherine R. Birck Professor of Electrical and Computer Engineering. His current research interests include image rendering, image quality, color imaging and color measurement, and document management. Prof. Allebach is a member of the IEEE Signal Processing (SP) Society, the Society for Imaging Science and Technology (IS&T), and SPIE. He has been especially active with the IEEE SP Society and IS&T. He is a Fellow of both these societies, has served as Distinguished/Visiting Lecturer for both societies, and has served as an Officer and on the Board of Directors of both societies. He is a past Associate Editor for the IEEE TRANSACTIONS ON SIGNAL PROCESSING and the IEEE TRANSACTIONS ON IMAGE PROCESSING. He is presently Editor for the IS&T/SPIE Journal of Electronic Imaging. He received the Senior (best paper) Award from the IEEE Signal Processing Society and the Bowman Award from IS&T. In 2004, he was named Electronic Imaging Scientist of the Year by IS&T and SPIE.
IEEE TRANSACTIONS ON IMAGE PROCESSING, VOL. 14, NO. 5, MAY 2005
Inkjet Printer Model-Based Halftoning Je-Ho Lee, Member, IEEE, and Jan P. Allebach, Fellow, IEEE
Abstract—The quality of halftone prints produced by inkjet (IJ) printers can be limited by random dot-placement errors. While a large literature addresses model-based halftoning for electrophotographic printers, little work has been done on model-based halftoning for IJ printers. In this paper, we propose model-based approaches to both iterative least-squares halftoning and tone-dependent error diffusion (TDED). The particular approach to iterative least-squares halftoning that we use is direct binary search (DBS). For DBS, we use a stochastic model for the equivalent gray-scale image, based on measured dot statistics of printed IJ halftone patterns. For TDED, we train the tone-dependent weights and thresholds to mimic the spectrum of halftone textures generated by model-based DBS. We do this under a metric that enforces both the correct radially averaged spectral profile and angular symmetry at each radial frequency. Experimental results generated with simulated printers and a real printer show that both IJ model-based DBS and IJ model-based TDED very effectively suppress IJ printer-induced artifacts. Index Terms—Digital halftoning, direct binary search (DBS), error diffusion, inkjet (IJ) printer models.
I. INTRODUCTION
D
IGITAL halftoning is the process of creating a pattern of binary pixels that the eye perceives as a continuoustone image. Digital halftoning is necessary for devices, such as printers, that can only produce a small set of output levels. Methods for digital halftoning may be categorized into three groups according to the computational complexity required to actually render the continuous-tone image in halftone form, independent of the computation required to design the halftoning algorithm [1]. The simplest group consists of point processes, in which each pixel of the halftone image is a function of only one pixel from the continuous-tone image. Screening or dithering, the digital analog of the contact screening process from the graphic arts and printing industry, is the predominant algorithm of this type. The second class includes algorithms that require processing of pixels from a local neighborhood in the continuous-tone image. The dominant example of this type of algorithm is error diffusion. The last and most complex class of algorithms are those that are iterative in nature, requiring several passes through the continuous-tone image to determine the halftone image. Iterative methods for halftoning come in two Manuscript received April 25, 2003; revised May 11, 2004. This work was supported by the Hewlett-Packard Company. The associate editor coordinating the review of the manuscript and approving it for publication was Dr. Reiner Eschbach. J.-H. Lee was with the School of Electrical and Computer Engineering, Purdue University, West Lafayette, IN 47907-1285 USA. He is now with Hewlett-Packard Company, Vancouver, WA 98683 USA (e-mail: [email protected]). J. P. Allebach is with the School of Electrical and Computer Engineering, Purdue University, West Lafayette, IN 47907-1285 USA (e-mail: [email protected]). Digital Object Identifier 10.1109/TIP.2005.843787
flavors—those that find the minimum or maximum of some sort of metric and those that satisfy constraints. In general, implicit in the development of the halftone algorithms is the assumption that the printer is an ideal device, i.e., it always puts ideal square dots at exactly the correct locations. Unfortunately, however, most printers do not obey these assumptions. When halftoning is done without accounting for this fact, the printed textures may not have the appearance predicted by the halftoning algorithm. In order to reproduce the original grayscale image correctly, it is necessary to incorporate within the halftone algorithm a model for the printing process. There has been a great deal of research considering how to model printers and how printer models may be incorporated within a halftoning algorithm. Depending on the approach taken to the modeling, printer models can be classified into two categories, print characterization models and process characterization models [2]. The print characterization models are based entirely on a characterization of the halftone image as it exists on the printed page. The process characterization models reflect the actual process whereby the digital halftone is transformed to colorant on the page. The most widely used print characterization model is the hard circular dot (HCD) model, which was first proposed by Roetling and Holladay [3] to account for spreading of colorant on the medium, which results in increased absorptance of the print. This phenomenon is commonly referred to as mechanical dot gain. In the HCD model, it is assumed that the printer generates a circular spot with constant absorptance at each printer addressable location and multiple dot overlap is resolved by a logical OR at each point in the print. Stucki [4] suggested an extension of error diffusion that incorporates an HCD model. Stevenson and Arce [5] extended error diffusion to the hexagonal lattice and used an HCD model to compensate for neighborhood dot interaction. Pappas et al. [6] popularized the HCD model and generalized the HCD model to a parametric method [7]. Knox et al. [8] developed a similar parametric model. Baqai and Allebach [9] incorporated the HCD model into the direct binary search (DBS) halftoning algorithm, which seeks to minimize the mean-squared error between the perceived rendered halftone and the continuous-tone original image [10], [11]. Lin and Wiseman [12] proposed a stochastic dot model based on modeling the probability of a toner particle occurring at a particular location as being Gaussian distributed around the center of the dot. Flohr et al. [13] also used this model to improve the quality of textures produced by DBS. Baqai and Allebach [9] proposed a stochastic model based on the tabular model, and incorporated it into the DBS algorithm. The origin of process characterization models can be traced from the work by Ruckdeschel and Hauser [14]. They analyzed optical dot gain, also known as the Yule–Nielsen effect, which is the apparent darkening of the print caused by trapping of
1057-7149/$20.00 © 2005 IEEE
LEE AND ALLEBACH: INKJET PRINTER MODEL-BASED HALFTONING
scattered light under colorant dots. Loce et al. [15] proposed a model for the electrophotographic (EP) printing process to analyze banding induced by mechanical vibration. Kacker and Allebach [16] developed a detailed model for the EP process within the DBS halftoning algorithm, focusing on the exposure step, in which the optical photoconductor drum is written with a Gaussian laser beam. The advantage of process characterization models compared to print characterization is that fewer measurements may be required to parameterize the model. However, these models are quite complex in general. Despite these advances, virtually all printer model-based halftoning work has focused on EP printers. The lack of comparable work for inkjet (IJ) printers may be due to the fact that IJ printers do render dots that have a nearly hard, ideal profile, and are much more stable than dots rendered by EP printers. However, as we shall see, IJ printers do exhibit dot-displacement errors that are caused by misaligned nozzles in the printhead, and which appear random in nature. The goal of this paper is to model the IJ printer and then to incorporate this model into halftoning algorithms so that high print quality can be achieved. Here, we develop a stochastic printer model based on the dot statistics of the IJ printer. Our IJ printer model is primarily based on print characterization. To model the IJ printer, we take an approach similar to the one used in [9]: We print a specially designed test pattern, scan the print with a high-resolution scanner, and estimate the dot statistics from the print. However, whereas [9] directly measures the statistics of pixels such as their mean and variance conditioned on a certain neighborhood configuration, we measure the displacement and profile of each individual dot and then infer the conditional pixel statistics based on this information. This enables us to reduce the number of measurements required to model the printer. We incorporate an IJ model based on these statistics into the DBS halftoning algorithm. We also develop a printer model-based version of tone-dependent error diffusion (TDED) [17], which uses tone-dependent error weights and thresholds obtained through a process of training with DBS halftone textures. Experimental results demonstrate the effectiveness of these model-based halftoning algorithms. The remainder of this paper is organized as follows. In Section II, we describe the printer characterization process and the underlying model for the printer based on that characterization. In Sections III and IV, we describe model-based DBS and model-based TDED, respectively, and present experimental results. In Section V, we summarize the key findings of the paper.
II. PRINTER MODEL In this section, we will discuss a stochastic model which is based on a print characterization process. Our paper is focused on high-resolution IJ printers. We use an HP DeskJet 970Cx printer as our test vehicle. As its contemporaries do, this printer supports various print options such as different print resolutions, speeds, directions (uni/bi), number of printing passes, and number of ink drops at each pixel. In fact, the printer reveals various artifacts depending on its print mode. The source of artifacts are typically twofold: dot irregularity and dot-placement error.
675
Fig. 1. Illustration of the two passes used to print the pixels in the shaded region.
The cause of dot irregularity may be either ink coalescence or creation of satellites. Ink coalescence can take place when adjacent nozzles are fired simultaneously. Firing the nozzles at a higher frequency than they can handle may cause puddling around the nozzle opening that can create the satellites. Multiple-pass print modes are usually employed to avoid these printing artifacts [18]. In a multiple-pass mode, the pen visits each pixel more than once and puts a drop there during a certain pass. Therefore, extra logic control is required to determine whether the pen should fire a drop at a given location on a given pass when the pixel at that location is to be marked. Print masks are applied to provide this extra logic. They are usually coded as an array of 0s and 1s. A 1 indicates firing the nozzle, if the halftone pixel at that point has value 1. A 0 means no firing regardless of the value of the corresponding halftone pixel. Fig. 1 [18] shows an example for a two-pass print mode, with print-mask width 4 and height 8. As this mask array is tiled over the media, the pen sweeps across the media and marks every other pixel location if the halftone pixel value there is 1, in a checkerboard pattern. Afterward, the media is advanced half the pen height and the printing resumes. All entries that were missed by M1 the first time are picked up by M2 during the second time when the pen visits. The print masks reflect hardware limitations on firing frequency and avoid consecutive firing of the same nozzle. Dot-placement errors are caused by misaligned nozzles in a print head. Fig. 2 illustrates the nozzle geometry. The print head contains nozzles for three colorants: cyan, magenta, and yellow; each colorant is assigned two columns of the nozzle. From the top view, we can see that the nozzle columns of each colorant are at the edge of the ink feed slot, and the nozzle plate bows down into the ink feed slot as it spans the void. The drop will travel roughly perpendicular to the nozzle plate at the nozzle. However, since the nozzle columns are located on the curved part of the plate, the normals to the nozzle plate at each nozzle column have a slight inward tilt, causing the drop to travel slightly inward. This phenomenon is sometimes referred as a toe-in or dimple effect. In addition to this systematic bias, the dot displacement fluctuates randomly, which suggests the use of a stochastic model. In this paper, we have chosen the 600-dpi, 10 in/s, unidirectional, multiple-pass print mode as our target print mode. Even though the discussion here will be restricted to this print mode,
676
IEEE TRANSACTIONS ON IMAGE PROCESSING, VOL. 14, NO. 5, MAY 2005
Fig. 2.
Structure of print head.
Fig. 3. The test pattern. (a) A portion of the scanned test pattern. (b) A portion of the segmented pattern plus segmentation mask (marked by yellow circles).
our framework for model-based halftoning can be applied to other print modes or other IJ printers with minor modifications. As the first step toward the printer modeling, we characterize the HP DeskJet 970Cx printer. We print a test pattern with the target printer/mode using magenta ink which is the darkest of the three colorants. The printed image is then scanned at 4000-dpi resolution with a Howtek D4000 drum scanner, and the resulting scanned image is transferred to a computer to obtain dot statistics such as the mean and variance of the dot profile, and the horizontal and vertical dot displacements. The test pattern consists of 600 600 pixels where only every fifth column and fifth row contains colored pixels. Hence, there is a total of 14, 400 sample dots in the printed test pattern. Fig. 3(a) shows a portion of the scanned test pattern, and Fig. 3(b) shows a portion of the segmented pattern, plus the segmentation mask (marked by yellow circles). After segmentation, we measure dot profiles and compute the centroid of each dot based on its absorptance. The horizontal (or vertical) displacement of a dot is estimated by computing the difference between its centroid and the average value of the centroids of all the dots in the same row (or column). We measure dot displacements separately for even and odd raster lines, since even and odd lines are printed with different nozzle columns, i.e., each raster will only be printed by the left column or the right column, not by both. Fig. 4 shows the measurement data. As can be seen from the mean and standard deviation of the dot profiles, the rendered dots are reasonably hard. The mean dot
Fig. 4. Measurement data for the HP DeskJet 970Cx printer used in the 600-dpi, 10 in/s, unidirectional print mode. Dot profiles: (a) the mean dot profile and (b) the standard deviation of dot profiles. Histogram of the dot displacements: (c) horizontal dot displacements for even lines, (d) horizontal dot displacements for odd lines, (e) vertical dot displacements for even lines, and (f) vertical dot displacements for odd lines. The vertical dashed lines indicate the mean dot displacements.
size is close to , where in is the spacing between printer addressable spots with this print mode. We can also see from Fig. 4(c)–(f) that dots are significantly displaced horizontally with different mean shifts for even and odd lines, for even lines and for as expected (approximately odd lines). The figures indicate that there also exists nonzero vertical dot displacement, the distribution of which can be approximated as Gaussian. However, we have observed that these vertical placement errors do not cause significant visual artifacts in the rendered image. Hence, we will ignore them in this paper. To ensure consistency of the dot statistics as well as reliability of the measurements, we collected data from five different ink cartridges with three measurements for each pen. The final measurement data shown in Fig. 4 is the average of all 15 of those data sets. We found that all the measurements were quite consistent. In particular, the standard deviation in the mean shift over the 15 data sets was less than 10% for both even and odd lines. Fig. 5 illustrates the artifacts caused by the printer. The most prominent artifacts shown in Fig. 5(b) are vertical white wormy patterns which are introduced by horizontal dot-placement errors.
LEE AND ALLEBACH: INKJET PRINTER MODEL-BASED HALFTONING
677
Fig. 5. Illustration of the artifacts introduced by the random dot-placement error. (a) Halftone image generated by DBS with ideal printer model and rendered with an ideal printer. (b) The scanned HP DeskJet 970Cx printer output for (a). Note the vertical white veins to the left of the propeller and nose cone.
Based on these measured dot statistics, we define a stochastic printer model for the target printer as follows: dot with flat dot profile; • square • random horizontal dot displacement with first-order statistics approximated by the measured data; • no vertical dot displacement; • displacements mutually independent from pixel to pixel. The rationale for the mutual independence assumption is coupled with the print mask for the multiple-pass print mode. It is most likely that adjacent dots will be inked by different nozzles in different passes. Next, we will develop frameworks for IJ printer model-based halftoning in the context of DBS and TDED in Sections III and IV, respectively. III. IJ PRINTER MODEL-BASED DBS A. Overview of DBS DBS is a recursive search heuristic which uses a human visual system (HVS) model and a printer model to minimize a measure of the perceived difference between the rendered halftone and the continuous-tone image [10], [11]. By modeling the low-pass characteristic of the HVS as a linear shift-invariant filtering operation, DBS seeks a halftone image that minimizes
or 1 denote the digital halftone image. The value. Let halftone image rendered by the ideal printer is then given by (2) where is the spot function of the ideal printer is the periodicity matrix whose columns comprise the and basis for the lattice of printer addressable dots. We can express and as the relation between (3) where embodies the effect of cascading the ideal printer rendering and HVS models. With a high-resolution output device such as a 600-dpi printer, the printer spot profile is sufficiently narrow compared to so that . Throughout this paper, we will assume that this relationship holds and refer to the point spread function of the HVS . In addition, we will assume that the continuous-tone as can be also expressed in terms of its samples image according to (2). Then, the error measure in (1) can be written [10], [11] as
(1) and . Here, where denotes the convolution operation, represents the point is the continuous-parameter spread function of the HVS, is the continuous-tone original rendered halftone, and image, which is assumed to lie between 0 (white) and 1 and as the perceived rendered (black). We refer to halftone and the perceived continuous-tone images, respectively. Throughout the remainder of this paper, we represent continuous and discrete spatial coordinates, respectively, with and . vectors In general, the DBS algorithm is implicitly based on an ideal printer model. An ideal printer is one that can reproduce an ideal square dot with absorptance equal to that of the input pixel
(4) where
, , and . Starting with an arbitrary initial halftone, DBS scans through the halftone image in raster order, from left to right and from top to bottom. At each pixel of the halftone image, DBS considers toggling its state, or swapping it with any of its neighbors that differ in state. If any one of these changes reduces the error measure , DBS keeps that change which reduces it the most. One iteration of the algorithm consists of a visit to every pixel in the halftone image. When no changes are retained throughout an entire iteration, the algorithm has converged. This typically requires ten or so iterations.
678
IEEE TRANSACTIONS ON IMAGE PROCESSING, VOL. 14, NO. 5, MAY 2005
^[m] FOR THE DESKJET HP970Cx PRINTER WHERE m = [ 0 1] TABLE I
STATISTICS OF g
{
^[m ]
Note that g is a function of g and their respective displacements.
i; j
,
m = [ ] , m = [ + 1] i; j
i; j
, AND
m = [ + 2] { i; j
[m ], [m ], and [m ], and their respective displacements, while ^[m ] is a function of [m ], [m ], and [m ], g
g
g
The frequency response of the HVS model that we use for DBS is given by [19]
the average absorptance of at [20]; so
(6)
B. Stochastic Printer Model Even though we assume a hard square dot with size for will not be constant within our printer model, the value of cell due to random dot displacements. To account each which is for this, we use an equivalent grayscale image
within the
g
g
cell centered (7)
(5) is the spatial frequency coordinate in cywhere cles/radian subtended at the retina, is the average luminance , and . of the light from the print in cd/m , Taking the inverse two-dimensional (2-D) continuous-space Fourier transform yields the radially symmetric point spread . Here, the spatial coordinate has function units of radians subtended at the retina. To convert to units measured on the printed page, we note that a length when viewed at a distance will subtend an angle radians for . Thus
g
where . The equivalent grayscale image summarizes the microstructure of the halftone dots by a printer addresssingle value that is constant over each able cell. Under the equivalent grayscale model, the rendered halftone image on the page is assumed to be given by (8) The validity of (8) rests with the notion that the human viewer cannot resolve the microstructure of on a scale less than and perceives only an average value. In our printer is a function of neighboring halftone pixel values model, as well as their respective random displacements. Let dedenote the set of pixels in the neighborhood of , and let note the displacement vector at pixel . Then, this dependency can be expressed as (9) to only Since we assume that dots can be misplaced from is a 1 3 neighborhood of in the horizontal direction,
LEE AND ALLEBACH: INKJET PRINTER MODEL-BASED HALFTONING
679
Fig. 6. DBS results with the simulated never-centered printer (printed at 100 dpi). Halftone image generated by DBS with no printer model and rendered with (a) an ideal printer and (b) the never-centered printer. (c) Halftone image generated by DBS with the never-centered printer model and rendered with the never-centered printer.
. The dot-displacement is a stochastic process; so, is also a random 2-D sequence. Whether refers to a random process or one realization of that random process will be clear from the context.
a random quantity, we will take its expectation with respect to . Then, DBS seeks the digital halftone that minimizes (10)
C. Error Metric for Printer Model-Based DBS To put the equivalent grayscale model in the context of DBS, we first substitute into (4). Since is now
The error measure is a statistical average over the ensemble of all possible realizations of the sample mean-squared perceived
680
IEEE TRANSACTIONS ON IMAGE PROCESSING, VOL. 14, NO. 5, MAY 2005
Fig. 7. DBS halftone images rendered with the simulated bi-displacement printer (printed at 100 dpi) (a) generated by DBS with no printer model, (b) generated by DBS with the mean dot model for the bi-displacement printer, and (c) generated by DBS with the stochastic model for the bi-displacement printer.
rendered error for a given digital halftone. Substituting (4) into (10), we obtain, yet, another expression for , which is (11)
(12) Equation (12) has a nice interpretation. The first term on the right hand side is the error metric for a deterministic model, , which measures the perceived difference based on
between the ensemble average of the rendered halftone and the original image for a given digital halftone. The second term accounts for random fluctuations between adjacent dots. This term will penalize the use of local halftone textures that are vulnerable to dot-placement errors. If the range of dot displacements is relatively small, there would exist only limited dot interaction between neighboring dots. In this case, we may approximate by the the first term in (12) and use it as an error metric denoted by (13)
LEE AND ALLEBACH: INKJET PRINTER MODEL-BASED HALFTONING
681
We will refer to DBS algorithms based on error measures and as DBS with a stochastic dot model and DBS with a mean dot model, respectively. is computationally The direct evaluation of either or prohibitive. We will, therefore, describe an efficient method to evaluate the effect of trial halftone changes, which is similar to that reported for conventional DBS [10], [11]. Our derivation of the method for efficient evaluation of halftone changes will be restricted to the stochastic dot model only, since the mean dot model may be viewed as a special case of this more general model. Let us, then, consider the effect of a trial change on . We denote everything after the change by a prime. From (11), the new error can be expressed as
as well as its neighbors , for generally alter . Therefore, the change in due to the toggle at can be written as for (19) for Substituting (19) into (18), the change in error can be expressed as
(20) Similarly, we have the following equation for the change in error and due to a trial swap between pixels
(14) Then, we have
(21) (15) . If the spatial because distance between and is sufficiently large, there will be no interaction between the pixels and , and we can assume that and are independent, which implies that the same is true for and . More specifically, for the displacement and (or and ) models that we shall consider, are only dependent if and are neighbors of each other. Thus, we have if if where fine
(16) . Now, let us deto be the cross correlation between and
Table I summarizes the statistics of based on the measurements in Section II. The values of , , and will be precomputed based on Table I, and stored in a LUT. We also can precompute and store it in a LUT, since it is independent of the halftone. However, depends on the halftone; so, we have to update it whenever we accept a trial change. The extent . The update of the changes is limited by the support of equation is given by
if accept toggle if accept
(17) By substituting (16) and (17) into (15), the change in error can be expressed as
(18) Consider the effect on the error metric of a trial toggle at in the halftone. The change in would any location
swap. (22) D. Experimental Results for IJ Printer Model-Based DBS In this section, we evaluate the efficacy of using the printer model with DBS. We will show results for DBS with the mean dot model and the stochastic dot model and compare them with DBS with an ideal printer model. With any type of printer model, there are two important, and somewhat distinct questions to be addressed. The first one is how well DBS can exploit the printer model to create textures that are visually pleasing yet robust to the limitations of the printing technology, as they are represented by the model. The second question is how well
682
IEEE TRANSACTIONS ON IMAGE PROCESSING, VOL. 14, NO. 5, MAY 2005
Fig. 8. DBS halftone images rendered with the simulated correlated-displacement printer (printed at 100 dpi) (a) generated by DBS with no printer model, (b) generated by DBS with the mean dot model for the correlated-displacement printer, and (c) generated by DBS with the stochastic model for the correlated-displacement printer.
the model actually characterizes the printer. To address the first question without complication from the second, we use simulated printers. 1) Simulated Printer Results: Our first simulated printer model is a “never-centered” printer. As its name implies, this synthetic printer always prints dots with a fixed nonzero displacement, such that every dot on even raster lines is shifted and every dot on odd lines is shifted by horizontally by . This is a deterministic printer. Therefore, a stochastic dot model and a mean dot model for this printer will produce identical halftone images. Fig. 6 compares images rendered by the never-centered printer using DBS halftones generated
with no printer model and with the never-centered printer model. For comparison, we also show the result of printing on an ideal printer the DBS halftone generated with no printer model. These images were all printed at 200-dpi resolution simulated printer using 2 2 real printer dots for each addressable cell. Hence, the print is actually 100 dpi. The DBS halftone generated with no printer model and rendered with the never-centered printer [Fig. 6(b)] shows white worm artifacts throughout the sky behind the aircraft, noisy texture on the fuselage of the aircraft, and loss of detail in the word “SIX SHOOTER” and the image of Yosemite Sam. The DBS halftone generated with the printer model and rendered with
LEE AND ALLEBACH: INKJET PRINTER MODEL-BASED HALFTONING
683
Fig. 9. HP DeskJet 970Cx printer outputs (scanned at 4000 dpi). (a) DBS with no printer model. (b) DBS with the mean dot model for this printer. (c) DBS with the stochastic dot model for this printer.
the never-centered printer [Fig. 6(c)] shows none of these problems, and achieves almost the same level of quality as a DBS halftone generated with no printer model and rendered with an ideal printer [Fig. 6(a)]. We developed a second simulated printer model which has characteristics similar to those of our target printer, the HP 970Cx. This simulated printer behaves in a stochastic way such that dots on even/odd raster lines are shifted horizontally by (0 )/(0 or ) with equal probability. The dot-placement or errors are mutually independent. The mean dot shifts for even and , respectively, which and odd lines are, thus, matches our measurement for the target printer. We will call this printer the “bi-displacement” printer. Simulation results are shown in Fig. 7. Both the mean [Fig. 7(b)] and stochastic [Fig. 7(c)] dot models significantly reduce the white worm artifacts in the highlight tone patches and the veining artifacts and graininess in the mid tone patches that can be seen in the halftone [Fig. 7(a)] generated with no printer model. The halftones generated by the mean and stochastic dot models are
very similar in appearance. This is due to the very limited interaction between dots, which makes the impact of the covariance term in (12) minimal. In order to show the merit of the stochastic dot model over the mean dot model, we developed a third simulated stochastic printer. This simulated printer has a larger range of horizontal for odd lines) than dot shift (0 or for even lines and 0 or the previous printer does. Also, the dot-displacement errors are now strongly correlated for those dots on the same line. For the first dot on each line, the dot displacement is equally likely to for odd lines. For the rebe 0 or for even lines and 0 or maining dots on the line, the displacements are governed by a sequence of independent, identically distributed random variables , that take on value 1 with probability 0.8 and value 1 with probability 0.2. If , then the displacement for the th dot is the same as that for the th dot. Otherwise, it has the opposite value. For example, on an even-numbered th dot is 0 and , line, if the displacement of the then the displacement for the th dot will be . We will call
684
IEEE TRANSACTIONS ON IMAGE PROCESSING, VOL. 14, NO. 5, MAY 2005
Fig. 10. Comparison of two error measures. (a) A bitmap for absorptance 57/255. (b) The power spectrum of (a). (c) A second bitmap for the same absorptance. (d) The power spectrum of (c).
this printer the “correlated-displacement” printer. Simulation results for this printer are shown in Fig. 8. As before the halftone generated with no printer model [Fig. 8(a)] shows objectionable white worm artifacts in light tone patches and severe graininess in the mid-tone patches. Both the mean [Fig. 8(b)] and stochastic [Fig. 8(c)] dot models reduce those artifacts significantly. However, the stochastic model clearly shows its benefit over the deterministic model for this printer, especially in the mid-tone patches. Here, we still see horizontal banding and graininess in the halftone generated with the mean dot model. These artifacts are much less noticeable with the correlated-displacement model. 2) Real Printer Results: The images printed by the actual HP DeskJet 970Cx printer are shown in Fig. 9. In order to demonstrate the performance of each halftoning algorithm more clearly, we scan each print at 4000 dpi. As with the simulated printers, lots of objectionable white wormy patterns can be observed in the rendered halftone image generated by
DBS with no printer model [Fig. 9(a)]. Both DBS with the mean and stochastic dot models get rid of this artifact and yield more homogeneous patterns as seen in Fig. 9(b) and (c). IV. IJ PRINTER MODEL-BASED TONE-DEPENDENT ERROR DIFFUSION A. Overview of TDED Despite its capability to produce the highest level of print quality, DBS is not practical for most desktop printing applications because of its computational complexity, which requires several passes through the continuous-tone image to determine the final halftone image. For this reason, error diffusion, which was first proposed by Floyd and Steinberg [21], has been widely used in desktop printing applications. Since Floyd and Steinberg’s seminal work, much effort has been expended toward improving the quality and extending the capabilities of error diffusion. This includes using different weight sets [22], threshold
LEE AND ALLEBACH: INKJET PRINTER MODEL-BASED HALFTONING
Fig. 11.
685
Block diagram for training IJ printer model-based TDED.
modulation [23], different scan paths [22], [24], and an adaptive filter [25]. While these methods have been shown to be very effective for improving halftone quality, the resulting quality is still not comparable to that produced by search methods such as DBS. In recent years, tone-dependent methods that use different weight sets for different input pixel values have been developed. Eschbach used a larger weight set for extreme gray levels and a smaller set for midtones [26]. Ostromoukhov designed a method that uses a two-term weight set for extreme gray levels and a four-term weight set for midtones [27]. He later [28] optimized the weight set by minimizing the difference between the Fourier spectra of certain gray levels and the corresponding blue noise spectra [22]. Li and Allebach [17] used tone-dependent weight and threshold sets in their TDED algorithm, where the weight and threshold sets are chosen through an off-line training process. In the TDED algorithm, the error weights and threshand , respectively, are functions of olds, denoted by is determined the input pixel value . The binary output according to if otherwise.
(23)
Here, is the modified continuous-tone image, and the threshold function is given by if otherwise where
and
the input continuous-tone image prior to thresholding, or turn . off this modulation by setting is thresholded, the quantization error is computed Once as (27) and distributed to future pixels according to the tone-dependent weights (28) where denotes the region of support of the weight set. In order to obtain the optimal weights and thresholds, Li and Allebach employed the frequency weighted mean-squared error measure (1). However, they found that although this metric can generally indicate the smoothness of the binary texture, it sometimes fails to measure the texture variety and homogeneity. It works well for DBS because DBS only finds a local minimum of the metric starting from a random initial halftone that is directionally unbiased and has good texture variety. When optimizing the weights and thresholds for TDED, the initial halftone texture is generated by error diffusion and, thus, may be inhomogeneous and lacking in variety, especially in the midtones. For this reason, Li and Allebach used the error measure (1) only for the extreme levels (from 1 to 20 and from 235 to 254). For the remaining levels, they employed the measure
(24) (29)
are tone-dependent parameters satisfying (25) (26)
with period and the doubly periodic function is a halftone image for absorptance 0.5 generated by DBS. The thresholding operation given in (24) can be interpreted as a conditional threshold modulation process, i.e., with proper values of and , TDED can add the optimal amount of dither signal to
based on matching the magnitude of the spectrum of the DBS texture for the corresponding gray level with that for a given weight set of the TDED spectrum and threshold . The training process then seeks the parameter sets and that minimize this error measure. The search strategy used in [17] was based on a simple multigrid search method, which requires intensive computation. However, once the optimal sets are obtained from this
686
IEEE TRANSACTIONS ON IMAGE PROCESSING, VOL. 14, NO. 5, MAY 2005
Fig. 12. TDED results with the simulated never-centered printer (printed at 100 dpi). Halftone image generated by TDED with no printer model and rendered with (a) an ideal printer and (b) the never-centered printer. (c) Halftone image generated by TDED with the never-centered printer model and rendered with the never-centered printer.
off-line process, TDED requires exactly same order of computation as does conventional error diffusion.
B. Error Metric and Training Strategy for IJ Printer Model-Based TDED The success of TDED has demonstrated the power of training-based methods for halftoning algorithm design. With
this capability, it would be advantageous to be able to train the algorithm to avoid textures that exhibit noise artifacts which are intrinsic to the IJ process. Thus, we train the TDED parameters so that the spectrum of the TDED halftone image rendered under a model for the IJ process will be similar to that of the ideal DBS halftone image. It should be noted that with IJ model-based DBS, we actually embed the IJ printer model in the halftoning algorithm. With TDED, however, the IJ printer model is only used as part of the training process.
LEE AND ALLEBACH: INKJET PRINTER MODEL-BASED HALFTONING
687
Fig. 13. TDED results with the simulated bi-displacement printer (printed at 200 dpi). Halftone image generated by TDED with no printer model and rendered with (a) an ideal printer and (b) the bi-displacement printer. (c) Halftone image generated by TDED with the bi-displacement printer model and rendered with the bi-displacement printer.
The architecture of the core TDED algorithm remains the same as before. The training process for the IJ printer model-based TDED seeks the parameter sets and that minimize
(30)
grayscale image with the parameter sets and for the cor, we first responding gray level . To estimate , generate 50 128 128 TDED halftone images , and then compute the autocorrelation functions based on the printer model. The discrete Fourier transform (DFT) of the autocorrelation , is given by function, which is denoted by
where and are, respectively, the power spectra of the DBS halftone and the TDED equivalent
(31)
688
for . Here, and mutually independent, unless other. Thus, (31) can be written as
IEEE TRANSACTIONS ON IMAGE PROCESSING, VOL. 14, NO. 5, MAY 2005
and are are neighbors of each
(32) for
. Then, can be approximated by , which is the sample average of over . Similarly, we approximate the power spectrum , the sample average of the of the DBS texture by DFTs of the 50 DBS halftone images as in [17]. However, we have observed that globally minimizing the spectral difference based on the total mean squared error often leads to an undesirable rendered TDED halftone image. The power spectrum of the DBS pattern has a characteristically blue noise nature. This suggests a new error metric which somehow accounts more explicitly for this spectral property. We adopt two metrics proposed by Ulichney in [22] and define a new error metric based on the combination of these two metrics. We use the radially averaged power spectrum of and its anisotropy, which are denoted by and , respectively, where . They are defined as (33) (34) Here, is the number of samples at a given radial frequency and is the sample variance of those samples of . In our new training strategy, we compute the difand , and measure the ference between anisotropy of . The resulting error metric is given by (35) where is a parameter that controls the relative emphasis of the two terms in the error metric. We determined empirically that yields good results. The first term in matches the radially averaged power spectrum of the DBS texture with that of the TDED halftone. The second term explicitly encourages radial symmetry in the TDED power spectrum. Fig. 10 illustrates the efficacy of the error measure over the measure . Although the halftone in Fig. 10(c) has a larger value than the halftone in Fig. 10(a), it has more homogeneous texture and a more radially symmetric spectrum. On the other hand, the resulting values of agree with these observations. To determine the optimal parameter set, we use the derivative-free downhill simplex optimization method due to Nelder and Mead [29]. We
Fig. 14. HP DeskJet 970Cx printer outputs (scanned at 4000 dpi). (a) TDED with no model for this printer and (b) TDED with printer model for this printer.
have found that this optimization algorithm reduces the training time by a factor of ten compared to the one used in [17]. Fig. 11 summarizes the training algorithm for IJ printer model-based TDED based on the error metric defined in (35). C. Experimental Results for IJ Printer Model-Based TDED We evaluate the IJ printer model-based TDED halftoning algorithm by applying it to two of the same simulated printers and the same real printer as in Section III with the same test images (the mustang and step wedge images). The results are shown in Figs. 12–14. The simulation results for TDED shown in Figs. 12 and 13 parallel those for DBS very closely. The TDED halftones generated with no printer model and rendered with the nonideal simulated printers [Figs. 12(b) and 13(b)] suffer from artifacts similar to those seen earlier with DBS. On the other hand, TDED trained with the IJ printer model [Figs. 12(c) and 13(c)] achieves almost the same level of quality as a TDED halftone generated with no printer model and rendered with an ideal printer [Figs. 12(a) and 13(a)]. The images printed by the HP DeskJet 970Cx printer are shown in Fig. 14. Again, TDED trained with the IJ printer model yields improved print quality by suppressing artifacts such as white worms and veins, which are easily observed in the rendered halftone generated by TDED with no printer model.
LEE AND ALLEBACH: INKJET PRINTER MODEL-BASED HALFTONING
V. CONCLUSION In this paper, we introduced IJ printer model-based halftoning algorithms within the frameworks of DBS and TDED. We determined the printer model based on a printer characterization process, and we successfully incorporated it into the existing halftoning algorithms. For model-based DBS, we proposed two different error metrics to handle the random dot-placement errors in the printer and showed that both work well. We also showed that the algorithm can be efficiently implemented by recursively evaluating the effect of trial changes. For model-based TDED, we proposed a training method that encourages TDED to mimic the spectral characteristics of DBS. For this purpose, we used a new error metric based on the radially averaged power spectrum and anisotropy. We also employed a new optimization method, the downhill simplex method, which reduces the training complexity dramatically. The printer model is incorporated within the halftoning algorithm only through the training of the error diffusion parameters. The structure of the TDED algorithm itself is not modified. Both the model-based DBS and TDED halftoning algorithms described here remove most of the visual artifacts caused by the printer. Although this paper has focused on a specific inkjet printer—the HP DeskJet 970cx—the modeling approach that we presented is quite general and could be applied to other printers, as well. REFERENCES [1] J. P. Allebach, Ed., Selected Papers on Digital Halftoning. Bellingham, WA: SPIE, 1999. [2] J. P. Allebach, “DBS: retrospective and future directions,” in Proc. SPIE Color Imaging: Device-Independent Color, Color Hardcopy, and Graphic Arts VI, 2001, pp. 358–376. [3] P. G. Roetling and T. M. Holladay, “Tone reproduction and screen design for pictorial electrographic printing,” J. Appl. Photo. Eng., vol. 15, no. 4, pp. 179–182, 1979. [4] P. Stucki, “MECCA—A multiple-error correcting computation algorithm for bilevel image hardcopy reproduction,” IBM Res. Lab., Zurich, Switzerland, Res. Rep. RZ1060, 1981. [5] R. L. Stevenson and G. R. Arce, “Binary display of hexagonally sampled continuous-tone images,” J. Opt. Soc. Amer. A, vol. 2, no. 7, pp. 1009–1013, Jul. 1985. [6] T. N. Pappas and D. L. Neuhoff, “Model-based halftoning,” in Proc. SPIE Human Vision, Visual Processing, and Digital Display II, 1991, pp. 244–255. [7] T. N. Pappas, C. Dong, and D. L. Neuhoff, “Measurement of printer parameters for model-based halftoning,” J. Imag. Technol., vol. 2, no. 3, pp. 193–204, Jul. 1993. [8] K. Knox, C. Hains, and G. Sharma, “Halftone-independent calibratin of black & white printers,” in Proc. SPIE Human Vision and Electronic Imaging, 1996, pp. 432–436. [9] F. A. Baqai and J. P. Allebach, “Halftoning via direct binary search using analytical and stochastic printer models,” IEEE Trans. Image Process., vol. 12, pp. 1–5, Jan. 2003. [10] M. Analoui and J. P. Allebach, “Model based halftoning using direct binary search,” in Proc. SPIE Human Vision, Visual Processing, and Digital Display III, 1992, pp. 96–108. [11] D. J. Lieberman and J. P. Allebach, “A dual interpretation for direct binary search and its implications for tone reproduction and texture quality,” IEEE Trans. Image Process., vol. 9, no. 11, pp. 1950–1963, Nov. 2000. [12] Q. Lin and J. Wiseman, “Impact of electrophotographic printer dot modeling on halftone image quality,” in SID Int. Symp. Tech. Papers, Seattle, WA, May 1993, pp. 147–150. [13] T. J. Flohr and J. P. Allebach, “Halftoning via direct binary search with a stochastic dot model,” presented at the IS&T’s 3rd Tech. Symp. Prepress, Proofing and Printing, Chicago, IL, Nov. 1993. [14] F. R. Ruckdeschel and O. G. Hauser, “Yule-Nielsen effect in printing: a physical analysis,” Appl. Opt., vol. 17, no. 21, pp. 3376–3383, Nov. 1978.
689
[15] R. P. Loce, W. L. Lama, and M. S. Maltz, “Modeling vibration-induced halftone banding in a xerographic laser printer,” Electron. Imag., vol. 4, no. 1, pp. 48–61, Jan. 1995. [16] D. Kacker, T. Camis, and J. P. Allebach, “Electrophographic process embedded in direct binary search,” IEEE Trans. Image Process., vol. 11, no. 3, pp. 234–257, Mar. 2002. [17] P. Li and J. P. Allebach, “Tone-dependent error diffusion,” IEEE Trans. Image Process., vol. 13, no. 2, pp. 201–215, Feb. 2004. [18] J. Yen, M. Carlsson, M. Chang, J. Carcia, and H. Nguyen, “Constraint solving for inkjet print mask design,” J. Imag. Sci. Technol., vol. 44, no. 5, pp. 391–397, Sep./Oct. 2000. [19] R. Näsänen, “Visibility of halftone dot textures,” IEEE Trans. Syst., Man, Cybern., vol. 14, no. 6, pp. 920–924, Dec. 1984. [20] T. N. Pappas and D. L. Neuhoff, “Printer models and error diffusion,” IEEE Trans. Image Process., vol. 4, no. 1, pp. 66–80, Jan. 1995. [21] R. W. Floyd and L. Steinberg, “An adaptive algorithm for spatial grayscale,” in Proc. SID, 1976, pp. 75–77. [22] R. A. Ulichney, Digital Halftoning. Cambridge, MA: MIT Press, 1987. [23] K. T. Knox, “Error diffusion: a theoretical view,” in Proc. SPIE Human Vision, Visual Processing, and Digital Display IV, 1993, pp. 326–331. [24] I. H. Witten and R. M. Neal, “Using Peano curves for bilevel display of continous-tone images,” IEEE Comput. Graph. Appl., vol. CGA-2, no. 3, pp. 47–52, May 1982. [25] J. Lee, Y. Kwon, and H. Kim, “A color halftoning algorithm for low-bit flat panel displays,” in Proc. IEEE Int. Conf. Image Process., vol. II, Washington, DC, Oct. 1995, pp. 331–334. [26] R. Eschbach, “Reduction of artifacts in error diffusion by means of input-dependent weights,” J. Electron. Imag., vol. 2, no. 4, pp. 352–358, Oct. 1993. [27] V. Ostromoukhov, “Enhanced error-diffusion method for color or blackand-white reproduction,” U.S. Patent 5 737 453, 1998. , “A simple and efficient error-diffusion algorithm,” presented at [28] the SIGGRAPH, Los Angeles, CA, Aug.. 2001. [29] W. H. Press, S. A. Teukolsky, W. T. Vetterling, and B. P. Flannery, Eds., Numerical Recipes in C, 2nd ed. New York: Cambridge Univ. Press, 1992. Je-Ho Lee (M’00) received the B.S. degree in electrical engineering from the Hanyang University, Seoul, Korea, the M.S. degree in electrical engineering from the Pennsylvania State University, Philadelphia, and the Ph.D. degree in electrical engineering from Purdue University, West Lafayette, IN, in 1990, 1992, and 2002, respectively. From 1992 to 1997, he was with the Korea Institute of Science and Technology as a Research Staffer. He is currently an Imaging Scientist with HewlettPackard Company, Vancouver, WA. His research interests include digital halftoning, image enhancement, color image processing, and image compression.
Jan P. Allebach (F’91) received the B.S.E.E. from the University of Delaware, Newark, and the Ph.D. degree from Princeton University, Princeton, NJ, in 1972 and 1976, respectively. He was on the faculty at the University of Delaware from 1976 to 1983. Since 1983, he has been with Purdue University West Lafayette, IN, where he is the Michael J. and Katherine R. Birck Professor of Electrical and Computer Engineering. His current research interests include image rendering, image quality, color imaging and color measurement, and document management. Prof. Allebach is a member of the IEEE Signal Processing (SP) Society, the Society for Imaging Science and Technology (IS&T), and SPIE. He has been especially active with the IEEE SP Society and IS&T. He is a Fellow of both these societies, has served as Distinguished/Visiting Lecturer for both societies, and has served as an Officer and on the Board of Directors of both societies. He is a past Associate Editor for the IEEE TRANSACTIONS ON SIGNAL PROCESSING and the IEEE TRANSACTIONS ON IMAGE PROCESSING. He is presently Editor for the IS&T/SPIE Journal of Electronic Imaging. He received the Senior (best paper) Award from the IEEE Signal Processing Society and the Bowman Award from IS&T. In 2004, he was named Electronic Imaging Scientist of the Year by IS&T and SPIE.
