Region-based Watermarking for Images - Semantic Scholar
Region-based Watermarking for Images Gareth Brisbane†, Rei Safavi-Naini† and Philip Ogunbona‡ †School of IT and CS University of Wollongong, Northfields Ave. Wollongong 2522, Australia
‡Digital Technology Research Lab Motorola Australian Research Centre
E-mail: {gareth, rei}@uow.edu.au, [email protected] Abstract. There is a spate of research activities investigating digital image watermarking schemes in an effort to control the problem of illegal proliferation of creative digital data across the Internet. Several schemes, with their merits and demerits, have been proposed to date. However, many of them rely on the presentation of a test image in an appropriate format before the detection can proceed. In most cases this is achieved through some form of pre-processing. This paper proposes a technique, using image region segmentation, which alleviates many of the problems associated with pre-processing and thus allowing the detection process to be more independent as well as reducing the need for human intervention.
1. Introduction Digital information is now readily available due to advances in the compression, storage and communications technologies. The amount of digital information that can be found on the Internet and the popularity of the Internet corroborate this observation.
Unfortunately, the protection of this
information, especially in circumstances where the owners hope to generate revenue through controlled dissemination, is yet to be standardized. There are at least two consequences of the status quo. First, digital information that is already available is being illegally re-distributed and thereby robbing the legal owners of deserved revenue. Second, this situation discourages content generators from sharing their work with the wider community. A solution to the latter case is the development of proprietary techniques to insert and verify digital watermarks. Several techniques have been proposed in the literature to insert watermarks in digital images. These techniques provide various degrees of robustness. Watermarking applications include, copyright protection,
authentication, embedded
and
hidden annotations,
and
secure
and invisible
communications. Watermarking systems that are intended for copyright protection are likely to be subjected to a wide range of attacks that are aimed at removing the watermark, or simply making it ineffective and so a very high degree of robustness is required. Watermarking systems for authentication belong to the fragile class of schemes where the slightest change to the image completely destroys the mark and hence the authenticity of the image. Finally watermarking for embedding annotations and secure and invisible communication require resistance against moderate level of modification due to routine image manipulations including digital to analog conversion and vice versa. In this paper we are concerned with a group of attacks on a watermarked image that result in the loss of synchronization and hence failure of the watermark verification process. By synchronization we mean correct correspondence between pixels: that is the ith pixel in the original image remains the ith pixel in the image under verification test. The loss of synchronization may be due to intentional attacks,
such as deleting a row or a column of pixels, or unintentional causes such as misalignment during the scanning of the image. The problems of misalignment and loss of synchronization are discussed in some detail in [1], where the authors reviewed several techniques for overcoming these problems. The solution proposed is to repeatedly test shifted versions of the verification signal, searching for a high peak to indicate correlation. However, the application of this scheme is only discussed in terms of binary insertion and detection, which places in the class of fragile watermark. Also, the notion of continuously testing different signals to search for matches is unappealing. Another attempt to produce a region-based watermarking algorithm was by Schneider and Chang [2], where individual objects in an image were targeted for watermarking. However, this was a procedure that required manual selection of image regions for both insertion and detection. Our objective is to use image segments, which may not necessarily correspond to an object, but are generated by a simple segmentation algorithm. Segmentation methods can be as simple as partitioning an image into tiles, or through the use of image features that capture the quantitative description of a homogeneity criterion. Region growing, clustering and watershed algorithms are some examples of feature based segmentation. In general, the pixels within each segmented region will be similar according to some criteria. Simple image partitioning or tiling is commonly used for increasing efficiency in watermarking systems. The problem of efficiency arises when image sizes do not lend themselves to operation by a fast transformation algorithm. An example can be found in the case of the DCT where the fast radix-2 N
algorithm is well known. Images with sizes that are not multiples of 2 , where N is an integer, have to be tiled into sub-images of sizes amenable to the fast algorithm. We propose segmentation for efficiency, robustness against synchronization loss and localization. Using segments and breaking the verification process into smaller chunks results in more efficient computation. This is especially true when a hardware implementation of the watermarking process is required. Using robust segmentation algorithms and inserting the watermark in selected secret segments will make attacker’s attempts to cause synchronization loss much more difficult. Lastly, it may be required to localize the regions of the image where possible changes have to be detected. This has the advantage of potentially allowing a form of group watermarking, where each member can place a watermark in a different component of the image. Segmenting an image into regions and inserting the watermark only in the desired regions allows various levels of protection for different part of an image. However the use of segmentation requires careful design of the system as it could adversely effect synchronization loss because of the introduction of many regions that need to be individually synchronized. The segmentation process during verification process must recover exactly the same regions that have been used during the insertion process. In other words, segments must remain invariant after the watermark insertion process.
For example, in [3], the authors proposed the use of an invariant domain to store the watermark information. An invariant domain is an area of the image that will always be perfectly reconstructed irrespective of the changes introduced to the image. In [3], the domain proposed is invariant to rotation, scaling and translation; such attacks will not have any effect on the retrieval of the watermark. Fleet and Heeger [4] introduced a watermarking scheme that is rotation and scale invariant. The insertion of the watermark was in the Yellow-Blue colour space of the image and invariance was achieved by modifying the amplitude of regularly spaced sine waves inserted in this domain. At a later stage, these waves can be recovered using the same technique and the image can be realigned and re-scaled to its original dimensions. Cox et al use the whole of frequency range to provide an invisible yet robust domain to insert the watermark [5]. While this domain is robust, it is not invariant, as changes in the spatial domain via rotation for example, will substantially affect the reconstruction of the domain. We propose a simple magnitude based ordering of pixels before the insertion process; the ordering is necessarily undone before verification takes place. The reasoning behind the ordering is that this process relies on the relative magnitude of data, in this case pixel values, and so is independent of the orientation of a segment during the re-scanning of the image. We note that in each of the abovementioned techniques, the domain of insertion is publicly accessible, although the location of the watermark itself may be preserved by use of the owner’s secret information. Using segmentation allows us to hide the domain of insertion if required. The rest of this paper is organized as follows. In section 2 we develop the intuition behind our approach and provide justification for its operation. In section 3 we give the outline of an implementation using a simple segmentation method in conjunction with the scheme proposed by Cox et al. Our experimental output is documented in section 4. Section 5 then provides some concluding remarks.
2. Segmentation in Watermarking Watermarking consists of two complementary processes: watermark insertion and watermark verification. During the insertion process a watermark signal, w, is embedded into a host object I and produces a watermarked object
I (w) . The watermarked object might undergo a range of intentional
and unintentional modifications. The aim of the verification process is to produce evidence in support of the existence of the watermark. The verification algorithm produces a true result by recovering the watermark, possibly in a distorted and weakened form, but with appreciable signal strength that supports existence of the mark with a high probability. The main challenge in robust watermarking is to ensure that the system produces correct results. In other words, verification should produce true for all valid marks, and false for all fraudulent objects, if the object has been subjected to a range of intentional and unintentional modifications.
It is important to note that because of various
unintentional modifications, such as filtering and compression, certain degree of robustness is necessary for all watermarking applications. Furthermore, synchronization is lost if an object is not perfectly aligned and may result in the failure of the verification process. To illustrate this problem, consider the case of computing a transform (e.g. DCT) of a tiled image in order to extract the watermark signal. The image needs to be properly aligned so that the same tiles used in the insertion process can be recovered. If the watermarked image were visually imperceptibly skewed or rotated the
verifier requires knowledge of this process to be able to perform the verification process accurately. This problem remains even when the whole image (rather than tiles) is used during the insertion process. Segmentation-based watermarking attempts to provide the required invariance property when verifying a watermark in the scenarios described above. Region segmentation based on textural properties of the image can be robust to affine transformations if appropriate descriptors are used. It is also robust against minor changes aimed at removing the strict correspondence between the original image and its watermarked form. Such image modifications might include row (or a column) removal, substitution of a row (or a column) with the average of adjacent ones, line insertion and resizing.
2.1 The System We assume an original image, or colour. Let
I s denote the s
th
I , consisting of an array N × M of pixels which could be grayscale segment of the image, where each segment consists of a subset of
pixels forming an arbitrarily shaped area of the image and obtained through a possibly human-assisted algorithm. Our definition of a segment is assumed broad enough to subsume the case of tiling and the case where a human user selects areas of an image as segments. Let S be the total number of segments. Each segment
I s from the image can be ordered into a sequence, x s , of pixels, with the
i th component denoted as x s (i ). X denotes the set of all such segments. The watermarking process usually requires some input called key and denoted by k which is only accessible to the owner of the object or in general the person who inserts the mark. We use K to denote the set of possible keys and
f K (x ) to denote a key-dependent transformation on an input segment, x s ; f K (x ) is for example a watermark insertion algorithm. A watermark detection algorithm,
g K (x, y ′), produces an estimate of
the likelihood that an image segment represented by the sequence
y ' contains the watermark. In other
y ′ = f k (x ) . For an image with segments y s , the likelihood of being derived through
words,
watermarking
from
the
image
segment
set
{x s ∈ X ; s = 1,, S }
is
calculated
as
S
W = ∑ g k (x s y s ) . s =1
2. 2 Producing Invariance in Segments Consider the simplest case where the number of segments, S , is equal to one. In other words the whole image is used for insertion and detection. One drawback of the whole image as one segment is computational and implementation inefficiencies. The system is also prone to loss of synchronization if the sequence, x s (in this case the whole image), derived from the original image does not bear a oneto-one correspondence to the sequence, y s , derived from the watermarked image. The problem faced when S > 1 is no less, because any inaccuracy in determining the segment boundaries will have a cumulative effect when more segments are considered.
The detection function
g k (x, y ) can be seen as a correlator that provides comparison between the
two vectors and ensures that the differences between the two are predictable by k. If the correspondence between segments of
I and I (w) is not accurate the correlation measure fails. Note
that the loss of segment synchronization can occur if a simple transformation such as cropping or adding an extra column to the image is performed. It is therefore necessary to reverse the operation before any comparison between two sequences are made. In the case of cropping, the remaining portion of the picture must be replaced in the original position, so that the correct neighbourhood can be reconstructed for detection (as indicated in [5]). If a column has been added, then it must be removed to regain synchronization. Finally for segmentation to work properly, we must ensure that each sequence
x s relates to y s on a point-to-point basis. This is achieved through the ordering mechanism described before. It is desirable that it be able to detect the change in segment boundaries and recognize the need to realign the segments. This requirement is particularly important with schemes that depend on inserting a watermark at certain positions in the image, e.g. “Patchwork”[6]. Here we have a fixed point somewhere within
I , the reference image. By mapping the watermarked image I (w) onto I we can
recover the differences and indicate the presence of the watermark. If the reference point is no longer valid, even if it has only been slightly tampered with, then the comparison cannot be correctly made. To counter these problems, some attempts have been made to provide an invariant domain of operation to allow
g k (x, y ) to work correctly, irrespective of changes made to the image [3]. In their
work [3], Ó Ruanaidh and Pun achieved invariance by transforming the image through the Fast Fourier Transform, then through the Log Polar Transform and finally through another Fast Fourier transform. The end result of this process is a domain that is invariant to rotations, scaling and translations. However, cropping is an effective attacking tool, which does not degrade the appearance of the original image, although it does reduce the amount of original information. Another technique attempts to introduce a small mask of sine waves across the image [4], which can be retrieved at a later stage to indicate how the image has been stretched or rotated. Unfortunately, these are produced so as to be public knowledge and so can be filtered out by a clever attacker, or modified to add credence to the attacker’s version of the image. The foregoing discussions demonstrate the need for an invariant segmentation technique which does not rely on a reference pixel which can be removed from an attackers duplicate, nor on the order of the pixels, which can be changed at whim through rotations or other symmetrical transformations. We propose a technique to redress this problem by segmenting the image. Our method does not regard the image as a segment since this would require the presence of all original pixels during verification. Furthermore, we do not simply tile the image, as this requires direct synchronization of the tiles for verification. In the implementation presented in this paper, we use simple region growing segmentation method that relies on pixel properties to define regions. We quickly point out that any image segmentation algorithm that uses invariant local properties to describe a pixel is applicable. Using such an approach, we expect to achieve high level of robustness against loss of synchronization. We can also
obviate the need for having the pixels in the same order by enforcing a fixed predetermined order on the pixels before inserting the mark. Thus, we can effectively provide efficient watermarking functions,
f K (x ) and g K (x, y ) , which are robust against loss of synchronization. In summary we
propose two methods which can provide an invariant domain: 1. Use a segmentation method based on some similarity criterion to partition the image into segments that only rely on pixel properties, and store enough information about the segments to enable exact recovery of segments during the verification phase. Techniques that can be employed to achieve the region segmentation include among others, k-means clustering, fuzzy c-means, region growing and watershed. This was performed in a manual fashion in [2], where the content of the image was targeted for image authentication. 2. Impose a fixed order, for example ascending order of magnitude of pixel values, on pixels within a segment before inserting the mark. The effect of the imposed ordering is to provide robustness (invariance) against rotation or misalignment in a single segment.
3. Feature-based Segmentation A simple region growing algorithm has been used in our tests. The feature-based segmentation utilizes each of the colours, red, green and blue in turn. It moves across the image, selecting key pixels. In our tests, each key pixel was 10 pixels apart from its nearest neighbour in all four directions. From each of these pixels, every adjacent pixel is added to the region if their difference is minor. The strict definition of a minor difference is left to the discretion of the user, but for our tests we considered two pixels to be similar if the colour being used is within 20 grey levels (on a scale of 256 grey levels). We also took the minimum size for a region to be 3,000 pixels in total area so as to prevent the watermark relying on too small an area for data hiding. Segments are required to have a minimum size to be considered for watermarking, otherwise they are merged with neighbouring regions. Therefore, arbitrarily small regions will not be marked as they are of less importance to the image than the larger regions. In this way, an image can be broken up into many different segments for watermarking with the important property that, if the image is cropped or rotated, we should still be able to recover the same segments. If the image is scaled, we then recover segments that are proportionately smaller. The value of this is that if an attacker decides to extract only the important regions of the image then the detection process will be able to identify the same segments as with our original image and use
g k (x, y ) in matching segments. In this way, we have achieved our
goal of invariant segmentation and watermark insertion. Because of the way we convert the two-dimensional regions into one-dimensional arrays, the scan line plays an important part in determining how the watermark is spread across the region. Suppose we extract S segments (or regions) from an image. Taking our sequence
x s , we then obtain
y s = f K (x s ) . Finally we recompose our image by replacing the segments in their original positions. The proposed method is able to achieve invariance even with rotated images.
Detection follows a similar process where we generate segments in the same fashion as for insertion. To allow correct verification we must be able to correctly recover the segments. However because of the insertion of the watermark, we cannot expect exact recovery unless some key values from the original segmentation process is stored. The amount of stored data is an important efficiency parameter of the system. An efficient system should require relatively small size of essential segmentation data, while a naïve system might require all segment outlines to be stored.
4. Experimental Results In order to verify the feasibility of the proposed techniques, the test image, “Peppers”, was segmented into regions using the simple region growing techniques described earlier. As mentioned above, we put the threshold for related pixels at 20 shades and regard all regions under 3,000 pixels as being too small. Simulation of watermark insertion based on Cox’s method [5] was performed. For each of the segments generated, the watermark amplitude was set to 0.1 and the lowest 1,000 frequencies of the transformed segments were modified. We note that in our simulation, we did not use the first post-processing suggested in [5], viz., mean subtraction. Rather, we employed only the second type of post-processing operation, by allowing for tolerance levels in the change in the transformation coefficient values. The actual watermarking pattern is based on the experiments mentioned in [5]. It is important to note that the detection scores we calculated were not quite as impressive as those demonstrated in that paper. For using an entire image with a thousand frequencies, the benchmark score was 32.0, while our results yielded 23.55. Likewise, after using JPEG compression at 90% level, they still retain enough watermark content to give 22.8, while for us, we achieve only 3.97. With the current results, we cannot claim robustness to distortions introduced by compression. There are, however, ways in which we can improve the technique. For example, the choice of a transform more amenable to the structure of the reordered pixels. The DCT transform used by Cox may not be the most appropriate transform to use, given that the sorting operation will create a domain which represents a ramp function.
4.1 Region Growing The detection process used the same regions that were generated in the original image, so there would be no discrepancies between the regions. Of course, the ultimate objective is to have each image independently generate its own region set and automatically compare the closest matches in the detection process.
Figure 4.1.1 Original Peppers Image
Figure 4.1.2 Peppers image showing the formed featurebased regions
Figure 4.1.3 Watermarked Peppers image using formed
Figure 4.1.4 Peppers image illustrating the confidence
feature-based regions
level of the recovered watermark in figure 4.1.3
Figure 4.1.1 shows the original “Peppers” image prior to the watermarking process. Figure 4.1.2 illustrates the five regions that were constructed by the clustering process. Figure 4.1.3 gives us the watermarked image, achieved after applying Cox’s method to the five regions found in Figure 4.1.2. Figure 4.1.4 attempts to show the level of confidence across the watermarked image. It shows a scale from dark to light, where black indicates no contribution to the watermarking score (due to the lack of an appropriate region), grey indicates no contribution the watermarking score, while white represents a strong confidence level in the presence of the watermark. The detection process scored the inserted watermark with 35.10. This value easily demonstrates the presence of the watermark. The Peak Signal to Noise Ratio (PSNR) was used as the image fidelity measure. The PSNR of image shown in Figure 4.1.3 relative to that in Figure 4.1.1 was 38.80dB and showed little visible variation between the two images.
However, the need to produce consistent and invariant regions in the image causes some difficulty. Here in this experiment, we stored the image map which shows how the regions are spread across the original image (as shown in Figure 4.1.2) in Run Length Encoding (RLE) form as a simple and effective technique for reducing the amount of stored information required. The image map by itself would have taken up 65,536 bytes but after the encoding, the size was reduced to 10,224 bytes. This was necessary, as using region-growing segmentation on the watermarked image produces regions that are too dissimilar to reliably detect the watermark. The reason for this is that the watermark insertion process modifies the pixel values. Thus, the regions which are detected using the same process vary from the originally generated regions.
4.2 K-means In these experiments the components of the feature vector used by the k-means algorithm are simply the pixel under consideration and its four-neigbour pixels. Effectively, we have not taken advantage of the availability of colour information. Five regions have been chosen in these experiments and the result is presented Figure 4.2.1.
Figure 4.2.1 Peppers image showing the 5 k-means
Figure 4.2.2 Watermarked Peppers image using k-
generated regions
means generated regions
Figure 4.2.3 Peppers image illustrating the confidence level of the recovered watermark in Figure 4.2.2
The codebook that generated the regions were stoored using RLE. The detection score for Figure 4.2.2 is 36.67 and has a PSNR of 36.16dB. Although the output image is noisier, it is important to note that the entire image now has a watermark inserted as opposed to Figure 4.1.3, which has only had the valid regions watermarked.
Number
Watermark
PSNR
of regions
detection score
(db)
3
33.39
37.32
4
34.18
37.00
5
35.04
37.00
6
37.00
35.12
7
36.61
34.44
8
39.49
35.35
Table 4.2.1 Average of 10 watermarked images seeded with consecutive numbers
The results presented in Table 4.2.1 indicate that the detection scores improve as we segment into more regions. We do expect this at the cost of robustness because we have given ourselves more locations to store information at the cost of the spread of the watermark. In general though, the benefit of providing more regions is that the locality of the watermark becomes stronger. This means that if an image is cropped, a greater portion of the watermark remains in that area, than if it had been spread over the entire image. The cost to the PSNR of using more regions is minimal.
4.3 Sorting The results obtained when sorting was incorporated with k-means or region-growing based segmentation were not encouraging because the insertion process via the DCT disrupted the order of
reconstructed pixels. This problem is being investigated further with the view to selecting a more appropriate transform.
5. Conclusion In this paper we proposed segmentation and pixel ordering as a general method of implementing robust watermarking systems. Segmentation can provide higher robustness and efficiency, and also a way of selectively watermarking parts of an image. The latter application is important if different parts of an image have different semantic significance and so require higher level of protection. However introducing segmentation can result in more susceptibility to loss synchronization. We noted that correct detection requires correct recovery of the segments during verification phase. Towards this goal some key information from the original segmentation process must be preserved and used in the verification phase. At present, we require an original “image-map” to recover the watermark. In the region growing segmentation method given in Section 3 the size of this key information was rather large, that is equal to the number of pixels in the image (65,536 bytes) or the RLE of the regions in the image (10,224 bytes). In the second clustering method, given in section [2], this could be reduced to (number of regions * 9 * 8 bytes) which is a dramatic reduction. The reduction is because of the use of a clustering algorithm that allows the recovery the segments from the stored values. Although finding the optimal clustering algorithm requires further research, the approach is potentially very powerful. It means that we are no longer bound by the dimensions of an image or by the relative position of the components within the image. Watermark verification can then be done by an automated process without the need for human pre-processing to guarantee good results. If necessary, the regions could be selected by hand, or pre-determined such as with object recognition or by provision of the digital media itself, to achieve good results. The use of an invariant technique would establish simple verification of any image to provide identification and/or verification of ownership as it is independent of the modifications of the image. It is important to note that the underlying watermarking algorithm is of no key significance. This is because most contemporary watermarking algorithms make no distinction between being given an image or given just a segment to work with. In this way, an invariant technique that provides these segments will allow the algorithm to work as effectively as if it were working on the entire image. Further work involves testing these relationships and verifying that segment comparisons are effective and efficient. We also need to establish other means to use sorting to resist the changes introduced by the DCT. There are many sides to this problem that need investigation, such as the way clustering is performed and also the level to which segments can be easily verified outside of their original contexts.
Acknowledgments: This research work is partly funded by Motorola Australian Research Centre, through the Australian Government SPIRT grant scheme.
References: [1] A. Z. Tirkel, C. F. Osborne, and T. E. Hall, “Image and watermark registration”, Signal Processing, vol 66, pp373-383, 1998. [2] M. Schneider and Shih-Fu Chang, “A Robust Content Based Digital Signature for Image Authentication”, Proceedings of the 1996 International Conference in Image Processing, pp 227230, 1997. [3] J. O. Ruanaidh, W. Dowling, and F. Boland, “Phase Watermarking of Digital Images”, Proceedings of the 1996 International Conference on Image Processing, vol. 3, pp 239-242, 1996. [4] D. Fleet, and D. Heeger, “Embedding Invisible Information in Color Images”, Proceedings of the 1997 International Conference on Image Processing, pp 532-534, 1997. [5] I. Cox, J. Kilian, T. Leighton, and T. Shamoon “A Secure, Robust Watermark for Multimedia”, NEC Research Institute, Technical Report 95-10, 1995. [6] W. Bender, D. Gruhl, N. Morimoto, and A. Lu, “Techniques for data hiding”, IBM Systems Journal, vol. 35, pp313-336, 1996. [7] A. Gersho and R. M. Gray, Vector Quantization and Signal Compression, Kluwer Academic Publishers Group, 1991.
‡Digital Technology Research Lab Motorola Australian Research Centre
E-mail: {gareth, rei}@uow.edu.au, [email protected] Abstract. There is a spate of research activities investigating digital image watermarking schemes in an effort to control the problem of illegal proliferation of creative digital data across the Internet. Several schemes, with their merits and demerits, have been proposed to date. However, many of them rely on the presentation of a test image in an appropriate format before the detection can proceed. In most cases this is achieved through some form of pre-processing. This paper proposes a technique, using image region segmentation, which alleviates many of the problems associated with pre-processing and thus allowing the detection process to be more independent as well as reducing the need for human intervention.
1. Introduction Digital information is now readily available due to advances in the compression, storage and communications technologies. The amount of digital information that can be found on the Internet and the popularity of the Internet corroborate this observation.
Unfortunately, the protection of this
information, especially in circumstances where the owners hope to generate revenue through controlled dissemination, is yet to be standardized. There are at least two consequences of the status quo. First, digital information that is already available is being illegally re-distributed and thereby robbing the legal owners of deserved revenue. Second, this situation discourages content generators from sharing their work with the wider community. A solution to the latter case is the development of proprietary techniques to insert and verify digital watermarks. Several techniques have been proposed in the literature to insert watermarks in digital images. These techniques provide various degrees of robustness. Watermarking applications include, copyright protection,
authentication, embedded
and
hidden annotations,
and
secure
and invisible
communications. Watermarking systems that are intended for copyright protection are likely to be subjected to a wide range of attacks that are aimed at removing the watermark, or simply making it ineffective and so a very high degree of robustness is required. Watermarking systems for authentication belong to the fragile class of schemes where the slightest change to the image completely destroys the mark and hence the authenticity of the image. Finally watermarking for embedding annotations and secure and invisible communication require resistance against moderate level of modification due to routine image manipulations including digital to analog conversion and vice versa. In this paper we are concerned with a group of attacks on a watermarked image that result in the loss of synchronization and hence failure of the watermark verification process. By synchronization we mean correct correspondence between pixels: that is the ith pixel in the original image remains the ith pixel in the image under verification test. The loss of synchronization may be due to intentional attacks,
such as deleting a row or a column of pixels, or unintentional causes such as misalignment during the scanning of the image. The problems of misalignment and loss of synchronization are discussed in some detail in [1], where the authors reviewed several techniques for overcoming these problems. The solution proposed is to repeatedly test shifted versions of the verification signal, searching for a high peak to indicate correlation. However, the application of this scheme is only discussed in terms of binary insertion and detection, which places in the class of fragile watermark. Also, the notion of continuously testing different signals to search for matches is unappealing. Another attempt to produce a region-based watermarking algorithm was by Schneider and Chang [2], where individual objects in an image were targeted for watermarking. However, this was a procedure that required manual selection of image regions for both insertion and detection. Our objective is to use image segments, which may not necessarily correspond to an object, but are generated by a simple segmentation algorithm. Segmentation methods can be as simple as partitioning an image into tiles, or through the use of image features that capture the quantitative description of a homogeneity criterion. Region growing, clustering and watershed algorithms are some examples of feature based segmentation. In general, the pixels within each segmented region will be similar according to some criteria. Simple image partitioning or tiling is commonly used for increasing efficiency in watermarking systems. The problem of efficiency arises when image sizes do not lend themselves to operation by a fast transformation algorithm. An example can be found in the case of the DCT where the fast radix-2 N
algorithm is well known. Images with sizes that are not multiples of 2 , where N is an integer, have to be tiled into sub-images of sizes amenable to the fast algorithm. We propose segmentation for efficiency, robustness against synchronization loss and localization. Using segments and breaking the verification process into smaller chunks results in more efficient computation. This is especially true when a hardware implementation of the watermarking process is required. Using robust segmentation algorithms and inserting the watermark in selected secret segments will make attacker’s attempts to cause synchronization loss much more difficult. Lastly, it may be required to localize the regions of the image where possible changes have to be detected. This has the advantage of potentially allowing a form of group watermarking, where each member can place a watermark in a different component of the image. Segmenting an image into regions and inserting the watermark only in the desired regions allows various levels of protection for different part of an image. However the use of segmentation requires careful design of the system as it could adversely effect synchronization loss because of the introduction of many regions that need to be individually synchronized. The segmentation process during verification process must recover exactly the same regions that have been used during the insertion process. In other words, segments must remain invariant after the watermark insertion process.
For example, in [3], the authors proposed the use of an invariant domain to store the watermark information. An invariant domain is an area of the image that will always be perfectly reconstructed irrespective of the changes introduced to the image. In [3], the domain proposed is invariant to rotation, scaling and translation; such attacks will not have any effect on the retrieval of the watermark. Fleet and Heeger [4] introduced a watermarking scheme that is rotation and scale invariant. The insertion of the watermark was in the Yellow-Blue colour space of the image and invariance was achieved by modifying the amplitude of regularly spaced sine waves inserted in this domain. At a later stage, these waves can be recovered using the same technique and the image can be realigned and re-scaled to its original dimensions. Cox et al use the whole of frequency range to provide an invisible yet robust domain to insert the watermark [5]. While this domain is robust, it is not invariant, as changes in the spatial domain via rotation for example, will substantially affect the reconstruction of the domain. We propose a simple magnitude based ordering of pixels before the insertion process; the ordering is necessarily undone before verification takes place. The reasoning behind the ordering is that this process relies on the relative magnitude of data, in this case pixel values, and so is independent of the orientation of a segment during the re-scanning of the image. We note that in each of the abovementioned techniques, the domain of insertion is publicly accessible, although the location of the watermark itself may be preserved by use of the owner’s secret information. Using segmentation allows us to hide the domain of insertion if required. The rest of this paper is organized as follows. In section 2 we develop the intuition behind our approach and provide justification for its operation. In section 3 we give the outline of an implementation using a simple segmentation method in conjunction with the scheme proposed by Cox et al. Our experimental output is documented in section 4. Section 5 then provides some concluding remarks.
2. Segmentation in Watermarking Watermarking consists of two complementary processes: watermark insertion and watermark verification. During the insertion process a watermark signal, w, is embedded into a host object I and produces a watermarked object
I (w) . The watermarked object might undergo a range of intentional
and unintentional modifications. The aim of the verification process is to produce evidence in support of the existence of the watermark. The verification algorithm produces a true result by recovering the watermark, possibly in a distorted and weakened form, but with appreciable signal strength that supports existence of the mark with a high probability. The main challenge in robust watermarking is to ensure that the system produces correct results. In other words, verification should produce true for all valid marks, and false for all fraudulent objects, if the object has been subjected to a range of intentional and unintentional modifications.
It is important to note that because of various
unintentional modifications, such as filtering and compression, certain degree of robustness is necessary for all watermarking applications. Furthermore, synchronization is lost if an object is not perfectly aligned and may result in the failure of the verification process. To illustrate this problem, consider the case of computing a transform (e.g. DCT) of a tiled image in order to extract the watermark signal. The image needs to be properly aligned so that the same tiles used in the insertion process can be recovered. If the watermarked image were visually imperceptibly skewed or rotated the
verifier requires knowledge of this process to be able to perform the verification process accurately. This problem remains even when the whole image (rather than tiles) is used during the insertion process. Segmentation-based watermarking attempts to provide the required invariance property when verifying a watermark in the scenarios described above. Region segmentation based on textural properties of the image can be robust to affine transformations if appropriate descriptors are used. It is also robust against minor changes aimed at removing the strict correspondence between the original image and its watermarked form. Such image modifications might include row (or a column) removal, substitution of a row (or a column) with the average of adjacent ones, line insertion and resizing.
2.1 The System We assume an original image, or colour. Let
I s denote the s
th
I , consisting of an array N × M of pixels which could be grayscale segment of the image, where each segment consists of a subset of
pixels forming an arbitrarily shaped area of the image and obtained through a possibly human-assisted algorithm. Our definition of a segment is assumed broad enough to subsume the case of tiling and the case where a human user selects areas of an image as segments. Let S be the total number of segments. Each segment
I s from the image can be ordered into a sequence, x s , of pixels, with the
i th component denoted as x s (i ). X denotes the set of all such segments. The watermarking process usually requires some input called key and denoted by k which is only accessible to the owner of the object or in general the person who inserts the mark. We use K to denote the set of possible keys and
f K (x ) to denote a key-dependent transformation on an input segment, x s ; f K (x ) is for example a watermark insertion algorithm. A watermark detection algorithm,
g K (x, y ′), produces an estimate of
the likelihood that an image segment represented by the sequence
y ' contains the watermark. In other
y ′ = f k (x ) . For an image with segments y s , the likelihood of being derived through
words,
watermarking
from
the
image
segment
set
{x s ∈ X ; s = 1,, S }
is
calculated
as
S
W = ∑ g k (x s y s ) . s =1
2. 2 Producing Invariance in Segments Consider the simplest case where the number of segments, S , is equal to one. In other words the whole image is used for insertion and detection. One drawback of the whole image as one segment is computational and implementation inefficiencies. The system is also prone to loss of synchronization if the sequence, x s (in this case the whole image), derived from the original image does not bear a oneto-one correspondence to the sequence, y s , derived from the watermarked image. The problem faced when S > 1 is no less, because any inaccuracy in determining the segment boundaries will have a cumulative effect when more segments are considered.
The detection function
g k (x, y ) can be seen as a correlator that provides comparison between the
two vectors and ensures that the differences between the two are predictable by k. If the correspondence between segments of
I and I (w) is not accurate the correlation measure fails. Note
that the loss of segment synchronization can occur if a simple transformation such as cropping or adding an extra column to the image is performed. It is therefore necessary to reverse the operation before any comparison between two sequences are made. In the case of cropping, the remaining portion of the picture must be replaced in the original position, so that the correct neighbourhood can be reconstructed for detection (as indicated in [5]). If a column has been added, then it must be removed to regain synchronization. Finally for segmentation to work properly, we must ensure that each sequence
x s relates to y s on a point-to-point basis. This is achieved through the ordering mechanism described before. It is desirable that it be able to detect the change in segment boundaries and recognize the need to realign the segments. This requirement is particularly important with schemes that depend on inserting a watermark at certain positions in the image, e.g. “Patchwork”[6]. Here we have a fixed point somewhere within
I , the reference image. By mapping the watermarked image I (w) onto I we can
recover the differences and indicate the presence of the watermark. If the reference point is no longer valid, even if it has only been slightly tampered with, then the comparison cannot be correctly made. To counter these problems, some attempts have been made to provide an invariant domain of operation to allow
g k (x, y ) to work correctly, irrespective of changes made to the image [3]. In their
work [3], Ó Ruanaidh and Pun achieved invariance by transforming the image through the Fast Fourier Transform, then through the Log Polar Transform and finally through another Fast Fourier transform. The end result of this process is a domain that is invariant to rotations, scaling and translations. However, cropping is an effective attacking tool, which does not degrade the appearance of the original image, although it does reduce the amount of original information. Another technique attempts to introduce a small mask of sine waves across the image [4], which can be retrieved at a later stage to indicate how the image has been stretched or rotated. Unfortunately, these are produced so as to be public knowledge and so can be filtered out by a clever attacker, or modified to add credence to the attacker’s version of the image. The foregoing discussions demonstrate the need for an invariant segmentation technique which does not rely on a reference pixel which can be removed from an attackers duplicate, nor on the order of the pixels, which can be changed at whim through rotations or other symmetrical transformations. We propose a technique to redress this problem by segmenting the image. Our method does not regard the image as a segment since this would require the presence of all original pixels during verification. Furthermore, we do not simply tile the image, as this requires direct synchronization of the tiles for verification. In the implementation presented in this paper, we use simple region growing segmentation method that relies on pixel properties to define regions. We quickly point out that any image segmentation algorithm that uses invariant local properties to describe a pixel is applicable. Using such an approach, we expect to achieve high level of robustness against loss of synchronization. We can also
obviate the need for having the pixels in the same order by enforcing a fixed predetermined order on the pixels before inserting the mark. Thus, we can effectively provide efficient watermarking functions,
f K (x ) and g K (x, y ) , which are robust against loss of synchronization. In summary we
propose two methods which can provide an invariant domain: 1. Use a segmentation method based on some similarity criterion to partition the image into segments that only rely on pixel properties, and store enough information about the segments to enable exact recovery of segments during the verification phase. Techniques that can be employed to achieve the region segmentation include among others, k-means clustering, fuzzy c-means, region growing and watershed. This was performed in a manual fashion in [2], where the content of the image was targeted for image authentication. 2. Impose a fixed order, for example ascending order of magnitude of pixel values, on pixels within a segment before inserting the mark. The effect of the imposed ordering is to provide robustness (invariance) against rotation or misalignment in a single segment.
3. Feature-based Segmentation A simple region growing algorithm has been used in our tests. The feature-based segmentation utilizes each of the colours, red, green and blue in turn. It moves across the image, selecting key pixels. In our tests, each key pixel was 10 pixels apart from its nearest neighbour in all four directions. From each of these pixels, every adjacent pixel is added to the region if their difference is minor. The strict definition of a minor difference is left to the discretion of the user, but for our tests we considered two pixels to be similar if the colour being used is within 20 grey levels (on a scale of 256 grey levels). We also took the minimum size for a region to be 3,000 pixels in total area so as to prevent the watermark relying on too small an area for data hiding. Segments are required to have a minimum size to be considered for watermarking, otherwise they are merged with neighbouring regions. Therefore, arbitrarily small regions will not be marked as they are of less importance to the image than the larger regions. In this way, an image can be broken up into many different segments for watermarking with the important property that, if the image is cropped or rotated, we should still be able to recover the same segments. If the image is scaled, we then recover segments that are proportionately smaller. The value of this is that if an attacker decides to extract only the important regions of the image then the detection process will be able to identify the same segments as with our original image and use
g k (x, y ) in matching segments. In this way, we have achieved our
goal of invariant segmentation and watermark insertion. Because of the way we convert the two-dimensional regions into one-dimensional arrays, the scan line plays an important part in determining how the watermark is spread across the region. Suppose we extract S segments (or regions) from an image. Taking our sequence
x s , we then obtain
y s = f K (x s ) . Finally we recompose our image by replacing the segments in their original positions. The proposed method is able to achieve invariance even with rotated images.
Detection follows a similar process where we generate segments in the same fashion as for insertion. To allow correct verification we must be able to correctly recover the segments. However because of the insertion of the watermark, we cannot expect exact recovery unless some key values from the original segmentation process is stored. The amount of stored data is an important efficiency parameter of the system. An efficient system should require relatively small size of essential segmentation data, while a naïve system might require all segment outlines to be stored.
4. Experimental Results In order to verify the feasibility of the proposed techniques, the test image, “Peppers”, was segmented into regions using the simple region growing techniques described earlier. As mentioned above, we put the threshold for related pixels at 20 shades and regard all regions under 3,000 pixels as being too small. Simulation of watermark insertion based on Cox’s method [5] was performed. For each of the segments generated, the watermark amplitude was set to 0.1 and the lowest 1,000 frequencies of the transformed segments were modified. We note that in our simulation, we did not use the first post-processing suggested in [5], viz., mean subtraction. Rather, we employed only the second type of post-processing operation, by allowing for tolerance levels in the change in the transformation coefficient values. The actual watermarking pattern is based on the experiments mentioned in [5]. It is important to note that the detection scores we calculated were not quite as impressive as those demonstrated in that paper. For using an entire image with a thousand frequencies, the benchmark score was 32.0, while our results yielded 23.55. Likewise, after using JPEG compression at 90% level, they still retain enough watermark content to give 22.8, while for us, we achieve only 3.97. With the current results, we cannot claim robustness to distortions introduced by compression. There are, however, ways in which we can improve the technique. For example, the choice of a transform more amenable to the structure of the reordered pixels. The DCT transform used by Cox may not be the most appropriate transform to use, given that the sorting operation will create a domain which represents a ramp function.
4.1 Region Growing The detection process used the same regions that were generated in the original image, so there would be no discrepancies between the regions. Of course, the ultimate objective is to have each image independently generate its own region set and automatically compare the closest matches in the detection process.
Figure 4.1.1 Original Peppers Image
Figure 4.1.2 Peppers image showing the formed featurebased regions
Figure 4.1.3 Watermarked Peppers image using formed
Figure 4.1.4 Peppers image illustrating the confidence
feature-based regions
level of the recovered watermark in figure 4.1.3
Figure 4.1.1 shows the original “Peppers” image prior to the watermarking process. Figure 4.1.2 illustrates the five regions that were constructed by the clustering process. Figure 4.1.3 gives us the watermarked image, achieved after applying Cox’s method to the five regions found in Figure 4.1.2. Figure 4.1.4 attempts to show the level of confidence across the watermarked image. It shows a scale from dark to light, where black indicates no contribution to the watermarking score (due to the lack of an appropriate region), grey indicates no contribution the watermarking score, while white represents a strong confidence level in the presence of the watermark. The detection process scored the inserted watermark with 35.10. This value easily demonstrates the presence of the watermark. The Peak Signal to Noise Ratio (PSNR) was used as the image fidelity measure. The PSNR of image shown in Figure 4.1.3 relative to that in Figure 4.1.1 was 38.80dB and showed little visible variation between the two images.
However, the need to produce consistent and invariant regions in the image causes some difficulty. Here in this experiment, we stored the image map which shows how the regions are spread across the original image (as shown in Figure 4.1.2) in Run Length Encoding (RLE) form as a simple and effective technique for reducing the amount of stored information required. The image map by itself would have taken up 65,536 bytes but after the encoding, the size was reduced to 10,224 bytes. This was necessary, as using region-growing segmentation on the watermarked image produces regions that are too dissimilar to reliably detect the watermark. The reason for this is that the watermark insertion process modifies the pixel values. Thus, the regions which are detected using the same process vary from the originally generated regions.
4.2 K-means In these experiments the components of the feature vector used by the k-means algorithm are simply the pixel under consideration and its four-neigbour pixels. Effectively, we have not taken advantage of the availability of colour information. Five regions have been chosen in these experiments and the result is presented Figure 4.2.1.
Figure 4.2.1 Peppers image showing the 5 k-means
Figure 4.2.2 Watermarked Peppers image using k-
generated regions
means generated regions
Figure 4.2.3 Peppers image illustrating the confidence level of the recovered watermark in Figure 4.2.2
The codebook that generated the regions were stoored using RLE. The detection score for Figure 4.2.2 is 36.67 and has a PSNR of 36.16dB. Although the output image is noisier, it is important to note that the entire image now has a watermark inserted as opposed to Figure 4.1.3, which has only had the valid regions watermarked.
Number
Watermark
PSNR
of regions
detection score
(db)
3
33.39
37.32
4
34.18
37.00
5
35.04
37.00
6
37.00
35.12
7
36.61
34.44
8
39.49
35.35
Table 4.2.1 Average of 10 watermarked images seeded with consecutive numbers
The results presented in Table 4.2.1 indicate that the detection scores improve as we segment into more regions. We do expect this at the cost of robustness because we have given ourselves more locations to store information at the cost of the spread of the watermark. In general though, the benefit of providing more regions is that the locality of the watermark becomes stronger. This means that if an image is cropped, a greater portion of the watermark remains in that area, than if it had been spread over the entire image. The cost to the PSNR of using more regions is minimal.
4.3 Sorting The results obtained when sorting was incorporated with k-means or region-growing based segmentation were not encouraging because the insertion process via the DCT disrupted the order of
reconstructed pixels. This problem is being investigated further with the view to selecting a more appropriate transform.
5. Conclusion In this paper we proposed segmentation and pixel ordering as a general method of implementing robust watermarking systems. Segmentation can provide higher robustness and efficiency, and also a way of selectively watermarking parts of an image. The latter application is important if different parts of an image have different semantic significance and so require higher level of protection. However introducing segmentation can result in more susceptibility to loss synchronization. We noted that correct detection requires correct recovery of the segments during verification phase. Towards this goal some key information from the original segmentation process must be preserved and used in the verification phase. At present, we require an original “image-map” to recover the watermark. In the region growing segmentation method given in Section 3 the size of this key information was rather large, that is equal to the number of pixels in the image (65,536 bytes) or the RLE of the regions in the image (10,224 bytes). In the second clustering method, given in section [2], this could be reduced to (number of regions * 9 * 8 bytes) which is a dramatic reduction. The reduction is because of the use of a clustering algorithm that allows the recovery the segments from the stored values. Although finding the optimal clustering algorithm requires further research, the approach is potentially very powerful. It means that we are no longer bound by the dimensions of an image or by the relative position of the components within the image. Watermark verification can then be done by an automated process without the need for human pre-processing to guarantee good results. If necessary, the regions could be selected by hand, or pre-determined such as with object recognition or by provision of the digital media itself, to achieve good results. The use of an invariant technique would establish simple verification of any image to provide identification and/or verification of ownership as it is independent of the modifications of the image. It is important to note that the underlying watermarking algorithm is of no key significance. This is because most contemporary watermarking algorithms make no distinction between being given an image or given just a segment to work with. In this way, an invariant technique that provides these segments will allow the algorithm to work as effectively as if it were working on the entire image. Further work involves testing these relationships and verifying that segment comparisons are effective and efficient. We also need to establish other means to use sorting to resist the changes introduced by the DCT. There are many sides to this problem that need investigation, such as the way clustering is performed and also the level to which segments can be easily verified outside of their original contexts.
Acknowledgments: This research work is partly funded by Motorola Australian Research Centre, through the Australian Government SPIRT grant scheme.
References: [1] A. Z. Tirkel, C. F. Osborne, and T. E. Hall, “Image and watermark registration”, Signal Processing, vol 66, pp373-383, 1998. [2] M. Schneider and Shih-Fu Chang, “A Robust Content Based Digital Signature for Image Authentication”, Proceedings of the 1996 International Conference in Image Processing, pp 227230, 1997. [3] J. O. Ruanaidh, W. Dowling, and F. Boland, “Phase Watermarking of Digital Images”, Proceedings of the 1996 International Conference on Image Processing, vol. 3, pp 239-242, 1996. [4] D. Fleet, and D. Heeger, “Embedding Invisible Information in Color Images”, Proceedings of the 1997 International Conference on Image Processing, pp 532-534, 1997. [5] I. Cox, J. Kilian, T. Leighton, and T. Shamoon “A Secure, Robust Watermark for Multimedia”, NEC Research Institute, Technical Report 95-10, 1995. [6] W. Bender, D. Gruhl, N. Morimoto, and A. Lu, “Techniques for data hiding”, IBM Systems Journal, vol. 35, pp313-336, 1996. [7] A. Gersho and R. M. Gray, Vector Quantization and Signal Compression, Kluwer Academic Publishers Group, 1991.
