A neuro-fuzzy technique for document binarisation
Neural Comput & Applic (2003) 12: 190–199 DOI 10.1007/s00521-003-0382-z
O R I GI N A L A R T IC L E
Nikos Papamarkos
A neuro-fuzzy technique for document binarisation
Received: 8 March 2002 / Accepted: 3 July 2003 / Published online: 8 November 2003 Springer-Verlag London Limited 2003
Abstract This paper proposes a new neuro-fuzzy technique suitable for binarisation or, in general, the colour reduction of digital documents. The proposed approach uses the image colour values and additional local spatial features extracted in the neighbourhood of the pixels. Both image and local features values feed a Kohonen self-organised feature map (SOFM) neural network classifier. After training, the neurons of the output competition layer of the SOFM define a first approach of the final classes. Using the content of these classes, fuzzy membership functions are obtained that are next used by the fuzzy C-means (FCM) algorithm in order to obtain the colours of the final document. The method can be applied to greyscale and colour documents; it is suitable for improving blurring and badly illuminated documents and can be easily modified to accommodate any type of spatial characteristics. Keywords Binarisation Æ Colour reduction Æ FCM Æ Fuzzy logic Æ Self-organised neural networks
1 Introduction A complex mixed-type document image contains text, line-drawings and graphics regions. In many practical applications we need only to recognise or improve the text content of the documents. In such cases, it is preferable to convert the documents into a binary form, or at least, into a limited number of grey levels (or colours in the case of colour documents). Doing this, we can transmit and process more efficiently the documents instead of the original greyscale forms. For many years the N. Papamarkos (&) Department of Electrical & Computer Engineering, Democritus University of Thrace, 67100 Xanthi, Greece E-mail: [email protected] Tel.: +30-25410-79585 Fax: +30-25410-79569
binarisation of greyscale documents was based on the standard bilevel techniques that are also called global thresholding algorithms [1, 2, 3, 4, 5, 6]. These statistical methods are suitable for converting any greyscale image into a binary form but are inappropriate for complex documents, and even more so, for degraded documents. In these special cases it is important to take into account the nature and the spatial structure of the document images. Based on this assumption, specialised binarisation techniques have been developed for complex document images. Strouthopoulos and Papamarkos [7] considered the problem of the grey level reduction of complex documents by using a combination of a page layout analysis technique [8] and a Kohonen seld-organised feature map (SOFM) neural network [9–10]. Such techniques are suitable for cleaning documents but do not work efficiently with degraded documents. Some other document binarisation techniques have been developed which are focused on the binarisation of degraded documents [11, 12, 13, 14]. In [12], OGorman proposes an algorithm that is based on the use of local connectivity information. Other techniques are based on some gradient and edge information [13,15], stroke analysis [16] and adaptive logical algorithms [11,14]. A good analysis and comparative results of these techniques are given in [11]. The abovementioned techniques provide good results in many cases. However, they are complex, require an extended analysis of the documents structure and are time consuming. In many applications, such as fax transmission and document retrieval systems, we mainly want the text to be as readable as possible. Therefore, an efficient document binarisation technique must be focused to primarily improve the text regions. The proposed technique for the binarisation and grey level (or colour) reduction of mixed-type documents, in order to improve the form of the final text regions, takes advantages not only of the image pixels values, but also of additional spatial information. Specifically, the proposed technique is based on a hybrid neuro-fuzzy system [17], which consists of a Kohonen SOFM and a fuzzy C-means (FCM) classifier [10,18]. According to the
191
proposed technique, the grey level values of the pixels are related to suitable local spatial features extracted in the pixels neighbouring regions. Thus, the one-dimensional histogram clustering approach of the classical multithresholding techniques is converted into a multi-dimensional feature clustering technique. For each image pixel, its greyscale value or its three colour component values for the case of colour documents are considered as the first feature. The entire feature set is completed by K additional spatial features, which are extracted from neighboring pixels. These features are associated with spatial image characteristics that emphasise and improve the shapes of the characters. In general, depending on the type of problem, the Kohonen SOM has K+1 (for the case of gay-scale images) of K+3 (for the case of colour documents) input neurons. The number of the output neurons is equal to the number of the final colour classes. Thus, for binarisation we have two output neurons and for general colour reduction, independently if the image isgreyscale or colour, the number of output neurons is equal to the number of final greyscale values or colours. In these cases, the training procedure follows the Kohonen learning rules. After training, the output neurons of the competition layer of the SOFM define the dominant classes. Using the content of these classes, fuzzy membership functions are automatically obtained and are next used in the FCM classifier. FCM is a powerful classifier but has two drawbacks associated with the computational cost and the requirement for a ‘‘good’’ starting point to avoid convergence into an unsatisfactory local sub-optimal point. These drawbacks are overcome in the proposed approach by using the SOFM results and a fractal sub-sampling procedure as a starting point. On the other hand, through proper classification the fuzzy data using the FCM classifier a significant character blurring reduction is achieved [18]. As it is analysed in Sect. 3, the proposed technique can be extended to cover the case of the reduction of the document grey levels or colours into a limited number by adding more output classes. In this case, a technique is initially applied that can estimates the proper number of the document greyscales (or colours) and therefore, the proper number of the final image greyscale (or colour) values. In order to reduce the storage requirement and the computation time, the training set can be a representative sample of the image pixels. In our approach, the sub-sampling is performed via a fractal scanning technique, based on the Hilberts space filling curve [19]. The proposed method was tested with a variety of documents. In addition, this paper presents characteristic examples and comparative results that confirm the effectiveness of the proposed method.
2 A description of the method for greyscale documents A digital greyscale document I (i, j), i = 1, ..., n, j = 1,..., m can be considered as an image consisting of n·m pixels, where the value of each pixel is a point in the
greyscale space. Usually, the total number of grey levels is restricted to 256, i.e., I(i, j) 2 [0,255]. The document greyscale reduction problem can be considered as the problem of best transforming the document grey level image into a new one, with only J grey levels, such as the final document to approximate not only the dominant grey level values, but also to preserve the principal text information included in it. A special case of the greyscale reduction problem is the binarisation process where the final image has only two greyscale values, usually 0 for foreground and 255 for background pixels, respectively. An effective approach to solve this problem is to consider it as a clustering problem and achieve its solution using a suitable self-organised neural network and the FCM algorithm. The proposed technique, for the binarisation orthe greyscale reduction of complex and degraded documents, is analytically described in the next section.
2.1 Document binarisation The proposed document binarisation method consists of the following three stages: – Initial document binarisation using a Kohonen SOFM neural network with suitable spatial features – Fuzzy membership extraction – Binarisation using the FCM algorithm The complete analysis of the above stages follows. 2.1.1 Initial document binarisation In order to take advantage on the document texture, it is obvious that we must not only consider the image greyscale values, but also suitable spatial features. It is obvious that most of the information in a document is associated with the text regions and the shapes of the characters. For this reason, local characteristics must be extracted and used in the binarisation process. If these local characteristics are considered as local spatial features, then the binarisation problem can be defined as a procedure that leads to a binary document by taking into account, except for greyscale values, suitable spatial characteristics. Now, the problem can be viewed as a clustering one and its solution can be obtained by using an appropriate classifier such as the Kohonen SOFM neural network. If k spatial features are used then we have a k+1 features clustering problem. The Kohonen SOFM classifier used in this stage is shown in Fig. 1. We can observe that k+1 features feed the neural network and the output consists of only two neurons. Certainly, instead of using the SOFM we could use another type of classifier as, for example, the KNN. However, we have found in our previous work that the Kohonen SOFM is a powerful classifier for general colour reduction [21]. It should be noticed that the spatial features must be suitably selected such as the document text areas to be
192
Fig. 1 The Kohonen SOFM classifier
enhanced. Different types of document degradation require different types of spatial features. From our experience we have found that edge extraction and sharpening masks, in combination, in some cases, with mean or min local values provide a good combination for the spatial feature set. Especially, the Laplacian feature values obtained by the 3·3 mask: 2 3 1 1 1 f2 ¼ 4 1 8 1 5 ð1Þ 1 1 1
Fig. 2 Document sub-sampling using the Hilbert curve
in cooperation with the min value in the 3·3 neighboring region of the sampling pixels constructs a suitable set of spatial features for document binarisation. To speed up the entire process and to reduce memory requirements, a fractal scanning sub-sampling technique based on the Hilbert curve is adopted [19] (Fig. 2). The important feature of the Hilbert curve is that it scans continuously the neighboring entry in the image [20]. As a first example, let us considered the badly illuminated document of Fig. 3a. Figure 3b depicts the binary images obtained by the method of Otsu. Figure 3c shows the binary image obtained by the method of Papamarkos and Atsalakis [21]. As it can be observed, both techniques result in a dark area on the left side of the document. Finally, Fig. 3d shows the binarisation results obtained in the first stage of the proposed algorithm where the Laplacian and min additional spatial features are applied. As it can be observed, the binary document obtained is better and does not have any dark area. 2.1.2 Fuzzy membership functions In order to improve the binarisation results by reducing the blurring effects, we propose a fuzzy procedure that can classify efficiently fuzzy pixels into foreground or background pixels. This technique requires a fuzzy membership extraction procedure that determines the membership functions of the two classes defined by the two output neurons of the SOFM. Referring to the abovementioned example, it is clear that each feature vector, which consists of three independent features, is classified in one of the two output classes. Let dij, i =
Fig. 3 a Original document. Document binarisation using the methods of b Otsu c Papamarkos and Atsalakis, and d SOFM with spatial features
1,...,n·m, j = 1,2 be the Euclidean distances of the feature vectors from the centres of the two classes obtained by the Kohonen SOFM. The membership functions for the two classes are obtained through the following relations: u1;j ¼
1 1 þ dj1 l1 =D1
ð2Þ
193
1 1 þ dj2 l2 =D2
u2;j ¼
ð3Þ
where li, i =1,2 is the mean Euclidean distance of class i, and Di ¼ maxðdji Þ minðdji Þ
ð4Þ
A similar definition of the membership functions is proposed by Huang and Wang [22].
other for the foreground pixels. The important effect of using the FCM is the reduction of the document blurring. Figure 4a depicts the document obtained after the application of the FCM algorithm. Comparing Figs. 3d and 4a it can be observed that the document in Fig. 4a is sharper than the document extracted by the SOFM. As a final stage, some simple post-processing tasks can be applied. For example, a mean 3·3 filter followed by a global binarisation procedure fills the character holes and makes them thicker. The results of the application of the post-processing procedure are depicted in Fig. 4b.
2.1.3 The final binarisation via the FCM algorithm After the determination of the membership functions, the clustering problem is converted into a fuzzy clustering one. The solution of this problem can be easily obtained by using the well-known FCM algorithm [18] where as initial membership values are those obtained by the method described above. The description of the FCM used in our approach is as follows. Let f1, f2,...,fn are the n feature vectors V = {v1, v2} are the cluster centres U = {uij} is the 2·n matrix of the membership values that satisfy the conditions: 06uij 61; i ¼ 1; 2 and j ¼ 1:::; n 2 X
uij ¼ 1;
ð5Þ
j ¼ 1; :::; n
ð6Þ
i ¼ 1; 2
ð7Þ
i¼1 n X uij\n; 0\ j¼1
The FCM algorithm is iterative and makes use of the following equations: n P j¼1
um ij fij
vi ¼ P n
i ¼ 1; 2
;
ð8Þ
2.2 An extension to the reduction of the documents greyscale values The proposed technique can be extended to cover the case of reduction of the image greyscale values into a limited number. This is similar to the multithresholding case, where we try to find appropriate thresholds Ti, i = 1,...,N)1 in the image histogram in order to reduce the number of the image greyscale values to N. In the proposed approach, if N is the desired number of final greyscale values, then the Kohonen SOFM must have N output neurons. The input vectors remain similar to the case of binarisation, that is, they consist of every pixel of the pixels greyscale value and additional values of the spatial features obtained locally. After training, each output neuron is associated with the centre of the feature classes obtained. A similar approach has been used in [23]. Now, using the Euclidean distance from the centre of the classes as a criterion, the membership functions, for every feature vector and for every class can be obtained. Specifically, if: N fi ; i ¼ 1; :::; n vj ; j ¼ 1; :::; N dij ; i ¼ 1; ::; n; j ¼ 1; :::; N
is the number of output neurons ðclassesÞ are the feature vectors are thecluster centers are the Euclidean distances
um ij
j¼1
h uij ¼
1 jfk vi j2
2 h P j¼1
i1=ðm1Þ
1 jfk vi j2
i1=ðm1Þ
;
i ¼ 1; 2 and k ¼ 1; 2; :::; n
ð9Þ where m is the fuzziness factor, m >1. The iterative procedure stops when there is not a significant change in the objective function: J ðU ; V Þ ¼
2 X n X
2 um ik jfk vi j
ð10Þ
i¼1 k¼1
The FCM algorithm classifies the document pixels into two classes, one for the background pixels and the
Fig. 4 a The document after the application of the FCM b The final document after post-processing
194
then the membership functions are obtained from the relation: h i1=ðm1Þ uik ¼
1 dik2
N h i1=ðm1Þ P 1 j¼1
;
i ¼ 1; :::N ;
and k ¼ 1; :::n
dik2
ð11Þ The clusters obtained by the Kohonen SOFM, and the membership values calculated from Eq. 11, are considered as the initial starting point of the FCM classifier. The use of the FCM algorithm provides more flexibility in the clustering results obtained. An important issue of the greyscale reduction of documents is the estimation of the proper number of the final greyscales. This problem can be solved by examining the validity of clusters obtained [18]. However, this approach leads into an adaptive and time-consuming procedure. To overcome these difficulties, the piece-wise histogram approximation (PWHA) algorithm, described in the next section, can be used. We present below an example of the applications of this technique for the reduction of the number of greyscales in a mixed-type document.
1. A line ( k1, h(k1)) ) ( k2, h(k2)) is not divided if k2 ) k1 < d where d1 a predefined threshold value. 2. A line ( k1, h(k1)) ) ( k2, h(k2)) is not split more if |h(k2) ) h(k1)|< d2 max(h), where max(h) the maximum value of the histogram, and d2 a suitable coefficient (usually d2 = 0.01). 3. A line is not split more if the maximum distance of the histogram points from this line is less than d3 max (h), where the threshold d3 is usually taken equal to 0.002. In order the have an acceptable limited number of histogram peaks, a histogram smoothing procedure can be applied. According to this technique, every value of the histogram is substituted by the mean value in a neighbouring: hðkÞ¼meanfhðk 0:5N þ0:5Þ;:::;hðkÞ;:::;hðk þ0:5N þ0:5Þg ð12Þ where N is usually taken equal to five. We call this technique piece-wise smoothing histogram approximation (PWSHA) and, as the experimental results show, this technique probably gives the most advantageous results.
An extension to colour documents 2.3 The PWHA This technique is based on the curve-fitting algorithm of Duda and Hart [24] which can be considered as a linear PWHA procedure. Specifically, the histogram curve is approximated by a set of straight-line segments. Fitting and convergence criteria are applied to control the entire procedure. According to the PWHA algorithm, the shape of a histogram can be approximated through a set of straight-line segments that are determined by means of a simple iterative process. This technique is relatively immune to small histogram bin variation (due to noise) by suitably choosing the fitting parameters. At the end of the fitting process the principal peaks of the histogram are obtained and their number specifies the optimal numbergreyscale values (ONPGV) of the image. If h(j) is the histogram function, then h(j) can be approximated as follows: initially h(j) is approximated by the straight-line 1 which passes through points (j0,h(j0)) and (j1,h(j1)), where j0 = 0 and j1 = 255. Then, the histogram point (j2,h(j2)), with j0 < j2 < j, having the maximum distance from 1, is obtained and the histogram curve is now approximated by the two lines 2 and 3 defined by the points (j0, h(j0)) ) (j2, h(j2)) and (j2, h(j2)) )(j1, h(j1)), respectively. This fitting procedure is repeated until some convergence conditions are satisfied. At the end of the PWHA procedure, the histogram scheme is approximated by a set of line segments. The convergence conditions for the PWHA algorithm are:
The proposed technique can be extended to the case of colour documents. The goal in this case is to achieve colour reduction of the document image. In general, an RGB colour document Ic(i,j), i = 1,...,n, j = 1,...,m and c = R, G or B can be considered as a colour image consisting of n ·m pixels, where the value of each pixel is a point in the 3D RGB space. Usually, the total number of colours is restricted to 2563 and each colour component is in the range [0,255]. In the case of colour documents, the Kohonen SOFM is fed by the three colour components and additional spatial feature values extracted from the image colour components. For example, if the RGM colour model is used, in every pixel the input vector consists of the three R, G and B components and additional values of the spatial features extracted locally from the colour components images. That is in every pixel (x,y) the feature vector has the following form: T Fðx;y Þ ¼ Rðx;y Þ ; Gðx;y Þ ; Bðx;y Þ ; f4 ; f5 ; f6 ; :::; fn
ð13Þ
The number of the output neurons is equal to the number of the final colours. After training, the centres of the output classes obtained define the final principal colours obtained. In the second stage, the Euclidean distance is used as a criterion in order to calculate the values of the membership functions, for every feature vector and for every class that can be used in the next with the FCM classifier. Again, and for the RGB colour model, if:
195
N fi ; i ¼ 1; :::; n vj ; j ¼ 1; :::; N dij ; i ¼ 1; ::; n; j ¼ 1; :::; N
is the number of output neurons ðclassesÞ are the feature vectors are thecluster centers are the Euclidean distances
then the membership functions are obtained again from Eq. 11. The centres of the final classes obtained, after the application of the FCM, define the final principal colours. In the case of colour documents, the estimation of the proper number of image principal colours can be done by applying the PWHA algorithm in the histogram of a specific colour component. An analytical and characteristic example is given in the experiment session that clarifies the technique for colour reduction in a document.
3 Examples 3.1 Example 1 In this experiment we apply the proposed method to the degradedgreyscale document of Fig. 5a. This document has many line drawings and graphics regions and is Fig. 5 a Original document b Binarisation using the SOFM c Binarisation using the proposed technique d The document after post-processing
badly illuminated. The binarisation of this document using thresholding techniques leads to unaccepted results. Figure 5b shows the binarisation results obtained by the SOFM without using any spatial features. Figure 5c shows the results obtained by the proposed method, where in the SOFM we use the Laplacian mask of Eq. 1. Figure 5d depicts the final image obtained if we apply as post-processing operations the mean 3·3 mask followed by the global binarisation via the SOFM. 3.2 Example 2 This example demonstrates the application of the proposed technique to the reduction of the number of grey levels in a digital document. The original badly illuminated mixed-type document, shown in Fig. 6, includes text and graphics regions. Its size is 1058·946 pixels with a 300 dpi resolution. In order to obtain the proper number of image grey levels, the PWSHA technique is initially applied with a smoothing value N=5, and d1=6, d2=0.01 and d3=0.002. As it can be observed in Fig. 7, the proper minimum value of imagegreyscales obtained is equal to five. The proposed technique is first applied for image binarisation without using spatial features. The image obtained is shown in Fig. 8a. We can observe that due to the mixed-type of the document and the low-scanning quality, the document binarisation results in an unac-
196
Fig. 7 Using the PWSHA technique for the estimation of the proper number of image greyscales Fig. 6 The original mixed-type document
ceptable document form with many broken characters. Even if we apply the powerful adaptive logical techniques of Yang and Yan [11], the binarisation results, shown in Fig. 8b, are still unsatisfactory. In this case we have better character segmentation results but in the graphics areas only edge pixels survive. Then, the proposed technique is applied to reduce the number of grey levels to five but without using spatial features. The image obtained is shown in Fig. 8c and the fivegreyscales are equal to 28, 92, 153, 202 and 246. As it is expected, in comparison with the binary image of Fig. 8a, the quality of the image has been improved. However, in some parts of the documents the characters appear to be blurred. To reduce the blurring effects, we apply the proposed technique with three sharpening spatial features extracted by the following masks:
" 1 1 1 # f2 ¼
1 1 " 1
f4 ¼
9
1 ;
1 2
1 1 #
2
5
2
1
2
1
" 0 f3 ¼
1
0 #
1
5
1 ;
0
1
0
The final image obtained is shown in Fig. 8d and has the five dominantgreyscale values: 38, 95, 159, 203 and 243. Comparing Figs. 8c and 8d it can be easily observed the improvement of the image blurring (this is obvious especially in the text area in the bottom of the document). Fig. 8e depicts the image pixels that have the maximum classification vagueness, i.e., their maximum membership values are close to 0.5. For comparison reasons, we reduce the imagegreyscale values to five by using other techniques. Figure 9 depicts the results obtained by the methods of Reddi
Fig. 8 a Image binarisation b Binarisation using the method [11] c The document with fivegreyscales obtained without using spatial features d The final document with five grey levels using the spatial features e The image pixels with a large vagueness
197
Fig. 10 The original colour document
be noticed again that the proposed technique, using a different set of spatial features, results in different images.
4 Example 3
Fig. 9 Reduction to five greyscales using these methods: a Reddi et al. b Median-cut c Minimum-variance d Dekkers
et al. [2], median-cut [25], minimum variance [26] and Dekker [27]. However, it is well known that there is not any objective criterion that can measure the perceived difference between the original and the final image. The perceived difference can be considered also as perceived image quality. In general, the type of application defines when an image is better than others. From the results obtained in this example, it is clear that the proposed technique had better perceived image quality results. On the other hand, in literature there are some image dependent distortion measures. The comparative results presented here are based on the following six criteria: – – – – –
mean squared error (MSE) Euclidean distance (ED) signal-to-noise ratio (SNR) maximum absolute difference (MAD) peak signal-to-noise ratio (PSNR)
Table 1 gives the values for the five criteria. These values clearly indicate that the superior results of the image of Fig. 8b, which obtained by the basic part of the proposed method (without using spatial features). It is interesting that even the maximum absolute difference is better than the other techniques. Finally, it should Table 1 Comparative results Proposed without spatial features Proposed with spatial features Reddi et al. Median-cut et al. Minimum variance Dekkers
This example demonstrates the application of the proposed technique for reduction of the number of colours in a digital colour document. The original document is shown in Fig. 10. This is not a trivial colour document because it has white and black text, graphics and no uniform background. However, it can be easily observed that the number of its dominant colours is only six. In order to smooth the background and obtain only the dominant colours we used: 1. a 3·3 mean filter for extraction of local features, and 2. the Lu*v* colour space. For comparison reasons, we perform binarisation, and colour reductions using other approaches. It can be easily concluded that the proposed technique, with the specifications defined above, results in better colour reduction results for the document. Figure 11 summarises the results obtained in the all the above cases.
5 Conclusions This paper proposes a neuro-fuzzy technique for binarisation and general colour reduction of badly illuminated documents. The method is based on a hybrid neuro-fuzzy classifier that consists of a Kohonen SOFM and a FCM. The feature vector consists of the imageMSE
ED
SNR
MAD
PSNR
171.54 333.54 625.66 382.20 287.02 365.07
11.08 14.02 20.24 12.38 13.80 12.06
229.03 117.33 56.96 104.52 135.95 107.74
34 112 62 86 71 92
25.79 22.90 20.17 22.30 23.56 22.51
198 Fig. 11 Experimental comparative results for Example 3
greyscale values and additional spatial features, extracted in the pixels neighborhood that emphasise and improve the final text regions of the document. First, the feature space is clustered by the Kohonen SOFM. The output neurons of the SOFM define the centres of the class obtained. Using the clustering results obtained by the SOFM and the Euclidean distance as a criterion, the membership functions of every feature vector are obtained. These functions are then taken as the initial staring point of the FCM classifier. Thus, the FCM starts from a point close to the optimal and quickly converges on it. The use of the FCM classifier takes advantages of the fuzzy clustering and results in the reduction of the characters blurring. In the case of general colour reduction, a PWHA technique is applied to a colour component to estimate initially the proper number of the document dominant colours. Even the spatial features that must be used depend on the form and the type degradation of the documents, and we have found that edge, sharpening and smoothing spatial features lead to satisfactory results.
References 1. Kittler J, lllingworth J (1986) Minimum error thresholding. Patt Recog 19:41–47
2. Reddi SS, Rudin SF and Keshavan HR (1984) An optimal multiple threshold scheme for image segmentation. IEEE Tran Sys Man Cybern 14(4):661–665 3. Otsu N (1979) A threshold selection method from grey level histograms. IEEE Tran Sys Man Cybern 9(1):62–69 4. Kapur JN, Sahoo PK and Wong AK (1985) A new method for grey level picture thresholding using the entropy of the histogram. Comp Vis Graph Imag Process 29:273–285 5. Papamarkos N, Gatos B (1994) A new approach for multithreshold selection. Comp Vis Graph Imag Process Graph Mod Imag Proc 56(5):357–370 6. Sahoo PK, Soltani S and Wong AKC (1988) A survey of thresholding techniques. Comp Vis Graph Imag Process 41:233–260 7. Strouthopoulos C, Papamarkos N (2000) Multithresholding of mixed type documents. Engin App Art Intellig 13(3):323–343 8. Strouthopoulos C, Papamarkos N (1998) Text identification for image analysis using a neural network. Imag Vis Comp 16:879– 896 9. Kohonen T (1997) Self-organizing maps. Springer, Berlin Heidelberg New York 10. Haykin S (1994) Neural networks: a comprehensive foundation. MacMillan, New York 11. Yang Y, Yan H (2000) An adaptive logical method for binarization of degraded document images. Patt Recog 33(5):787– 807 12. OGorman L (1994) Binarization and multithresholding of document images using connectivity. CVGIP: Grap Mod Imag Proc 56(6):494–506 13. Parker JR (1991) Gray level thresholding in badly illuminated images. IEEE Trans Patt Anal Mach Intell 13(8):813-819 14. Sauvola J, Pietika¨inen M (2000) Adaptive document image binarization. Patt Recog 33(2):225–236
199 15. Trier OD, Taxt T (1995) Improvement of integrated function algorithm for binarization of document images. Patt Recog Lett 16(3):277–283 16. Liu Y, Srihari SN (1997) Document image binarization based on texture features. IEEE Trans Pattern Anal Mach Intell 19(5):540–544 17. Nauck D, Klawonn F and Kruse R (1997) Neuro-fuzzy systems. Wiley, New York 18. Chi Z, Yan H and Pham T (1996) Fuzzy algorithms: with applications to image processing and pattern recognition. World Scientific, Singapore 19. Sagan H (1994) Space-filling curves. Springer, Berlin Heidelberg New York 20. Chung KL, Tsai YH and Hu FC (2000)Space-filling approach for fast window query on compressed images. IEEE Tran Imag Proc 9(12):2109–2116
21. Papamarkos N, Atsalakis A (2000) Grey level reduction using local spatial features. Comp Vis Imag Under 78:336–350 22. Huang LK, Wang MJ (1995) Image thresholding by minimizing the measure of fuzziness. Patt Recog 28:41–51 23. Papamarkos N, Strouthopoulos C and Andreadis I (2000) Multithresholding of color and grey level images through a neural network technique. Imag Vis Comp 18:213–222 24. Duda RO, Hart PE (1973) Pattern recognition and scene analysis. Wiley, New York 25. Heckbert P (1982) Color image quantization for frame buffer display. Comp Graph 16:297–307 26. Wan SJ, Prusinkiewicz P and Wong SKM (1990) Variance based color image quantization for frame buffer display. Col Res Appl 15(1):52–58 27. Dekker AH (1994) Kohonen neural networks for optimal colour quantization. Ntwk: Comp Neur Sys 5:351–367
O R I GI N A L A R T IC L E
Nikos Papamarkos
A neuro-fuzzy technique for document binarisation
Received: 8 March 2002 / Accepted: 3 July 2003 / Published online: 8 November 2003 Springer-Verlag London Limited 2003
Abstract This paper proposes a new neuro-fuzzy technique suitable for binarisation or, in general, the colour reduction of digital documents. The proposed approach uses the image colour values and additional local spatial features extracted in the neighbourhood of the pixels. Both image and local features values feed a Kohonen self-organised feature map (SOFM) neural network classifier. After training, the neurons of the output competition layer of the SOFM define a first approach of the final classes. Using the content of these classes, fuzzy membership functions are obtained that are next used by the fuzzy C-means (FCM) algorithm in order to obtain the colours of the final document. The method can be applied to greyscale and colour documents; it is suitable for improving blurring and badly illuminated documents and can be easily modified to accommodate any type of spatial characteristics. Keywords Binarisation Æ Colour reduction Æ FCM Æ Fuzzy logic Æ Self-organised neural networks
1 Introduction A complex mixed-type document image contains text, line-drawings and graphics regions. In many practical applications we need only to recognise or improve the text content of the documents. In such cases, it is preferable to convert the documents into a binary form, or at least, into a limited number of grey levels (or colours in the case of colour documents). Doing this, we can transmit and process more efficiently the documents instead of the original greyscale forms. For many years the N. Papamarkos (&) Department of Electrical & Computer Engineering, Democritus University of Thrace, 67100 Xanthi, Greece E-mail: [email protected] Tel.: +30-25410-79585 Fax: +30-25410-79569
binarisation of greyscale documents was based on the standard bilevel techniques that are also called global thresholding algorithms [1, 2, 3, 4, 5, 6]. These statistical methods are suitable for converting any greyscale image into a binary form but are inappropriate for complex documents, and even more so, for degraded documents. In these special cases it is important to take into account the nature and the spatial structure of the document images. Based on this assumption, specialised binarisation techniques have been developed for complex document images. Strouthopoulos and Papamarkos [7] considered the problem of the grey level reduction of complex documents by using a combination of a page layout analysis technique [8] and a Kohonen seld-organised feature map (SOFM) neural network [9–10]. Such techniques are suitable for cleaning documents but do not work efficiently with degraded documents. Some other document binarisation techniques have been developed which are focused on the binarisation of degraded documents [11, 12, 13, 14]. In [12], OGorman proposes an algorithm that is based on the use of local connectivity information. Other techniques are based on some gradient and edge information [13,15], stroke analysis [16] and adaptive logical algorithms [11,14]. A good analysis and comparative results of these techniques are given in [11]. The abovementioned techniques provide good results in many cases. However, they are complex, require an extended analysis of the documents structure and are time consuming. In many applications, such as fax transmission and document retrieval systems, we mainly want the text to be as readable as possible. Therefore, an efficient document binarisation technique must be focused to primarily improve the text regions. The proposed technique for the binarisation and grey level (or colour) reduction of mixed-type documents, in order to improve the form of the final text regions, takes advantages not only of the image pixels values, but also of additional spatial information. Specifically, the proposed technique is based on a hybrid neuro-fuzzy system [17], which consists of a Kohonen SOFM and a fuzzy C-means (FCM) classifier [10,18]. According to the
191
proposed technique, the grey level values of the pixels are related to suitable local spatial features extracted in the pixels neighbouring regions. Thus, the one-dimensional histogram clustering approach of the classical multithresholding techniques is converted into a multi-dimensional feature clustering technique. For each image pixel, its greyscale value or its three colour component values for the case of colour documents are considered as the first feature. The entire feature set is completed by K additional spatial features, which are extracted from neighboring pixels. These features are associated with spatial image characteristics that emphasise and improve the shapes of the characters. In general, depending on the type of problem, the Kohonen SOM has K+1 (for the case of gay-scale images) of K+3 (for the case of colour documents) input neurons. The number of the output neurons is equal to the number of the final colour classes. Thus, for binarisation we have two output neurons and for general colour reduction, independently if the image isgreyscale or colour, the number of output neurons is equal to the number of final greyscale values or colours. In these cases, the training procedure follows the Kohonen learning rules. After training, the output neurons of the competition layer of the SOFM define the dominant classes. Using the content of these classes, fuzzy membership functions are automatically obtained and are next used in the FCM classifier. FCM is a powerful classifier but has two drawbacks associated with the computational cost and the requirement for a ‘‘good’’ starting point to avoid convergence into an unsatisfactory local sub-optimal point. These drawbacks are overcome in the proposed approach by using the SOFM results and a fractal sub-sampling procedure as a starting point. On the other hand, through proper classification the fuzzy data using the FCM classifier a significant character blurring reduction is achieved [18]. As it is analysed in Sect. 3, the proposed technique can be extended to cover the case of the reduction of the document grey levels or colours into a limited number by adding more output classes. In this case, a technique is initially applied that can estimates the proper number of the document greyscales (or colours) and therefore, the proper number of the final image greyscale (or colour) values. In order to reduce the storage requirement and the computation time, the training set can be a representative sample of the image pixels. In our approach, the sub-sampling is performed via a fractal scanning technique, based on the Hilberts space filling curve [19]. The proposed method was tested with a variety of documents. In addition, this paper presents characteristic examples and comparative results that confirm the effectiveness of the proposed method.
2 A description of the method for greyscale documents A digital greyscale document I (i, j), i = 1, ..., n, j = 1,..., m can be considered as an image consisting of n·m pixels, where the value of each pixel is a point in the
greyscale space. Usually, the total number of grey levels is restricted to 256, i.e., I(i, j) 2 [0,255]. The document greyscale reduction problem can be considered as the problem of best transforming the document grey level image into a new one, with only J grey levels, such as the final document to approximate not only the dominant grey level values, but also to preserve the principal text information included in it. A special case of the greyscale reduction problem is the binarisation process where the final image has only two greyscale values, usually 0 for foreground and 255 for background pixels, respectively. An effective approach to solve this problem is to consider it as a clustering problem and achieve its solution using a suitable self-organised neural network and the FCM algorithm. The proposed technique, for the binarisation orthe greyscale reduction of complex and degraded documents, is analytically described in the next section.
2.1 Document binarisation The proposed document binarisation method consists of the following three stages: – Initial document binarisation using a Kohonen SOFM neural network with suitable spatial features – Fuzzy membership extraction – Binarisation using the FCM algorithm The complete analysis of the above stages follows. 2.1.1 Initial document binarisation In order to take advantage on the document texture, it is obvious that we must not only consider the image greyscale values, but also suitable spatial features. It is obvious that most of the information in a document is associated with the text regions and the shapes of the characters. For this reason, local characteristics must be extracted and used in the binarisation process. If these local characteristics are considered as local spatial features, then the binarisation problem can be defined as a procedure that leads to a binary document by taking into account, except for greyscale values, suitable spatial characteristics. Now, the problem can be viewed as a clustering one and its solution can be obtained by using an appropriate classifier such as the Kohonen SOFM neural network. If k spatial features are used then we have a k+1 features clustering problem. The Kohonen SOFM classifier used in this stage is shown in Fig. 1. We can observe that k+1 features feed the neural network and the output consists of only two neurons. Certainly, instead of using the SOFM we could use another type of classifier as, for example, the KNN. However, we have found in our previous work that the Kohonen SOFM is a powerful classifier for general colour reduction [21]. It should be noticed that the spatial features must be suitably selected such as the document text areas to be
192
Fig. 1 The Kohonen SOFM classifier
enhanced. Different types of document degradation require different types of spatial features. From our experience we have found that edge extraction and sharpening masks, in combination, in some cases, with mean or min local values provide a good combination for the spatial feature set. Especially, the Laplacian feature values obtained by the 3·3 mask: 2 3 1 1 1 f2 ¼ 4 1 8 1 5 ð1Þ 1 1 1
Fig. 2 Document sub-sampling using the Hilbert curve
in cooperation with the min value in the 3·3 neighboring region of the sampling pixels constructs a suitable set of spatial features for document binarisation. To speed up the entire process and to reduce memory requirements, a fractal scanning sub-sampling technique based on the Hilbert curve is adopted [19] (Fig. 2). The important feature of the Hilbert curve is that it scans continuously the neighboring entry in the image [20]. As a first example, let us considered the badly illuminated document of Fig. 3a. Figure 3b depicts the binary images obtained by the method of Otsu. Figure 3c shows the binary image obtained by the method of Papamarkos and Atsalakis [21]. As it can be observed, both techniques result in a dark area on the left side of the document. Finally, Fig. 3d shows the binarisation results obtained in the first stage of the proposed algorithm where the Laplacian and min additional spatial features are applied. As it can be observed, the binary document obtained is better and does not have any dark area. 2.1.2 Fuzzy membership functions In order to improve the binarisation results by reducing the blurring effects, we propose a fuzzy procedure that can classify efficiently fuzzy pixels into foreground or background pixels. This technique requires a fuzzy membership extraction procedure that determines the membership functions of the two classes defined by the two output neurons of the SOFM. Referring to the abovementioned example, it is clear that each feature vector, which consists of three independent features, is classified in one of the two output classes. Let dij, i =
Fig. 3 a Original document. Document binarisation using the methods of b Otsu c Papamarkos and Atsalakis, and d SOFM with spatial features
1,...,n·m, j = 1,2 be the Euclidean distances of the feature vectors from the centres of the two classes obtained by the Kohonen SOFM. The membership functions for the two classes are obtained through the following relations: u1;j ¼
1 1 þ dj1 l1 =D1
ð2Þ
193
1 1 þ dj2 l2 =D2
u2;j ¼
ð3Þ
where li, i =1,2 is the mean Euclidean distance of class i, and Di ¼ maxðdji Þ minðdji Þ
ð4Þ
A similar definition of the membership functions is proposed by Huang and Wang [22].
other for the foreground pixels. The important effect of using the FCM is the reduction of the document blurring. Figure 4a depicts the document obtained after the application of the FCM algorithm. Comparing Figs. 3d and 4a it can be observed that the document in Fig. 4a is sharper than the document extracted by the SOFM. As a final stage, some simple post-processing tasks can be applied. For example, a mean 3·3 filter followed by a global binarisation procedure fills the character holes and makes them thicker. The results of the application of the post-processing procedure are depicted in Fig. 4b.
2.1.3 The final binarisation via the FCM algorithm After the determination of the membership functions, the clustering problem is converted into a fuzzy clustering one. The solution of this problem can be easily obtained by using the well-known FCM algorithm [18] where as initial membership values are those obtained by the method described above. The description of the FCM used in our approach is as follows. Let f1, f2,...,fn are the n feature vectors V = {v1, v2} are the cluster centres U = {uij} is the 2·n matrix of the membership values that satisfy the conditions: 06uij 61; i ¼ 1; 2 and j ¼ 1:::; n 2 X
uij ¼ 1;
ð5Þ
j ¼ 1; :::; n
ð6Þ
i ¼ 1; 2
ð7Þ
i¼1 n X uij\n; 0\ j¼1
The FCM algorithm is iterative and makes use of the following equations: n P j¼1
um ij fij
vi ¼ P n
i ¼ 1; 2
;
ð8Þ
2.2 An extension to the reduction of the documents greyscale values The proposed technique can be extended to cover the case of reduction of the image greyscale values into a limited number. This is similar to the multithresholding case, where we try to find appropriate thresholds Ti, i = 1,...,N)1 in the image histogram in order to reduce the number of the image greyscale values to N. In the proposed approach, if N is the desired number of final greyscale values, then the Kohonen SOFM must have N output neurons. The input vectors remain similar to the case of binarisation, that is, they consist of every pixel of the pixels greyscale value and additional values of the spatial features obtained locally. After training, each output neuron is associated with the centre of the feature classes obtained. A similar approach has been used in [23]. Now, using the Euclidean distance from the centre of the classes as a criterion, the membership functions, for every feature vector and for every class can be obtained. Specifically, if: N fi ; i ¼ 1; :::; n vj ; j ¼ 1; :::; N dij ; i ¼ 1; ::; n; j ¼ 1; :::; N
is the number of output neurons ðclassesÞ are the feature vectors are thecluster centers are the Euclidean distances
um ij
j¼1
h uij ¼
1 jfk vi j2
2 h P j¼1
i1=ðm1Þ
1 jfk vi j2
i1=ðm1Þ
;
i ¼ 1; 2 and k ¼ 1; 2; :::; n
ð9Þ where m is the fuzziness factor, m >1. The iterative procedure stops when there is not a significant change in the objective function: J ðU ; V Þ ¼
2 X n X
2 um ik jfk vi j
ð10Þ
i¼1 k¼1
The FCM algorithm classifies the document pixels into two classes, one for the background pixels and the
Fig. 4 a The document after the application of the FCM b The final document after post-processing
194
then the membership functions are obtained from the relation: h i1=ðm1Þ uik ¼
1 dik2
N h i1=ðm1Þ P 1 j¼1
;
i ¼ 1; :::N ;
and k ¼ 1; :::n
dik2
ð11Þ The clusters obtained by the Kohonen SOFM, and the membership values calculated from Eq. 11, are considered as the initial starting point of the FCM classifier. The use of the FCM algorithm provides more flexibility in the clustering results obtained. An important issue of the greyscale reduction of documents is the estimation of the proper number of the final greyscales. This problem can be solved by examining the validity of clusters obtained [18]. However, this approach leads into an adaptive and time-consuming procedure. To overcome these difficulties, the piece-wise histogram approximation (PWHA) algorithm, described in the next section, can be used. We present below an example of the applications of this technique for the reduction of the number of greyscales in a mixed-type document.
1. A line ( k1, h(k1)) ) ( k2, h(k2)) is not divided if k2 ) k1 < d where d1 a predefined threshold value. 2. A line ( k1, h(k1)) ) ( k2, h(k2)) is not split more if |h(k2) ) h(k1)|< d2 max(h), where max(h) the maximum value of the histogram, and d2 a suitable coefficient (usually d2 = 0.01). 3. A line is not split more if the maximum distance of the histogram points from this line is less than d3 max (h), where the threshold d3 is usually taken equal to 0.002. In order the have an acceptable limited number of histogram peaks, a histogram smoothing procedure can be applied. According to this technique, every value of the histogram is substituted by the mean value in a neighbouring: hðkÞ¼meanfhðk 0:5N þ0:5Þ;:::;hðkÞ;:::;hðk þ0:5N þ0:5Þg ð12Þ where N is usually taken equal to five. We call this technique piece-wise smoothing histogram approximation (PWSHA) and, as the experimental results show, this technique probably gives the most advantageous results.
An extension to colour documents 2.3 The PWHA This technique is based on the curve-fitting algorithm of Duda and Hart [24] which can be considered as a linear PWHA procedure. Specifically, the histogram curve is approximated by a set of straight-line segments. Fitting and convergence criteria are applied to control the entire procedure. According to the PWHA algorithm, the shape of a histogram can be approximated through a set of straight-line segments that are determined by means of a simple iterative process. This technique is relatively immune to small histogram bin variation (due to noise) by suitably choosing the fitting parameters. At the end of the fitting process the principal peaks of the histogram are obtained and their number specifies the optimal numbergreyscale values (ONPGV) of the image. If h(j) is the histogram function, then h(j) can be approximated as follows: initially h(j) is approximated by the straight-line 1 which passes through points (j0,h(j0)) and (j1,h(j1)), where j0 = 0 and j1 = 255. Then, the histogram point (j2,h(j2)), with j0 < j2 < j, having the maximum distance from 1, is obtained and the histogram curve is now approximated by the two lines 2 and 3 defined by the points (j0, h(j0)) ) (j2, h(j2)) and (j2, h(j2)) )(j1, h(j1)), respectively. This fitting procedure is repeated until some convergence conditions are satisfied. At the end of the PWHA procedure, the histogram scheme is approximated by a set of line segments. The convergence conditions for the PWHA algorithm are:
The proposed technique can be extended to the case of colour documents. The goal in this case is to achieve colour reduction of the document image. In general, an RGB colour document Ic(i,j), i = 1,...,n, j = 1,...,m and c = R, G or B can be considered as a colour image consisting of n ·m pixels, where the value of each pixel is a point in the 3D RGB space. Usually, the total number of colours is restricted to 2563 and each colour component is in the range [0,255]. In the case of colour documents, the Kohonen SOFM is fed by the three colour components and additional spatial feature values extracted from the image colour components. For example, if the RGM colour model is used, in every pixel the input vector consists of the three R, G and B components and additional values of the spatial features extracted locally from the colour components images. That is in every pixel (x,y) the feature vector has the following form: T Fðx;y Þ ¼ Rðx;y Þ ; Gðx;y Þ ; Bðx;y Þ ; f4 ; f5 ; f6 ; :::; fn
ð13Þ
The number of the output neurons is equal to the number of the final colours. After training, the centres of the output classes obtained define the final principal colours obtained. In the second stage, the Euclidean distance is used as a criterion in order to calculate the values of the membership functions, for every feature vector and for every class that can be used in the next with the FCM classifier. Again, and for the RGB colour model, if:
195
N fi ; i ¼ 1; :::; n vj ; j ¼ 1; :::; N dij ; i ¼ 1; ::; n; j ¼ 1; :::; N
is the number of output neurons ðclassesÞ are the feature vectors are thecluster centers are the Euclidean distances
then the membership functions are obtained again from Eq. 11. The centres of the final classes obtained, after the application of the FCM, define the final principal colours. In the case of colour documents, the estimation of the proper number of image principal colours can be done by applying the PWHA algorithm in the histogram of a specific colour component. An analytical and characteristic example is given in the experiment session that clarifies the technique for colour reduction in a document.
3 Examples 3.1 Example 1 In this experiment we apply the proposed method to the degradedgreyscale document of Fig. 5a. This document has many line drawings and graphics regions and is Fig. 5 a Original document b Binarisation using the SOFM c Binarisation using the proposed technique d The document after post-processing
badly illuminated. The binarisation of this document using thresholding techniques leads to unaccepted results. Figure 5b shows the binarisation results obtained by the SOFM without using any spatial features. Figure 5c shows the results obtained by the proposed method, where in the SOFM we use the Laplacian mask of Eq. 1. Figure 5d depicts the final image obtained if we apply as post-processing operations the mean 3·3 mask followed by the global binarisation via the SOFM. 3.2 Example 2 This example demonstrates the application of the proposed technique to the reduction of the number of grey levels in a digital document. The original badly illuminated mixed-type document, shown in Fig. 6, includes text and graphics regions. Its size is 1058·946 pixels with a 300 dpi resolution. In order to obtain the proper number of image grey levels, the PWSHA technique is initially applied with a smoothing value N=5, and d1=6, d2=0.01 and d3=0.002. As it can be observed in Fig. 7, the proper minimum value of imagegreyscales obtained is equal to five. The proposed technique is first applied for image binarisation without using spatial features. The image obtained is shown in Fig. 8a. We can observe that due to the mixed-type of the document and the low-scanning quality, the document binarisation results in an unac-
196
Fig. 7 Using the PWSHA technique for the estimation of the proper number of image greyscales Fig. 6 The original mixed-type document
ceptable document form with many broken characters. Even if we apply the powerful adaptive logical techniques of Yang and Yan [11], the binarisation results, shown in Fig. 8b, are still unsatisfactory. In this case we have better character segmentation results but in the graphics areas only edge pixels survive. Then, the proposed technique is applied to reduce the number of grey levels to five but without using spatial features. The image obtained is shown in Fig. 8c and the fivegreyscales are equal to 28, 92, 153, 202 and 246. As it is expected, in comparison with the binary image of Fig. 8a, the quality of the image has been improved. However, in some parts of the documents the characters appear to be blurred. To reduce the blurring effects, we apply the proposed technique with three sharpening spatial features extracted by the following masks:
" 1 1 1 # f2 ¼
1 1 " 1
f4 ¼
9
1 ;
1 2
1 1 #
2
5
2
1
2
1
" 0 f3 ¼
1
0 #
1
5
1 ;
0
1
0
The final image obtained is shown in Fig. 8d and has the five dominantgreyscale values: 38, 95, 159, 203 and 243. Comparing Figs. 8c and 8d it can be easily observed the improvement of the image blurring (this is obvious especially in the text area in the bottom of the document). Fig. 8e depicts the image pixels that have the maximum classification vagueness, i.e., their maximum membership values are close to 0.5. For comparison reasons, we reduce the imagegreyscale values to five by using other techniques. Figure 9 depicts the results obtained by the methods of Reddi
Fig. 8 a Image binarisation b Binarisation using the method [11] c The document with fivegreyscales obtained without using spatial features d The final document with five grey levels using the spatial features e The image pixels with a large vagueness
197
Fig. 10 The original colour document
be noticed again that the proposed technique, using a different set of spatial features, results in different images.
4 Example 3
Fig. 9 Reduction to five greyscales using these methods: a Reddi et al. b Median-cut c Minimum-variance d Dekkers
et al. [2], median-cut [25], minimum variance [26] and Dekker [27]. However, it is well known that there is not any objective criterion that can measure the perceived difference between the original and the final image. The perceived difference can be considered also as perceived image quality. In general, the type of application defines when an image is better than others. From the results obtained in this example, it is clear that the proposed technique had better perceived image quality results. On the other hand, in literature there are some image dependent distortion measures. The comparative results presented here are based on the following six criteria: – – – – –
mean squared error (MSE) Euclidean distance (ED) signal-to-noise ratio (SNR) maximum absolute difference (MAD) peak signal-to-noise ratio (PSNR)
Table 1 gives the values for the five criteria. These values clearly indicate that the superior results of the image of Fig. 8b, which obtained by the basic part of the proposed method (without using spatial features). It is interesting that even the maximum absolute difference is better than the other techniques. Finally, it should Table 1 Comparative results Proposed without spatial features Proposed with spatial features Reddi et al. Median-cut et al. Minimum variance Dekkers
This example demonstrates the application of the proposed technique for reduction of the number of colours in a digital colour document. The original document is shown in Fig. 10. This is not a trivial colour document because it has white and black text, graphics and no uniform background. However, it can be easily observed that the number of its dominant colours is only six. In order to smooth the background and obtain only the dominant colours we used: 1. a 3·3 mean filter for extraction of local features, and 2. the Lu*v* colour space. For comparison reasons, we perform binarisation, and colour reductions using other approaches. It can be easily concluded that the proposed technique, with the specifications defined above, results in better colour reduction results for the document. Figure 11 summarises the results obtained in the all the above cases.
5 Conclusions This paper proposes a neuro-fuzzy technique for binarisation and general colour reduction of badly illuminated documents. The method is based on a hybrid neuro-fuzzy classifier that consists of a Kohonen SOFM and a FCM. The feature vector consists of the imageMSE
ED
SNR
MAD
PSNR
171.54 333.54 625.66 382.20 287.02 365.07
11.08 14.02 20.24 12.38 13.80 12.06
229.03 117.33 56.96 104.52 135.95 107.74
34 112 62 86 71 92
25.79 22.90 20.17 22.30 23.56 22.51
198 Fig. 11 Experimental comparative results for Example 3
greyscale values and additional spatial features, extracted in the pixels neighborhood that emphasise and improve the final text regions of the document. First, the feature space is clustered by the Kohonen SOFM. The output neurons of the SOFM define the centres of the class obtained. Using the clustering results obtained by the SOFM and the Euclidean distance as a criterion, the membership functions of every feature vector are obtained. These functions are then taken as the initial staring point of the FCM classifier. Thus, the FCM starts from a point close to the optimal and quickly converges on it. The use of the FCM classifier takes advantages of the fuzzy clustering and results in the reduction of the characters blurring. In the case of general colour reduction, a PWHA technique is applied to a colour component to estimate initially the proper number of the document dominant colours. Even the spatial features that must be used depend on the form and the type degradation of the documents, and we have found that edge, sharpening and smoothing spatial features lead to satisfactory results.
References 1. Kittler J, lllingworth J (1986) Minimum error thresholding. Patt Recog 19:41–47
2. Reddi SS, Rudin SF and Keshavan HR (1984) An optimal multiple threshold scheme for image segmentation. IEEE Tran Sys Man Cybern 14(4):661–665 3. Otsu N (1979) A threshold selection method from grey level histograms. IEEE Tran Sys Man Cybern 9(1):62–69 4. Kapur JN, Sahoo PK and Wong AK (1985) A new method for grey level picture thresholding using the entropy of the histogram. Comp Vis Graph Imag Process 29:273–285 5. Papamarkos N, Gatos B (1994) A new approach for multithreshold selection. Comp Vis Graph Imag Process Graph Mod Imag Proc 56(5):357–370 6. Sahoo PK, Soltani S and Wong AKC (1988) A survey of thresholding techniques. Comp Vis Graph Imag Process 41:233–260 7. Strouthopoulos C, Papamarkos N (2000) Multithresholding of mixed type documents. Engin App Art Intellig 13(3):323–343 8. Strouthopoulos C, Papamarkos N (1998) Text identification for image analysis using a neural network. Imag Vis Comp 16:879– 896 9. Kohonen T (1997) Self-organizing maps. Springer, Berlin Heidelberg New York 10. Haykin S (1994) Neural networks: a comprehensive foundation. MacMillan, New York 11. Yang Y, Yan H (2000) An adaptive logical method for binarization of degraded document images. Patt Recog 33(5):787– 807 12. OGorman L (1994) Binarization and multithresholding of document images using connectivity. CVGIP: Grap Mod Imag Proc 56(6):494–506 13. Parker JR (1991) Gray level thresholding in badly illuminated images. IEEE Trans Patt Anal Mach Intell 13(8):813-819 14. Sauvola J, Pietika¨inen M (2000) Adaptive document image binarization. Patt Recog 33(2):225–236
199 15. Trier OD, Taxt T (1995) Improvement of integrated function algorithm for binarization of document images. Patt Recog Lett 16(3):277–283 16. Liu Y, Srihari SN (1997) Document image binarization based on texture features. IEEE Trans Pattern Anal Mach Intell 19(5):540–544 17. Nauck D, Klawonn F and Kruse R (1997) Neuro-fuzzy systems. Wiley, New York 18. Chi Z, Yan H and Pham T (1996) Fuzzy algorithms: with applications to image processing and pattern recognition. World Scientific, Singapore 19. Sagan H (1994) Space-filling curves. Springer, Berlin Heidelberg New York 20. Chung KL, Tsai YH and Hu FC (2000)Space-filling approach for fast window query on compressed images. IEEE Tran Imag Proc 9(12):2109–2116
21. Papamarkos N, Atsalakis A (2000) Grey level reduction using local spatial features. Comp Vis Imag Under 78:336–350 22. Huang LK, Wang MJ (1995) Image thresholding by minimizing the measure of fuzziness. Patt Recog 28:41–51 23. Papamarkos N, Strouthopoulos C and Andreadis I (2000) Multithresholding of color and grey level images through a neural network technique. Imag Vis Comp 18:213–222 24. Duda RO, Hart PE (1973) Pattern recognition and scene analysis. Wiley, New York 25. Heckbert P (1982) Color image quantization for frame buffer display. Comp Graph 16:297–307 26. Wan SJ, Prusinkiewicz P and Wong SKM (1990) Variance based color image quantization for frame buffer display. Col Res Appl 15(1):52–58 27. Dekker AH (1994) Kohonen neural networks for optimal colour quantization. Ntwk: Comp Neur Sys 5:351–367
