Using Wavelet Extraction for Haptic Texture Classification
Using Wavelet Extraction for Haptic Texture Classification
Waskito Adi1, Suziah Sulaiman1 1
Computer and Information Sciences Department, Universiti Teknologi PETRONAS Bandar Seri Iskandar, 31750 Tronoh, Perak, Malaysia {[email protected], [email protected]}
Abstract. While visual texture classification is a widely-researched topic in image analysis, little is known on its counterpart i.e. the haptic (touch) texture. This paper examines the visual texture classification in order to investigate how well it could be used for haptic texture search engine. In classifying the visual textures, feature extraction for a given image involving wavelet decomposition is used to obtain the transformation coefficients. Feature vectors are formed using energy signature from each wavelet sub-band coefficient. We conducted an experiment to investigate the extent in which wavelet decomposition could be used in haptic texture search engine. The experimental result, based on different testing data, shows that feature extraction using wavelet decomposition achieve accuracy rate more than 96%. This demonstrates that wavelet decomposition and energy signature is effective in extracting information from a visual texture. Based on this finding, we discuss on the suitability of wavelet decomposition for haptic texture searching, in terms of extracting information from image and haptic information. Keywords: Texture recognition, supervised learning, machine learning, haptic texture search engine, wavelet decomposition.
1 Introduction Lately, computer vision has become one of the most popular research subjects. Computer vision contributes in various fields that include medical, engineering, and robotics. One of the most interesting topics in computer vision is texture recognition. Being the smallest entity, textures could be used as a parameter to recognize a particular object. There are many different types of applications involving texture analysis, including medical imaging, industrial inspection, remote sensing, document segmentation, and computer based image retrieval [1]. Feature extraction plays an important role in a classification process. The effectiveness of such classification relies greatly on the choice of this feature. In this case, a suitable extraction algorithm influences the process end result. Parallel to computer vision, computer haptic is also another area which is gaining its popularity among researchers. Computer haptic enable user to touch and interact with the virtual
objects. Similar to computer vision, haptic texture is a growing research topics and its application has been noted in medical field, arts, and textile, to name a few. Even though computer vision and computer haptic are among the favorite topics of late, no attempt has been made to integrate both texture classifications into the same system such as a search engine. Current practices have seen visual and haptic textures been dealt with separately; and the trend is that visual aspect has been the dominant topics of the two texture types. The absence of such integration may hinder one from getting a complete set of surface information of a particular object. This is because visual textures could inform us on the macro-geometric information while haptic textures on the micro-geometric details [15]. The lacking in combining both visual and haptic textures needs addressing. In this paper, we attempt to address the issue of lacking the effort in integrating visual and haptic textures. We propose a way to analyze and classify haptic textures based on an approach used from computer vision. The focus of our work here is on visual textures and examines how such result could inform those researchers working in haptic texture classifications. This involves conducting an experiment to test the visual texture classification and feature extraction performance. In the classification process, supervised learning has been used as a testing method. The outline of this paper is organized as follows: in section 2, we review some related works pertaining to visual and haptic textures analysis. This is followed by section 3 in which we describe the method involved in our experiment that include wavelet transformation, and statistical classification. In section 4, we present our scheme in extracting texture information, and also how the experiments were conducted. Finally, in section 5 we analyze the experimental results and discuss the findings.
2 Related Works
2.1 On Visual Textures Visual texture analysis has been studied over the past three decades. Most of the early research focused on the analysis of the first or second order statistics of textures and stochastic model such as Gaussian Markov random fields and auto regression [2]. Recently, the development of texture classification and analysis has been using frequency and spatial analysis, such as Fourier transform, Gabor filter, and Wavelet transform. Since a texture image is in 2D representation, measuring its characteristics could be done by employing a 2D transformation or filtering. For example, to extract texture information, a 2D Fourier transform can be employed. Azencott et al [3] define a distance between textures for texture classification from texture features based on windowed Fourier filters. In the research, Fourier transformation is used as a method to extract information from textures. In employing a more advanced technique, Fountain [4] has conducted an experiment using Gabor filter for texture extraction. Also, in [7], Miaohong conducted an experiment using Gabor filter to extract features at different orientation
from hyperspectral images. In that particular research, it is said that accuracy of the classification is over 90%. Using a different technique, Shankar conducted an experiment [5] by using wavelet transform to extract information from finger print. Similarly, Chi-Man Pun [6] has also conducted an experiment using wavelet to extract information from textures. It is reported in his research that wavelet gives a better result than Gabor filter. In this particular case, although Gabor filter is a multiresolution filter, wavelet transform still outperforms. 2.2 On Haptic Textures Texture is the smallest independent quantity in an object. Research works that involve texture are mostly conducted in image and signal processing. When including texture in haptics field, image and signal processing algorithm will be involved. To create a realistic object the simplest way is by employing sine wave to generate the texture surfaces. In [16] Seungmoon Choi and Hong Z. Tan employed sine wave to render the texture surface in order to analyze the stability during haptic texture rendering. In order to simulate the surface of haptic texture, there are some algorithms that can be used [16][17]. The simplest one is by using sine wave. By adjusting amplitude and frequency of the sine wave the texture of surfaces could be obtained. In [14] Wall developed a haptic texture rendering technique using Fourier series. The basic idea of this rendering technique is similar with rendering using sine wave because Fourier series using sine wave as basis to create new wave. In order to create realistic texture for haptic rendering, Juan Wu introduced a new rendering method based on image processing [17].
3 Methodology
3.1 Equating Haptic Texture to Visual Texture To obtain a texture sensation, a simple technique such as sinusoidal waves could be used to render the force feedback. Different texture pattern or profile is obtained by adjusting frequency, and amplitude of the sine waves which could result in different haptic sensation. For example, Wall [14] conducted an experiment that simulates texture surface by employing Fourier series to achieve different types of haptic texture surface. The texture profile used in haptic rendering and its 2D image counterpart share a similar characteristic. When rendering a property of an object, such as its haptic texture using sinusoidal waves, the (haptic) surface will have surface properties similar to (visual) sine waves, but in 3D plane. However, when these haptic properties are plotted onto a 2D plane they will appear similar to 2D images. Fig. 1 shows a 2D representation of sine and square haptic waves (Fig 1.1 & 1.2) and the actual visual images (Fig 1.3 & 1.4).
Fig 1.1. sine wave (haptic) texture
Fig 1.2. square wave (haptic) texture
Fig 1.3 cloth texture (visual)
Fig 1.4 cloth texture (visual)
Fig. 1. Image visualization from haptic texture
Comparing the four images in Fig. 1, it could be assumed that haptic texture images generated using both sine square waves have similar patterns with those non artificial visual textures (Fig. 1.3 & 1.4). Based on the similarity of their appearances on a 2D plane, we could claim that computer vision (CV) algorithm could also be employed to extract information from haptic rendering data. 3.2 Determining a Method to Represent Haptic Texture Transformation is one of the most powerful tools to analyze signal information. With transformation, signal can be analyze and manipulated. The most common method to analyze signal is by using Fourier transform. Fourier transforms use sine wave as basis of the transformation, by using Fourier transform assumption, it can be inferred that every signal is constructed by sine waves superposition. Fourier transform is denoted as: ∞ (1)
F ( w) =
∫ f ( x )e
− 2πiωx
∂x
−∞
The inverse of Fourier transform, which allows going from spectrum to the signal is: ∞ (2)
f ( x) = ∫ F ( x)e 2πiωx ∂ω −∞
Consider a square wave, (Fig. 2), f (x) of length 2 L .The range [0,2 L] , could be written:
x x f ( x ) = 2 H − H − 1 − 1 (3) L L By using Fourier transform, information of square signal can be retrieved. In this case, square signal can be decomposed into different sine wave frequency and amplitude. Referring to the Fourier transform, if a square wave equation is transformed, it will produce a Fourier series as follows: ∞ (4) 4 1 nπx
f ( x) =
π
∫
n n =1, 3, 5 ,...
sin L
Unlike Fourier series that uses sine wave, wavelet is a transformation that uses transient signal as its basis. In other word wavelet transform is better than Fourier for analyzing transient signal. Wavelet transform of f (x) is denoted as: (5) W f (a, b) = ψ ( x ) f ( x)∂x ψ
∫
a ,b
R
Consider a signal shown in (Fig. 2), in wavelet analysis.
Fig. 2. Decomposition of square wave using Fourier Transform
The signal can be decomposed into different small basis transient signal. Otherwise if Fourier transform is employed, the signal will be decomposed into different amplitude and frequency of sine wave.
Fig 3. Illustration of 1-D wavelet transforms. Unlike Fourier transform that used sine wave as basis, wavelet transform use small transient signal to transform or decompose the source signal.
Wavelet multiresolution analysis has been used as texture characterization. It provides spatial or frequency information of an image. There are two method extractions in wavelet multiresolution: (i) wavelet decomposition (ii) wavelet packet decomposition.
In our experiment we use 2D wavelet decomposition to extract texture images. The decomposition offer results for image analysis. Although it is not as rich as 2D wavelet packet (2D-WPT), it is adequate to do analysis to an image with 2D wavelet decomposition. In 2D wavelet decomposition transform (2D-WDT), an image x is split into an approximation and three detail image (Fig. 3). 2D-WDT can be described as a pair of filter L and H [9], H is low pass filter, and L is high pass filter. The 2DWDT of a discrete N x M image can be recursively defined as: (6) A = C p +1 = ∑∑ h(m)h(n)C p 1
k ,( m+ 2 i , n + 2 j )
4 k ,( i , j )
m
n
H1 = C 4pk++11,(i , j ) = ∑∑ h(m)h(n)C kp,( m+2 i ,n +2 j ) m
(7)
n
V1 = C 4pk++12,(i , j ) = ∑∑ h(m)h(n)C kp,( m+ 2i ,n + 2 j ) m
n
m
n
(8)
D1 = C4pk++13,(i , j ) = ∑∑ h(m)h(n)Ckp,( m+2i ,n + 2 j )
(9)
A1 is approximation coefficient, H1 is horizontal detail, V1 is vertical detail, and D1 is diagonal detail, illustrated in (Fig. 4). In practice, the image X only has finite number of pixels, for boundary handling different technique such as symmetric, periodic could be used. The difference between 2D-WDT and 2D-WPT is in the extraction process. In 2D-WDT only approximation coefficient will be transformed whereas in 2D-WPT all of the coefficient will be recursively transformed. In the proposed method, an image is decomposed into 4-levels. 4-level decomposition results in 3n+1 sub-band, it means that decomposition results in 13 sub-bands. The general transformation processes compress information from data. In wavelet transform, if the basis signal for transformation has correlation with transformed signal it will produce homogenous transformed coefficient. Referring to those characteristics, characteristics of each sub-band can be used as information to form feature vector. In order to form a feature vector, f=(x1, x2, x3,. . ., xn), the energy or characteristic each sub-band of images have been calculated using formula (Table 1). Table 1. Energy Calculation Formula Energy Type Norm 2 Norm 1
Formula
1 NM 1 e2 = NM e1 =
Standard Deviation
e3 = Average Residual
N
2 M ,N
m =1 n =1 M N
∑∑ C
M ,N
m =1 n =1
1 NM
M
N
∑∑ (C
M ,N
− µ)2
m =1 n =1
e4 = ∑∑ C M , N − µ M
Entropy
M
∑∑ C
e5 = −
N
1 NM
∑∑ C M
N
2 M ,N
log C M , N
2
It is a common issue that data stored in feature vector have redundancy. To reduce redundancy form the feature vector, PCA (Principle Component analysis) algorithm should be employed [7][12][1]. With PCA algorithm, classification process could be more effective and efficient 3.3 Testing the Classification Performance In our research, we will employ the supervised learning to classify the texture. In our study, in order to test the classification performance, we employ linear, quadratic and mahalanobis classifiers. Bayesian rule has been used as a fundamental concept for the entire classifiers. Bayesian rule is denoted as:
P(ω j | x) =
p( x | ω j ) P(ω j )
(10)
p( x )
Let say there are two classes of data ω1 and ω2, P(ωj) is prior probability and p(x) is probability density function (PDF). Referring to the maximum likelihood theorem, by maximizing the conditional probabilities P(ωj|x) decision of the class could be obtained. In this case if P(ω1|x)>P(ω2|x) the data will belong to class ω1, otherwise if ω2 data belong to ω2. Referring to Bayesian rule, if it assumes PDF is a normal distribution or Gaussian, linear, quadratic and mahalamobis classifier could be obtained. Quadratic, linear and mahalanobis could be denoted as: Quadratic classifier: 1 max | − ( x − µ j ) T ∑ −j1 ( x − µ j ) + ln P(ϖ j ) − ln | ∑ j | 2
(11)
Linear Classifier: 1 max | − ( x − µ j ) T ∑ −j1 ( x − µ j ) + ln P(ϖ j ) 2
(12)
Mahalanobis Classifier: max | −( x − µ j ) T ∑ −j 1 ( x − µ j )
(13)
To obtain the data classes (refer to equation 11, 12, 13), there is a need to maximize the conditional probability. In equations 11 and 12, all data are calculated based on the relative distance between the data and data distribution, and also using prior probability of the class as weight parameter. In mahalanobis classifier algorithm, (equation 13), it calculates relative distance between the data and the data distribution.
4 Proposed Framework and Experiment To this end, we have claimed that computer haptic texture classification could replicate similar technique from computer vision. In the previous section, we have
described how wavelet and its feature extraction could be a potential candidate to address this situation. Fig. 4 presents a proposed framework for a system that can adopt both visual and haptic search. The framework combines both visual and haptic texture extractions and classifications.
x1 x2 f = x3 ... x n
Fig. 4. Proposed Framework involving the extraction and classification processes
The visual information as shown in Fig. 4 consists of the texture image data; this information is passed to the extraction process. The process involves the wavelet decomposition (using db-8) to get the characteristics from each image. At this stage the energy calculation formula will be used. The feature vector f(x) is formed as a result of this process. In the classification stage, we employ the linear, quadratic and mahalanobis classifiers to examine the most suitable classifier in terms of its performance. Using the chosen classifier, the system will suggest some possible matching results. 4.1 Experiment The effectiveness of wavelet decomposition for texture analysis and classification has been tested in an experiment, using 76 samples of texture images. The texture set consists of natural, artificial, and semi natural textures. The objectives of the experiments are: • To determine the texture classification performance for each classifier • To identify the most effective and efficient variable to be used in a classification process. During the performance test, we use three different types of classifiers. We employ linear, quadratic, and mahalanobis classifiers. Daubechies wavelet family has been employed as an algorithm to extract the textures. Table 2 shows 16 tap of daubechies wavelet transform filter coefficient.
Table 2. 16 Tap of Daubechies-8 Wavelet Transform Filter Coefficient
h(0) h(1) h(2) h(3) h(4) h(5) h(6) h(7)
0.0385 0.2212 0.4777 0.4139 -0.0112 -0.2008 0.0003 0.0910
h(8) h(9) h(10) h(11) h(12) h(13) h(14) h(15)
-0.0123 -0.0312 0.0099 0.0062 -0.0034 -0.0003 0.0005 -0.0001
The textures have been decomposed into 4-level. It means that decomposition processes results in 13 coefficient sub-band. To conduct the experiment we prepare a learning sample and testing data sets. We create a random 512x512 pixel image from the texture image. For each 512x512 pixel image we partition this image into 4 265x265 without overlapping each other. We generate 100 random images 128x128 from each 256x256 images. In other words, for each sample we have 400 samples. We use 200 samples for learning process and the other 200 samples for testing. To test the performance of the classification, we conduct three experiments with different feature vectors and different classification methods. We employed the linear, quadratic, and mahalanobis classifiers in the experiments, consecutively. For the first experiment, we use the entire feature of vector element, meaning that all the extracted information was used. The second experiment was to test the effect of the element number with classification process. We start by using 1 feature vector element up to the entire elements. In the third experiment, we use only one type of energy calculation algorithm.
5 Findings and Analysis As described earlier, in the experiment we have used the three classifiers to measure the accuracy of classification performance. In each classification process we measure the error rate of each method. The classification testing involves: 1) Measuring the number of feature vector element versus error rate 2) Measuring the effect using different type of feature vector 3) Comparing the classification method 4) Finding the optimum number of feature vector element The first experiment was conducted by measuring the error rate for each feature vector that have different element. Since we have Level-4 wavelet decomposition, it means that we have 13 different sub-bands. For each sub-band we calculate the energy using the formula in (Table 1), which means we have 65 different values to form a feature vector. In this experiment we form a feature vector started using 1 element to 65 elements, f(x1) to f(x1, x1, x1,. . ., x65). In this test, quadratic classifier gives the most accurate results when compared with linear and mahalanobis classifiers. The classification results are summarized in Figure 5 and Table 3.
Table 3. 16 Tap of Daubechies-8 Wavelet Transform Filter Coefficient Energy 1 2 3 4 5 6 7 8 9 10 11 12 13
Norm1 0.8661 0.6899 0.4606 0.3198 0.3068 0.1672 0.1072 0.0917 0.0527 0.0348 0.0210 0.0126 0.0068
Norm2 0.8656 0.8379 0.8200 0.7755 0.7460 0.7065 0.6691 0.6361 0.6205 0.6057 0.5903 0.5580 0.5432
Std dev 0.8584 0.7063 0.5540 0.5337 0.3248 0.1607 0.1057 0.0744 0.0610 0.0562 0.0327 0.0199 0.1066
Ave. residual 0.9212 0.5719 0.3503 0.2409 0.2226 0.1461 0.1022 0.0605 0.0311 0.0228 0.0179 0.0140 0.0119
Entropy 0.8666 0.6389 0.5241 0.4041 0.3936 0.2459 0.1727 0.1594 0.1113 0.0840 0.0683 0.0524 0.0411
We carried out a second experiment similar to the first but for this time we only use one type of energy to form the feature vector. This means that the feature vector only formed from norm1, norm2, standard deviation, average residual and entropy (Table 3). The experimental results show that the best performance was achieved with average residual energy calculation. In this experiment quadratic classifier has been employed and overall error rate is 1.19%. This suggests that average residual is more effective than other energy calculation method. From the experimental result, it shows that the number of element of feature vector, and the number of energy used are affecting the accuracy of classification process. Using the data of the experiment, optimization of the classification system could be done by reducing the element of the feature vector. In Figure 5.1, it is indicated that the optimum number element is 20. The visual representation shows that the error rate starts to convergence when the number of feature vector element is 20. This means that if we add more elements in, more energy will be used. This does not show any significant improvement to Figure 5.1. The experiment has resulted in the quadratic classifier being the most accurate method. Error rate for quadratic classifier is 0.18%. This result shows that feature extraction using wavelet is suitable for texture classification. Based on the second experiment it shows that the most effective and efficient energy calculation algorithm is using the average residual, (Figure 5.2). Overall error rate for 13 number of energy, i.e. element of the feature vector, is 1.19%. Overall accuracy is about 98%. From this result we can suggest that wavelet feature extraction should be suitable for haptic texture searching that deals not only with visual representation, but also the touch sensation.
Fig 5.1
Fig 5.2
Fig 5. Classification Accuracy experiment by using different type of energy. In this particular experiment, quadratic classifier has been employed. In this experiment, average residual gives most effective feature vector element for classification. 1). error propagation for quadratic, linear and mahalanobis classifier. 2). Propagation error for quadratic classifier for each type of energy.
6 Conclusion and Future Work In this paper, we have addressed the problem in haptic searching method and proposed a framework that integrates both visual and haptic in a texture search engine. We claimed that techniques in computer vision could be replicated for computer haptic. We predicted that wavelet decomposition could be the best candidate. In justifying our claim, we reviewed the standard 2D wavelet decomposition. Then we defined an algorithm to extract the texture. The feature extraction involves wavelet transform and five different energy measurements. The construction of feature vector is done using different techniques. Firstly, we used the entire energy from entire sub-band; secondly, we used only one type of energy for entire sub-band. The several series of experiments to find out the suitable method for extraction and searching process show that quadratic classifier outperforms two other classification methods. Also, based on the experiment, average residual is the most effective and efficient energy measurements method, beside norm 2. The overall error rate for average residual energy calculation in quadratic classification was 1.19%. Since haptic rendering data and 2D images have similarity, based on this experiment, we can conclude that 2D wavelet decomposition and wavelet energy signature could be employed in haptic data extraction. With this experiment results we could suggest that this method could be suitable for our proposed haptic texture searching. Future work will include implementation of this method in our haptic texture search engine.
References 1. Datta, R.: Image Retrieval: Ideas, Influences, and Trends of New Age, ACM Computing surveys, vol. 40, No. 2, Article 5, (2008) 2. Conners, R., Wand C.A., Harlow, A.: Theoretical Comparison of Texture Algorithms, IEEE Trans. Pattern Analysis and Machine Intelligence. 2, 204-222, (1980) 3. Azencott R.: Texture Classification Using Windowed Fourier Filters, IEEE Trans. Pattern Analysis and Machine Intelligence. 19, 148--153, (1997) 4. Fountain S. R.: Rotation Invariant Texture Features from Gabor Filters, Computer Vision ACCV'98, vol. 1352 (1997) 5. Nikam, S.B., Agarwal, S.: Wavelet Energy Signature and GLCM Features-based Fingerprint Anti-spoofing. In Conference on Wavelet Analysis and Pattern Recognition. ICWAPR '08. International, (2008) 6. Pun, C.: Log-Polar Wavelet Energy Signatures for Rotation and Scale Invariant Texture Classification IEEE Transactions on Pattern Analysis and Machine Intelligence. 25(5), (2003) 7. Shi, M.: Hyperspectral Texture Recognition Using a Multiscale Opponent Representation. IEEE Transaction on Geoscience and Remote Sensing. 41(5), (2003) 8. Yektaii, M.: Cumulative Global Distance for Dimension Reduction in Hand written Digits Database. Lecture Note in Computer Science, vol. 4781, pp. 216--222. (2007) 9. Mallat S.: A Theory for Multiresolution Signal Decomposition: The Wavelet Representation, IEEE Trans on Pattern Analysis and Machine Intelligence. 11(7), 674--693, (1989) 10.Daubechies, I.: Orthonormal Bases of Compactly Supported Wavelets. Comm. Pure and Applied Math. 41, 909--996, (1988) 11.Cédric, V.: Generalized Daubechies Wavelet Families, IEEE Transactions on Signal Processing. 55(9), (2007) 12.Farrell, M.D.: On The Impact of PCA Dimension Reduction for Hyperspectral Detection of Difficult Targets, IEEE Transaction on Geoscience and Remote Sensing Letters. l.2(2). (2005) 13.Hadsell, R., Chopra, S., LeCun, Y.: Dimensionality Reduction by Learning an Invariant Mapping. In IEEE Conference in Computer Vision and Pattern Recognition, pp. 1735-1742. IEEE Computer Society Press, Los Alamitos (2006) 14. Wall, S.: An Investigation of Temporal and Spatial Limitation of Haptic Interfaces. PhD Thesis, Department of Cybernetics, University of Reading (2004) 15.Klatzky, R.L. and Lederman, S.J. Touch, In A.F. Healy & R.W. Proctor (Eds.), Experimental Psychology, vol. 4. In I.B. Weiner (Editor-in-Chief), Handbook of Psychology, New York: Wiley. (2004) 16.Seungmoon, C., Tan, H.Z.: An Analysis of Perceptual Instability During Haptic Texture Rendering. In 10th Symposium on Haptic Interfaces for Virtual Environment and Teleoperator Systems, (2002) 17.Wu, J., Song, A., Zou, C.: A Novel Haptic Texture Display Based on Image Processing. In: IEEE International Conference on Robotics and Biomimetics, pp. 1315--1320. (2007)
Waskito Adi1, Suziah Sulaiman1 1
Computer and Information Sciences Department, Universiti Teknologi PETRONAS Bandar Seri Iskandar, 31750 Tronoh, Perak, Malaysia {[email protected], [email protected]}
Abstract. While visual texture classification is a widely-researched topic in image analysis, little is known on its counterpart i.e. the haptic (touch) texture. This paper examines the visual texture classification in order to investigate how well it could be used for haptic texture search engine. In classifying the visual textures, feature extraction for a given image involving wavelet decomposition is used to obtain the transformation coefficients. Feature vectors are formed using energy signature from each wavelet sub-band coefficient. We conducted an experiment to investigate the extent in which wavelet decomposition could be used in haptic texture search engine. The experimental result, based on different testing data, shows that feature extraction using wavelet decomposition achieve accuracy rate more than 96%. This demonstrates that wavelet decomposition and energy signature is effective in extracting information from a visual texture. Based on this finding, we discuss on the suitability of wavelet decomposition for haptic texture searching, in terms of extracting information from image and haptic information. Keywords: Texture recognition, supervised learning, machine learning, haptic texture search engine, wavelet decomposition.
1 Introduction Lately, computer vision has become one of the most popular research subjects. Computer vision contributes in various fields that include medical, engineering, and robotics. One of the most interesting topics in computer vision is texture recognition. Being the smallest entity, textures could be used as a parameter to recognize a particular object. There are many different types of applications involving texture analysis, including medical imaging, industrial inspection, remote sensing, document segmentation, and computer based image retrieval [1]. Feature extraction plays an important role in a classification process. The effectiveness of such classification relies greatly on the choice of this feature. In this case, a suitable extraction algorithm influences the process end result. Parallel to computer vision, computer haptic is also another area which is gaining its popularity among researchers. Computer haptic enable user to touch and interact with the virtual
objects. Similar to computer vision, haptic texture is a growing research topics and its application has been noted in medical field, arts, and textile, to name a few. Even though computer vision and computer haptic are among the favorite topics of late, no attempt has been made to integrate both texture classifications into the same system such as a search engine. Current practices have seen visual and haptic textures been dealt with separately; and the trend is that visual aspect has been the dominant topics of the two texture types. The absence of such integration may hinder one from getting a complete set of surface information of a particular object. This is because visual textures could inform us on the macro-geometric information while haptic textures on the micro-geometric details [15]. The lacking in combining both visual and haptic textures needs addressing. In this paper, we attempt to address the issue of lacking the effort in integrating visual and haptic textures. We propose a way to analyze and classify haptic textures based on an approach used from computer vision. The focus of our work here is on visual textures and examines how such result could inform those researchers working in haptic texture classifications. This involves conducting an experiment to test the visual texture classification and feature extraction performance. In the classification process, supervised learning has been used as a testing method. The outline of this paper is organized as follows: in section 2, we review some related works pertaining to visual and haptic textures analysis. This is followed by section 3 in which we describe the method involved in our experiment that include wavelet transformation, and statistical classification. In section 4, we present our scheme in extracting texture information, and also how the experiments were conducted. Finally, in section 5 we analyze the experimental results and discuss the findings.
2 Related Works
2.1 On Visual Textures Visual texture analysis has been studied over the past three decades. Most of the early research focused on the analysis of the first or second order statistics of textures and stochastic model such as Gaussian Markov random fields and auto regression [2]. Recently, the development of texture classification and analysis has been using frequency and spatial analysis, such as Fourier transform, Gabor filter, and Wavelet transform. Since a texture image is in 2D representation, measuring its characteristics could be done by employing a 2D transformation or filtering. For example, to extract texture information, a 2D Fourier transform can be employed. Azencott et al [3] define a distance between textures for texture classification from texture features based on windowed Fourier filters. In the research, Fourier transformation is used as a method to extract information from textures. In employing a more advanced technique, Fountain [4] has conducted an experiment using Gabor filter for texture extraction. Also, in [7], Miaohong conducted an experiment using Gabor filter to extract features at different orientation
from hyperspectral images. In that particular research, it is said that accuracy of the classification is over 90%. Using a different technique, Shankar conducted an experiment [5] by using wavelet transform to extract information from finger print. Similarly, Chi-Man Pun [6] has also conducted an experiment using wavelet to extract information from textures. It is reported in his research that wavelet gives a better result than Gabor filter. In this particular case, although Gabor filter is a multiresolution filter, wavelet transform still outperforms. 2.2 On Haptic Textures Texture is the smallest independent quantity in an object. Research works that involve texture are mostly conducted in image and signal processing. When including texture in haptics field, image and signal processing algorithm will be involved. To create a realistic object the simplest way is by employing sine wave to generate the texture surfaces. In [16] Seungmoon Choi and Hong Z. Tan employed sine wave to render the texture surface in order to analyze the stability during haptic texture rendering. In order to simulate the surface of haptic texture, there are some algorithms that can be used [16][17]. The simplest one is by using sine wave. By adjusting amplitude and frequency of the sine wave the texture of surfaces could be obtained. In [14] Wall developed a haptic texture rendering technique using Fourier series. The basic idea of this rendering technique is similar with rendering using sine wave because Fourier series using sine wave as basis to create new wave. In order to create realistic texture for haptic rendering, Juan Wu introduced a new rendering method based on image processing [17].
3 Methodology
3.1 Equating Haptic Texture to Visual Texture To obtain a texture sensation, a simple technique such as sinusoidal waves could be used to render the force feedback. Different texture pattern or profile is obtained by adjusting frequency, and amplitude of the sine waves which could result in different haptic sensation. For example, Wall [14] conducted an experiment that simulates texture surface by employing Fourier series to achieve different types of haptic texture surface. The texture profile used in haptic rendering and its 2D image counterpart share a similar characteristic. When rendering a property of an object, such as its haptic texture using sinusoidal waves, the (haptic) surface will have surface properties similar to (visual) sine waves, but in 3D plane. However, when these haptic properties are plotted onto a 2D plane they will appear similar to 2D images. Fig. 1 shows a 2D representation of sine and square haptic waves (Fig 1.1 & 1.2) and the actual visual images (Fig 1.3 & 1.4).
Fig 1.1. sine wave (haptic) texture
Fig 1.2. square wave (haptic) texture
Fig 1.3 cloth texture (visual)
Fig 1.4 cloth texture (visual)
Fig. 1. Image visualization from haptic texture
Comparing the four images in Fig. 1, it could be assumed that haptic texture images generated using both sine square waves have similar patterns with those non artificial visual textures (Fig. 1.3 & 1.4). Based on the similarity of their appearances on a 2D plane, we could claim that computer vision (CV) algorithm could also be employed to extract information from haptic rendering data. 3.2 Determining a Method to Represent Haptic Texture Transformation is one of the most powerful tools to analyze signal information. With transformation, signal can be analyze and manipulated. The most common method to analyze signal is by using Fourier transform. Fourier transforms use sine wave as basis of the transformation, by using Fourier transform assumption, it can be inferred that every signal is constructed by sine waves superposition. Fourier transform is denoted as: ∞ (1)
F ( w) =
∫ f ( x )e
− 2πiωx
∂x
−∞
The inverse of Fourier transform, which allows going from spectrum to the signal is: ∞ (2)
f ( x) = ∫ F ( x)e 2πiωx ∂ω −∞
Consider a square wave, (Fig. 2), f (x) of length 2 L .The range [0,2 L] , could be written:
x x f ( x ) = 2 H − H − 1 − 1 (3) L L By using Fourier transform, information of square signal can be retrieved. In this case, square signal can be decomposed into different sine wave frequency and amplitude. Referring to the Fourier transform, if a square wave equation is transformed, it will produce a Fourier series as follows: ∞ (4) 4 1 nπx
f ( x) =
π
∫
n n =1, 3, 5 ,...
sin L
Unlike Fourier series that uses sine wave, wavelet is a transformation that uses transient signal as its basis. In other word wavelet transform is better than Fourier for analyzing transient signal. Wavelet transform of f (x) is denoted as: (5) W f (a, b) = ψ ( x ) f ( x)∂x ψ
∫
a ,b
R
Consider a signal shown in (Fig. 2), in wavelet analysis.
Fig. 2. Decomposition of square wave using Fourier Transform
The signal can be decomposed into different small basis transient signal. Otherwise if Fourier transform is employed, the signal will be decomposed into different amplitude and frequency of sine wave.
Fig 3. Illustration of 1-D wavelet transforms. Unlike Fourier transform that used sine wave as basis, wavelet transform use small transient signal to transform or decompose the source signal.
Wavelet multiresolution analysis has been used as texture characterization. It provides spatial or frequency information of an image. There are two method extractions in wavelet multiresolution: (i) wavelet decomposition (ii) wavelet packet decomposition.
In our experiment we use 2D wavelet decomposition to extract texture images. The decomposition offer results for image analysis. Although it is not as rich as 2D wavelet packet (2D-WPT), it is adequate to do analysis to an image with 2D wavelet decomposition. In 2D wavelet decomposition transform (2D-WDT), an image x is split into an approximation and three detail image (Fig. 3). 2D-WDT can be described as a pair of filter L and H [9], H is low pass filter, and L is high pass filter. The 2DWDT of a discrete N x M image can be recursively defined as: (6) A = C p +1 = ∑∑ h(m)h(n)C p 1
k ,( m+ 2 i , n + 2 j )
4 k ,( i , j )
m
n
H1 = C 4pk++11,(i , j ) = ∑∑ h(m)h(n)C kp,( m+2 i ,n +2 j ) m
(7)
n
V1 = C 4pk++12,(i , j ) = ∑∑ h(m)h(n)C kp,( m+ 2i ,n + 2 j ) m
n
m
n
(8)
D1 = C4pk++13,(i , j ) = ∑∑ h(m)h(n)Ckp,( m+2i ,n + 2 j )
(9)
A1 is approximation coefficient, H1 is horizontal detail, V1 is vertical detail, and D1 is diagonal detail, illustrated in (Fig. 4). In practice, the image X only has finite number of pixels, for boundary handling different technique such as symmetric, periodic could be used. The difference between 2D-WDT and 2D-WPT is in the extraction process. In 2D-WDT only approximation coefficient will be transformed whereas in 2D-WPT all of the coefficient will be recursively transformed. In the proposed method, an image is decomposed into 4-levels. 4-level decomposition results in 3n+1 sub-band, it means that decomposition results in 13 sub-bands. The general transformation processes compress information from data. In wavelet transform, if the basis signal for transformation has correlation with transformed signal it will produce homogenous transformed coefficient. Referring to those characteristics, characteristics of each sub-band can be used as information to form feature vector. In order to form a feature vector, f=(x1, x2, x3,. . ., xn), the energy or characteristic each sub-band of images have been calculated using formula (Table 1). Table 1. Energy Calculation Formula Energy Type Norm 2 Norm 1
Formula
1 NM 1 e2 = NM e1 =
Standard Deviation
e3 = Average Residual
N
2 M ,N
m =1 n =1 M N
∑∑ C
M ,N
m =1 n =1
1 NM
M
N
∑∑ (C
M ,N
− µ)2
m =1 n =1
e4 = ∑∑ C M , N − µ M
Entropy
M
∑∑ C
e5 = −
N
1 NM
∑∑ C M
N
2 M ,N
log C M , N
2
It is a common issue that data stored in feature vector have redundancy. To reduce redundancy form the feature vector, PCA (Principle Component analysis) algorithm should be employed [7][12][1]. With PCA algorithm, classification process could be more effective and efficient 3.3 Testing the Classification Performance In our research, we will employ the supervised learning to classify the texture. In our study, in order to test the classification performance, we employ linear, quadratic and mahalanobis classifiers. Bayesian rule has been used as a fundamental concept for the entire classifiers. Bayesian rule is denoted as:
P(ω j | x) =
p( x | ω j ) P(ω j )
(10)
p( x )
Let say there are two classes of data ω1 and ω2, P(ωj) is prior probability and p(x) is probability density function (PDF). Referring to the maximum likelihood theorem, by maximizing the conditional probabilities P(ωj|x) decision of the class could be obtained. In this case if P(ω1|x)>P(ω2|x) the data will belong to class ω1, otherwise if ω2 data belong to ω2. Referring to Bayesian rule, if it assumes PDF is a normal distribution or Gaussian, linear, quadratic and mahalamobis classifier could be obtained. Quadratic, linear and mahalanobis could be denoted as: Quadratic classifier: 1 max | − ( x − µ j ) T ∑ −j1 ( x − µ j ) + ln P(ϖ j ) − ln | ∑ j | 2
(11)
Linear Classifier: 1 max | − ( x − µ j ) T ∑ −j1 ( x − µ j ) + ln P(ϖ j ) 2
(12)
Mahalanobis Classifier: max | −( x − µ j ) T ∑ −j 1 ( x − µ j )
(13)
To obtain the data classes (refer to equation 11, 12, 13), there is a need to maximize the conditional probability. In equations 11 and 12, all data are calculated based on the relative distance between the data and data distribution, and also using prior probability of the class as weight parameter. In mahalanobis classifier algorithm, (equation 13), it calculates relative distance between the data and the data distribution.
4 Proposed Framework and Experiment To this end, we have claimed that computer haptic texture classification could replicate similar technique from computer vision. In the previous section, we have
described how wavelet and its feature extraction could be a potential candidate to address this situation. Fig. 4 presents a proposed framework for a system that can adopt both visual and haptic search. The framework combines both visual and haptic texture extractions and classifications.
x1 x2 f = x3 ... x n
Fig. 4. Proposed Framework involving the extraction and classification processes
The visual information as shown in Fig. 4 consists of the texture image data; this information is passed to the extraction process. The process involves the wavelet decomposition (using db-8) to get the characteristics from each image. At this stage the energy calculation formula will be used. The feature vector f(x) is formed as a result of this process. In the classification stage, we employ the linear, quadratic and mahalanobis classifiers to examine the most suitable classifier in terms of its performance. Using the chosen classifier, the system will suggest some possible matching results. 4.1 Experiment The effectiveness of wavelet decomposition for texture analysis and classification has been tested in an experiment, using 76 samples of texture images. The texture set consists of natural, artificial, and semi natural textures. The objectives of the experiments are: • To determine the texture classification performance for each classifier • To identify the most effective and efficient variable to be used in a classification process. During the performance test, we use three different types of classifiers. We employ linear, quadratic, and mahalanobis classifiers. Daubechies wavelet family has been employed as an algorithm to extract the textures. Table 2 shows 16 tap of daubechies wavelet transform filter coefficient.
Table 2. 16 Tap of Daubechies-8 Wavelet Transform Filter Coefficient
h(0) h(1) h(2) h(3) h(4) h(5) h(6) h(7)
0.0385 0.2212 0.4777 0.4139 -0.0112 -0.2008 0.0003 0.0910
h(8) h(9) h(10) h(11) h(12) h(13) h(14) h(15)
-0.0123 -0.0312 0.0099 0.0062 -0.0034 -0.0003 0.0005 -0.0001
The textures have been decomposed into 4-level. It means that decomposition processes results in 13 coefficient sub-band. To conduct the experiment we prepare a learning sample and testing data sets. We create a random 512x512 pixel image from the texture image. For each 512x512 pixel image we partition this image into 4 265x265 without overlapping each other. We generate 100 random images 128x128 from each 256x256 images. In other words, for each sample we have 400 samples. We use 200 samples for learning process and the other 200 samples for testing. To test the performance of the classification, we conduct three experiments with different feature vectors and different classification methods. We employed the linear, quadratic, and mahalanobis classifiers in the experiments, consecutively. For the first experiment, we use the entire feature of vector element, meaning that all the extracted information was used. The second experiment was to test the effect of the element number with classification process. We start by using 1 feature vector element up to the entire elements. In the third experiment, we use only one type of energy calculation algorithm.
5 Findings and Analysis As described earlier, in the experiment we have used the three classifiers to measure the accuracy of classification performance. In each classification process we measure the error rate of each method. The classification testing involves: 1) Measuring the number of feature vector element versus error rate 2) Measuring the effect using different type of feature vector 3) Comparing the classification method 4) Finding the optimum number of feature vector element The first experiment was conducted by measuring the error rate for each feature vector that have different element. Since we have Level-4 wavelet decomposition, it means that we have 13 different sub-bands. For each sub-band we calculate the energy using the formula in (Table 1), which means we have 65 different values to form a feature vector. In this experiment we form a feature vector started using 1 element to 65 elements, f(x1) to f(x1, x1, x1,. . ., x65). In this test, quadratic classifier gives the most accurate results when compared with linear and mahalanobis classifiers. The classification results are summarized in Figure 5 and Table 3.
Table 3. 16 Tap of Daubechies-8 Wavelet Transform Filter Coefficient Energy 1 2 3 4 5 6 7 8 9 10 11 12 13
Norm1 0.8661 0.6899 0.4606 0.3198 0.3068 0.1672 0.1072 0.0917 0.0527 0.0348 0.0210 0.0126 0.0068
Norm2 0.8656 0.8379 0.8200 0.7755 0.7460 0.7065 0.6691 0.6361 0.6205 0.6057 0.5903 0.5580 0.5432
Std dev 0.8584 0.7063 0.5540 0.5337 0.3248 0.1607 0.1057 0.0744 0.0610 0.0562 0.0327 0.0199 0.1066
Ave. residual 0.9212 0.5719 0.3503 0.2409 0.2226 0.1461 0.1022 0.0605 0.0311 0.0228 0.0179 0.0140 0.0119
Entropy 0.8666 0.6389 0.5241 0.4041 0.3936 0.2459 0.1727 0.1594 0.1113 0.0840 0.0683 0.0524 0.0411
We carried out a second experiment similar to the first but for this time we only use one type of energy to form the feature vector. This means that the feature vector only formed from norm1, norm2, standard deviation, average residual and entropy (Table 3). The experimental results show that the best performance was achieved with average residual energy calculation. In this experiment quadratic classifier has been employed and overall error rate is 1.19%. This suggests that average residual is more effective than other energy calculation method. From the experimental result, it shows that the number of element of feature vector, and the number of energy used are affecting the accuracy of classification process. Using the data of the experiment, optimization of the classification system could be done by reducing the element of the feature vector. In Figure 5.1, it is indicated that the optimum number element is 20. The visual representation shows that the error rate starts to convergence when the number of feature vector element is 20. This means that if we add more elements in, more energy will be used. This does not show any significant improvement to Figure 5.1. The experiment has resulted in the quadratic classifier being the most accurate method. Error rate for quadratic classifier is 0.18%. This result shows that feature extraction using wavelet is suitable for texture classification. Based on the second experiment it shows that the most effective and efficient energy calculation algorithm is using the average residual, (Figure 5.2). Overall error rate for 13 number of energy, i.e. element of the feature vector, is 1.19%. Overall accuracy is about 98%. From this result we can suggest that wavelet feature extraction should be suitable for haptic texture searching that deals not only with visual representation, but also the touch sensation.
Fig 5.1
Fig 5.2
Fig 5. Classification Accuracy experiment by using different type of energy. In this particular experiment, quadratic classifier has been employed. In this experiment, average residual gives most effective feature vector element for classification. 1). error propagation for quadratic, linear and mahalanobis classifier. 2). Propagation error for quadratic classifier for each type of energy.
6 Conclusion and Future Work In this paper, we have addressed the problem in haptic searching method and proposed a framework that integrates both visual and haptic in a texture search engine. We claimed that techniques in computer vision could be replicated for computer haptic. We predicted that wavelet decomposition could be the best candidate. In justifying our claim, we reviewed the standard 2D wavelet decomposition. Then we defined an algorithm to extract the texture. The feature extraction involves wavelet transform and five different energy measurements. The construction of feature vector is done using different techniques. Firstly, we used the entire energy from entire sub-band; secondly, we used only one type of energy for entire sub-band. The several series of experiments to find out the suitable method for extraction and searching process show that quadratic classifier outperforms two other classification methods. Also, based on the experiment, average residual is the most effective and efficient energy measurements method, beside norm 2. The overall error rate for average residual energy calculation in quadratic classification was 1.19%. Since haptic rendering data and 2D images have similarity, based on this experiment, we can conclude that 2D wavelet decomposition and wavelet energy signature could be employed in haptic data extraction. With this experiment results we could suggest that this method could be suitable for our proposed haptic texture searching. Future work will include implementation of this method in our haptic texture search engine.
References 1. Datta, R.: Image Retrieval: Ideas, Influences, and Trends of New Age, ACM Computing surveys, vol. 40, No. 2, Article 5, (2008) 2. Conners, R., Wand C.A., Harlow, A.: Theoretical Comparison of Texture Algorithms, IEEE Trans. Pattern Analysis and Machine Intelligence. 2, 204-222, (1980) 3. Azencott R.: Texture Classification Using Windowed Fourier Filters, IEEE Trans. Pattern Analysis and Machine Intelligence. 19, 148--153, (1997) 4. Fountain S. R.: Rotation Invariant Texture Features from Gabor Filters, Computer Vision ACCV'98, vol. 1352 (1997) 5. Nikam, S.B., Agarwal, S.: Wavelet Energy Signature and GLCM Features-based Fingerprint Anti-spoofing. In Conference on Wavelet Analysis and Pattern Recognition. ICWAPR '08. International, (2008) 6. Pun, C.: Log-Polar Wavelet Energy Signatures for Rotation and Scale Invariant Texture Classification IEEE Transactions on Pattern Analysis and Machine Intelligence. 25(5), (2003) 7. Shi, M.: Hyperspectral Texture Recognition Using a Multiscale Opponent Representation. IEEE Transaction on Geoscience and Remote Sensing. 41(5), (2003) 8. Yektaii, M.: Cumulative Global Distance for Dimension Reduction in Hand written Digits Database. Lecture Note in Computer Science, vol. 4781, pp. 216--222. (2007) 9. Mallat S.: A Theory for Multiresolution Signal Decomposition: The Wavelet Representation, IEEE Trans on Pattern Analysis and Machine Intelligence. 11(7), 674--693, (1989) 10.Daubechies, I.: Orthonormal Bases of Compactly Supported Wavelets. Comm. Pure and Applied Math. 41, 909--996, (1988) 11.Cédric, V.: Generalized Daubechies Wavelet Families, IEEE Transactions on Signal Processing. 55(9), (2007) 12.Farrell, M.D.: On The Impact of PCA Dimension Reduction for Hyperspectral Detection of Difficult Targets, IEEE Transaction on Geoscience and Remote Sensing Letters. l.2(2). (2005) 13.Hadsell, R., Chopra, S., LeCun, Y.: Dimensionality Reduction by Learning an Invariant Mapping. In IEEE Conference in Computer Vision and Pattern Recognition, pp. 1735-1742. IEEE Computer Society Press, Los Alamitos (2006) 14. Wall, S.: An Investigation of Temporal and Spatial Limitation of Haptic Interfaces. PhD Thesis, Department of Cybernetics, University of Reading (2004) 15.Klatzky, R.L. and Lederman, S.J. Touch, In A.F. Healy & R.W. Proctor (Eds.), Experimental Psychology, vol. 4. In I.B. Weiner (Editor-in-Chief), Handbook of Psychology, New York: Wiley. (2004) 16.Seungmoon, C., Tan, H.Z.: An Analysis of Perceptual Instability During Haptic Texture Rendering. In 10th Symposium on Haptic Interfaces for Virtual Environment and Teleoperator Systems, (2002) 17.Wu, J., Song, A., Zou, C.: A Novel Haptic Texture Display Based on Image Processing. In: IEEE International Conference on Robotics and Biomimetics, pp. 1315--1320. (2007)
