Thermal condition monitoring system using log-polar mapping ...

This article appeared in a journal published by Elsevier. The attached copy is furnished to the author for internal non-commercial research and education use, including for instruction at the authors institution and sharing with colleagues. Other uses, including reproduction and distribution, or selling or licensing copies, or posting to personal, institutional or third party websites are prohibited. In most cases authors are permitted to post their version of the article (e.g. in Word or Tex form) to their personal website or institutional repository. Authors requiring further information regarding Elsevier’s archiving and manuscript policies are encouraged to visit: http://www.elsevier.com/copyright

Author's personal copy Neurocomputing 74 (2010) 164–177

Contents lists available at ScienceDirect

Neurocomputing journal homepage: www.elsevier.com/locate/neucom

Thermal condition monitoring system using log-polar mapping, quaternion correlation and max-product fuzzy neural network classification Wai-Kit Wong n, Chu-Kiong Loo, Way-Soong Lim, Poi-Ngee Tan Faculty of Engineering and Technology, Multimedia University, 75450 Jln Ayer Keroh Lama, Malaysia

a r t i c l e in f o

a b s t r a c t

Article history: Received 28 October 2008 Received in revised form 20 January 2010 Accepted 12 February 2010 Communicated by A. Zobaa Available online 1 August 2010

Nowadays, most factories rely on machines to help boost up their production and process. Therefore, an effective machine condition monitoring system plays an important role in these factories to ensure that their production and process are running smoothly all the time. In this paper, a new and effective machine condition monitoring system using log-polar mapper, quaternion based thermal image correlator and max-product fuzzy neural network classifier is proposed. Two classification characteristics, namely peak to sidelobe ratio (PSR) and real to complex ratio of the discrete quaternion correlation output (p-value) are applied in this proposed machine condition monitoring system. Large PSR and p-value showed a good match among correlation of the input thermal image with a particular reference image, but reversely for small PSR and p-value match. In the simulation, log-polar mapping is found to have solved the rotation and scaling invariant problems in quaternion based thermal image correlation. Besides, log-polar mapping can possess two fold data compression capability. Log-polar mapping helps smoothen up the output correlation plane, hence making better measurement for PSR and p-values. The simulation results have also proven that the proposed system is an efficient machine condition monitoring system with an accuracy of more than 94%. & 2010 Elsevier B.V. All rights reserved.

Keywords: Thermal condition monitoring system Log-polar mapping Quaternion correlation Max-product fuzzy neural network Thermal imaging

the quaternion numbers, respectively [3]:

1. Introduction The recently developed concept of quaternion correlation, based on quaternion algebra introduced by Hamilton in 1843 [1], is found useful in color pattern recognition (e.g. human face recognition [2], color alphanumeric words recognition [3]). A quaternion array is generalized from the complex number representation and it can be considered as a number with a real and imaginary term consisting of three orthogonal components as follows [4]: f ¼ fr þ fi i þ fj j þfk k

ð1Þ

where fr, fi, fj, fk are real quaternion numbers and i, j, k, are imaginary operators. In conventional correlation for pattern recognition, patterns have to be converted to gray level scale or by processing the three color channels (R, G, B) separately before combining them again. But for the quaternion correlation technique, all color channels are processed together using the quaternion array. In quaternion array, the R, G, B in color image can be represented by inserting the value of three color channels into the three imaginary terms of n

Corresponding author. Tel.: +60 6 2523005; fax: + 60 6 2316552. E-mail addresses: [email protected] (W.-K. Wong), [email protected] (C.-K. Loo), [email protected] (W.-S. Lim), [email protected] (P.-N. Tan). 0925-2312/$ - see front matter & 2010 Elsevier B.V. All rights reserved. doi:10.1016/j.neucom.2010.02.027

f ðm,nÞ ¼ fR ðm,nÞiþ fG ðm,nÞj þ fB ðm,nÞk

ð2Þ

where fR(m, n), fG(m, n) and fB(m, n) represent the R, G and B patterns, respectively, and m, n are the 2-D pixels coordinates. Quaternion correlator has so far been used in color human face recognition [2] and color alphanumeric words recognition [3]. It is not found in object recognition yet, especially in thermal object/ image recognition. In this paper, a new approach is presented, whereby quaternion correlator is used in thermal image recognition for machine condition monitoring system. Thermal monitoring is useful for revealing some serious electrical problems in a factory that often go undetected until a serious breakdown occurs. In factories, there are various types of operating machines to be monitored. When there is any malfunctioning of machine, extra heat will be generated which can be picked up by a thermal camera. The algorithm proposed in this paper has made detection and monitoring of conditions (overheat) of multiple machines in a single view simple and efficient. The monitoring and analyses of the images will then alert maintenance personnel to take corrective action and/or repair the overheat/faulty machine. An image mapping method for solving problem of scaling and rotational invariant due to the change of position in either thermal camera or machine during monitoring process is also proposed. This method is called logpolar mapping [5].

Author's personal copy W.-K. Wong et al. / Neurocomputing 74 (2010) 164–177

In quaternion correlator, a strong and sharp peak can be observed in the output correlation plane when the input thermal image comes from the authentic class (input thermal image matches with a particular training/reference image stored in the database), and there will be no discernible peak if the input thermal image comes from imposter class (input thermal image does not match with the particular reference image). For better recognition, peak-to-sidelobe ratio (PSR) [6] is introduced to test whether an input thermal image belongs to the authentic class or not. It is because by considering the peak value with the region around the peak value is more accurate than just a single peak point. The higher the value of PSR, the more likely the input thermal image belonging to the reference image class. Another parameter, the real to complex ratio of the discrete quaternion correlation output (p-value [3]), is also used in measuring the quaternion correlation output between the colors, shape, size and brightness of the input thermal image and a particular reference thermal image. Apart from the quaternion correlator, a max-product fuzzy neural network classifier to perform classification on the thermal images based on the PSR and p-value output from quaternion correlator is also proposed. Classification in pattern recognition [7] refers to a procedure whereby individual patterns are placed into groups based on quantitative information on one or more characteristics inherent in the patterns and based on a training set of previously labeled patterns, known as classes. The aim of classification is to establish a rule whereby a new observed pattern will map into one of the existing classes. These classes are predefined by a partition of the sample space, which are the attributes themselves. For example, a machine may be classified as overheated if the color display in the thermal image is brighter than the predetermined limits. In this paper, the max-product fuzzy neural network classifier [8] is modified to perform the classification in machine condition monitoring system. In the max-product fuzzy neural network classifier, both the PSR and p-value output from the quaternion correlator are first fuzzified with Gaussian membership function. The max-product fuzzy neural network classifier is applied for accurate classification with the weights obtained from training reference images. The weights are then applied for classification of input images in real time application. Simulation results show that in the authentic case, if an input image is well matched with a particular reference image in the database, followed by performing quaternion correlation on these two images, their output correlation plane will have sharp peaks. However in imposter case, if an input thermal image is not matched with a particular reference image in the database, the output correlation plane is flat. Large peak to sidelobe ratio (PSR) and real to complex ratio of the discrete quaternion correlation output (p-value) are proven to have a good match among correlation of the input thermal image with a particular reference image, while small PSR and p-value reflect reversely. In the simulation, log-polar mapping is discovered to solve the rotation and scaling invariant problems in quaternion based thermal image correlation. Besides that, logpolar mapping can have two-fold data compression capability. Log-polar mapping helps smoothen up the output correlation plane, hence improving measurement for PSR and p-values. Simulation results also show that the proposed system is an efficient machine condition monitoring system with accuracy above 94%. This paper is organized in the following order: Section 2 briefly comments on the machines monitoring system and Section 3 summarizes the log-polar image geometry and the mapping techniques. The algorithm of the proposed quaternion based thermal image correlator is described in Section 4. Section 5 describes the structure of the max-product fuzzy neural network

165

classifier. In Section 6, the experimental results are discussed. Finally, Section 7 summarizes the work and some suggestions are proposed for future work.

2. Machine condition monitoring system model The machine condition monitoring system developed in this paper is shown in Fig. 1. The thermal camera used in this paper is cost effective and has acceptable resolution model: AXT100 manufactured by Ann Arbor Sensor Systems [9]. The thermal camera costs about US$5,000 and can capture thermal images with resolution up to 256  248 display resolution pixels. Besides, AXT100 also has some advanced signal processing features such as linear or logarithmic scaling, false color and atmospheric correction. The digital control of AXT100 is also accomplished through the 10/100 Ethernet port connected to a laptop or PC located in a monitoring room via an embedded firmware called InternalWeb, which can be interfaced with Matlab or other image processing software. Therefore, it is best fitted for machine condition monitoring system. Image partitioner is used for partitioning the input thermal image into S-partitioned sections, where S is the number of machines to be monitored. Each partitioned section consists of one machine to be monitored. An example of a thermal image with S ¼3 partition sections is shown in Fig. 2. Applying the log polar mapping technique [5] converts a Cartesian image into a retina-like (log-polar) image giving an important and useful property that scaling and rotating an object in a Cartesian plane corresponds to translate the object in the logpolar plane. Another advantage of log-polar image representation is that it possesses data compression characteristics. Detailed discussion on log-polar mapping will be provided in Section 3. Quaternion based thermal image correlator is used to obtain correlation plane for each correlated input thermal image captured live with reference images of all possible machine conditions storing in a database, the data is used to calculate some of the classification characteristics such as the real to complex ratio of the discrete quaternion correlation (DQCR) output, pvalue and the peak-to-sidelobe ratio, PSR. These classification characteristics will later be fed into the max-product fuzzy neural network classifier. Detailed descriptions on quaternion based thermal image correlator will be discussed in Section 4. The max-product fuzzy neural network classifier is first applied to train for accurate classification with the weight (w) obtained from training reference images of all possible machine conditions stored in the database. During application, the PSR and p-value output from quaternion based thermal image correlator

Fig. 1. Machine condition monitoring system model.

Fig. 2. Example of a thermal image with S¼ 3 partition sections.

Author's personal copy 166

W.-K. Wong et al. / Neurocomputing 74 (2010) 164–177

are first fuzzified into Gaussian membership function. Then, the product value is calculated based on multiplication of PSR and pvalue in Gaussian membership value. The product values are stored in an array and multiplied with the weight (w). Maxcomposition is performed on the output based on two sets of fuzzy IF-THEN rules, whereas defuzzification is performed to classify each machine’s condition under monitoring. Detailed discussion on max-product fuzzy neural network classifier is given in Section 5.

3. Log-polar mapping Log-polar imaging geometry is a biological inspiration approach to human eye vision whereby it transforms the original two dimensional captured image into a spatially variant (retinalike) representation [10]. Generally, the Cartesian image in any vision system can be resampled to retina-like image through the use of the mask, inspiring the retina. The retina can be roughly divided into two distinct regions: fovea and periphery. The fovea region (central portion of the retina) is formed by approximately constant size receptive fields and organized in hexagonal form [10]. In the periphery region, the receptive fields are circularly distributed with an area exponentially increasing as a function of the distance to the retina center. Overlapping may exist among the receptive fields in order to prevent some Cartesian areas from not being covered by the retina transformation (due to the circular geometry of the receptive fields) [10]. In many vision systems, there has been a trend to design and use true retina-like sensors [11,12] or simulate the log-polar images by software conversion [13,14]. In the software conversion of log-polar images, practitioners in pattern recognition usually named it as logpolar mapping [13,15]. Log-polar mapping has been used in designing systems that are scale and rotation invariant [16,17], and showed good performance. Another advantage is log-polar image representation has data compression manner. Therefore, in this proposed machine condition monitoring system, log-polar mapping is used to convert the input thermal image captured by thermal camera into log-polar image for solving the scaling and rotational invariant due to the change of position in either thermal camera or machines during monitoring process, and preserving fine image quality in a higher data compression manner. An example

illustrating the log-polar mapping of a hypothetical N-rings retina containing a 3 rings fovea is shown in Fig. 3. The log-polar mapping can be summarized as follows. Initially, the geometry of each section of the partitioned thermal image is in Cartesian form (x1, y1). Log-polar sampling is applied to sample the Cartesian input image into log-polar sampling image. Then, the log-polar image is mapped to another Cartesian form (x2, y2) whereby in this process, the log-polar sampling image is unwarped into a log-polar mapping image. Since the output logpolar mapping image has been compressed and also in Cartesian form, subsequent image processing task will become much easier. The center of pixel for log-polar sampling expression is described in [18]:

rðx1 ,y1 Þ ¼ logl yðx1 ,y1 Þ ¼

R ro

ð3Þ

Ny y1 arctan 2p x1

ð4Þ

The center of pixel for log-polar mapping expression is described in [18]:   2py r ð5Þ x2 ðr, yÞ ¼ l ro cos Ny r

y2 ðr, yÞ ¼ l ro sin

  2py Ny

ð6Þ

where R is the distance between the given point and the center of qffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi mapping ¼ x21 þ y21 , ro is a scaling factor defining the size of the circle at r(x1, y1)¼0 and l is the base of the algorithm.

l¼

1 þsinðp=Ny Þ 1sinðp=Ny Þ

ð7Þ

Ny is the total number of pixels per ring in log-polar geometry. This value is assigned by the user. For example, if the user assigns Ny ¼5, the ring is divided into 5 sectors or 5 pixels per ring. If Ny ¼100, each ring is divided into 100 sectors or 100 pixels per ring. The higher the Ny, the higher the resolution in y-axis (angular). In our experimental case, we assign Ny ¼ 70 and it allows a 2 fold data compression in a fine image resolution manner. This issue is discussed in detail in Section 6.

Fig. 3. An example of log-polar mapping of a N rings retina containing 3 rings fovea. To simplify the figure, no overlapping was use. Note that periphery portion is log-polar.

Author's personal copy W.-K. Wong et al. / Neurocomputing 74 (2010) 164–177

167

The number of rings in the fovea region expression is given in [18]: Nfov ¼

l l1

ð8Þ

The number of rings is an integer. Hence, the calculated Nfov is rounded to the closer integer value. To sample the Cartesian pixels (x1, y1) into log polar pixels (r, y), at each center point calculated using (3) and (4), the corresponding log-polar pixel (rn, yn) is covers a region of Cartesian pixels with radius rn ¼ lrn1

ð9Þ Fig. 6. Unwarping process.

where n ¼0, 1, y, N 1. Fig. 4 shows the conventional circle sampling method of log-polar mapping [12,15]. One of the disadvantages of using circle sampling is that certain regions of Cartesian pixels outside sampling circle did not cover any log-polar pixels. Therefore, some researchers [13,19,20] had adopted sector sampling method as shown in Fig. 5, which could maximize the coverage of Cartesian pixels for each log polar pixel. The region of Cartesian pixels cover by an individual logpolar pixel will have the same color intensity according to the respective original Cartesian center sampling point. During unwarping process, the (r, y) pixels will map each corresponding (x2, y2) pixel as shown in Fig. 6. The intensity value in each individual pixel equals the mean intensity value of all pixels inside the sampling circle on the original Cartesian image (x1, y1).

4. Quaternion based thermal image correlator In this section, the algorithm of the quaternion based thermal image correlator is described. 4.1. Algorithm for quaternion based thermal image correlation The reference image after performing discrete quaternion Fourier transforms (DQFT) [3] is given by Iðm,nÞ ¼ IR ðm,nÞi þ IG ðm,nÞj þIB ðm,nÞk

ð10Þ

where m and n are the pixel coordinates of the reference image. R, G, B reference images are represented by IR(m, n), IG(m, n) and IB(m, n), respectively, and i, j and k are the imaginary terms of quaternion complex number [1] and the real part is set to zero. Similarly, hi(m, n) is used for representing input image. Then, output b(m, n) is produced to judge whether the input image matches the reference image or not. If hi(m, n) is the space shift of the reference image hi ðm,nÞ ¼ Iðmm0 ,nn0 Þ

ð11Þ

then after some calculation, Maxðbr ðm,nÞÞ ¼ br ðm0 ,n0 Þ

ð12Þ

where br ðm,nÞ is the real part of bðm,nÞ and br ðm0 ,n0 Þ ¼

M 1 N 1 X X

2

9Iðm,nÞ9

ð13Þ

m¼0n¼0

where m and n are the image x-axis and y-axis dimensions, respectively. At the location (  m0, n0), the multipliers of i-, j-, kimaginary part of b( m0, n0) are equal to zero: Fig. 4. Conventional circle sampling.

bi ðm0 ,n0 Þ ¼ bj ðm0 ,n0 Þ ¼ bk ðm0 ,n0 Þ ¼ 0

ð14Þ

Thus, the following process can be utilized for thermal image correlation [3]: (1) Calculate energy of reference image Iðm,nÞ: EI ¼

M 1 N 1 X X

9Iðm,nÞ9

2

ð15Þ

m¼0n¼0

normalizing the reference image I(m, n) and the input image hi(m, n) as pffiffiffiffi Ia ðm,nÞ ¼ Iðm,nÞ= EI ð16Þ pffiffiffiffi ð17Þ Ha ðm,nÞ ¼ hi ðm,nÞ= EI (2) Calculate the output of discrete quaternion correlation (DQCR): ga ðm,nÞ ¼ Fig. 5. Sector sampling method for method log-polar image.

M 1 N 1 X X

t¼0Z¼0

Ia ðt, ZÞHa ðtm, ZnÞ

ð18Þ

Author's personal copy 168

W.-K. Wong et al. / Neurocomputing 74 (2010) 164–177

where ‘—’ means the quaternion conjugation operation and do the space reverse operation: gðm,nÞ ¼ ga ðm,nÞ ð19Þ (3) Perform inverse discrete quaternion Fourier transform (IDQFT) on (19) to obtain the correlation plane P(m, n). (4) Search all the local peaks on the correlation plane and record the location of the local peaks as (ms, ns). (5) Then at all the location of local peaks (ms, ns) found in step 4, we calculate the real to  complex  value of the DQCR output Pr ðms ,ns Þ        ð20Þ p ¼  Pr ðms ,ns Þ þ Pi ðms ,ns Þ þ Pj ðms ,ns Þ þ Pk ðms ,ns Þ

where Pr(ms, ns) is the real part of P(ms, ns). Pi(ms, ns), Pj(ms, ns) and Pk(ms, ns) are the i, j k parts of P(ms, ns), respectively. If p Zd1 and c1 o9P(ms, ns)9 oc2, it implies that at location (ms, ns), there is an object that has the same shape, size, color and brightness as the reference image. If d1 o 1, c1 o1 oc2 and all the variables are near to 1, the value of p decays faster with the color difference between the matching image and the reference image. Another classification characteristic used in quaternion based thermal image correlation is the peak-to-sidelobe ratio (PSR), which will be discussed in detail as below. The quaternion based thermal image correlation consists of 2 stages: enrollment stage and recognition stage. In the enrollment stage, one or multiple images of each machine condition are acquired. These multiple reference images have variety in color tones for different temperature conditions of the machines. The DQFT of the reference images are used to train fuzzy neural network and to determine the correlation filter for each possible machine’s conditions. In recognition stage, the thermal camera captures live machine thermal image and the DQFT of the image is correlated with the one of the reference images, stored in the database together with their corresponding filter coefficients. The inverse DQFT of this product is the result in the correlation output of that filter. A strong peak can be observed in the correlation output if the input image comes from imposter class. The peak-to-sidelobe ratio (PSR) is a method for measuring the peak sharpness which is defined as follows [2]: peakmeanðsidelobeÞ PSR ¼ sðsidelobeÞ

ð21Þ

where peak is the value of the peak on the correlation output plane. sidelobe refers to a fixed-sized surrounding area off the peak. mean is the average value of the sidelobe region. s is the standard deviation of the sidelobe region. Large PSR values indicate better match of the input image and the corresponding reference image. Enrollment stage and recognition stage are discussed in detail in the next sections.

4.2. Enrollment stage The schematic of enrollment stage is shown in Fig. 7. In the enrollment stage, the reference thermal images for each possible machine’s conditions in the database are partitioned according to S machine sections. Each machine section consists of one single machine to be monitored. S is the total number of machines to be monitored. These partitioned reference images are then encoded into a two dimensional quaternion array (QA) as follows: Isðt1 Þ ¼ Isrðt1 Þ þIsRðt1 Þ i þIsGðt1 Þ j þIsBðt1 Þ k

ð22Þ

where t1 ¼1, 2, y, T represents the number of reference images, Isrðt1 Þ represents the real part of quaternion array of sth machine section for reference image t1, s¼1, 2, y, S represents the number of partitioned machines’ sections. IsRðt1 Þ , IsGðt1 Þ and IsBðt1 Þ each represent the i-, j-, k-imaginary part of sth machine section for reference image t1, respectively. Discrete quaternion Fourier transform (DQFT) is then performed on the quaternion array in (22) to transform the quaternion image to the quaternion frequency domain. A twoside form of DQFT has been proposed by Ell [21,22] as follows: Isðt1 Þ ðm,nÞ ¼

M 1 N 1 X X

em1 2pðmt=MÞ Isðt1 Þ ðt, ZÞem2 2pðnZ=NÞ

ð23Þ

t¼0Z¼0

where e is exponential term, m1 and m2 are two units pure quaternion (the quaternion unit with real term equal to zero) that are orthogonal to each other [23]:

m1 ¼ m1,i i þ m1,j j þ m1,k k

ð24Þ

m2 ¼ m2,i i þ m2,j j þ m2,k k

ð25Þ

m21,i þ m21,j þ m21,k ¼ m22,i þ m22,j þ m22,k ¼ 1ði:e : m21 ¼ m22 ¼ 1Þ

ð26Þ

m1,i m2,i þ m1,j m2,j þ m1,k m2,k ¼ 0

ð27Þ

The output of DQFT, Isðt1 Þ , is used to train the max-product fuzzy neural network classifier and also to design the correlation filter. 4.2.1. Quaternion correlator (QC) To train the max-product fuzzy neural network classifier, the output of the DQFT is first passed to a quaternion correlator (QC) as shown in Fig. 8. The function of the QC is summarized as follows: For DQFT output of sth machine section, discrete quaternion correlation (DQCR) [24,25] is performed on reference image Isðt1 Þ and reference image Isðt2 Þ , then multiply to the corresponding filter coefficients (filtðt2 Þ ): gsðt1 ,

t2 Þ ðm,nÞ ¼

M 1 N 1 X X

t¼0Z¼0

I s(t1 ) I s(t1 )

Fig. 7. Schematic of enrollment stage.

Isðt1 Þ Isðt2 Þ ðtm, ZnÞ filtðt2 Þ

ð28Þ

Author's personal copy W.-K. Wong et al. / Neurocomputing 74 (2010) 164–177

169

(usually 1), for each of the reference images. Lagrange multiplier is used for optimization, yielding filtMACE ¼ D1 XðX 0 D1 XÞ1 c

Fig. 8. Quaternion correlator (QC).

where t1, t2 ¼ 1, 2, y, T are the number of reference image. Then, inverse DQFT is performed on (28) to obtain the correlation plane function: Psðt1 ,

t2 Þ ðm,nÞ ¼

1 N 1 X X 1 M em1 2pðmt=MÞ gsðt1 , 4p2 t ¼ 0 Z ¼ 0

t2 Þ ðm,

This equation is the closed form solution to be the linear constrained quadratic minimization. D is diagonal matrix with the average power spectrum of the reference images placed as elements along diagonal of the matrix. X contains Fourier transform of the reference images lexicographically re-ordered and placed along each column. As an example, if there are T thermal reference images of size 100  243(¼ 24300), then X will be a 24300  T matrix. X0 is the matrix transpose of X. c is a column vector of length T with all entries equal to 1. The second type of MACE filter is the unconstrained MACE (UMACE) filter [30]. Just like conventional MACE filter, the UMACE filter also minimizes the average correlation energy of the reference images and maximizes the correlation output at the origin. The difference between conventional MACE filter and UMACE filter is the optimization scheme. Conventional MACE filter is using Lagrange multiplier whereas for UMACE filter, it uses the Raleigh quotient, which leads to the following equation: filtUMACE ¼ D1 m

nÞem2 2pðnZ=NÞ ð29Þ

The correlation plane is a collection of correlation values, each one obtained by performing a pixel-by pixel comparison (inner product) of two images (Isðt1 Þ and Isðt2 Þ . A sharp peak in the correlation plane indicates the similarity of Isðt1 Þ and Isðt2 Þ , while the absence or a lower value of such peak indicates the dissimilarity of the two. Subsequently, psðt1 ,t2 Þ and PSRsðt1 ,t2 Þ are calculated from the correlation plane as in (29) using (20) and (21), respectively. psðt1 ,t2 Þ are the p-values of reference image Iðt1 Þ correlate on reference image Iðt2 Þ in sth machine section, while PSRsðt1 ,t2 Þ represents PSR values of reference image Iðt1 Þ correlate on reference image Iðt2 Þ in sth machine section. These values are then fed into the max-product fuzzy neural network classifier for training and to calculate the weight, which will be discussed in Section 5. 4.2.2. Correlation filter Conventional filtering methods [26] emphasized on applying matched filters. Matched filters are optimal for detecting a known reference image in additive white Gaussian noise environment. If the input image changes slightly from the known reference image (scale, rotation and pose invariant), the detection of the matched filters degrades rapidly. However the emerge of correlation filter designs [27] have changed to handle such types of distortions. The minimum average correlation energy (MACE) filters [28] are one of such design and show good results in the field of automatic target recognition and applications in biometric verification [6,29]. MACE filters different from matched filters in that more than one reference image are used to synthesize a single filter template, therefore making its classification performance invariant to shift of the input image [27]. There are two types of MACE filters in general, namely: (1) conventional MACE filter [28] and (2) unconstrained MACE (UMACE) filter [30], both with the goal to produce sharp peaks that resemble two dimensional delta-type correlation outputs when the input image belongs to the authentic class and low peaks in imposter class. Conventional MACE filter [28] minimizes the average correlation energy of the reference images while constraining the correlation output at the origin to a specific value

ð30Þ

ð31Þ

where D is the diagonal matrix, the same as that in the conventional MACE filter. m is a column vector containing the mean values of the Fourier transform of the reference images. Besides MACE filters, there is a type of correlation filter, namely the unconstrained optimal tradeoff synthetic discriminant filter (UOTSDF) shown by Refreiger [31] and Kumar et al. [32], which has yielded good verification performance. The UOTSDF is given by filtUOTSDF ¼ ðaD þ

pffiffiffiffiffiffiffiffiffiffiffiffi 1a2 CÞ1 m

ð32Þ

where D is a diagonal matrix with average power spectrum of the training image placed along the diagonal elements. m is a column vector containing the mean values of the Fourier transform of the reference images. C is the power spectral density of the noise. White noise spectrum is the dominant source in predicting the performance of a thermal imaging system [46]. It is caused by the fluctuation in the detector output. Other noise sources (total up as background noise) are not that significant and normally limited/filtered out by internal filter of advanced thermal imaging system. For most of the applications, a white noise power spectral density is assumed; therefore C reduces to the identity matrix. According to the derivation work done in [32], to determine the OTSDF, the authors minimized the energy function, which obtains: a2 + b2 + g2 + d2 ¼1. In UOTSDF, the constants b, g, d E0. a term is typically set to be close to 1 to achieve good performance even in the presence of noise; however it also helps improve generalization to distortions outside the reference images. By comparing the three correlation filters listed above, conventional MACE filter is complicated to implement whereby it requires inversion of a T  T matrix. UMACE filter is simpler to implement from the computational viewpoint as it involves inverting the diagonal matrix only, and the performance is close to the conventional MACE but poorer than UOTSDF. Therefore, we plan to extend UOTSDF into quaternion version to use in our quaternion based thermal image correlator for the classification of machine condition since it is less complicated from computational viewpoint than conventional MACE filter and achieves good performance. Further details about the comparison performance of these filters may be obtained from [27].

Author's personal copy 170

W.-K. Wong et al. / Neurocomputing 74 (2010) 164–177

Fig. 9. Schematic of recognition stage.

4.3. Recognition stage The schematic of recognition stage for classification of machine condition by quaternion correlation is shown in Fig. 9. During the recognition stage, thermal image captured live is first partitioned according to S machine sections. The partitioned image is then encoded into two dimensional quaternion array (QA) as follows: hsðiÞ ¼ hsrðiÞ þ hsRðiÞ i þhsGðiÞ j þ hsBðiÞ k

ð33Þ

where i represents the input image, hsr(i) represents the real part of quaternion array of sth machine section for input image i, s ¼1, 2, y, S represents the number of partitioned machines’ sections. hsR(i), hsG(i) and hsB(i) each represents the i-, j-, k-imaginary parts of sth machine section for input image i, respectively. The quaternion array in (33) performs DQFT to transform the quaternion image to the quaternion frequency domain. A twoside form of DQFT is used: hsðiÞ ðm,nÞ ¼

M 1 N 1 X X

em1 2pðmt=MÞ hsðiÞ ðt, ZÞem2 2pðnZ=NÞ

ð34Þ

t¼0Z¼0

where e is exponential term, m1 and m2 are two units pure quaternion as shown in (24) and (25), respectively. The output of the DQFT, hs(i), is cross correlated with every quaternion correlation filter in the database using the quaternion correlator (QC) just as the one shown in Fig. 8, but the DQFT output is now hs(i). In QC, quaternion correlation is performed on hs(i) with reference images Isðt2 Þ from the database, and is multiplied with corresponding filter coefficients (filtðt2 Þ ): gsði, t2 Þ ðm, nÞ ¼

M 1 N 1 X X

hsðiÞ Isðt2 Þ ðtm, ZnÞ filtðt2 Þ

ð35Þ

t¼0Z¼0

5.1. Define 2 classes, namely: overheat class and non-overheat class

Then, inverse DQFT is performed on (35) to obtain the correlation plane function Psði,t2 Þ ðm,nÞ ¼

M 1 N 1 X X

1 em1 2pðmt=MÞ gsði,t2 Þ ðm,nÞ em2 2pðnZ=NÞ 4p2 t ¼ 0 Z ¼ 0

appropriately handle both the uncertainty and imprecision in linguistic semantics, model expert heuristics and provide requisite high level organizing principles [33]. Neural network in engineering field refer to a mathematical/computational model based on biological neural network. Neural network provides selforganizing substrates for low level representation of information with adaptation capabilities. Fuzzy logic and neural network are complementary technologies. Therefore, it is plausible and justified to combine both these approaches in the design of classification systems. Such integrated system is referred to as fuzzy neural network classifier [33]. Various fuzzy neural network classifiers have been proposed in the literature [34–37], and there has been much interest of many fuzzy neural networks applying max–min composition as functional basis [38–40]. However, in [41], Leotamonphong and Fang mentioned that the max–min composition is ‘‘suitable only when a system allows no compensatability among the elements of a solution vector’’. He proposed to use max-product composition in fuzzy neural network rather than max–min composition. Another work by Bourke and Fisher in [42] also commented that the max-product composition gives better results than the traditional max-min operator. Therefore, efficient learning algorithms have been studied by others [43,44] using the max-product composition afterwards. In this paper, a fuzzy neural network classifier using maxproduct composition is proposed for thermal image classification for machine condition monitoring system. The max-product composition is the same as a single perceptron except that summation is replaced by maximization, and in the max–min threshold unit, min is replaced by product.

ð36Þ

psði,t2 Þ and PSRsði,t2 Þ are calculated from the correlation plane as in (36) using Eqs. (20) and (21), respectively. psðt1 ,t2 Þ means pvalues of input image h(i) correlated on reference image Iðt2 Þ in sth machine section, while PSRsði,t2 Þ means PSR values of input image h(i) correlated on reference image Iðt2 Þ in sth machine section. These values are then fed into max-product fuzzy neural network classifier to perform classification for machines’ conditions, which will be discussed in Section 5.

5. Max-product fuzzy neural network classifier Fuzzy logic is a type of multi-valued logic that is derived from fuzzy set theory to deal with approximate reasoning. Fuzzy logic provides high level framework for approximate reasoning that can

The reference images for all possible machine conditions are captured and stored in a database. Each reference image is assigned with a unique number starting from 1 to T, where T is the total number of reference images. These reference images are interpreted by an operator (human observer), the overall description of which could be called the ‘Operator perceived activity’ (OPA) [45]. The operator comments on each of the reference images and classified it into either overheat class or non-overheat class by storing the unique number of reference images according to the classes type.

5.2. Training max-product fuzzy neural network classifier The max-product fuzzy neural network classifier is trained in 4 steps: (1) PSRsðt1 ,t2 Þ and psðt1,t2Þ outputs from the quaternion correlator of the enrollment stage are fuzzified through the activation

Author's personal copy W.-K. Wong et al. / Neurocomputing 74 (2010) 164–177

(6) Determine which element in Ys classification matrix match with Classs:

functions (Gaussian membership function): " # ðPSRsðt1 ,t2 Þ 1Þ2 GPSRsðt1 ,t2 Þ ¼ exp 2

ð37Þ

" # ðPSRsðt1 ,t2 Þ 1Þ2

ð38Þ

s

GPsðt1 ,t2 Þ ¼ exp

s2

c ¼ the position number of element in Ys classification

where s is the smoothing factor, which is the deviation of the Gaussian functions. (2) Calculate the product value for sth machine section of the fuzzy neural network classifier at each correlated image: Gsðt1 ,t2 Þ ¼ GPSRsðt1 ,t2 Þ  Gpsðt1 ,t2 Þ

ð39Þ

Xs training

Gsð1,1Þ 6G 6 sð2,1Þ ¼6 6 ^ 4

GsðT,1Þ

Gsð1,2Þ



Gsð2,2Þ



^

&

GsðT,2Þ



Gsð1,TÞ

ð40Þ

(4) The output is set to produce 1 if it is authentic class and 0 for imposter class, and it is in an array of Yidentity, with identity matrix of dimension T  T. To calculate the weight w for sth machine section, the equation is ws ¼ Xstraining 1 Yidentity

ð48Þ

c corresponds to the assigned number of reference image in database. (7) Based on two sets of fuzzy IF-THEN rules, perform defuzzification: R1s : IF matches with the number stored in overheat class

ð49Þ

3

Gsð2,TÞ 7 7 7 ^ 7 5 GsðT,TÞ

matrix which has the equal value with Classs :

of sth machine, THEN alarm : ‘machines overheat’:

(3) Gather and store the product values in an array: 2

171

ð41Þ

R2s : IF c matches with the number stored in non-overheat class of sth machine, THEN alarm : ‘machines function properly’:

ð50Þ

6. Experimental results In this section, the application of log-polar mapper, quaternion based thermal image correlator together with max-product fuzzy neural network classifier for machine condition monitoring system is illustrated. Here, some experiments were used to prove the algorithms introduced in Sections 3–5.

5.3. Max-product fuzzy neural network classification The max-product fuzzy neural network classification is carried out through 7 steps:

6.1. Database of reference thermal images for all possible machine conditions

(1) PSRsði,t2 Þ and psði,t2 Þ outputs from the quaternion correlator of the recognition stage are fuzzified through the activation functions (Gaussian membership function):

Thermal images collected at the Applied Mechanics Lab in Faculty of Engineering and Technology, Multimedia University, is used to test the proposed machine condition monitoring system. The database consists of log-polar images unwarped from Cartesian thermal images captured from three functioning machines using the AXT100 thermal camera. An image captured by a digital camera on the site is shown in Fig. 10. A thermal image is also captured using AXT100 at the same position for all the functioning machines with overheated condition, as shown in Fig. 11, and the corresponding log-polar form of it is shown in Fig. 12. In Fig. 10, machine A (leftmost) and machine C (rightmost) are vibro test machines of the same model and same specifications, whereas machine B (center) is a fatigue test machine. Three machines are considered to be overheated when their motor temperature reach 90 1C. The captured thermal images using AXT100 via MATLAB has 256  248 display resolution pixels (after camera interpolation). It is cropped with MATLAB for the region of interest for 3 machine sections, which is 243  100 pixels. Each machine section is

" GPSRsði,t Þ ¼ exp

ðPSRsði,t2 Þ 1Þ2

ð42Þ

s2

2

Gpsði,t2 Þ ¼ exp

#

" # PSRsði,t2 Þ 12

ð43Þ

s2

(2) Calculate the product value for sth machine section of the fuzzy neural network classifier at input image on the training images in the database: Gsði,t2 Þ ¼ GPSRsði,t Þ  Gpsði,t2 Þ

ð44Þ

2

(3) Gather and store the product values in an array: h Xs classification ¼ Gsði,1Þ

Gsði,2Þ



Gsði,TÞ

i

ð45Þ

(4) Obtain the classification outcomes for each machine condition in the sth section by multiplying (45) with the weight trained at (41): Ys classification ¼ Xs classification  ws

ð46Þ

(5) Classify the input machine condition with the class of machine condition it belongs to by using max composition: h Xs classification ¼ Gsði,1Þ

Gsði,2Þ

...

Gsði,TÞ

i

ð47Þ

Fig. 10. Image on the site capture by digital camera.

Author's personal copy 172

W.-K. Wong et al. / Neurocomputing 74 (2010) 164–177

machine C overheated, (5) machine A and machine B overheated, (6) machine A and machine C overheated, (7) machine B and machine C overheated, (8) machine A, B, and C are overheated.

6.2. Quaternion based thermal image correlation using unconstrained optimal tradeoff synthetic discriminant filter (UOTSDF)

Fig. 11. Image on the site capture by thermal camera for all machines are in overheats condition.

Fig. 12. Log-polar mapping of Fig. 11 as divided into 3 partitioned machines’ sections.

Fig. 13. Available Color Palettes in AXT100.

81  100 pixels. After performing log-polar mapping, it becomes 60  70 pixels for each machine section. The database has T¼30 reference images, each machine section has dimension of 180 horizontal pixels  70 vertical pixels with varying conditions (temperature level with different color tones in HOT color palette ranging from black, brown, dark red, red, orange, yellow, light yellow to white). Fig. 13 shows the color palette of AXT100. The type GRAY is grayscale, which is not color. Since the proposed algorithm is color based (quaternion based); hence it is not suitable. The types GRAYRAIN, IRONRAIN, SPECTRA 1, FLOW 1–4 are not suitable too because there are black regions intermediate of many of its range as gaps. Type COOL and MOTION only required majority 2 channels (G–B, or R–B). This again limited user choice only on IRON, HOT and SPECTRA 2. Type IRON and SPECTRA 2 have a temperature range of magenta color, which consists of the mixture of R and B. This will cause error/ misjudgment in certain processing/manipulation tasks. Hence the ideal one is by using HOT color palette, and that’s the reason HOT color palette is pick for the proposed image processing tools. They can be divided into 8 major outcomes, namely (1) All machines function properly (none of the machines are overheated), (2) machine A overheated, (3) machine B overheated, (4)

In the experiment, 30 reference images in database are used to synthesize a single UOTSDF using (32). D and m are calculated from the reference images, C is an identity matrix of dimension 30  30 and a set to 1. These values are substituted into (32) to calculate the filter’s coefficients. Then in enrollment stage, for each filter line as in Fig. 8, cross  correlation   is performed on all the DQFT reference images Isðt1 Þ with Isðt2 Þ in the database, and multiplying the output value with corresponding filter coefficients, respectively, where t1, t2 ¼1, 2, y, 30; s¼1, 2, 3. In recognition stage, for each filter line, cross correlation of the DQFT form is performed on input image (hs(i)) with the DQFT form of   reference images in database Isðt1 Þ and multiplying the output value with corresponding filter coefficients, respectively. For authentic case (good match between two images), the correlation plane will have sharp peaks and it will not exhibit such strong peaks for imposter case (bad or no match between two images). The investigation for these two cases is given below. Authentic case: Fig. 16(a, b and c) shows the samples correlation plane for input thermal image (for every machine is overheated as in Fig. 14, log polar transform in Fig. 15) matching with one of the reference image of overheated class in the database (as in Fig. 12), for section machine A, section machine B and section machine C, respectively. Since the three pairs of images are in good match, smooth and sharp peaks correlation plane can be observed Fig. 16. Imposter case: Fig. 19(a, b and c) shows the samples correlation plane for input thermal image (for every machine is not-overheated as in Fig. 17, log polar transform in Fig. 18) matching with one of the reference images of overheated class in the database (as in Fig. 12), for section machine A, section machine B and section machine C, respectively. Since the three pairs of images are not in good match, no sharp peak was observed from their correlation.

Fig. 14. Sample input thermal image (all machines in overheat condition).

Fig. 15. Log-polar mapping of Fig. 14 as divided into 3 partitioned machines’ sections.

Author's personal copy W.-K. Wong et al. / Neurocomputing 74 (2010) 164–177

173

Fig. 16. Samples correlation plane for input thermal image (for every machine is overheated) matching with one of the reference image of overheated class for both all the machines in the database (authentic case) (a) section machine A, (b) section machine B, (c) section machine C.

Table 1 Normalized PSR and p-values for both authentic and imposter case.

Fig. 17. Sample input thermal image (all machines in overheat condition).

Authentic case

Normalized PSR

Normalized p-value

Section mac. A Section mac. B Section mac. C

0.95281 1.0000 0.9674

0.7971 0.8987 0.8317

Imposter case

Normalized PSR

Normalized p-value

Section mac. A Section mac. B Section mac. C

0.0338 0.0421 0.0371

0.0287 0.0373 0.0345

normalized PSR and p-value in authentic case of section machine A, B and C, whereas small PSR and p-value exhibiting in the imposter case of section machine A, B and C.

Fig. 18. Log polar mapping of Fig. 16 as divided into 3 partitioned machines’ sections.

Table 1 shows the PSR and p-value for both authentic and imposter case as shown in Fig. 16 and 19 for section machine A, B and C. Note that the sharp correlation peak resulting in large

6.3. Log-polar mapping for solving rotation and scaling invariant Log-polar mapping is used in the proposed machine condition monitoring system to solve the rotation and scaling invariant. For each of the partitioned machine section (with total x1 ¼81 and total y1 ¼100), log-polar sampling is used to sample Cartesian

Author's personal copy 174

W.-K. Wong et al. / Neurocomputing 74 (2010) 164–177

Fig. 19. Samples correlation plane for input thermal image (for every machine is not-overheated) matching with one of the reference image of overheated class for both all the machines in the database (imposter case) (a) section machine A, (b) section machine B, (c) section machine C.

Fig. 20. An example case of rotational invariant (a) Cartesian input image capture by digital camera (1.6 m), (b) the same image as in (a) capture using thermal camera during the machine being overheated, (c) log polar mapping image of (b), (d) rotational invariant of (a) capture by digital camera, (e) thermal image of (d) in Cartesian form, (f) log-polar form of (e).

input image into log-polar sampling image (with total r ¼60 and total y ¼70) and then mapping into log-polar mapping image with total x2 ¼60 and y2 ¼70. By applying log-polar mapping, a data reduction ratio of 8100:4200 is obtained with almost 2 fold of data compression in a fine image resolution manner. Fig. 20 shows an example of log-polar mapping for section machine A. Fig. 20b is captured using thermal camera at the same position of the machine in Fig. 20a when the machine is overheated. Say, if a rotational invariant occurs on the same machine as shown in Fig. 20d, the corresponding thermal images in Cartesian and log polar forms are shown in Fig. 20e f, respectively. The output correlation plane obtained from (i) Fig. 20b and b itself, (ii) Fig. 20b and e, (iii) Fig. 20c and c itself, (iv) Fig. 20c and f are shown in Fig. 21a, b, c and d, respectively. From Fig. 21a and b, the correlation planes are obtained among thermal images that are captured directly without log-polar mapping. It is observable that the correlation planes are actually not smooth and consist of more than one focusing sharp peaks. It is difficult to detect PSR and pvalues from such correlation planes. However, in Fig. 21c and d, after performing log-polar mapping on the captured images, the correlation planes are smooth and they are almost identical in shape. The calculated PSR and p-values from these two planes after normalization are (PSRFig. 21c ¼1.0000, PSRFig. 21d ¼0.9267, pFig. 21c ¼ 0.8894, pFig. 21d ¼0.8054). PSRFig. 21d is of 7.4% deviate from PSRFig. 21c and pFig. 21d is of 8.5% deviate from pFig. 21c.

Author's personal copy W.-K. Wong et al. / Neurocomputing 74 (2010) 164–177

175

Fig. 21. Output correlation planes among: (a) Fig. 20b and b itself, (b) Fig. 20b and e, (c) Fig. 20c and c itself, (d) Fig. 20c and f.

Fig. 22. An example case of scaling invariant (a) Cartesian input image capture by digital camera (3.2 m), (b) the same image as in (a) capture using thermal camera during the machine is overheated, (c) log polar mapping image of (b).

The deviation is less than 9%, which can be considered close to each other. This shows that log-polar mapping not only provides a smoother correlation plane for PSR and p-values measurement, it actually solves rotational invariant problem in quaternion based thermal image correlation too. For scaling invariant, a test is conducted on machine section A but at different distance between the machine and the camera, as shown in Fig. 20a and Fig. 22a. The distance of machine–camera is 1.6 m in Fig. 20a and 3.2 m in Fig. 22a. Their thermal images captured at overheated condition are shown in Fig. 20b and Fig. 22b, respectively, and their log-polar images are shown in Fig. 20c and Fig. 22c respectively. The output correlation plane obtained for (i) Fig. 20b and Fig. 22b, and (ii) Fig. 20c and Fig. 22c are shown in Fig. 23a and b, respectively. The output correlation plane in Fig. 23a (correlated

images without log-polar mapping) is with rough surfaces and overall low normalized amplitudes as compared to Fig. 23b (correlated images with log-polar mapping). By comparing output correlation planes in Fig. 23b and Fig. 21c, the overall normalized amplitudes in the correlation plane of Fig. 23b are reduced to almost 1=4 of that in Fig. 21c. However, their plane shapes are almost identical. Sharp peaks can still be detected around central regions, and their PSR and p-values after normalizing are (PSRFig. 21c ¼1.0000, PSRFig. 23b ¼0.8647, pFig. 21c ¼0.8894, pFig. 23b ¼0.7681). PSRFig. 23b is of 13.5% deviate from PSRFig. 21c and pFig. 23d is of 12.3% deviate from pFig. 21c. The deviation is less than 14%, which can be considered near to each other. Hence, it is proven that log-polar mapping can be used to solve scaling invariant problem in quaternion based thermal image correlation.

6.4. Efficiency of the machine monitoring system The machine monitoring system was evaluated based on the thermal images captured live and displayed on monitor scene as interpreted by an operator (human observer) the overall description of which could be called the ‘operator perceived activity’ (OPA) [45]. The operator comments on the images captured by the thermal camera, whether any of the machines are overheated or not. The system was evaluated with 30,000 samples images captured by the thermal camera for monitoring the machines, as in Fig. 10. Out of the 30,000 samples images, one-third (10,000) sample images are captured in a fixed position (same angle—01 face to face camera– machine as in Fig. 11, same distance—1.6 meter camera–machine),

Author's personal copy 176

W.-K. Wong et al. / Neurocomputing 74 (2010) 164–177

Fig. 23. Output correlation planes among: (a) Fig. 20b and Fig. 22b, (b) Fig. 20c and Fig. 22c.

one-third (10,000) sample images are captured in a fixed distance (1.6 m camera–machine) but different angles (ranging between  301 camera–machine—rotational invariant) and another one-third (10,000) samples images are captured in a fixed angle (01 face to face camera–machine as in Fig. 11) but different distances (ranges between 1.6 and 3.6 m camera–machine—scaling invariant). In fixed position case, 9851 samples are tracked correctly (machine conditions agreed by both observer and the machine condition monitoring system), with an accuracy of 98.51%, whereas in rotational invariant case, 9006 samples are tracked correctly, with an accuracy of 90.06%. In scaling invariant case, 9366 samples are tracked correctly, with an accuracy of 93.66%. From the experimental results, it is proven that the proposed machine monitoring system achieved the best performance in monitoring fixed position machines while its performance degrades a little in scaling invariant and rotational invariant cases. Out of 30,000 samples images, 28,223 were tracked perfectly (output machine conditions agreed by both observer and the machine condition monitoring system), i.e. an overall accuracy of 94.08%. The proposed machine monitoring system is also discovered to be capable of accepting rotational invariant angle up to  301 and scaling invariant distance up to maximum 3.6 m. Beyond this angle and distance range, the machine sections seriously overlap/overshape in images, and seriously degrade the classification results. The proposed machine monitoring system for now seems to only accept mild overlap/overshape for assigned machine sections in images caused by mild scaling and rotational invariant in camera viewpoint. This issue can be further studied in future for the enhancement of the machine monitoring system to be used in serious scaling and rotational invariant environment.

7. Conclusion This paper presented a system capable of monitoring machine condition using log-polar mapping method, quaternion based thermal image correlator and max-product fuzzy neural network classifier. The results show that the proposed machine condition monitoring system achieved very high accuracy, i.e. more than 94%. One of the advantages using quaternion correlation rather than conventional correlation method is that quaternion correlation method deals with color images without converting those images into grayscale images. Hence, important color information can be preserved. Max-product fuzzy neural network provides high level framework for approximate reasoning; hence it is best

suited for classification use. We also found that the use of logpolar mapping solves rotation and scaling problems in quaternion based thermal image correlation. In addition, log-polar mapping has data compression capability. Therefore the use of log-polar mapping reduces the computation time and memory storage needs. Log-polar mapping also smoothens the output correlation plane, hence making better measurement for PSR and p-value. Based on the experimental results, the use of PSR and p-value provides higher accuracy in tracking thermal condition in the proposed machine condition monitoring system. In future, there is a plan to implement this machine condition monitoring system in a wide area coverage using minimum hardware manner, whereby the machines surrounded in an omnidirectional (3601) view can be monitored by using a single thermal camera and a specific design hyperbolic mirror. Therefore, a mathematical model has to be formulated to design the geometry of such mirror and an unwarping processing method also needs to be researched for unwarping the omnidirectional image into panoramic forms for better machine partitioning purpose. These topics will be addressed in future work. References [1] W.R. Hamilton, in: Elements of Quaternions, Longmans, Green, London, U.K, 1866. [2] C. Xie, M. Savvides, B.V.K. Vijaya Kumar, Quaternion correlation filters for face recognition in wavelet domain, in: Proceedings of the International Conference on Acoustic, Speech and Signal Processing (ICASSP 2005), pp. II85–II88. [3] S.C. Pei, J.J. Ding, J. Chang, Color pattern recognition by quaternion correlation, in: Proceedings of the International Conference on Image Processing, vol. 1, 2001, pp. 894–897. [4] C. Eddie Moxey, S.J. Sangwine, T.A. Ell, Hypercomplex correlation techniques for vector images, IEEE Trans. Signal Process. 51 (7) (2003) 1941–1953. [5] C. Weiman, Log-polar vision for mobile robot navigation, in: Proceedings of the Electronic Imaging Conferences, November 1990Boston, USA, pp. 382–385. [6] B.V.K. Vijaya Kumar, M. Savvides, K. Venkataramani, C. Xie, Spatial frequency domain image processing for biometric recognition, Proc. Int. Conf. Image Process. 1 (2002) I53–I56. [7] R.O. Duda, P.E. Hart, D.G. Stork, in: Pattern Classification, second ed., Wiley, N.Y, 2001. [8] R.K. Brouwer, A fuzzy threshold max-product unit, with learning algorithm, for classification of pattern vectors, in: Proceedings of the VI Brazillian Symposium on Neural Networks, January 2000, pp. 208–212. [9] /http://www.aas2.com/products/axt100/S. [10] H.M. Gomes, R.B. Fisher, Learning-based versus model-based log-polar feature extraction operators: a comparative study, XVI Brazillian Symposium on Computer Graphics and Image Processing, October 2003, pp. 299–306. [11] LIRA Lab, Document on specification, Technical Report, Espirit Project n.31951-SVAVISCA-available at http://www.lira.dist.unige.it. [12] R. Wodnicki, G.W. Roberts, M.D. Levine, A foveated image sensor in standard CMOS technology, in: Proceedings of the Custom Integrated Circuits Conference, May 1995Santa Clara, pp. 357–360.

Author's personal copy W.-K. Wong et al. / Neurocomputing 74 (2010) 164–177

[13] F. Jurie, A new log-polar mapping for space variant imaging: application to face detection and tracking, Pattern Recognition, 32, Elsevier Science, 1999, pp. 865–875. [14] V.J. Traver, Motion estimation algorithms in log-polar images and application to monocular active tracking, Ph.D. Thesis, Dep. Llenguatges I, Sistemes Informatics, University Jaume I, Castellon, Spain, September 2002. [15] H. Araujo, J.M. Dias, An introduction to the log-polar mapping, Proc. 2nd Workshop Cybernatic Vision (1996) 139–144. [16] G. Sandidni, M. Tristarelli, Vision and space-variant sensing, Neural Networks for Perception, vol. 1, Academic Press, 1992, pp. 398–425. [17] H. Gomes, R. Fisher, Structural learning from iconic representations, Lecture Notes in Artificial Intelligence, Springer, 2000, pp. 399–408. [18] F. Berton, A brief introduction to log-polar mapping, Technical Report, LIRALab, University of Genova, 2006. [19] M. Bolduc, M.D. Levine, A review of biologically-motivated space variant data reduction models for robotic vision, Comput. Vision Image Understanding 69 (2) (1998) 170–184. [20] C.G. Ho, R.C.D. Young, C.R. Chatwin, Sensor geometry and sampling methods for space variant image processing, Pattern Analysis and Application, Springer Verlag, 2002, pp. 369–384. [21] T.A. Ell, Quaternion-Fourier transforms for analysis of two-dimensional linear time-invariant partial differential systems, Proc. 32nd Conf. Decision Contr. (1993) 1830–1841 Dec. [22] T.A. Ell, Hypercomplex spectral transforms, Ph.D. Dissertation, University of Minnesota, Minneapolis, 1992. [23] S.C. Pei, J.J. Ding, J.H. Chang, Efficient implementation of quaternion Fourier transform convolution and correlation by 2-D complex FFT, IEEE Trans. Signal Process. 49 (11) (2001) 2783–2797. [24] S.J. Sangwine, T.A. Ell, Hypercomplex auto- and cross-correlation of colour images, Proc. Int. Conf. Image Process., ICIP, 1999, pp. 319–323. [25] T.A. Ell, S.J. Sangwine, Colour-sensitive Edge Detection using Hypercomplex Filters, EUSIPCO, 2000, pp. 151–154. [26] A. Vanderlugt, Signal detection by complex spatial filtering, IEEE Trans. Inf. Theory 10 (1964) 139–145. [27] M. Saviddes, K. Venkataramani, B.V.K. Vijaya Kumar, Incremental updating of advanced correlation filters for biometric authentication systems, Proc. Int. Conf. Multimedia Expo, 3, ICME, 2003 p.p. 229–232. [28] A. Mahalanobis, B.V.K. Vijaya Kumar, D. Casasent, Minimum average correlation energy filters, Appl. Opt. 26 (1987) 3633–3640. [29] M. Savvides, B.V.K. Vijaya Kumar, P. Khosla, Face verification using correlations filters, in: Proceedings of the Third IEEE Automatic Identification Advanced Technologies, Tarrytown, NY, 2002) pp. 56–61. [30] A. Mahalanobis, B.V.K. Vijaya Kumar, S.R.F. Sims, J.F. Epperson, Unconstrained correlation filters, Appl. Opt. 33 (1994) 3751–3759. [31] P. Refreiger, Filter design for optical pattern recognition: multi-criteria optimization approach, Opt. Lett. 15 (1990) 854–856. [32] B.V.K. Kumar, D.W. Carlson, A. Mahalanobis, Optimal trade-off synthetic discriminant function filters for arbitrary devices, Opt. Lett. 19 (19) (1994) 1556–1558. [33] S. Kumar, in: McGraw Hill Int. (Ed.), Neural Networks: A Classroom Approach, 2004. [34] J.J. Buckley, Y. Hayashi, Fuzzy neural networks: a survey, Fuzzy Sets Syst. 66 (1994) 1–13. [35] C.T. Lin, C.S.G. Lee, in: Neural Fuzzy Systems: A Neuro-fuzzy Synergism to Intelligent Systems, Prentice Hall, Upper Saddle River, NJ, 1996. [36] D. Nauck, F. Klawonn, R. Kurse, in: Foundations of Neuro-fuzzy Systems, Wiley, Chichester, U.K, 1997. [37] S.K. Pal, S. Mitra, in: Neuro-fuzzy Pattern Recognition: Methods in Soft Computing, Wiley, Chichester, UK, 1999. [38] R. Ostermark, A fuzzy neural network algorithm for multigroup classification, Elsevier Science, Fuzzy Sets and Systems, 105 (1999) 113–122. [39] H.K. Kwan, Y. Cai, A fuzzy neural network and its application to pattern recognition, IEEE Trans. Fuzzy Syst. 2 (3) (1997) 185–193. [40] G.Z. Li, S.C. Fang, Solving interval-valued fuzzy relation equations, IEEE Trans. Fuzzy Syst. 6 (2) (1998) 321–324. [41] J. Leotamonphong, S. Fang, An efficient solution procedure for fuzzy relation equations with max product composition, IEEE Trans. Fuzzy Syst. 7 (4) (1999) 441–445. [42] M.M. Bourke, D.G. Fisher, A predictive fuzzy relational controller, in: Proceedings of the Fifth International Conference on Fuzzy Systems, 1996, pp. 1464–1470.

177

[43] M.M. Bourke, D.G. Fisher, Solution algorithms for fuzzy relational equations with max-product composition, Fuzzy Sets Syst. 94 (1998) 61–69. [44] P. Xiao, Y. Yu, Efficient learning algorithm for fuzzy max-product associative memory network, SPIE 3077 (1997) 388–395. [45] J. Owen, A. Hunter, E. Fletcher, A fast model-free morphology-based object tracking algorithm, Br. Mach. Vision Conf. (2002) 767–776. [46] H.V. Kennedy, Modeling noise in thermal imaging systems, Proc. SPIE 1969 (1993) 66–70.

Wai-Kit Wong was born in Bahau, Malaysia. He received his B. Eng. Sc. Degree (Honors) in electronics engineering majoring in Telecommunications and M. Eng. Sc. Degree in the years 2003 and 2006, respectively, both from Multimedia University (MMU), Malaysia. In November 2006, he joined the Faculty of Engineering and Technology, MMU, as a lecturer and currently he is pursuing his PhD degree in MMU. His research interests fall in the general areas of telecommunications and image processing. He has worked in the areas of digital signal processing, multiuser detection, power line communication and image processing. His current research interests include face recognition, pattern recognition, omnidirectional surveillance system and thermal condition monitoring.

Chu-Kiong Loo was born in Melaka, Malaysia. He received his PhD (University Sains Malaysia), B. Eng. (First class Hons in Mechanical Engineering from University Malaya), PhD in AI and robotics from University Science of Malaysia (USM). He is the Project leader and the Dean of Faculty of Information Science and Technology, Multimedia University, Malaysia. Formerly he was a design Engineer in various Industrial firms in different capacities as well as he is the former chairman of Centre for Robotics and automation in Multimedia University. Currently, he is an expert in Quantum Soft-computing includes Brain inspired quantum neural network, quantum clustering, constructivism Inspired Neural Network, synergetic Neural Networks, Robotic Intelligent System Development and Applications.

Way-Soong Lim was born in Melaka, Malaysia. He received his PhD (Multimedia University Malaysia) in year 2007. He is currently the deputy Dean of Faculty of Engineering and Technology, Multimedia University. His research interests include the general areas of Neural Networks, Pattern Recognition, Robotics and Automation

Poi-Ngee Tan received his B. Eng. (Hons) Electronics majoring in Telecommunications from Faculty of Engineering and Technology, Multimedia University, Malaysia in 2009. In the same year, he joined Unitele Multimedia Sdn. Bhd, which is wholly owned by Multimedia University as research officer. His research mainly focuses on thermal infrared imaging and also antenna design for synthethic aperture radar. His research interests include thermal imaging, image analysis, as well as antenna fabrication.