classification of human epithelial type 2 cell ... - Semantic Scholar
CLASSIFICATION OF HUMAN EPITHELIAL TYPE 2 CELL IMAGES USING INDEPENDENT COMPONENT ANALYSIS Yan Yang1,2 , Arnold Wiliem1 , Azadeh Alavi1,2 , Peter Hobson3 1
The University of Queensland, School of ITEE, QLD 4072, Australia NICTA Queensland Research Laboratory, Australia, 3 Sullivan Nicolaides Pathology, Australia SNPHEp-2
2
ABSTRACT
ICPRContest
Identifying the presence of Anti-Nuclear Antibody in Human Epithelial type 2 (HEp-2) cells via Indirect Immunofluorescence (IIF) is commonly used to diagnose various connective tissue diseases in clinical pathology tests. This pathology test can be automated by computer vision algorithms. However, the existing automated systems, namely Computer Aided Diagnostic (CAD) systems, suffer from numerous shortcomings such as using pre-selected features. To overcome such shortcomings, we propose a novel approach by learning filters from image statistics. Specifically, we train a filter bank from unlabelled cell images by using Independent Component Analysis (ICA). The filter bank is then applied to images in order to extract a set of filter responses. We extract regions from this set of responses and stack them into “cubic regions”. Average filter responses in 1 × 1, 2 × 2, 4 × 4 grids from the cubic-region are used as “ICA feature”. ICA features in multiple regions are stored in a feature collection matrix to represent each image. Finally, we use Support Vector Machine (SVM) in conjunction with histogram correlation kernel to classify the cell images. We show that our approach outperforms three recently proposed CAD systems on two publicly available datasets: ICPR HEp-2 contest and SNPHEp-2.
Homogeneous
Fine speckled
Nucleolar
Centromere Cytoplasmic
Fig. 1. Example images of HEp-2 cells from SNPHEp-2 and ICPRContest dataset. SNPHEp-2 dataset does not include Cytoplasmic cells. such as standard deviation, entropy, spectral measurements (e.g. Fourier and wavelet transforms), descriptors for shape and structure, morphological features, and the combination of above have been used as discriminative features [2, 3, 5, 8]. However, it has been shown that most of the current CAD systems using naive handpicked features are not robust enough to operate under different laboratory environments [8]. In realistic cases, they produce good results in one laboratory using specific assays and microscopes, but may not on another laboratory’s hardware. A more promising approach is to use codebook to build probability model for input feature. Then classification of cell image is based on the “term frequency” of single or combined features [8]. Despite of the improvement, codebook approach of certain features still needs to use specific pre-selected patch-level feature (e.g. Discrete Cosine Transform (DCT), LBP descriptors). Thus it only works when one knows what the most optimised set of features to use. One popular approach for image classification in recent year is to apply unsupervised feature learning on natural images, which is inspired by Olshausen’s discovery on the properties of primary visual cortex [9]. Olshausen’s work shows that localised, oriented and bandpass filters are similar to the receptive fields in primary visual cortex. These filters can be trained by maximizing the sparseness of the response in some linear superposition while sources are separated as independent components, namely Independent Component Analysis (ICA). ICA filters exhibit sparse activities similar to V 1 neurons, and these responses can be used as image feature. One popular use of ICA filter is to measure the change of filter ac-
Index Terms— HEp-2 cells classification, Independent Component Analysis, Indirect Immunofluorescence, AntiNuclear Antibodies Pathology Test, Computer Aided Diagnostic 1. INTRODUCTION Computer Aided Diagnostic (CAD) systems have been rapidly developed in recent years for various clinical pathology tests on Anti-Nuclear Antibody (ANA) via Indirect Immunofluorescence (IIF) [1, 2, 3, 4, 5, 6, 7, 8]. These systems help to improve the efficiency of previously labour intensive diagnosis process, as well as providing less subjective judgement on determining HEp-2 patterns, as shown in Fig 1. The existing CAD systems mostly follow the trend of using handpicked features [8]. Local Binary Pattern (LBP) is a common choice to describe cell texture [1, 7]. Other features
978-1-4799-2341-0/13/$31.00 ©2013 IEEE
Coarse speckled
733
ICIP 2013
tivities for saliency detection [10]. It is also possible to apply ICA for object classification and it has been shown that ICA saliency map and ICA filter approach can outperform traditional systems based on SIFT features [11]. In this paper, we show that applying ICA filters for the cell image classification domain leads to higher accuracy. A related approach in object classification is proposed by Kanan and Cottrell [11]. They use saliency map generated by ICA filters, then extract multiple regions by simulating human eye sweeping movement around salient region. Features in these regions are classified by Nearest Neighbour Kernel Density Estimation. We note the difference between our approach and Kanan and Cottrell as: 1) ICA filter responses are directly used in our approach, rather than the saliency map derived from the responses. The reason is: cell images are all properly cropped and centered, which makes the saliency map nonimportant for classification. 2) our method focuses on the correlation between regions, which is more meaningful than searching only the nearest neighbour of salient patches. Contributions: We propose to learn ICA filters for challenging HEp-2 cell classification. We apply convolution of each ICA filter and the input image. The responses of learned filters are used as features. Thus, we avoid to use any traditional “handpicked” features. We also propose a novel classification approach based on features in multiple cubic regions from a stack of filter responses, histogram correlation kernel and an SVM classifier. Experiment evaluation shows that our approach is more robust to different microscope settings, as proposed system outperforms three recently proposed CAD systems on two HEp-2 cell datasets taken under different laboratory conditions.
Fig. 2. Filters learned via Independent Component Analysis (ICA) for HEp-2 Cell images [w1 , . . . , wd ], such that the response vector s = W T I(x, y) is sparse (i.e. s = [s1 , . . . , sd ]T has a small number of nonzero values). To learn ICA filters W , HEp-2 cell images from training set are processed without using the class labels. For each image, K small image patches with size b × b are extracted from random locations, and stored as column vectors in patch collection matrix IˆK . Each patch vector is normalised to zero-mean and unit-length. We use Principal Component Analysis (PCA) on the patch collection matrix IˆK and reduce its dimensionality from b × b to d. Then, we apply FastICA to patch collection matrix to train d filters. This produces a linear transformation matrix W representing collected patches as their statistically independent components. W is also called “ICA filters” or “filter bank”. The filters learned from HEp-2 cells appear to be more similar to the shape of Gaussian/Laplacian Filters, as shown in Fig. 2. It is visually distinct from the common ICA filter bank whose shape is similar to the Gabor filter [9, 10, 11, 12]. The reason is that most natural images contain many edges and rigid corners, while cell images contain mostly speckled or bright spots. We note that ICA filter learning is different from the “codebook” or “Bag-of-Words” approach, as the latter does not constraint the individual atoms to be statistically independent.
2. PROPOSED METHOD Proposed approach contains three steps: 1) We use unlabelled training images of all classes to learn ICA filters. 2) Learned ICA filters are used to extract filter responses from sub-regions of cell image. 3) We apply histogram correlation kernel SVM to classify cell images based on extracted regional features obtained from learned ICA filter.
2.2. Feature Extraction We propose a region-based feature extraction process as illustrated in Fig 3. First, learned ICA filters are applied through convoluting the input image, producing d layers of filtered responses for all pixels (Fig 3(a)). Sub-regions of size q × q at fixated locations are extracted from all responses by densely scanning the frame 2 . These sub-regions are stacked together and represented as “cubic-regions”: volumes of q × q × d filter responses (as shown in Fig 3(b)). We use a three-level spatial pyramid pooling to divide each sub-region into 1 × 1, 2 × 2 and 4 × 4 grids. Therefore the stack of filter responses are reduced to the length of D = (1 + 4 + 16) × d = 21d (Fig 3(c)). Then we compute the average of filter response ht,j in T cubic regions (t = 1, . . . , T ) at each layer of the
2.1. Learning Filter Bank We employ a standard ICA learning algorithm: FastICA 1 for independent component analysis [12]. The generative ICA model of d bases is: Xd I(x, y) = ai si (x, y) (1) i=1
where I(x, y) is the pixel intensity of a small image patch at location (x, y), si is the weight corresponding to individual basis component ai . The goal of ICA is to find bases A = [a1 , . . . , ad ] and the corresponding ICA filters W = 1 http://research.ics.aalto.fi/ica/fastica/
2 We
734
note that the “sub-region” size q is not related to the filter size b.
d filters
... level0
level1
... d (a) Apply filters (b) Collect cubic regions
level2
...
D = (1+4+16) x d
(c) Spatial pooling
d
H=
[
Level0 d
h1,1 ... h1,d h2,1 ... h2,d h T,1 ... hT,d
Level1 4d
Level2 16d
h1,d+1 ... h1,5d h1,5d+1 ... h1,21d h2,d+1 ... h2,5d h2,5d+1 ... h2,21d ... hT,d+1 ... hT,5d hT,5d+1 ... h T,21d
(d) Feature collection matrix
Fig. 3. Feature extraction of sub-regions at fixation locations from filter responses stack on cell images. stack (j = 1, . . . , D) according to the grid cell. A set of scaling parameter α = {α0 , α1 , α2 } can be used if the coarser grids contain much noise, i.e. αl at higher grid level l is assigned a higher value in order to emphasize matches found in finer region [13]. The average of filter responses by pyramid grids are concatenated as row vector, and stored one row per region (Fig 3(d)). Overall, we get a feature matrix with T rows and D columns to present each input image. The advantages of proposed feature are: 1) filter responses can be regarded as a histogram-like feature, where histogram bins are probabilities of the filter activities average across the image. Thus, it tolerates local transformations; 2) using multiple “cubic-regions” from the stack of filter responses make our approach more resistant to shift, rotation and scale variation in cell images. Basically, if more cubic-regions are similar between two cell images, it is more likely that they belong to the same class.
3. EXPERIMENTS We first introduce the two HEp-2 cell datasets used in our experiments, and the details on the parameters settings of these datasets. Then, the proposed system is evaluated in different filter setting, classifier, and contrasted with three recently proposed systems. Our implementation is using OpenCV C++ library 3 and LIBSVM tools 4 . 3.1. Datasets and Parameters Two datasets are used in our experiments: 1) SNP HEp-2 Cell dataset (SNPHEp-2) was obtained at Sullivan Nicolaides Pathology laboratory, Australia [8]. It has 1884 cell images of five patterns: homogeneous, nucleolar, centromere, coarse speckled, fine speckled (as shown in Fig 1). The dataset is divided into training and testing sets, which contain 905 and 979 images, respectively. We follow the test protocol and validation sets used in [8], each validation fold contains around 900 images with 450 images for both training and testing. 2) ICPR HEp-2 Cell classification Contest dataset (ICPRContest) contains 1457 cells images [1]. There are six patterns in this dataset: centromere, coarse speckled, cytoplasmic, fine speckled, homogeneous and nucleolar. There are 721 images in training set and 734 images in testing set for ICPRContest. Only one-fold test for ICPRContest dataset is used in order to follow the test protocol. The cell images in these two dataset were captured by different microscope configurations. For instance, SNPHEp-2 used objective lens magnitude 20x, while ICPRContest used 40x. Although hand labelled image masks are available for both datasets, we need not to use these masks in our proposed system. We train two sets of ICA filters for SNPHEp-2 and ICPRContest datasets separately, from the given training images. A median filter is applied on ICPRContest images to reduce its noise. We learn ICA filters of size b = 16, and use PCA to reduce the feature dimension from 16 × 16 to around 128. As the images in SNPHEp-2 were taken using a high dynamic range cooled microscopy camera, the noise is very minimal. As such, scaling parameter on grid
2.3. Classification We use Kernel Support Vector Machine (KSVM) as the classifier. Correlation kernel based on Pearson correlation (Eqn. 2) is used in our SVM classifier: ¯ 1 )(H 2 (i) − H ¯ 2) −H ¯ 2 )2 ¯ 2 PN (H 2 (i) − H i=1 (H 1 (i) − H 1 ) i=1 (2) PN
K(H 1 , H 2 ) = qP N
i=1 (H 1 (i)
where H 1 and H 2 represent the feature collection matrices ¯ is the sample of image 1 and image 2 of size N = T × D, H mean of histogram bins. The correlation matrix is always symmetric and positive semidefinite [14], thus it is a Mercer kernel. Correlation Kernel SVM (CKSVM) is different from Nearest Neighbour Kernel Density Estimation (NNKDE) which has been recently applied to classify a similar multiple fixation region features [11]. The advantages of CKSVM are: 1) CKSVM does not need to assume that extracted regions are the statistically independent as NNKDE; 2) Similarity measure based on overall correlation between sub-region is less sensitive to the “outliers” than the nearest neighbour metric.
3 http://opencv.willowgarage.com/ 4 http://www.csie.ntu.edu.tw/
735
˜cjlin/libsvm/
]
124
126
128
132
SNPHEp-2
78.6%
82.1%
79.9%
78.7%
ICPRContest
55.4%
55.8%
58.6%
55.2%
Proposed
Accuracy (%)
Num of Filters
Table 1. Classification accuracy of proposed system using different number of filters. Classifier
NNKDE
CKSVM (proposed)
SNPHEp-2
60.3%
82.1%
ICPRContest
45.8%
58.6%
Wiliem
Strandmark
Cordelli
80 70 60 50 40
ICPRContest
SNPHEp-2
Fig. 4. Correct classification rate of proposed system comparing to three recent proposed systems: Wiliem [8], Strandmark [3] and Cordelli [1]. SNPHEp-2 dataset and ICPRContest 5 . The system employs various image statistics features (i.e. mean and standard deviation) and morphological features (e.g. area). We denote this system as “Strandmark”. We implemented the best reported descriptor in Cordelli et al. [1], which is a combination of image feature as LBP, image energy and entropy on both dataset. This system is denoted as “Cordelli”. Performance of these three systems and proposed approach are compared side-by-side on Fig 4. On ICPRContest dataset, our system outperforms Strandmark by 10.6 percentage points and Cordelli by 8.6 percentage points. On SNPHEp-2 dataset, we outperform both systems by an even larger margin: 24.1 percentage points more than Strandmark and 40.6 percentage points more than Cordelli. This indicates that proposed feature is more robust to different laboratory equipment, while the traditional features often fail to obtain consistent performance when the hardware settings change. Wiliem’s codebook approach has successfully handled the disadvantage in traditional feature by applying a probability model using codebook, but is less accurate than our approach: 80.4% v.s. 82.1% on SNPHEp-2 and 55.0% v.s. 58.6% on ICPRContest.
Table 2. Classification accuracy of Correlation Kernel SVM (CKSVM) classifier comparing to Nearest Neighbour Kernel Density Estimation (NNKDE) in [11]. We apply optimised filter set for both classifier (128 components for ICPRContest, and 126 components for SNPHEp-2). level during spatial pooling has little effect. While using α = {0.25, 0.25, 0.5} to weight grid levels can help to improve the accuracy on ICPRContest dataset [13]. We note that above hyperparameters were found in the training set. 3.2. Result Evaluation and Comparison Proposed system is evaluated when various number of filters are used. It is fairly simple to determine the optimal number of filters as the number of learned ICA filters is restricted by the patch size b. For example, 16 × 16 patch would obtain maximum 256 filters. However not all filters are useful because image data always contains noise. Hyv¨arinen [12] has suggested that feature dimension should be reduced to at least 70% of its original size before learning ICA. We found that, for cell images, feature dimension should be reduced to approximately 50% of it’s original size, as shown in Table 1. Proposed system obtains the best performance when reduce the original data dimension to 128 for ICPRContest and 126 for SNPHEp-2 . Next, we compare Correlation Kernel SVM (CKSVM) classifier against Nearest Neighbour Kernel Density Estimation (NNKDE) [11]. Proposed CKSVM outperforms NNKDE as shown in Table 2. We argue that correlation kernel is more suitable for classifying features in multipleregions than the nearest neighbour approach. It is because that input images are more likely in the same class if they have more correlated regions. Thus, it is better than NNKDE which searches for two regions that are the most similar. Finally, our system is contrasted with three recently proposed systems in Wiliem et al. [8], Strandmark et al. [3] and Cordelli et al. [1]. Wiliem et al. [8] apply a codebook approach using DCT features, we use authors’ code to produce result on both SNPHEp-2 and ICPRContest and denote their system as “Wiliem”. We also used the code provided by authors from Strandmark et al. [3] to produce result on
4. CONCLUSIONS In this paper, we proposed a novel approach for HEp-2 cell image classification via features generated by ICA filters learned from the statistic in the data. We show this approach is more robust than traditional CAD systems (mostly are using handpicked features) on processing images taken from different laboratory equipment. The proposed approach is contrasted with three recently proposed methods on two challenging HEp-2 cell classification datasets. Results show that our learned ICA filters produce robust features and consistently outperform recently proposed CAD systems on those datasets. 5. REFERENCES [1] E. Cordelli and P. Soda, “Color to grayscale staining pattern representation in iif,” in International Symposium on 5 We apply ICPR2012 HEp-2 cell classification contest evaluation protcol to produce results on ICPRContest dataset, which is different from the protocol adopted in [1, 8, 3]
736
Computer-Based Medical Systems (CBMS), 2011, pp. 1–6. [2] P. Elbischger, S. Geerts, K. Sander, G. Ziervogel-Lukas, and P. Sinah, “Algorithmic framework for hep-2 fluorescence pattern classification to aid auto-immune diseases diagnosis,” in Biomedical Imaging: From Nano to Macro, 2009, pp. 562– 565. [3] P. Strandmark, J. Ul´en, and F. Kahl, “Hep-2 staining pattern classification,” in International Conference Pattern Recognition, 2012, pp. 562–565. [4] R. Hiemann, T. B¨uttner, T. Krieger, D. Roggenbuck, U. Sack, and K. Conrad, “Challenges of automated screening and differentiation of non-organ specific autoantibodies on hep-2 cells,” Autoimmunity Reviews, vol. 9, no. 1, pp. 17–22, 2009. [5] T. Hsieh, Y. Huang, C. Chung, and Y. Huang, “Hep-2 cell classification in indirect immunofluorescence images,” in International Conference on Information, Communications and Signal Processing, 2009. [6] P. Soda and G. Iannello, “Aggregation of classifiers for staining pattern recognition in antinuclear autoantibodies analysis,” IEEE Transactions on Information Technology in Biomedicine, vol. 13, no. 3, pp. 322–329, 2009. [7] I. Theodorakopoulos, D. Kastaniotis, G. Economou, and S. Fotopoulos, “Hep-2 cells classification via fusion of morphological and textural features,” in IEEE 12th International Conference on Bioinformatics & Bioengineering, 2012, pp. 689–694. [8] A. Wiliem, Y. Wong, C. Sanderson, P. Hobson, S. Chen, and B. Lovell, “Classification of human epithelial type 2 cell indirect immunofluoresence images via codebook based descriptors,” in IEEE Workshop on Applications of Computer Vision (WACV), January 2013. [9] B.A. Olshausen et al., “Emergence of simple-cell receptive field properties by learning a sparse code for natural images,” Nature, vol. 381, no. 6583, pp. 607–609, 1996. [10] X. Hou and L. Zhang, “Dynamic visual attention: Searching for coding length increments,” Advances in neural information processing systems, vol. 21, pp. 681–688, 2008. [11] C. Kanan and G. Cottrell, “Robust classification of objects, faces, and flowers using natural image statistics,” in IEEE Conference on Computer Vision and Pattern Recognition (CVPR),, 2010, pp. 2472–2479. [12] A. Hyv¨arinen, J. Hurri, and P.O. Hoyer, Natural Image Statistics: A Probabilistic Approach to Early Computational Vision., vol. 39, Springer, 2009. [13] S. Lazebnik, C. Schmid, and J. Ponce, “Beyond bags of features: Spatial pyramid matching for recognizing natural scene categories,” in IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2006, pp. 2169–2178. [14] H. Jiang and W.K. Ching, “Correlation kernels for support vector machines classification with applications in cancer data,” Computational and Mathematical Methods in Medicine, 2012.
737
The University of Queensland, School of ITEE, QLD 4072, Australia NICTA Queensland Research Laboratory, Australia, 3 Sullivan Nicolaides Pathology, Australia SNPHEp-2
2
ABSTRACT
ICPRContest
Identifying the presence of Anti-Nuclear Antibody in Human Epithelial type 2 (HEp-2) cells via Indirect Immunofluorescence (IIF) is commonly used to diagnose various connective tissue diseases in clinical pathology tests. This pathology test can be automated by computer vision algorithms. However, the existing automated systems, namely Computer Aided Diagnostic (CAD) systems, suffer from numerous shortcomings such as using pre-selected features. To overcome such shortcomings, we propose a novel approach by learning filters from image statistics. Specifically, we train a filter bank from unlabelled cell images by using Independent Component Analysis (ICA). The filter bank is then applied to images in order to extract a set of filter responses. We extract regions from this set of responses and stack them into “cubic regions”. Average filter responses in 1 × 1, 2 × 2, 4 × 4 grids from the cubic-region are used as “ICA feature”. ICA features in multiple regions are stored in a feature collection matrix to represent each image. Finally, we use Support Vector Machine (SVM) in conjunction with histogram correlation kernel to classify the cell images. We show that our approach outperforms three recently proposed CAD systems on two publicly available datasets: ICPR HEp-2 contest and SNPHEp-2.
Homogeneous
Fine speckled
Nucleolar
Centromere Cytoplasmic
Fig. 1. Example images of HEp-2 cells from SNPHEp-2 and ICPRContest dataset. SNPHEp-2 dataset does not include Cytoplasmic cells. such as standard deviation, entropy, spectral measurements (e.g. Fourier and wavelet transforms), descriptors for shape and structure, morphological features, and the combination of above have been used as discriminative features [2, 3, 5, 8]. However, it has been shown that most of the current CAD systems using naive handpicked features are not robust enough to operate under different laboratory environments [8]. In realistic cases, they produce good results in one laboratory using specific assays and microscopes, but may not on another laboratory’s hardware. A more promising approach is to use codebook to build probability model for input feature. Then classification of cell image is based on the “term frequency” of single or combined features [8]. Despite of the improvement, codebook approach of certain features still needs to use specific pre-selected patch-level feature (e.g. Discrete Cosine Transform (DCT), LBP descriptors). Thus it only works when one knows what the most optimised set of features to use. One popular approach for image classification in recent year is to apply unsupervised feature learning on natural images, which is inspired by Olshausen’s discovery on the properties of primary visual cortex [9]. Olshausen’s work shows that localised, oriented and bandpass filters are similar to the receptive fields in primary visual cortex. These filters can be trained by maximizing the sparseness of the response in some linear superposition while sources are separated as independent components, namely Independent Component Analysis (ICA). ICA filters exhibit sparse activities similar to V 1 neurons, and these responses can be used as image feature. One popular use of ICA filter is to measure the change of filter ac-
Index Terms— HEp-2 cells classification, Independent Component Analysis, Indirect Immunofluorescence, AntiNuclear Antibodies Pathology Test, Computer Aided Diagnostic 1. INTRODUCTION Computer Aided Diagnostic (CAD) systems have been rapidly developed in recent years for various clinical pathology tests on Anti-Nuclear Antibody (ANA) via Indirect Immunofluorescence (IIF) [1, 2, 3, 4, 5, 6, 7, 8]. These systems help to improve the efficiency of previously labour intensive diagnosis process, as well as providing less subjective judgement on determining HEp-2 patterns, as shown in Fig 1. The existing CAD systems mostly follow the trend of using handpicked features [8]. Local Binary Pattern (LBP) is a common choice to describe cell texture [1, 7]. Other features
978-1-4799-2341-0/13/$31.00 ©2013 IEEE
Coarse speckled
733
ICIP 2013
tivities for saliency detection [10]. It is also possible to apply ICA for object classification and it has been shown that ICA saliency map and ICA filter approach can outperform traditional systems based on SIFT features [11]. In this paper, we show that applying ICA filters for the cell image classification domain leads to higher accuracy. A related approach in object classification is proposed by Kanan and Cottrell [11]. They use saliency map generated by ICA filters, then extract multiple regions by simulating human eye sweeping movement around salient region. Features in these regions are classified by Nearest Neighbour Kernel Density Estimation. We note the difference between our approach and Kanan and Cottrell as: 1) ICA filter responses are directly used in our approach, rather than the saliency map derived from the responses. The reason is: cell images are all properly cropped and centered, which makes the saliency map nonimportant for classification. 2) our method focuses on the correlation between regions, which is more meaningful than searching only the nearest neighbour of salient patches. Contributions: We propose to learn ICA filters for challenging HEp-2 cell classification. We apply convolution of each ICA filter and the input image. The responses of learned filters are used as features. Thus, we avoid to use any traditional “handpicked” features. We also propose a novel classification approach based on features in multiple cubic regions from a stack of filter responses, histogram correlation kernel and an SVM classifier. Experiment evaluation shows that our approach is more robust to different microscope settings, as proposed system outperforms three recently proposed CAD systems on two HEp-2 cell datasets taken under different laboratory conditions.
Fig. 2. Filters learned via Independent Component Analysis (ICA) for HEp-2 Cell images [w1 , . . . , wd ], such that the response vector s = W T I(x, y) is sparse (i.e. s = [s1 , . . . , sd ]T has a small number of nonzero values). To learn ICA filters W , HEp-2 cell images from training set are processed without using the class labels. For each image, K small image patches with size b × b are extracted from random locations, and stored as column vectors in patch collection matrix IˆK . Each patch vector is normalised to zero-mean and unit-length. We use Principal Component Analysis (PCA) on the patch collection matrix IˆK and reduce its dimensionality from b × b to d. Then, we apply FastICA to patch collection matrix to train d filters. This produces a linear transformation matrix W representing collected patches as their statistically independent components. W is also called “ICA filters” or “filter bank”. The filters learned from HEp-2 cells appear to be more similar to the shape of Gaussian/Laplacian Filters, as shown in Fig. 2. It is visually distinct from the common ICA filter bank whose shape is similar to the Gabor filter [9, 10, 11, 12]. The reason is that most natural images contain many edges and rigid corners, while cell images contain mostly speckled or bright spots. We note that ICA filter learning is different from the “codebook” or “Bag-of-Words” approach, as the latter does not constraint the individual atoms to be statistically independent.
2. PROPOSED METHOD Proposed approach contains three steps: 1) We use unlabelled training images of all classes to learn ICA filters. 2) Learned ICA filters are used to extract filter responses from sub-regions of cell image. 3) We apply histogram correlation kernel SVM to classify cell images based on extracted regional features obtained from learned ICA filter.
2.2. Feature Extraction We propose a region-based feature extraction process as illustrated in Fig 3. First, learned ICA filters are applied through convoluting the input image, producing d layers of filtered responses for all pixels (Fig 3(a)). Sub-regions of size q × q at fixated locations are extracted from all responses by densely scanning the frame 2 . These sub-regions are stacked together and represented as “cubic-regions”: volumes of q × q × d filter responses (as shown in Fig 3(b)). We use a three-level spatial pyramid pooling to divide each sub-region into 1 × 1, 2 × 2 and 4 × 4 grids. Therefore the stack of filter responses are reduced to the length of D = (1 + 4 + 16) × d = 21d (Fig 3(c)). Then we compute the average of filter response ht,j in T cubic regions (t = 1, . . . , T ) at each layer of the
2.1. Learning Filter Bank We employ a standard ICA learning algorithm: FastICA 1 for independent component analysis [12]. The generative ICA model of d bases is: Xd I(x, y) = ai si (x, y) (1) i=1
where I(x, y) is the pixel intensity of a small image patch at location (x, y), si is the weight corresponding to individual basis component ai . The goal of ICA is to find bases A = [a1 , . . . , ad ] and the corresponding ICA filters W = 1 http://research.ics.aalto.fi/ica/fastica/
2 We
734
note that the “sub-region” size q is not related to the filter size b.
d filters
... level0
level1
... d (a) Apply filters (b) Collect cubic regions
level2
...
D = (1+4+16) x d
(c) Spatial pooling
d
H=
[
Level0 d
h1,1 ... h1,d h2,1 ... h2,d h T,1 ... hT,d
Level1 4d
Level2 16d
h1,d+1 ... h1,5d h1,5d+1 ... h1,21d h2,d+1 ... h2,5d h2,5d+1 ... h2,21d ... hT,d+1 ... hT,5d hT,5d+1 ... h T,21d
(d) Feature collection matrix
Fig. 3. Feature extraction of sub-regions at fixation locations from filter responses stack on cell images. stack (j = 1, . . . , D) according to the grid cell. A set of scaling parameter α = {α0 , α1 , α2 } can be used if the coarser grids contain much noise, i.e. αl at higher grid level l is assigned a higher value in order to emphasize matches found in finer region [13]. The average of filter responses by pyramid grids are concatenated as row vector, and stored one row per region (Fig 3(d)). Overall, we get a feature matrix with T rows and D columns to present each input image. The advantages of proposed feature are: 1) filter responses can be regarded as a histogram-like feature, where histogram bins are probabilities of the filter activities average across the image. Thus, it tolerates local transformations; 2) using multiple “cubic-regions” from the stack of filter responses make our approach more resistant to shift, rotation and scale variation in cell images. Basically, if more cubic-regions are similar between two cell images, it is more likely that they belong to the same class.
3. EXPERIMENTS We first introduce the two HEp-2 cell datasets used in our experiments, and the details on the parameters settings of these datasets. Then, the proposed system is evaluated in different filter setting, classifier, and contrasted with three recently proposed systems. Our implementation is using OpenCV C++ library 3 and LIBSVM tools 4 . 3.1. Datasets and Parameters Two datasets are used in our experiments: 1) SNP HEp-2 Cell dataset (SNPHEp-2) was obtained at Sullivan Nicolaides Pathology laboratory, Australia [8]. It has 1884 cell images of five patterns: homogeneous, nucleolar, centromere, coarse speckled, fine speckled (as shown in Fig 1). The dataset is divided into training and testing sets, which contain 905 and 979 images, respectively. We follow the test protocol and validation sets used in [8], each validation fold contains around 900 images with 450 images for both training and testing. 2) ICPR HEp-2 Cell classification Contest dataset (ICPRContest) contains 1457 cells images [1]. There are six patterns in this dataset: centromere, coarse speckled, cytoplasmic, fine speckled, homogeneous and nucleolar. There are 721 images in training set and 734 images in testing set for ICPRContest. Only one-fold test for ICPRContest dataset is used in order to follow the test protocol. The cell images in these two dataset were captured by different microscope configurations. For instance, SNPHEp-2 used objective lens magnitude 20x, while ICPRContest used 40x. Although hand labelled image masks are available for both datasets, we need not to use these masks in our proposed system. We train two sets of ICA filters for SNPHEp-2 and ICPRContest datasets separately, from the given training images. A median filter is applied on ICPRContest images to reduce its noise. We learn ICA filters of size b = 16, and use PCA to reduce the feature dimension from 16 × 16 to around 128. As the images in SNPHEp-2 were taken using a high dynamic range cooled microscopy camera, the noise is very minimal. As such, scaling parameter on grid
2.3. Classification We use Kernel Support Vector Machine (KSVM) as the classifier. Correlation kernel based on Pearson correlation (Eqn. 2) is used in our SVM classifier: ¯ 1 )(H 2 (i) − H ¯ 2) −H ¯ 2 )2 ¯ 2 PN (H 2 (i) − H i=1 (H 1 (i) − H 1 ) i=1 (2) PN
K(H 1 , H 2 ) = qP N
i=1 (H 1 (i)
where H 1 and H 2 represent the feature collection matrices ¯ is the sample of image 1 and image 2 of size N = T × D, H mean of histogram bins. The correlation matrix is always symmetric and positive semidefinite [14], thus it is a Mercer kernel. Correlation Kernel SVM (CKSVM) is different from Nearest Neighbour Kernel Density Estimation (NNKDE) which has been recently applied to classify a similar multiple fixation region features [11]. The advantages of CKSVM are: 1) CKSVM does not need to assume that extracted regions are the statistically independent as NNKDE; 2) Similarity measure based on overall correlation between sub-region is less sensitive to the “outliers” than the nearest neighbour metric.
3 http://opencv.willowgarage.com/ 4 http://www.csie.ntu.edu.tw/
735
˜cjlin/libsvm/
]
124
126
128
132
SNPHEp-2
78.6%
82.1%
79.9%
78.7%
ICPRContest
55.4%
55.8%
58.6%
55.2%
Proposed
Accuracy (%)
Num of Filters
Table 1. Classification accuracy of proposed system using different number of filters. Classifier
NNKDE
CKSVM (proposed)
SNPHEp-2
60.3%
82.1%
ICPRContest
45.8%
58.6%
Wiliem
Strandmark
Cordelli
80 70 60 50 40
ICPRContest
SNPHEp-2
Fig. 4. Correct classification rate of proposed system comparing to three recent proposed systems: Wiliem [8], Strandmark [3] and Cordelli [1]. SNPHEp-2 dataset and ICPRContest 5 . The system employs various image statistics features (i.e. mean and standard deviation) and morphological features (e.g. area). We denote this system as “Strandmark”. We implemented the best reported descriptor in Cordelli et al. [1], which is a combination of image feature as LBP, image energy and entropy on both dataset. This system is denoted as “Cordelli”. Performance of these three systems and proposed approach are compared side-by-side on Fig 4. On ICPRContest dataset, our system outperforms Strandmark by 10.6 percentage points and Cordelli by 8.6 percentage points. On SNPHEp-2 dataset, we outperform both systems by an even larger margin: 24.1 percentage points more than Strandmark and 40.6 percentage points more than Cordelli. This indicates that proposed feature is more robust to different laboratory equipment, while the traditional features often fail to obtain consistent performance when the hardware settings change. Wiliem’s codebook approach has successfully handled the disadvantage in traditional feature by applying a probability model using codebook, but is less accurate than our approach: 80.4% v.s. 82.1% on SNPHEp-2 and 55.0% v.s. 58.6% on ICPRContest.
Table 2. Classification accuracy of Correlation Kernel SVM (CKSVM) classifier comparing to Nearest Neighbour Kernel Density Estimation (NNKDE) in [11]. We apply optimised filter set for both classifier (128 components for ICPRContest, and 126 components for SNPHEp-2). level during spatial pooling has little effect. While using α = {0.25, 0.25, 0.5} to weight grid levels can help to improve the accuracy on ICPRContest dataset [13]. We note that above hyperparameters were found in the training set. 3.2. Result Evaluation and Comparison Proposed system is evaluated when various number of filters are used. It is fairly simple to determine the optimal number of filters as the number of learned ICA filters is restricted by the patch size b. For example, 16 × 16 patch would obtain maximum 256 filters. However not all filters are useful because image data always contains noise. Hyv¨arinen [12] has suggested that feature dimension should be reduced to at least 70% of its original size before learning ICA. We found that, for cell images, feature dimension should be reduced to approximately 50% of it’s original size, as shown in Table 1. Proposed system obtains the best performance when reduce the original data dimension to 128 for ICPRContest and 126 for SNPHEp-2 . Next, we compare Correlation Kernel SVM (CKSVM) classifier against Nearest Neighbour Kernel Density Estimation (NNKDE) [11]. Proposed CKSVM outperforms NNKDE as shown in Table 2. We argue that correlation kernel is more suitable for classifying features in multipleregions than the nearest neighbour approach. It is because that input images are more likely in the same class if they have more correlated regions. Thus, it is better than NNKDE which searches for two regions that are the most similar. Finally, our system is contrasted with three recently proposed systems in Wiliem et al. [8], Strandmark et al. [3] and Cordelli et al. [1]. Wiliem et al. [8] apply a codebook approach using DCT features, we use authors’ code to produce result on both SNPHEp-2 and ICPRContest and denote their system as “Wiliem”. We also used the code provided by authors from Strandmark et al. [3] to produce result on
4. CONCLUSIONS In this paper, we proposed a novel approach for HEp-2 cell image classification via features generated by ICA filters learned from the statistic in the data. We show this approach is more robust than traditional CAD systems (mostly are using handpicked features) on processing images taken from different laboratory equipment. The proposed approach is contrasted with three recently proposed methods on two challenging HEp-2 cell classification datasets. Results show that our learned ICA filters produce robust features and consistently outperform recently proposed CAD systems on those datasets. 5. REFERENCES [1] E. Cordelli and P. Soda, “Color to grayscale staining pattern representation in iif,” in International Symposium on 5 We apply ICPR2012 HEp-2 cell classification contest evaluation protcol to produce results on ICPRContest dataset, which is different from the protocol adopted in [1, 8, 3]
736
Computer-Based Medical Systems (CBMS), 2011, pp. 1–6. [2] P. Elbischger, S. Geerts, K. Sander, G. Ziervogel-Lukas, and P. Sinah, “Algorithmic framework for hep-2 fluorescence pattern classification to aid auto-immune diseases diagnosis,” in Biomedical Imaging: From Nano to Macro, 2009, pp. 562– 565. [3] P. Strandmark, J. Ul´en, and F. Kahl, “Hep-2 staining pattern classification,” in International Conference Pattern Recognition, 2012, pp. 562–565. [4] R. Hiemann, T. B¨uttner, T. Krieger, D. Roggenbuck, U. Sack, and K. Conrad, “Challenges of automated screening and differentiation of non-organ specific autoantibodies on hep-2 cells,” Autoimmunity Reviews, vol. 9, no. 1, pp. 17–22, 2009. [5] T. Hsieh, Y. Huang, C. Chung, and Y. Huang, “Hep-2 cell classification in indirect immunofluorescence images,” in International Conference on Information, Communications and Signal Processing, 2009. [6] P. Soda and G. Iannello, “Aggregation of classifiers for staining pattern recognition in antinuclear autoantibodies analysis,” IEEE Transactions on Information Technology in Biomedicine, vol. 13, no. 3, pp. 322–329, 2009. [7] I. Theodorakopoulos, D. Kastaniotis, G. Economou, and S. Fotopoulos, “Hep-2 cells classification via fusion of morphological and textural features,” in IEEE 12th International Conference on Bioinformatics & Bioengineering, 2012, pp. 689–694. [8] A. Wiliem, Y. Wong, C. Sanderson, P. Hobson, S. Chen, and B. Lovell, “Classification of human epithelial type 2 cell indirect immunofluoresence images via codebook based descriptors,” in IEEE Workshop on Applications of Computer Vision (WACV), January 2013. [9] B.A. Olshausen et al., “Emergence of simple-cell receptive field properties by learning a sparse code for natural images,” Nature, vol. 381, no. 6583, pp. 607–609, 1996. [10] X. Hou and L. Zhang, “Dynamic visual attention: Searching for coding length increments,” Advances in neural information processing systems, vol. 21, pp. 681–688, 2008. [11] C. Kanan and G. Cottrell, “Robust classification of objects, faces, and flowers using natural image statistics,” in IEEE Conference on Computer Vision and Pattern Recognition (CVPR),, 2010, pp. 2472–2479. [12] A. Hyv¨arinen, J. Hurri, and P.O. Hoyer, Natural Image Statistics: A Probabilistic Approach to Early Computational Vision., vol. 39, Springer, 2009. [13] S. Lazebnik, C. Schmid, and J. Ponce, “Beyond bags of features: Spatial pyramid matching for recognizing natural scene categories,” in IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2006, pp. 2169–2178. [14] H. Jiang and W.K. Ching, “Correlation kernels for support vector machines classification with applications in cancer data,” Computational and Mathematical Methods in Medicine, 2012.
737
