Denoising, Segmentation, and Visualization of ... - Semantic Scholar
Removal of Surface Artifacts of Material Volume Data with Defects Jie Shen, Vela Diego and David Yoon University of Michigan, Dearborn, Michigan, USA Abstract The three-dimensional defect distribution in material test specimens is a very important piece of information for us to understand the deformation and failure mechanism of materials. This distribution is sometimes complicated by the surface roughness of specimens in the defect detection of computed tomography data. In this paper, we proposed a new local differentiation algorithm to remove the surface artifacts caused by surface roughness in the defect detection of material specimens from computed tomography (CT) volume data. The accuracy of our method is compared with a traditional scan-line algorithm in terms of defect volume fraction measured in an independent scanning electron microscope (SEM) test. The experimental result indicates that our method is significantly better than the existing scan-line approach for predicting the defect volume fraction. 1. Introduction Dynamic spatial distribution and evolution of voids/cracks in engineering materials are crucial information for both engineers and scientists to understand damage and failure mechanisms in different loading conditions. Unfortunately, there are very few published experimental data with respect to the spatial distribution and evolution of multiple threedimensional (3D) voids and cracks. Although electron microscopy has been used to achieve a nanometer level resolution, only surface microstructures can be observed unless tedious and destructive dissection of samples is involved. The dissection is not only prohibitively labor-intensive and costly, but also destructive to some micro/mesoscale 3D cracks. Optical microscope shares the same drawbacks as above with relatively lower resolutions and restricted depths of field. Computer tomography on the basis of microfocus X-ray microscope, has been recently extended from medical field to material research [1], providing an affordable opportunity to nondestructive investigation on the dynamic propagation of 3D cracks. The surface of some material specimens may not be polished well in practice. In such cases, the surface/volume recognition of material defects (voids and cracks) is influenced by surface roughness. The peaks and valleys of surface roughness may be recognized as defects if there is no special treatment is applied. For metal alloy materials in which the defect volume fraction is not large (less than 0.05%), our experience on computed tomography (CT) and scanning electron microscopy (SEM) tests indicates that the prediction value on defect volume fraction could be two times different from the true value. The major error of this discrepancy is caused by surface roughness, especially in the cases of metal alloy materials. Thus, it is important to design a method that eliminates or alleviates the impact of surface roughness on the accurate detection of defect distribution.
1
Many existing studies were limited to the two-dimensional image processing for identifying and/or classifying defects [2-6]. Tsai et al. [3] used an anisotropic diffusion algorithm to micro-crack defects in solar wafer manufacturing. Both gray-level and gradient information were used to adjust the diffusion coefficients of the model. Fourier analysis is a common way for two-dimensional defect detection. One typical work was conducted by Chan et al. [4] for detecting fabric defects. Fourier transforms were applied for monitoring the spatial frequency spectrum of a fabric. Wavelet transform is another way of detecting patterned fabric defects [7]. Traditional Hough transform [6] was also used by some researchers for defect pattern detection. Neural network and support vector machine are yet another major method for defect pattern recognition. Jia et al. [2] used support vector machine for a real-time vision system of surface defect detection in hot rolling processes. Most of the above approaches were focused on the recognition of defect patterns rather than an accurate description of defect distribution. Shape from shading [8] can be used to identify three-dimensional surface defects. The drawback of this approach is no detection on interior defects. Sun et al. [9] utilized X-ray imaging for detecting weld defects in steel pipe. A real-time detection system was established on the basis of fuzzy pattern recognition. Three-dimensional computed tomography (CT) was used by Nakazawa et al. [10] for evaluating internal deformation/fracture characteristics of materials. Big particles were selected as landmarks on the basis of volume/frequency and landmark matching was designed for the analysis. X-ray micro-focus CT was also tested on characterizing the internal damage of composite materials [11]. Most of the existing methods were mainly aimed at the identification of material defects or the landmarks of material deformation instead of the exact distribution of defects. The latter is important in a multiresolution simulation of mechanical behavior on the digital models measured from the computed tomography system. In terms of filtering out the false defects caused by surface roughness, two possible approaches are i) surface/volume smoothing; and ii) removal of outliers. In the area of surface denoising, one group of existing studies were characterized by adopting smoothing algorithms in signal processing such as Laplacian [12-14], biLaplacian [15], mean filter [16], median filter [17, 18], alpha-trimming mean-median filter [19], Gaussian filter [20, 21], Weiner filter [22-24], band-pass filtering [24, 25], etc. Another group of studies were based upon the second-order geometric flow such as mean curvature flow [26-30], area-decreasing [31] or Laplacian of mean curvature [32]. A number of studies have been conducted to preserve sharp features during a mesh smoothing process. One technique was anisotropic diffusion that was used in height fields [28], triangle meshes [26, 33-36], level set surfaces [37], surface and function [38], surface reconstruction [39-41]; and the other method was bilateral filtering [42, 43], which was further extended to a trilateral filter [44]. Yet, another major approach is moving least-squares fitting [41, 45-49]. The drawback of smoothing algorithms is the alteration of defect geometry, which is not desirable in subsequent numerical simulations.
2
In the area of outlier removal, mono-oriented grouping method [50] is more suited to handling isolated outliers. With non-isolated outlier clusters, the dual orientation for outliers may not exist such that the detection of such outliers might fail. The spectral graph partitioning method [51] is also suited to dealing with isolated outliers, from which a normal cut could be easily conducted on the basis of certain distance metric. The cut becomes more difficult with non-isolated outlier clusters. The experimental results in [51] demonstrated the success of their approach only on synthetic discrete outliers without any particular address on the non-isolated outlier clusters that are associated with surface roughness in this study. In summary, very little work has been reported in the open literature on the accurate distribution of material defects associated with surface roughness. The main objective of this paper is to develop an effective approach to determine three-dimensional material defects in an adverse environment with poor quality of surface roughness. The rest of this paper is organized as follows. In Section 2, a new local differentiation algorithm is proposed to identify all the material defects with removal of artifacts (or false alarms) caused by surface roughness. Then, experimental setup and discussion are provided in Section 3. Finally, some conclusions are drawn in Section 4. 2. Local Differentiation Method for Removing False Defects at Rough Surfaces The surface roughness may cause significant error in identifying material defects, because the artifacts due to surface roughness may be mistakenly recognized as material defects. In this paper, we propose a new local algorithm to handle the material defects at rough surfaces. The input data format is limited to volume data from computed tomography (CT) or magnetic resonance imaging (MRI). Traditionally, a scan-line algorithm can be used to detect all the material defects inside a material domain, as shown in Figure 1. The basic idea is that a sequence of scan lines are created in each coordinate direction. Each line starts from one boundary (say left border) and traverse the material domain until the other end of boundary is encountered (e.g., right border). During the line traversal, if the volume element (voxel) intensity decreases from material voxel to defect voxel, it marks the beginning of a defect feature; next, when the pixel intensity jumps from defect volume voxel to material voxel, it represents the end of that particular defect. Here, an implicit assumption is that the intensity of material voxels is higher than that of defect voxels. If this is not the case, an inversion operation can be easily applied to the intensity of all the voxels of the CT data. Since the lighting environment inside the X-ray CT chamber is well controlled in this study, no non-uniform illumination problem is concerned, and thus an adaptive local treatment of thresholding is not needed. On the other hand, the scanning resolution is crucial to identify small defects at the level of micrometer. Smoothing operation and edge detection are not desired. As a result, we choose the approach of simple histogram thresholding to separate defects from material and background. The above scan-line algorithm works fine with a material domain that contains largesize defects. But, alloy metals frequently contain only a small fraction defects. For instance, the defect volume fraction of aluminum alloys could be less than 0.05% before
3
any mechanical loading is applied. In such cases, surface roughness plays a detrimental role of causing a significant relative error of detecting material defects. Our test experience indicates that the effect of surface roughness may deviate the estimation of true volume fraction of defects by two times. Thus, it is crucial to filter out those false defects caused by surface roughness. The basic idea of our new method for detecting material defects at rough surfaces is to utilize the information of voxel state change in a local neighborhood along three coordinate directions. Here, state refers to different types of voxels (background, material, and defect), and the state change means a transition from one voxel type to another in a spatially differential way. We name this method as local differentiation, which produces different labels around internal defects and small surface artifacts due to surface roughness, as illustrated in Figure 2. It demonstrates that an internal defect is surrounded by material elements (‘m’), while a surface opening has boundary elements (‘b’) as neighbors. Y-direction scanning
Y
X
X-direction scanning
Figure 1: Schematic view of scan-line Figure 2: Identification of an internal segmentation of defect elements from defect and a surface opening in our local material elements. differentiation method. In the local differentiation method, it is first necessary to segment the material from everything else. Because of no non-uniform illumination in this study, a simple histogram thresholding method is used without an adaptive treatment. To do this we run a loop through every voxel. If the voxel is within the given threshold, that voxel is labeled as material (‘m’); otherwise it is temporarily labeled as a defect (‘d’). Then the algorithm in Table 1 is used to outline the material. The way the algorithm works is by moving in a specific plane that is perpendicular to one of the coordinate axis. Then once in this plane the algorithm moves along a line that is parallel to one of the coordinate axis. In this line a scan goes from one side to the other until you reach a material voxel. Next, this voxel is changed to a boundary voxel. Since a boundary voxel is one that is next to an outsider voxel it follows that every material voxel next to the voxels scanned must also be a boundary voxel. So the algorithm goes to the previous line and change the label of every material voxel that is next to a boundary voxel. 4
Table 1: Local differentiation algorithm for identifying material defects with rough surfaces. Step Input: A volume data set obtained from CT or MRI 1 Given the threshold determine which voxels are voids and which voxels are materials. Label void voxels as DEFECT and material voxels as MATERIAL 2
Use three nested loops. The first loop is to limit movement in a plane that is perpendicular to one of the axis but cover every plane. The second loop is to limit movement along a line that is parallel to one of the axis but in the plane. The third loop is to move along the line.
3
Scan along the line, changing the label to BACKGROUND, until you reach a voxel that has been labeled MATERIAL.
4
Change the label from MATERIAL to BOUNDARY, and store the number of voxels scanned before arriving at the BOUNDARY.
5
If there is a previous line then change the label of each MATERIAL voxel to BOUNDRY if it is before the first BOUNDARY of the next line.
6
Repeat 2-5 in the opposite direction.
7
Repeat 2-6 on the other side
8
Repeat 2-7 for each face Output: MATERIAL and BOUNDARY are material elements; DEFECT refers to internal defect; BACKGROUND represents background space.
The underlying assumption of the local differentiation (LD) algorithm is that the material specimen or material domain is located at the center of each CT images, and is surrounded by the background. The size of the background at each border line of the images could be zero, one, or more voxels. The LD is basically a sequential scanning in all the coordinate directions, and therefore needs at least three passes for all the three coordinate directions. Since the local differentiation aims at surface roughness, twelve passes are designed with two passes for either the positive or negative direction of each coordinate. Figure 3 demonstrates two passes out of the twelve passes specifically for the direction of positive coordinate x. For the sake of simplicity, two-dimensional drawing is given in the figure, even though the third coordinate direction is considered in the implementation of the algorithm.
5
Figure 3(a) depicts a pass from left to right and bottom-up scan. Symbol ‘o’ represents an outsider element, while ‘m’ refers to a material element. During the scan, some elements become ‘b’, which denotes a boundary element. In each pass the computer scans the domain until it reaches the first material element. This element is converted into a boundary element, and the status of all the elements before it is changed to a background element (for the sake of clarity, transition from ‘d’ to background ‘o’ is not shown in Figure 3). Then the next line is scanned until the end of the slice is reached. If on the next line the first material element encountered is farther than that on the previous line (relative to the scanning), then each element adjacent to a non-material element is labeled as a boundary element. If the next line encounters a material element before the previous line, the next line is not modified because it will be changed when the lines are processed in the opposite direction. Similarly, Figure 3(b) is another pass that follows pass 1. After the twelve passes are completed, both boundary ‘b’ and material ‘m’ elements are considered as material, and outsider ‘o’ elements denote background elements. The figure does not show any internal defect element, which is designated by ‘d’, because we are focusing on the boundary of material domain.
(a) Pass 1 – Left to right and bottom-up scan
(b) Pass 2 – Left to right and up-bottom scan Figure 3: Two typical passes out of twelve passes in the local differentiation. 3. Experiment and Discussions In order to validate our local differentiation (LD) method, a comparison is conducted between scanning electron microscopy (SEM) images and X-ray computed tomography (XCT) data. Each SEM image is processed by using a simple image histogram method, and the XCT data is processed by the LD method as well as a traditional scan-line (SL) method with respect to a series of cross-section images obtained from three-dimensional XCT data. Figure 4(a) is an SEM image of an etched cross-section of a material specimen (an aluminum alloy), and Figure 4(b) demonstrates a three-dimensional XCT domain of the specimen. Porosity of defects is used as a measure to compare the results from SEM and XCT. Since the XCT domain is three-dimensional, a series of virtual cross-sections are made and the maximum porosity of these sections is then compared with the porosity 6
from the SEM image. Figure 4(c) indicates that the value predicted by our LD method is close to that obtained from SEM. The porosity predicted by the traditional scan-line algorithm [52] is 0.604 %, which is about two times greater than that of SEM.
(a) SEM image (black dots – voids)
(b) XCT data (c) porosity of defects (SEM: (white color – scanning electron defects) microscopy; LD-XCT: local differentiation; SL-XCT: scan-line algorithm) Figure 4: Comparison of defect porosity between an SEM image and XCT data.
Since there are many thousands of defects in a material specimen, it is very difficult, if not impossible, to validate the distribution of material defects in a complete way. The above measure, porosity of defects, is a convenient metric of defect sizing characterization. Another indirect way to show the efficacy of our LD method is to conduct a subsequent numerical analysis on the basis of defect distribution predicted by our method. The analysis is used to predict the mechanical property (effective Young’s modulus), which is then compared with the true value obtained from material testing. In Figure 5, defect mechanics is our new approach with defect distribution determined by the LD method. Its accuracy in predicting the effective Young’s modulus is significantly better than that of traditional approaches, including damage mechanics, micromechanics (self-consistent and Eshelby’s methods), and Gibson’s cellular model. The details of defect mechanics are beyond the scope of this paper.
7
Figure 5: Numerical errors of different approaches in predicting the effective Young’s modulus of a material specimen. Furthermore, multiresolution visualization of material defects is implemented as a visual way to inspect the correctness of our LD method. It is based on VOREEN [53], which is an open-source volume rendering engine. Figure 6 illustrates the visualization of the spatial distribution of material defects in a lightweight material.
Figure 6: Multiresolution visualization of material defects. 4. Conclusions In this paper an effective algorithm, local differentiation, has been proposed to remove surface artifacts in detecting material defects. Those artifacts are caused by surface roughness of material test specimens. Two types of comparisons were conducted to 8
demonstrate the efficacy of our new method, compared with a tradition scan-line algorithm. Experiment results (scanning electron imaging and material testing) indicate that our method and subsequent defect mechanics are significantly better than existing approaches in terms of predicting defect porosity and mechanical property (effective Young’s modulus). Acknowledgements This work was in part supported by U.S. National Science Foundation CMMI-0721625, a University of Michigan OVPR grant, and a University of Michigan – Dearborn CEEP grant. References [1] L. Salvo, P. Cloetens, E. Maire, S. Zabler, J. J. Blandin, J. Y. Buffiere, W. Ludwig, E. Boller, D. Bellet and C. Josserond, X-ray micro-tomography an attractive characterisation technique in materials science. Nuclear Instruments and Methods in Physics Research B 200, 273-286 (2003). [2] H. Jia, L. Y. Murphey, T. Chang, J. Shi and D. Gutchess. Real-time surface defect detection in hot rolling process. CD-ROM. 2003. Proceedings of Iron and Steel Exposition and 2003 AISE Annual Convention. [3] D. M. Tsai, C. C. Chang and S. M. Chao. Micro-crack inspection in heterogeneously textured solar wafers using anisotropic diffusion. Image and Vision Computing 28[3], 491-501. 2010. [4] C. H. Chan and K. H. Pang. Fabric defect detection by Fourier analysis. IEEE Transactions on Industry Applications 36[5], 1267-1276. 2000. [5] G. Paschos. Fast color texture recognition using chromaticity moments. Pattern Recognition Letters 21[8], 837-841. 2000. [6] K. Maruo, T. Shibata, T. Yamaguchi, M. Ichikawa and T. Ohmi. Automatic defect pattern detection on LSI wafers using image processing techniques. IEICE Transactions on Electronics E Series C 86[2], 1003-1012. 1999. [7] H. Y. T. Ngan, G. K. H. Pang, S. P. Yung and M. K. Ng. Wavelet based methods on patterned fabric defect detection. Pattern Recognition 38[4], 559-576. 2005. [8] L. Song, X. Qu, K. Xu and L. Lv. Three-dimensional measurement and defect measurement based on single image. Journal of Optoelectronics and Advanced Materials 7[2], 1029-1038. 2005. [9] Y. Sun, P. Bai, H. Y. Sun and P. Zhou. Real-time automatic detection of weld defects in steel pipe. NDT & E International 38[7], 522-528. 2005. [10] M. Nakazawa, Y. Aoki, M. Kobayashi and H. Toda. 3D image analysis for evaluating internal deformation/fracture characteristics of materials. 6718, 67180C.1-67180C.8. 2007. Proceedings of SPIE, the International Society for Optical Engineering. [11] P. J. Schilling, B. P. Karedla, A. K. Tatiparthi, M. A. Verges and P. D. Herrington. X-ray computed microtomography of internal damage in fiber reinforced polymer matrix composites. Composites Science and Technology 65[15-16], 2071-2078. 2005.
9
[12] G. Taubin. A Signal Processing Approach to Fair Surface Design. 351-358. 1995. Proceedings of the 22nd Annual Conference on Computer Graphics and Interactive Techniques. [13] G. Taubin. Linear anisotropic mesh filtering. 2001. IBM Research Division. Technical Report RC22213. [14] J. Vollmer, R. Mencl and H. Muller. Improved Laplacian smoothing of noisy surface meshes. Computer Graphics Forum 18[3], 131-138. 1999. [15] L. Kobbelt, S. Campagna, J. Vorsatz and H. Seidel. Interactive Multi-resolution Modeling on Arbitrary Meshes. 105-114. 1998. Proceedings of the 25th Annual Conference on Computer Graphics and Interactive Techniques. [16] A. Belyaev and Y. Ohtake. A comparison of mesh smoothing methods. 83-87. 2003. Israel-Korea Bi-National Conference on Geometric Modeling and Computer Graphics. [17] H. Yagou, Y. Ohtake and A. Belyaev. Mesh Smoothing via Mean and Median Filtering Applied to Face Normals. 124-131. 2002. Geometric Modeling and Processing 2002. [18] Y. Shen and K. E. Barner. Fuzzy vector median-based surface smoothing. IEEE Transactions on Visualization and Computer Graphics 10[3], 266-277. 2004. [19] H. Yagou, Y. Ohtake and A. Belyaev. Mesh denoising via iterative alphatrimming and non-linear diffusion of normals with automatic thresholding. 28-33. 2003. Proceedings of Computer Graphics International 2003. [20] Y. Ohtake, A. Belyaev and H. Seidel. Mesh smoothing by adaptive and anisotropic Gaussian filter. 203-210. 2002. Vision, Modeling, and Visualization 2002. [21] J. Oliensis. Local reproducible smoothing without shrinkage. IEEE Transactions on Pattern Analysis and Machine Intelligence 15[3], 307-312. 1993. [22] J. Peng, V. Strela and D. Zorin. A simple algorithm for surface denoising. 107112. 2001. IEEE Visualization 2001. [23] M. Alexa. Wiener filtering of meshes. 51-57. 2002. Proceedings of Shape Modeling International. [24] M. Pauly and M. Gross. Spectral processing of point-sampled geometry. 379386. 2001. Proceedings of the 25th Annual Conference on Computer Graphics and Interactive Techniques. [25] I. Guskov, W. Sweldens and P. Schroder. Multiresolution signal processing for meshes. 325-334. 1999. Proceedings of the 26th Annual Conference on Computer Graphics and Interactive Techniques. [26] U. Clarenz, U. Diewald and M. Rumpf. Anisotropic geometric diffusion in surface processing. 397-405. 2000. Proceedings of IEEE Visualization. [27] U. Clarenz, M. Rumpf and A. Telea. Robust feature detection and local classification for surfaces based on moment analysis. IEEE Transactions on Visualization and Computer Graphics 10[5], 516-524. 2004. [28] M. Desbrun, M. Meyer, P. Schroder and A. H. Barr. Anisotropic featurepreserving denoising of height fields and bivariate data. 145-152. 2000. Graphics Interface.
10
[29] U. Clarenz, G. Dziuk and M. Rumpf, On generalized mean curvature flow in surface processing. In. (Ed. H. Karcher and S. Hildebrandt) pp. 217-248, Springer, 2003. [30] M. Desbrun, M. Meyer, P. Schroder and A. H. Barr. Implicit Fairing of Irregular Meshes Using Diffusion and Curvature Flow. 317-324. 1999. Proceedings of the 26th Annual Conference on Computer Graphics and Interactive Techniques. [31] Y. Sun, D. L. Page, J. K. Paik, A. Koschan and M. A. Abidi. Triangle mesh-based surface modeling using adaptive smoothing and implicit texture integration. 588597. 2002. Proceedings of First International Symposium on 3D Data Processing Visualization and Transmission . [32] R. Schneider and L. Kobbelt. Generating fair meshes with g1 boundary conditions. 251-261. 2000. Proceedings of Geometric Modeling and Processing. [33] M. Meyer, M. Desbrun, P. Schroder and A. H. Barr, Discrete differentialgeometry operators for triangulated 2-manifolds. In Visualization and Mathematics III. pp. 35-57, Springer-Verlag, Heidelberg 2003. [34] H. Zhang and E. L. Fiume, Mesh smoothing with shape or feature preservation. In. (Ed. J. Vince and R. Earnshaw) pp. 167-182, Springer, 2002. [35] Y. Ohtake, A. Belyaev and I. A. Bogaevski. Mesh regularization and adaptive smoothing. Computer Aided Design 33[11], 789-800. 2001. [36] K. Hildebrandt and K. Polthier. Anisotropic filtering of non-linear surface features. Computer Graphics Forum 23[3], 391-400. 2004. [37] T. Tasdizen, R. T. Whitaker, P. Burchard and S. Osher. Anisotropic geometric diffusion in surface processing. 125-132. 2002. IEEE Visualization 2002. [38] C. Bajaj and G. Xu. Anisotropic diffusion of subdivision surfaces and functions on surfaces. ACM Transactions on Graphics 22[1], 4-32. 2003. [39] T. Bulow. Spherical diffusion for 3D surface smoothing. IEEE Transactions on Pattern Analysis and Machine Intelligence 26[12], 1650-1654. 2004. [40] T. Tasdizen and R. T. Whitaker. High-order nonlinear priors for surface reconstruction. IEEE Transactions on Pattern Analysis and Machine Intelligence 26[7], 878-891. 2004. [41] B. Mederos, L. Velho and L. H. De Figueiredo. Smoothing surface reconstruction from noisy clouds. Journal of the Brazilian Computer Society 9[3], 52-66. 2004. [42] T. R. Jones, F. Durant and M. Desbrun. Non-iterative, feature-preserving mesh smoothing. 943-949. 2003. Proceedings of the 30th Annual Conference on Computer Graphics and Interactive Techniques. [43] S. Fleishman, I. Drori and D. Cohen-Or. Bilateral mesh denoising. 950-953. 2003. Proceedings of the 30th Annual Conference on Computer Graphics and Interactive Techniques. [44] P. Choudhury and J. Tumblin. The trilateral filter for high contrast images and meshes. Christensen, Per. H. and Cohen, D. 186-196. 2003. Proceedings of the Eurographics Symposium on Rendering. [45] D. Levin, Mesh-independent surface interpolation. In Geometric Modeling for Scientific Visualization. pp. 37-49, Springer-Verlag, 2003. [46] S. Fleishman, D. Cohen-Or and C. Silva. Robust moving least-squares fitting with sharp features. ACM Transactions on Graphics. 24[3], 544-552. 2005.
11
[47] T. K. Dey and J. Sun. An adaptive MLS surface for reconstruction with guarantees. 2005. Eurographics Symposium on Geometry Processing (2005). Desbrun, M. and Pottmann, H. [48] C. Shen, J. F. O'Brien and J. R. Shewchuk. Interpolating and approximating implicit surfaces for polygon soup. 641-650. 2004. Proceedings of ACM SIGGRAPH 2003. [49] N. Amenta and Y. Kil. Defining point-set surfaces. ACM Transactions on Graphics 23[3], 264-270. 2004. [50] H. Xie, K. T. McDonnell and H. Qin. Surface reconstruction of noisy and defective data sets. 259-266. 2004. IEEE Visualization 2004. [51] R. Kolluri, J. R. Shewchuk and J. F. O'Brien. Spectral surface reconstruction from noisy point clouds. 11-21. 2004. Symposium on Geometry Processing. [52] J. D. Foley, A. van Dam, S. K. Feiner and J. F. Hughes, Computer Graphics, Principles and Practice, Addison-Wesley Publishing Company, 1997. [53] J. Meyer-Spadow, T. Ropinski, J. Mensmann and K. Hinrichs. Voreen: a rapidprototyping environment for ray-casting-based volume visualization. IEEE Computer Graphics & Applications 29[6], 6-13. 2009.
12
1
Many existing studies were limited to the two-dimensional image processing for identifying and/or classifying defects [2-6]. Tsai et al. [3] used an anisotropic diffusion algorithm to micro-crack defects in solar wafer manufacturing. Both gray-level and gradient information were used to adjust the diffusion coefficients of the model. Fourier analysis is a common way for two-dimensional defect detection. One typical work was conducted by Chan et al. [4] for detecting fabric defects. Fourier transforms were applied for monitoring the spatial frequency spectrum of a fabric. Wavelet transform is another way of detecting patterned fabric defects [7]. Traditional Hough transform [6] was also used by some researchers for defect pattern detection. Neural network and support vector machine are yet another major method for defect pattern recognition. Jia et al. [2] used support vector machine for a real-time vision system of surface defect detection in hot rolling processes. Most of the above approaches were focused on the recognition of defect patterns rather than an accurate description of defect distribution. Shape from shading [8] can be used to identify three-dimensional surface defects. The drawback of this approach is no detection on interior defects. Sun et al. [9] utilized X-ray imaging for detecting weld defects in steel pipe. A real-time detection system was established on the basis of fuzzy pattern recognition. Three-dimensional computed tomography (CT) was used by Nakazawa et al. [10] for evaluating internal deformation/fracture characteristics of materials. Big particles were selected as landmarks on the basis of volume/frequency and landmark matching was designed for the analysis. X-ray micro-focus CT was also tested on characterizing the internal damage of composite materials [11]. Most of the existing methods were mainly aimed at the identification of material defects or the landmarks of material deformation instead of the exact distribution of defects. The latter is important in a multiresolution simulation of mechanical behavior on the digital models measured from the computed tomography system. In terms of filtering out the false defects caused by surface roughness, two possible approaches are i) surface/volume smoothing; and ii) removal of outliers. In the area of surface denoising, one group of existing studies were characterized by adopting smoothing algorithms in signal processing such as Laplacian [12-14], biLaplacian [15], mean filter [16], median filter [17, 18], alpha-trimming mean-median filter [19], Gaussian filter [20, 21], Weiner filter [22-24], band-pass filtering [24, 25], etc. Another group of studies were based upon the second-order geometric flow such as mean curvature flow [26-30], area-decreasing [31] or Laplacian of mean curvature [32]. A number of studies have been conducted to preserve sharp features during a mesh smoothing process. One technique was anisotropic diffusion that was used in height fields [28], triangle meshes [26, 33-36], level set surfaces [37], surface and function [38], surface reconstruction [39-41]; and the other method was bilateral filtering [42, 43], which was further extended to a trilateral filter [44]. Yet, another major approach is moving least-squares fitting [41, 45-49]. The drawback of smoothing algorithms is the alteration of defect geometry, which is not desirable in subsequent numerical simulations.
2
In the area of outlier removal, mono-oriented grouping method [50] is more suited to handling isolated outliers. With non-isolated outlier clusters, the dual orientation for outliers may not exist such that the detection of such outliers might fail. The spectral graph partitioning method [51] is also suited to dealing with isolated outliers, from which a normal cut could be easily conducted on the basis of certain distance metric. The cut becomes more difficult with non-isolated outlier clusters. The experimental results in [51] demonstrated the success of their approach only on synthetic discrete outliers without any particular address on the non-isolated outlier clusters that are associated with surface roughness in this study. In summary, very little work has been reported in the open literature on the accurate distribution of material defects associated with surface roughness. The main objective of this paper is to develop an effective approach to determine three-dimensional material defects in an adverse environment with poor quality of surface roughness. The rest of this paper is organized as follows. In Section 2, a new local differentiation algorithm is proposed to identify all the material defects with removal of artifacts (or false alarms) caused by surface roughness. Then, experimental setup and discussion are provided in Section 3. Finally, some conclusions are drawn in Section 4. 2. Local Differentiation Method for Removing False Defects at Rough Surfaces The surface roughness may cause significant error in identifying material defects, because the artifacts due to surface roughness may be mistakenly recognized as material defects. In this paper, we propose a new local algorithm to handle the material defects at rough surfaces. The input data format is limited to volume data from computed tomography (CT) or magnetic resonance imaging (MRI). Traditionally, a scan-line algorithm can be used to detect all the material defects inside a material domain, as shown in Figure 1. The basic idea is that a sequence of scan lines are created in each coordinate direction. Each line starts from one boundary (say left border) and traverse the material domain until the other end of boundary is encountered (e.g., right border). During the line traversal, if the volume element (voxel) intensity decreases from material voxel to defect voxel, it marks the beginning of a defect feature; next, when the pixel intensity jumps from defect volume voxel to material voxel, it represents the end of that particular defect. Here, an implicit assumption is that the intensity of material voxels is higher than that of defect voxels. If this is not the case, an inversion operation can be easily applied to the intensity of all the voxels of the CT data. Since the lighting environment inside the X-ray CT chamber is well controlled in this study, no non-uniform illumination problem is concerned, and thus an adaptive local treatment of thresholding is not needed. On the other hand, the scanning resolution is crucial to identify small defects at the level of micrometer. Smoothing operation and edge detection are not desired. As a result, we choose the approach of simple histogram thresholding to separate defects from material and background. The above scan-line algorithm works fine with a material domain that contains largesize defects. But, alloy metals frequently contain only a small fraction defects. For instance, the defect volume fraction of aluminum alloys could be less than 0.05% before
3
any mechanical loading is applied. In such cases, surface roughness plays a detrimental role of causing a significant relative error of detecting material defects. Our test experience indicates that the effect of surface roughness may deviate the estimation of true volume fraction of defects by two times. Thus, it is crucial to filter out those false defects caused by surface roughness. The basic idea of our new method for detecting material defects at rough surfaces is to utilize the information of voxel state change in a local neighborhood along three coordinate directions. Here, state refers to different types of voxels (background, material, and defect), and the state change means a transition from one voxel type to another in a spatially differential way. We name this method as local differentiation, which produces different labels around internal defects and small surface artifacts due to surface roughness, as illustrated in Figure 2. It demonstrates that an internal defect is surrounded by material elements (‘m’), while a surface opening has boundary elements (‘b’) as neighbors. Y-direction scanning
Y
X
X-direction scanning
Figure 1: Schematic view of scan-line Figure 2: Identification of an internal segmentation of defect elements from defect and a surface opening in our local material elements. differentiation method. In the local differentiation method, it is first necessary to segment the material from everything else. Because of no non-uniform illumination in this study, a simple histogram thresholding method is used without an adaptive treatment. To do this we run a loop through every voxel. If the voxel is within the given threshold, that voxel is labeled as material (‘m’); otherwise it is temporarily labeled as a defect (‘d’). Then the algorithm in Table 1 is used to outline the material. The way the algorithm works is by moving in a specific plane that is perpendicular to one of the coordinate axis. Then once in this plane the algorithm moves along a line that is parallel to one of the coordinate axis. In this line a scan goes from one side to the other until you reach a material voxel. Next, this voxel is changed to a boundary voxel. Since a boundary voxel is one that is next to an outsider voxel it follows that every material voxel next to the voxels scanned must also be a boundary voxel. So the algorithm goes to the previous line and change the label of every material voxel that is next to a boundary voxel. 4
Table 1: Local differentiation algorithm for identifying material defects with rough surfaces. Step Input: A volume data set obtained from CT or MRI 1 Given the threshold determine which voxels are voids and which voxels are materials. Label void voxels as DEFECT and material voxels as MATERIAL 2
Use three nested loops. The first loop is to limit movement in a plane that is perpendicular to one of the axis but cover every plane. The second loop is to limit movement along a line that is parallel to one of the axis but in the plane. The third loop is to move along the line.
3
Scan along the line, changing the label to BACKGROUND, until you reach a voxel that has been labeled MATERIAL.
4
Change the label from MATERIAL to BOUNDARY, and store the number of voxels scanned before arriving at the BOUNDARY.
5
If there is a previous line then change the label of each MATERIAL voxel to BOUNDRY if it is before the first BOUNDARY of the next line.
6
Repeat 2-5 in the opposite direction.
7
Repeat 2-6 on the other side
8
Repeat 2-7 for each face Output: MATERIAL and BOUNDARY are material elements; DEFECT refers to internal defect; BACKGROUND represents background space.
The underlying assumption of the local differentiation (LD) algorithm is that the material specimen or material domain is located at the center of each CT images, and is surrounded by the background. The size of the background at each border line of the images could be zero, one, or more voxels. The LD is basically a sequential scanning in all the coordinate directions, and therefore needs at least three passes for all the three coordinate directions. Since the local differentiation aims at surface roughness, twelve passes are designed with two passes for either the positive or negative direction of each coordinate. Figure 3 demonstrates two passes out of the twelve passes specifically for the direction of positive coordinate x. For the sake of simplicity, two-dimensional drawing is given in the figure, even though the third coordinate direction is considered in the implementation of the algorithm.
5
Figure 3(a) depicts a pass from left to right and bottom-up scan. Symbol ‘o’ represents an outsider element, while ‘m’ refers to a material element. During the scan, some elements become ‘b’, which denotes a boundary element. In each pass the computer scans the domain until it reaches the first material element. This element is converted into a boundary element, and the status of all the elements before it is changed to a background element (for the sake of clarity, transition from ‘d’ to background ‘o’ is not shown in Figure 3). Then the next line is scanned until the end of the slice is reached. If on the next line the first material element encountered is farther than that on the previous line (relative to the scanning), then each element adjacent to a non-material element is labeled as a boundary element. If the next line encounters a material element before the previous line, the next line is not modified because it will be changed when the lines are processed in the opposite direction. Similarly, Figure 3(b) is another pass that follows pass 1. After the twelve passes are completed, both boundary ‘b’ and material ‘m’ elements are considered as material, and outsider ‘o’ elements denote background elements. The figure does not show any internal defect element, which is designated by ‘d’, because we are focusing on the boundary of material domain.
(a) Pass 1 – Left to right and bottom-up scan
(b) Pass 2 – Left to right and up-bottom scan Figure 3: Two typical passes out of twelve passes in the local differentiation. 3. Experiment and Discussions In order to validate our local differentiation (LD) method, a comparison is conducted between scanning electron microscopy (SEM) images and X-ray computed tomography (XCT) data. Each SEM image is processed by using a simple image histogram method, and the XCT data is processed by the LD method as well as a traditional scan-line (SL) method with respect to a series of cross-section images obtained from three-dimensional XCT data. Figure 4(a) is an SEM image of an etched cross-section of a material specimen (an aluminum alloy), and Figure 4(b) demonstrates a three-dimensional XCT domain of the specimen. Porosity of defects is used as a measure to compare the results from SEM and XCT. Since the XCT domain is three-dimensional, a series of virtual cross-sections are made and the maximum porosity of these sections is then compared with the porosity 6
from the SEM image. Figure 4(c) indicates that the value predicted by our LD method is close to that obtained from SEM. The porosity predicted by the traditional scan-line algorithm [52] is 0.604 %, which is about two times greater than that of SEM.
(a) SEM image (black dots – voids)
(b) XCT data (c) porosity of defects (SEM: (white color – scanning electron defects) microscopy; LD-XCT: local differentiation; SL-XCT: scan-line algorithm) Figure 4: Comparison of defect porosity between an SEM image and XCT data.
Since there are many thousands of defects in a material specimen, it is very difficult, if not impossible, to validate the distribution of material defects in a complete way. The above measure, porosity of defects, is a convenient metric of defect sizing characterization. Another indirect way to show the efficacy of our LD method is to conduct a subsequent numerical analysis on the basis of defect distribution predicted by our method. The analysis is used to predict the mechanical property (effective Young’s modulus), which is then compared with the true value obtained from material testing. In Figure 5, defect mechanics is our new approach with defect distribution determined by the LD method. Its accuracy in predicting the effective Young’s modulus is significantly better than that of traditional approaches, including damage mechanics, micromechanics (self-consistent and Eshelby’s methods), and Gibson’s cellular model. The details of defect mechanics are beyond the scope of this paper.
7
Figure 5: Numerical errors of different approaches in predicting the effective Young’s modulus of a material specimen. Furthermore, multiresolution visualization of material defects is implemented as a visual way to inspect the correctness of our LD method. It is based on VOREEN [53], which is an open-source volume rendering engine. Figure 6 illustrates the visualization of the spatial distribution of material defects in a lightweight material.
Figure 6: Multiresolution visualization of material defects. 4. Conclusions In this paper an effective algorithm, local differentiation, has been proposed to remove surface artifacts in detecting material defects. Those artifacts are caused by surface roughness of material test specimens. Two types of comparisons were conducted to 8
demonstrate the efficacy of our new method, compared with a tradition scan-line algorithm. Experiment results (scanning electron imaging and material testing) indicate that our method and subsequent defect mechanics are significantly better than existing approaches in terms of predicting defect porosity and mechanical property (effective Young’s modulus). Acknowledgements This work was in part supported by U.S. National Science Foundation CMMI-0721625, a University of Michigan OVPR grant, and a University of Michigan – Dearborn CEEP grant. References [1] L. Salvo, P. Cloetens, E. Maire, S. Zabler, J. J. Blandin, J. Y. Buffiere, W. Ludwig, E. Boller, D. Bellet and C. Josserond, X-ray micro-tomography an attractive characterisation technique in materials science. Nuclear Instruments and Methods in Physics Research B 200, 273-286 (2003). [2] H. Jia, L. Y. Murphey, T. Chang, J. Shi and D. Gutchess. Real-time surface defect detection in hot rolling process. CD-ROM. 2003. Proceedings of Iron and Steel Exposition and 2003 AISE Annual Convention. [3] D. M. Tsai, C. C. Chang and S. M. Chao. Micro-crack inspection in heterogeneously textured solar wafers using anisotropic diffusion. Image and Vision Computing 28[3], 491-501. 2010. [4] C. H. Chan and K. H. Pang. Fabric defect detection by Fourier analysis. IEEE Transactions on Industry Applications 36[5], 1267-1276. 2000. [5] G. Paschos. Fast color texture recognition using chromaticity moments. Pattern Recognition Letters 21[8], 837-841. 2000. [6] K. Maruo, T. Shibata, T. Yamaguchi, M. Ichikawa and T. Ohmi. Automatic defect pattern detection on LSI wafers using image processing techniques. IEICE Transactions on Electronics E Series C 86[2], 1003-1012. 1999. [7] H. Y. T. Ngan, G. K. H. Pang, S. P. Yung and M. K. Ng. Wavelet based methods on patterned fabric defect detection. Pattern Recognition 38[4], 559-576. 2005. [8] L. Song, X. Qu, K. Xu and L. Lv. Three-dimensional measurement and defect measurement based on single image. Journal of Optoelectronics and Advanced Materials 7[2], 1029-1038. 2005. [9] Y. Sun, P. Bai, H. Y. Sun and P. Zhou. Real-time automatic detection of weld defects in steel pipe. NDT & E International 38[7], 522-528. 2005. [10] M. Nakazawa, Y. Aoki, M. Kobayashi and H. Toda. 3D image analysis for evaluating internal deformation/fracture characteristics of materials. 6718, 67180C.1-67180C.8. 2007. Proceedings of SPIE, the International Society for Optical Engineering. [11] P. J. Schilling, B. P. Karedla, A. K. Tatiparthi, M. A. Verges and P. D. Herrington. X-ray computed microtomography of internal damage in fiber reinforced polymer matrix composites. Composites Science and Technology 65[15-16], 2071-2078. 2005.
9
[12] G. Taubin. A Signal Processing Approach to Fair Surface Design. 351-358. 1995. Proceedings of the 22nd Annual Conference on Computer Graphics and Interactive Techniques. [13] G. Taubin. Linear anisotropic mesh filtering. 2001. IBM Research Division. Technical Report RC22213. [14] J. Vollmer, R. Mencl and H. Muller. Improved Laplacian smoothing of noisy surface meshes. Computer Graphics Forum 18[3], 131-138. 1999. [15] L. Kobbelt, S. Campagna, J. Vorsatz and H. Seidel. Interactive Multi-resolution Modeling on Arbitrary Meshes. 105-114. 1998. Proceedings of the 25th Annual Conference on Computer Graphics and Interactive Techniques. [16] A. Belyaev and Y. Ohtake. A comparison of mesh smoothing methods. 83-87. 2003. Israel-Korea Bi-National Conference on Geometric Modeling and Computer Graphics. [17] H. Yagou, Y. Ohtake and A. Belyaev. Mesh Smoothing via Mean and Median Filtering Applied to Face Normals. 124-131. 2002. Geometric Modeling and Processing 2002. [18] Y. Shen and K. E. Barner. Fuzzy vector median-based surface smoothing. IEEE Transactions on Visualization and Computer Graphics 10[3], 266-277. 2004. [19] H. Yagou, Y. Ohtake and A. Belyaev. Mesh denoising via iterative alphatrimming and non-linear diffusion of normals with automatic thresholding. 28-33. 2003. Proceedings of Computer Graphics International 2003. [20] Y. Ohtake, A. Belyaev and H. Seidel. Mesh smoothing by adaptive and anisotropic Gaussian filter. 203-210. 2002. Vision, Modeling, and Visualization 2002. [21] J. Oliensis. Local reproducible smoothing without shrinkage. IEEE Transactions on Pattern Analysis and Machine Intelligence 15[3], 307-312. 1993. [22] J. Peng, V. Strela and D. Zorin. A simple algorithm for surface denoising. 107112. 2001. IEEE Visualization 2001. [23] M. Alexa. Wiener filtering of meshes. 51-57. 2002. Proceedings of Shape Modeling International. [24] M. Pauly and M. Gross. Spectral processing of point-sampled geometry. 379386. 2001. Proceedings of the 25th Annual Conference on Computer Graphics and Interactive Techniques. [25] I. Guskov, W. Sweldens and P. Schroder. Multiresolution signal processing for meshes. 325-334. 1999. Proceedings of the 26th Annual Conference on Computer Graphics and Interactive Techniques. [26] U. Clarenz, U. Diewald and M. Rumpf. Anisotropic geometric diffusion in surface processing. 397-405. 2000. Proceedings of IEEE Visualization. [27] U. Clarenz, M. Rumpf and A. Telea. Robust feature detection and local classification for surfaces based on moment analysis. IEEE Transactions on Visualization and Computer Graphics 10[5], 516-524. 2004. [28] M. Desbrun, M. Meyer, P. Schroder and A. H. Barr. Anisotropic featurepreserving denoising of height fields and bivariate data. 145-152. 2000. Graphics Interface.
10
[29] U. Clarenz, G. Dziuk and M. Rumpf, On generalized mean curvature flow in surface processing. In. (Ed. H. Karcher and S. Hildebrandt) pp. 217-248, Springer, 2003. [30] M. Desbrun, M. Meyer, P. Schroder and A. H. Barr. Implicit Fairing of Irregular Meshes Using Diffusion and Curvature Flow. 317-324. 1999. Proceedings of the 26th Annual Conference on Computer Graphics and Interactive Techniques. [31] Y. Sun, D. L. Page, J. K. Paik, A. Koschan and M. A. Abidi. Triangle mesh-based surface modeling using adaptive smoothing and implicit texture integration. 588597. 2002. Proceedings of First International Symposium on 3D Data Processing Visualization and Transmission . [32] R. Schneider and L. Kobbelt. Generating fair meshes with g1 boundary conditions. 251-261. 2000. Proceedings of Geometric Modeling and Processing. [33] M. Meyer, M. Desbrun, P. Schroder and A. H. Barr, Discrete differentialgeometry operators for triangulated 2-manifolds. In Visualization and Mathematics III. pp. 35-57, Springer-Verlag, Heidelberg 2003. [34] H. Zhang and E. L. Fiume, Mesh smoothing with shape or feature preservation. In. (Ed. J. Vince and R. Earnshaw) pp. 167-182, Springer, 2002. [35] Y. Ohtake, A. Belyaev and I. A. Bogaevski. Mesh regularization and adaptive smoothing. Computer Aided Design 33[11], 789-800. 2001. [36] K. Hildebrandt and K. Polthier. Anisotropic filtering of non-linear surface features. Computer Graphics Forum 23[3], 391-400. 2004. [37] T. Tasdizen, R. T. Whitaker, P. Burchard and S. Osher. Anisotropic geometric diffusion in surface processing. 125-132. 2002. IEEE Visualization 2002. [38] C. Bajaj and G. Xu. Anisotropic diffusion of subdivision surfaces and functions on surfaces. ACM Transactions on Graphics 22[1], 4-32. 2003. [39] T. Bulow. Spherical diffusion for 3D surface smoothing. IEEE Transactions on Pattern Analysis and Machine Intelligence 26[12], 1650-1654. 2004. [40] T. Tasdizen and R. T. Whitaker. High-order nonlinear priors for surface reconstruction. IEEE Transactions on Pattern Analysis and Machine Intelligence 26[7], 878-891. 2004. [41] B. Mederos, L. Velho and L. H. De Figueiredo. Smoothing surface reconstruction from noisy clouds. Journal of the Brazilian Computer Society 9[3], 52-66. 2004. [42] T. R. Jones, F. Durant and M. Desbrun. Non-iterative, feature-preserving mesh smoothing. 943-949. 2003. Proceedings of the 30th Annual Conference on Computer Graphics and Interactive Techniques. [43] S. Fleishman, I. Drori and D. Cohen-Or. Bilateral mesh denoising. 950-953. 2003. Proceedings of the 30th Annual Conference on Computer Graphics and Interactive Techniques. [44] P. Choudhury and J. Tumblin. The trilateral filter for high contrast images and meshes. Christensen, Per. H. and Cohen, D. 186-196. 2003. Proceedings of the Eurographics Symposium on Rendering. [45] D. Levin, Mesh-independent surface interpolation. In Geometric Modeling for Scientific Visualization. pp. 37-49, Springer-Verlag, 2003. [46] S. Fleishman, D. Cohen-Or and C. Silva. Robust moving least-squares fitting with sharp features. ACM Transactions on Graphics. 24[3], 544-552. 2005.
11
[47] T. K. Dey and J. Sun. An adaptive MLS surface for reconstruction with guarantees. 2005. Eurographics Symposium on Geometry Processing (2005). Desbrun, M. and Pottmann, H. [48] C. Shen, J. F. O'Brien and J. R. Shewchuk. Interpolating and approximating implicit surfaces for polygon soup. 641-650. 2004. Proceedings of ACM SIGGRAPH 2003. [49] N. Amenta and Y. Kil. Defining point-set surfaces. ACM Transactions on Graphics 23[3], 264-270. 2004. [50] H. Xie, K. T. McDonnell and H. Qin. Surface reconstruction of noisy and defective data sets. 259-266. 2004. IEEE Visualization 2004. [51] R. Kolluri, J. R. Shewchuk and J. F. O'Brien. Spectral surface reconstruction from noisy point clouds. 11-21. 2004. Symposium on Geometry Processing. [52] J. D. Foley, A. van Dam, S. K. Feiner and J. F. Hughes, Computer Graphics, Principles and Practice, Addison-Wesley Publishing Company, 1997. [53] J. Meyer-Spadow, T. Ropinski, J. Mensmann and K. Hinrichs. Voreen: a rapidprototyping environment for ray-casting-based volume visualization. IEEE Computer Graphics & Applications 29[6], 6-13. 2009.
12
