NOISE AND ARTIFACT REMOVAL IN KNIFE-EDGE SCANNING ...
NOISE AND ARTIFACT REMOVAL IN KNIFE-EDGE SCANNING MICROSCOPY D. Mayerich, B.H. McCormick, J. Keyser Texas A&M University Department of Computer Science College Station, TX ABSTRACT Knife-Edge Scanning Microscopy (KESM) is a recently developed technique that allows fast and automated imaging of several hundred cubic millimeters of tissue at sub-micron resolution. Successive sections are captured in registration by imaging the specimen concurrently with cutting by a diamondknife ultramicrotome. Because this imaging technique is relatively new, we are currently investigating ways to improve image quality and data rate. In addition, certain imaging artifacts are unique to this technology and the time required to perform corrective image processing is a concern due to the high rate of image capture. In this paper, we describe algorithms that can be used to process KESM images in order to obtain the quality necessary for subsequent segmentation and modeling. There is also emphasis on making these algorithms independent of global information within the image so that they can be more easily parallelized. Index Terms— knife-edge microscopy, chatter, lighting artifacts, volumetric data capture 1. INTRODUCTION The acquisition of large cellular-level datasets is a next step in the understanding of the anatomical structure of organisms. Well-known techniques such as confocal [1], multi-photon [2], and, more recently, serial block face scanning electron microscopy (SBF-SEM) [3] have been used to image small volumes of cellular structures. Staining techniques have also been developed to take advantage of the imaging methods used by fluorescence microscopy [4, 5, 6]. These methods are known to produce high-resolution cellular-level datasets, however fluorescence microscopy is limited to the surface of the tissue sample while SBF-SEM is limited in general to very small samples of tissue. We have recently introduced a technique known as knifeedge scanning microscopy (KESM) [7] in order to overcome This work was funded in part by National Institute of Neurological Disorders and Stroke grant #R01 NS 54252, National Science Foundation grants CCF-0220047 and #0079874, Texas Higher Education Coordinating Board grant #ATP-000512-0146-2001, the Office of the Vice President for Research at Texas A&M University.
Fig. 1. Knife-Edge Scanning Microscopy. the above limits on tissue volume. Captured KESM images are subject to various forms of noise and artifacts due to the imaging methodology, some of which are common to other forms of microscopy, and some of which are unique to KESM. In this paper, we present methods for processing KESM images in order to remove noise and artifacts, thus enabling more useful volumetric datasets for subsequent analysis. 2. KNIFE-EDGE SCANNING MICROSCOPY KESM datasets are created by concurrently cutting and imaging embedded tissue samples. A diamond knife ultramicrotome is used to cut thin sections of the sample while simultaneously the top facet of the knife is imaged by an optical microscope (fig. 1). Illumination is provided through the diamond knife by a liquid-optic illuminator. Imaging is performed by a high sensitivity line-scan camera mounted at the end of the optical train of the microscope. Camera firing is keyed to the digitally encoded position of the tissue under the diamond knife. Therefore, as the tissue sample is translated using a linear stage, the camera records sequential lines of data as the section moves over the knife surface. In order to obtain the most information from the trans-
mitted light, Nikon water immersion objectives are used for their larger numerical aperture. For this reason, all cutting is performed under water. In order to keep debris from building up on the knife surface, a current is induced across the knife edge by applying suction at the rear of the knife. This causes sections to be pulled upwards towards the base of the knife, away from the field of view (FOV) of the objective. This imaging process results in large volumes of volumetric data. At the resolutions we work with (typically 3001000nm in each dimension), a typical single pass over a specimen block produces an image of size approximately 2048 x 9000 pixels. Here the x-axis is along the edge of the knife; thus the x-resolution is the resolution of the line-scan camera that images the tip of the knife blade. The y-axis in any one slice is dependent on the length of the slice taken. Since each slice is taken at constant speed (though subsequent slices might differ in speed), the y-axis is also equivalent to time. Subsequent slices can be obtained adjacent to or below that slice; we currently have scanned more than 1000 layers deep. The resulting dataset can be massive, totalling several terabytes for a single organ of a small animal.
Fig. 2. The overall intensity shift along the x-axis is due to knife misalignment and two knife defects leave streaks in the image (arrows).
3. KESM VOLUME NOISE KESM exhibits several types of noise common to and different from traditional microscopy. These generally involve small lighting irregularities due to camera alignment, illumination frequency, and knife vibration during the cutting process.
Fig. 3. High frequency chatter is visible throughout the image and particularly bad cases (arrows) may cause loss of information.
3.1. Lighting Defects
Lighting defects such as those listed above form regular patterns in the output data that are easily removed from any given image. Unfortunately, irregular illumination tends to occur in KESM images due to knife vibration. Knife chatter is well known in machining [8, 9, 10], but usually is not an issue in imaging, since other imaging techniques do not image data as it is being cut. We use several mechanical techniques to reduce knife vibration, including increasing the mechanical stiffness of the specimen and cutting tool and randomizing cutting velocities in order to prevent reinforcement at any frequency. Even with these precautions, knife vibration is visible in the resulting image as changes in illumination along the y-axis (fig. 3). When using KESM it is important to sample as close to the edge of the knife as possible in order to get the best alignment between sections as well as to ensure that the section is coherent and not warped or torn due to water current or knife friction. Sampling at the very tip of the knife, however, makes knife vibration more visible since light intensity quickly drops past the edge of the knife. Knife vibration is therefore visible as dark horizontal stripes across the image which, due to misalignments described above, may or may not be continuous along the x-axis. In addition, particularly severe occurrences of chatter can cause loss of data.
The most visible lighting defect involves a variation in illumination across the x-axis. There are two main sources: • Small misalignments between the camera and the surface of the knife. This misalignment of the camera results in a steady change in the overall illumination across the image. • Defects (e.g. chips and uneven areas) on the surface of the knife edge itself. The defects cause refraction and reflection variations at the knife surface, resulting in uneven lighting across the knife edge. This is visible in images as brighter or darker strips that extend in the y direction (fig. 2). Regular lighting defects are also visible along the y-axis. The major source is: • The illumination power source can cause fluctuations in the illumination over time. This produces an oscillating fluctuation in illumination creating visible stripes in the image. Depending on the sampling rate of the image, the frequency of the fluctuations changes in the image space however they are constant over the time domain.
3.2. Knife Chatter
4. KESM IMAGE PROCESSING TECHNIQUES We have assembled several known imaging algorithms to help remove lighting irregularities. New methods are also demonstrated that take advantage of the unique noise found in KESM images in order to better prepare the volumetric datasets for segmentation. In addition, we specifically focus on image processing algorithms that require only local information in an image so that processing can be distributed across several systems for faster results. Such parallelism is important for maintaining the high data rate that gives KESM an advantage over other forms of microscopy. 4.1. Light Normalization Equalizing the illumination across the x-axis of the image is the first step in removing noise. Since these irregularities are due to misalignment of the camera or artifacts in the lighting pipeline, they are repeated over every sample in every image. For example, if an artifact present in the image is due to a defect in the knife (such as a chip in the knife surface), this artifact will be present in every sample at a given x value in the image. This type of artifact is also visible even when tissue is not being sectioned. The constant presence of this type of artifact allows a base sample without tissue to be imaged at any time during the cutting process. Since these variations of lighting are constant during the sectioning process, each sample taken along the y-axis can be normalized using this base light vector. Note that such base information might need to be taken at regular intervals, since a knife might gradually develop surface defects, or gradually move out of alignment. Cyclical illumination artifacts produced by high-frequency noise from the light source are less consistent between images. Since the initial sample can take place at any point during the illuminator’s frequency fluctuation, a phase shift is observed along the y-axis. We compensate for this by sectioning slightly less tissue than the knife and objective allows. This leaves a small portion of the knife edge visible without any interfering tissue. A sample line of pixels along the edge of the image can then be used to normalize light frequency fluctuations along the y-axis. 4.2. Removing Chatter Artifacts Knife chatter artifacts are more difficult to remove from KESM images. Although they are lines that extend along the xaxis, they tend to be discontinuous and not perfectly (though nearly) horizontal. This precludes scaling the line by any given scalar value, such as the mean of the current sample. Local smoothing is a method often used to eliminate highfrequency noise [11, 12]. However, it is difficult to apply these techniques to KESM data since most of the information is high frequency and low contrast. Instead, we scale the value of a pixel in the sample by the mean of a small window of pixels surrounding the current pixel in the sample. As this
Fig. 4. Original mouse brain stem section with close-up. window moves across the image, it removes streaks caused by tissue folding and knife vibration by scaling the intensity values up to match other pixels in the sample. This preserves the high-frequency details, providing that there was very little data lost due to the intensity shift, and removes the streaks making segmentation a simpler matter. 5. RESULTS We have applied these techniques to many different KESM data sets and have come to rely on them as a pre-processing step before segmentation. Here we present these methods for processing Nissl-stained mouse brain stem and cerebellum. This dataset was selected because of the high frequency chatter in the cerebellum and low contrast in the spinal cord (fig. 4 - 6). 6. REFERENCES [1] J. B. Pauley, Ed., Handbook of Biological Confocal Microscopy, Plenum Press, New York, 1995. [2] Winfried Denk, James H. Strickler, and Watt W. Webb, “Two-photon laser scanning fluorescence microscopy,” Science, vol. 248, no. 4951, pp. 73–76, Apr. 1990.
Fig. 5. Removal of lighting and knife irregularities. [3] Winfried Denk and Heinz Horstmann, “Serial blockface scanning electron microscopy to reconstruct threedimensional tissue nanostructure,” PLoS Biology, vol. 2, no. 11, pp. e329, Nov. 2004. [4] P. Greenspan, E. P. Mayer, and S. D. Fowler, “Nile red: a selective fluorescent stain for intracellular lipid droplets,” The Journal of Cell Biology, vol. 100, pp. 965–973, 1985. [5] M. Chalfie, Y. Tu, G. Euskirchen, W. W. Ward, and D. C. Prasher, “Green fluorescent protein as a marker for gene expression,” Science, vol. 263, no. 5148, pp. 802–805, Feb. 1994. [6] Tatsuyuki Takada, Keiko Iida, Takeo Awaji, Kazumi Itoh, Riichi Takahashi, Akiko Shibui, Kenichi Yoshida, Sumio Sugano, and Gozoh Tsujimoto, “Selective production of transgenic mice using green fluorescent protein as a marker,” Nature Biotechnology, vol. 15, no. 5, pp. 458, May 1997. [7] Bruce McCormick, “Knife-edge scanning microscope (kesm 1.5): Optics and cameras,” Tech. Rep., Department of Computer Science, Texas A&M University, College Station, TX, Oct. 2006.
Fig. 6. Final pass for the reduction of knife chatter. High resolution views are taken from the same image as fig. 5. Images are shown after removal of light artifacts (left) and after removal of chatter (right). [8] R. Phillips, “Diamond knife ultra microtomy of metals and the structure of microtomed sections,” British Journal of Applied Physics, vol. 12, pp. 554–558, Oct. 1961. [9] Marian Wiercigroch and Erhan Budak, “Sources of nonlinearities, chatter generation and suppression in metal cutting,” Philosophical Transactions of the Royal Society A: Mathematical, Physical and Engineering Sciences, vol. 359, no. 1781, pp. 663–693, Apr. 2001. [10] B. H. McCormick, D. M. Mayerich, and M. Wiercigroch, “Nanomachining for high-resolution scanning of mammalian brain microstructure,” in Proceedings, 11th International Conference on Fracture, Turin, Italy, March 2005. [11] Rafael C. Gonzalez and Richard E. Woods, Digital Image Processing, Prentice Hall, 2nd edition, 2002. [12] F. T. Kuan, A. A. Sawchuk, T. C. Strand, and P. Chavel, “Adaptive noise smoothing filter for images with signaldependent noise,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 7, no. 2, pp. 165–177, 1985.
Fig. 1. Knife-Edge Scanning Microscopy. the above limits on tissue volume. Captured KESM images are subject to various forms of noise and artifacts due to the imaging methodology, some of which are common to other forms of microscopy, and some of which are unique to KESM. In this paper, we present methods for processing KESM images in order to remove noise and artifacts, thus enabling more useful volumetric datasets for subsequent analysis. 2. KNIFE-EDGE SCANNING MICROSCOPY KESM datasets are created by concurrently cutting and imaging embedded tissue samples. A diamond knife ultramicrotome is used to cut thin sections of the sample while simultaneously the top facet of the knife is imaged by an optical microscope (fig. 1). Illumination is provided through the diamond knife by a liquid-optic illuminator. Imaging is performed by a high sensitivity line-scan camera mounted at the end of the optical train of the microscope. Camera firing is keyed to the digitally encoded position of the tissue under the diamond knife. Therefore, as the tissue sample is translated using a linear stage, the camera records sequential lines of data as the section moves over the knife surface. In order to obtain the most information from the trans-
mitted light, Nikon water immersion objectives are used for their larger numerical aperture. For this reason, all cutting is performed under water. In order to keep debris from building up on the knife surface, a current is induced across the knife edge by applying suction at the rear of the knife. This causes sections to be pulled upwards towards the base of the knife, away from the field of view (FOV) of the objective. This imaging process results in large volumes of volumetric data. At the resolutions we work with (typically 3001000nm in each dimension), a typical single pass over a specimen block produces an image of size approximately 2048 x 9000 pixels. Here the x-axis is along the edge of the knife; thus the x-resolution is the resolution of the line-scan camera that images the tip of the knife blade. The y-axis in any one slice is dependent on the length of the slice taken. Since each slice is taken at constant speed (though subsequent slices might differ in speed), the y-axis is also equivalent to time. Subsequent slices can be obtained adjacent to or below that slice; we currently have scanned more than 1000 layers deep. The resulting dataset can be massive, totalling several terabytes for a single organ of a small animal.
Fig. 2. The overall intensity shift along the x-axis is due to knife misalignment and two knife defects leave streaks in the image (arrows).
3. KESM VOLUME NOISE KESM exhibits several types of noise common to and different from traditional microscopy. These generally involve small lighting irregularities due to camera alignment, illumination frequency, and knife vibration during the cutting process.
Fig. 3. High frequency chatter is visible throughout the image and particularly bad cases (arrows) may cause loss of information.
3.1. Lighting Defects
Lighting defects such as those listed above form regular patterns in the output data that are easily removed from any given image. Unfortunately, irregular illumination tends to occur in KESM images due to knife vibration. Knife chatter is well known in machining [8, 9, 10], but usually is not an issue in imaging, since other imaging techniques do not image data as it is being cut. We use several mechanical techniques to reduce knife vibration, including increasing the mechanical stiffness of the specimen and cutting tool and randomizing cutting velocities in order to prevent reinforcement at any frequency. Even with these precautions, knife vibration is visible in the resulting image as changes in illumination along the y-axis (fig. 3). When using KESM it is important to sample as close to the edge of the knife as possible in order to get the best alignment between sections as well as to ensure that the section is coherent and not warped or torn due to water current or knife friction. Sampling at the very tip of the knife, however, makes knife vibration more visible since light intensity quickly drops past the edge of the knife. Knife vibration is therefore visible as dark horizontal stripes across the image which, due to misalignments described above, may or may not be continuous along the x-axis. In addition, particularly severe occurrences of chatter can cause loss of data.
The most visible lighting defect involves a variation in illumination across the x-axis. There are two main sources: • Small misalignments between the camera and the surface of the knife. This misalignment of the camera results in a steady change in the overall illumination across the image. • Defects (e.g. chips and uneven areas) on the surface of the knife edge itself. The defects cause refraction and reflection variations at the knife surface, resulting in uneven lighting across the knife edge. This is visible in images as brighter or darker strips that extend in the y direction (fig. 2). Regular lighting defects are also visible along the y-axis. The major source is: • The illumination power source can cause fluctuations in the illumination over time. This produces an oscillating fluctuation in illumination creating visible stripes in the image. Depending on the sampling rate of the image, the frequency of the fluctuations changes in the image space however they are constant over the time domain.
3.2. Knife Chatter
4. KESM IMAGE PROCESSING TECHNIQUES We have assembled several known imaging algorithms to help remove lighting irregularities. New methods are also demonstrated that take advantage of the unique noise found in KESM images in order to better prepare the volumetric datasets for segmentation. In addition, we specifically focus on image processing algorithms that require only local information in an image so that processing can be distributed across several systems for faster results. Such parallelism is important for maintaining the high data rate that gives KESM an advantage over other forms of microscopy. 4.1. Light Normalization Equalizing the illumination across the x-axis of the image is the first step in removing noise. Since these irregularities are due to misalignment of the camera or artifacts in the lighting pipeline, they are repeated over every sample in every image. For example, if an artifact present in the image is due to a defect in the knife (such as a chip in the knife surface), this artifact will be present in every sample at a given x value in the image. This type of artifact is also visible even when tissue is not being sectioned. The constant presence of this type of artifact allows a base sample without tissue to be imaged at any time during the cutting process. Since these variations of lighting are constant during the sectioning process, each sample taken along the y-axis can be normalized using this base light vector. Note that such base information might need to be taken at regular intervals, since a knife might gradually develop surface defects, or gradually move out of alignment. Cyclical illumination artifacts produced by high-frequency noise from the light source are less consistent between images. Since the initial sample can take place at any point during the illuminator’s frequency fluctuation, a phase shift is observed along the y-axis. We compensate for this by sectioning slightly less tissue than the knife and objective allows. This leaves a small portion of the knife edge visible without any interfering tissue. A sample line of pixels along the edge of the image can then be used to normalize light frequency fluctuations along the y-axis. 4.2. Removing Chatter Artifacts Knife chatter artifacts are more difficult to remove from KESM images. Although they are lines that extend along the xaxis, they tend to be discontinuous and not perfectly (though nearly) horizontal. This precludes scaling the line by any given scalar value, such as the mean of the current sample. Local smoothing is a method often used to eliminate highfrequency noise [11, 12]. However, it is difficult to apply these techniques to KESM data since most of the information is high frequency and low contrast. Instead, we scale the value of a pixel in the sample by the mean of a small window of pixels surrounding the current pixel in the sample. As this
Fig. 4. Original mouse brain stem section with close-up. window moves across the image, it removes streaks caused by tissue folding and knife vibration by scaling the intensity values up to match other pixels in the sample. This preserves the high-frequency details, providing that there was very little data lost due to the intensity shift, and removes the streaks making segmentation a simpler matter. 5. RESULTS We have applied these techniques to many different KESM data sets and have come to rely on them as a pre-processing step before segmentation. Here we present these methods for processing Nissl-stained mouse brain stem and cerebellum. This dataset was selected because of the high frequency chatter in the cerebellum and low contrast in the spinal cord (fig. 4 - 6). 6. REFERENCES [1] J. B. Pauley, Ed., Handbook of Biological Confocal Microscopy, Plenum Press, New York, 1995. [2] Winfried Denk, James H. Strickler, and Watt W. Webb, “Two-photon laser scanning fluorescence microscopy,” Science, vol. 248, no. 4951, pp. 73–76, Apr. 1990.
Fig. 5. Removal of lighting and knife irregularities. [3] Winfried Denk and Heinz Horstmann, “Serial blockface scanning electron microscopy to reconstruct threedimensional tissue nanostructure,” PLoS Biology, vol. 2, no. 11, pp. e329, Nov. 2004. [4] P. Greenspan, E. P. Mayer, and S. D. Fowler, “Nile red: a selective fluorescent stain for intracellular lipid droplets,” The Journal of Cell Biology, vol. 100, pp. 965–973, 1985. [5] M. Chalfie, Y. Tu, G. Euskirchen, W. W. Ward, and D. C. Prasher, “Green fluorescent protein as a marker for gene expression,” Science, vol. 263, no. 5148, pp. 802–805, Feb. 1994. [6] Tatsuyuki Takada, Keiko Iida, Takeo Awaji, Kazumi Itoh, Riichi Takahashi, Akiko Shibui, Kenichi Yoshida, Sumio Sugano, and Gozoh Tsujimoto, “Selective production of transgenic mice using green fluorescent protein as a marker,” Nature Biotechnology, vol. 15, no. 5, pp. 458, May 1997. [7] Bruce McCormick, “Knife-edge scanning microscope (kesm 1.5): Optics and cameras,” Tech. Rep., Department of Computer Science, Texas A&M University, College Station, TX, Oct. 2006.
Fig. 6. Final pass for the reduction of knife chatter. High resolution views are taken from the same image as fig. 5. Images are shown after removal of light artifacts (left) and after removal of chatter (right). [8] R. Phillips, “Diamond knife ultra microtomy of metals and the structure of microtomed sections,” British Journal of Applied Physics, vol. 12, pp. 554–558, Oct. 1961. [9] Marian Wiercigroch and Erhan Budak, “Sources of nonlinearities, chatter generation and suppression in metal cutting,” Philosophical Transactions of the Royal Society A: Mathematical, Physical and Engineering Sciences, vol. 359, no. 1781, pp. 663–693, Apr. 2001. [10] B. H. McCormick, D. M. Mayerich, and M. Wiercigroch, “Nanomachining for high-resolution scanning of mammalian brain microstructure,” in Proceedings, 11th International Conference on Fracture, Turin, Italy, March 2005. [11] Rafael C. Gonzalez and Richard E. Woods, Digital Image Processing, Prentice Hall, 2nd edition, 2002. [12] F. T. Kuan, A. A. Sawchuk, T. C. Strand, and P. Chavel, “Adaptive noise smoothing filter for images with signaldependent noise,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 7, no. 2, pp. 165–177, 1985.
