simulation of a tire inspection system - CiteSeerX
SIMULATION OF A TIRE INSPECTION SYSTEM W. H. GRAY1, C. DUMONT1,2, and M. A. ABIDI1 1
2
IRIS Laboratory, University of Tennessee, Knoxville, TN, 37996 U.S.A. Le2I laboratory, IUT Le Creusot, University of Burgundy, Le Creusot, 71200, France
Abstract - Simulation is an important aspect to consider when designing an inspection system based upon machine vision. The tire inspection system uses thermal cameras to inspect the surface of a tire. The simulation of a tire inspection system can yield an optimal set of parameters for a thermal camera, or given a specific thermal camera provide information on the optimal placement of the camera. Since the design process takes place by simulation the design can be validated before committing to the expense of new equipment. The simulation can also be used to map acquired thermal data back onto a model of a tire to aid in the inspection process. 1. INTRODUCTION Simulation is a key component for efficient machine vision building. A quality control system by artificial vision aims at capturing information about an object of interest in order to decide whether or not the object is defective. A defect can be a dust particle located under the metallic layer of a cosmetic product [1] or a scratch on the surface of a metallic object [2][3]. A defect can also be a smooth modification of the object surface [4]. Defects are to be detected using specific sensors capturing visible and/or non-visible information about the object to be inspected. Sensors can be associated with a lighting system in some cases to improve defect detection. For other nonvisible sensors, the use of supplementary equipment is not required: the thermal information taken from an infrared camera can suffice to reveal a defect on the surface of the object being inspected. Camera placement is a key issue for efficient defect detection. For any vision system, the placement of the camera(s) determines the effectiveness of the defect detection: i.e. a black and white camera associated with a lighting system can be set with respect to an object so that the defects clearly appear in the image [5]. Many applications of quality control tend to
optimize the camera placement using a heuristic approach. This approach consists in approaching the optimal solution with an empirical method when the vision system is being built [3]. Although this approach does not rely on any theories, it allows construction of robust and effective vision systems. Other applications of quality control attempt to formalize or simulate the vision system in order to optimize its parameters. Hence, a lighting system can be optimally chosen and placed with respect to the camera in order to reveal defects that are located on the surface of metallic objects (see [1][2][5][6]). In the application described in [1][7][8], the lighting system is complex and requires to be characterized before its implementation. In this application a mathematical model of the vision system is built and an optimal set of parameters is chosen according to an effectiveness function based on the intensity of a defect in an image. A good simulation relies on an accurate model of reality [1][9][10][11]. A model of the type of sensor to be used in the vision system is built. Simulation consists of building a virtual vision system that simulates the camera(s) taking images of objects according to the camera model. A complete system is built by integrating the camera model and the object to be inspected into a complete virtual vision system. A simulation mimics not only the image capture, but also the mechanical system that controls object rotation and support. Furthermore, it can synchronize actions between each other: for example the image capture is performed after receiving a signal emitted when the object being inspected has completed a full rotation. Simulation allows low cost and fast design of vision systems (see [1][9][10][11]). For complex vision systems involving heavy equipment, the cost of design is a critical issue. High performance of the vision system has to be proven before making a purchase of very expensive equipment or sensors such as a thermal camera. Simulation allows building of very complex system with a cost of development
relying only on salaries and use of computers and software. Simulation provides engineers a means for easy modification of both the vision system and the algorithms controlling object inspection. The camera placement can be optimized off-line before construction of the prototype. The algorithms are studied and optimized to meet the needs and requirements necessary for future implementation and use of the inspection system. The goal of any imaging system is to provide users with quality data about an object. In this case the object is a tire with a specific thermal signature. The setup of such a system relies on the interrelation between several key thermal camera parameters and the positioning of the thermal camera relative to the tire in the system. The inclusion of 3D modeling of the tire, and the subsequent mapping of thermal data back onto the tire model should provide the user with a level of interaction not possible with 2D imaging alone. The theoretical information gained from such a simulation can be used to determine the best possible detection setup according to the users needs and resources.
system a high quality tire model is used and is shown in Figures 1 and 2.
Figure 1: Detailed Tire Model
2. SYSTEM DESIGN METHODOLOGY A. Interrelated Parts The relationships between the parts of machine vision system must be taken into account for that system to be effective. The most common example in quality control is the heuristic approach where the positioning of sensors follows an empirical method. The users setting up the system take the relationships between the parts of the machine vision system into account. While this approach can be quite effective, it typically involves moving expensive and/or delicate equipment, something that should be avoided if at all possible. The accurate simulation of a machine vision system can provide a solution to this problem. Tire size, camera parameters, and camera positioning can all be used to determine an optimal placement of the camera(s) relative to the tire. This simulation will provide the users a system that is already optimized before it is set up. B. Modeling of Parts Any simulation of a machine vision system is only as good as its individual models. To facilitate creating a realistic tire inspection
Figure 2: Wireframe of Detailed Tire Model The tire model is to be placed in a scene simulating an endurance drum (dynamometer) test machine. The dynamometer test machine consists of three major components: a steel drum to rotate the tire against, the tire mount, and a hydraulic ram to keep the tire in contact with the drum. The machine allows a tire to be rotated at a given speed and load for lengthy amounts of time. The modeling of the dynamometer setup was deemed to have secondary importance, as the thermal data from the tire is what the users desire. A coarse dynamometer model was developed in order to show the general orientation of the setup, and where a thermal camera could likely be mounted. This setup is shown in Figure 3.
3. THERMAL INSPECTION RESULTS A. 2D/3D Imaging
Figure 3: Dynamometer Model The model of the thermal camera used was a pinhole [12] model used in many applications when cameras focus at infinity. This model is used in this paper to model an infrared camera and its optical lenses. The camera has several internal parameters adjustable by the user, i. e. resolution, field of view, and aspect ratio. In addition the user can specify the position of the camera relative to the tire.
The initial goal of this project was to provide a way of capturing the thermal signature of a tire. To accomplish this goal a model of the tire model in Figure 4 was used in an Open Inventor scene. A camera with adjustable parameters was placed in the same scene, and various images were taken from the viewpoint of the camera. A sample representation of camera viewpoints is shown in Figure 6, and the images taken from those viewpoints are shown in Figures 7-9. Viewpoint 1
Viewpoint 2
C. Thermal Data Acquisition As can be seen from Figure 1, there is no thermal data known about the tire model. In order to build a system to test thermal data acquisition it is imperative to have a model that actually reflects real world thermal data. To achieve this requirement, the tire models geometry was used, but the color was altered. Instead of retaining its uniform dark gray color, a program was written to encode the surface with a full range of colors in order to mimic a real world thermal signature. A sample tire model with pseudo thermal data is shown in Figure 4.
Viewpoint 3 Figure 6: Orientation of Camera Viewpoints
Figure 7: Image from Viewpoint 1
Figure 4: Tire Model with Thermal Information Figure 8: Image from Viewpoint 2
Figure 9: Image from Viewpoint 3
Figure 10: Surface Mesh
As can be seen from the preceding figures, a high quality thermal image can be obtained from the theoretical tire model. These images can then be inspected individually by the user of the system. While this method has proven itself quite capable, the decision was made to incorporate the 3D model of the tire back into the display process. Each of the individual images generated from the thermal model can be mapped back to the original model using texture mapping. The program can decide which texture maps to use based upon what regions of the tire are being displayed. B. 2.5D Imaging While the previous approach provides many good results, it also assumes several items that can have an effect on a real world system. The first is that a sufficiently detailed model of the tire is available to the users. The second assumption is that the tire being examined suffers from no manufacturing defects that may alter the geometry of the tire. The solution that has been found to combat these problems is to not only take thermal images at the various viewpoints, but also to take range images at the same viewpoints. The range information can be used to generate a 2.5D surface mesh of the tire that will provide information about the geometry of the tire. This information can then be used in conjunction with the thermal image taken from the same location. Figures 10 and 11 show a pair of range and thermal information taken from the same viewpoint (the surface mesh is rotated slightly to show its 2.5D nature).
Figure 11: Thermal Image The thermal image can then be used as a texture map for the surface mesh rather than the original 3D model. This approach minimizes the amount of knowledge the user must have concerning the geometry of the tire such as tread patterns or physical defects. An example using the surface mesh from figure 10 and the thermal image from figure 11 is shown in figure 12.
Figure 12: Surface Mesh with Thermal Texture Since the data in figure 12 is 2.5D it can be rotated and examined from many different angles, something not possible with 2D images
alone. This approach also intrinsically ties the thermal information to a location on the tire, something that can be confusing if looking at 2D images alone. Multiple range scans can be merged together using zippering or a volumetric approach. These combined range scans can form a reconstructed 3D tire model that is textured using multiple thermal images. Users can then inspect this textured 3D tire model.
4. CONCLUSION Simulation is an important aspect to consider when designing an inspection system based upon machine vision. A thermal tire inspection system that takes into account camera parameters and location has been described. Results from simulated thermal cameras have been shown. The benefits of incorporating thermal data with range data have been described and illustrated. The promise of having a fully reconstructed 3D tire model with associated thermal textures has been described.
5. REFERENCES [1] D. Aluse, "Systeme de detection et de caracterisation de defauts d'aspects sur des surfaces parfaitement specu;aire et non planes: application au controle de produits destines a l'emballage cosmetique", Ph.D. dissertation, University of Burgundy, France, Nov. 1998 [2] C. Coulot, "Etude de l'eclairage de surfaces metalliques pour la vision artificielle: application au controle dimensionnel", Ph.D. dissertation, University of Burgundy, France, June 1997 [3] H. Jender, "Contrôle temps réel par vision artificielle de tubes métalliques en défilement continu", Ph.D. dissertation, University of Burgundy, France, 1993 [4] D. Perard, J. Beyerer, "Three-dimensional measurement of free-form surfaces with a structured-lighting reflection technique",
SPIE conference on machine vision application in industrial inspection, Pittsburgh (USA), 1997, vol. 3204, pp. 7480 [5] D. Aluze, C. Coulot, F. Meriaudeau, P. Gorria, C. Dumont, "Machine vision for the control of reflecting non plane surface", Journal of the machine vision association, Society of Manufacturing Engineers, 1998, Vol. 14, No 3, pp. 1-4. 1D-2D [6] C. Coulot, S. Kohler-Hermmerlin, C. Dumont, D. Aluze, B. Lamalle, "Simulation of lighting for an optimal inspection of metallic-objects", AIM'97, Tokyo (Japan), 16-20 June 97 [7] D. Aluze, C. Dumont, P. Gorria, M. A. Abidi, "Machine Vision for controlling reflective 3-D Objects", Society of Manufacturing Engineers, Conference on Applied Machine Vision, Nashville (USA), May 1998, pp 45-62 [8] D. Aluze, C. Coulot, F. Meriaudeau, P. Gorria, C. Dumont, "Machine vision for the control of reflecting non plane surfaces", SPIE, Pittsburgh (USA), Machine vision application in industrial inspection, 15-17 Oct 1997, pp 180-186 [9] L. M. Wong, C. Dumont and M. A. Abidi, "An Algorithm for Finding the Next Best View in Object Reconstruction", Photonics EAST, Intelligent System and Advanced Manufacturing 98, SPIE Conference on Sensor Fusion and Decentralized Control in Robotics Systems, Boston (USA), November 1998, 3523, pp 191-200 [10] L. M. Wong, C. Dumont, and M. A. Abidi, "Determining Optimal Sensor Poses in 3-D Object Inspection", Conference on Quality Control By Artificial Vision, Takamatsu (Japan), Nov. 98, ISBN: 4-921073-01-5, pp 371-377 [11] L. D. Han, M. Qureshi, M. A. Abidi, H. Gray, Truck rollover warning system simulations, accepted to ISATA, Vienna, Austria, 14-18 June 99. [12] R. C. Gonzales, P. Wintz, Digital image processing, Addison-Wesley Publishing Company, 1992, 2nd edition
2
IRIS Laboratory, University of Tennessee, Knoxville, TN, 37996 U.S.A. Le2I laboratory, IUT Le Creusot, University of Burgundy, Le Creusot, 71200, France
Abstract - Simulation is an important aspect to consider when designing an inspection system based upon machine vision. The tire inspection system uses thermal cameras to inspect the surface of a tire. The simulation of a tire inspection system can yield an optimal set of parameters for a thermal camera, or given a specific thermal camera provide information on the optimal placement of the camera. Since the design process takes place by simulation the design can be validated before committing to the expense of new equipment. The simulation can also be used to map acquired thermal data back onto a model of a tire to aid in the inspection process. 1. INTRODUCTION Simulation is a key component for efficient machine vision building. A quality control system by artificial vision aims at capturing information about an object of interest in order to decide whether or not the object is defective. A defect can be a dust particle located under the metallic layer of a cosmetic product [1] or a scratch on the surface of a metallic object [2][3]. A defect can also be a smooth modification of the object surface [4]. Defects are to be detected using specific sensors capturing visible and/or non-visible information about the object to be inspected. Sensors can be associated with a lighting system in some cases to improve defect detection. For other nonvisible sensors, the use of supplementary equipment is not required: the thermal information taken from an infrared camera can suffice to reveal a defect on the surface of the object being inspected. Camera placement is a key issue for efficient defect detection. For any vision system, the placement of the camera(s) determines the effectiveness of the defect detection: i.e. a black and white camera associated with a lighting system can be set with respect to an object so that the defects clearly appear in the image [5]. Many applications of quality control tend to
optimize the camera placement using a heuristic approach. This approach consists in approaching the optimal solution with an empirical method when the vision system is being built [3]. Although this approach does not rely on any theories, it allows construction of robust and effective vision systems. Other applications of quality control attempt to formalize or simulate the vision system in order to optimize its parameters. Hence, a lighting system can be optimally chosen and placed with respect to the camera in order to reveal defects that are located on the surface of metallic objects (see [1][2][5][6]). In the application described in [1][7][8], the lighting system is complex and requires to be characterized before its implementation. In this application a mathematical model of the vision system is built and an optimal set of parameters is chosen according to an effectiveness function based on the intensity of a defect in an image. A good simulation relies on an accurate model of reality [1][9][10][11]. A model of the type of sensor to be used in the vision system is built. Simulation consists of building a virtual vision system that simulates the camera(s) taking images of objects according to the camera model. A complete system is built by integrating the camera model and the object to be inspected into a complete virtual vision system. A simulation mimics not only the image capture, but also the mechanical system that controls object rotation and support. Furthermore, it can synchronize actions between each other: for example the image capture is performed after receiving a signal emitted when the object being inspected has completed a full rotation. Simulation allows low cost and fast design of vision systems (see [1][9][10][11]). For complex vision systems involving heavy equipment, the cost of design is a critical issue. High performance of the vision system has to be proven before making a purchase of very expensive equipment or sensors such as a thermal camera. Simulation allows building of very complex system with a cost of development
relying only on salaries and use of computers and software. Simulation provides engineers a means for easy modification of both the vision system and the algorithms controlling object inspection. The camera placement can be optimized off-line before construction of the prototype. The algorithms are studied and optimized to meet the needs and requirements necessary for future implementation and use of the inspection system. The goal of any imaging system is to provide users with quality data about an object. In this case the object is a tire with a specific thermal signature. The setup of such a system relies on the interrelation between several key thermal camera parameters and the positioning of the thermal camera relative to the tire in the system. The inclusion of 3D modeling of the tire, and the subsequent mapping of thermal data back onto the tire model should provide the user with a level of interaction not possible with 2D imaging alone. The theoretical information gained from such a simulation can be used to determine the best possible detection setup according to the users needs and resources.
system a high quality tire model is used and is shown in Figures 1 and 2.
Figure 1: Detailed Tire Model
2. SYSTEM DESIGN METHODOLOGY A. Interrelated Parts The relationships between the parts of machine vision system must be taken into account for that system to be effective. The most common example in quality control is the heuristic approach where the positioning of sensors follows an empirical method. The users setting up the system take the relationships between the parts of the machine vision system into account. While this approach can be quite effective, it typically involves moving expensive and/or delicate equipment, something that should be avoided if at all possible. The accurate simulation of a machine vision system can provide a solution to this problem. Tire size, camera parameters, and camera positioning can all be used to determine an optimal placement of the camera(s) relative to the tire. This simulation will provide the users a system that is already optimized before it is set up. B. Modeling of Parts Any simulation of a machine vision system is only as good as its individual models. To facilitate creating a realistic tire inspection
Figure 2: Wireframe of Detailed Tire Model The tire model is to be placed in a scene simulating an endurance drum (dynamometer) test machine. The dynamometer test machine consists of three major components: a steel drum to rotate the tire against, the tire mount, and a hydraulic ram to keep the tire in contact with the drum. The machine allows a tire to be rotated at a given speed and load for lengthy amounts of time. The modeling of the dynamometer setup was deemed to have secondary importance, as the thermal data from the tire is what the users desire. A coarse dynamometer model was developed in order to show the general orientation of the setup, and where a thermal camera could likely be mounted. This setup is shown in Figure 3.
3. THERMAL INSPECTION RESULTS A. 2D/3D Imaging
Figure 3: Dynamometer Model The model of the thermal camera used was a pinhole [12] model used in many applications when cameras focus at infinity. This model is used in this paper to model an infrared camera and its optical lenses. The camera has several internal parameters adjustable by the user, i. e. resolution, field of view, and aspect ratio. In addition the user can specify the position of the camera relative to the tire.
The initial goal of this project was to provide a way of capturing the thermal signature of a tire. To accomplish this goal a model of the tire model in Figure 4 was used in an Open Inventor scene. A camera with adjustable parameters was placed in the same scene, and various images were taken from the viewpoint of the camera. A sample representation of camera viewpoints is shown in Figure 6, and the images taken from those viewpoints are shown in Figures 7-9. Viewpoint 1
Viewpoint 2
C. Thermal Data Acquisition As can be seen from Figure 1, there is no thermal data known about the tire model. In order to build a system to test thermal data acquisition it is imperative to have a model that actually reflects real world thermal data. To achieve this requirement, the tire models geometry was used, but the color was altered. Instead of retaining its uniform dark gray color, a program was written to encode the surface with a full range of colors in order to mimic a real world thermal signature. A sample tire model with pseudo thermal data is shown in Figure 4.
Viewpoint 3 Figure 6: Orientation of Camera Viewpoints
Figure 7: Image from Viewpoint 1
Figure 4: Tire Model with Thermal Information Figure 8: Image from Viewpoint 2
Figure 9: Image from Viewpoint 3
Figure 10: Surface Mesh
As can be seen from the preceding figures, a high quality thermal image can be obtained from the theoretical tire model. These images can then be inspected individually by the user of the system. While this method has proven itself quite capable, the decision was made to incorporate the 3D model of the tire back into the display process. Each of the individual images generated from the thermal model can be mapped back to the original model using texture mapping. The program can decide which texture maps to use based upon what regions of the tire are being displayed. B. 2.5D Imaging While the previous approach provides many good results, it also assumes several items that can have an effect on a real world system. The first is that a sufficiently detailed model of the tire is available to the users. The second assumption is that the tire being examined suffers from no manufacturing defects that may alter the geometry of the tire. The solution that has been found to combat these problems is to not only take thermal images at the various viewpoints, but also to take range images at the same viewpoints. The range information can be used to generate a 2.5D surface mesh of the tire that will provide information about the geometry of the tire. This information can then be used in conjunction with the thermal image taken from the same location. Figures 10 and 11 show a pair of range and thermal information taken from the same viewpoint (the surface mesh is rotated slightly to show its 2.5D nature).
Figure 11: Thermal Image The thermal image can then be used as a texture map for the surface mesh rather than the original 3D model. This approach minimizes the amount of knowledge the user must have concerning the geometry of the tire such as tread patterns or physical defects. An example using the surface mesh from figure 10 and the thermal image from figure 11 is shown in figure 12.
Figure 12: Surface Mesh with Thermal Texture Since the data in figure 12 is 2.5D it can be rotated and examined from many different angles, something not possible with 2D images
alone. This approach also intrinsically ties the thermal information to a location on the tire, something that can be confusing if looking at 2D images alone. Multiple range scans can be merged together using zippering or a volumetric approach. These combined range scans can form a reconstructed 3D tire model that is textured using multiple thermal images. Users can then inspect this textured 3D tire model.
4. CONCLUSION Simulation is an important aspect to consider when designing an inspection system based upon machine vision. A thermal tire inspection system that takes into account camera parameters and location has been described. Results from simulated thermal cameras have been shown. The benefits of incorporating thermal data with range data have been described and illustrated. The promise of having a fully reconstructed 3D tire model with associated thermal textures has been described.
5. REFERENCES [1] D. Aluse, "Systeme de detection et de caracterisation de defauts d'aspects sur des surfaces parfaitement specu;aire et non planes: application au controle de produits destines a l'emballage cosmetique", Ph.D. dissertation, University of Burgundy, France, Nov. 1998 [2] C. Coulot, "Etude de l'eclairage de surfaces metalliques pour la vision artificielle: application au controle dimensionnel", Ph.D. dissertation, University of Burgundy, France, June 1997 [3] H. Jender, "Contrôle temps réel par vision artificielle de tubes métalliques en défilement continu", Ph.D. dissertation, University of Burgundy, France, 1993 [4] D. Perard, J. Beyerer, "Three-dimensional measurement of free-form surfaces with a structured-lighting reflection technique",
SPIE conference on machine vision application in industrial inspection, Pittsburgh (USA), 1997, vol. 3204, pp. 7480 [5] D. Aluze, C. Coulot, F. Meriaudeau, P. Gorria, C. Dumont, "Machine vision for the control of reflecting non plane surface", Journal of the machine vision association, Society of Manufacturing Engineers, 1998, Vol. 14, No 3, pp. 1-4. 1D-2D [6] C. Coulot, S. Kohler-Hermmerlin, C. Dumont, D. Aluze, B. Lamalle, "Simulation of lighting for an optimal inspection of metallic-objects", AIM'97, Tokyo (Japan), 16-20 June 97 [7] D. Aluze, C. Dumont, P. Gorria, M. A. Abidi, "Machine Vision for controlling reflective 3-D Objects", Society of Manufacturing Engineers, Conference on Applied Machine Vision, Nashville (USA), May 1998, pp 45-62 [8] D. Aluze, C. Coulot, F. Meriaudeau, P. Gorria, C. Dumont, "Machine vision for the control of reflecting non plane surfaces", SPIE, Pittsburgh (USA), Machine vision application in industrial inspection, 15-17 Oct 1997, pp 180-186 [9] L. M. Wong, C. Dumont and M. A. Abidi, "An Algorithm for Finding the Next Best View in Object Reconstruction", Photonics EAST, Intelligent System and Advanced Manufacturing 98, SPIE Conference on Sensor Fusion and Decentralized Control in Robotics Systems, Boston (USA), November 1998, 3523, pp 191-200 [10] L. M. Wong, C. Dumont, and M. A. Abidi, "Determining Optimal Sensor Poses in 3-D Object Inspection", Conference on Quality Control By Artificial Vision, Takamatsu (Japan), Nov. 98, ISBN: 4-921073-01-5, pp 371-377 [11] L. D. Han, M. Qureshi, M. A. Abidi, H. Gray, Truck rollover warning system simulations, accepted to ISATA, Vienna, Austria, 14-18 June 99. [12] R. C. Gonzales, P. Wintz, Digital image processing, Addison-Wesley Publishing Company, 1992, 2nd edition
