Improved Accurate Extrinsic Calibration Algorithm ... - Semantic Scholar
JOURNAL OF MULTIMEDIA, VOL. 8, NO. 6, DECEMBER 2013
777
Improved Accurate Extrinsic Calibration Algorithm of Camera and Two-dimensional Laser Scanner Jianlei Kong, Lei Yan*, Jinhao Liu, Qingqing Huang, and Xiaokang Ding College of Technology, Beijing Forestry University, Beijing, China Email: [email protected], [email protected], [email protected], [email protected], [email protected] *Corresponding Author
Abstract—In the object detecting system composed of a camera and a two-dimensional laser scanner (2DLS), the extrinsic calibration is an essential step to operate properly. However, the edge points of object cannot be detected accurately by 2DLS due to angular resolution limit, which caused loss of accuracy of the extrinsic calibration between camera and 2DLS. In this paper a new algorithm is proposed to solve this problem. Firstly, the least-squares method was applied to linearly fit laser data located on the scanning plane of the three-dimensional rectangular calibration object. Secondly, half of the value of angular resolution was used as the angular incremental component of the laser edge points of object to improve the accuracy of laser data captured by 2DLS. Finally a general optimization was proposed to refine the extrinsic calibration parameters of camera-2DLS by minimizing the re-projection error between those adjusted laser points and their corresponding projections under the image coordinate of the camera. Compared with related methods, experimental results showed that the accuracy of extrinsic calibration between camera and 2DLS was significantly improved. Index Terms—Two-dimensional Laser Scanner; Accuracy of Extrinsic Calibration; Ranging Error; Step-angular Error; Camera
I.
INTRODUCTION
In the recent studies, combination of a camera and a two-dimensional laser scanner (2DLS) has been widely used to provide applications in detection [1], autonomous vehicle navigation [2], robot motion and control [3] and 3D visualization measurement [4]. In the camera-2DLS object detecting system, 2DLS provides real time ranging measurements in a wide angular field while a camera provides rich visual information. In order to effectively fusion the data from the camera and 2DLS, it is important to know their relative position and orientation from each other, which affects the geometric interpretation of their measurements. The calibration between camera and 2DLS can be decomposed into intrinsic calibration parameters and extrinsic calibration parameters. The extrinsic calibration parameters are the rotation and translation of some fiducial coordinate systems relative to the sensors. And
© 2013 ACADEMY PUBLISHER doi:10.4304/jmm.8.6.777-783
the intrinsic calibration parameters, such as the calibration matrix of a camera, affect how the sensor samples the scene. This work assumes the intrinsic calibration is known as the perspective pinhole model, and focuses on the accuracy improvement of the extrinsic calibration, which is critical to be solved. （ R c ， Tc ）
u
K
p(u,v)
Yc Xc
Pc v Image in CCD Camera
Zc
CCD Camera
Zw Pw
Xw
Yw
（ Φ， Δ ） Yl
Pl( x l,Y l) a1
Zl
Pl a
a2
Xl
Laser Data in 2DLS
Yl Xl
2 D Laser Scanner(2D LS)
Figure 1. Schematic of the extrinsic calibration problem of camera-2DLS. The three-dimensional rectangular calibration object was posed in the both views of the camera and 2DLS. This paper reduces the ranging error and the step-angular error to improve the laser data used for extrinsic calibration. The goal of this paper is to study a new algorithm to improve the accuracy of extrinsic calibration by refining the rotation matrix and the transformed vector with those modified laser data.
A large number of researchers have contributed many works on detecting some specific points or stripes to solve the camera-2DLS extrinsic calibration. Those feature points or shapes were obtained to establish a homogeneous transformation between the 2DLS coordinate and the camera coordinate [5-7]. However, the extraction process of corresponding data pairs was cumbersome and time-consuming, and the procedure of extrinsic calibration was inaccurate. To simplify the calibration process, the most widely used method was proposed by Zhang and Pless in [8], and described a camera-2DLS extrinsic calibration procedure where a checkerboard pattern was freely placed in same views simultaneously of the two sensors. For each various positions of the planar pattern, the method constrained the extrinsic parameters by registering the laser scanline on the planar pattern with the estimated pattern plane from the camera image. The solution of equations provided an
778
JOURNAL OF MULTIMEDIA, VOL. 8, NO. 6, DECEMBER 2013
initial estimate for the relative rotation and translation, which was subsequently refined by iterative minimization of the re-projection error. However, this algorithm existed for a problem that it needed a specific orientation of calibration plate to obtain the optimal results, which was difficult to be established. Since the method was based on linear equations which did not directly enforce the rotation and translation matrix. It often leaded to poor initialization of the data pairs corresponding to the 2DLS data and the images viewed by the camera, and the estimation errors of the data pairs would affect the calibration accuracy also. An algorithm for extrinsic calibration has been proposed to establish a coordinate transform relationships by using straight line features by Li in [9]. The method designed a right-angled triangular checkerboard to employ the invisible intersection points of the slice plane obtained by 2DLS with the edges of the checkerboard to set up the constraints equations. The extrinsic parameters are then calibrated by minimizing the algebraic errors between the measured intersections points and their corresponding projections on the image plane of the camera. However, the checkerboard used in this method was unsuitable for intrinsic calibration, which was also not convenient in extrinsic calibration. Another method proposed by F. Vasconcelos in [10] fit lines to the laser depth readings and carried the Euclidean registration of 3 planes with 3 co-planar lines in an optimal and closed-form manner. It is proved for the first time that the alignment of 3 plane-line correspondences had at most 8 solutions, which determined the rotation and translation by solving a standard p3p problem. u p( u, v
O1 O2 ( u0,v
0
)
X f
)
Y
Xc Optical Axis
Oc
Zc Yc
v
Pw(Xw,Yw,Zw)
Zw Xw Ow
Yw
Figure 2. Geometry of perspective projection in camera model. In machine vision system, the transformation of objects from the world coordinates to the pixel coordinates involves four reference coordinate systems: pixel coordinate system, image coordinate system, camera coordinate system, and world coordinate system [13]. This work solves intrinsic calibration parameters with Matlab.
However, the inaccurate measurement of 2DLS caused loss of accuracy of the extrinsic calibration between the camera and 2DLS severely. When a single laser pulse of 2DLS was sent out and reflected by an object surface within the range of the scanner, the elapsed time between emission and reception of the laser pulse served to calculate the distance between object and 2DLS [11]. Since the reflectivity was based on the material and position of the various objects, the ranging error existed in extrinsic calibration of camera-2DLS.This work applied the least-squares method to linearly fit the discretely distributing laser points, which were located on the surface of a three-dimensional rectangular calibration object, as the intersecting lines. Since the laser rays from 2DLS were invisible, the calibration data must be on the edges of calibration objects. However, 2DLS could not © 2013 ACADEMY PUBLISHER
detect the edge points of object accurately because of angular resolution, which also caused the inaccuracy of 2DLS [12]. To solve this problem, half of the value of angular resolution was used as the increased value of step-angular error to modify the laser data points on the edges of object. By minimizing the re-projection error between those adjusted points captured by 2DLS and their corresponding projections on the image coordinate of the camera, the extrinsic calibration parameters were refined with a general optimization. The rest of this paper is as follows: Section 2 presents the principle and notations of extrinsic calibration. Section 3 presents a new algorithm to improve the accuracy of 2DLS for the accurate improvement of extrinsic calibration between the camera and 2DLS. Compared with related methods, the experimental results are described in Section 4. Lastly, Section 5 concludes the algorithm in this paper. II.
THE CALIBRATION PROBLEM
The camera can be described by the perspective pinhole model as shown in Fig.2. A projection of a random edge point from the camera coordinates system Pc [ X c , Yc , Zc ]T in the world coordinate to the image coordinates p [u, v]T can be represented as follows: u X c Zc fu v c.K . Y Z c Z 0 c c c 1 1 0
s fv
u0 v0
0
1
X 0 c Y 0 c Z 0 c 1
(1)
where c is a proportional factor and K is the camera related to the camera’s internal structure. fu, fv are the equivalent focal lengths on u-axe and v-axe of the image coordinate system respectively. Here, (u0,v0) is called the coordinates of the principal point and s is the parameter describing the skewness of the two image axes. This paper applied the Camera Calibration Toolbox in reference [14] to calculate the camera intrinsic matrix with Matlab. A three-dimensional calibration object was viewed by a camera and 2DLS as shown in Fig.3. 2DLS could detect direction and distance measurements of the object lying on a scanning plane parallel to the floor. The 2DLS coordinate system was defined with an origin at 2DLS, and the laser scanning plane was the plane Zl=ŋ, where ŋ is a constant representing the distance from the origin of the 2DLS coordinate system to the origin of the world coordinate system. Supposing an edge point Pw detected by 2DLS was reported as Pl [ D cos(n ), D sin(n ), ]T in a normalized Cartesian coordinate system, where δ is the angular resolution, n is the number of laser beam, and D represents the distance from the center of 2DLS to the calibration object. And Pl was projected in the camera coordinate system Pc as follows:
Pc Pl
(2)
JOURNAL OF MULTIMEDIA, VOL. 8, NO. 6, DECEMBER 2013
779
where is a 3×3 orthonormal matrix representing the rotation matrix corresponding to the 3×1 transformed vector . Combining (1) and (2) leads to the following extend homogeneous equation between the camera coordinate system and the 2DLS coordinate system:
the red lines L1 and L2 as shown in Fig. 4 (a)). The lines represented the projection of the calibration object in the 2DLS coordinate system and were expressed as: w 1
A. X b . (3)
k 1
where A is a 3×12 matrix and b is a 3-vector which can T b uZ k , vZ k , Z k c c c be expressed as , and X is a 9×1 vector representing 12 unknown extrinsic calibration parameters. n pairs of (Pl, p) representing the projections of the same point Pw viewed in the 2DLS and the camera were selected to solve the overdetermined homogeneous equations. When the value of n was greater than 4, the least square solution of (3) was obtained as following:
PLij
w 1
L1 {( xk , yk )
(4)
The details about this matrix computation can be referred in Matrix Computation [15]. This concluded the initial rotate matrix and the transformed vector in extrinsic calibration of camera-2DLS.
k 1
w 1
y k 1
m
2 k
m
k w
k w
m
m
k w
k w
xk2 +yk2 Dk2 }
where (Xk,Yk) represents the coordinate of the actual laser points and Dk gives the distance from the laser center in 2DLS coordinate which satisfies Dk xk2 +yk2 , k defines the laser point on the k-th beam, w stands for the number of laser beam at the minimum distance, a1 and a2 are the slopes of the lines, and b1 and b2 represent the intercepts of lines in 2DLS coordinate system. The intercepts and slopes were obtained by using the method of linear fitting. The least squares resolution was applied to express the linear parameters of two intersecting lines L1 and L2 as following:
(a) (b)
This section described in detail the algorithm to improve the accuracy of extrinsic calibration. A three-dimensional rectangular calibration object was placed at different poses as long as its feature could be detected by a camera and 2DLS. The method of least squares was used to linearly fit laser data of the object scanned by 2DLS for reducing the ranging error. And half of the value of the angular resolution was applied to modify the edge laser points and the laser apexes of the object. Lastly, the adjusted laser data compared with the nonlinear optimization and the general optimization was used to refine and of the extrinsic calibration between camera and 2DLS for accurate improvement. A. Reducing the Ranging Error The combination of the ranging error and the step-angular error reduced the accuracy of extrinsic calibration of camera-2DLS. Hence a new algorithm was proposed to reduce ranging error to improve the accuracy of laser data as shown in Fig.4 (a). A black dot P indicated the actual measurement of the laser points and deviated from the calibration object surface. And the green point P’ represented the adjusted laser data corresponding to P. This work applied the method of least squares to linearly fit laser points of the calibration object captured by 2DLS as two intersecting lines (seeing © 2013 ACADEMY PUBLISHER
(5)
yk a2 xk b2 ,
w 1
w 1
w 1 xk yk xk yk
w 1
k 1 2
, b1
m
m
m
k w
k m
k m 2
k 1
k 1
w 1 w 1 x xk k 1 k 1 w 1
2 k
PROPOSED SCHEME
k 1
2 k
\ L2 {( xk , yk )
a1
III.
a1 xk b1 ,
x +y D } 2 k
w 1
Figure 3. A three-dimensional calibration object in the both views of a camera (a) and 2DLS (b)
w 1
k
a2
k 1
w 1
k
a1 xk k 1
w 1
(6)
m w 1 xk yk xk yk w 1 w 1 xk2 xk k 1 k 1 w 1
y
m
, b2
y
k w
m
k
a2 xk k w
m w 1
Combining (5) and (6), the optimized laser points (green dots in Fig.4) corresponding to the original laser points (black dots in Fig.4) were used to reduce the ranging error for improving accuracy of laser data. B. Reducing the Step-angular Error Taking into consideration that the laser rays were invisible, the calibration data must be on the edges of calibration objects. However, the edge points could not be detected accurately by 2DLS due to angular resolution, which was called as the step-angular error. It was a main reason reducing the accuracy of extrinsic calibration. When the laser rays hit an object edge, the point on the left edge of the calibration object had a real world coordinate as B1= (X1, Y1, Z1), which polar coordinate could be written as B1= (,D). However, since the 2DLS had an angular resolution , a nearby detected point P1= (n,D) on the calibration object was used to replace B1. The step-angle error of the point B1 with respect to the laser point P1 as (see Fig. 4 (b)), where is the included angle between B1 and P1. The original data was refined as B1= (nD). Based on observations and analysis, the value of satisfied two conditions: Firstly, the step-angular error was always less than one. Because if not, the point B1 must also be detected by the laser and
780
JOURNAL OF MULTIMEDIA, VOL. 8, NO. 6, DECEMBER 2013
Three-dimentional Calibration Object
L 1 B1 (n+m-1)δ L2 (n-1)δ
...
(n+ξ)δ
P1
nδ
...
wδ
ξδ
...
Three-dimentional Calibration Object
L1 nδ ( n-1)δ (n+1)δ P P'
...
mδ
L2 B2 (m-1)δ
τδ wδ B3(w+1)δ O
O
2 D Laser Scanner
2 D Laser Scanner
(a) (b) Figure 4. Analysis of ranging error and step-angular error in extrinsic calibration
it should represent the edge instead of P1.The second condition was that the laser points P2 met normal distribution and its mean approaches 1/2, which was chosen as the estimated value of step-angular error. Similarly, the right edge point B2 and the apex point B3 of the calibration object were also chose to calculate the parameters of extrinsic calibration (see Fig.4 (b)). According to the linear relationships of between L1 and L2, the following equations were obtained:
X , Y X D cos n , Y a X b X , Y X D cos n , Y a X b
B1 B2
1
2
1
1
2
1
2
1
2
1
2
1
2
2
A X 3 , Y3 Y3 a1 X 3 b1 , Y3 a2 X 3 b2 B1
X , Y X 1
1
1
D cos
(7)
Using to estimate the increasing value of step-angular error, the accuracy of laser data was improved greatly as well as the accuracy extrinsic calibration in this paper. C. Nonlinear Optimization Based on current adjusted laser data, a nonlinear optimization with the Levenberg-Marquardt method was proposed [16] to reduce the re-projection error between the original laser points and the optimized laser points as following: E2 DLS arg min w1i . B1 L1 , P1i P1i
2
i
w . B2 L2 , P P2i
2
w . B3 L3 , P P
2
i 2
i 3
i 2
i 3
i 3
D. General Optimization The above solution was obtained by modifying the laser data which was not directly related to extrinsic calibration of camera-2DLS. And both the camera and 2DLS had some noises in their outputs which affected the final accuracy of extrinsic calibration. For the further accurate improvement of extrinsic calibration, extrinsic parameters were nonlinearly refined with a general optimization, which was more physically meaningful. A parameter was defined to represent the Euclidean distance of the error from the modified laser points to the corresponding projections in image coordinate as following: E f PL xL , yL , zL , p u, v
Ecam 2 DLS
)
corresponding to the right edge point and P3i corresponding to apex point. B1, B2 and B3 are the modified points of original laser points P1i , P2i , P3i on the
(9)
A nonlinear minimization on E with the Levenberg-Marquardt method was used to refine the rotate matrix and the transformed vector of extrinsic calibration as following:
(8)
where the left edge point B1, the right edge point B2 and the apex point B3 of the calibration object were selected to solve the nonlinear problem. where P1i indicates the left edge point of the calibration-object projected on the i-th pose in 2DLS coordinate system, as same as the P2i
© 2013 ACADEMY PUBLISHER
two intersecting lines . w1i , w2i , w3i indicate the weight of the left edge point, the right edge point and the apex point at i-th pose in whole progress of extrinsic calibration. E2DLS denotes the re-projection error of the original laser points to the optimized laser points. By minimizing the Euclidean distance of the error E2DLS between the original laser points and those adjusted ones to refine laser data, the accuracy of laser data obtained by 2DLS was improved further.
c min i . .K . .PLij pij u, v Z i j c
2
(10)
where the index i represents the pose number of the calibration object, PLij the j-th laser radar point at the i-th poses of the calibration object in 2DLS coordinate, pij u, v the projection corresponding to PLij represents in the image coordinate system, and i is the weight proportion of the i-th pose in the overall data. Combining (9) with (10), a general nonlinear optimization was proposed by minimizing the combination of the error from the original laser points to the optimized laser points
JOURNAL OF MULTIMEDIA, VOL. 8, NO. 6, DECEMBER 2013
and the error from the optimized laser points to image points as follows: Egeneral Ecam 2 DLS E2 DLS
(11)
Here, is a scalar weighting parameter to normalize the relative contribution between Ecam 2 DLS and E2DLS . This requires an initial value of and in extrinsic calibration, which is obtained in the previous section. The general optimization of extrinsic calibration was carried to refine the extrinsic calibration parameters to improve the accuracy of extrinsic calibration significantly. IV.
EXPERIMENTAL RESULTS
A. Comparison The intrinsic matrix of the camera was calculated within the Camera Calibration Toolbox for Matlab proposed in [17] as: 1.6782 1319.0398 6297.7294 K 0 6278.3645 871.8265 . 0 0 1 And the proposed algorithm for the data modifying and the accurate improvement of extrinsic calibration has been implemented in Matlab. Comparing the adjusted laser data with the general optimization, the extrinsic calibration parameters and were refined with the assumption of 10 and c.Zc 2950 as following: [150.9765, 214.3636, 2920]T 0.8708 0.0409 1.0000 0.0044 0.0413 2.0000 0.0000 0.0000 3.0000 A three-dimensional rectangular calibration-object was used for extrinsic calibration in this paper. Each plane of the calibration object was defined by 10×6 grids and the size of the pattern square was 30×30 mm. The calibration object was placed at the distance ranging from 1 meter to 5 meters away from the camera-2DLS system. In order to compare with the Zhang’s algorithm (Zhang (2004)) and the Sattaratnamai’s algorithm (Sattaratnamai (2011) ), Gaussian noises with mean 0 and standard deviation 0.5 pixels were added to the projected image points of edge points and apexes of the calibration object. A range of ±2cm as uniform noise was also added into the ranging measurement of laser points. 100 trials of independent poses were selected to decrease the influence of Gaussian noises. Several contrast parameters were selected to verify high accuracy of calibration with the new algorithm proposed in this paper as showed in Table I. As showed in Table 1, the u-axis average error of this algorithm was limited into 30 pixels, which was increased by 61.4% compared with Zhang (2004) and by 25.8% compared with Sattaratnamai (2011). And the v-axis average error was limited into 3 pixels, which was obviously improved by 47.5% corresponding to Zhang (2004) and by 43.5% corresponding to Sattaratnamai (2011). The u-axis and v-axis average errors were both presenting the extrinsic calibration accurate improvement © 2013 ACADEMY PUBLISHER
781
of the new algorithm in the image coordinate system. Here this work applied images of 2592×1728 pixels to calibration procedure compared with ones of 640×480 pixels in the paper proposed by Zhang (2004) and compared with images of 780×582 pixels and 1920×1080 pixels mentioned in Sattaratnamai (2011), which presented higher resolution of camera in calibration. This caused a large magnitude of the average errors in this paper corresponding to relative works. The three-dimensional error in this algorithm, which presented the calibration error in the ranging measurement, was about 0.991 cm, which was less than the error about 2.49 cm obtained by the method in Zhang (2004) and the error about 1.29 cm corresponding to Sattaratnamai (2011). Similarly, the value of angle error in this new algorithm obtained the smallest value about 0.176 degree in world coordinate system. The minimal results of the three-dimensional error and the angular error obtained by the algorithm showed that, the calibration errors were weakly affected by angles and poses of the calibration object and the extrinsic calibration system of camera and 2DLS in this algorithm was more stable and accurate. Whereas, the results obtained by the method in Zhang (2004) and Sattaratnamai (2011) were both strongly influenced by this angle and poses. TABLE I.
EXPERIMENTAL RESULT OF NEW ALGORITHM COMPARED WITH RELATED WORKS
Methods u-axis average error (pixel) v-axis average error (pixel) Three-dimensional error (cm) Angle error (degree) RMS (pixel) STD
Zhang (2004)
Sattaratnamai (2011)
New algorithm
72.23799022
37.58364553
27.90318309
4.994270337
4.64171778
2.6211516
2.491210195
1.288852679
0.991360647
0.439002248 97.58660382 65.11802068
0.22662919 48.08085658 29.88941575
0.176153146 34.48339795 18.61776107
The Root Mean Square (RMS) of the calibration errors in this algorithm was closed to 34.5 pixel, which was the smallest among the three methods. The value of RMS in this paper showed the obvious improvement by 64.7% compared with Zhang (2004) and by 28.3% corresponding to Sattaratnamai (2011), which indicated the higher precision of the extrinsic calibration. What’s more, the minimum value of the Standard Deviation (STD) of the calibration errors presented that the error distribution was not very discrete and the extrinsic calibration was stable and robust in this paper. A further experiment dealing with the influence of the number of data pairs on the calibration error was performed. The number of the data pairs was varied from 5 to 55 and the calibration error of every number was computed by running 51 trials as showed in Fig.5. Where x-axis represents the number of the data pairs and y-axis represents the calibration error. The new algorithm had the smallest calibration error curve which showed more accurate in extrinsic calibration. What’s more, the error curve of this algorithm reached a stable value at about 15
782
JOURNAL OF MULTIMEDIA, VOL. 8, NO. 6, DECEMBER 2013
pairs against to about 27 pairs in Zhang (2004) and Sattaratnamai (2011), which showed the higher speed in extrinsic calibration of this algorithm. 25
Calibration Error(pixel)
Zhang(2004) Sattaratnamai(2011)
20
New Algorithm 15
10
5
was composed with two planar calibration patterns, which were defined by 10×6 grids. And each square grid was 30 mm ×30 mm. This calibration was designed for intrinsic calibration of camera and extrinsic calibration of camera-2DLS simultaneously for efficiency improvement and time saving. Since the image sensor size was 22.3 mm×14.9 mm and the camera focal length was fixed at 55mm, one pixel in image coordinate system was equal to 0.034027cm in world coordinate system depending on the resolution of image is 72dpi. On real data, the method operates well given reliable calibration parameters of the camera.
0 1
5
10
15
20
25
30
35
40
45
50
55
Number of Data Pairs Figure 5. The influence of the number of data pairs on the calibration error
Three-dimensional Error(m)
0.1
Zhang(2004)
0.09
Sattaratnamai(2011)
0.08
New Algorithm
0.07 0.06 0.05
Figure 7. Projection of the 2DLS points of the calibration object into the images. Red points represented the projection using the extrinsic calibration results obtained with the new algorithm corresponding to blue points obtained by Zhang (2004) and green points obtained by Sattaratnamai (2011). It was obviously showed that the new algorithm had the best performance in extrinsic calibration of camera-2DLS.
0.04 0.03 0.02 0.01 0 3 6 9 12 15 18 21 24 27 30 33 36 39 42 45 48 51 54 57 60 63 66 69 72 75 78 81 84 87 90
Angles of Calibration Object (degree) Figure 6. The influence of the angles of the calibration object on the three-dimensional error
This experiment was performed to reveal the influence of the angles of the calibration object on the three-dimensional error as showed in Fig.6. Where x-axis represents the rotation degree of the calibration object changed from 0 to 90 degree every 3 degree. And the y-axis represents the three-dimensional error. The result of Zhang (2004) and Sattaratnamai (2011) However, the results in the new algorithm were weakly affected by the angles from the calibration object to the 2DLS slice plane. while the results obtained by the method in Zhang (2004) and Sattaratnamai (2011) are strongly influenced by this angle. Therefore, it can be concluded that the design of the three-dimensional rectangular calibration object is optimal in terms of minimization of calibration errors. B. Experimental Results with Real Data Experiments were carried out on the equipment as shown in Fig. 1. A CANOY EOS550D SLR camera with images of 2592×1728 pixels resolution was mounted on a SICK LMS291-S05 laser scanner. This two-dimensional laser scanner provides range measurements by scanning 180 degrees of the environment parallel to the floor and maximum 80 (m) measurements with 0.25 degree angular resolution. A three-dimensional rectangular calibration-object was moved in front of camera and 2DLS at the distance ranging from 1 meter to 5 meters for extrinsic calibration in this work. The calibration object
© 2013 ACADEMY PUBLISHER
Comparing with Zhang (2004) and Sattaratnamai (2011), the 2DLS points were projected into the images using the optimized extrinsic calibration results obtained in this paper as shown in Fig.7. Obviously, it concluded that the new algorithm could improve the accuracy of extrinsic calibration greatly. V.
CONCLUSIONS
In this paper, a new algorithm was proposed to improve accuracy of laser data to be used in extrinsic calibration between camera and 2DLS.The least-squares method for reducing the ranging error and an angular estimation for reducing the step-angular error were applied to modify the laser data of the three-dimensional rectangular calibration object obtained by 2DLS. Finally, by minimizing re-projection error between adjusted laser points and their corresponding projections on the image coordinate, extrinsic parameters were refined generally to improve the accuracy of extrinsic calibration. Comparing with Zhang (2004) and Sattaratnamai (2011), this algorithm has the advantages of being fast to calibrate and yielding better performance. Experimental results showed that the angular error was down to about 0.2 degree and the three-dimensional error was down to about 1 cm in the measuring range from 1m to 5m. And the accuracy of extrinsic calibration of camera-2DLS has been verified to improve greatly. ACKNOWLEDGMENT This study is financially supported by National Natural Science Foundation of China (Grant No.31070634), China Postdoctoral Science Foundation (Grant No. 2012M510330), 948 project supported by State Forestry Administration, China (Grant No. 2011-4-02).
JOURNAL OF MULTIMEDIA, VOL. 8, NO. 6, DECEMBER 2013
783
REFERENCES [1] F. Rottensteiner, J. Trinder, S. Clode and K. Kubik, “Building detection using LIDAR data and multispectral images,” Digital Image Computing: Techniques and Applications. Sydney, vol. 2, pp. 673-682, December 2003. [2] H. Lili, and M. Barth, “A novel multi-planar LIDAR and computer vision calibration procedure using 2D patterns for automated navigation,” Intelligent Vehicles Symposium. 2009 IEEE, pp. 117-122, June 2009. [3] P. Nunez, P. Jr. Drews, R. Rocha, and J. Dias. “Data fusion calibration for a 3d laser range finder and a camera using inertial data. European Conference on Mobile”. Proc. of 4th European Conf. on Mobile Robots, pp. 31-36. September 2009. [4] D. Amarasinghe, G. K. I. Mann, and R. G. Gosine. “Integrated laser/camera sensor for the detection and localization of landmarks for robotic applications”. Robotics and Automation, pp. 4012-4017, May 2008. [5] F. Rodriguez, V. Fremont, and P. Bonnifait. “Extrinsic calibration between a multi-layer LIDAR and a camera”. Multi-sensor Fusion and Integration for Intelligent Systems, pp. 214-219, August 2008. [6] J. Santolaria, D. Guillomía, C. Cajal, J. A. Albajez and J. J. Aguilar, “Modelling and Calibration Technique of Laser Triangulation Sensors for Integration in Robot Arms and Articulated Arm Coordinate Measuring Machines”, Sensors, pp. 7374-7396, September 2009. [7] G. Lisca, P. Jeong, and S. Nedevschi “Automatic one step extrinsic calibration of a multi-layer laser ranger finder relative to a stereo camera”. Intelligent Computer Communication and Processing (ICCP), pp. 223-230, August 2010. [8] Q. Zhang and R. Pless. “Extrinsic calibration of a camera and laser range finder (improves camera calibration)”.
© 2013 ACADEMY PUBLISHER
[9]
[10]
[11] [12]
[13]
[14] [15] [16]
[17]
Intelligent Robots and systems, vol. 3, pages 2301-2306, Sept. / Oct. 2004. G. Li, Y. Liu, L. Dong, X. Cai and D. Zhou. “An algorithm for extrinsic parameters calibration of a camera and a laser range finder using line features”. Intelligent Robots and Systems, pp. 3854-3859, Oct. / Nov. 2007. Vasconcelos, Francisco, J. Barreto, and U. Nunes. “A Minimal Solution for the Extrinsic Calibration of a Camera and a Laser-Range finder”. Pattern Analysis and Machine Intelligence, vol. 32, pp. 2097-2107, November 2012. S. Sattaratnamai, N. Niparnan, A. Sudsang. “On improving laser data for extrinsic 2DLS/camera calibration”. Robotics and Biomimetics (ROBIO), pp. 343-348, December 2011 L. Kneip, F. Tâche, G. Caprari and R. Siegwart. “Characterization of the compact Hokuyo URG-04LX 2D laser range scanner”. Robotics and Automation, pp. 1447-1454, May 2009. M. Yi, K. Jana, S. Stefano, S. Shankar. An Invitation to 3-D Vision: From Images to Models. Opaque this, November 2001. C. Mei, I. S. Antipolis and J. Y. Bouguet “Camera Calibration Toolbox for Matlab”. Available: http://www. vision. caltech. edu/bouguetj/calib_doc/. July 2010. Jennings, Alan, J. J. McKeown and A. Jennings. Matrix computation for engineers and scientists. J. Wiley, 1992. W. Chen, Q. Gong, L. Zhang, M. Yang. “Parameters identification of twelve-phase synchronous generator based on Levenberg-Marquardt algorithm”. Electrical Machines and Systems, pp. 3992 – 3995, October 2008. Zhengyou Zhang. “Flexible camera calibration by viewing a plane from unknown orientations”. Proceedings of the Seventh IEEE International Conference on IEEE, Vol. 1, pp. 666-673, September 1999.
777
Improved Accurate Extrinsic Calibration Algorithm of Camera and Two-dimensional Laser Scanner Jianlei Kong, Lei Yan*, Jinhao Liu, Qingqing Huang, and Xiaokang Ding College of Technology, Beijing Forestry University, Beijing, China Email: [email protected], [email protected], [email protected], [email protected], [email protected] *Corresponding Author
Abstract—In the object detecting system composed of a camera and a two-dimensional laser scanner (2DLS), the extrinsic calibration is an essential step to operate properly. However, the edge points of object cannot be detected accurately by 2DLS due to angular resolution limit, which caused loss of accuracy of the extrinsic calibration between camera and 2DLS. In this paper a new algorithm is proposed to solve this problem. Firstly, the least-squares method was applied to linearly fit laser data located on the scanning plane of the three-dimensional rectangular calibration object. Secondly, half of the value of angular resolution was used as the angular incremental component of the laser edge points of object to improve the accuracy of laser data captured by 2DLS. Finally a general optimization was proposed to refine the extrinsic calibration parameters of camera-2DLS by minimizing the re-projection error between those adjusted laser points and their corresponding projections under the image coordinate of the camera. Compared with related methods, experimental results showed that the accuracy of extrinsic calibration between camera and 2DLS was significantly improved. Index Terms—Two-dimensional Laser Scanner; Accuracy of Extrinsic Calibration; Ranging Error; Step-angular Error; Camera
I.
INTRODUCTION
In the recent studies, combination of a camera and a two-dimensional laser scanner (2DLS) has been widely used to provide applications in detection [1], autonomous vehicle navigation [2], robot motion and control [3] and 3D visualization measurement [4]. In the camera-2DLS object detecting system, 2DLS provides real time ranging measurements in a wide angular field while a camera provides rich visual information. In order to effectively fusion the data from the camera and 2DLS, it is important to know their relative position and orientation from each other, which affects the geometric interpretation of their measurements. The calibration between camera and 2DLS can be decomposed into intrinsic calibration parameters and extrinsic calibration parameters. The extrinsic calibration parameters are the rotation and translation of some fiducial coordinate systems relative to the sensors. And
© 2013 ACADEMY PUBLISHER doi:10.4304/jmm.8.6.777-783
the intrinsic calibration parameters, such as the calibration matrix of a camera, affect how the sensor samples the scene. This work assumes the intrinsic calibration is known as the perspective pinhole model, and focuses on the accuracy improvement of the extrinsic calibration, which is critical to be solved. （ R c ， Tc ）
u
K
p(u,v)
Yc Xc
Pc v Image in CCD Camera
Zc
CCD Camera
Zw Pw
Xw
Yw
（ Φ， Δ ） Yl
Pl( x l,Y l) a1
Zl
Pl a
a2
Xl
Laser Data in 2DLS
Yl Xl
2 D Laser Scanner(2D LS)
Figure 1. Schematic of the extrinsic calibration problem of camera-2DLS. The three-dimensional rectangular calibration object was posed in the both views of the camera and 2DLS. This paper reduces the ranging error and the step-angular error to improve the laser data used for extrinsic calibration. The goal of this paper is to study a new algorithm to improve the accuracy of extrinsic calibration by refining the rotation matrix and the transformed vector with those modified laser data.
A large number of researchers have contributed many works on detecting some specific points or stripes to solve the camera-2DLS extrinsic calibration. Those feature points or shapes were obtained to establish a homogeneous transformation between the 2DLS coordinate and the camera coordinate [5-7]. However, the extraction process of corresponding data pairs was cumbersome and time-consuming, and the procedure of extrinsic calibration was inaccurate. To simplify the calibration process, the most widely used method was proposed by Zhang and Pless in [8], and described a camera-2DLS extrinsic calibration procedure where a checkerboard pattern was freely placed in same views simultaneously of the two sensors. For each various positions of the planar pattern, the method constrained the extrinsic parameters by registering the laser scanline on the planar pattern with the estimated pattern plane from the camera image. The solution of equations provided an
778
JOURNAL OF MULTIMEDIA, VOL. 8, NO. 6, DECEMBER 2013
initial estimate for the relative rotation and translation, which was subsequently refined by iterative minimization of the re-projection error. However, this algorithm existed for a problem that it needed a specific orientation of calibration plate to obtain the optimal results, which was difficult to be established. Since the method was based on linear equations which did not directly enforce the rotation and translation matrix. It often leaded to poor initialization of the data pairs corresponding to the 2DLS data and the images viewed by the camera, and the estimation errors of the data pairs would affect the calibration accuracy also. An algorithm for extrinsic calibration has been proposed to establish a coordinate transform relationships by using straight line features by Li in [9]. The method designed a right-angled triangular checkerboard to employ the invisible intersection points of the slice plane obtained by 2DLS with the edges of the checkerboard to set up the constraints equations. The extrinsic parameters are then calibrated by minimizing the algebraic errors between the measured intersections points and their corresponding projections on the image plane of the camera. However, the checkerboard used in this method was unsuitable for intrinsic calibration, which was also not convenient in extrinsic calibration. Another method proposed by F. Vasconcelos in [10] fit lines to the laser depth readings and carried the Euclidean registration of 3 planes with 3 co-planar lines in an optimal and closed-form manner. It is proved for the first time that the alignment of 3 plane-line correspondences had at most 8 solutions, which determined the rotation and translation by solving a standard p3p problem. u p( u, v
O1 O2 ( u0,v
0
)
X f
)
Y
Xc Optical Axis
Oc
Zc Yc
v
Pw(Xw,Yw,Zw)
Zw Xw Ow
Yw
Figure 2. Geometry of perspective projection in camera model. In machine vision system, the transformation of objects from the world coordinates to the pixel coordinates involves four reference coordinate systems: pixel coordinate system, image coordinate system, camera coordinate system, and world coordinate system [13]. This work solves intrinsic calibration parameters with Matlab.
However, the inaccurate measurement of 2DLS caused loss of accuracy of the extrinsic calibration between the camera and 2DLS severely. When a single laser pulse of 2DLS was sent out and reflected by an object surface within the range of the scanner, the elapsed time between emission and reception of the laser pulse served to calculate the distance between object and 2DLS [11]. Since the reflectivity was based on the material and position of the various objects, the ranging error existed in extrinsic calibration of camera-2DLS.This work applied the least-squares method to linearly fit the discretely distributing laser points, which were located on the surface of a three-dimensional rectangular calibration object, as the intersecting lines. Since the laser rays from 2DLS were invisible, the calibration data must be on the edges of calibration objects. However, 2DLS could not © 2013 ACADEMY PUBLISHER
detect the edge points of object accurately because of angular resolution, which also caused the inaccuracy of 2DLS [12]. To solve this problem, half of the value of angular resolution was used as the increased value of step-angular error to modify the laser data points on the edges of object. By minimizing the re-projection error between those adjusted points captured by 2DLS and their corresponding projections on the image coordinate of the camera, the extrinsic calibration parameters were refined with a general optimization. The rest of this paper is as follows: Section 2 presents the principle and notations of extrinsic calibration. Section 3 presents a new algorithm to improve the accuracy of 2DLS for the accurate improvement of extrinsic calibration between the camera and 2DLS. Compared with related methods, the experimental results are described in Section 4. Lastly, Section 5 concludes the algorithm in this paper. II.
THE CALIBRATION PROBLEM
The camera can be described by the perspective pinhole model as shown in Fig.2. A projection of a random edge point from the camera coordinates system Pc [ X c , Yc , Zc ]T in the world coordinate to the image coordinates p [u, v]T can be represented as follows: u X c Zc fu v c.K . Y Z c Z 0 c c c 1 1 0
s fv
u0 v0
0
1
X 0 c Y 0 c Z 0 c 1
(1)
where c is a proportional factor and K is the camera related to the camera’s internal structure. fu, fv are the equivalent focal lengths on u-axe and v-axe of the image coordinate system respectively. Here, (u0,v0) is called the coordinates of the principal point and s is the parameter describing the skewness of the two image axes. This paper applied the Camera Calibration Toolbox in reference [14] to calculate the camera intrinsic matrix with Matlab. A three-dimensional calibration object was viewed by a camera and 2DLS as shown in Fig.3. 2DLS could detect direction and distance measurements of the object lying on a scanning plane parallel to the floor. The 2DLS coordinate system was defined with an origin at 2DLS, and the laser scanning plane was the plane Zl=ŋ, where ŋ is a constant representing the distance from the origin of the 2DLS coordinate system to the origin of the world coordinate system. Supposing an edge point Pw detected by 2DLS was reported as Pl [ D cos(n ), D sin(n ), ]T in a normalized Cartesian coordinate system, where δ is the angular resolution, n is the number of laser beam, and D represents the distance from the center of 2DLS to the calibration object. And Pl was projected in the camera coordinate system Pc as follows:
Pc Pl
(2)
JOURNAL OF MULTIMEDIA, VOL. 8, NO. 6, DECEMBER 2013
779
where is a 3×3 orthonormal matrix representing the rotation matrix corresponding to the 3×1 transformed vector . Combining (1) and (2) leads to the following extend homogeneous equation between the camera coordinate system and the 2DLS coordinate system:
the red lines L1 and L2 as shown in Fig. 4 (a)). The lines represented the projection of the calibration object in the 2DLS coordinate system and were expressed as: w 1
A. X b . (3)
k 1
where A is a 3×12 matrix and b is a 3-vector which can T b uZ k , vZ k , Z k c c c be expressed as , and X is a 9×1 vector representing 12 unknown extrinsic calibration parameters. n pairs of (Pl, p) representing the projections of the same point Pw viewed in the 2DLS and the camera were selected to solve the overdetermined homogeneous equations. When the value of n was greater than 4, the least square solution of (3) was obtained as following:
PLij
w 1
L1 {( xk , yk )
(4)
The details about this matrix computation can be referred in Matrix Computation [15]. This concluded the initial rotate matrix and the transformed vector in extrinsic calibration of camera-2DLS.
k 1
w 1
y k 1
m
2 k
m
k w
k w
m
m
k w
k w
xk2 +yk2 Dk2 }
where (Xk,Yk) represents the coordinate of the actual laser points and Dk gives the distance from the laser center in 2DLS coordinate which satisfies Dk xk2 +yk2 , k defines the laser point on the k-th beam, w stands for the number of laser beam at the minimum distance, a1 and a2 are the slopes of the lines, and b1 and b2 represent the intercepts of lines in 2DLS coordinate system. The intercepts and slopes were obtained by using the method of linear fitting. The least squares resolution was applied to express the linear parameters of two intersecting lines L1 and L2 as following:
(a) (b)
This section described in detail the algorithm to improve the accuracy of extrinsic calibration. A three-dimensional rectangular calibration object was placed at different poses as long as its feature could be detected by a camera and 2DLS. The method of least squares was used to linearly fit laser data of the object scanned by 2DLS for reducing the ranging error. And half of the value of the angular resolution was applied to modify the edge laser points and the laser apexes of the object. Lastly, the adjusted laser data compared with the nonlinear optimization and the general optimization was used to refine and of the extrinsic calibration between camera and 2DLS for accurate improvement. A. Reducing the Ranging Error The combination of the ranging error and the step-angular error reduced the accuracy of extrinsic calibration of camera-2DLS. Hence a new algorithm was proposed to reduce ranging error to improve the accuracy of laser data as shown in Fig.4 (a). A black dot P indicated the actual measurement of the laser points and deviated from the calibration object surface. And the green point P’ represented the adjusted laser data corresponding to P. This work applied the method of least squares to linearly fit laser points of the calibration object captured by 2DLS as two intersecting lines (seeing © 2013 ACADEMY PUBLISHER
(5)
yk a2 xk b2 ,
w 1
w 1
w 1 xk yk xk yk
w 1
k 1 2
, b1
m
m
m
k w
k m
k m 2
k 1
k 1
w 1 w 1 x xk k 1 k 1 w 1
2 k
PROPOSED SCHEME
k 1
2 k
\ L2 {( xk , yk )
a1
III.
a1 xk b1 ,
x +y D } 2 k
w 1
Figure 3. A three-dimensional calibration object in the both views of a camera (a) and 2DLS (b)
w 1
k
a2
k 1
w 1
k
a1 xk k 1
w 1
(6)
m w 1 xk yk xk yk w 1 w 1 xk2 xk k 1 k 1 w 1
y
m
, b2
y
k w
m
k
a2 xk k w
m w 1
Combining (5) and (6), the optimized laser points (green dots in Fig.4) corresponding to the original laser points (black dots in Fig.4) were used to reduce the ranging error for improving accuracy of laser data. B. Reducing the Step-angular Error Taking into consideration that the laser rays were invisible, the calibration data must be on the edges of calibration objects. However, the edge points could not be detected accurately by 2DLS due to angular resolution, which was called as the step-angular error. It was a main reason reducing the accuracy of extrinsic calibration. When the laser rays hit an object edge, the point on the left edge of the calibration object had a real world coordinate as B1= (X1, Y1, Z1), which polar coordinate could be written as B1= (,D). However, since the 2DLS had an angular resolution , a nearby detected point P1= (n,D) on the calibration object was used to replace B1. The step-angle error of the point B1 with respect to the laser point P1 as (see Fig. 4 (b)), where is the included angle between B1 and P1. The original data was refined as B1= (nD). Based on observations and analysis, the value of satisfied two conditions: Firstly, the step-angular error was always less than one. Because if not, the point B1 must also be detected by the laser and
780
JOURNAL OF MULTIMEDIA, VOL. 8, NO. 6, DECEMBER 2013
Three-dimentional Calibration Object
L 1 B1 (n+m-1)δ L2 (n-1)δ
...
(n+ξ)δ
P1
nδ
...
wδ
ξδ
...
Three-dimentional Calibration Object
L1 nδ ( n-1)δ (n+1)δ P P'
...
mδ
L2 B2 (m-1)δ
τδ wδ B3(w+1)δ O
O
2 D Laser Scanner
2 D Laser Scanner
(a) (b) Figure 4. Analysis of ranging error and step-angular error in extrinsic calibration
it should represent the edge instead of P1.The second condition was that the laser points P2 met normal distribution and its mean approaches 1/2, which was chosen as the estimated value of step-angular error. Similarly, the right edge point B2 and the apex point B3 of the calibration object were also chose to calculate the parameters of extrinsic calibration (see Fig.4 (b)). According to the linear relationships of between L1 and L2, the following equations were obtained:
X , Y X D cos n , Y a X b X , Y X D cos n , Y a X b
B1 B2
1
2
1
1
2
1
2
1
2
1
2
1
2
2
A X 3 , Y3 Y3 a1 X 3 b1 , Y3 a2 X 3 b2 B1
X , Y X 1
1
1
D cos
(7)
Using to estimate the increasing value of step-angular error, the accuracy of laser data was improved greatly as well as the accuracy extrinsic calibration in this paper. C. Nonlinear Optimization Based on current adjusted laser data, a nonlinear optimization with the Levenberg-Marquardt method was proposed [16] to reduce the re-projection error between the original laser points and the optimized laser points as following: E2 DLS arg min w1i . B1 L1 , P1i P1i
2
i
w . B2 L2 , P P2i
2
w . B3 L3 , P P
2
i 2
i 3
i 2
i 3
i 3
D. General Optimization The above solution was obtained by modifying the laser data which was not directly related to extrinsic calibration of camera-2DLS. And both the camera and 2DLS had some noises in their outputs which affected the final accuracy of extrinsic calibration. For the further accurate improvement of extrinsic calibration, extrinsic parameters were nonlinearly refined with a general optimization, which was more physically meaningful. A parameter was defined to represent the Euclidean distance of the error from the modified laser points to the corresponding projections in image coordinate as following: E f PL xL , yL , zL , p u, v
Ecam 2 DLS
)
corresponding to the right edge point and P3i corresponding to apex point. B1, B2 and B3 are the modified points of original laser points P1i , P2i , P3i on the
(9)
A nonlinear minimization on E with the Levenberg-Marquardt method was used to refine the rotate matrix and the transformed vector of extrinsic calibration as following:
(8)
where the left edge point B1, the right edge point B2 and the apex point B3 of the calibration object were selected to solve the nonlinear problem. where P1i indicates the left edge point of the calibration-object projected on the i-th pose in 2DLS coordinate system, as same as the P2i
© 2013 ACADEMY PUBLISHER
two intersecting lines . w1i , w2i , w3i indicate the weight of the left edge point, the right edge point and the apex point at i-th pose in whole progress of extrinsic calibration. E2DLS denotes the re-projection error of the original laser points to the optimized laser points. By minimizing the Euclidean distance of the error E2DLS between the original laser points and those adjusted ones to refine laser data, the accuracy of laser data obtained by 2DLS was improved further.
c min i . .K . .PLij pij u, v Z i j c
2
(10)
where the index i represents the pose number of the calibration object, PLij the j-th laser radar point at the i-th poses of the calibration object in 2DLS coordinate, pij u, v the projection corresponding to PLij represents in the image coordinate system, and i is the weight proportion of the i-th pose in the overall data. Combining (9) with (10), a general nonlinear optimization was proposed by minimizing the combination of the error from the original laser points to the optimized laser points
JOURNAL OF MULTIMEDIA, VOL. 8, NO. 6, DECEMBER 2013
and the error from the optimized laser points to image points as follows: Egeneral Ecam 2 DLS E2 DLS
(11)
Here, is a scalar weighting parameter to normalize the relative contribution between Ecam 2 DLS and E2DLS . This requires an initial value of and in extrinsic calibration, which is obtained in the previous section. The general optimization of extrinsic calibration was carried to refine the extrinsic calibration parameters to improve the accuracy of extrinsic calibration significantly. IV.
EXPERIMENTAL RESULTS
A. Comparison The intrinsic matrix of the camera was calculated within the Camera Calibration Toolbox for Matlab proposed in [17] as: 1.6782 1319.0398 6297.7294 K 0 6278.3645 871.8265 . 0 0 1 And the proposed algorithm for the data modifying and the accurate improvement of extrinsic calibration has been implemented in Matlab. Comparing the adjusted laser data with the general optimization, the extrinsic calibration parameters and were refined with the assumption of 10 and c.Zc 2950 as following: [150.9765, 214.3636, 2920]T 0.8708 0.0409 1.0000 0.0044 0.0413 2.0000 0.0000 0.0000 3.0000 A three-dimensional rectangular calibration-object was used for extrinsic calibration in this paper. Each plane of the calibration object was defined by 10×6 grids and the size of the pattern square was 30×30 mm. The calibration object was placed at the distance ranging from 1 meter to 5 meters away from the camera-2DLS system. In order to compare with the Zhang’s algorithm (Zhang (2004)) and the Sattaratnamai’s algorithm (Sattaratnamai (2011) ), Gaussian noises with mean 0 and standard deviation 0.5 pixels were added to the projected image points of edge points and apexes of the calibration object. A range of ±2cm as uniform noise was also added into the ranging measurement of laser points. 100 trials of independent poses were selected to decrease the influence of Gaussian noises. Several contrast parameters were selected to verify high accuracy of calibration with the new algorithm proposed in this paper as showed in Table I. As showed in Table 1, the u-axis average error of this algorithm was limited into 30 pixels, which was increased by 61.4% compared with Zhang (2004) and by 25.8% compared with Sattaratnamai (2011). And the v-axis average error was limited into 3 pixels, which was obviously improved by 47.5% corresponding to Zhang (2004) and by 43.5% corresponding to Sattaratnamai (2011). The u-axis and v-axis average errors were both presenting the extrinsic calibration accurate improvement © 2013 ACADEMY PUBLISHER
781
of the new algorithm in the image coordinate system. Here this work applied images of 2592×1728 pixels to calibration procedure compared with ones of 640×480 pixels in the paper proposed by Zhang (2004) and compared with images of 780×582 pixels and 1920×1080 pixels mentioned in Sattaratnamai (2011), which presented higher resolution of camera in calibration. This caused a large magnitude of the average errors in this paper corresponding to relative works. The three-dimensional error in this algorithm, which presented the calibration error in the ranging measurement, was about 0.991 cm, which was less than the error about 2.49 cm obtained by the method in Zhang (2004) and the error about 1.29 cm corresponding to Sattaratnamai (2011). Similarly, the value of angle error in this new algorithm obtained the smallest value about 0.176 degree in world coordinate system. The minimal results of the three-dimensional error and the angular error obtained by the algorithm showed that, the calibration errors were weakly affected by angles and poses of the calibration object and the extrinsic calibration system of camera and 2DLS in this algorithm was more stable and accurate. Whereas, the results obtained by the method in Zhang (2004) and Sattaratnamai (2011) were both strongly influenced by this angle and poses. TABLE I.
EXPERIMENTAL RESULT OF NEW ALGORITHM COMPARED WITH RELATED WORKS
Methods u-axis average error (pixel) v-axis average error (pixel) Three-dimensional error (cm) Angle error (degree) RMS (pixel) STD
Zhang (2004)
Sattaratnamai (2011)
New algorithm
72.23799022
37.58364553
27.90318309
4.994270337
4.64171778
2.6211516
2.491210195
1.288852679
0.991360647
0.439002248 97.58660382 65.11802068
0.22662919 48.08085658 29.88941575
0.176153146 34.48339795 18.61776107
The Root Mean Square (RMS) of the calibration errors in this algorithm was closed to 34.5 pixel, which was the smallest among the three methods. The value of RMS in this paper showed the obvious improvement by 64.7% compared with Zhang (2004) and by 28.3% corresponding to Sattaratnamai (2011), which indicated the higher precision of the extrinsic calibration. What’s more, the minimum value of the Standard Deviation (STD) of the calibration errors presented that the error distribution was not very discrete and the extrinsic calibration was stable and robust in this paper. A further experiment dealing with the influence of the number of data pairs on the calibration error was performed. The number of the data pairs was varied from 5 to 55 and the calibration error of every number was computed by running 51 trials as showed in Fig.5. Where x-axis represents the number of the data pairs and y-axis represents the calibration error. The new algorithm had the smallest calibration error curve which showed more accurate in extrinsic calibration. What’s more, the error curve of this algorithm reached a stable value at about 15
782
JOURNAL OF MULTIMEDIA, VOL. 8, NO. 6, DECEMBER 2013
pairs against to about 27 pairs in Zhang (2004) and Sattaratnamai (2011), which showed the higher speed in extrinsic calibration of this algorithm. 25
Calibration Error(pixel)
Zhang(2004) Sattaratnamai(2011)
20
New Algorithm 15
10
5
was composed with two planar calibration patterns, which were defined by 10×6 grids. And each square grid was 30 mm ×30 mm. This calibration was designed for intrinsic calibration of camera and extrinsic calibration of camera-2DLS simultaneously for efficiency improvement and time saving. Since the image sensor size was 22.3 mm×14.9 mm and the camera focal length was fixed at 55mm, one pixel in image coordinate system was equal to 0.034027cm in world coordinate system depending on the resolution of image is 72dpi. On real data, the method operates well given reliable calibration parameters of the camera.
0 1
5
10
15
20
25
30
35
40
45
50
55
Number of Data Pairs Figure 5. The influence of the number of data pairs on the calibration error
Three-dimensional Error(m)
0.1
Zhang(2004)
0.09
Sattaratnamai(2011)
0.08
New Algorithm
0.07 0.06 0.05
Figure 7. Projection of the 2DLS points of the calibration object into the images. Red points represented the projection using the extrinsic calibration results obtained with the new algorithm corresponding to blue points obtained by Zhang (2004) and green points obtained by Sattaratnamai (2011). It was obviously showed that the new algorithm had the best performance in extrinsic calibration of camera-2DLS.
0.04 0.03 0.02 0.01 0 3 6 9 12 15 18 21 24 27 30 33 36 39 42 45 48 51 54 57 60 63 66 69 72 75 78 81 84 87 90
Angles of Calibration Object (degree) Figure 6. The influence of the angles of the calibration object on the three-dimensional error
This experiment was performed to reveal the influence of the angles of the calibration object on the three-dimensional error as showed in Fig.6. Where x-axis represents the rotation degree of the calibration object changed from 0 to 90 degree every 3 degree. And the y-axis represents the three-dimensional error. The result of Zhang (2004) and Sattaratnamai (2011) However, the results in the new algorithm were weakly affected by the angles from the calibration object to the 2DLS slice plane. while the results obtained by the method in Zhang (2004) and Sattaratnamai (2011) are strongly influenced by this angle. Therefore, it can be concluded that the design of the three-dimensional rectangular calibration object is optimal in terms of minimization of calibration errors. B. Experimental Results with Real Data Experiments were carried out on the equipment as shown in Fig. 1. A CANOY EOS550D SLR camera with images of 2592×1728 pixels resolution was mounted on a SICK LMS291-S05 laser scanner. This two-dimensional laser scanner provides range measurements by scanning 180 degrees of the environment parallel to the floor and maximum 80 (m) measurements with 0.25 degree angular resolution. A three-dimensional rectangular calibration-object was moved in front of camera and 2DLS at the distance ranging from 1 meter to 5 meters for extrinsic calibration in this work. The calibration object
© 2013 ACADEMY PUBLISHER
Comparing with Zhang (2004) and Sattaratnamai (2011), the 2DLS points were projected into the images using the optimized extrinsic calibration results obtained in this paper as shown in Fig.7. Obviously, it concluded that the new algorithm could improve the accuracy of extrinsic calibration greatly. V.
CONCLUSIONS
In this paper, a new algorithm was proposed to improve accuracy of laser data to be used in extrinsic calibration between camera and 2DLS.The least-squares method for reducing the ranging error and an angular estimation for reducing the step-angular error were applied to modify the laser data of the three-dimensional rectangular calibration object obtained by 2DLS. Finally, by minimizing re-projection error between adjusted laser points and their corresponding projections on the image coordinate, extrinsic parameters were refined generally to improve the accuracy of extrinsic calibration. Comparing with Zhang (2004) and Sattaratnamai (2011), this algorithm has the advantages of being fast to calibrate and yielding better performance. Experimental results showed that the angular error was down to about 0.2 degree and the three-dimensional error was down to about 1 cm in the measuring range from 1m to 5m. And the accuracy of extrinsic calibration of camera-2DLS has been verified to improve greatly. ACKNOWLEDGMENT This study is financially supported by National Natural Science Foundation of China (Grant No.31070634), China Postdoctoral Science Foundation (Grant No. 2012M510330), 948 project supported by State Forestry Administration, China (Grant No. 2011-4-02).
JOURNAL OF MULTIMEDIA, VOL. 8, NO. 6, DECEMBER 2013
783
REFERENCES [1] F. Rottensteiner, J. Trinder, S. Clode and K. Kubik, “Building detection using LIDAR data and multispectral images,” Digital Image Computing: Techniques and Applications. Sydney, vol. 2, pp. 673-682, December 2003. [2] H. Lili, and M. Barth, “A novel multi-planar LIDAR and computer vision calibration procedure using 2D patterns for automated navigation,” Intelligent Vehicles Symposium. 2009 IEEE, pp. 117-122, June 2009. [3] P. Nunez, P. Jr. Drews, R. Rocha, and J. Dias. “Data fusion calibration for a 3d laser range finder and a camera using inertial data. European Conference on Mobile”. Proc. of 4th European Conf. on Mobile Robots, pp. 31-36. September 2009. [4] D. Amarasinghe, G. K. I. Mann, and R. G. Gosine. “Integrated laser/camera sensor for the detection and localization of landmarks for robotic applications”. Robotics and Automation, pp. 4012-4017, May 2008. [5] F. Rodriguez, V. Fremont, and P. Bonnifait. “Extrinsic calibration between a multi-layer LIDAR and a camera”. Multi-sensor Fusion and Integration for Intelligent Systems, pp. 214-219, August 2008. [6] J. Santolaria, D. Guillomía, C. Cajal, J. A. Albajez and J. J. Aguilar, “Modelling and Calibration Technique of Laser Triangulation Sensors for Integration in Robot Arms and Articulated Arm Coordinate Measuring Machines”, Sensors, pp. 7374-7396, September 2009. [7] G. Lisca, P. Jeong, and S. Nedevschi “Automatic one step extrinsic calibration of a multi-layer laser ranger finder relative to a stereo camera”. Intelligent Computer Communication and Processing (ICCP), pp. 223-230, August 2010. [8] Q. Zhang and R. Pless. “Extrinsic calibration of a camera and laser range finder (improves camera calibration)”.
© 2013 ACADEMY PUBLISHER
[9]
[10]
[11] [12]
[13]
[14] [15] [16]
[17]
Intelligent Robots and systems, vol. 3, pages 2301-2306, Sept. / Oct. 2004. G. Li, Y. Liu, L. Dong, X. Cai and D. Zhou. “An algorithm for extrinsic parameters calibration of a camera and a laser range finder using line features”. Intelligent Robots and Systems, pp. 3854-3859, Oct. / Nov. 2007. Vasconcelos, Francisco, J. Barreto, and U. Nunes. “A Minimal Solution for the Extrinsic Calibration of a Camera and a Laser-Range finder”. Pattern Analysis and Machine Intelligence, vol. 32, pp. 2097-2107, November 2012. S. Sattaratnamai, N. Niparnan, A. Sudsang. “On improving laser data for extrinsic 2DLS/camera calibration”. Robotics and Biomimetics (ROBIO), pp. 343-348, December 2011 L. Kneip, F. Tâche, G. Caprari and R. Siegwart. “Characterization of the compact Hokuyo URG-04LX 2D laser range scanner”. Robotics and Automation, pp. 1447-1454, May 2009. M. Yi, K. Jana, S. Stefano, S. Shankar. An Invitation to 3-D Vision: From Images to Models. Opaque this, November 2001. C. Mei, I. S. Antipolis and J. Y. Bouguet “Camera Calibration Toolbox for Matlab”. Available: http://www. vision. caltech. edu/bouguetj/calib_doc/. July 2010. Jennings, Alan, J. J. McKeown and A. Jennings. Matrix computation for engineers and scientists. J. Wiley, 1992. W. Chen, Q. Gong, L. Zhang, M. Yang. “Parameters identification of twelve-phase synchronous generator based on Levenberg-Marquardt algorithm”. Electrical Machines and Systems, pp. 3992 – 3995, October 2008. Zhengyou Zhang. “Flexible camera calibration by viewing a plane from unknown orientations”. Proceedings of the Seventh IEEE International Conference on IEEE, Vol. 1, pp. 666-673, September 1999.
