A New Camera Calibration Algorithm Based On Rotating Object Hui Chen1 , Hong Yu2 , and Aiqun Long1 1
2
School of Information Science and Engineering, Shandong University, Jinan, 250100, China {huichen, longaiqun}@sdu.edu.cn Yantai Normal University, Yantai, Shandong, China
[email protected]
Abstract. With 3D vision measuring, camera calibration is necessary for extracting metric information from 2D images. We present a new algorithm for camera calibration that only requires a spatial triangle with known size. In our method, the camera is fixed while the triangle is rotated freely at one of its vertices. By taking a few (at least two) pictures of the rotating object at an identical camera view position and direction, the camera intrinsic matrix can be obtained. It advances other traditional methods in that the extrinsic parameters are separated from the intrinsic ones, which simplifies the problem with more efficiency and accuracy. Experiments also show that our method is robust and results in high resolution. Key words: camera calibration; pinhole camera model; intrinsic matrix
1
Introduction
Recover 3D information from 2D images is a fundamental problem in computer vision. The recovery of metric information requires knowledge of camera parameters –also known as camera calibration. Camera calibration is widely used in reverse engineering, object recognition and synthesis of virtual environment, etc [1]. The techniques of camera calibration can be roughly classified into three categories: self-calibration, active vision based camera calibration and traditional ways of calibration. Self-calibration doesn’t need calibration object. It achieves calibration only with a moving camera to take several images and find corresponding points. As the technique is based on absolute conic and conicoid, it is very flexible but not mature yet. Since there are many parameters to estimate, it cannot always obtain reliable results [7], [8], [9]. Active vision based camera calibration requires the movement of camera to be exactly known [10], [11]. This technique needs precise equipment to catch the track of the moving camera, which is expensive.
Until now the traditional ways of calibration is still the most widely used technique in camera calibration. It is performed by observing a calibration object whose size or geometry is known, where images of the calibration object is used to acquire the relation between spatial points and their image correspondences for solving the calibration [2–6]. By reasonable design of the calibration object, precise result can be obtained. The technique presented here belongs to the traditional calibration technique. The calibration object is designed as a space triangle that can be easily made with fine precision and little cost. In the classical methods, the calibration object is fixed while moving the camera. However, this will cause the extrinsic parameters to alter with the intrinsic matrix. In our method, we set the camera fixed and one vertex of the calibration object fixed, while rotate the object about the fixed point freely. In doing so, we can solve the intrinsic matrix directly by using a few (at least two) calibration images. The paper is organized as follows: Section 2 introduces the basic principles and notations. Section 3 describes the calibration object and explains how to solve the calibration problem in detail. Section 4 provides experimental results with both computer simulated data and real images. Finally, Section 5 concludes the paper with perspective of this work.
2
Camera Model and Geometry
We first introduce the pinhole mathematical model and the mapping relation between 3D and 2D points with the notations used in this paper. The pinhole camera model is shown in Fig. 1. There are three coordinate
Xc U
P ( x w, y w, z w)
p (u,v)
( x c, y c, z c)
O
Xw
Oc Ow
Zc
f Yc
V
Yw
P
Zw
(u 0, v 0)
Fig. 1. Pinhole Camera Model
systems in the model: world coordinate system XW YW ZW , camera coordinate system XC YC ZC and image coordinate system U V O. The mapping relation between space point P (XW , YW , ZW ) and its corresponding image one p(u, v) is
given by ⎡ ⎤ ⎤ ⎡ ⎤ ⎡ ⎡ ⎤ XW XW fu γ u 0 u ⎢ ⎥ ⎢ YW ⎥ ⎥ = ⎣ 0 fv v0 ⎦ [R, T] ⎢ YW ⎥ . s˜ p = s ⎣ v ⎦ = K[R, T] ⎢ ⎣ ZW ⎦ ⎣ ZW ⎦ 0 0 1 1 1 1
(1)
Where s is an arbitrary scale factor; K is called the camera intrinsic matrix, with (u0 , v0 ) the coordinates of the principal point, fu and fv the scale factors in image U and V axes, and γ the parameter describing the deflection of the two image axes; (R, T), called the extrinsic parameters, is the rotation and translation which relates the world coordinate system to the camera coordinate system. The task of camera calibration is to determine the five intrinsic parameters in ˜ to denote the K. We use the abbreviation K−T for (K−1 )T or (K−T )−1 and x T ˜ = u v 1 . Considering augmented vector by adding 1 as the last element: p that the world coordinate system can be denoted anywhere when the camera is fixed while moving the calibration object, we set it to be superpositioned with the camera coordinate system. Thus there are R = I and T = 0, Equ. (1) becomes ⎡ ⎤ ⎡ ⎤ u Xw ˜ = Zw ⎣ v ⎦ = K ⎣ Yw ⎦ = KP . (2) Zw p Zw 1 The calibration method of this paper is performed based on the above relationship.
3
Description of Calibration Model and Solving Calibration Problem
A(fixed point) C1
C2 l2
l1 l3
B1
c1 b1
a
B2
c2 b2
Oc
Fig. 2. Calibration Object
As shown in Fig. 2, the calibration object is a space triangle AB1 B2 with known length of three sides. One of its vertices A is fixed, the triangle can be rotated freely about this fixed point. C1 and C2 are two points on AB1 , AB2 , their positions on the sides can be measured. In one image, we need 3 parameters to define the fixed point A (space coordinate), and another 3 parameters to define B1 . Consider the length of AB1 is known, we actually only need another 2 parameters to define the point B1 . As the same, point B2 can be defined by 2 parameters with the length of AB2 . Because the length of B1 B2 is also known, with this information, only 1 parameter is enough to define B2 . As following, we can see there are 8 + 3N unknown parameters in N images, in which the first 8 is for 3 parameters to define the fixed point A plus 5 intrinsic parameters, and next 3N is for parameters to define the moving B1 and B2 . By Equ. (2), it has 2 equations for each point in the image, so there are total 2 + 8N equations in N images for the five features, in which the fixed point introduces 2 equations and the other moving four points do that of 8N . Considering the collinearity of image points a, c1 , b1 and a, c2 , b2 , actually there are 2 + 6N equations in N image. That is in theory, with more than two observations (2 + 6N ≥ 8 + 3N , i.e. N ≥ 2) we can get enough information to solve all the unknown parameters. Next, we will explain the calibration process in details. By the squared length of three calibration triangle’s sides, denoted as l1 , l2 , and l3 , we have (3) (B1 − A)T (B1 − A) = l1 , (B2 − A)T (B2 − A) = l2
,
(B2 − B1 )T (B2 − B1 ) = l3 .
(4) (5)
The position of C1 , C2 on sides AB1 , AB2 can be measured as Ci = λAi A + λBi Bi
,
(6)
where λAi + λBi = 1 (i = 1, 2). Substituting the coordinates of A, B1 , C1 into (2) and (6), it follows that ZB1 = − Assume
˜ 1 × c˜1 ) ZA λA1 (˜ a × c˜1 ) · (b . ˜ ˜ λB1 (b1 × c˜1 ) · (b1 × c˜1 )
(7)
˜ × ˜c) a × c˜) · (b ˜ ˜c, λ1 , λ2 ) = λ1 (˜ P (˜ a, b, ˜ ˜ × c˜) λ2 (b × ˜c) · (b
˜ ˜ ˜ , b, where a c are 3 × 1 matrix , λ1 and λ2 are constants, and let ˜ 1 , ˜c1 , λA1 , λB1 )b ˜1 . ˜ + P (˜ h1 = a a, b Combinning (2) and (7), Equ. (3) becomes 2 T ZA h1 K−T K−1 h1 = l1 .
(8)
Similarly, (4) and (5) can be written as 2 T ZA h2 K−T K−1 h2 = l2 ,
(9)
2 T h3 K−T K−1 h3 = l3 , ZA
(10)
where ˜ 2 , ˜c2 , λA2 , λB2 )b ˜2 , ˜ + P (˜ a, b h2 = a and ˜ 2 , c˜2 , λA2 , λB2 )b ˜ 2 − P (˜ ˜ 1 , ˜c1 , λA1 , λB1 )b ˜1 . h3 = P (˜ a, b a, b The above deduction shows that three equations (8), (9) and (10) can be obtained from one calibration image, each of that has 6 unknown parameters. It means in theory that we can get enough information to solve the calibration problem from only a few (at least two) calibration images. Since K−T K−1 is a symmetrical matrix, we can define ⎤ ⎡ m1 m2 m3 M = K−T K−1 = ⎣ m2 m4 m5 ⎦ , m3 m5 m6 let
m = [m1 , m2 , m3 , m4 , m5 , m6 ]T , hi = [hi1 , hi2 , hi3 ]T , x = [x1 , x2 , x3 , x4 , x5 , x6 ]T ,
2 m. Combinning (8), (9) and (10) yields and let x = ZA
[v11 , v22 , v33 ]T x = [l1 , l2 , l3 ]T ,
(11)
where vij = [hi1 hj1 , hi1 hj2 + hi2 hj1 , hi3 hj1 + hi1 hj3 , hi2 hj2 , hi2 hj3 + hi3 hj2 , hi3 hj3 ] . From N calibration images, we have N such linear equations as (11). All together it has the following special form Vx = L , where V= and
1 1 1 v11 , v22 , v33
T
(12)
N N N T T , · · · , v11 , v22 , v33
L = [l1 , l2 , l3 , · · · , l1 , l2 , l3 ]T .
The size of V is 3N × 6, and that of L is 3N × 1. When N > 2, the least-squares solution is given by x = (VT V)−1 VT L .
Once x is estimated, without difficulty, we can obtain the intrinsic parameters and ZA as follow v0 = (x2 x3 − x1 x5 ) (x1 x4 − x22 ) ,
ZA = x6 − x23 + v0 (x2 x3 − x1 x5 ) x1 , √ fu = ZA / x1 ,
fv = ZA x1 x1 x4 − x22 , 2 γ = −x2 fu2 fv ZA , 2 . u0 = γv0 /fu − x3 fu2 ZA At this point, the features A, B1 , B2 , C1 , C2 can be computed from Equ. (2), (6) and (7). In real case, due to the adding noise of calibration images, the extracted feature points will not comply exactly with the Equ. (2). Therefore, it is not sufficient that we only use Equ. (12) to estimate the unknown parameters. Optimization of the initial result should be performed. The optimization model is given by E = min
N
2 2 ˜ ˜ 2 (˜ ai − φ(K, A) + b 1i − φ(K, B1i ) + b2i − φ(K, B2i )
i=1
+ ˜ c1i − φ(K, C1i )2 + ˜c2i − φ(K, C2i )2 ) where φ(K, P) = (1/Zp )KP. The above minization is a nonlinear problem that can be solved by using the Levenberg-Marquardt algorithm [12]. It requires an initial guess of K, A, B1i , B2i , C1i , and C2i , which can be obtained by the technique described in this section.
4
Experimental Results
The proposed algorithm has been tested on both computer simulated data and real data. 4.1
Computer Simulate
In order to make the simulation generalized, we set the simulated camera’s parameters as (1) The image resolution is of 640 × 480, and u0 = 320, v0 = 240, fu = 1000, fv = 1000, γ = 0. (2) The simulated calibration object is shown in Fig. 2, which is an equal side triangle with a side length of 100cm. Assume C1 and C2 are the center points of AB1 and AB2 , let λA1 = λB1 = λA2 = λB2 = 0.5, and set the fixed point A at [0, 35, 150]T .
(3) The movement of the simulated object is set randomly as the method in [6]. (4) In order to check the capability of anti-jamming, Gaussian noise is added on the simulated feature points with 0 mean and standard deviation from 0.0 to 1.0 pixel. The simulation experiment mainly exams the effect of result by altering the number of the calibration images and the noise level. Let the number of images range from 2 to 40 and the noise level from 0.1 to 1.0. For each noise level or image number, we perform 40 independent trials. The average result is shown in Fig. 3.
Fig. 3. Calibration errors (a) with respect to the number of images (b) with respect to the noise level
From Fig. 3, it shows that the calibration error will be reduced by increasing the number of calibration images with the same noise level, and the calibration result tends to convergence when the number of images exceeds 20; while with the same number of calibration images, the calibration error will increase with that of noise level; when the noise level is less than 0.7, the relative error is changed slowly and small enough to be accepted. From the simulated experiment results, we can see that the proposed calibration technique has good capability in anti-jamming. 4.2
Real Data
In the real experiment, the resolution of calibration image is 640 × 480, and the number of images is 50. As shown in Fig. 4, one vertex of the calibration triangle is fixed, the lengths of the three sides are AB1 =33cm, AB2 =37cm and B1 B2 =28cm. C1 and C2 are the center points of AB1 and AB2 . The calibration technique here requires to precisely locate the feature points, but it is complicated to extract these feature points directly from gray images. Instead, we first detect the feature edges of the calibration object, and then compute the intersection points of these edges to determine the features (Fig. 4). The result is shown in Table 1.
Fig. 4. Calibration Image Table 1. Calibration Result with Real Data N um
fu
fv
u0
v0
γ
5
1049.23 1077.35 239.14 343.59 7.86(89.57◦ )
10
1076.47 1031.47 259.49 327.44 4.15 (88.78◦ )
15
1016.65 1058.08 267.49 308.83 3.03(89.83◦ )
20
830.57
738.45 316.26 313.86 2.99(89.79◦ )
25
889.55
810.32 312.47 302.48 2.71(89.83◦ )
30
880.88
809.59 323.22 283.94 2.63(89.83◦ )
35
879.24
811.47 331.46 280.43 2.67(89.83◦ )
40
879.93
814.89 334.66 279.47 2.69(89.82◦ )
50
879.92
809.57 340.64 275.07 2.69(89.82◦ )
From the real experiment data, we can see that the result converges to the ideal one when the number of calibration images exceeds 25.
5
Conclusion
In this paper, we have presented an algorithm for camera calibration using a rotating spatial triangle with edge length given. The relationship between 3D points and its 2D correspondences is analyzed to eliminate the extrinsic parameters in the calibration processing. By making full use of the geometric information of the calibration object, only two calibration images at minimal are required to solve the calibration. Compared with the traditional techniques, the proposed one is more computationally efficient in that the intrinsic parameters are solved independently with only a few images(at least two), and therefore more accurate. Simulation and real data experiments also show that the tech-
nique has good performance in anti-jamming. The present result can be further improved by considering radial distortion. Acknowledgment This work is supported by the Grand Project of Shandong Nature Science Fund under No.Z2005G02 and Shandong Outstanding MiddleYoung-Aged Scientist Reward Fund under No.03BS001.
References 1. Agrawal, M., Davis, L. S.: Camera calibration using spheres : A semi-definite programming approach. Proceedings of the Ninth IEEE International Conference on Computer Vision(ICCV’03) (2003) 2. Tsai, R. Y.: A Versatile Camera Calibration Technique for High-Accuracy 3D Machine Vision Metrology Using Off-the-Shelf TVCameras and Lenses. IEEE J. Robotics and Automation,3(2) (1987)323–344 3. Faugeras, O.: Three-Dimensional Computer Vision: a Geometric Viewpoint. MIT Press (1993) 4. Sturm, P., Maybank, S.: On Plane-Based Camera Calibration: A General Algorithm, Singularities, Applications. Proc. IEEE Conf. Computer Vision and Pattern Recognition (1999) 432–437 5. Zhang, Z.: A Flexible New Technique for Camera Calibration. IEEE Trans. Pattern Analysis and Machine Intelligence 22(11) (2000) 1330–1334 6. Zhang, Z.: Camera Calibration with One-Dimensional Objects. Proc. European Conf. Computer Vision 4 (2002) 161–174 7. Hartley, R. I.: An Algorithm for Self Calibration from Several Views. IEEE Conf. Computer Vision and Pattern Recognition (1994) 908–912 8. Luong, Q.-T.: Matrice Fondamentale et Calibration Visuelle surl’EnvironnementVers une plus Grande Autonomie des Systemes Robotiques. PhD thesis, Universite de Paris-Sud, Centre d’Orsay (1992) 9. Maybank,S. J., Faugeras, O. D.: A Theory of Self-Calibration of a Moving Camera. Int’l J. Computer Vision 8(2) (1992) 123–152 10. Wan, W., Xu, G. Y.: Camera parameters estimation and evaluation in active vision system. Pattern Recognition 29(3) (1996) 429–447 11. Wei, G. Q., Arbter, K., Hirzinger, G.: Active self -calibration of robotic eyes and Hand-Eye relationship with model identification IEEE Trans Robotics and Automation 29(3) (1998)158–166 12. More, J. J.: The Levenberg-Marquardt Algorithm, Implementation and Theory. Numerical Analysis, G.A. Watson, ed. (1977)
