A Factorization-based method for Projective ... - Semantic Scholar
A Factorization-based method for Projective Reconstruction with minimization of 2-D reprojection errors W. K. Tang and Y. S. Hung Department of Electrical and Electronic Engineering The University of Hong Kong Pokfulam Road, Hong Kong {wktang,yshung}@eee.hku.hk
Abstract. In this paper, we consider the problem of projective reconstruction based on the factorization method. Unlike existing factorization based methods which minimize the SVD reprojection error, we propose to estimate the projective depths by minimizing the 2-D reprojection errors. An iterative algorithm is developed to minimize 2-D reprojection errors. This algorithm reconstructs the projective depths robustly and does not rely on any geometric knowledge, such as epipolar geometry. Simulation results using synthetic data are given to illustrate the performance of the algorithm.
1
Introduction
In the area of computer vision, there are many existing approaches [1][2][3][4][5] to reconstruct the 3-D Euclidean structure from multiple views. One of the popular strategies is ’stratification’ [1]. It decomposes the reconstruction process into different procedures that recover the projective space first and then upgrade it to Euclidean space by applying metric constraints. One of the main benefits of this approach is that each procedure can be considered and developed independently. Projective reconstruction is the first step of the stratification strategy to reconstruct the 3-D shape and the projection matrices in projective space from correspondences between 2-D planar images. Some existing methods [6][7] reconstruct the 3-D structure by solving the geometric properties between two views, three views or even four views with multilinear constraints. All of them are linear methods but they can relate only two to four views at the same time. Since there is no general representation for multiple (more than four) views based on geometric properties, the advantages of multiple views cannot be fully applied. The well-known projective factorization method [8], proposed by Tomasi and Kanade, generalizes projective reconstruction for multiple-view configuration. However, one set of parameters, the projective depths, need to be estimated in the reconstruction. Han and Kanade [2] use an iterative method to solve for the depths in a decomposition process iteratively without calculating the fundamental matrices. Poelman and Kanade
2
W. K. Tang and Y. S. Hung
[9] also propose other methods to estimate the depths in paraperspective or orthogonal projection as the starting points for the iterative method [2]. Triggs [10] proposes a method to solve for the depths by calculating fundamental matrices between any two views. It provides a very accurate result and fast converging on noise-free situations. However, it has the same problems as the above geometric methods since the fundamental matrices are calculated in advance. There exist several projective reconstruction methods [11][12][13] for missing data. The sequential updating method by Beardsley [11] is able to deal with missing data but it does not optimize all the views at the same time. Jacobs [12] proposes a method that is good in noise-free situations and the result can be the starting points for iterative methods. Shum, Ikeuchi and Reddy [13] parameterize the minimization problem of factorization method as weighted least squares (WLS) minimization problems. The minimizations are set up by existing data only. The problem is stated as minimizing the sum of squares of the SVD reprojection errors [14]. Unlike 2-D reprojection errors which represent the distance between the input point and the reprojected point on a 2-D image plane, SVD reprojection errors lack physical meaning. Most of the factorization-based methods minimize the SVD reprojection errors rather than 2-D reprojection errors. A difficulty of minimizing 2-D reprojection errors is that the minimization problem [3] becomes non-linear. In this paper, we propose an algorithm for projective reconstruction and it is based on the factorization method and WLS minimization. The proposed algorithm can deal with missing data and projective depth estimation and it minimizes 2D reprojection errors from all views at the same time. This paper describes all the background and existing methods first. Our algorithm is developed in Sections 2, 3 and 4. In Section 5, simulation results using synthetic data are given to illustrate the performance of our algorithm in comparison with an existing method. Section 6 contains some concluding remarks.
2
Problem formulation
£ ¤T For a 3-D point Xj = xj yj zj 1 , let its projection in the ith view be the £ ¤T image point xij = uij vij 1 in homogenous coordinates. The perspective projection equation relates these two sets of coordinates as: λij xij = Pi Xj
(1)
where λij is the real depth of the point and Pi is a 3 × 4 projection matrix. The Joint projection matrix P combines all the m projection matrices Pi as ¤ £ T T ∈
Abstract. In this paper, we consider the problem of projective reconstruction based on the factorization method. Unlike existing factorization based methods which minimize the SVD reprojection error, we propose to estimate the projective depths by minimizing the 2-D reprojection errors. An iterative algorithm is developed to minimize 2-D reprojection errors. This algorithm reconstructs the projective depths robustly and does not rely on any geometric knowledge, such as epipolar geometry. Simulation results using synthetic data are given to illustrate the performance of the algorithm.
1
Introduction
In the area of computer vision, there are many existing approaches [1][2][3][4][5] to reconstruct the 3-D Euclidean structure from multiple views. One of the popular strategies is ’stratification’ [1]. It decomposes the reconstruction process into different procedures that recover the projective space first and then upgrade it to Euclidean space by applying metric constraints. One of the main benefits of this approach is that each procedure can be considered and developed independently. Projective reconstruction is the first step of the stratification strategy to reconstruct the 3-D shape and the projection matrices in projective space from correspondences between 2-D planar images. Some existing methods [6][7] reconstruct the 3-D structure by solving the geometric properties between two views, three views or even four views with multilinear constraints. All of them are linear methods but they can relate only two to four views at the same time. Since there is no general representation for multiple (more than four) views based on geometric properties, the advantages of multiple views cannot be fully applied. The well-known projective factorization method [8], proposed by Tomasi and Kanade, generalizes projective reconstruction for multiple-view configuration. However, one set of parameters, the projective depths, need to be estimated in the reconstruction. Han and Kanade [2] use an iterative method to solve for the depths in a decomposition process iteratively without calculating the fundamental matrices. Poelman and Kanade
2
W. K. Tang and Y. S. Hung
[9] also propose other methods to estimate the depths in paraperspective or orthogonal projection as the starting points for the iterative method [2]. Triggs [10] proposes a method to solve for the depths by calculating fundamental matrices between any two views. It provides a very accurate result and fast converging on noise-free situations. However, it has the same problems as the above geometric methods since the fundamental matrices are calculated in advance. There exist several projective reconstruction methods [11][12][13] for missing data. The sequential updating method by Beardsley [11] is able to deal with missing data but it does not optimize all the views at the same time. Jacobs [12] proposes a method that is good in noise-free situations and the result can be the starting points for iterative methods. Shum, Ikeuchi and Reddy [13] parameterize the minimization problem of factorization method as weighted least squares (WLS) minimization problems. The minimizations are set up by existing data only. The problem is stated as minimizing the sum of squares of the SVD reprojection errors [14]. Unlike 2-D reprojection errors which represent the distance between the input point and the reprojected point on a 2-D image plane, SVD reprojection errors lack physical meaning. Most of the factorization-based methods minimize the SVD reprojection errors rather than 2-D reprojection errors. A difficulty of minimizing 2-D reprojection errors is that the minimization problem [3] becomes non-linear. In this paper, we propose an algorithm for projective reconstruction and it is based on the factorization method and WLS minimization. The proposed algorithm can deal with missing data and projective depth estimation and it minimizes 2D reprojection errors from all views at the same time. This paper describes all the background and existing methods first. Our algorithm is developed in Sections 2, 3 and 4. In Section 5, simulation results using synthetic data are given to illustrate the performance of our algorithm in comparison with an existing method. Section 6 contains some concluding remarks.
2
Problem formulation
£ ¤T For a 3-D point Xj = xj yj zj 1 , let its projection in the ith view be the £ ¤T image point xij = uij vij 1 in homogenous coordinates. The perspective projection equation relates these two sets of coordinates as: λij xij = Pi Xj
(1)
where λij is the real depth of the point and Pi is a 3 × 4 projection matrix. The Joint projection matrix P combines all the m projection matrices Pi as ¤ £ T T ∈
