Epipolar Geometry Estimation via RANSAC ... - Semantic Scholar
1/18
Epipolar Geometry Estimation via RANSAC Benefits from the Oriented Epipolar Constraint Ondˇrej Chum
Toma´ sˇ Werner
Jiˇr´ı Matas
Center for Machine Perception Department of Cybernetics Faculty of Electrical Engineering Czech Technical University, Prague
August 23, 2004 International Conference on Pattern Recognition, Cambridge
Projective vs. Oriented Projective Geometry 2/18
Let a, b ∈ Rn+1 \ {0} Projective Geometry a∼b if λa = b for λ 6= 0
Oriented Projective Geometry Equivalence
+ a∼ b if λa = b for λ > 0
Projective vs. Oriented Projective Geometry 2/18
Let a, b ∈ Rn+1 \ {0} Projective Geometry
Oriented Projective Geometry
a∼b if λa = b for λ 6= 0
Equivalence
+ a∼ b if λa = b for λ > 0
Pn = Rn+1/ ∼
Projective spaces
+ Sn = Rn+1/ ∼
Projective vs. Oriented Projective Geometry 2/18
Let a, b ∈ Rn+1 \ {0} Projective Geometry
Oriented Projective Geometry
a∼b if λa = b for λ 6= 0
Equivalence
+ a∼ b if λa = b for λ > 0
Pn = Rn+1/ ∼
Projective spaces
+ Sn = Rn+1/ ∼
x2
Camera Model x2 x1
x3
x3
x1
−x3
x ∼ PX rays are lines
+ x∼ PX rays are half lines
Oriented Epipolar Constraint 3/18 + Fx ∼ e0 × x0
Oriented Epipolar Constraint 3/18
⇐ ⇐
+ Fx ∼ e0 × x0
+ x0>Fx ∼ x0>(e0 × x0)
x0>Fx = 0 Oriented epipolar constraint is stronger than epipolar constraint.
Oriented Epipolar Constraint 3/18
⇐ ⇐
+ Fx ∼ e0 × x0
+ x0>Fx ∼ x0>(e0 × x0)
x0>Fx = 0 Oriented epipolar constraint is stronger than epipolar constraint.
How often does the epipolar constraint hold and the oriented does not?
Previous Work on Oriented Projective Geometry 4/18
• Oriented projective geometry known to mathematicians at least from the end of the 19th century; was not considered very interesting
• Stolfi [’91] showed its importance for computer graphics
• Laveau and Faugeras [ECCV ’96] used it to reject wrong matches after unoriented epipolar geometry was estimated
• Werner and Pajdla [ICCV’98, BMVC’01] introduced oriented epipolar geometry
Algorithm 5/18
How to exploit oriented epipolar constraint to efficiently reject incorrect hypotheses in RANSAC?
• Let fundamental matrix F be computed from seven randomly selected correspondences xi ↔ x0i, i = 1 . . . 7 • Let e0 be epipole in the second image, so that e0>F = 0 • Reject hypothesis F, if there is no such λ ∈ {−1, 1} so that + λe0 × x0i ∼ Fxi
holds for all seven correspondences xi ↔ x0i
Oriented Epipolar Geometry - Model 1 6/18
OK
Oriented Epipolar Geometry - Model 2 7/18
Rejected
Oriented Epipolar Geometry - Model 3 8/18
Rejected
Experiment Image Pairs 9/18
Juice
Shelf
Valbonne
Great Wall
Leuven
Corridor
Experiment Details 10/18
Juice Correspondences: Inliers: Fraction of inliers ε:
447 274 61.30
Models passed: Time speed-up:
37.77% 35.08%
Experiment Details 11/18
Shelf Correspondences: Inliers: Fraction of inliers ε:
126 43 34.13
Models passed: Time speed-up:
11.29% 15.82%
Experiment Details 12/18
Valbonne Correspondences: Inliers: Fraction of inliers ε:
216 42 19.44
Models passed: Time speed-up:
6.41% 37.17%
Experiment Details 13/18
Great Wall Correspondences: Inliers: Fraction of inliers ε:
318 68 21.38
Models passed: Time speed-up:
7.29% 45.96%
Experiment Details 14/18
Leuven Correspondences: Inliers: Fraction of inliers ε:
793 379 47.79
Models passed: Time speed-up:
71.82% 18.52%
Experiment Details 15/18
Corridor Correspondences: Inliers: Fraction of inliers ε:
607 394 64.91
Models passed: Time speed-up:
89.82% 5.77%
Experiment Summary - Number of Models 16/18
500,000 samples drawn models
rejected
passed [%]
ε [%]
Juice
1,221,932
760,354
37.77
61.30
Shelf
1,233,770
1,094,533
11.29
34.13
Valbonne
1,256,648
1,176,042
6.41
19.44
Great Wall
1,274,018
1,181,084
7.29
21.38
Leuven
1,194,238
336,515
71.82
47.79
Corridor
1,187,380
120,916
89.82
64.91
Experiment Summary - Time 17/18
standard
oriented
speed-up [%]
Juice
1.4
0.9
35.08
Shelf
53.6
45.1
15.82
Valbonne
930.8
584.8
37.17
Great Wall
1109.0
599.3
45.96
Leuven
18.5
15.1
18.52
Corridor
1.2
1.2
5.77
Conclusions 18/18
• In comparison to standard RANSAC, introduction of the oriented epipolar constraint reduced the running time by 5% to 46%, compared with standard RANSAC. • The evaluation of the orientation test takes virtually no time compared to the epipolar geometry computation. • More models were rejected in the wide-baseline setting than in the narrow-baseline one. • The fraction of rejected models is proportional to the fraction of outliers among the tentative correspondences.
x2 x1
x3
x2 x3
x1
−x3
Epipolar Geometry Estimation via RANSAC Benefits from the Oriented Epipolar Constraint Ondˇrej Chum
Toma´ sˇ Werner
Jiˇr´ı Matas
Center for Machine Perception Department of Cybernetics Faculty of Electrical Engineering Czech Technical University, Prague
August 23, 2004 International Conference on Pattern Recognition, Cambridge
Projective vs. Oriented Projective Geometry 2/18
Let a, b ∈ Rn+1 \ {0} Projective Geometry a∼b if λa = b for λ 6= 0
Oriented Projective Geometry Equivalence
+ a∼ b if λa = b for λ > 0
Projective vs. Oriented Projective Geometry 2/18
Let a, b ∈ Rn+1 \ {0} Projective Geometry
Oriented Projective Geometry
a∼b if λa = b for λ 6= 0
Equivalence
+ a∼ b if λa = b for λ > 0
Pn = Rn+1/ ∼
Projective spaces
+ Sn = Rn+1/ ∼
Projective vs. Oriented Projective Geometry 2/18
Let a, b ∈ Rn+1 \ {0} Projective Geometry
Oriented Projective Geometry
a∼b if λa = b for λ 6= 0
Equivalence
+ a∼ b if λa = b for λ > 0
Pn = Rn+1/ ∼
Projective spaces
+ Sn = Rn+1/ ∼
x2
Camera Model x2 x1
x3
x3
x1
−x3
x ∼ PX rays are lines
+ x∼ PX rays are half lines
Oriented Epipolar Constraint 3/18 + Fx ∼ e0 × x0
Oriented Epipolar Constraint 3/18
⇐ ⇐
+ Fx ∼ e0 × x0
+ x0>Fx ∼ x0>(e0 × x0)
x0>Fx = 0 Oriented epipolar constraint is stronger than epipolar constraint.
Oriented Epipolar Constraint 3/18
⇐ ⇐
+ Fx ∼ e0 × x0
+ x0>Fx ∼ x0>(e0 × x0)
x0>Fx = 0 Oriented epipolar constraint is stronger than epipolar constraint.
How often does the epipolar constraint hold and the oriented does not?
Previous Work on Oriented Projective Geometry 4/18
• Oriented projective geometry known to mathematicians at least from the end of the 19th century; was not considered very interesting
• Stolfi [’91] showed its importance for computer graphics
• Laveau and Faugeras [ECCV ’96] used it to reject wrong matches after unoriented epipolar geometry was estimated
• Werner and Pajdla [ICCV’98, BMVC’01] introduced oriented epipolar geometry
Algorithm 5/18
How to exploit oriented epipolar constraint to efficiently reject incorrect hypotheses in RANSAC?
• Let fundamental matrix F be computed from seven randomly selected correspondences xi ↔ x0i, i = 1 . . . 7 • Let e0 be epipole in the second image, so that e0>F = 0 • Reject hypothesis F, if there is no such λ ∈ {−1, 1} so that + λe0 × x0i ∼ Fxi
holds for all seven correspondences xi ↔ x0i
Oriented Epipolar Geometry - Model 1 6/18
OK
Oriented Epipolar Geometry - Model 2 7/18
Rejected
Oriented Epipolar Geometry - Model 3 8/18
Rejected
Experiment Image Pairs 9/18
Juice
Shelf
Valbonne
Great Wall
Leuven
Corridor
Experiment Details 10/18
Juice Correspondences: Inliers: Fraction of inliers ε:
447 274 61.30
Models passed: Time speed-up:
37.77% 35.08%
Experiment Details 11/18
Shelf Correspondences: Inliers: Fraction of inliers ε:
126 43 34.13
Models passed: Time speed-up:
11.29% 15.82%
Experiment Details 12/18
Valbonne Correspondences: Inliers: Fraction of inliers ε:
216 42 19.44
Models passed: Time speed-up:
6.41% 37.17%
Experiment Details 13/18
Great Wall Correspondences: Inliers: Fraction of inliers ε:
318 68 21.38
Models passed: Time speed-up:
7.29% 45.96%
Experiment Details 14/18
Leuven Correspondences: Inliers: Fraction of inliers ε:
793 379 47.79
Models passed: Time speed-up:
71.82% 18.52%
Experiment Details 15/18
Corridor Correspondences: Inliers: Fraction of inliers ε:
607 394 64.91
Models passed: Time speed-up:
89.82% 5.77%
Experiment Summary - Number of Models 16/18
500,000 samples drawn models
rejected
passed [%]
ε [%]
Juice
1,221,932
760,354
37.77
61.30
Shelf
1,233,770
1,094,533
11.29
34.13
Valbonne
1,256,648
1,176,042
6.41
19.44
Great Wall
1,274,018
1,181,084
7.29
21.38
Leuven
1,194,238
336,515
71.82
47.79
Corridor
1,187,380
120,916
89.82
64.91
Experiment Summary - Time 17/18
standard
oriented
speed-up [%]
Juice
1.4
0.9
35.08
Shelf
53.6
45.1
15.82
Valbonne
930.8
584.8
37.17
Great Wall
1109.0
599.3
45.96
Leuven
18.5
15.1
18.52
Corridor
1.2
1.2
5.77
Conclusions 18/18
• In comparison to standard RANSAC, introduction of the oriented epipolar constraint reduced the running time by 5% to 46%, compared with standard RANSAC. • The evaluation of the orientation test takes virtually no time compared to the epipolar geometry computation. • More models were rejected in the wide-baseline setting than in the narrow-baseline one. • The fraction of rejected models is proportional to the fraction of outliers among the tentative correspondences.
x2 x1
x3
x2 x3
x1
−x3
