Extrinsic camera calibration
CS4495/6495 Introduction to Computer Vision 3C-L1 Extrinsic camera calibration
Recall: Modeling projection Projection equations • Compute intersection with
Perspective Projection of ray from (x,y,z) to COP • Derived using similar triangles
( X ,Y , Z ) ( d
X Z
, d
Y
, d )
Z
(assumes normal Z negative – we’ll change later)
Recall: Modeling projection Projection equations ( X ,Y , Z ) ( d
X
, d
Z
Y
, d )
Z
We get the projection by throwing out the last coordinate:
( x ', y ') ( d
X Z
,d
Y Z
)
Recall: Homogeneous coordinates Is this a linear transformation? No – division by the (not constant) Z is non-linear
Trick: add one more coordinate:
homogeneous image (2D) coordinates
homogeneous scene (3D) coordinates
Recall: Homogeneous coordinates Converting from homogeneous coordinates:
(this makes homogenous coordinates invariant under scale)
Recall: Perspective Projection • Projection is a matrix multiply using
homogeneous coordinates (and |z|): 1 0 0
0
0
1
0
0
1/ f
0 0 0
x y | z 1
|
x x y y f , f | z | | z | | z / f |
u, v
S. Seitz
But… • In all this discussion we have the notion of a camera’s
coordinate system – an origin and an orientation. • We put the center of projection at this origin and the optic axis down the z axis. • So to do geometric reasoning about the world we need to relate the coordinate system of the world to that of the camera and the image. • Today we’ll do from the world to the camera, and next time from the camera to the image.
Geometric Camera calibration • We want to use the camera to tell us things
about the world. • So we need the relationship between coordinates in
the world and coordinates in the image: geometric camera calibration • For reference see Forsyth and Ponce, sections
1.2 and 1.3.
Geometric Camera calibration Composed of 2 transformations: • From some (arbitrary) world coordinate system to
the camera’s 3D coordinate system. Extrinisic parameters (or camera pose) • From the 3D coordinates in the camera frame to the
2D image plane via projection. Intrinisic parameters
Camera Pose 𝑦 𝑥 𝑧 c w
World Coordinates
𝑦 𝑥
T
𝑧
Camera Coordinates
Quiz How many degrees of freedom are there in specifying the extrinsic parameters? a) 5 b) 6 c) 3 d) 9
Rigid Body Transformations Need a way to specify the six degrees-offreedom of a rigid body. Why 6?
A rigid body is a collection of points whose positions relative to each other can’t change
Fix one point, 3 DOF 3
Fix second point, 2 more Third point adds 1 DOF (must maintain more DOF, for distance constraint) rotation around line +2
+1
Notation (from F&P) • Superscript references coordinate frame
•
AP
is coordinates of P in frame A • BP is coordinates of P in frame B
x P y OP z
k
A
O
A
i
j
A
P
A
A
A
A
A
A
x ·i
A
A
y· j
A
A
z ·k
A
Translation Only B
P
A
P
B
k
j
OA
k
O A
i
or B
P
j B
OA
A
P
B
i
A
O
A
A
P
B
B
B
Translation • Using homogeneous coordinates, translation can
be expressed as a matrix multiplication. B
P
A
P OA
BP I T 1 0
B
B
3x3 identity A OA P 1 1
(Translation is commutative)
Rotation O P iA
B
𝐵 𝐴𝑅
P
B A
A
R P
means describing frame A in the coordinate system of frame B
jA
Ax A k A y iB A z
jB
Bx B kB y B z
Rotation i A .i B B R i .j A A B i A .k B B = i A
j A .i B j A .jB j A .k B B
jA
k A .i B k A .jB k A .k B B
k A The columns of the rotation matrix are the axes of frame A expressed in frame B. Why?
Rotation i A .i B B R i .j A A B i A .k B B = iA
j A .i B j A .jB j A .k B B
A iBT A T = jB Ak T B
jA
k A .i B k A .jB k A .k B B
k A
Orthogonal matrix! Why?
Example: Rotation about z axis
What is the rotation matrix?
cos( ) R ( ) sin( ) 0 Z
sin( ) cos( ) 0
0 0 1
Combine 3 to get arbitrary rotation • Euler angles: Z, X’, Z’’ • Or heading, pitch roll: world Z, new X, new Y …
• Or roll, pitch and yaw … • Or Azimuth, elevation, roll… • Three basic matrices: order matters, but we’ll
not focus on that
Combine 3 to get arbitrary rotation 1 R ( ) 0 0 X
0 cos( ) sin( )
0 sin( ) cos( )
co s( ) R ( ) sin ( ) 0
sin ( ) co s( ) 0
cos( ) R ( ) 0 sin( )
0 1 0
Z
Y
0 0 1
sin( ) 0 cos( )
Rotation in homogeneous coordinates • Using homogeneous coordinates, rotation can be
expressed as a matrix multiplication. B
P
B A
A
R P
B P AB R T 1 0
0 P 1 1
• Rotation is not commutative
A
Rigid transformations
B
P
B A
R P OA A
B
Rigid transformations (con’t) Unified treatment using homogeneous coordinates: BP 1 T 1 0
AB R T 0
B
OA A R T 1 0
0 P 1 1
B
B
A
OA P 1 1 A
Rigid transformations (con’t) And even better: B P AB R T 1 0
B
A A OA P B P AT 1 1 1
so AP A BP BT 1 1
B A
T
1
AP 1
Invertible!
Translation and rotation From frame A to B: Non-homogeneous (“regular) coordinates B
p
B A
R
A
p
B A
t
3x3 rotation matrix
3x1 translation vector
Translation and rotation Homogeneous coordinates: B
B
p AT
p 0
B
B A
A
R
0
p
0
| x B t y A | z 1 1
Homogenous coordinates allows us to write coordinate transforms as a single matrix!
From World to Camera Translation from world to camera frame
Rotation from world to camera frame C
p
Point now in camera frame
C W
R
W
p
C W
t Point in world frame
Nonhomogeneous coordinates
From World to Camera
C
p 0
C W
R
0
0
| C W t p W | 1
Homogeneous coordinates
From world to camera is the extrinsic parameter matrix (4x4) (sometimes 3x4 if using for next step in projection – not worrying about inversion)
Quiz How many degrees of freedom are there in the 3x4 extrinsic parameter matrix? a) 12 b) 6 c) 9 d) 3
From World to Camera
C
p 0
C W
R
0
0
| C W t p W | 1
Homogeneous coordinates
From world to camera is the extrinsic parameter matrix (4x4) (sometimes 3x4 if using for next step in projection – not worrying about inversion)
Recall: Modeling projection Projection equations • Compute intersection with
Perspective Projection of ray from (x,y,z) to COP • Derived using similar triangles
( X ,Y , Z ) ( d
X Z
, d
Y
, d )
Z
(assumes normal Z negative – we’ll change later)
Recall: Modeling projection Projection equations ( X ,Y , Z ) ( d
X
, d
Z
Y
, d )
Z
We get the projection by throwing out the last coordinate:
( x ', y ') ( d
X Z
,d
Y Z
)
Recall: Homogeneous coordinates Is this a linear transformation? No – division by the (not constant) Z is non-linear
Trick: add one more coordinate:
homogeneous image (2D) coordinates
homogeneous scene (3D) coordinates
Recall: Homogeneous coordinates Converting from homogeneous coordinates:
(this makes homogenous coordinates invariant under scale)
Recall: Perspective Projection • Projection is a matrix multiply using
homogeneous coordinates (and |z|): 1 0 0
0
0
1
0
0
1/ f
0 0 0
x y | z 1
|
x x y y f , f | z | | z | | z / f |
u, v
S. Seitz
But… • In all this discussion we have the notion of a camera’s
coordinate system – an origin and an orientation. • We put the center of projection at this origin and the optic axis down the z axis. • So to do geometric reasoning about the world we need to relate the coordinate system of the world to that of the camera and the image. • Today we’ll do from the world to the camera, and next time from the camera to the image.
Geometric Camera calibration • We want to use the camera to tell us things
about the world. • So we need the relationship between coordinates in
the world and coordinates in the image: geometric camera calibration • For reference see Forsyth and Ponce, sections
1.2 and 1.3.
Geometric Camera calibration Composed of 2 transformations: • From some (arbitrary) world coordinate system to
the camera’s 3D coordinate system. Extrinisic parameters (or camera pose) • From the 3D coordinates in the camera frame to the
2D image plane via projection. Intrinisic parameters
Camera Pose 𝑦 𝑥 𝑧 c w
World Coordinates
𝑦 𝑥
T
𝑧
Camera Coordinates
Quiz How many degrees of freedom are there in specifying the extrinsic parameters? a) 5 b) 6 c) 3 d) 9
Rigid Body Transformations Need a way to specify the six degrees-offreedom of a rigid body. Why 6?
A rigid body is a collection of points whose positions relative to each other can’t change
Fix one point, 3 DOF 3
Fix second point, 2 more Third point adds 1 DOF (must maintain more DOF, for distance constraint) rotation around line +2
+1
Notation (from F&P) • Superscript references coordinate frame
•
AP
is coordinates of P in frame A • BP is coordinates of P in frame B
x P y OP z
k
A
O
A
i
j
A
P
A
A
A
A
A
A
x ·i
A
A
y· j
A
A
z ·k
A
Translation Only B
P
A
P
B
k
j
OA
k
O A
i
or B
P
j B
OA
A
P
B
i
A
O
A
A
P
B
B
B
Translation • Using homogeneous coordinates, translation can
be expressed as a matrix multiplication. B
P
A
P OA
BP I T 1 0
B
B
3x3 identity A OA P 1 1
(Translation is commutative)
Rotation O P iA
B
𝐵 𝐴𝑅
P
B A
A
R P
means describing frame A in the coordinate system of frame B
jA
Ax A k A y iB A z
jB
Bx B kB y B z
Rotation i A .i B B R i .j A A B i A .k B B = i A
j A .i B j A .jB j A .k B B
jA
k A .i B k A .jB k A .k B B
k A The columns of the rotation matrix are the axes of frame A expressed in frame B. Why?
Rotation i A .i B B R i .j A A B i A .k B B = iA
j A .i B j A .jB j A .k B B
A iBT A T = jB Ak T B
jA
k A .i B k A .jB k A .k B B
k A
Orthogonal matrix! Why?
Example: Rotation about z axis
What is the rotation matrix?
cos( ) R ( ) sin( ) 0 Z
sin( ) cos( ) 0
0 0 1
Combine 3 to get arbitrary rotation • Euler angles: Z, X’, Z’’ • Or heading, pitch roll: world Z, new X, new Y …
• Or roll, pitch and yaw … • Or Azimuth, elevation, roll… • Three basic matrices: order matters, but we’ll
not focus on that
Combine 3 to get arbitrary rotation 1 R ( ) 0 0 X
0 cos( ) sin( )
0 sin( ) cos( )
co s( ) R ( ) sin ( ) 0
sin ( ) co s( ) 0
cos( ) R ( ) 0 sin( )
0 1 0
Z
Y
0 0 1
sin( ) 0 cos( )
Rotation in homogeneous coordinates • Using homogeneous coordinates, rotation can be
expressed as a matrix multiplication. B
P
B A
A
R P
B P AB R T 1 0
0 P 1 1
• Rotation is not commutative
A
Rigid transformations
B
P
B A
R P OA A
B
Rigid transformations (con’t) Unified treatment using homogeneous coordinates: BP 1 T 1 0
AB R T 0
B
OA A R T 1 0
0 P 1 1
B
B
A
OA P 1 1 A
Rigid transformations (con’t) And even better: B P AB R T 1 0
B
A A OA P B P AT 1 1 1
so AP A BP BT 1 1
B A
T
1
AP 1
Invertible!
Translation and rotation From frame A to B: Non-homogeneous (“regular) coordinates B
p
B A
R
A
p
B A
t
3x3 rotation matrix
3x1 translation vector
Translation and rotation Homogeneous coordinates: B
B
p AT
p 0
B
B A
A
R
0
p
0
| x B t y A | z 1 1
Homogenous coordinates allows us to write coordinate transforms as a single matrix!
From World to Camera Translation from world to camera frame
Rotation from world to camera frame C
p
Point now in camera frame
C W
R
W
p
C W
t Point in world frame
Nonhomogeneous coordinates
From World to Camera
C
p 0
C W
R
0
0
| C W t p W | 1
Homogeneous coordinates
From world to camera is the extrinsic parameter matrix (4x4) (sometimes 3x4 if using for next step in projection – not worrying about inversion)
Quiz How many degrees of freedom are there in the 3x4 extrinsic parameter matrix? a) 12 b) 6 c) 9 d) 3
From World to Camera
C
p 0
C W
R
0
0
| C W t p W | 1
Homogeneous coordinates
From world to camera is the extrinsic parameter matrix (4x4) (sometimes 3x4 if using for next step in projection – not worrying about inversion)
