2) Subject model - Geometric primitives - Detailed Body Scans - Human Shape models
3) Inference - Observation likelihood - Local optimization - Particle Based optimization
Title Overview of the chapter
• 3
Title of the chapter Human Motion
Title of the chapter Kinematic Chains Motivated from robotics: The human motion can be expressed via a „kinematic chain“, a series of local rigid body motions (along the limbs).
The model parameters to optimize for are rigid body motions. ? How to model RBM‘S ? Bregler et.al. CVPR-98
TitleDefinition of the chapter A rigid body motion is an affine transformation that preserves distances and orientations
0 1 Euclidean a1 : (non linear) X = @a2 A ! X` = RX + t; R 2 R3£3 ; t 2 R3 a3 RRT = I; det(R) = 1
Affine: (linear) 0 1 a1 µ ¶ Ba C R t 2 C ! X` = X=B X @a A 0 1 3 1
Title of the chapter Euler Angles
S
Titleconfusion of the chapter Euler Angles:
Careful: Euler angles are a typical source of confusion When using Euler angles 2 things have to be specified
1) Convention: X-Y-Z, Z-Y-X, Z-Y-Z … 2) Rotations about the static spatial frame or the moving body frame
of the chapter Euler Angles: Title drawbacks I
• Gimbal lock: When two of the axis align one degree of freedom is lost !
• Parameterization is not unique • Lots of conventions for Euler angles
Title of the chapter Quaternions
• A quaternion has 4 components: • They generalize complex numbers with additional properties
• Unit length quaternions can be used to carry out rotations. The set they form is called
Title of the chapter Quaternions
• Rotations can be carried away directly in parameter space via the quaternion product: - Concatenation of rotations:
- If we want to rotate a vector
where
is the quat conjugate
Title of the chapter Quaternions Quaternions have no singularities
Derivatives exist and are linearly independent Quaternion product allows to perform rotations But all this comes at the expense of using 4 numbers instead of 3 - Enforce quadratic constraint
Title of the chapter Axis-angle For human motion modeling it is often needed to specify the axis of rotation of a joint Any rotation about the origin can be expressed in terms of the axis of rotation and the angle of rotation with the exponential map
Title of the chapter Lie Groups / Lie Algebras Definition: A group is an n-dimensional Lie-group, if the set of its elements can be represented as a continuously differentiable manifold of dimension n, on which the group product and inverse are continuously differentiable functions as well
Lie Group M
M
0
exp
Lie algebra
Title of the chapter Lie Groups / Lie Algebras so(2) =
µ
=µ
cos(Á) sin(Á)
¡sin(Á)
µ¡ sin(Á)
cos(Á)
cos(Á)
¶
; µ@
µ
cos(Á) sin(Á)
µ ¡cos(Á)¶ j =µ 0 ¡sin(Á) 0 1
¡sin(Á) cos(Á)
¡1¶ µ! ^ 0 =
¶
j
0
so(2) = fA 2 R2£2 jA = ¡AT g
If a body rotates at constant velocity about an axis, the velocity can be written as
q(t) _ =! ^ q(t) µ
0 Example: 1
(1) ¡1¶ µ1¶ 0
0
µ ¶ µ 0 0 = 1 ; 1
¡1¶ µ0¶ µ¡1¶ 0 1 = 0
(1) Is a time invariant linear differential equation which may be integrated to give:
q(t) = exp(^ ! t)q(0)
Title of the chapter Axis-angle (3D) Given a vector
the skew symetric matrix is
It is the matrix form of the cross-product:
Title of the chapter Exponential map Exponential map (3D):
If we rotate
units of time
Title of the chapter Exponential map
Exploiting the properties of skew symetric matrices
Rodriguez formula
Closed form!
Title of the chapter Twists What about rotation & translation ? The twist coordinates are defined as
And the twist is defined as
Note: A degenerate screw can be used to model rotations around axes in space !
Title of the chapter Exponential map The rigid body motion can be computed in closed form as well
2) Subject model - Geometric primitives - Detailed Body Scans - Human Shape models
3) Inference - Observation likelihood - Local optimization - Particle Based optimization
Title Overview of the chapter
Title of the chapter Articulation
B
S
In a rest position we have
Title of the chapter Articulation
S
Title of the chapter Articulation
S
Title of the chapter Articulation
S
The coordinates of the point in the spatial frame
Title of the chapter Product of exponentials Product of exponentials formula
of the chapter InverseTitle Kinematics Supose we want to find the angles to reach a specific goal
of the chapter InverseTitle Kinematics Supose we want to find the angles to reach a specific goal
• The problem is non-linear • Linearize with the articulated Jacobian
Title of the chapter Articulated Jacobian
The Jacobian using twists is extremely simple and easy to compute
1) Every column corresponds to the contribution of i-th joint to the end-effector motion 2) Maps an increment of joint angles to the end-effector twist
Title of the chapter Articulated Jacobian Intuition: Linear combination of twists B
Title of the chapter Articulated Jacobian Intuition: Linear combination of twists
Title of the chapter Articulated Jacobian Intuition: Linear combination of twists
of the chapter PoseTitle Parameters Pose parameters: root + joint angles
Title of the chapter Pose Jacobian Maps increments in the pose parameters to increments in end-effector position
Any feature that can be predicted from the model and is fast to compute
Title of the chapter Optimization Optimize
Model-Image
O
Image features
Project model
Title of the chapter Optimization
Extract features
Predict and match
Optimize
Title Matching of the chapter TIPS: 1) Match image to model and model to image 2) Careful removing outliers 1)
Look along normal model contour directions
2) Discretize and match
DAGM ´11
Title ofsquares the chapter Least Given a set of correspondences we can model the likelihood as
Map is found by minimizing the log-likelihood
Model predictions
Image observations
Title ofsquares the chapter Least Express the problem in vector form
Residual for match 1
Title of the chapter Local Optimization
Gradient
Take a step in that direction
~Hessian
Title of the chapter Jacobian 1) Pose 3) Projection
C 2) World-camera transformation
O
3
2
1
Title of the chapter Other likelihoods
2D-3D error point-to-line distance
3D-3D error point-to-point distance
of the chapter DistanceTitle transforms
inconsistent consistent 1) Push model inside silhouette 2) Force the model to explain the image Distance transform + overlapp term Sminchisescu F & G 2001
Title of the chapter Region based Region-based
Rosenhahn et.al.
Use model as region mask Q that separates foreground from background
Optical flow
Bregler and Malik Parameterize flow with human motion model
of the chapter Local Title optimization It is fast and accurate
Prone to local minima Requires initialization Matching cost is ambiguous
2) Subject model - Geometric primitives - Detailed Body Scans - Human Shape models
3) Inference - Observation likelihood - Local optimization - Particle Based inference
Title Overview of the chapter
Title of the Filter chapter Particle First order Markov process
Image observations at t State space, pose parameters at t
• Once I know
,
is independent on previous measurements
• Once I know the state, the new measurement becomes independent on the others
Title of the Filter chapter Particle
Distribution approximated with a set of weighted samples
Title of the Filter chapter Particle
Posterior t-1
Posterior t
Temporal Dynamics
Diffusion
Title of the Filter chapter Particle
Condensation, Isaard and Blake 1996
TitleSampling of the chapter
sample weight
weight
TitleProblems of the chapter Observation likelihood is highly multimodal !
Video from Sminchisescu and Triggs
1) Multiple optima 2) Huge search space
Title of the Filter chapter Annealed Particle Iteratively evaluate smooth versions of Particles reduced by a factor >10 Less prone to local optima Not as robust as Bayesian Still computationaly expensive
Deutscher et.al. Gall et.al.
weight resample diffuse
Title of the chapter Efficient sampling I Hybrid MCMC
Localy optimize every sample of MCMC Likelihood levelsets Cho and Fleet
Samples
of the chapter EfficientTitle sampling II
Covariance Scaled Sampling Scatter particles along cost function valley Sminchisescu and Triggs
Explore high dimensional space more efficiently Dedicates some particles to explore globaly
Title of the chapter Discussion Generative modeling: - Need to model Kinematics - Need to model Shape - Need to model Observation Likelihood - Texture - Ilumination - ufff lots of work so … IS THIS THE END OF GENERATIVE ?
Title of the chapter End of Generative ? Well, depends on the application…
In totaly uncontrolled scenarios will never work! But the accuracy is still higher and they generalize to complex motions better than discriminative approaches Useful as refinement stage coupled with discriminative initialization