slides - Stanford Computer Graphics Laboratory
SCAPE: Shape Completion and Animation of People By Dragomir Anguelov, Praveen Srinivasan, Daphne Koller, Sebastian Thrun, Jim Rodgers, James Davis From SIGGRAPH 2005 Presentation for CS468 by Emilio Antúnez
1
Motivation
It is difficult to get highresolution body scans It is even harder at video rates By building up a human model, you could synthesize a highresolution scan from sparse/incomplete data Accurate model is most easily created by learning from sample scans 2
Preexisting Work in Deformable Human Models I
Deformations described relative to a template shape Pose deformations given relative to local joints in an articulated model Bodyshape deformations described using displacement vectors from PCA 3
Preexisting Work in Deformable Human Models II
Pose and shape deformations rarely addressed together Most similar work by Sumner and Popović – Retargets pose deformation to another mesh – Does not learn a model
4
Paper Contributions
Learning an affine deformation model for both pose and shape Shape completion for scan of an arbitrary human target Body shape manipulation for motion capture animation 5
Presentation Overview
Data Acquisition Learning the Human Model Applications – Shape Completion – Motion Capture Animation
Limitations
6
Data Format / Assumptions
Each input model is a deformation of a fixedtopology triangle mesh Models divided into three categories – One template model – Template subject in different poses – Different people in (roughly) same pose
Articulated skeleton assigned to each mesh 7
Data Acquisition and Processing
8
Learning the Human Model
Pose and shape deformations described pertriangle using linear transformations Pose transformations learned from template subject in different poses Body shape transformations learned by comparing different subjects to template 9
Pose Deformation I
Rigid (skeletal) deformations are represented separately from nonrigid ones Transformations are given in relative coordinate system where one of the corners is fixed at the origin
final triangle
Rl[k]
Qk
O template triangle
10
Pose Deformation II
Triangle edges are not forced to be consistent Final synthesized mesh reduces the leastsquares error between mesh points and triangle deformations
final triangle
Rl[k]
Qk
O template triangle
11
Learning Pose Deformation Model I
Rigid rotation is known from skeleton Nonrigid transformation is underdefined Q matrix is computed by requiring adjacent triangles’ nonrigid transformations to be similar 12
Learning Pose Deformation Model II
Nonrigid deformation modeled as an affine function of adjacent joint angles
In practice, some of the degrees of freedom are removed for constrained joings 13
Pose Deformation Learning Results
14
BodyShape Deformation
Body shape is modeled as an additional linear transform, S
S is underdetermined (like Q) Again, solved using a smoothness constraint
15
Learning the Shape Deformation Model
The matrix coefficients for all body shape transformations are vectorized Principal component analysis is used to parameterize the shape transform vectors
16
Shape Deformation Learning Results
17
Shape Completion I
Assuming you know some of the node positions, estimate the others Must estimate pose and body shape This optimization is highly nonlinear in the pose Empirically found that optimizing over all variables at once produces bad results Instead, SCAPE iterates solving 18
Shape Completion II
Empirically found that optimizing over all variables at once produces bad results Instead, SCAPE iterates, solving each of these in order: – – –
Pose Mesh estimate Body shape
Results in a “completed” mesh and a “predicted” mesh 19
Partial View Completion
Skeletal and pointcorrespondences may be off if too much data is missing Iterate between the shape completion algorithm previously described and remapping the point correspondences
20
Partial View Completion Results
21
Motion Capture Animation
Motion capture data provides the pose data Body shape parameters can be set arbitrarily Since markers are generally placed on body surface (not in the bones), mesh is constrained to lie in the space of body shapes encoded by the model 22
Motion Capture Animation Results
23
Limitations
Assumes that pose deformation and body shape are mostly independent Models only pose deformations from skeletal motion
24
Conclusion
SCAPE learns simple body model which distinguishes pose and body shape deformations Creates reasonable shape completions, even when large features are missing Allows for flexible reconstruction of moving model from motion capture data 25
1
Motivation
It is difficult to get highresolution body scans It is even harder at video rates By building up a human model, you could synthesize a highresolution scan from sparse/incomplete data Accurate model is most easily created by learning from sample scans 2
Preexisting Work in Deformable Human Models I
Deformations described relative to a template shape Pose deformations given relative to local joints in an articulated model Bodyshape deformations described using displacement vectors from PCA 3
Preexisting Work in Deformable Human Models II
Pose and shape deformations rarely addressed together Most similar work by Sumner and Popović – Retargets pose deformation to another mesh – Does not learn a model
4
Paper Contributions
Learning an affine deformation model for both pose and shape Shape completion for scan of an arbitrary human target Body shape manipulation for motion capture animation 5
Presentation Overview
Data Acquisition Learning the Human Model Applications – Shape Completion – Motion Capture Animation
Limitations
6
Data Format / Assumptions
Each input model is a deformation of a fixedtopology triangle mesh Models divided into three categories – One template model – Template subject in different poses – Different people in (roughly) same pose
Articulated skeleton assigned to each mesh 7
Data Acquisition and Processing
8
Learning the Human Model
Pose and shape deformations described pertriangle using linear transformations Pose transformations learned from template subject in different poses Body shape transformations learned by comparing different subjects to template 9
Pose Deformation I
Rigid (skeletal) deformations are represented separately from nonrigid ones Transformations are given in relative coordinate system where one of the corners is fixed at the origin
final triangle
Rl[k]
Qk
O template triangle
10
Pose Deformation II
Triangle edges are not forced to be consistent Final synthesized mesh reduces the leastsquares error between mesh points and triangle deformations
final triangle
Rl[k]
Qk
O template triangle
11
Learning Pose Deformation Model I
Rigid rotation is known from skeleton Nonrigid transformation is underdefined Q matrix is computed by requiring adjacent triangles’ nonrigid transformations to be similar 12
Learning Pose Deformation Model II
Nonrigid deformation modeled as an affine function of adjacent joint angles
In practice, some of the degrees of freedom are removed for constrained joings 13
Pose Deformation Learning Results
14
BodyShape Deformation
Body shape is modeled as an additional linear transform, S
S is underdetermined (like Q) Again, solved using a smoothness constraint
15
Learning the Shape Deformation Model
The matrix coefficients for all body shape transformations are vectorized Principal component analysis is used to parameterize the shape transform vectors
16
Shape Deformation Learning Results
17
Shape Completion I
Assuming you know some of the node positions, estimate the others Must estimate pose and body shape This optimization is highly nonlinear in the pose Empirically found that optimizing over all variables at once produces bad results Instead, SCAPE iterates solving 18
Shape Completion II
Empirically found that optimizing over all variables at once produces bad results Instead, SCAPE iterates, solving each of these in order: – – –
Pose Mesh estimate Body shape
Results in a “completed” mesh and a “predicted” mesh 19
Partial View Completion
Skeletal and pointcorrespondences may be off if too much data is missing Iterate between the shape completion algorithm previously described and remapping the point correspondences
20
Partial View Completion Results
21
Motion Capture Animation
Motion capture data provides the pose data Body shape parameters can be set arbitrarily Since markers are generally placed on body surface (not in the bones), mesh is constrained to lie in the space of body shapes encoded by the model 22
Motion Capture Animation Results
23
Limitations
Assumes that pose deformation and body shape are mostly independent Models only pose deformations from skeletal motion
24
Conclusion
SCAPE learns simple body model which distinguishes pose and body shape deformations Creates reasonable shape completions, even when large features are missing Allows for flexible reconstruction of moving model from motion capture data 25
