sliding
A N U N S U P E RV I S E D A P P R O A C H TO D E T E C T I N G A N D I S O L AT I N G AT H L E T I C M O V E M E N T S Terry T. Um and Dana Kulić
University of Waterloo Department of Electrical & Computer Engineering
Terry Taewoong Um
Terry Taewoong Um ([email protected])
Motivation Scenario We want to analyze athletic movements from a long sequence of skeleton data
Procedure CMU motion capture dataset (http://www.cs.cmu.edu/~jkh/uobio/bio.html)
Data collection
Segmentation (Extracting athletic movements)
Movement analysis B. Tekin. et., “Direct Prediction of 3D Body Poses from Motion Compensated Sequences” (2016)
1 / 12
What are athletic movements? • There can be various ways to define athletic movements • Universal principles that apply to all sports tasks (R. Bartlett, Introduction to sports biomechanics: Analysing human movement patterns. Routledge, 2007)
- Use of the stretch–shortening cycle of muscle contraction. - Minimisation of energy used to perform the task. - Control of redundant degrees of freedom in the segmental chain.
• In this research, we attempt to detect stretch-shortening cycles from a long sequence of skeleton data
2 / 12
What are athletic movements? • We will detect flexed poses followed by a explosive and coherent movement http://goo.gl/Gc5Gej
PreStretch Stretch
pre- stretch stretch
• •
Storing potential energy (Usually) flexing body limbs
•
Stretching body limbs explosively & coherently
0
T Pre-Stretch Stretch
https://en.wikipedia.org/wiki/Pitcher
proposed measure
Stretch followed by pre-stretch detect!
pre-stretch
stretch 3 / 12
Detection of Athletic Movements • I will measure the extent of Pre-Stretch &
Stretch
, and blend them
for detecting athletic movement
9 / 12
Detection of Pre-Stretch Phase • Manipulability
(T. Yoshikawa, 1985)
The robot arm’s ability to change the position or orientation of its endpoint in each direction
pre-stretch
stretch
• From the Jacobian 𝑱 of the limb, pre-stretch close to 1 stretch
close to 0
4 / 12
Detection of Stretching Phase • In the stretching phase, all joints should move coherently • Different joint movements make a coherent kinematic synergy at the limb’s endpoint.
problem2
problem1
• Prob. 1) How can we represent kinematic synergy of joints? Forward Kinematics
in products of exponentials (POE) formula (by Lie group formulation)
joint axis
Kinematic Synergy
joint angle
An approximate abstraction of all joints’ instantaneous movements
5 / 12
Detection of Stretching Phase • Baker–Campbell–Hausdorff (BCH) formula
If A and B is enough small,
• If we keep merging POEs one after another, Kinematic Dimensionality Reduction (KDR) 𝜔𝐵𝐶𝐻 ∈ ℝ3 for rotations, ∈ ℝ6 rotations & translations • Note that 𝜔𝑖 𝑞𝑖 is small enough if we set 𝑞𝑖 as instantaneous changes of joint angles
6 / 12
Detection of Stretching Phase • Prob. 2) How can we detect coherency from the kinematic synergy, 𝜔𝐵𝐶𝐻 ?
Displaying the trajectory of kinematic synergy values (𝜔𝐵𝐶𝐻 ∈ ℝ3 )
𝜔𝐵𝐶𝐻
𝜔𝐵𝐶𝐻
(Right leg)
(Left leg)
7 / 12
Detection of Stretching Phase • Prob. 2) How can we detect coherency from the kinematic synergy, 𝜔𝐵𝐶𝐻 ?
stretching part
• In the stretching part, 𝜔𝐵𝐶𝐻 travels long distance toward a certain direction.
• Or we can say 𝜔𝐵𝐶𝐻 momentarily forms a submanifold (lower dimensional manifold).
[eigenvalue ratio for detecting submanifold]
[Scaling factor for measuring distance]
Coherency
8 / 12
Detection of Athletic Movements 1.
Use fixed size (e.g. 0.25sec) sliding window (Let’s assume that the window contains 2m+1 data from 𝑥𝑖−𝑚 to 𝑥𝑖+𝑚 )
2.
Calculate manipulability at (𝑖 − 𝑚) for detecting pre-stretches 𝜇(𝑖−𝑚)
3.
Calculate scaled coherency from 𝑥(𝑖−𝑚 ∶ 𝑖+𝑚) for detecting stretches
4.
Blend them with a ratio 𝛽
5.
Report athletic movements when 𝜐𝑖 > 𝜐𝑡ℎ𝑟𝑒𝑠ℎ𝑜𝑙𝑑
9 / 12
Experiments • The proposed method is evaluated on the CMU motion capture dataset • We experimented with 4 athletic motions (which has about 60 trials in total), which are jumping, soccer kicking, baseball pitching, and golf, and 2 long sequences of random motions (walking, hand waving, squat, etc.)
All codes are available from http://terryum.io/ publications/#EM BC2016
9 / 12
Results Measure Pre-stretch
Stretch
340
10 / 12
Results
(a)
: Test with long sequences of activities
(b)
11 / 12
Remarks • No prior knowledge for movements is required • No machine learning techniques have been applied, that is, no training data or training time is required. • You can enhance the detection performance with the combination of machine learning techniques. • Or, you can use the proposed representation (kinematic synergy) for machine learning tasks, e.g., human activity classification • In the future work, we will verified the proposed concepts in machine learning tasks
“An Unsupervised approach to Detecting and isolating athletic movements,” T. T. Um and D. Kulić, EMBC2016
12 / 12
Summary • An approach to detect athletic movements of body parts is proposed • Pre-stretching motions are captured by using manipulability
• We proposed kinematic dimensionality reduction (KDR) method for representing kinematic synergy of joint movements • Stretching motions are captured by detecting submanifold in the kinematic synergy • By detecting sequential pre-stretching and stretching motions, we can detect athletic movements of the body parts • The proposed approach has been verified with CMU mocap dataset (The Matlab code is available from http://terryum.io/publications/#EMBC2016)
“An Unsupervised approach to Detecting and isolating athletic movements,” T. T. Um and D. Kulić, EMBC2016
12 / 12
University of Waterloo Department of Electrical & Computer Engineering
Terry Taewoong Um
Terry Taewoong Um ([email protected])
Motivation Scenario We want to analyze athletic movements from a long sequence of skeleton data
Procedure CMU motion capture dataset (http://www.cs.cmu.edu/~jkh/uobio/bio.html)
Data collection
Segmentation (Extracting athletic movements)
Movement analysis B. Tekin. et., “Direct Prediction of 3D Body Poses from Motion Compensated Sequences” (2016)
1 / 12
What are athletic movements? • There can be various ways to define athletic movements • Universal principles that apply to all sports tasks (R. Bartlett, Introduction to sports biomechanics: Analysing human movement patterns. Routledge, 2007)
- Use of the stretch–shortening cycle of muscle contraction. - Minimisation of energy used to perform the task. - Control of redundant degrees of freedom in the segmental chain.
• In this research, we attempt to detect stretch-shortening cycles from a long sequence of skeleton data
2 / 12
What are athletic movements? • We will detect flexed poses followed by a explosive and coherent movement http://goo.gl/Gc5Gej
PreStretch Stretch
pre- stretch stretch
• •
Storing potential energy (Usually) flexing body limbs
•
Stretching body limbs explosively & coherently
0
T Pre-Stretch Stretch
https://en.wikipedia.org/wiki/Pitcher
proposed measure
Stretch followed by pre-stretch detect!
pre-stretch
stretch 3 / 12
Detection of Athletic Movements • I will measure the extent of Pre-Stretch &
Stretch
, and blend them
for detecting athletic movement
9 / 12
Detection of Pre-Stretch Phase • Manipulability
(T. Yoshikawa, 1985)
The robot arm’s ability to change the position or orientation of its endpoint in each direction
pre-stretch
stretch
• From the Jacobian 𝑱 of the limb, pre-stretch close to 1 stretch
close to 0
4 / 12
Detection of Stretching Phase • In the stretching phase, all joints should move coherently • Different joint movements make a coherent kinematic synergy at the limb’s endpoint.
problem2
problem1
• Prob. 1) How can we represent kinematic synergy of joints? Forward Kinematics
in products of exponentials (POE) formula (by Lie group formulation)
joint axis
Kinematic Synergy
joint angle
An approximate abstraction of all joints’ instantaneous movements
5 / 12
Detection of Stretching Phase • Baker–Campbell–Hausdorff (BCH) formula
If A and B is enough small,
• If we keep merging POEs one after another, Kinematic Dimensionality Reduction (KDR) 𝜔𝐵𝐶𝐻 ∈ ℝ3 for rotations, ∈ ℝ6 rotations & translations • Note that 𝜔𝑖 𝑞𝑖 is small enough if we set 𝑞𝑖 as instantaneous changes of joint angles
6 / 12
Detection of Stretching Phase • Prob. 2) How can we detect coherency from the kinematic synergy, 𝜔𝐵𝐶𝐻 ?
Displaying the trajectory of kinematic synergy values (𝜔𝐵𝐶𝐻 ∈ ℝ3 )
𝜔𝐵𝐶𝐻
𝜔𝐵𝐶𝐻
(Right leg)
(Left leg)
7 / 12
Detection of Stretching Phase • Prob. 2) How can we detect coherency from the kinematic synergy, 𝜔𝐵𝐶𝐻 ?
stretching part
• In the stretching part, 𝜔𝐵𝐶𝐻 travels long distance toward a certain direction.
• Or we can say 𝜔𝐵𝐶𝐻 momentarily forms a submanifold (lower dimensional manifold).
[eigenvalue ratio for detecting submanifold]
[Scaling factor for measuring distance]
Coherency
8 / 12
Detection of Athletic Movements 1.
Use fixed size (e.g. 0.25sec) sliding window (Let’s assume that the window contains 2m+1 data from 𝑥𝑖−𝑚 to 𝑥𝑖+𝑚 )
2.
Calculate manipulability at (𝑖 − 𝑚) for detecting pre-stretches 𝜇(𝑖−𝑚)
3.
Calculate scaled coherency from 𝑥(𝑖−𝑚 ∶ 𝑖+𝑚) for detecting stretches
4.
Blend them with a ratio 𝛽
5.
Report athletic movements when 𝜐𝑖 > 𝜐𝑡ℎ𝑟𝑒𝑠ℎ𝑜𝑙𝑑
9 / 12
Experiments • The proposed method is evaluated on the CMU motion capture dataset • We experimented with 4 athletic motions (which has about 60 trials in total), which are jumping, soccer kicking, baseball pitching, and golf, and 2 long sequences of random motions (walking, hand waving, squat, etc.)
All codes are available from http://terryum.io/ publications/#EM BC2016
9 / 12
Results Measure Pre-stretch
Stretch
340
10 / 12
Results
(a)
: Test with long sequences of activities
(b)
11 / 12
Remarks • No prior knowledge for movements is required • No machine learning techniques have been applied, that is, no training data or training time is required. • You can enhance the detection performance with the combination of machine learning techniques. • Or, you can use the proposed representation (kinematic synergy) for machine learning tasks, e.g., human activity classification • In the future work, we will verified the proposed concepts in machine learning tasks
“An Unsupervised approach to Detecting and isolating athletic movements,” T. T. Um and D. Kulić, EMBC2016
12 / 12
Summary • An approach to detect athletic movements of body parts is proposed • Pre-stretching motions are captured by using manipulability
• We proposed kinematic dimensionality reduction (KDR) method for representing kinematic synergy of joint movements • Stretching motions are captured by detecting submanifold in the kinematic synergy • By detecting sequential pre-stretching and stretching motions, we can detect athletic movements of the body parts • The proposed approach has been verified with CMU mocap dataset (The Matlab code is available from http://terryum.io/publications/#EMBC2016)
“An Unsupervised approach to Detecting and isolating athletic movements,” T. T. Um and D. Kulić, EMBC2016
12 / 12
