poster - Institute For Systems and Robotics
Generation of Meaningful Robot Expressions with Active Learning Giovanni Saponaro, Alexandre Bernardino VisLab, Institute for Systems and Robotics, Instituto Superior Técnico, Lisbon, Portugal
Computer Vision Laboratory
PÓLO DO I.S.T
Active Learning
Overview
We address the problem of communicating emotions with a humanoid robot merely with its body joint movements, without facial information, querying human users for emotional scores attributed to the movements. Machine learning can help convey the intended emotions more clearly, by selecting the next actions and parameters that need tuning and rewarding successful action–emotion matches.
Motivation • Transmit emotions to human observers with the iCub humanoid robot • Explore expressivity (or lack thereof) of body movements when facial expressions are disabled
Setup and Proposed Approach Each combination is a simple Bayesian Network with an Action and an Emotion node:
Library of Movements select movement with max entropy gain
θ11 θ12 P (E = el |A = an) = [θln] = . . θ1A
• Action–emotion matches modelled as a multinomial distribution • Rows: actions (with a discretized joint parameter), columns: emotions, Σl θln = 1 ∀ action • Active: learner selects the row yielding highest entropy gain as next query, we update the
parameters of that row P (E=el |A=an) P (E=el |A=an)+s • ← : update step, where #an is the number of trials performed #an #an+1 with A = an up to this point and s is the Likert score resulting from the current trial answer (normalized to a probability) • The framework is scalable to use a matrix with hundres of rows, where clusters of rows encode the same action with different joint parameters, and can be awarded/penalized
Results • Head actions are associated to emotions clearly, arm gestures are often ambiguous • Unclear actions are chosen and displayed more times than clear ones, as reflected in the
higher denominators of match scores
0.8
0.6
match score
human scores E
Active Learning
θ11=0.87/2
uniform θln=0.2/1
0.8
4/5 θ41=0.53/2
0.6
0.4 0.2
0.2
nod
punch
punch
look out disapproval success distraction
look out disapproval success distraction
thumbs up thumbs down
anger agreement
Action
5.02/10
0.2
3.95/18
0.4 0.2
6.45/23 0.4 0.2
0
0
nod
nod
nod
punch
punch
look out
Action
disapproval success distraction
look out disapproval success distraction
thumbs up thumbs down
anger agreement
Action
thumbs down
Emotion
Entropy evolution for the "nod" action
Emotion
Entropy evolution for the "punch" action
Entropy evolution for the "look out" action
1.8
1.8
1.6
1.6
1.6
1.4
1.4
1.4
1.2
1.2
1.2
1
1
1
0.8
0.8
0.8
0.6
0.6
0.6
0.4
0.4
0.4
0.2
0.2
0.2
1
5
10
15
anger agreement
Action
1.8
init
disapproval success distraction
thumbs up
anger agreement
Emotion
Figure: The iCub performing the “thumbs down” gesture.
20
0
init
Number of interviewed subjects
1
5
10
15
20
Number of interviewed subjects
Entropy evolution for the "thumbs up" action
Meaning initial position of first neck joint final position of first neck joint initial velocity of first neck joint time to transition from (0) to (1) final velocity of first neck joint time to transition from (1) to (0)
4.51/18
0
thumbs down
Parameter (0) x0 (1) x0 (0) x˙ 0 t0→1 (1) x˙ 0 t1→0
θ14=6.46/23
0.6
Score
Score
Score
2.97/12
thumbs up
Table: Parameters of the “nod” action
12.08/20 12.07/19 0.8
0.6
look out
Description Expected perceived emotion el head tilts up and down agreement rapidly extend fist in front of robot anger abruptly deviate robot head and gaze to a side distraction show fist and move thumb up success show fist and move thumb down disapproval
Matches after interviewing 20 people
0.8
punch
Table: Library of robot action movements
Emotion
θ42=4.77/18
3/12
anger agreement
Action
Matches after interviewing 15 people
0.6 0.4
thumbs down
anger agreement
6.28/10 7.8/11
disapproval success distraction
thumbs up
Emotion
Matches after interviewing 10 people
0.8
Example Library of Movements and Parameters
0.2
nod
Emotion
show (the one requiring more correction, i.e., with most entropy) • Active learner can also choose an action movement parameter and a discretized value to fine-tune
0.4
nod
look out
0
1.34/7
0
Action
• Input human opinions into an active machine learning program that selects the next action to
0.6
0
thumbs down
Figure: Human users being interviewed about what emotion they perceive from robot movements.
3.34/6
θ33=θ44=1.2/2
0
thumbs up
• Survey human users about what they feel when the robot performs said movements
2.9/6
0.8
0.4
punch
• Design a library of pre-defined movements
Matches after interviewing 5 people
Score
A
Matches after interviewing 1 person
Score
tuned movement parameters
Initial matches
Score
Questionnaire
Action an nod punch look out thumbs up thumbs down
· · · θL1 · · · θL2 . . . .. · · · θLA
0
init
1
5
10
15
20
Number of interviewed subjects
Entropy evolution for the "thumbs down" action
1.8
1.8
1.6
1.6
1.4
1.4
1.2
1.2
1
1
0.8
0.8
0.6
0.6
0.4
0.4
0.2
0.2
0
0
Entropy Posterior entropy Entropy gain init
1
5
10
15
Number of interviewed subjects
20
init
1
5
10
15
20
Number of interviewed subjects
Future Work • Improve corpus of robot movements, explore using other limbs (legs), systematically analyze
the human affective experience with or without robot facial expressions • Study the degree of robot emotional expressivity due to its appearance vs the mechanical range of permitted movements • Try different parameter initialization: uniform vs expert knowledge • Explore optimization strategies other than maximum-entropy criterion: artificial neural networks, reinforcement learning
Summary We model a mapping between robot movements (with no facial expressions) and emotions perceived by human users. Match parameters are updated with a questionnaire, and as correspondences take shape we can observe (i) which body joint actions are expressive, (ii) which actions should be performed by a robot in order to transmit a given emotion. As opposed to random action selection or querying for a fixed sequence of actions, the Active Learning framework gives the system the ability to inquire about movements that are ambiguous, showing them more often than easily-perceived ones.
User Survey • Human subjects see robot actions and respond to a Likert questionnaire • After each robot movement, a user is asked to evaluate statements such as “This action
expresses anger” with a score among [strongly disagree] [disagree] [neither agree nor disagree] [agree] [strongly agree] • Responses are sent as a probability vector to the Active Learning module, which updates action–emotion parameters • We interviewed 20 people (non-roboticists): 50% males, 50% females, mean age 29.7, stddev 4.49 • Type of study: within-subject with self-assessment evaluation
References [1] J. Li and M. Chignell. Communication of Emotion in Social Robots through Simple Head and Arm Movements. International Journal of Social Robotics, 2010. [2] S. Tong and D. Koller. Active Learning for Parameter Estimation in Bayesian Networks. In T. K. Leen, T. G. Dietterich, and V. Tresp, editors, Neural Information Processing Systems (NIPS), pages 647–653. MIT Press, 2000. This work was supported by Fundação para a Ciência e a Tecnologia (ISR/IST pluriannual funding and grant SFRH/BD/61910/2009).
Computer Vision Laboratory
PÓLO DO I.S.T
Active Learning
Overview
We address the problem of communicating emotions with a humanoid robot merely with its body joint movements, without facial information, querying human users for emotional scores attributed to the movements. Machine learning can help convey the intended emotions more clearly, by selecting the next actions and parameters that need tuning and rewarding successful action–emotion matches.
Motivation • Transmit emotions to human observers with the iCub humanoid robot • Explore expressivity (or lack thereof) of body movements when facial expressions are disabled
Setup and Proposed Approach Each combination is a simple Bayesian Network with an Action and an Emotion node:
Library of Movements select movement with max entropy gain
θ11 θ12 P (E = el |A = an) = [θln] = . . θ1A
• Action–emotion matches modelled as a multinomial distribution • Rows: actions (with a discretized joint parameter), columns: emotions, Σl θln = 1 ∀ action • Active: learner selects the row yielding highest entropy gain as next query, we update the
parameters of that row P (E=el |A=an) P (E=el |A=an)+s • ← : update step, where #an is the number of trials performed #an #an+1 with A = an up to this point and s is the Likert score resulting from the current trial answer (normalized to a probability) • The framework is scalable to use a matrix with hundres of rows, where clusters of rows encode the same action with different joint parameters, and can be awarded/penalized
Results • Head actions are associated to emotions clearly, arm gestures are often ambiguous • Unclear actions are chosen and displayed more times than clear ones, as reflected in the
higher denominators of match scores
0.8
0.6
match score
human scores E
Active Learning
θ11=0.87/2
uniform θln=0.2/1
0.8
4/5 θ41=0.53/2
0.6
0.4 0.2
0.2
nod
punch
punch
look out disapproval success distraction
look out disapproval success distraction
thumbs up thumbs down
anger agreement
Action
5.02/10
0.2
3.95/18
0.4 0.2
6.45/23 0.4 0.2
0
0
nod
nod
nod
punch
punch
look out
Action
disapproval success distraction
look out disapproval success distraction
thumbs up thumbs down
anger agreement
Action
thumbs down
Emotion
Entropy evolution for the "nod" action
Emotion
Entropy evolution for the "punch" action
Entropy evolution for the "look out" action
1.8
1.8
1.6
1.6
1.6
1.4
1.4
1.4
1.2
1.2
1.2
1
1
1
0.8
0.8
0.8
0.6
0.6
0.6
0.4
0.4
0.4
0.2
0.2
0.2
1
5
10
15
anger agreement
Action
1.8
init
disapproval success distraction
thumbs up
anger agreement
Emotion
Figure: The iCub performing the “thumbs down” gesture.
20
0
init
Number of interviewed subjects
1
5
10
15
20
Number of interviewed subjects
Entropy evolution for the "thumbs up" action
Meaning initial position of first neck joint final position of first neck joint initial velocity of first neck joint time to transition from (0) to (1) final velocity of first neck joint time to transition from (1) to (0)
4.51/18
0
thumbs down
Parameter (0) x0 (1) x0 (0) x˙ 0 t0→1 (1) x˙ 0 t1→0
θ14=6.46/23
0.6
Score
Score
Score
2.97/12
thumbs up
Table: Parameters of the “nod” action
12.08/20 12.07/19 0.8
0.6
look out
Description Expected perceived emotion el head tilts up and down agreement rapidly extend fist in front of robot anger abruptly deviate robot head and gaze to a side distraction show fist and move thumb up success show fist and move thumb down disapproval
Matches after interviewing 20 people
0.8
punch
Table: Library of robot action movements
Emotion
θ42=4.77/18
3/12
anger agreement
Action
Matches after interviewing 15 people
0.6 0.4
thumbs down
anger agreement
6.28/10 7.8/11
disapproval success distraction
thumbs up
Emotion
Matches after interviewing 10 people
0.8
Example Library of Movements and Parameters
0.2
nod
Emotion
show (the one requiring more correction, i.e., with most entropy) • Active learner can also choose an action movement parameter and a discretized value to fine-tune
0.4
nod
look out
0
1.34/7
0
Action
• Input human opinions into an active machine learning program that selects the next action to
0.6
0
thumbs down
Figure: Human users being interviewed about what emotion they perceive from robot movements.
3.34/6
θ33=θ44=1.2/2
0
thumbs up
• Survey human users about what they feel when the robot performs said movements
2.9/6
0.8
0.4
punch
• Design a library of pre-defined movements
Matches after interviewing 5 people
Score
A
Matches after interviewing 1 person
Score
tuned movement parameters
Initial matches
Score
Questionnaire
Action an nod punch look out thumbs up thumbs down
· · · θL1 · · · θL2 . . . .. · · · θLA
0
init
1
5
10
15
20
Number of interviewed subjects
Entropy evolution for the "thumbs down" action
1.8
1.8
1.6
1.6
1.4
1.4
1.2
1.2
1
1
0.8
0.8
0.6
0.6
0.4
0.4
0.2
0.2
0
0
Entropy Posterior entropy Entropy gain init
1
5
10
15
Number of interviewed subjects
20
init
1
5
10
15
20
Number of interviewed subjects
Future Work • Improve corpus of robot movements, explore using other limbs (legs), systematically analyze
the human affective experience with or without robot facial expressions • Study the degree of robot emotional expressivity due to its appearance vs the mechanical range of permitted movements • Try different parameter initialization: uniform vs expert knowledge • Explore optimization strategies other than maximum-entropy criterion: artificial neural networks, reinforcement learning
Summary We model a mapping between robot movements (with no facial expressions) and emotions perceived by human users. Match parameters are updated with a questionnaire, and as correspondences take shape we can observe (i) which body joint actions are expressive, (ii) which actions should be performed by a robot in order to transmit a given emotion. As opposed to random action selection or querying for a fixed sequence of actions, the Active Learning framework gives the system the ability to inquire about movements that are ambiguous, showing them more often than easily-perceived ones.
User Survey • Human subjects see robot actions and respond to a Likert questionnaire • After each robot movement, a user is asked to evaluate statements such as “This action
expresses anger” with a score among [strongly disagree] [disagree] [neither agree nor disagree] [agree] [strongly agree] • Responses are sent as a probability vector to the Active Learning module, which updates action–emotion parameters • We interviewed 20 people (non-roboticists): 50% males, 50% females, mean age 29.7, stddev 4.49 • Type of study: within-subject with self-assessment evaluation
References [1] J. Li and M. Chignell. Communication of Emotion in Social Robots through Simple Head and Arm Movements. International Journal of Social Robotics, 2010. [2] S. Tong and D. Koller. Active Learning for Parameter Estimation in Bayesian Networks. In T. K. Leen, T. G. Dietterich, and V. Tresp, editors, Neural Information Processing Systems (NIPS), pages 647–653. MIT Press, 2000. This work was supported by Fundação para a Ciência e a Tecnologia (ISR/IST pluriannual funding and grant SFRH/BD/61910/2009).
