Engineering and Technology Degree Level: Ph.D. Abstract ID#: 1051
Graduate Category: Engineering and Technology Degree Level: Ph.D. Abstract ID#: 1051
Assessment of Human Vulnerability in a Touch-screen Game; Metrics and Analysis Payam Parsinejad
Rifat Sipahi (PI)
Problem Definition
Physiological Metric: pNN50 [3]
• Human under excessive stress/mental workload prone to error and they are vulnerable [1].
• pNN50 is the temporal changes of the normal-to-normal (NN) heartbeat intervals.
• Affective computing infers human vulnerability by analyzing human physiological responses [8].
• pNN50 decreases when mental workload (or stress) level increases. IBI
• However, studies show that affective computing NOT RELIABLE [2]
Objective • We propose an alternative way to sense human mental states while interacting with a machine. • HYPOTHESIS: Humans’ behavioral patterns must be affected by task difficulty changes.
where #NN50 is number of 𝚫𝚫𝚫𝚫𝚫𝚫 > 𝟓𝟓𝟓𝟓 ms, and
Results: Performance Metrics
• We verify the reliability of the proposed behavioral measurements to infer human mental states using the performance and physiological metrics, pNN50 as a baseline.
Air Traffic Management Game • An air traffic management game within MATLAB is designed with two levels of difficulty, namely Easy and Difficult, to induce different workload demands.
White
White
White
Green
White
Red
# of Strokes # of Assignments • Subjects are challenged by the complexity of the difficult game (effect of game difficulty). • Subjects’ performance are lower in difficult game vs. easy games.
Figure: Left: Easy Game. Right: Difficult Game. Red and Blue arrow: un-assigned and assigned trajectories.
• Subjects show consistent playing in the two identical easy games.
Results: Behavioral Metrics
Experimental Protocol • Easy (E) 1 and 2 are identical to the easy Level. • Rest (R) 1 and 2 are the rest period: Subjects relaxed and did not play the game. 60 sec.
60 sec.
60 sec.
REST 1
EASY 1
DIFFICULT
REST 2
60 sec.
EASY 2
60 sec.
• Easy (E) 1 and 2 are identical to the easy Level. • Rest (R) 1 and 2 are the rest period: Subjects relaxed and did not play the game. Effort: Mean Energy (ME) Response Time: Mean Delay Time (MSD) • Fewer strokes, drawn faster in D & more strokes drawn slower in E1 and E2.
Performance Metric Subjects’ Goal
• longer decision-making time to decide to assign the airplanes to the airports in D [??].
Object
A
Target
B
Number of Finger Strokes
Number of Successful Airplane Assignment
• Limited statistical power on SE and MSDur (modest sample size) results not reported.
Results: Physiological Metric (pNN50)
Proposed Performance Metrics
C
• pNN50 is significantly affected by the game difficulty.
Behavioral Metric
• pNN50 shows subjects’ subjective perception of game difficulty [4]. • pNN50 the lowest in D & larger in E1 and E2 inverse correlation to mental workload.
Effort kinetic energy for a lumped point mass
Sum of the Strokes Energy in Game g
mass m unknown, ½ m removed from energy calculation
Mean Energy of Strokes in Game g
Total Energy of Stroke i
• Game order has non-linear effect on pNN50 Dependent on game order.
Conclusion • ME and MSD show strong correlation with performance and physiological (pNN50) metrics. • Behavioral metrics can differentiate between difficult and easy tasks encountered by the subjects. • Behavioral metrics infer subjects’ task load increases, and thereby to infer subjects vulnerability.
Stroke Duration & Stroke Delay Time (Decision Making) [5-7] (Hick-Hyman Law and Fitt’s Law)
References 1. 2. 3. 4. 5. 6. 7. 8.
Mean Stroke Duration in Game g
Mean Stroke Delay in Game g
g = E1, D, E2 | N = Number of points in stroke i | n = Number of strokes in game g
Mehler, B., Reimer, B., and Coughlin, J. F., 2012. “Sensitivity of physiological measures for detecting systematic variations in cognitive demand from a working memory task an on-road study across three age groups”. Human Factors: The Journal of the Human Factors and Ergonomics Society, 54(3), pp. 396–412. Quintana, D. S., and Heathers, J. A., 2014. “Considerations in the assessment of heart rate variability in bio-behavioral research”. Frontiers in Psychology, 5. Malik, M., Bigger, J. T., Camm, A. J., Kleiger, R. E., Malliani, A., Moss, A. J., and Schwartz, P. J., 1996. “Heart rate variability standards of measurement, physiological interpretation, and clinical use”. European Heart Journal, 17(3), pp. 354–381. Parsinejad, P., and Sipahi, R., 2014. “A touchscreen game to induce mental workload on human subjects”. In 40th Annual Northeast Bioengineering Conference (NEBEC), IEEE, pp. 1–2. Hick, W. E., 1952. “On the rate of gain of information”. Quarterly Journal of Experimental Psychology, 4(1), pp. 11–26. Hyman, R., 1953. “Stimulus information as a determinant of reaction time”. Journal of experimental psychology, 45(3), p. 188. Fitts, P. M., 1954. “The information capacity of the human motor system in controlling the amplitude of movement”. Journal of experimental psychology, 47(6), p. 381. Picard, Rosalind W., and Roalind Picard. Affective computing. Vol. 252. Cambridge: MIT press, 1997.
Acknowledgment The work of R. Sipahi is supported by DARPA N66001‐11‐1‐4161 grant. Any opinions presented on this poster are those of authors, not those of the funding agency.
Assessment of Human Vulnerability in a Touch-screen Game; Metrics and Analysis Payam Parsinejad
Rifat Sipahi (PI)
Problem Definition
Physiological Metric: pNN50 [3]
• Human under excessive stress/mental workload prone to error and they are vulnerable [1].
• pNN50 is the temporal changes of the normal-to-normal (NN) heartbeat intervals.
• Affective computing infers human vulnerability by analyzing human physiological responses [8].
• pNN50 decreases when mental workload (or stress) level increases. IBI
• However, studies show that affective computing NOT RELIABLE [2]
Objective • We propose an alternative way to sense human mental states while interacting with a machine. • HYPOTHESIS: Humans’ behavioral patterns must be affected by task difficulty changes.
where #NN50 is number of 𝚫𝚫𝚫𝚫𝚫𝚫 > 𝟓𝟓𝟓𝟓 ms, and
Results: Performance Metrics
• We verify the reliability of the proposed behavioral measurements to infer human mental states using the performance and physiological metrics, pNN50 as a baseline.
Air Traffic Management Game • An air traffic management game within MATLAB is designed with two levels of difficulty, namely Easy and Difficult, to induce different workload demands.
White
White
White
Green
White
Red
# of Strokes # of Assignments • Subjects are challenged by the complexity of the difficult game (effect of game difficulty). • Subjects’ performance are lower in difficult game vs. easy games.
Figure: Left: Easy Game. Right: Difficult Game. Red and Blue arrow: un-assigned and assigned trajectories.
• Subjects show consistent playing in the two identical easy games.
Results: Behavioral Metrics
Experimental Protocol • Easy (E) 1 and 2 are identical to the easy Level. • Rest (R) 1 and 2 are the rest period: Subjects relaxed and did not play the game. 60 sec.
60 sec.
60 sec.
REST 1
EASY 1
DIFFICULT
REST 2
60 sec.
EASY 2
60 sec.
• Easy (E) 1 and 2 are identical to the easy Level. • Rest (R) 1 and 2 are the rest period: Subjects relaxed and did not play the game. Effort: Mean Energy (ME) Response Time: Mean Delay Time (MSD) • Fewer strokes, drawn faster in D & more strokes drawn slower in E1 and E2.
Performance Metric Subjects’ Goal
• longer decision-making time to decide to assign the airplanes to the airports in D [??].
Object
A
Target
B
Number of Finger Strokes
Number of Successful Airplane Assignment
• Limited statistical power on SE and MSDur (modest sample size) results not reported.
Results: Physiological Metric (pNN50)
Proposed Performance Metrics
C
• pNN50 is significantly affected by the game difficulty.
Behavioral Metric
• pNN50 shows subjects’ subjective perception of game difficulty [4]. • pNN50 the lowest in D & larger in E1 and E2 inverse correlation to mental workload.
Effort kinetic energy for a lumped point mass
Sum of the Strokes Energy in Game g
mass m unknown, ½ m removed from energy calculation
Mean Energy of Strokes in Game g
Total Energy of Stroke i
• Game order has non-linear effect on pNN50 Dependent on game order.
Conclusion • ME and MSD show strong correlation with performance and physiological (pNN50) metrics. • Behavioral metrics can differentiate between difficult and easy tasks encountered by the subjects. • Behavioral metrics infer subjects’ task load increases, and thereby to infer subjects vulnerability.
Stroke Duration & Stroke Delay Time (Decision Making) [5-7] (Hick-Hyman Law and Fitt’s Law)
References 1. 2. 3. 4. 5. 6. 7. 8.
Mean Stroke Duration in Game g
Mean Stroke Delay in Game g
g = E1, D, E2 | N = Number of points in stroke i | n = Number of strokes in game g
Mehler, B., Reimer, B., and Coughlin, J. F., 2012. “Sensitivity of physiological measures for detecting systematic variations in cognitive demand from a working memory task an on-road study across three age groups”. Human Factors: The Journal of the Human Factors and Ergonomics Society, 54(3), pp. 396–412. Quintana, D. S., and Heathers, J. A., 2014. “Considerations in the assessment of heart rate variability in bio-behavioral research”. Frontiers in Psychology, 5. Malik, M., Bigger, J. T., Camm, A. J., Kleiger, R. E., Malliani, A., Moss, A. J., and Schwartz, P. J., 1996. “Heart rate variability standards of measurement, physiological interpretation, and clinical use”. European Heart Journal, 17(3), pp. 354–381. Parsinejad, P., and Sipahi, R., 2014. “A touchscreen game to induce mental workload on human subjects”. In 40th Annual Northeast Bioengineering Conference (NEBEC), IEEE, pp. 1–2. Hick, W. E., 1952. “On the rate of gain of information”. Quarterly Journal of Experimental Psychology, 4(1), pp. 11–26. Hyman, R., 1953. “Stimulus information as a determinant of reaction time”. Journal of experimental psychology, 45(3), p. 188. Fitts, P. M., 1954. “The information capacity of the human motor system in controlling the amplitude of movement”. Journal of experimental psychology, 47(6), p. 381. Picard, Rosalind W., and Roalind Picard. Affective computing. Vol. 252. Cambridge: MIT press, 1997.
Acknowledgment The work of R. Sipahi is supported by DARPA N66001‐11‐1‐4161 grant. Any opinions presented on this poster are those of authors, not those of the funding agency.
