Applying qualitative reasoning to a driver's cognitive mental load
Applying qualitative reasoning to a driver's cognitive mental load Shinichiro Sega, Hirotoshi Iwasaki
Denso IT Laboratory Hironori Hiraishi
Akita National College of Tech. Fumio Mizoguchi
Tokyo Univ. of Science
11/8/18
Copyright (C) 2011 Denso IT Laboratory, Inc. All Rights Reserved Copyright (C) 2011 WisdomTex, Inc. All Rights Reserved
1/12
Contents
11/8/18
Background Services with Driver's Cognitive Load Qualitative Reasoning Simulation Driver's Cognitive Load Model Verification COGSIM: Cognitive Qualitative SIMulation Conclusions
Copyright (C) 2011 Denso IT Laboratory, Inc. All Rights Reserved Copyright (C) 2011 WisdomTex, Inc. All Rights Reserved
2/12
Background
Next Generation Services in Car
Telematics
Services in Cooperation with Smartphone
Entune, G-BOOK Terminal Mode, NaviCon®
Common Internet Services
Denso NaviCon® 11/8/18
Copyright (C) 2011 Denso IT Laboratory, Inc. All Rights Reserved Copyright (C) 2011 WisdomTex, Inc. All Rights Reserved
3/12
Services with Driver's Cognitive Load
As automobile services increase, it is necessary to consider the driver's cognitive load.
Not Considering Cognitive Load
11/8/18
Considering Cognitive Load
Copyright (C) 2011 Denso IT Laboratory, Inc. All Rights Reserved Copyright (C) 2011 WisdomTex, Inc. All Rights Reserved
4/12
Qualitative Reasoning Simulation Qualitative Model of Eye Movement Perception Temporal Change
temp
sac
decide
Saccade
Fixation
Eye Movement
Eye Movement 11/8/18
decide
Decision
fix
+
extern
extern temp
+
External Change
Cognition
Constraints of the Model
sac
fix
Description Method of Kuipers (1994)
Acquisition of Eye Movement
Copyright (C) 2011 Denso IT Laboratory, Inc. All Rights Reserved Copyright (C) 2011 WisdomTex, Inc. All Rights Reserved
5/12
Driver's Cognitive Load Model Design the model based on four techniques
11/8/18
Qualitative Reasoning Simulation [Kuipers, 1994] Perception-Cognition-Action Theory Eye Movement [Findlay, 1999] Resource Allocation Theory [Norman, 1975]
Copyright (C) 2011 Denso IT Laboratory, Inc. All Rights Reserved Copyright (C) 2011 WisdomTex, Inc. All Rights Reserved
6/12
Interpretation of “Cognition” Model
Use case “Action for drive and Environment change” 1. 2. 3.
intern increases caused by Action → used_a increases env increases caused by Eye movement → used_p increases used_a and used_p increase → rest and decide decrease M+ intern
M+
perform+
used_a perform‐
M+
used_p M+
+
env +
rest
extern
temp
M‐
M‐
+
used
resource
11/8/18
+
+
decide
sac fix Copyright (C) 2011 Denso IT Laboratory, Inc. All Rights Reserved Copyright (C) 2011 WisdomTex, Inc. All Rights Reserved
7/12
Verification
Purpose
Verify the Model Using Real Data Verify the Interpretation Rate between env and used_p
Method
Eye Movement Data ・Saccade ・Fixation
Acquisition of Driving Data from in-vehicle LAN
Driving Data
Acquisition of Eye Movement by NAC Eye Mark Recorder 11/8/18
・Velocity ・Braking signal ・Steering angle ・Accelerator pressing rate
Copyright (C) 2011 Denso IT Laboratory, Inc. All Rights Reserved Copyright (C) 2011 WisdomTex, Inc. All Rights Reserved
8/12
Interpretation of Real Data Relations of acquired data and the parameters Parameter Acquired data sac
Frequency of saccade (times/sec)
fix
Continuous time of fixation (sec)
perform+
Rate of pressing the accelerator (%) Steering angle (degree)
perform-
Braking signal Steering angle (contradicts accel.)
intern
Velocity (km/h)
Constraints of the Model
Parameters other than the acquired data can be determined by using our model as in the following equations.
extern = sac – intern (1) decide = -(perform+) (2) temp = fix - decide = fix + perform+ (3) env = extern + temp = sac - intern + fix + perform+ = sac + fix (4) rest = perform(5) used = -rest = -(perform-) (6) used_a = intern = perform+ (7) used_p = used - used_a = -(perform- + perform+) (8) Red: Parameters related with acquired data
11/8/18
Copyright (C) 2011 Denso IT Laboratory, Inc. All Rights Reserved Copyright (C) 2011 WisdomTex, Inc. All Rights Reserved
9/12
COGSIM
COGSIM: Cognitive Qualitative SIMulation
Using QSIM: Qualitative SIMulation Analyzing real data
Eye Movement Driving Data
Analyzing Eye Movement 11/8/18
Analyzing Driving Data
Copyright (C) 2011 Denso IT Laboratory, Inc. All Rights Reserved Copyright (C) 2011 WisdomTex, Inc. All Rights Reserved
10/12
Interpretation rate of real data
env M+ used_p
Results of the changes of env and used_p Subject A
Road1 Subject B Subject A Road2 Subject B Subject A Road3 Subject B
Average
Our model can interpret 84% of the data. 11/8/18
Copyright (C) 2011 Denso IT Laboratory, Inc. All Rights Reserved Copyright (C) 2011 WisdomTex, Inc. All Rights Reserved
11/12
Conclusions
We applied Qualitative reasoning simulation to designing the cognitive model of the driver. We verified the model using real data. We interpreted the driver's cognitive load from the change of real data.
Thank you very much for kind attention.
11/8/18
Copyright (C) 2011 Denso IT Laboratory, Inc. All Rights Reserved Copyright (C) 2011 WisdomTex, Inc. All Rights Reserved
12/12
Denso IT Laboratory Hironori Hiraishi
Akita National College of Tech. Fumio Mizoguchi
Tokyo Univ. of Science
11/8/18
Copyright (C) 2011 Denso IT Laboratory, Inc. All Rights Reserved Copyright (C) 2011 WisdomTex, Inc. All Rights Reserved
1/12
Contents
11/8/18
Background Services with Driver's Cognitive Load Qualitative Reasoning Simulation Driver's Cognitive Load Model Verification COGSIM: Cognitive Qualitative SIMulation Conclusions
Copyright (C) 2011 Denso IT Laboratory, Inc. All Rights Reserved Copyright (C) 2011 WisdomTex, Inc. All Rights Reserved
2/12
Background
Next Generation Services in Car
Telematics
Services in Cooperation with Smartphone
Entune, G-BOOK Terminal Mode, NaviCon®
Common Internet Services
Denso NaviCon® 11/8/18
Copyright (C) 2011 Denso IT Laboratory, Inc. All Rights Reserved Copyright (C) 2011 WisdomTex, Inc. All Rights Reserved
3/12
Services with Driver's Cognitive Load
As automobile services increase, it is necessary to consider the driver's cognitive load.
Not Considering Cognitive Load
11/8/18
Considering Cognitive Load
Copyright (C) 2011 Denso IT Laboratory, Inc. All Rights Reserved Copyright (C) 2011 WisdomTex, Inc. All Rights Reserved
4/12
Qualitative Reasoning Simulation Qualitative Model of Eye Movement Perception Temporal Change
temp
sac
decide
Saccade
Fixation
Eye Movement
Eye Movement 11/8/18
decide
Decision
fix
+
extern
extern temp
+
External Change
Cognition
Constraints of the Model
sac
fix
Description Method of Kuipers (1994)
Acquisition of Eye Movement
Copyright (C) 2011 Denso IT Laboratory, Inc. All Rights Reserved Copyright (C) 2011 WisdomTex, Inc. All Rights Reserved
5/12
Driver's Cognitive Load Model Design the model based on four techniques
11/8/18
Qualitative Reasoning Simulation [Kuipers, 1994] Perception-Cognition-Action Theory Eye Movement [Findlay, 1999] Resource Allocation Theory [Norman, 1975]
Copyright (C) 2011 Denso IT Laboratory, Inc. All Rights Reserved Copyright (C) 2011 WisdomTex, Inc. All Rights Reserved
6/12
Interpretation of “Cognition” Model
Use case “Action for drive and Environment change” 1. 2. 3.
intern increases caused by Action → used_a increases env increases caused by Eye movement → used_p increases used_a and used_p increase → rest and decide decrease M+ intern
M+
perform+
used_a perform‐
M+
used_p M+
+
env +
rest
extern
temp
M‐
M‐
+
used
resource
11/8/18
+
+
decide
sac fix Copyright (C) 2011 Denso IT Laboratory, Inc. All Rights Reserved Copyright (C) 2011 WisdomTex, Inc. All Rights Reserved
7/12
Verification
Purpose
Verify the Model Using Real Data Verify the Interpretation Rate between env and used_p
Method
Eye Movement Data ・Saccade ・Fixation
Acquisition of Driving Data from in-vehicle LAN
Driving Data
Acquisition of Eye Movement by NAC Eye Mark Recorder 11/8/18
・Velocity ・Braking signal ・Steering angle ・Accelerator pressing rate
Copyright (C) 2011 Denso IT Laboratory, Inc. All Rights Reserved Copyright (C) 2011 WisdomTex, Inc. All Rights Reserved
8/12
Interpretation of Real Data Relations of acquired data and the parameters Parameter Acquired data sac
Frequency of saccade (times/sec)
fix
Continuous time of fixation (sec)
perform+
Rate of pressing the accelerator (%) Steering angle (degree)
perform-
Braking signal Steering angle (contradicts accel.)
intern
Velocity (km/h)
Constraints of the Model
Parameters other than the acquired data can be determined by using our model as in the following equations.
extern = sac – intern (1) decide = -(perform+) (2) temp = fix - decide = fix + perform+ (3) env = extern + temp = sac - intern + fix + perform+ = sac + fix (4) rest = perform(5) used = -rest = -(perform-) (6) used_a = intern = perform+ (7) used_p = used - used_a = -(perform- + perform+) (8) Red: Parameters related with acquired data
11/8/18
Copyright (C) 2011 Denso IT Laboratory, Inc. All Rights Reserved Copyright (C) 2011 WisdomTex, Inc. All Rights Reserved
9/12
COGSIM
COGSIM: Cognitive Qualitative SIMulation
Using QSIM: Qualitative SIMulation Analyzing real data
Eye Movement Driving Data
Analyzing Eye Movement 11/8/18
Analyzing Driving Data
Copyright (C) 2011 Denso IT Laboratory, Inc. All Rights Reserved Copyright (C) 2011 WisdomTex, Inc. All Rights Reserved
10/12
Interpretation rate of real data
env M+ used_p
Results of the changes of env and used_p Subject A
Road1 Subject B Subject A Road2 Subject B Subject A Road3 Subject B
Average
Our model can interpret 84% of the data. 11/8/18
Copyright (C) 2011 Denso IT Laboratory, Inc. All Rights Reserved Copyright (C) 2011 WisdomTex, Inc. All Rights Reserved
11/12
Conclusions
We applied Qualitative reasoning simulation to designing the cognitive model of the driver. We verified the model using real data. We interpreted the driver's cognitive load from the change of real data.
Thank you very much for kind attention.
11/8/18
Copyright (C) 2011 Denso IT Laboratory, Inc. All Rights Reserved Copyright (C) 2011 WisdomTex, Inc. All Rights Reserved
12/12
