Recognition of Human Driving Behaviors based on ... - IEEE Xplore
2008 IEEE/RSJ International Conference on Intelligent Robots and Systems Acropolis Convention Center Nice, France, Sept, 22-26, 2008
Recognition of Human Driving Behaviors based on Stochastic Symbolization of Time Series Signal Wataru Takano, Akihiro Matsushita, Keijiro Iwao and Yoshihiko Nakamura Abstract— This paper describes an imitative learning of driving time series data for intellectual cognition toward future automobiles. The driving pattern primitives consisting of states of the environment, vehicle and driver are symbolized by Hidden Markov Models (HMMs), which can be used for both recognition and generation of the driving patterns. The relationship among the HMMs can be represented by locating the HMMs in a multidimensional space. The contribution of each variable to the HMM space can be analyzed such that important variables can be selected out of the driving data in order to reduce the size of the HMMs. Moreover, this paper presents a hierarchical model with the HMMs abstracting the primitive driving patterns in the lower layer, and another HMMs abstracting the longterm contextual driving patterns which are representation in the HMM space. Tests with a driving simulator and a actual vehicle demonstrate not only the validity of symbolization of driving pattern primitives, recognition and generation, but also availability of key feature selection. The extended hierarchical model is also proved to have a potential to predict the driving data appropriately.
I. I NTRODUCTION Recent advances in sensing and computational technologies have led many researchers to develop various intelligent vehicle control systems, such as longitudinal control system [1][2][3] or lateral control system [4][5]. These control systems depend on manually designed continuous models. However, a data-driven framework for autonomous abstraction or symbolization of a lot of driving patterns is required in order to construct higher-level intellectual cognition for automobiles. In robotics, some researches on symbolization of sensorimotor information have been proposed, such as Recurrent Neural Network with Parametric Bias (RNNPB) [6][7], Module Selection And Identification for Control (MOSAIC) [8], Hidden Markov Models (HMMs) [9][10][11][12]. We have been also developing the cognitive frameworks for humanoid robots by using HMMs, such as a physical communication model[13] or a linguistic model [14], inspired by the discovery of mirror neurons in primates [15][16]. The mirror neurons activate both when a primate observes a demonstrator’s behavior and when he performs the same kind of behavior. This fact means the mirror neurons integrate the motion recognition with motion generation through symbolization of motion patterns. In this paper, we propose an approach to stochastically intellectual cognition for automobiles by using the HMMs. The W. Takano and Y. Nakamura are with Mechano-Informatics, University of Tokyo, Hongo 7-3-1, Bunkyo-ku, Tokyo, 113-8656, Japan
{takano,nakamura}@ynl.t.u-tokyo.ac.jp
A. Matsusita and K. Iwao are with Nissan Research Center, Nissan Motor Co.,Ltd, {matsushita,k-iwao}@mail.nissan.co.jp
978-1-4244-2058-2/08/$25.00 ©2008 IEEE.
approach consists of segmentation, symbolization, recognition and generation of the driving patterns. The validity of the proposed cognitive framework is demonstrated by conducting experiments with a driving simulator and a actual vehicle. We apply this framework to selection of important variables from the driving time series signal. Moreover, we propose a hierarchical model which has the HMMs for the primitive driving patterns in the lower layer and HMMs for longterm contextual ones in the upper layer. The experimental result verifies that the hierarchical model can be used to predict the driving data appropriately. II. V EHICLE C OGNITIVE F RAMEWORK -S YMBOL OF D RIVING PATTERN A cognitive vehicle requires to sense data about states of environment, vehicle and driver, recognize driving situation, and generate the data appropriate for the current situation in order to control itself or support the driver. Imitative symbolization of driving time series signal is a powerful framework to model the driving pattern data. In this paper, we propose the cognitive framework, which allows the vehicle to symbolize, recognize and generate the driving patterns. A. Symbolization of Driving Time Series Signal We use left-to-right Hidden Markov Models to symbolize the driving patterns.An HMM is defined by a set of variables λ = {Q, A, B, Π}, where Q = {q1 , · · · , qn } is a set of nodes, A = {aij } is the matrix whose (i, j) element represents the transition probability from the i-th node to the j-th node, B = {b1 , b2 , · · · , bn } is a set of probability density functions and Π = {π1 , π2 , · · · , πn } is a set of initial node distributions. The driving time series signal O consists of multimodal data, such as environmental data, vehicle data and driver data. The driving time series signal is a sequence of driving patterns. So the measured signal has to be segmented into the driving patterns in order to symbolize the driving skills. We apply the segmentation method to the driving signal such that the driving pattern data Ok can be obtained [17]. The segmentation method is based on the criterion for prediction of following driving data. The parameters of HMMs are optimized using the segmented driving patterns by BaumWelch algorithm [18]. Note that each segmented driving pattern data is classified into an HMM which outputs the largest likelihood that the HMM generates the driving data, and the driving pattern data is used as a training data for the HMM. In this way, the HMMs are gradually optimized by using similar driving pattern data. These optimized HMMs
167
Generation of Driving Pattern
Proto Symbol
where OiG is the driving pattern data generated by the proto symbol λi , and d(λi , λj ) is Kullback Leibler Information between two proto symbols, λi and λj , which indicates the dissimilarity. However, this measure is not symmetric. The symmetric measure D(λi , λj ) is introduced in equation (3).
Symbolization, Recognition of Driving Pattern
HMM
Segmentation of driving pattern
d(λi , λj ) + d(λj , λi ) (3) 2 The proto-symbols are placed in a multidimensional space such that the distances among the proto-symbols in the space become as close as possible to the actual distances, thus minimizing the topological distortions in the mapping [20]. We use the multidimensional scaling method to locate the proto-symbols in the space. D(λi , λj ) =
Driving time series data (environment, vehicle, and driver states)
Fig. 1. Overview of a framework based on symbolization, recognition and generation of driving pattern.
extract features of the driving patterns. We name the HMMs proto-symbols since the HMMs form to the origins of symbols. B. Recognition of Driving Patterns based on Proto-symbols The acquired HMMs can be used for recognition of the driving patterns. The HMM corresponding to the driving pattern should output the largest likelihood and other HMMs should output the small likelihood that HMMs generate the driving pattern data. Therefore, motion recognition can be achieved by detecting the HMM which outputs the largest likelihood as follows. R = arg
max
λi :i=1,2,3,···,N
P (Ok |λi )
(1)
where N is the number of HMMs, P (Ok |λi ) is likelihood that the HMM λi generates the driving pattern data Ok , and R is the recognition result. C. Generation of Driving Patterns based on Proto-symbols The HMMs can be also used to generate the driving patterns by Monte-Carlo methods [11]. At first a sequence of representative nodes is computed from expected duration of staying in each node. Note that the expected duration of 1 i-th node, ti = , is calculated based on the node 1 − ai,i transition probability, where ai,i is the recurrent transition probability in i-th node. The driving pattern data is generated from this sequence of nodes and the probability density functions. A lot of samples of these generated driving pattern data are averaged to the resulting generated driving pattern. Therefore, a cognitive framework for vehicles can be constructed by using the HMMs, which allow for symbolization, recognition and generation of the driving patterns. This is illustrated schematically in Fig.1. D. Geometry of Driving Pattern Symbols It is important not only to symbolize the driving patterns but also to clarify the relationships among the proto-symbols. We focus on the distances among the proto-symbols in order to construct the geometric structure of driving patterns. We use the dissimilarities among the proto-symbols in place of the distances. The dissimilarity measure between the protosymbol λi and λj is given by equation (2)[19]. d(λi , λj ) =
ln P (OiG |λi ) − ln P (OiG |λj )
(2)
E. Key Feature Extraction Based on Proto-symbol Space In the proto-symbol space, the relationships among the proto-symbols are defined by the dissimilarity of them. It means that relative distance of a pair of two proto symbols is important in the proto-symbol space. In this paper, we analyze the degree of influence by each variable of the driving time series signal to the proto-symbol space by comparing an original proto-symbol space with a reconstructed symbol space as depicted by Fig.2. The original proto-symbol space is constructed based on the dissimilarity of the proto symbols which can be computed by using all of the variables included in the driving time series signal as described above. The new proto-symbol space is reconstructed based on the dissimilarity of the proto-symbols where a specific variable is eliminated. Note that the new proto-symbol space is calculated by multidimensional scaling after coordinates of the proto-symbols in the original protosymbol space are set to the initial ones in the new protosymbol space. The degree of change from the original protosymbol space to the reconstructed proto-symbol space can be computed as following. ∆k =
N X
k xoi − xki k2
(4)
i=1
where xoi is the coordinate of i-th proto symbol in the original proto-symbol space, xki is the coordinate of ith proto symbol in the new proto-symbol space which is reconstructed without using k-th variable for the dissimilarity of the proto symbols, and ∆k indicates the degree of change induced by the k-th variable. Here, we define the degree of influence by each variable to the proto symbol space as this measure. We can extract the key features of the variables with large ∆k since the large ∆k represents large contribution to the original proto symbol by the k-th variable. III. H IERARCHIZATION BASED ON P ROTO - SYMBOLS The proto symbols abstract short primitive driving patterns. such as “left-hand turn” or “straight drive”. However, there should be long and complicated driving patterns consisting of the multiple primitive patterns in our daily driving situation. For example, the driving pattern of “overtaking”
168
HMM (Meta-proto-symbol)
Proto-symbols in original proto-symbol space Proto-symbols in reconstructed proto-symbol space
x io
Proto-symbols
|| x io - x ik || x ik Symbol trajectory
HMM HMM
HMM
HMM
HMM
HMM
Driving time series data
Fig. 2. Overview of comparison between the original proto-symbol space and the reconstructed proto-symbol space.
is formed by a sequence of primitive patterns such as “accelerating the speed”, “right-hand turn”, “left-hand turn” and “straight drive in the constant speed”. In this paper, we propose the hierarchical structure with the proto-symbols in the lower layer and the meta-proto-symbols in the upper layer. The latter essentially correspond to models which abstract long-term driving patterns.
Sliding window
Fig. 3. The sliding windows of the driving time series data are converted to HMMs by optimizing the parameters of the HMMs using the windows as the training data. The HMMs can be located in the proto-symbol space. The symbol trajectory is formed by a sequence of their location. HMMs in the upper layer learn the symbol trajectory and develop to meta-proto symbols. [m] 500
A. Acquisition of Meta-proto-symbols The driving time series signal is represented by a trajectory in the proto-symbol space by calculating HMMs corresponding to sliding windows of the driving data and locating the HMMs in the proto-symbol space. The trajectory is illustrated by Fig.3. We call this trajectory “symbol trajectory”. The symbol trajectories are abstracted by another HMMs in the upper layer, which are meta-proto-symbols.
0
-500
B. Application of Meta-proto-symbols to Prediction of Driving The meta-proto-symbols have the potential for various applications. In this paper, we used the meta-proto-symbols to predict the driving. The current node in the meta-protosymbol is estimated from the symbol trajectory. The estimation can be achieved by Viterbi algorithm [19]. The node transition s1 , s2 , · · · , sT also can be predicted from the expected duration of each node by transiting from the estimated current node for T frames. Note that T is the term of prediction. A point in the proto-symbol space is output from the probability density function on the predicted node sT . sT is the node to which the driving situation is predicted to transit T frames later. The point is converted to its equivalent proto-symbol which is the closest to the point. The predicted proto-symbol can generate the driving pattern data. We assume the start time of the generation to be when the proto-symbol different from the previous one is predicted. While the same proto-symbol is predicted, the proto symbol successively generates the driving pattern data along the node transition. IV. E XPERIMENTAL R ESULTS A. Segmentation and Recognition of Driving Patterns 1) Test with Driving Simulator: We conducted experiments for segmentation and recognition of driving patterns with a driving simulator. The driving time series signal
-200 Boundary
0 Left-hand Turn
200
[m] Straight Drive
Right-hand Turn
Fig. 4. The segmentation and recognition results of the driving pattern data. The data markers indicate the boundaries or recognition results of the driving patterns during a subject drives around the test course for four times.
contains 4 kinds of measurements, vehicle speed, yaw rate, driver’s accelerator stroke and steering angle. The driving time series signal was measured during a subject’s driving along a test course. 10 proto-symbols with 10 nodes can be acquired by using the measured driving data through automatic segmentation and competitive learning framework [17]. Fig.4 shows the segmentation and recognition results of driving patterns during driving around the test course for four times. In this figure, the colors and the shapes of each markers indicate the boundaries of the driving patterns or the proto-symbols which the driving patterns are recognized as. The driving patterns in the same parts of the course tend to be recognized as the same proto-symbols. Realtime segmentation and recognition results by using the driving simulator are shown in Fig.5. In this experiment, the boundaries of the driving patterns are detected in realtime. If the current state is detected as the boundary, the driving pattern data from the previous boundary to the current one is
169
t=0.0
t=3.3
t=8.3
Right-hand Turn
HMM0 HMM1 HMM2 HMM3 HMM4 HMM5 HMM6 HMM7 HMM8 HMM9
0.1 0.05 0
Boundary
t=9.9
t=13.2
Straight Drive
-0.05 -0.1
Recognition Result (Symbol Index) t=14.5
-0.15
Left-hand Turn
-0.2
0.1
0 0
-0.1
-0.1 -0.2
-0.2
Fig. 6. The proto symbols are located in a multidimensional space based on dissimilarities among them. The proto symbols can be classified into one of driving patterns, “straight drive”, “right-hand turn” or “left-hand turn”.
t=14.9
t=16.5
(a)
t=18.0
Boundaries of driving patterns
(b)
(c)
Straight drive
Boundaries of driving patterns Fig. 5. The measured driving data is segmented, then the segmented driving pattern data is recognized as a proto-symbol in realtime. The displays of Boundary indicate the time when boundaries of driving patterns is detected, and the symbol indices, 9,1 and 9, indicate the recognition results of segmented driving pattern data. In the left, the boundary of driving data is detected just before entering right curve. The driving data from t = 0s to t = 8.3s can be labeled as “straight drive”. In the middle, the boundary is detected at the end of right curve, then the segmented driving data from t = 9.9s to t = 14.5s can be labeled as “right-hand turn”. Right figure shows that the boundary is also detected at the entrance of right curve. the segmented driving data from t = 14.9s to t = 18.0s can be labeled as “straight drive”. The similar driving pattern data from t = 0s to t = 8.3s, and from t = 14.9s to t = 18.0s are recognized as the same proto symbol 9
(d)
(e)
(g)
(h)
(f)
Left-hand turn
Boundaries of driving patterns (i)
Right-hand turn
Fig. 7. The boundaries of driving pattern data were detected between (c)(d) and (f)-(g). The extracted driving pattern data respectively correspond to “straight drive”, “left-hand turn” and “right-hand turn”.
recognized as a proto-symbol. We can see that the boundaries are found at the appropriate situation, and the driving pattern data are recognized as the suitable proto-symbols in Fig.5. The boundaries are detected at t = 8.3s, t = 14.5s and t = 18.0s, when the driver starts to turn the steering wheel to the right, finish driving in the right curve, and start to turn it to the right again. The driving situation changes drastically around these points. The driving data from t = 0s to t = 8.3s is very similar to one from t = 14.9s to t = 18.0s, which can be subjectively recognized as straight drive. These driving pattern data are classified as the same proto-symbol,“9”. This experiment demonstrates the validity of the framework of the proto-symbols. We construct the geometric space for the 10 proto symbols, which is shown in Fig.6. The proto symbols are clustered into three divisions, which correspond to driving pattern of “straight drive”, “right-hand turn” or “left-hand turn” respectively. Note that we subjectively give the proto-symbols these
labels such as “straight drive” by checking the recognition results shown in Fig.4 and the data generated from each proto symbol. Therefore, similar symbols are located close to one another. The symbol space can be established adequately. 2) Test with Actual Vehicle: The framework based on symbolization of driving patterns was tested on driving time series signal measured from an actual vehicle. The driving data consists of 9 kinds of measurements, vehicle angular velocities (roll, pitch, and yaw rates), vehicle accelerations (lateral, longitudinal, and vertical accelerations), vehicle speed, driver’s accelerator stroke and steering angle. The driving data were measured during 4 subjects driving along S-curve, which contains a right-hand curve with curvature radius 70m, a left-hand curve with curvature radius 80m, and a right-hand curve with curvature radius 100m. The sampling rate is set to 10Hz.
170
λ7 100 0
0
1
o
2
{
{ {
Time(s)
o
100
30
Accelerator stroke (%)
λ6
{ {
Accelerator stroke(%)
λ9
λ6
o
3
o
4
o
30 20 10 0
100
λ0
Left-hand turn 0
0
2
100
λ1
Step on the accelerator
0 100 0
30 20 10 0
2
0 Right-hand turn
30 20 10 0
2
30 20 10 0
Fig. 8. This figure shows the time profile of the accelerator stroke and the steering angle, the segmentation result and the recognition result.
Left-hand turn 0 100
0 Right-hand turn
100
4
4
100 100
λ6 0 Right-hand turn
100 2
4 100
λ7
Ease up on the accelerator
0 0
30 20 10 0
2
100
4
100
λ8
0 Right-hand turn
100
30 20 10 0
2
4 100
λ9 0 Right-hand turn
100 0
2
Time(s)
4
Time(s)
Degree of influence
Fig. 9. The driving data generated from 10 proto-symbols. The proto symbols, λ1 , λ3 , represent the driving pattern of “left-hand turn”. The proto symbols, λ2 , λ4 , λ5 , λ6 , λ8 , λ9 , abstract the driving pattern of “right-hand turn”. The proto symbol, λ1 represents the driving pattern that the driver steps on the accelerator pedal. In contrast, the proto symbol, λ7 represents the driving pattern that the driver releases the pedal.
㪇㪅㪍 㪇㪅㪌 㪇㪅㪋 㪇㪅㪊 㪇㪅㪉 㪇㪅㪈 㪇 㩷
Vehicle speed
Accelerator
Steer angle
Vertical acceleration
Lateral acceleration
Longitudinal acceleration
Roll
We have analyzed the degree of influence by each variable in the driving time series signal, which was measured in the driving test by using the actual vehicle. These degrees are shown in Fig.10, which indicates that the vehicle yaw rate, longitudinal acceleration, lateral acceleration, speed, the driver’s steering angle, and accelerator stroke are significant to construct the proto-symbol space. Since the driving data was measured during driving along a flat course, the vehicle
λ4 2
2
0 100
0
30 20 10 0
4
Pitch
B. Key Feature Extraction
2
Yaw
Fig.7 shows that the driving data is segmented into driving patterns, such as “straight drive”, “right-hand turn”, and “lefthand turn”. The first boundary is detected between (c) and (d) frame when the driver turns the steering wheel to the left. The extracted driving pattern ((a)-(c)) is classified into “straight drive”. The second boundary between (f) and (g) frame corresponds to the end of left-hand curve, then the driving pattern can be given a label of “left-hand turn”. In this way, the boundaries are detected around the changes of driving operation. These segmented driving pattern data are used as training data for HMMs such that the HMMs were optimized to develop to the proto-symbols. Note that the number of HMMs was set to 10 in advance in this experiment. Fig.8 and Fig.9 show the recognition and generation result of the driving data by using the acquired proto-symbols. The driving pattern data, O 2 and O 4 , which correspond to the driving pattern of “right-hand turn”, are recognized as the proto-symbol, “λ6 ”. The proto-symbol, “λ6 ” is representation of “righthand turn” as can be seen in Fig.9. The driving pattern data, O 3 , was measured during driving along the left-hand curve. This driving data is also appropriately recognized as the proto symbol, “λ3 ”, which symbolizes the driving pattern of “left-hand turn” as shown by Fig.9. Moreover, The driving pattern data, O 5 has the feature that the driver reduced the pressure on the accelerator pedal. This driving data is classified as the proto-symbol, λ7 . This proto symbol has the same feature as shown in Fig.9, then this driving data is also recognized correctly. Therefore, the proposed framework adequately performs the recognition of the driving pattern data.
100
4 100
λ3
0 Right-hand turn
0
0 100
0
100
λ5
30 20 10 0
4
λ2
0
5
100
4
30 20 10 0
Steering angle (deg)
30 20 10 0
λ3
30 20 10 0
Steering Angle(deg)
Recognition Result
Accelerator stroke Steering angle
Accelerator stroke Steering Angle
Boundary
Fig. 10. The degree of influence by variables of vehicle angular velocities (yaw, pitch, and roll rates), vehicle accelerations (longitudinal, lateral, and vertical accelerations), driver’s steering angle and accelerator stroke, and vehicle speed. This graph indicates that vehicle yaw rate, longitudinal acceleration, lateral acceleration, driver’s steering angle, accelerator stroke and vehicle speed are important to construct the proto-symbol space.
pitch rate, roll rate and vertical acceleration seem to be insignificant for the proto-symbols. Therefore, this result validates the proposed analytical method. C. Prediction based on Hierarchy of Proto-symbols and Meta-proto-symbols It is significant to predict the driver’s operation from measured driving data in order to support the driver by warning or controlling vehicle dynamics, or improve the driver’s skill by teaching expert’s driving technique. In this paper, we conducted a test on predictive performance of the steering angle. Fig.11 shows the comparison between the measured steering angle and predicted one in the two case of prediction term of 100ms and 3s. The error from 10s to 20s are very large and almost reach 40deg. We need to improve the prediction algorithm such that the error can be decreased. The average absolute value of error between the predicted steering angle and the measured one increases as the prediction term becomes larger as seen in Fig.12. However, the average error is about 7deg even in
171
Steering Angle [deg]
Measurement
Estimation
Prediction term : 100ms
and longterm contextual ones. The comparison between the actual measured driving data and the predicted one demonstrates the validity of the hierarchical model, which is expected to be useful to improve skills of novice drivers.
Prediction term : 3s
20 0
R EFERENCES
-40 0
Time[s]
80 0
80
Time[s]
Error of Steering Angle [deg]
Fig. 11. The comparison between the measured steering angle and predicted one. The left and right graphs respectively show the comparison in the case of prediction term of 100ms and 3s.
7
1 0
Prediction Term [s]
5
Fig. 12. The error between the measured steering angle and predicted one increases as the prediction term becomes larger. Even in the case that the prediction term is set to 5s, the steering angle can be predicted with about 7deg average error.
the case that the prediction term is set to 5s. It is difficult to use the prediction method with the degree of accuracy described above for automated driving systems. However, the prediction based on the meta-proto-symbols is expected to be applicable to driving support, navigation and warning systems. V. C ONCLUSION In this paper, we proposed an imitative learning approach to intellectual cognition for automobiles. Driving time-series signal consisting of states of the environment, vehicle and driver is segmented into driving pattern data. The driving pattern data are classified into HMMs which represent the data the most suitably. The parameters of the HMMs are optimized by using the classified driving pattern data. The autonomous segmentation and abstraction of the driving patterns enable the HMMs to form to the origins of the driving pattern symbols. The HMMs can be used for not only the recognition and but also the generation of the driving patterns. This framework allows the vehicles to grow intelligent by storing the knowlegde of a lot of driving pattern as driving pattern symbols through observing expert’s driving. Then this framework is expected to teach novice drivers by comparing their driving pattern with acquired driving pattern symbols. Moreover, the geometrical analysis of the HMMs allows the important variables to be selected out of the driving data. Experimental result shows the vehicle roll and pitch rate, lateral and vertical acceleration are insignificant for modeling the driving patterns during driving along a flat course. The symbolic approach is extended to a hierarchical framework in order to abstract both primitive driving patterns
[1] P. Ioannou and C. C. Chien, “Autonomous intelligent cruise control,” IEEE Transactions on Vehicular Technology, vol. 42, pp. 657–672, 1993. [2] D. N. Godbole and J. Lygeros, “Longitudinal control of the lead car of platoon,” IEEE Transactions on Vehicular Technology, vol. 43, pp. 1125–1135, 1994. [3] S. Huang and W. Ren, “Vehicle longitudinal control using throttles and brakes,” Robotics and Autonomous Systems, vol. 26, no. 5, pp. 241–253, 1999. [4] M. Ikegaya, N. Asanuma, S. Ishida, and S. Kondo, “Development of a lane following assistance system,” in Proceedings of the International Symposium on Advanced Vehicle Control, 1998, pp. 473–478. [5] C. Taylor, J. Seck, R. Blasi, and J. Malik, “A comparative study of vision-based lateral control strategies for autonomous highway driving,” Robotics and Autonomous Systems, vol. 18, no. 5, pp. 442– 453, 1999. [6] J. Tani and M. Ito, “Self-organization of behavioral primitives as multiple attractor dynamics: A robot experiment,” IEEE Transactions on Systems, Man and Cybernetics Part A: Systems and Humans, vol. 33, no. 4, pp. 481–488, 2003. [7] T. Ogata, S. Matsumoto, J. Tani, K. Komatani, and H. G. Okuno, “Human-robot cooperation using quasi-symbols generated by rnnpb model,” in Proceedings of the IEEE International Conference on Robotics and Automation, 2007, pp. 2156–2161. [8] M. Haruno, D. Wolpert, and M. Kawato, “Mosaic model for sensorimotor learning and control,” Neural Computation, vol. 13, pp. 2201– 2220, 2001. [9] J. Yang, Y. Xu, and C. S. Chen, “Hidden markov model approach to skill learning and its application to telerobotics,” IEEE Transactions on Robotics And Automation, vol. 10, no. 5, pp. 621–631, 1994. [10] A. Billard and R. Siegwart, “Robot learning from demonstration,” Robotics and Autonomous Systems, vol. 47, pp. 65–57, 2004. [11] T. Inamura, I. Toshima, H. Tanie, and Y. Nakamura, “Embodied symbol emergence based on mimesis theory,” International Journal of Robotics Research, vol. 23, no. 4, pp. 363–377, 2004. [12] D. Kulic, W. Takano, and Y. Nakamura, “Representability of human motions by factorial hidden markov models,” in Proceedings of thef IEEE/RSJ 2007 International Conference on Intelligent Robots and Systems, 2007, pp. 2388–2393. [13] W. Takano, K. Yamane, T. Sugihara, K. Yamamoto, and Y. Nakamura, “Primitive communication based on motion recognition and generation with hierarchical mimesis model,” in Proceedings of the IEEE International Conference on Robotics and Automation, 2006, pp. 3602–2609. [14] W. Takano, K. Yamane, and Y. Nakamura, “Capture database through symbolization, recognition and generation of motion patterns,” in Proceedings of the IEEE International Conference on Robotics and Automation, 2007, pp. 3092–3097. [15] G. Rizzolatti, L. Fogassi, and V. Gallese, “Neurophysiological mechanisms underlying the understanding and imitation of action,” Nature Reviews, pp. 661–670, 2001. [16] V. Gallese and A. Goldman, “Mirror neuron and the simulation theory of mind-reading,” Trends in Cognitive Sciences, vol. 2, no. 12, pp. 493–501, 1998. [17] W. Takano and Y. Nakamura, “Humanoid robot’s autonomous acquisition of proto-symbols through motion segmentation,” in Proceedings of the IEEE-RAS International Conference on Humanoid Robots, 2006, pp. 425–431. [18] L. Rabiner and B. H. Juang, “Fundamentals of speech recognition,” in Prentice Hall Signal Processing Series, 1993. [19] L. Rabiner, “A tutorial on hidden markov models and selected applications in speech recognition,” in Proceedings of the IEEE,, vol. 77, no. 2, 1989, pp. 257–286. [20] Y. Takane, F. W. Young, and J. D. Leeuw, “Nonmetric individual differences multidimensional scaling: An alternating least squres method with optimal scaling features,” Psychometrika, vol. 42, pp. 7–67, 1977.
172
Recognition of Human Driving Behaviors based on Stochastic Symbolization of Time Series Signal Wataru Takano, Akihiro Matsushita, Keijiro Iwao and Yoshihiko Nakamura Abstract— This paper describes an imitative learning of driving time series data for intellectual cognition toward future automobiles. The driving pattern primitives consisting of states of the environment, vehicle and driver are symbolized by Hidden Markov Models (HMMs), which can be used for both recognition and generation of the driving patterns. The relationship among the HMMs can be represented by locating the HMMs in a multidimensional space. The contribution of each variable to the HMM space can be analyzed such that important variables can be selected out of the driving data in order to reduce the size of the HMMs. Moreover, this paper presents a hierarchical model with the HMMs abstracting the primitive driving patterns in the lower layer, and another HMMs abstracting the longterm contextual driving patterns which are representation in the HMM space. Tests with a driving simulator and a actual vehicle demonstrate not only the validity of symbolization of driving pattern primitives, recognition and generation, but also availability of key feature selection. The extended hierarchical model is also proved to have a potential to predict the driving data appropriately.
I. I NTRODUCTION Recent advances in sensing and computational technologies have led many researchers to develop various intelligent vehicle control systems, such as longitudinal control system [1][2][3] or lateral control system [4][5]. These control systems depend on manually designed continuous models. However, a data-driven framework for autonomous abstraction or symbolization of a lot of driving patterns is required in order to construct higher-level intellectual cognition for automobiles. In robotics, some researches on symbolization of sensorimotor information have been proposed, such as Recurrent Neural Network with Parametric Bias (RNNPB) [6][7], Module Selection And Identification for Control (MOSAIC) [8], Hidden Markov Models (HMMs) [9][10][11][12]. We have been also developing the cognitive frameworks for humanoid robots by using HMMs, such as a physical communication model[13] or a linguistic model [14], inspired by the discovery of mirror neurons in primates [15][16]. The mirror neurons activate both when a primate observes a demonstrator’s behavior and when he performs the same kind of behavior. This fact means the mirror neurons integrate the motion recognition with motion generation through symbolization of motion patterns. In this paper, we propose an approach to stochastically intellectual cognition for automobiles by using the HMMs. The W. Takano and Y. Nakamura are with Mechano-Informatics, University of Tokyo, Hongo 7-3-1, Bunkyo-ku, Tokyo, 113-8656, Japan
{takano,nakamura}@ynl.t.u-tokyo.ac.jp
A. Matsusita and K. Iwao are with Nissan Research Center, Nissan Motor Co.,Ltd, {matsushita,k-iwao}@mail.nissan.co.jp
978-1-4244-2058-2/08/$25.00 ©2008 IEEE.
approach consists of segmentation, symbolization, recognition and generation of the driving patterns. The validity of the proposed cognitive framework is demonstrated by conducting experiments with a driving simulator and a actual vehicle. We apply this framework to selection of important variables from the driving time series signal. Moreover, we propose a hierarchical model which has the HMMs for the primitive driving patterns in the lower layer and HMMs for longterm contextual ones in the upper layer. The experimental result verifies that the hierarchical model can be used to predict the driving data appropriately. II. V EHICLE C OGNITIVE F RAMEWORK -S YMBOL OF D RIVING PATTERN A cognitive vehicle requires to sense data about states of environment, vehicle and driver, recognize driving situation, and generate the data appropriate for the current situation in order to control itself or support the driver. Imitative symbolization of driving time series signal is a powerful framework to model the driving pattern data. In this paper, we propose the cognitive framework, which allows the vehicle to symbolize, recognize and generate the driving patterns. A. Symbolization of Driving Time Series Signal We use left-to-right Hidden Markov Models to symbolize the driving patterns.An HMM is defined by a set of variables λ = {Q, A, B, Π}, where Q = {q1 , · · · , qn } is a set of nodes, A = {aij } is the matrix whose (i, j) element represents the transition probability from the i-th node to the j-th node, B = {b1 , b2 , · · · , bn } is a set of probability density functions and Π = {π1 , π2 , · · · , πn } is a set of initial node distributions. The driving time series signal O consists of multimodal data, such as environmental data, vehicle data and driver data. The driving time series signal is a sequence of driving patterns. So the measured signal has to be segmented into the driving patterns in order to symbolize the driving skills. We apply the segmentation method to the driving signal such that the driving pattern data Ok can be obtained [17]. The segmentation method is based on the criterion for prediction of following driving data. The parameters of HMMs are optimized using the segmented driving patterns by BaumWelch algorithm [18]. Note that each segmented driving pattern data is classified into an HMM which outputs the largest likelihood that the HMM generates the driving data, and the driving pattern data is used as a training data for the HMM. In this way, the HMMs are gradually optimized by using similar driving pattern data. These optimized HMMs
167
Generation of Driving Pattern
Proto Symbol
where OiG is the driving pattern data generated by the proto symbol λi , and d(λi , λj ) is Kullback Leibler Information between two proto symbols, λi and λj , which indicates the dissimilarity. However, this measure is not symmetric. The symmetric measure D(λi , λj ) is introduced in equation (3).
Symbolization, Recognition of Driving Pattern
HMM
Segmentation of driving pattern
d(λi , λj ) + d(λj , λi ) (3) 2 The proto-symbols are placed in a multidimensional space such that the distances among the proto-symbols in the space become as close as possible to the actual distances, thus minimizing the topological distortions in the mapping [20]. We use the multidimensional scaling method to locate the proto-symbols in the space. D(λi , λj ) =
Driving time series data (environment, vehicle, and driver states)
Fig. 1. Overview of a framework based on symbolization, recognition and generation of driving pattern.
extract features of the driving patterns. We name the HMMs proto-symbols since the HMMs form to the origins of symbols. B. Recognition of Driving Patterns based on Proto-symbols The acquired HMMs can be used for recognition of the driving patterns. The HMM corresponding to the driving pattern should output the largest likelihood and other HMMs should output the small likelihood that HMMs generate the driving pattern data. Therefore, motion recognition can be achieved by detecting the HMM which outputs the largest likelihood as follows. R = arg
max
λi :i=1,2,3,···,N
P (Ok |λi )
(1)
where N is the number of HMMs, P (Ok |λi ) is likelihood that the HMM λi generates the driving pattern data Ok , and R is the recognition result. C. Generation of Driving Patterns based on Proto-symbols The HMMs can be also used to generate the driving patterns by Monte-Carlo methods [11]. At first a sequence of representative nodes is computed from expected duration of staying in each node. Note that the expected duration of 1 i-th node, ti = , is calculated based on the node 1 − ai,i transition probability, where ai,i is the recurrent transition probability in i-th node. The driving pattern data is generated from this sequence of nodes and the probability density functions. A lot of samples of these generated driving pattern data are averaged to the resulting generated driving pattern. Therefore, a cognitive framework for vehicles can be constructed by using the HMMs, which allow for symbolization, recognition and generation of the driving patterns. This is illustrated schematically in Fig.1. D. Geometry of Driving Pattern Symbols It is important not only to symbolize the driving patterns but also to clarify the relationships among the proto-symbols. We focus on the distances among the proto-symbols in order to construct the geometric structure of driving patterns. We use the dissimilarities among the proto-symbols in place of the distances. The dissimilarity measure between the protosymbol λi and λj is given by equation (2)[19]. d(λi , λj ) =
ln P (OiG |λi ) − ln P (OiG |λj )
(2)
E. Key Feature Extraction Based on Proto-symbol Space In the proto-symbol space, the relationships among the proto-symbols are defined by the dissimilarity of them. It means that relative distance of a pair of two proto symbols is important in the proto-symbol space. In this paper, we analyze the degree of influence by each variable of the driving time series signal to the proto-symbol space by comparing an original proto-symbol space with a reconstructed symbol space as depicted by Fig.2. The original proto-symbol space is constructed based on the dissimilarity of the proto symbols which can be computed by using all of the variables included in the driving time series signal as described above. The new proto-symbol space is reconstructed based on the dissimilarity of the proto-symbols where a specific variable is eliminated. Note that the new proto-symbol space is calculated by multidimensional scaling after coordinates of the proto-symbols in the original protosymbol space are set to the initial ones in the new protosymbol space. The degree of change from the original protosymbol space to the reconstructed proto-symbol space can be computed as following. ∆k =
N X
k xoi − xki k2
(4)
i=1
where xoi is the coordinate of i-th proto symbol in the original proto-symbol space, xki is the coordinate of ith proto symbol in the new proto-symbol space which is reconstructed without using k-th variable for the dissimilarity of the proto symbols, and ∆k indicates the degree of change induced by the k-th variable. Here, we define the degree of influence by each variable to the proto symbol space as this measure. We can extract the key features of the variables with large ∆k since the large ∆k represents large contribution to the original proto symbol by the k-th variable. III. H IERARCHIZATION BASED ON P ROTO - SYMBOLS The proto symbols abstract short primitive driving patterns. such as “left-hand turn” or “straight drive”. However, there should be long and complicated driving patterns consisting of the multiple primitive patterns in our daily driving situation. For example, the driving pattern of “overtaking”
168
HMM (Meta-proto-symbol)
Proto-symbols in original proto-symbol space Proto-symbols in reconstructed proto-symbol space
x io
Proto-symbols
|| x io - x ik || x ik Symbol trajectory
HMM HMM
HMM
HMM
HMM
HMM
Driving time series data
Fig. 2. Overview of comparison between the original proto-symbol space and the reconstructed proto-symbol space.
is formed by a sequence of primitive patterns such as “accelerating the speed”, “right-hand turn”, “left-hand turn” and “straight drive in the constant speed”. In this paper, we propose the hierarchical structure with the proto-symbols in the lower layer and the meta-proto-symbols in the upper layer. The latter essentially correspond to models which abstract long-term driving patterns.
Sliding window
Fig. 3. The sliding windows of the driving time series data are converted to HMMs by optimizing the parameters of the HMMs using the windows as the training data. The HMMs can be located in the proto-symbol space. The symbol trajectory is formed by a sequence of their location. HMMs in the upper layer learn the symbol trajectory and develop to meta-proto symbols. [m] 500
A. Acquisition of Meta-proto-symbols The driving time series signal is represented by a trajectory in the proto-symbol space by calculating HMMs corresponding to sliding windows of the driving data and locating the HMMs in the proto-symbol space. The trajectory is illustrated by Fig.3. We call this trajectory “symbol trajectory”. The symbol trajectories are abstracted by another HMMs in the upper layer, which are meta-proto-symbols.
0
-500
B. Application of Meta-proto-symbols to Prediction of Driving The meta-proto-symbols have the potential for various applications. In this paper, we used the meta-proto-symbols to predict the driving. The current node in the meta-protosymbol is estimated from the symbol trajectory. The estimation can be achieved by Viterbi algorithm [19]. The node transition s1 , s2 , · · · , sT also can be predicted from the expected duration of each node by transiting from the estimated current node for T frames. Note that T is the term of prediction. A point in the proto-symbol space is output from the probability density function on the predicted node sT . sT is the node to which the driving situation is predicted to transit T frames later. The point is converted to its equivalent proto-symbol which is the closest to the point. The predicted proto-symbol can generate the driving pattern data. We assume the start time of the generation to be when the proto-symbol different from the previous one is predicted. While the same proto-symbol is predicted, the proto symbol successively generates the driving pattern data along the node transition. IV. E XPERIMENTAL R ESULTS A. Segmentation and Recognition of Driving Patterns 1) Test with Driving Simulator: We conducted experiments for segmentation and recognition of driving patterns with a driving simulator. The driving time series signal
-200 Boundary
0 Left-hand Turn
200
[m] Straight Drive
Right-hand Turn
Fig. 4. The segmentation and recognition results of the driving pattern data. The data markers indicate the boundaries or recognition results of the driving patterns during a subject drives around the test course for four times.
contains 4 kinds of measurements, vehicle speed, yaw rate, driver’s accelerator stroke and steering angle. The driving time series signal was measured during a subject’s driving along a test course. 10 proto-symbols with 10 nodes can be acquired by using the measured driving data through automatic segmentation and competitive learning framework [17]. Fig.4 shows the segmentation and recognition results of driving patterns during driving around the test course for four times. In this figure, the colors and the shapes of each markers indicate the boundaries of the driving patterns or the proto-symbols which the driving patterns are recognized as. The driving patterns in the same parts of the course tend to be recognized as the same proto-symbols. Realtime segmentation and recognition results by using the driving simulator are shown in Fig.5. In this experiment, the boundaries of the driving patterns are detected in realtime. If the current state is detected as the boundary, the driving pattern data from the previous boundary to the current one is
169
t=0.0
t=3.3
t=8.3
Right-hand Turn
HMM0 HMM1 HMM2 HMM3 HMM4 HMM5 HMM6 HMM7 HMM8 HMM9
0.1 0.05 0
Boundary
t=9.9
t=13.2
Straight Drive
-0.05 -0.1
Recognition Result (Symbol Index) t=14.5
-0.15
Left-hand Turn
-0.2
0.1
0 0
-0.1
-0.1 -0.2
-0.2
Fig. 6. The proto symbols are located in a multidimensional space based on dissimilarities among them. The proto symbols can be classified into one of driving patterns, “straight drive”, “right-hand turn” or “left-hand turn”.
t=14.9
t=16.5
(a)
t=18.0
Boundaries of driving patterns
(b)
(c)
Straight drive
Boundaries of driving patterns Fig. 5. The measured driving data is segmented, then the segmented driving pattern data is recognized as a proto-symbol in realtime. The displays of Boundary indicate the time when boundaries of driving patterns is detected, and the symbol indices, 9,1 and 9, indicate the recognition results of segmented driving pattern data. In the left, the boundary of driving data is detected just before entering right curve. The driving data from t = 0s to t = 8.3s can be labeled as “straight drive”. In the middle, the boundary is detected at the end of right curve, then the segmented driving data from t = 9.9s to t = 14.5s can be labeled as “right-hand turn”. Right figure shows that the boundary is also detected at the entrance of right curve. the segmented driving data from t = 14.9s to t = 18.0s can be labeled as “straight drive”. The similar driving pattern data from t = 0s to t = 8.3s, and from t = 14.9s to t = 18.0s are recognized as the same proto symbol 9
(d)
(e)
(g)
(h)
(f)
Left-hand turn
Boundaries of driving patterns (i)
Right-hand turn
Fig. 7. The boundaries of driving pattern data were detected between (c)(d) and (f)-(g). The extracted driving pattern data respectively correspond to “straight drive”, “left-hand turn” and “right-hand turn”.
recognized as a proto-symbol. We can see that the boundaries are found at the appropriate situation, and the driving pattern data are recognized as the suitable proto-symbols in Fig.5. The boundaries are detected at t = 8.3s, t = 14.5s and t = 18.0s, when the driver starts to turn the steering wheel to the right, finish driving in the right curve, and start to turn it to the right again. The driving situation changes drastically around these points. The driving data from t = 0s to t = 8.3s is very similar to one from t = 14.9s to t = 18.0s, which can be subjectively recognized as straight drive. These driving pattern data are classified as the same proto-symbol,“9”. This experiment demonstrates the validity of the framework of the proto-symbols. We construct the geometric space for the 10 proto symbols, which is shown in Fig.6. The proto symbols are clustered into three divisions, which correspond to driving pattern of “straight drive”, “right-hand turn” or “left-hand turn” respectively. Note that we subjectively give the proto-symbols these
labels such as “straight drive” by checking the recognition results shown in Fig.4 and the data generated from each proto symbol. Therefore, similar symbols are located close to one another. The symbol space can be established adequately. 2) Test with Actual Vehicle: The framework based on symbolization of driving patterns was tested on driving time series signal measured from an actual vehicle. The driving data consists of 9 kinds of measurements, vehicle angular velocities (roll, pitch, and yaw rates), vehicle accelerations (lateral, longitudinal, and vertical accelerations), vehicle speed, driver’s accelerator stroke and steering angle. The driving data were measured during 4 subjects driving along S-curve, which contains a right-hand curve with curvature radius 70m, a left-hand curve with curvature radius 80m, and a right-hand curve with curvature radius 100m. The sampling rate is set to 10Hz.
170
λ7 100 0
0
1
o
2
{
{ {
Time(s)
o
100
30
Accelerator stroke (%)
λ6
{ {
Accelerator stroke(%)
λ9
λ6
o
3
o
4
o
30 20 10 0
100
λ0
Left-hand turn 0
0
2
100
λ1
Step on the accelerator
0 100 0
30 20 10 0
2
0 Right-hand turn
30 20 10 0
2
30 20 10 0
Fig. 8. This figure shows the time profile of the accelerator stroke and the steering angle, the segmentation result and the recognition result.
Left-hand turn 0 100
0 Right-hand turn
100
4
4
100 100
λ6 0 Right-hand turn
100 2
4 100
λ7
Ease up on the accelerator
0 0
30 20 10 0
2
100
4
100
λ8
0 Right-hand turn
100
30 20 10 0
2
4 100
λ9 0 Right-hand turn
100 0
2
Time(s)
4
Time(s)
Degree of influence
Fig. 9. The driving data generated from 10 proto-symbols. The proto symbols, λ1 , λ3 , represent the driving pattern of “left-hand turn”. The proto symbols, λ2 , λ4 , λ5 , λ6 , λ8 , λ9 , abstract the driving pattern of “right-hand turn”. The proto symbol, λ1 represents the driving pattern that the driver steps on the accelerator pedal. In contrast, the proto symbol, λ7 represents the driving pattern that the driver releases the pedal.
㪇㪅㪍 㪇㪅㪌 㪇㪅㪋 㪇㪅㪊 㪇㪅㪉 㪇㪅㪈 㪇 㩷
Vehicle speed
Accelerator
Steer angle
Vertical acceleration
Lateral acceleration
Longitudinal acceleration
Roll
We have analyzed the degree of influence by each variable in the driving time series signal, which was measured in the driving test by using the actual vehicle. These degrees are shown in Fig.10, which indicates that the vehicle yaw rate, longitudinal acceleration, lateral acceleration, speed, the driver’s steering angle, and accelerator stroke are significant to construct the proto-symbol space. Since the driving data was measured during driving along a flat course, the vehicle
λ4 2
2
0 100
0
30 20 10 0
4
Pitch
B. Key Feature Extraction
2
Yaw
Fig.7 shows that the driving data is segmented into driving patterns, such as “straight drive”, “right-hand turn”, and “lefthand turn”. The first boundary is detected between (c) and (d) frame when the driver turns the steering wheel to the left. The extracted driving pattern ((a)-(c)) is classified into “straight drive”. The second boundary between (f) and (g) frame corresponds to the end of left-hand curve, then the driving pattern can be given a label of “left-hand turn”. In this way, the boundaries are detected around the changes of driving operation. These segmented driving pattern data are used as training data for HMMs such that the HMMs were optimized to develop to the proto-symbols. Note that the number of HMMs was set to 10 in advance in this experiment. Fig.8 and Fig.9 show the recognition and generation result of the driving data by using the acquired proto-symbols. The driving pattern data, O 2 and O 4 , which correspond to the driving pattern of “right-hand turn”, are recognized as the proto-symbol, “λ6 ”. The proto-symbol, “λ6 ” is representation of “righthand turn” as can be seen in Fig.9. The driving pattern data, O 3 , was measured during driving along the left-hand curve. This driving data is also appropriately recognized as the proto symbol, “λ3 ”, which symbolizes the driving pattern of “left-hand turn” as shown by Fig.9. Moreover, The driving pattern data, O 5 has the feature that the driver reduced the pressure on the accelerator pedal. This driving data is classified as the proto-symbol, λ7 . This proto symbol has the same feature as shown in Fig.9, then this driving data is also recognized correctly. Therefore, the proposed framework adequately performs the recognition of the driving pattern data.
100
4 100
λ3
0 Right-hand turn
0
0 100
0
100
λ5
30 20 10 0
4
λ2
0
5
100
4
30 20 10 0
Steering angle (deg)
30 20 10 0
λ3
30 20 10 0
Steering Angle(deg)
Recognition Result
Accelerator stroke Steering angle
Accelerator stroke Steering Angle
Boundary
Fig. 10. The degree of influence by variables of vehicle angular velocities (yaw, pitch, and roll rates), vehicle accelerations (longitudinal, lateral, and vertical accelerations), driver’s steering angle and accelerator stroke, and vehicle speed. This graph indicates that vehicle yaw rate, longitudinal acceleration, lateral acceleration, driver’s steering angle, accelerator stroke and vehicle speed are important to construct the proto-symbol space.
pitch rate, roll rate and vertical acceleration seem to be insignificant for the proto-symbols. Therefore, this result validates the proposed analytical method. C. Prediction based on Hierarchy of Proto-symbols and Meta-proto-symbols It is significant to predict the driver’s operation from measured driving data in order to support the driver by warning or controlling vehicle dynamics, or improve the driver’s skill by teaching expert’s driving technique. In this paper, we conducted a test on predictive performance of the steering angle. Fig.11 shows the comparison between the measured steering angle and predicted one in the two case of prediction term of 100ms and 3s. The error from 10s to 20s are very large and almost reach 40deg. We need to improve the prediction algorithm such that the error can be decreased. The average absolute value of error between the predicted steering angle and the measured one increases as the prediction term becomes larger as seen in Fig.12. However, the average error is about 7deg even in
171
Steering Angle [deg]
Measurement
Estimation
Prediction term : 100ms
and longterm contextual ones. The comparison between the actual measured driving data and the predicted one demonstrates the validity of the hierarchical model, which is expected to be useful to improve skills of novice drivers.
Prediction term : 3s
20 0
R EFERENCES
-40 0
Time[s]
80 0
80
Time[s]
Error of Steering Angle [deg]
Fig. 11. The comparison between the measured steering angle and predicted one. The left and right graphs respectively show the comparison in the case of prediction term of 100ms and 3s.
7
1 0
Prediction Term [s]
5
Fig. 12. The error between the measured steering angle and predicted one increases as the prediction term becomes larger. Even in the case that the prediction term is set to 5s, the steering angle can be predicted with about 7deg average error.
the case that the prediction term is set to 5s. It is difficult to use the prediction method with the degree of accuracy described above for automated driving systems. However, the prediction based on the meta-proto-symbols is expected to be applicable to driving support, navigation and warning systems. V. C ONCLUSION In this paper, we proposed an imitative learning approach to intellectual cognition for automobiles. Driving time-series signal consisting of states of the environment, vehicle and driver is segmented into driving pattern data. The driving pattern data are classified into HMMs which represent the data the most suitably. The parameters of the HMMs are optimized by using the classified driving pattern data. The autonomous segmentation and abstraction of the driving patterns enable the HMMs to form to the origins of the driving pattern symbols. The HMMs can be used for not only the recognition and but also the generation of the driving patterns. This framework allows the vehicles to grow intelligent by storing the knowlegde of a lot of driving pattern as driving pattern symbols through observing expert’s driving. Then this framework is expected to teach novice drivers by comparing their driving pattern with acquired driving pattern symbols. Moreover, the geometrical analysis of the HMMs allows the important variables to be selected out of the driving data. Experimental result shows the vehicle roll and pitch rate, lateral and vertical acceleration are insignificant for modeling the driving patterns during driving along a flat course. The symbolic approach is extended to a hierarchical framework in order to abstract both primitive driving patterns
[1] P. Ioannou and C. C. Chien, “Autonomous intelligent cruise control,” IEEE Transactions on Vehicular Technology, vol. 42, pp. 657–672, 1993. [2] D. N. Godbole and J. Lygeros, “Longitudinal control of the lead car of platoon,” IEEE Transactions on Vehicular Technology, vol. 43, pp. 1125–1135, 1994. [3] S. Huang and W. Ren, “Vehicle longitudinal control using throttles and brakes,” Robotics and Autonomous Systems, vol. 26, no. 5, pp. 241–253, 1999. [4] M. Ikegaya, N. Asanuma, S. Ishida, and S. Kondo, “Development of a lane following assistance system,” in Proceedings of the International Symposium on Advanced Vehicle Control, 1998, pp. 473–478. [5] C. Taylor, J. Seck, R. Blasi, and J. Malik, “A comparative study of vision-based lateral control strategies for autonomous highway driving,” Robotics and Autonomous Systems, vol. 18, no. 5, pp. 442– 453, 1999. [6] J. Tani and M. Ito, “Self-organization of behavioral primitives as multiple attractor dynamics: A robot experiment,” IEEE Transactions on Systems, Man and Cybernetics Part A: Systems and Humans, vol. 33, no. 4, pp. 481–488, 2003. [7] T. Ogata, S. Matsumoto, J. Tani, K. Komatani, and H. G. Okuno, “Human-robot cooperation using quasi-symbols generated by rnnpb model,” in Proceedings of the IEEE International Conference on Robotics and Automation, 2007, pp. 2156–2161. [8] M. Haruno, D. Wolpert, and M. Kawato, “Mosaic model for sensorimotor learning and control,” Neural Computation, vol. 13, pp. 2201– 2220, 2001. [9] J. Yang, Y. Xu, and C. S. Chen, “Hidden markov model approach to skill learning and its application to telerobotics,” IEEE Transactions on Robotics And Automation, vol. 10, no. 5, pp. 621–631, 1994. [10] A. Billard and R. Siegwart, “Robot learning from demonstration,” Robotics and Autonomous Systems, vol. 47, pp. 65–57, 2004. [11] T. Inamura, I. Toshima, H. Tanie, and Y. Nakamura, “Embodied symbol emergence based on mimesis theory,” International Journal of Robotics Research, vol. 23, no. 4, pp. 363–377, 2004. [12] D. Kulic, W. Takano, and Y. Nakamura, “Representability of human motions by factorial hidden markov models,” in Proceedings of thef IEEE/RSJ 2007 International Conference on Intelligent Robots and Systems, 2007, pp. 2388–2393. [13] W. Takano, K. Yamane, T. Sugihara, K. Yamamoto, and Y. Nakamura, “Primitive communication based on motion recognition and generation with hierarchical mimesis model,” in Proceedings of the IEEE International Conference on Robotics and Automation, 2006, pp. 3602–2609. [14] W. Takano, K. Yamane, and Y. Nakamura, “Capture database through symbolization, recognition and generation of motion patterns,” in Proceedings of the IEEE International Conference on Robotics and Automation, 2007, pp. 3092–3097. [15] G. Rizzolatti, L. Fogassi, and V. Gallese, “Neurophysiological mechanisms underlying the understanding and imitation of action,” Nature Reviews, pp. 661–670, 2001. [16] V. Gallese and A. Goldman, “Mirror neuron and the simulation theory of mind-reading,” Trends in Cognitive Sciences, vol. 2, no. 12, pp. 493–501, 1998. [17] W. Takano and Y. Nakamura, “Humanoid robot’s autonomous acquisition of proto-symbols through motion segmentation,” in Proceedings of the IEEE-RAS International Conference on Humanoid Robots, 2006, pp. 425–431. [18] L. Rabiner and B. H. Juang, “Fundamentals of speech recognition,” in Prentice Hall Signal Processing Series, 1993. [19] L. Rabiner, “A tutorial on hidden markov models and selected applications in speech recognition,” in Proceedings of the IEEE,, vol. 77, no. 2, 1989, pp. 257–286. [20] Y. Takane, F. W. Young, and J. D. Leeuw, “Nonmetric individual differences multidimensional scaling: An alternating least squres method with optimal scaling features,” Psychometrika, vol. 42, pp. 7–67, 1977.
172
