Learning and Adaptation of a Stylistic Myoelectric ... - Semantic Scholar
Learning and Adaptation of a Stylistic Myoelectric Interface: EMG-based Robotic Control with Individual User Differences Takamitsu Matsubara ∗† , Sang-Ho Hyon∗‡ and Jun Morimoto∗
∗ Department † Graduate
of Brain Robot Interface, ATR Computational Neuroscience Laboratories, Japan School of Information Science, Nara Institute of Science and Technology, Japan ‡ Department of Robotics, Ritsumeikan University, Japan
Abstract—In this study, we propose an interface to intuitively control robotic devices by using myoelectric signals detected from human users. In particular, we show experiments in which myoelectric signals measured from a forearm are used to control a robotic hand. When a user performs different motions (e.g., grasping and pinching), different myoelectric signals are measured. On the other hand, when different users perform the same motion (e.g, grasping), also, different myoelectric signals are measured. Therefore, designing a myoelectric interface that can be used for different users to perform different motions is difficult. In this study, we propose a bilinear model to explain myoelectric signals that depend on users and motions. The bilinear model is composed of two linear factors: 1) the userdependent factor and 2) the motion-dependent factor. Since the motion-dependent factor can be interpreted as a common representation of motion among multiple users, it allows to construct a myoelectric interface that is commonly applicable to multiple users to robotic devices. We present a learning procedure for the model using a set of myoelectric signals captured from multiple subjects, and present an adaptation procedure that adapts the model to a new user through only a few interactions. We call the combination of the model and the adaptation procedure the Stylistic Myoelectric Interface. Through experiments, the interface was applied to EMG-based robotic hand control and its effectiveness for multiple users was demonstrated.
I. I NTRODUCTION Myoelectric systems have received much attention for controlling multifunction prosthetic and robotic devices. The use of surface electromyography (EMG) is popular because it can be measured on a human skin surface with noninvasive electrodes. The dominant approach for making a controller with these surface EMG signals is to create a discriminative function between a human and a device as a myoelectric interface; the input is the EMG signals and the output is one of the pre-designed motions for the device. Thus, the discriminative function is basically a classification task. Designing such a discriminative function by hand would be too difficult especially in the case with multiple electrodes and multiple target motions. To automatically construct a highaccuracy classifier from data, machine learning or pattern recognition techniques have been extensively explored as with the AR models [1], [2], multi-layer neural networks [3], [4], fuzzy-rules [5], a combination of PCA and SOM [6], a HMMbased classifier [7], Gaussian Mixture Models (GMMs) [8] and Support Vector Machines [9]–[11]. Further advances can be
Robot Control
Surface EMG of the known user Myoelectric (EMG) signals
Mapping to robot moon (Mul-class SVM)
(a) Convenonal Scheme for EMG-based Roboc Control Surface EMG of a novel user
Robot Control Myoelectric (EMG) signals
Stylisc - Moon intenons Myoelectric Interface
Mapping to robot moon (Mul-class SVM)
(b) Proposed Scheme for EMG-based Roboc Control
Fig. 1. A schematic diagram of EMG-based robotic control with the stylistic myoelectric interface. In our schematic diagram for EMG-based robotic control, EMG signals are first factorized into the user-dependent factor and the motion-dependent factor. Then the motion-dependent factor is used in a motion classifier as input, and the output indicates the motion of the robot for control.
found in (e.g., [12], [13]). Commonly for the above studies, the task is to learn a classifier as a map g s : y → l where y ∈ RK is a EMG signal as the input, K is the number of its dimension and l ∈ {1, · · · L} indicates a corresponding motion of the device as the output and L is the number of pre-designed motion classes. Since an EMG signal has an user-dependent nature; the signals measured even performing the same intended motions can be largely different among different users. This is in large part due to the fact that EMG signals depend on userdependent factors such as the quantity of subcutaneous fat, skin impedance, and the pattern of muscle synergies. Thus, in most previous studies as [1]–[10], [12], [13], a classifier g s is constructed in a “tailor-made manner” for each individual user; each user must execute long-time experiments to capture a sufficient number of EMG signals to construct the tailormade classifier before starting to use it. We believe that the inconvenience of this process has prevented popularization of EMG-based myoelectric systems, although its potential has been recognized for a long time. To eliminate this inconvenience from myoelectric systems, in this paper we propose a novel approach for constructing a practical myoelectric interface between a human and a device. Roughly speaking, by using a data set captured from multiple subjects, we learn a stylistic myoelectric model that
models an EMG signal as a result of interactions between the user-dependent factor and the motion-dependent factor. Since the motion-dependent factor can be interpreted as a common representation of motion among multiple users, the model allows us to develop a novel myoelectric interface with a motion classifier commonly applicable for multiple users. For a new user, the model can be adapted to the user through only a few interactions. With the adapted model, the motion-dependent factor is extracted from myoelectric signals by separating its user-dependent factor, and used as inputs for the motion classifier for accurate classification. We call the combination of the model and the adaptation procedure the Stylistic Myoelectric Interface. Figure 1 depicts a schematic diagram of our EMG-based robotic device control with the stylistic myoelectric interface. The model adaptation approach proposed by Orabona et al. [14] also uses a database collected from multiple subjects. However, in their method, motion classifiers are trained for all subjects independently, then for a new subject, the model parameters of the best matched classifier is selected from among them to be used for a “smart initialization”. Although the adaptation process attempts to modify the initialized model to fit for a new subject, the process is executed in a high dimensional parameter space of the classifier, that requires a large amount of data to make the adaptation complete. Online adaptation of artificial neural networks [15], [16] also has a similar problem. It can be effective for tracking slow time-dependent changes in the characteristics of the subject; however, it would not be appropriate as an adaptation process of the model for a new subject since it tends to require a large amount of data. Section II presents the concept of the stylistic myoelectric interface for robot control. The novel concept and modeling approach of EMG signals are also presented in the section. Section III describes the learning and adaptation procedures of the interface. Section IV explains our experimental task and its settings. The proposed interface is applied to an EMG based robotic hand control system. Section V shows the experimental results. Section VI describes the concluding remarks of this paper. II. T HE S TYLISTIC M YOELECTRIC I NTERFACE In this section, we present the concept and the details of our stylistic myoelectric interface. As mentioned in the introduction, the interface is composed of the stylistic myoelectric model and the adaptation procedure to a new user. In Section II-A, we present the details of the stylistic myoelectric model based on a novel modeling approach for EMG signals. Then, Section II-B presents an overview of the myoelectric interface with the stylistic myoelectric model and the adaptation procedure. A. The Stylistic Myoelectric Model of EMG Signals Let y ∈ RK be an EMG signal (a raw data or a feature vector extracted by preprocessing) observed at a certain time through multiple electrodes placed on human skin over certain
User-dependent Factor (Style):
Moon-dependent Factor (Content):
Myoelectric Signal:
z x
y
⊗
Fig. 2. A rough sketch of the stylistic myoelectric model for EMG signals. The myoelectric (EMG) signal y is modeled as the result of an interaction between the user-dependent factor z and the motion-dependent factor x.
muscles. The value y may change according to the motion of the subject because the EMG signal is strongly related with the myoelectric signals of the muscles. However, the EMG signals are greatly different among subjects since they have individual dependencies related with the quantity of subcutaneous fat, skin impedance, the pattern of muscle synergies and so on. Thus, the EMG signal can be considered as a result of interactions between user-dependent factor and motion-dependent factor. With the above understanding as the ground, we propose to model an EMG signal y by a symmetric bilinear model as follows: wij zi xj (1) y = f (z, x) = i,j
or in element-wise as: yk = fe (z, x) = zT Wk x
(2)
where z ∈ RI is a latent variable representing user-dependent factor and x ∈ R J is another latent variable representing motion-dependent factor. W k ∈ RI×J (or w ∈ Rk ) is the interaction matrix that maps z and x to y. In the rest of the paper, we call this bilinear model of EMG signals in Eqs.(1) and (2) the stylistic myoelectric model. We also call z the style variable and x the content variable. A rough sketch of the model is depicted in Fig.2. B. Overview of The Stylistic Myoelectric Interface Once the stylistic myoelectric model is obtained, we can construct a map between the content variable x and the motion label l ∈ {1, · · · , L} as gc : x → l that can be commonly used for multiple subjects, where L is the number of motions predesigned for the robot. For a subject whose style variable is known as z, we could estimate the content variable x from an EMG signal y with the model f . However, for a new user whose style variable is unknown, it would be difficult to estimate the content variable from an EMG signal because there are two unknown variables of the style variable z and the content variable x to be determined from one observation y. Thus, to construct a practical myoelectric interface, an adaptation procedure for estimating the style variable z through only a few interactions would be required. In our scenario for the adaptation procedure, we ask the subject to demonstrate a few
motions by following directions for deriving certain intended motions. This scenario allows us to obtain a set of y and corresponding values of the content variable x to be known. With the pairs of data {y, x}, we can easily estimate z by a simple computation. After adaptation, the content variables are estimated from the EMG signals and used as input for the motion classifier as in Fig.1. III. L EARNING AND A DAPTATION P ROCEDURES FOR T HE S TYLISTIC M YOELECTRIC I NTERFACE In this section, we describe how to construct a stylistic myoelectric model. We present a learning procedure for learning the stylistic myoelectric model of EMG signals from EMG data captured from multiple subjects in Section III-A. We also describe an adaptation procedure of the model to a new subject in Section III-B. The scheme estimates latent variables of the ˆ from y ˆ . Section III-C explains a motion model as ˆ z and x classifier that maps the content variable to the motion label. In the following, we assume that we have a set of EMG data observed from multiple subjects over a set of common motions as D = ysc ∈ RK , lc ∈ {1, · · · , L} , s = 1 ∼ S, c = 1 ∼ C where K is the number of dimensions of the EMG signal, S is the number of subjects and C is the number of motions and L is the number of motions pre-designed for the robot. Thus, ysc represents an EMG signal observed from the subject s for the motion c. Each value of c has a corresponding motion class label lc for classification. A. A Learning Procedure for The Stylistic Myoelectric Model of EMG Signals The objective function of the learning procedure for the stylistic myoelectric model can be written with the elementwise bilinear model in Eq. (2) as E=
S C K
||yksc − zsT Wk xc ||2 .
(3)
s=1 c=1 k=1
The aim of the learning is to find variables {x c , zs , Wk } for all s, c, k that minimize the objective function in Eq. (3) where zs is the style variable for the subject s, and x c is the content variable for the content c. To find a solution for the minimization, we need a few matrix definitions. Let Y be a SK × C matrix and its vector-transpose Y VT be a CK × S matrix as 2
y11 6 Y = 4 ... yS1
··· .. .
y1C
3
2
7 VT 5,Y
ySC
y11 6 = 4 ... y1C
··· .. .
yS1
3
7 5,
(4)
ySC
where the vector-transpose {·} VT is defined as a process stacking for a SK × C matrix into a CK × X S matrix. and Z are also defined accordingly as Z = z1 · · · zS and X = x1 · · · xC . With these definitions and Eq. (2), we can obtain two equivalent equations for the data set: VT Y = WVT Z X, (5) YVT
=
VT
[WX]
Z
(6)
where W is the IK × J stacked matrix consisting of the kdimensional interaction vectors w ij and its vector transpose WVT is defined as follows: 2
w11 6 . W = 4 .. wI1
··· .. .
w1J wIJ
3
7 VT 5,W
2
w11 6 = 4 ... w1J
··· .. .
wI1 w
3
7 5.
IJ
These equations help to derive an iterative procedure to optimize the objective function in Eq.(3) with respect to both Z and X although the interaction matrix W k is unknown [17]. We first initialize X using the closed form of the Singular Value Decomposition (SVD) for Y as Y → UΣV T and the ˆ can be found as the first J rows of V T , where “→” estimate X represents the process of the SVD. Then, using Eq. (5) and the T ˆ ˆ VTX, we estimate Z as the first I rows of V where estimated ˆT YX → UΣVT . Next, using Eq. (6) and the estimated T ˆ ˆ Z, we estimate VT the new X as the first J rows of V where ˆT YVT Z → UΣVT . These two procedures provide one iteration of the learning procedure, and it typically converges within 10 iterations in our experience. After converging, we
VT VT ˆT ˆ = YX ˆT from Eq. (5) with Z finally obtain W
ˆ and X. ˆ the estimates Z B. Adaptation Procedure to A New Subject ˆ for estiTo use the stylistic myoelectric model with W mating the intended motion of an unknown subject u, the corresponding style variable ˆzu must be estimated with ob¯ = [¯ ¯ uC ] obtained from the subject. servations Y yu1 , · · · , y However, it is obviously an ill-posed problem because both z u ˆ = [x1 , · · · , xC ] are unknown, so it cannot be solved and X analytically. In our scenario, we ask the subject to demonstrate a few motions by following directions for deriving certain intended ¯ adapt motions. This scenario allows us to obtain a set of Y ¯ and corresponding values of Xadapt to be known. With this scenario, the adaptation problem becomes well-posed, thus, we can easily estimate ˆzu by using a simple computation derived from Eq. (6) as VT + u ˆ ¯ ¯ VT ˆz = WXadapt Y (8) adapt where {·}+ represents its Moore-Penrose pseudo inverse matrix. Note that the style variable z u is a I dimensional vector and it does not depend on the motion, thus, the adaptation ¯ adapt and procedure only requires a small amount of data as Y ¯ adapt obtained by a few interactions (data is not required for X all kinds of motions). After the adaptation of the model, we can directly estimate the content variable x test from a new observation y test with Eq. (5) by VT + ˆ VT zˆu ˆ test = W ytest . (9) x
(7)
C. Motion Classification from Human Motion After learning the stylistic myoelectric model of EMG signals, we can obtain a pair-wise data set composed of a ˆ c and corresponding number of estimated content variables x c c c labels of motion class l as {{ˆ x , l } , c = 1, · · · , C}. Using this data set, we construct a motion classifier commonly shared by multiple users. Among many possible candidate methods for constructing such a classifier, we selected the Support Vector Machine (SVM) based on its generalization performance as reported in many recent papers, such as in general [18] and in the application for EMG signals [9]–[11]. The SVM attempts to find a hyperplane that separates the input data into two classes (l c ∈ {−1, 1}) as a result of optimization with training data. The optimal classifier is obtained by optimizing the following objective function with constraints: max α
s.t.
C i=1 C
αi −
C 1 ˆj ) αi αj li lj k(ˆ xi , x 2 i,j=1
Open
Grasp
Neutral Extension Extensio n
Flexion
Fig. 3.
Human hand gestures and corresponding robotic hand motions.
(10) Electrode 4 Electrode 1
αi li = 0,
0 ≤ αi ≤ ξ,
(11)
Electrode 3
i=1
where k(·, ·) is a kernel function, and the parameter ξ is the regularization constant that decides the degree of miss classification for managing noisy training data. The obtained classifier C xi , x)+b. is expressed as sgn (g(x)) where g(x) = i li αi k(ˆ ˆ test The obtained classifier will label a content variable x as +1 or −1 to indicate a motion class, based on whether g(ˆ xtest ) is greater than 1, or less than −1. To extend the above binary classification using SVM for multi-class classification, we use a technique of the “one-against-one” method. In the method for the case of O classes, O(O+1)/2 binary classifiers are independently constructed for all combinations, that is, for each pair of classes, a binary classifier is constructed by using SVM. Then, to classify a test data, it is classified by each binary classifier and each result is counted as a vote for the respective class. Finally, the test data is classified as the class label with the maximum number of votes. IV. TASK D ESIGN AND S ETTING In this section, we explain the experimental task and the settings used to validate our approach. A. Task: Robotic Hand Control by Human Hand Gestures As a task for EMG-based robotic device control, we selected a task of robotic hand control by performing human intended hand gestures by mostly following the settings constructed by Yoshikawa et al. [10]. The robotic hand motion is controlled by the subject’s EMG-signals measured through electrodes placed on the forearm. First, the correspondence between humanintended hand gestures and robotic hand motions must be designed. Following the study [10], we selected five hand gestures as intended motions: flexion, extension, grasp, open, and neutral (L = 5). The corresponding motions of the robotic hand are designed as depicted in Fig. 3.
Electrode 2
Fig. 4. Electrodes placement on the forearm. Electrodes 1 to 4 are placed on the flexor carpi ulnaris (ch1), extensor carpi radialis (ch2), flexor digitorum profundus (ch3) and extensor digitorum (ch4).
B. Data Collection The EMG signals were measured with bipolar surface electrodes each of which has two parallel Ag-AgCl bars (Oisaka Electronic Device Ltd, LP-MS1020). Four channels were simultaneously measured by the instrument at 1000Hz, and full-wave rectification and filtering with 2.4 Hz cut-off frequency were applied for the measured signals. We located four electrodes on the subject’s forearm surface. The target muscles of all electrodes were the flexor carpi ulnaris (ch1), extensor carpi radialis (ch2), flexor digitorum profundus (ch3) and extensor digitorum (ch4). The placements of the electrodes are shown in Fig. 4. To capture the EMG data during intended motions, the subject was required to perform four motions sequentially in the order of flexion, extension, grasp, open. According to the 60 Hz sounds made by an electric metronome, the motions are performed with the rest between motions as neutral and it was subsequently performed as a trial. In this setting, we can collect a training data set from multiple subjects over a set of common intended motions for learning stylistic myoelectric interface. We collected data from twelve healthy subjects (eleven males (S = 11), aged between 23 and 30). C. Feature Extraction We used a simple feature extraction strategy by following [11]. We calculated the root-mean-square (rms) of the windowed steady-state EMG signals from each channel. 128
samples as the window size were empirically used and shifted 25 samples for each time, and the rms amplitude in the window was computed for each of the four electrodes. This meant that 40 samples were obtained for one second (during one intended motion) and it was assumed that each sample had motiondependent factor (K = 4, C = 200). V. E XPERIMENTS We conducted an experiment to investigate the effectiveness of our approach for EMG-based robotic control by multiple users with individual differences in EMG signals. A. Experimental Settings To validate our approach across individual user differences in EMG signals, we executed an experiment to measure the performance of the proposed method by the motion prediction accuracy for new (unknown) users. To achieve this, we used the leave-one-out approach, that is, S − 1 out of S subjects’ data were used as the training data set, and the remaining subject’s data were used as the test data. The interface constructed from the training data was first adapted to the test user through the adaptation procedure. In this experiment, the EMG signals and corresponding content variables during one test motion (flexion) from the test data were used. Then, the adapted model and the learned intended-motion classifier were validated with the whole test data. The prediction accuracy for robot motions from intended motions for the test subject’s data was defined as E = Ncorrect/N where Ncorrect is the number of corrected motions by predictions and N is the size of the test data for predictions. All cases of S were performed and the averages and standard deviations were calculated as reliable criteria for measuring the effectiveness of the proposed method for individual user differences. For learning the stylistic myoelectric model from the training data, we set the dimensions of the style and content variables as I = 4 and J = 12 so that the model could explain around 80 percent of the training data. The iterative learning procedure converged within 10 trials for all cases. In the learning settings of SVM, we used the RBF kernel as k(xi , x) = exp(−γ||xi − x||2 ). Since the hyperparameters in the above SVM were only γ and ξ, we employed a “gridsearch” on γ and ξ using cross-validation as an effective approach. Multiple pairs of (ξ, γ) were tried and the one with the best cross-validation accuracy was selected. We used 5 fold cross-validation in this study. As the comparisons in this scenario, we prepared two methods: In the first method, we picked a subject as a test, the test data was validated by the SVMs learned from other subjects’ data and its averaged prediction accuracy was calculated as a comparison (cross-SVM1). In the second method, the training data were partially renewed with the portion of the test data (as used for the adaptation procedure in the proposed method) and the SVMs were relearned with the renewed training data set for each subject to adapt the SVMs for the test subject (cross-SVM2), similarly to previous studies [14]–[16]. The
TABLE I C OMPARISONS OF P REDICTION A CCURACY. Method Cross-SVM1 Cross-SVM2 Proposed Method
Learning[s] 830 —– 6.0
Adaptation[s] —– 830 0.015
Accuracy 0.49(0.11) 0.54(0.081) 0.70(0.13)
performance of the cross-SVM1 would suggest the prediction accuracy without adaptation process. Since both the proposed method and the cross-SVM2 used a portion of the test data for adaptation, the comparison of the two methods in performance would validate the effectiveness of estimating the style variable for the test subject in the proposed method. Note that for the cross-SVM2, a better result would be expected than the cross-SVM1 due to the adaptation for the test subject; however, it is computationally demanding and impractical since training of multiple SVMs with the renewed training data set are required for each test subject. For all cases, the hyperparameters were set by the grid-search in the same way as for the proposed method. B. Experimental Results The experimental results are shown in Table I. The prediction accuracy by the cross-SVM1 was 0.49 and the crossSVM2 resulted in a better performance as 0.54. This performance improvement may be caused by renewing data with a portion of the test data and with a large computational cost in the cross-SVM2. The proposed method resulted in the best performance among all methods with the accuracy of 0.70. These results suggest that the stylistic myoelectric model could effectively represent a diversity of EMG signals and the learning and adaptation procedures work well. The difference in performance between the cross-SVM2 and the proposed method may be caused by the way each used a portion of the test data for adaptation: the cross-SVM2 used the data only for improving the prediction accuracy of the corresponding intended motion (i.e., flexion). On the other hand, in the proposed method, the data were used to estimate the style variable over all intended motions (not only for the motion of flexion), which contributes to improving the prediction accuracy for all motions. The executed time for learning and adaptation procedures for all methods are also shown in Table I. The proposed method is computationally efficient compared with the comparisons because it learns the one model for multiple users and the adaptation procedure requires very little extra computation as presented in Eq. (8). Snapshots of the successful robotic hand control by a new user with our method are depicted in Fig.5. VI. D ISCUSSION In this paper, we proposed a novel concept of a myoelectric interface for robotic and prosthetic device control by humans. In our approach, myoelectric signals are modeled as a result of interactions between the user-dependent factor (style) and the motion-dependent factor (content). Based on such a
Fig. 5. Snapshots of the successful robotic hand control with our proposed method by a new user’s EMG signals. The intended motions are performed and corresponding robotic motions are surely predicted.
factorial model of myoelectric signals, we proposed a novel myoelectric interface that could separate myoelectric signals into the user-dependent factor and motion-dependent factor. Since the motion-dependent factor can be interpreted as a common representation of motions among multiple users, we can construct an motion classifier that is commonly applicable for multiple users. The proposed interface was applied in an EMG-based robotic hand control system and the experimental results validated its effectiveness with individual differences in EMG signals. The factorized modelling approach, utilized for modeling of the stylistic myoelectric model in this paper, has been used for representing facial pictures in computer vision [17], motion capture data in computer graphics [19] and robotics [20] to capture the diversity of multiple data sets. To the best of our knowledge, this paper describes the first study to construct a factorized model for myoelectric signals and to demonstrate its effectiveness for EMG-based robotic device control with individual user differences in EMG signals through experiments. In our experiment, the dimensions of the style and content variables were experimentally set by a trial-and-error manner. Automatic selection of the dimensions from training data would be one of our future works. Other future work includes the use of several features of EMG signals already developed in previous studies as in [1]–[10], [12], [13] in our approach to achieve higher prediction accuracy. Development of the realtime control system of the robotic hand with a larger number of motions and electrodes will also be addressed with the proposed approach. R EFERENCES [1] D. Graupe and W. Cline, “Functional separation of emg signal via arma identification methods for prosthetic control purposes,” IEEE Trans. Sys. Man Cyber., vol. 5, no. 2, pp. 252–259, 1975. [2] D. Graupe, J. Salahi, and R. Scott, “Mutifunctional prosthesis and orthosis control via microcomputer identification of temporal pattern differences in single-site myoelectric signals,” J. Biomechanical Engineering, vol. 4, 1982. [3] M. F. Kelly, P. A. Parker, and S. Robert N, “The application of neural networks to myoelectric signal analysis: A preliminary study,” IEEE Transactions on Biomedical Engineering, vol. 37, no. 3, pp. 221–230, 1990. [4] B. Hudgins, P. Parker, and S. Robert N, “A new strategy for multifunction myoelectric control,” IEEE Transactions on Biomedical Engineering, vol. 40, no. 1, pp. 82–94, 1993. [5] F. H. Y. Chan, Y. Yang, F. K. Lam, Y. Zhang, and P. A. Parker, “Fuzzy EMG Classification for Prosthesis Control,” IEEE Transactions on Rehabilitation Engineering, vol. 8, no. 3, pp. 305–311, 2000.
[6] J. Chu, I. Moon, and M. Mun, “A real-time EMG pattern recognition system based on linear-nonlinear feature projection for a multifunction myoelectric hand,” IEEE Transaction on Biomedical Engineering, vol. 53, no. 11, pp. 2232–2239, 2006. [7] A. D. C. Chan and K. B. Englehart, “Continuous Myoelectric Control for Powered Prostheses Using Hidden Markov Models,” IEEE Transactions on Biomedical Engineering, vol. 52, no. 1, pp. 121–124, 2005. [8] Y. Huang, K. B. Englehart, B. Hudgins, and A. D. C. Chan, “A gaussian mixture model based classification scheme for myoelectric control of powered upper limb prostheses,” IEEE Transactions on Biomedical Engineering, vol. 52, no. 11, pp. 1801–1811, 2005. [9] S. Bitzer and P. Smagt, “Learning EMG control of a robotic hand: Towards active prostheses,” in ICRA, 2006, pp. 2819–2823. [10] M. Yoshikawa, M. Mikawa, and K. Tanaka, “A myoelectric interface for robotic hand control using support vector machine,” in IROS, 2007, pp. 2723–2728. [11] P. Shenoy, K. J. Miller, B. Crawford, and R. P. N. Rao, “Online electromyographic control of a robotic prosthesis,” IEEE Transactions on Biomedical Engineering, vol. 55, no. 3, pp. 1128–1135, 2008. [12] R. N. Khushaba, A. Al-Ani, and A. Al-Jumaily, “Orthogonal fuzzy neighborhood discriminant analysis for multifunction myoelectric hand control,” IEEE Transactions on Biomedical Engineering, vol. 57, no. 6, pp. 1410–1419, 2010. [13] L. J. Hargrove, E. J. Scheme, K. B. Englehart, and B. S. Hudgins, “Multiple binary classifications via linear discriminant analysis for improved controllability of a powered prosthesis,” IEEE Transactions on Neural Systems and Rehabilitation Engineering, vol. 18, no. 1, pp. 49–57, 2010. [14] F. Orabona, C. Castellini, B. Caputo, A. E. Fiorilla, and G. Sandini, “Model adaptation with least-squares SVM for adaptive hand prosthetics,” in ICRA, 2009, pp. 2897–2903. [15] O. Fukuda, T. Tsuji, A. Ohtsuka, and M. Kaneko, “EMG-based HumanRobot Interface for Rehabilitation Aid,” in ICRA, 1998, pp. 3492–3497. [16] R. Kato, H. Yokoi, and T. Arai, “Real-time Learning Method for Adaptable Motion- Discrimination using Surface EMG Signal,” in IROS, 2006, pp. 2127–2132. [17] J. B. Tenenbaum and W. T. Freeman, “Separating style and content with bilinear models,” Neural Computation, vol. 12, pp. 1247–1283, 2000. [18] N. Cristianini and J. Shawe-Taylor, An Introduction to Support Vector Machines and Other Kernel-based Learning Methods. Cambridge University Press, 2000. [19] J. M. Wang, D. J. Fleet, and A. Hertzmann, “Multifactor gaussian process models for style-content separation,” in ICML, 2007, pp. 975– 982. [20] T. Matsubara, S. Hyon, and J. Morimoto, “Learning Stylistic Dynamic Movement Primitives from Multiple Demonstrations,” in IROS, 2010, pp. 1277–1283.
ACKNOWLEDGEMENT This study is the result of “Brain Machine Interface Development” carried out under the SRBPS, MEXT. This study is partially supported by the Grant-in-Aid for Scientific Research from Japan Society for the Promotion of Science (WAKATEB22700177).
∗ Department † Graduate
of Brain Robot Interface, ATR Computational Neuroscience Laboratories, Japan School of Information Science, Nara Institute of Science and Technology, Japan ‡ Department of Robotics, Ritsumeikan University, Japan
Abstract—In this study, we propose an interface to intuitively control robotic devices by using myoelectric signals detected from human users. In particular, we show experiments in which myoelectric signals measured from a forearm are used to control a robotic hand. When a user performs different motions (e.g., grasping and pinching), different myoelectric signals are measured. On the other hand, when different users perform the same motion (e.g, grasping), also, different myoelectric signals are measured. Therefore, designing a myoelectric interface that can be used for different users to perform different motions is difficult. In this study, we propose a bilinear model to explain myoelectric signals that depend on users and motions. The bilinear model is composed of two linear factors: 1) the userdependent factor and 2) the motion-dependent factor. Since the motion-dependent factor can be interpreted as a common representation of motion among multiple users, it allows to construct a myoelectric interface that is commonly applicable to multiple users to robotic devices. We present a learning procedure for the model using a set of myoelectric signals captured from multiple subjects, and present an adaptation procedure that adapts the model to a new user through only a few interactions. We call the combination of the model and the adaptation procedure the Stylistic Myoelectric Interface. Through experiments, the interface was applied to EMG-based robotic hand control and its effectiveness for multiple users was demonstrated.
I. I NTRODUCTION Myoelectric systems have received much attention for controlling multifunction prosthetic and robotic devices. The use of surface electromyography (EMG) is popular because it can be measured on a human skin surface with noninvasive electrodes. The dominant approach for making a controller with these surface EMG signals is to create a discriminative function between a human and a device as a myoelectric interface; the input is the EMG signals and the output is one of the pre-designed motions for the device. Thus, the discriminative function is basically a classification task. Designing such a discriminative function by hand would be too difficult especially in the case with multiple electrodes and multiple target motions. To automatically construct a highaccuracy classifier from data, machine learning or pattern recognition techniques have been extensively explored as with the AR models [1], [2], multi-layer neural networks [3], [4], fuzzy-rules [5], a combination of PCA and SOM [6], a HMMbased classifier [7], Gaussian Mixture Models (GMMs) [8] and Support Vector Machines [9]–[11]. Further advances can be
Robot Control
Surface EMG of the known user Myoelectric (EMG) signals
Mapping to robot moon (Mul-class SVM)
(a) Convenonal Scheme for EMG-based Roboc Control Surface EMG of a novel user
Robot Control Myoelectric (EMG) signals
Stylisc - Moon intenons Myoelectric Interface
Mapping to robot moon (Mul-class SVM)
(b) Proposed Scheme for EMG-based Roboc Control
Fig. 1. A schematic diagram of EMG-based robotic control with the stylistic myoelectric interface. In our schematic diagram for EMG-based robotic control, EMG signals are first factorized into the user-dependent factor and the motion-dependent factor. Then the motion-dependent factor is used in a motion classifier as input, and the output indicates the motion of the robot for control.
found in (e.g., [12], [13]). Commonly for the above studies, the task is to learn a classifier as a map g s : y → l where y ∈ RK is a EMG signal as the input, K is the number of its dimension and l ∈ {1, · · · L} indicates a corresponding motion of the device as the output and L is the number of pre-designed motion classes. Since an EMG signal has an user-dependent nature; the signals measured even performing the same intended motions can be largely different among different users. This is in large part due to the fact that EMG signals depend on userdependent factors such as the quantity of subcutaneous fat, skin impedance, and the pattern of muscle synergies. Thus, in most previous studies as [1]–[10], [12], [13], a classifier g s is constructed in a “tailor-made manner” for each individual user; each user must execute long-time experiments to capture a sufficient number of EMG signals to construct the tailormade classifier before starting to use it. We believe that the inconvenience of this process has prevented popularization of EMG-based myoelectric systems, although its potential has been recognized for a long time. To eliminate this inconvenience from myoelectric systems, in this paper we propose a novel approach for constructing a practical myoelectric interface between a human and a device. Roughly speaking, by using a data set captured from multiple subjects, we learn a stylistic myoelectric model that
models an EMG signal as a result of interactions between the user-dependent factor and the motion-dependent factor. Since the motion-dependent factor can be interpreted as a common representation of motion among multiple users, the model allows us to develop a novel myoelectric interface with a motion classifier commonly applicable for multiple users. For a new user, the model can be adapted to the user through only a few interactions. With the adapted model, the motion-dependent factor is extracted from myoelectric signals by separating its user-dependent factor, and used as inputs for the motion classifier for accurate classification. We call the combination of the model and the adaptation procedure the Stylistic Myoelectric Interface. Figure 1 depicts a schematic diagram of our EMG-based robotic device control with the stylistic myoelectric interface. The model adaptation approach proposed by Orabona et al. [14] also uses a database collected from multiple subjects. However, in their method, motion classifiers are trained for all subjects independently, then for a new subject, the model parameters of the best matched classifier is selected from among them to be used for a “smart initialization”. Although the adaptation process attempts to modify the initialized model to fit for a new subject, the process is executed in a high dimensional parameter space of the classifier, that requires a large amount of data to make the adaptation complete. Online adaptation of artificial neural networks [15], [16] also has a similar problem. It can be effective for tracking slow time-dependent changes in the characteristics of the subject; however, it would not be appropriate as an adaptation process of the model for a new subject since it tends to require a large amount of data. Section II presents the concept of the stylistic myoelectric interface for robot control. The novel concept and modeling approach of EMG signals are also presented in the section. Section III describes the learning and adaptation procedures of the interface. Section IV explains our experimental task and its settings. The proposed interface is applied to an EMG based robotic hand control system. Section V shows the experimental results. Section VI describes the concluding remarks of this paper. II. T HE S TYLISTIC M YOELECTRIC I NTERFACE In this section, we present the concept and the details of our stylistic myoelectric interface. As mentioned in the introduction, the interface is composed of the stylistic myoelectric model and the adaptation procedure to a new user. In Section II-A, we present the details of the stylistic myoelectric model based on a novel modeling approach for EMG signals. Then, Section II-B presents an overview of the myoelectric interface with the stylistic myoelectric model and the adaptation procedure. A. The Stylistic Myoelectric Model of EMG Signals Let y ∈ RK be an EMG signal (a raw data or a feature vector extracted by preprocessing) observed at a certain time through multiple electrodes placed on human skin over certain
User-dependent Factor (Style):
Moon-dependent Factor (Content):
Myoelectric Signal:
z x
y
⊗
Fig. 2. A rough sketch of the stylistic myoelectric model for EMG signals. The myoelectric (EMG) signal y is modeled as the result of an interaction between the user-dependent factor z and the motion-dependent factor x.
muscles. The value y may change according to the motion of the subject because the EMG signal is strongly related with the myoelectric signals of the muscles. However, the EMG signals are greatly different among subjects since they have individual dependencies related with the quantity of subcutaneous fat, skin impedance, the pattern of muscle synergies and so on. Thus, the EMG signal can be considered as a result of interactions between user-dependent factor and motion-dependent factor. With the above understanding as the ground, we propose to model an EMG signal y by a symmetric bilinear model as follows: wij zi xj (1) y = f (z, x) = i,j
or in element-wise as: yk = fe (z, x) = zT Wk x
(2)
where z ∈ RI is a latent variable representing user-dependent factor and x ∈ R J is another latent variable representing motion-dependent factor. W k ∈ RI×J (or w ∈ Rk ) is the interaction matrix that maps z and x to y. In the rest of the paper, we call this bilinear model of EMG signals in Eqs.(1) and (2) the stylistic myoelectric model. We also call z the style variable and x the content variable. A rough sketch of the model is depicted in Fig.2. B. Overview of The Stylistic Myoelectric Interface Once the stylistic myoelectric model is obtained, we can construct a map between the content variable x and the motion label l ∈ {1, · · · , L} as gc : x → l that can be commonly used for multiple subjects, where L is the number of motions predesigned for the robot. For a subject whose style variable is known as z, we could estimate the content variable x from an EMG signal y with the model f . However, for a new user whose style variable is unknown, it would be difficult to estimate the content variable from an EMG signal because there are two unknown variables of the style variable z and the content variable x to be determined from one observation y. Thus, to construct a practical myoelectric interface, an adaptation procedure for estimating the style variable z through only a few interactions would be required. In our scenario for the adaptation procedure, we ask the subject to demonstrate a few
motions by following directions for deriving certain intended motions. This scenario allows us to obtain a set of y and corresponding values of the content variable x to be known. With the pairs of data {y, x}, we can easily estimate z by a simple computation. After adaptation, the content variables are estimated from the EMG signals and used as input for the motion classifier as in Fig.1. III. L EARNING AND A DAPTATION P ROCEDURES FOR T HE S TYLISTIC M YOELECTRIC I NTERFACE In this section, we describe how to construct a stylistic myoelectric model. We present a learning procedure for learning the stylistic myoelectric model of EMG signals from EMG data captured from multiple subjects in Section III-A. We also describe an adaptation procedure of the model to a new subject in Section III-B. The scheme estimates latent variables of the ˆ from y ˆ . Section III-C explains a motion model as ˆ z and x classifier that maps the content variable to the motion label. In the following, we assume that we have a set of EMG data observed from multiple subjects over a set of common motions as D = ysc ∈ RK , lc ∈ {1, · · · , L} , s = 1 ∼ S, c = 1 ∼ C where K is the number of dimensions of the EMG signal, S is the number of subjects and C is the number of motions and L is the number of motions pre-designed for the robot. Thus, ysc represents an EMG signal observed from the subject s for the motion c. Each value of c has a corresponding motion class label lc for classification. A. A Learning Procedure for The Stylistic Myoelectric Model of EMG Signals The objective function of the learning procedure for the stylistic myoelectric model can be written with the elementwise bilinear model in Eq. (2) as E=
S C K
||yksc − zsT Wk xc ||2 .
(3)
s=1 c=1 k=1
The aim of the learning is to find variables {x c , zs , Wk } for all s, c, k that minimize the objective function in Eq. (3) where zs is the style variable for the subject s, and x c is the content variable for the content c. To find a solution for the minimization, we need a few matrix definitions. Let Y be a SK × C matrix and its vector-transpose Y VT be a CK × S matrix as 2
y11 6 Y = 4 ... yS1
··· .. .
y1C
3
2
7 VT 5,Y
ySC
y11 6 = 4 ... y1C
··· .. .
yS1
3
7 5,
(4)
ySC
where the vector-transpose {·} VT is defined as a process stacking for a SK × C matrix into a CK × X S matrix. and Z are also defined accordingly as Z = z1 · · · zS and X = x1 · · · xC . With these definitions and Eq. (2), we can obtain two equivalent equations for the data set: VT Y = WVT Z X, (5) YVT
=
VT
[WX]
Z
(6)
where W is the IK × J stacked matrix consisting of the kdimensional interaction vectors w ij and its vector transpose WVT is defined as follows: 2
w11 6 . W = 4 .. wI1
··· .. .
w1J wIJ
3
7 VT 5,W
2
w11 6 = 4 ... w1J
··· .. .
wI1 w
3
7 5.
IJ
These equations help to derive an iterative procedure to optimize the objective function in Eq.(3) with respect to both Z and X although the interaction matrix W k is unknown [17]. We first initialize X using the closed form of the Singular Value Decomposition (SVD) for Y as Y → UΣV T and the ˆ can be found as the first J rows of V T , where “→” estimate X represents the process of the SVD. Then, using Eq. (5) and the T ˆ ˆ VTX, we estimate Z as the first I rows of V where estimated ˆT YX → UΣVT . Next, using Eq. (6) and the estimated T ˆ ˆ Z, we estimate VT the new X as the first J rows of V where ˆT YVT Z → UΣVT . These two procedures provide one iteration of the learning procedure, and it typically converges within 10 iterations in our experience. After converging, we
VT VT ˆT ˆ = YX ˆT from Eq. (5) with Z finally obtain W
ˆ and X. ˆ the estimates Z B. Adaptation Procedure to A New Subject ˆ for estiTo use the stylistic myoelectric model with W mating the intended motion of an unknown subject u, the corresponding style variable ˆzu must be estimated with ob¯ = [¯ ¯ uC ] obtained from the subject. servations Y yu1 , · · · , y However, it is obviously an ill-posed problem because both z u ˆ = [x1 , · · · , xC ] are unknown, so it cannot be solved and X analytically. In our scenario, we ask the subject to demonstrate a few motions by following directions for deriving certain intended ¯ adapt motions. This scenario allows us to obtain a set of Y ¯ and corresponding values of Xadapt to be known. With this scenario, the adaptation problem becomes well-posed, thus, we can easily estimate ˆzu by using a simple computation derived from Eq. (6) as VT + u ˆ ¯ ¯ VT ˆz = WXadapt Y (8) adapt where {·}+ represents its Moore-Penrose pseudo inverse matrix. Note that the style variable z u is a I dimensional vector and it does not depend on the motion, thus, the adaptation ¯ adapt and procedure only requires a small amount of data as Y ¯ adapt obtained by a few interactions (data is not required for X all kinds of motions). After the adaptation of the model, we can directly estimate the content variable x test from a new observation y test with Eq. (5) by VT + ˆ VT zˆu ˆ test = W ytest . (9) x
(7)
C. Motion Classification from Human Motion After learning the stylistic myoelectric model of EMG signals, we can obtain a pair-wise data set composed of a ˆ c and corresponding number of estimated content variables x c c c labels of motion class l as {{ˆ x , l } , c = 1, · · · , C}. Using this data set, we construct a motion classifier commonly shared by multiple users. Among many possible candidate methods for constructing such a classifier, we selected the Support Vector Machine (SVM) based on its generalization performance as reported in many recent papers, such as in general [18] and in the application for EMG signals [9]–[11]. The SVM attempts to find a hyperplane that separates the input data into two classes (l c ∈ {−1, 1}) as a result of optimization with training data. The optimal classifier is obtained by optimizing the following objective function with constraints: max α
s.t.
C i=1 C
αi −
C 1 ˆj ) αi αj li lj k(ˆ xi , x 2 i,j=1
Open
Grasp
Neutral Extension Extensio n
Flexion
Fig. 3.
Human hand gestures and corresponding robotic hand motions.
(10) Electrode 4 Electrode 1
αi li = 0,
0 ≤ αi ≤ ξ,
(11)
Electrode 3
i=1
where k(·, ·) is a kernel function, and the parameter ξ is the regularization constant that decides the degree of miss classification for managing noisy training data. The obtained classifier C xi , x)+b. is expressed as sgn (g(x)) where g(x) = i li αi k(ˆ ˆ test The obtained classifier will label a content variable x as +1 or −1 to indicate a motion class, based on whether g(ˆ xtest ) is greater than 1, or less than −1. To extend the above binary classification using SVM for multi-class classification, we use a technique of the “one-against-one” method. In the method for the case of O classes, O(O+1)/2 binary classifiers are independently constructed for all combinations, that is, for each pair of classes, a binary classifier is constructed by using SVM. Then, to classify a test data, it is classified by each binary classifier and each result is counted as a vote for the respective class. Finally, the test data is classified as the class label with the maximum number of votes. IV. TASK D ESIGN AND S ETTING In this section, we explain the experimental task and the settings used to validate our approach. A. Task: Robotic Hand Control by Human Hand Gestures As a task for EMG-based robotic device control, we selected a task of robotic hand control by performing human intended hand gestures by mostly following the settings constructed by Yoshikawa et al. [10]. The robotic hand motion is controlled by the subject’s EMG-signals measured through electrodes placed on the forearm. First, the correspondence between humanintended hand gestures and robotic hand motions must be designed. Following the study [10], we selected five hand gestures as intended motions: flexion, extension, grasp, open, and neutral (L = 5). The corresponding motions of the robotic hand are designed as depicted in Fig. 3.
Electrode 2
Fig. 4. Electrodes placement on the forearm. Electrodes 1 to 4 are placed on the flexor carpi ulnaris (ch1), extensor carpi radialis (ch2), flexor digitorum profundus (ch3) and extensor digitorum (ch4).
B. Data Collection The EMG signals were measured with bipolar surface electrodes each of which has two parallel Ag-AgCl bars (Oisaka Electronic Device Ltd, LP-MS1020). Four channels were simultaneously measured by the instrument at 1000Hz, and full-wave rectification and filtering with 2.4 Hz cut-off frequency were applied for the measured signals. We located four electrodes on the subject’s forearm surface. The target muscles of all electrodes were the flexor carpi ulnaris (ch1), extensor carpi radialis (ch2), flexor digitorum profundus (ch3) and extensor digitorum (ch4). The placements of the electrodes are shown in Fig. 4. To capture the EMG data during intended motions, the subject was required to perform four motions sequentially in the order of flexion, extension, grasp, open. According to the 60 Hz sounds made by an electric metronome, the motions are performed with the rest between motions as neutral and it was subsequently performed as a trial. In this setting, we can collect a training data set from multiple subjects over a set of common intended motions for learning stylistic myoelectric interface. We collected data from twelve healthy subjects (eleven males (S = 11), aged between 23 and 30). C. Feature Extraction We used a simple feature extraction strategy by following [11]. We calculated the root-mean-square (rms) of the windowed steady-state EMG signals from each channel. 128
samples as the window size were empirically used and shifted 25 samples for each time, and the rms amplitude in the window was computed for each of the four electrodes. This meant that 40 samples were obtained for one second (during one intended motion) and it was assumed that each sample had motiondependent factor (K = 4, C = 200). V. E XPERIMENTS We conducted an experiment to investigate the effectiveness of our approach for EMG-based robotic control by multiple users with individual differences in EMG signals. A. Experimental Settings To validate our approach across individual user differences in EMG signals, we executed an experiment to measure the performance of the proposed method by the motion prediction accuracy for new (unknown) users. To achieve this, we used the leave-one-out approach, that is, S − 1 out of S subjects’ data were used as the training data set, and the remaining subject’s data were used as the test data. The interface constructed from the training data was first adapted to the test user through the adaptation procedure. In this experiment, the EMG signals and corresponding content variables during one test motion (flexion) from the test data were used. Then, the adapted model and the learned intended-motion classifier were validated with the whole test data. The prediction accuracy for robot motions from intended motions for the test subject’s data was defined as E = Ncorrect/N where Ncorrect is the number of corrected motions by predictions and N is the size of the test data for predictions. All cases of S were performed and the averages and standard deviations were calculated as reliable criteria for measuring the effectiveness of the proposed method for individual user differences. For learning the stylistic myoelectric model from the training data, we set the dimensions of the style and content variables as I = 4 and J = 12 so that the model could explain around 80 percent of the training data. The iterative learning procedure converged within 10 trials for all cases. In the learning settings of SVM, we used the RBF kernel as k(xi , x) = exp(−γ||xi − x||2 ). Since the hyperparameters in the above SVM were only γ and ξ, we employed a “gridsearch” on γ and ξ using cross-validation as an effective approach. Multiple pairs of (ξ, γ) were tried and the one with the best cross-validation accuracy was selected. We used 5 fold cross-validation in this study. As the comparisons in this scenario, we prepared two methods: In the first method, we picked a subject as a test, the test data was validated by the SVMs learned from other subjects’ data and its averaged prediction accuracy was calculated as a comparison (cross-SVM1). In the second method, the training data were partially renewed with the portion of the test data (as used for the adaptation procedure in the proposed method) and the SVMs were relearned with the renewed training data set for each subject to adapt the SVMs for the test subject (cross-SVM2), similarly to previous studies [14]–[16]. The
TABLE I C OMPARISONS OF P REDICTION A CCURACY. Method Cross-SVM1 Cross-SVM2 Proposed Method
Learning[s] 830 —– 6.0
Adaptation[s] —– 830 0.015
Accuracy 0.49(0.11) 0.54(0.081) 0.70(0.13)
performance of the cross-SVM1 would suggest the prediction accuracy without adaptation process. Since both the proposed method and the cross-SVM2 used a portion of the test data for adaptation, the comparison of the two methods in performance would validate the effectiveness of estimating the style variable for the test subject in the proposed method. Note that for the cross-SVM2, a better result would be expected than the cross-SVM1 due to the adaptation for the test subject; however, it is computationally demanding and impractical since training of multiple SVMs with the renewed training data set are required for each test subject. For all cases, the hyperparameters were set by the grid-search in the same way as for the proposed method. B. Experimental Results The experimental results are shown in Table I. The prediction accuracy by the cross-SVM1 was 0.49 and the crossSVM2 resulted in a better performance as 0.54. This performance improvement may be caused by renewing data with a portion of the test data and with a large computational cost in the cross-SVM2. The proposed method resulted in the best performance among all methods with the accuracy of 0.70. These results suggest that the stylistic myoelectric model could effectively represent a diversity of EMG signals and the learning and adaptation procedures work well. The difference in performance between the cross-SVM2 and the proposed method may be caused by the way each used a portion of the test data for adaptation: the cross-SVM2 used the data only for improving the prediction accuracy of the corresponding intended motion (i.e., flexion). On the other hand, in the proposed method, the data were used to estimate the style variable over all intended motions (not only for the motion of flexion), which contributes to improving the prediction accuracy for all motions. The executed time for learning and adaptation procedures for all methods are also shown in Table I. The proposed method is computationally efficient compared with the comparisons because it learns the one model for multiple users and the adaptation procedure requires very little extra computation as presented in Eq. (8). Snapshots of the successful robotic hand control by a new user with our method are depicted in Fig.5. VI. D ISCUSSION In this paper, we proposed a novel concept of a myoelectric interface for robotic and prosthetic device control by humans. In our approach, myoelectric signals are modeled as a result of interactions between the user-dependent factor (style) and the motion-dependent factor (content). Based on such a
Fig. 5. Snapshots of the successful robotic hand control with our proposed method by a new user’s EMG signals. The intended motions are performed and corresponding robotic motions are surely predicted.
factorial model of myoelectric signals, we proposed a novel myoelectric interface that could separate myoelectric signals into the user-dependent factor and motion-dependent factor. Since the motion-dependent factor can be interpreted as a common representation of motions among multiple users, we can construct an motion classifier that is commonly applicable for multiple users. The proposed interface was applied in an EMG-based robotic hand control system and the experimental results validated its effectiveness with individual differences in EMG signals. The factorized modelling approach, utilized for modeling of the stylistic myoelectric model in this paper, has been used for representing facial pictures in computer vision [17], motion capture data in computer graphics [19] and robotics [20] to capture the diversity of multiple data sets. To the best of our knowledge, this paper describes the first study to construct a factorized model for myoelectric signals and to demonstrate its effectiveness for EMG-based robotic device control with individual user differences in EMG signals through experiments. In our experiment, the dimensions of the style and content variables were experimentally set by a trial-and-error manner. Automatic selection of the dimensions from training data would be one of our future works. Other future work includes the use of several features of EMG signals already developed in previous studies as in [1]–[10], [12], [13] in our approach to achieve higher prediction accuracy. Development of the realtime control system of the robotic hand with a larger number of motions and electrodes will also be addressed with the proposed approach. R EFERENCES [1] D. Graupe and W. Cline, “Functional separation of emg signal via arma identification methods for prosthetic control purposes,” IEEE Trans. Sys. Man Cyber., vol. 5, no. 2, pp. 252–259, 1975. [2] D. Graupe, J. Salahi, and R. Scott, “Mutifunctional prosthesis and orthosis control via microcomputer identification of temporal pattern differences in single-site myoelectric signals,” J. Biomechanical Engineering, vol. 4, 1982. [3] M. F. Kelly, P. A. Parker, and S. Robert N, “The application of neural networks to myoelectric signal analysis: A preliminary study,” IEEE Transactions on Biomedical Engineering, vol. 37, no. 3, pp. 221–230, 1990. [4] B. Hudgins, P. Parker, and S. Robert N, “A new strategy for multifunction myoelectric control,” IEEE Transactions on Biomedical Engineering, vol. 40, no. 1, pp. 82–94, 1993. [5] F. H. Y. Chan, Y. Yang, F. K. Lam, Y. Zhang, and P. A. Parker, “Fuzzy EMG Classification for Prosthesis Control,” IEEE Transactions on Rehabilitation Engineering, vol. 8, no. 3, pp. 305–311, 2000.
[6] J. Chu, I. Moon, and M. Mun, “A real-time EMG pattern recognition system based on linear-nonlinear feature projection for a multifunction myoelectric hand,” IEEE Transaction on Biomedical Engineering, vol. 53, no. 11, pp. 2232–2239, 2006. [7] A. D. C. Chan and K. B. Englehart, “Continuous Myoelectric Control for Powered Prostheses Using Hidden Markov Models,” IEEE Transactions on Biomedical Engineering, vol. 52, no. 1, pp. 121–124, 2005. [8] Y. Huang, K. B. Englehart, B. Hudgins, and A. D. C. Chan, “A gaussian mixture model based classification scheme for myoelectric control of powered upper limb prostheses,” IEEE Transactions on Biomedical Engineering, vol. 52, no. 11, pp. 1801–1811, 2005. [9] S. Bitzer and P. Smagt, “Learning EMG control of a robotic hand: Towards active prostheses,” in ICRA, 2006, pp. 2819–2823. [10] M. Yoshikawa, M. Mikawa, and K. Tanaka, “A myoelectric interface for robotic hand control using support vector machine,” in IROS, 2007, pp. 2723–2728. [11] P. Shenoy, K. J. Miller, B. Crawford, and R. P. N. Rao, “Online electromyographic control of a robotic prosthesis,” IEEE Transactions on Biomedical Engineering, vol. 55, no. 3, pp. 1128–1135, 2008. [12] R. N. Khushaba, A. Al-Ani, and A. Al-Jumaily, “Orthogonal fuzzy neighborhood discriminant analysis for multifunction myoelectric hand control,” IEEE Transactions on Biomedical Engineering, vol. 57, no. 6, pp. 1410–1419, 2010. [13] L. J. Hargrove, E. J. Scheme, K. B. Englehart, and B. S. Hudgins, “Multiple binary classifications via linear discriminant analysis for improved controllability of a powered prosthesis,” IEEE Transactions on Neural Systems and Rehabilitation Engineering, vol. 18, no. 1, pp. 49–57, 2010. [14] F. Orabona, C. Castellini, B. Caputo, A. E. Fiorilla, and G. Sandini, “Model adaptation with least-squares SVM for adaptive hand prosthetics,” in ICRA, 2009, pp. 2897–2903. [15] O. Fukuda, T. Tsuji, A. Ohtsuka, and M. Kaneko, “EMG-based HumanRobot Interface for Rehabilitation Aid,” in ICRA, 1998, pp. 3492–3497. [16] R. Kato, H. Yokoi, and T. Arai, “Real-time Learning Method for Adaptable Motion- Discrimination using Surface EMG Signal,” in IROS, 2006, pp. 2127–2132. [17] J. B. Tenenbaum and W. T. Freeman, “Separating style and content with bilinear models,” Neural Computation, vol. 12, pp. 1247–1283, 2000. [18] N. Cristianini and J. Shawe-Taylor, An Introduction to Support Vector Machines and Other Kernel-based Learning Methods. Cambridge University Press, 2000. [19] J. M. Wang, D. J. Fleet, and A. Hertzmann, “Multifactor gaussian process models for style-content separation,” in ICML, 2007, pp. 975– 982. [20] T. Matsubara, S. Hyon, and J. Morimoto, “Learning Stylistic Dynamic Movement Primitives from Multiple Demonstrations,” in IROS, 2010, pp. 1277–1283.
ACKNOWLEDGEMENT This study is the result of “Brain Machine Interface Development” carried out under the SRBPS, MEXT. This study is partially supported by the Grant-in-Aid for Scientific Research from Japan Society for the Promotion of Science (WAKATEB22700177).
