Brain-Controlled Finite State Machine for ... - Semantic Scholar
Brain-Controlled Finite State Machine for Wheelchair Navigation Amir Teymourian, Thorsten Lüth, Axel Gräser Institute of Automation, University of Bremen Otto-Hahn-Allee 1, 28359 Bremen, Germany
Torsten Felzer, Rainer Nordmann Department of Mechatronics in Mechanical Engineering, Darmstadt University of Technology Petersenstr. 30, 64287 Darmstadt, Germany
{felzer,nordmann}@mim.tudarmstadt.de
{teymourian,lueth,ag}@iat.unibremen.de ABSTRACT
2. BASIC ARCHITECTURE
This proposal is about a brain-controlled electrically powered wheelchair. The system comprises a brain-computer interface based on steady-state visual evoked potentials and a processing unit relying on a finite state machine (FSM). Results of first simulation experiments comparing two different FSMs are presented.
The two main components of the proposed input interface – the BCI and the FSM-based processing unit – are realized as two sepR processes. Upon detection of an input signal, the arate Windows BCI process sends a corresponding Window Message to the processing unit which updates an internal state governing wheelchair motion according to a transition diagram to be explained in the next section. A closed loop structure involves feedback referring to the movement of the wheelchair as perceived by the user. The processing unit is in accordance with the HAnds-free Wheelchair COntrol System (HaWCoS) which allows its user to control an EPW with the help of two simple muscle-related (as opposed to mental) input signals [3].
Categories and Subject Descriptors K.4.2 [Computers and Society]: Social Issues—Assistive technologies for persons with disabilities; H.5.2 [Information Interfaces and Presentation]: User Interfaces—Input devices and strategies
General Terms
3. METHODS
Experimentation, Human Factors
3.1 Interpretation of Control Commands
Keywords
Two options regarding the processing FSM are considered – they are detailed below.
Human-computer interaction, brain-computer interface (BCI), steady-state visual evoked potentials (SSVEP), finite state machine
1.
3.1.1 Option I: Using Two Input Signals The first option accepts the two input signals IS_A and IS_B. The resulting FSM (see fig. 1) conforms to the one used by HaWCoS, while an IS_A in the STOP state is used either to toggle FORWARD and BACKWARD or to interchange LEFT and RIGHT. This solution does not allow curves and has, therefore, to struggle with the same “digital” driving style as reported in [4].
INTRODUCTION
An electrically powered wheelchair (EPW) can be an extremely valuable aid for a person with a severe physical disability. However, members of the target population are often not able to utilize the standard manual joystick for controlling the wheelchair. An alternative is a brain-computer interface (BCI). BCIs interpret specific brain activity patterns and translate them into commands to control soft- or hardware devices [1]. The BCI used in the following is based on steady-state visual evoked potentials (SSVEP), which are a measurable response of the visual cortex to a stimulus light source flickering with a constant frequency (5 - 20 Hz, ideally) [2]. A periodic component of the same frequency as the stimulus can be obtained in brain activity. According to that detected frequency, a corresponding control command is issued. In the following, the interpretation of control commands generated by a BCI to navigate a simulated wheelchair is described.
↑
IS_A
IS_B
← (LEFT)
IS_A∗
STOP
IS_B
IS_B IS_B
IS_A
IS_A
(STRAIGHT’)
↑
→ (RIGHT)
IS_B IS_A
(STRAIGHT)
Copyright is held by the author/owner(s). ASSETS’08, October 13–15, 2008, Halifax, Nova Scotia, Canada. ACM 978-1-59593-976-0/08/10.
Figure 1: Transition diagram belonging to option I
257
IS_1 ∨ IS_3
IS_1 ∨ IS_3
IS_1 ∨ IS_2
IS_1
IS_1 I ∨ IS_2 IS_1 ∨ IS_3 IS_1
IS_2
? FLT
IS_2
IS_1
?
IS_4 R
IS_2
IS_3
? BLT
FRT
IS_4
IS_1 ∨ IS_2
IS_1 I ∨ IS_2 IS_1 ∨ IS_3
IS_2
IS_3
IS_4
IS_4 - S IS_4 i IS_4 6
IS_1
+ IS_2 R IS_3 - BRC BLC B
+ IS_2 R IS_3 - FRC FLC F
I
IS_2 ∨ IS_3
IS_4
IS_3
IS_4 - S’ IS_4 IS_4 * 6 IS_4 R
IS_4
? BRT
I IS_3
Figure 2: Transition diagram belonging to option II (abbreviations: S=Stop, F=Forward, FLC=Forward-Left Curve, FLT=ForwardLeft Turn, B=Backward, others=analogous); it may be divided into two more or less identical 6-state FSM’s (one for Forward, one for Backward), which are toggled with IS_2 or IS_3 in the Stop state (IS stands for “input signal”).
5. CONCLUSION
3.1.2 Option II: Using Four Input Signals Option II deals with the four input signals IS_1, IS_2, IS_3, and IS_4. The resulting transition diagram (which allows curves and turning) is depicted in fig. 2. The difference between the FLC and FLT states is that turning involves no straight component at all, while the curve state corresponds to pushing the manual control joystick into diagonal direction.
The results show that the first option (considering only two stimulus frequencies) is less effective than option II. More time is necessary to navigate the wheelchair through the corridor with the state machine of option I. Also, more handling errors are made by the subject. These handling errors occur due to the low information transfer rate of the BCI. As mentioned before, two seconds of data are necessary to classify the intended frequency. That makes the used BCI (and BCIs in general) too slow to handle a state machine effectively.
3.2 Subjects and Procedures Three non-impaired subjects (two male, one female), between 25 and 30 years old, participated in this study. All of them took already part in several earlier BCI studies and needed no vision correction. The subjects’ task was to navigate a simulated wheelchair through a corridor. If the wheelchair hits the wall, an error in handling the state machine is defined. Required time and errors are measured for each subject. The data was recorded non-invasively with an EEGcap. The electrodes for measuring the SSVEP response were placed at sites P O3 , P O4 , O9 , O10 , Oz and Pz with Cz as reference and AFz as ground electrode using the enhanced 10-20 system of electrode placement. The segment length of the data to be analyzed in each trial is 2s – one trial every 100ms. Therefore, if a frequency was classified successfully, an idle period of 2s has to pass before the next classification trial to give the subject the opportunity to stop focusing at a light source (without unintentionally evoking the same control command once more).
4.
Of course, more training of the subjects will decrease the number of errors, because the subjects will learn to focus at a frequency at the best point in time for a good navigation of the wheelchair. Although having to distinguish between four frequencies, the accuracy of the BCI using option II is quite similar to the one using option I. Future work will investigate the effects of training and will optimize the BCI for faster classification of the intended frequency. The ultimate goal is to devise an efficient and safe control mechanism for an actual (rather than simulated) EPW, usable by persons with severe motor impairments.
6. ACKNOWLEDGMENTS This work is partially supported by DFG grant FE 936/3-1 and a Marie Curie Transfer of Knowledge Fellowship of the European Community‘s.
RESULTS
Results of navigating the simulated wheelchair for all three subjects are shown in table 1. The table contains numerical results for both of the options detailed above, i.e., considering two and four stimulus frequencies, respectively.
7. REFERENCES [1] J. Wolpaw, N. Birbaumer, D. McFarland, G. Pfurtscheller and T. Vaughan. Brain-computer interfaces for communication and control. In Clinical Neurophysiology, volume 113(6), pages 767–791, 2002. [2] C. S. Herrmann. Human EEG responses to 1-100 hz flicker: resonance phenomena in visual cortex and their potential correlation to cognitive phenomena. In Exp. Brain Res., volume 137(3–4), pages 346–353, 2001. [3] T. Felzer and B. Freisleben. HaWCoS: The “Hands-free” Wheelchair Control System. In Proc. ASSETS 2002, pages 127–134. ACM Press, 2002. [4] T. Felzer and R. Nordmann. Alternative wheelchair control. In Proc. RAT’07, pages 67–74. IEEE Computer Society, 2007.
Table 1: Results of navigating the simulated wheelchair for all three subjects. The time as well as handling errors (wheelchair hit the wall) and the accuracy of the BCI (true positive classification rate) are given for both options. Subject Time [min] Handling errors accuracy [%] Opt. I/Opt. II Opt. I/Opt. II Opt. I/Opt. II 1 16:10/11:10 7/5 100/100 2 22:09/12:07 11/5 92/97 3 15:03/08:47 5/1 100/98
258
Torsten Felzer, Rainer Nordmann Department of Mechatronics in Mechanical Engineering, Darmstadt University of Technology Petersenstr. 30, 64287 Darmstadt, Germany
{felzer,nordmann}@mim.tudarmstadt.de
{teymourian,lueth,ag}@iat.unibremen.de ABSTRACT
2. BASIC ARCHITECTURE
This proposal is about a brain-controlled electrically powered wheelchair. The system comprises a brain-computer interface based on steady-state visual evoked potentials and a processing unit relying on a finite state machine (FSM). Results of first simulation experiments comparing two different FSMs are presented.
The two main components of the proposed input interface – the BCI and the FSM-based processing unit – are realized as two sepR processes. Upon detection of an input signal, the arate Windows BCI process sends a corresponding Window Message to the processing unit which updates an internal state governing wheelchair motion according to a transition diagram to be explained in the next section. A closed loop structure involves feedback referring to the movement of the wheelchair as perceived by the user. The processing unit is in accordance with the HAnds-free Wheelchair COntrol System (HaWCoS) which allows its user to control an EPW with the help of two simple muscle-related (as opposed to mental) input signals [3].
Categories and Subject Descriptors K.4.2 [Computers and Society]: Social Issues—Assistive technologies for persons with disabilities; H.5.2 [Information Interfaces and Presentation]: User Interfaces—Input devices and strategies
General Terms
3. METHODS
Experimentation, Human Factors
3.1 Interpretation of Control Commands
Keywords
Two options regarding the processing FSM are considered – they are detailed below.
Human-computer interaction, brain-computer interface (BCI), steady-state visual evoked potentials (SSVEP), finite state machine
1.
3.1.1 Option I: Using Two Input Signals The first option accepts the two input signals IS_A and IS_B. The resulting FSM (see fig. 1) conforms to the one used by HaWCoS, while an IS_A in the STOP state is used either to toggle FORWARD and BACKWARD or to interchange LEFT and RIGHT. This solution does not allow curves and has, therefore, to struggle with the same “digital” driving style as reported in [4].
INTRODUCTION
An electrically powered wheelchair (EPW) can be an extremely valuable aid for a person with a severe physical disability. However, members of the target population are often not able to utilize the standard manual joystick for controlling the wheelchair. An alternative is a brain-computer interface (BCI). BCIs interpret specific brain activity patterns and translate them into commands to control soft- or hardware devices [1]. The BCI used in the following is based on steady-state visual evoked potentials (SSVEP), which are a measurable response of the visual cortex to a stimulus light source flickering with a constant frequency (5 - 20 Hz, ideally) [2]. A periodic component of the same frequency as the stimulus can be obtained in brain activity. According to that detected frequency, a corresponding control command is issued. In the following, the interpretation of control commands generated by a BCI to navigate a simulated wheelchair is described.
↑
IS_A
IS_B
← (LEFT)
IS_A∗
STOP
IS_B
IS_B IS_B
IS_A
IS_A
(STRAIGHT’)
↑
→ (RIGHT)
IS_B IS_A
(STRAIGHT)
Copyright is held by the author/owner(s). ASSETS’08, October 13–15, 2008, Halifax, Nova Scotia, Canada. ACM 978-1-59593-976-0/08/10.
Figure 1: Transition diagram belonging to option I
257
IS_1 ∨ IS_3
IS_1 ∨ IS_3
IS_1 ∨ IS_2
IS_1
IS_1 I ∨ IS_2 IS_1 ∨ IS_3 IS_1
IS_2
? FLT
IS_2
IS_1
?
IS_4 R
IS_2
IS_3
? BLT
FRT
IS_4
IS_1 ∨ IS_2
IS_1 I ∨ IS_2 IS_1 ∨ IS_3
IS_2
IS_3
IS_4
IS_4 - S IS_4 i IS_4 6
IS_1
+ IS_2 R IS_3 - BRC BLC B
+ IS_2 R IS_3 - FRC FLC F
I
IS_2 ∨ IS_3
IS_4
IS_3
IS_4 - S’ IS_4 IS_4 * 6 IS_4 R
IS_4
? BRT
I IS_3
Figure 2: Transition diagram belonging to option II (abbreviations: S=Stop, F=Forward, FLC=Forward-Left Curve, FLT=ForwardLeft Turn, B=Backward, others=analogous); it may be divided into two more or less identical 6-state FSM’s (one for Forward, one for Backward), which are toggled with IS_2 or IS_3 in the Stop state (IS stands for “input signal”).
5. CONCLUSION
3.1.2 Option II: Using Four Input Signals Option II deals with the four input signals IS_1, IS_2, IS_3, and IS_4. The resulting transition diagram (which allows curves and turning) is depicted in fig. 2. The difference between the FLC and FLT states is that turning involves no straight component at all, while the curve state corresponds to pushing the manual control joystick into diagonal direction.
The results show that the first option (considering only two stimulus frequencies) is less effective than option II. More time is necessary to navigate the wheelchair through the corridor with the state machine of option I. Also, more handling errors are made by the subject. These handling errors occur due to the low information transfer rate of the BCI. As mentioned before, two seconds of data are necessary to classify the intended frequency. That makes the used BCI (and BCIs in general) too slow to handle a state machine effectively.
3.2 Subjects and Procedures Three non-impaired subjects (two male, one female), between 25 and 30 years old, participated in this study. All of them took already part in several earlier BCI studies and needed no vision correction. The subjects’ task was to navigate a simulated wheelchair through a corridor. If the wheelchair hits the wall, an error in handling the state machine is defined. Required time and errors are measured for each subject. The data was recorded non-invasively with an EEGcap. The electrodes for measuring the SSVEP response were placed at sites P O3 , P O4 , O9 , O10 , Oz and Pz with Cz as reference and AFz as ground electrode using the enhanced 10-20 system of electrode placement. The segment length of the data to be analyzed in each trial is 2s – one trial every 100ms. Therefore, if a frequency was classified successfully, an idle period of 2s has to pass before the next classification trial to give the subject the opportunity to stop focusing at a light source (without unintentionally evoking the same control command once more).
4.
Of course, more training of the subjects will decrease the number of errors, because the subjects will learn to focus at a frequency at the best point in time for a good navigation of the wheelchair. Although having to distinguish between four frequencies, the accuracy of the BCI using option II is quite similar to the one using option I. Future work will investigate the effects of training and will optimize the BCI for faster classification of the intended frequency. The ultimate goal is to devise an efficient and safe control mechanism for an actual (rather than simulated) EPW, usable by persons with severe motor impairments.
6. ACKNOWLEDGMENTS This work is partially supported by DFG grant FE 936/3-1 and a Marie Curie Transfer of Knowledge Fellowship of the European Community‘s.
RESULTS
Results of navigating the simulated wheelchair for all three subjects are shown in table 1. The table contains numerical results for both of the options detailed above, i.e., considering two and four stimulus frequencies, respectively.
7. REFERENCES [1] J. Wolpaw, N. Birbaumer, D. McFarland, G. Pfurtscheller and T. Vaughan. Brain-computer interfaces for communication and control. In Clinical Neurophysiology, volume 113(6), pages 767–791, 2002. [2] C. S. Herrmann. Human EEG responses to 1-100 hz flicker: resonance phenomena in visual cortex and their potential correlation to cognitive phenomena. In Exp. Brain Res., volume 137(3–4), pages 346–353, 2001. [3] T. Felzer and B. Freisleben. HaWCoS: The “Hands-free” Wheelchair Control System. In Proc. ASSETS 2002, pages 127–134. ACM Press, 2002. [4] T. Felzer and R. Nordmann. Alternative wheelchair control. In Proc. RAT’07, pages 67–74. IEEE Computer Society, 2007.
Table 1: Results of navigating the simulated wheelchair for all three subjects. The time as well as handling errors (wheelchair hit the wall) and the accuracy of the BCI (true positive classification rate) are given for both options. Subject Time [min] Handling errors accuracy [%] Opt. I/Opt. II Opt. I/Opt. II Opt. I/Opt. II 1 16:10/11:10 7/5 100/100 2 22:09/12:07 11/5 92/97 3 15:03/08:47 5/1 100/98
258
