EEG-based Communication: Improved Accuracy By Response ...

326

IEEE TRANSACTIONS ON REHABILITATION ENGINEERING, VOL. 6, NO. 3, SEPTEMBER 1998

EEG-Based Communication: Improved Accuracy by Response Verification Jonathan R. Wolpaw, Herbert Ramoser, Dennis J. McFarland, and Gert Pfurtscheller Abstract—Humans can learn to control the amplitude of electroencephalographic (EEG) activity in specific frequency bands over sensorimotor cortex and use it to move a cursor to a target on a computer screen. EEG-based communication could provide a new augmentative communication channel for individuals with motor disabilities. In the present system, each dimension of cursor movement is controlled by a linear equation. While the intercept in the equation is continually updated, it does not perfectly eliminate the impact of spontaneous variations in EEG amplitude. This imperfection reduces the accuracy of cursor movement. We evaluated a response verification (RV) procedure in which each outcome is determined by two opposite trials (e.g., one top-target trial and one bottom-target trial). Success, or failure, on both is required for a definitive outcome. The RV procedure reduces errors due to imperfection in intercept selection. Accuracy for opposite-trial pairs exceeds that predicted from the accuracies of individual trials, and greatly exceeds that for same-trial pairs. The RV procedure should be particularly valuable when the first trial has >2 possible targets, because the second trial need only confirm or deny the outcome of the first, and it should be applicable to nonlinear as well as to linear algorithms. Index Terms— Augmentative communication, electroencephalography, mu rhythm, operant conditioning, prosthesis, rehabilitation, sensorimotor cortex.

I. INTRODUCTION

R

ECENT studies show that people can learn to control the amplitude of electroencephalographic activity (EEG) over sensorimotor cortex and use that control to move a cursor from the center of a computer screen to a target on the periphery [1]–[3]. This work might lead to a new alternative communication channel for individuals with severe motor impairments, such as those resulting from brainstem stroke or amyotrophic lateral sclerosis (ALS) [4], [5]. It might also lead to a new method for controlling neural prostheses [6]. Realization of this potential depends on further improvements in the capacity of the current EEG-based communication system. In theory, the capacity of a communication system is given by its information transfer rate, normally measured in bits/second [7]. In practice, capacity is often measured Manuscript received September 30, 1997; revised January 26, 1998 and May 15 1998. This work was supported by the National Center for Medical Rehabilitation Research of the National Institutes of Health under Grant HD30146 and the “Fonds zur Forderung der wissenschaftlichen Forschung (Project P9043),” Austria. J. R. Wolpaw and D. J. McFarland are with the Wadsworth Center, New York State Department of Health and the State University of New York, Albany, NY 12201 USA. H. Ramoser is with the Department of Medical Informatics, Institute of Biomedical Engineering, Graz University of Technology, Graz, Austria. G. Pfurtscheller is with the Ludwig Boltzmann Institute of Medical Informatics and Neuroinformatics, Graz, Austria. Publisher Item Identifier S 1063-6528(98)05923-0.

by the accuracy and speed of the system in a specified application. While maximizing bit rate is a constant goal, our immediate goal is to maximize the number of bits that can be communicated with high accuracy in a discrete message (i.e., command) occupying a period of up to several seconds. This goal seems most relevant to the probable initial uses of EEGbased communication. For example, if the goal is to answer a yes/no question, change room temperature, choose a television channel, select a menu item, or control hand grasp through a neural prosthesis, the accuracy of the message is often more important than the speed with which it is conveyed. This study evaluates a method for increasing the accuracy of the message and maximizing the number of bits it contains. In the present prototype application, the message is selection of a target located on the edge of a computer screen. Selection is achieved by moving a cursor from the center of the screen to the target. Cursor movement in each dimension is a linear function of EEG amplitude. For example, vertical cursor is defined by the equation movement (1) is the EEG amplitude at a certain location and where ), and is the intercept. Examfrequency, is the gain ( ination of the function shows that an amplitude smaller than the intercept moves the cursor down and a higher amplitude moves it up. An intercept that is too large makes hitting top targets more difficult and one that is too small makes hitting bottom targets more difficult. Thus, selection of the intercept is crucial to system performance. The goal is an intercept that eliminates bias in either direction and thereby maximizes the influence that the subject’s EEG control has on the direction of cursor movement. If mean EEG amplitude in well-trained subjects (i.e., mean amplitude over a set of trials with equal numbers of top and bottom targets) displayed no variation except that correlated with target position (e.g., top or bottom of screen for vertical movement), the solution would be simple. Assuming that top and bottom targets have equal frequency, a subject’s past performance could be used to set the intercept so that net cursor movement over future trials would be zero. This would ensure that top and bottom targets were equally accessible. In reality, however, mean EEG amplitude, even in the besttrained subjects, varies significantly from minute to minute and session to session. In an effort to reduce the impact of this variation on cursor movement, a recent study evaluated a variety of different methods for using previous performance to select the intercept for the next trial [8]. The computationally simplest

1063–6528/98$10.00  1998 IEEE

Authorized licensed use limited to: CALIFORNIA STATE UNIVERSITY LA. Downloaded on April 14,2010 at 20:47:09 UTC from IEEE Xplore. Restrictions apply.

WOLPAW et al.: EEG-BASED COMMUNICATION

method—averaging the EEG from the 10 to 12 immediately preceding trials to obtain the intercept—proved to be as successful as more sophisticated methods, and is currently used during online operation. Nevertheless, even this method is only partially successful in eliminating the effects of spontaneous variation in the mean amplitude of the EEG component controlling cursor movement. The residual effects limit accuracy. For example, in controlling movement in one dimension (e.g., vertical movement to a top target or a bottom target), welltrained subjects can consistently achieve accuracies greater than 90%. However, in many possible applications, such as answering yes/no questions, higher accuracy, e.g., 99%, would be desirable. The present study analyzes the potential benefits of an errordetection procedure designed to reduce the deleterious effects on performance of the apparently inevitable imperfections in intercept selection. This “response verification” or RV procedure is comparable to the simplest error-detection options offered by information theory [7]. In the RV procedure, each outcome is determined not by a single trial, but by a pair of two successive and opposite trials (e.g., for vertical movement: one top-target trial and one bottom-target trial). If an individual hits both targets, the outcome is a success; if he or she misses both, the outcome is an error, and if he or she hits one and misses one, the outcome is undecided. Unless spontaneous variation in mean amplitude is rapid enough to eliminate any correlation between amplitudes in two consecutive trials (which is not the case), errors due to an imperfect intercept should be converted by this procedure to undecided outcomes (e.g., if the intercept is too low, the top target will be hit and the bottom one will be missed). Thus, the RV procedure should increase accuracy more than would be predicted from the observed single-trial accuracy. This greater accuracy would be purchased at the cost of a higher than expected number of undecided outcomes. To evaluate the probable effect of the RV procedure, we performed an offline analysis of single-trial data previously collected from many individuals who had achieved varying levels of one-dimensional EEG control, and from several who had achieved a substantial level of two-dimensional control. For each subject’s data, we determined the RV outcomes for pairs of opposite trials (e.g., top/bottom or bottom/top) and for pairs of same trials (e.g., top/top or bottom/bottom), and compared these outcomes to the outcome predicted from the accuracies of two single trials. The results confirm the impact of imperfections in the intercept. Accuracy for opposite pairs is significantly greater than predicted and accuracy for same pairs is significantly lower than predicted. At the same time, percent decided for opposite pairs is lower than predicted and percent decided for same pairs is higher than predicted. Thus, the data suggest that the RV procedures should be useful for certain applications, particularly those in which high accuracy is important. II. METHODS A. Online Operation and Data Collection The data analyzed here were gathered from 16 adults (13 men and three women, ages 29–68). Eleven had no neuro-

327

logical disorder, three had spinal cord injuries (at vertebral levels C4–5, C6–7, and T4, respectively) and were wheelchairbound, and two had ALS (one was ambulatory; and one was paralyzed, except for eye and limited eyebrow movements, and depended on a ventilator). All gave informed consent for the study, which had been reviewed and approved by the New York State Department of Health Institutional Review Board. During performance, each sat in a reclining chair or in a wheelchair, faced a video screen, and was asked to remain motionless. Scalp electrodes recorded 64 channels of EEG [9], each referred to a reference electrode on the right ear. All 64 channels were amplified with a bandpass of 1–60 Hz and digitized at 128 Hz for later analysis (though as noted below, only a subset of electrodes actually controlled cursor movement online). System operation has been described in detail elsewhere [10] and is summarized here. 1) One-Dimensional (1-D) Cursor Movement: All 16 of the subjects trained on controlling 1-D (i.e., vertical) cursor movement. After digitization, one or two EEG channels over sensorimotor cortex of each hemisphere (usually C3 and C4 [9]), were re-referenced either to a common average reference (seven subjects), a Laplacian reference (six subjects), or a bipolar reference (three subjects) [11]. The Laplacian reference used four next-nearest-neighbor electrodes (e.g., for C3, the neighbors were CZ, P3, T7, and F3). The bipolar reference used electrodes 3 cm anterior and posterior to C3 and C4. (Ten of the subjects were part of a study that compared cursor movement accuracies obtained with these three referencing methods.) Ten times/s, the most recent 200-ms segment from each channel was analyzed by an autoregressive algorithm [12], [13], and the square root of power in a 3, 4, or 5-Hz wide frequency band was calculated. These frequency bands encompassed the arch-shaped 8–12 Hz mu rhythm or the 20–25 Hz central beta rhythm, both of which are generated in sensorimotor cortex [14]–[21]. For 12 subjects, the sum of the values from the two channels (e.g., C3 and C4) was in the independent variable in the linear equation [i.e., (1)] that controlled cursor movement. For the remaining four subjects, the value from a single channel (i.e., C3 or C4) was . (The intersubject differences in referencing method, EEG channels, and frequency band had no discernible influence on the effectiveness of the RV procedure.) The gain [ in (1)] was set on the basis of each subject’s past performance so that the average duration of cursor movement was usually 1.5–3.0 s. The intercept [ in (1)] was continually updated on the basis of the average value of A for the subject’s most recent trials (see Introduction). In addition, the EEG amplitudes controlling cursor movements (as well as the digitized EEG from all 64 electrodes) and all online system parameters were stored for later analysis. Each subject participated in two or three training sessions per week. In each session, 1-D cursor control was practiced in eight runs of 3 min each, separated by 1-min breaks. A run consisted of a series of trials (usually 25–30). Each trial began with a one-s period during which the screen was blank. Then, a target appeared at the top or bottom edge of the screen. One second later, the cursor appeared in the center of the screen and began to move vertically 10 times/s according to the


328


linear equation described above. The cursor had 188 possible vertical positions. The subject’s task was to move the cursor to the target. The trial ended when the cursor touched the top or bottom edge. When it touched the correct edge, the target flashed for one second as a reward and the computer registered a hit. When it touched the other edge, the target disappeared, the cursor remained fixed on the screen for one s, and the computer registered a miss. In either case, the next trial then began with one second of blank screen. Top and bottom targets were randomized in blocks of 8 (i.e., four of each), blocks interrupted by the end of a run were continued in the next run, and a miss did not cause the target to be repeated. Thus, single-trial accuracy expected in the absence of any EEG control was 50%. 2) Two-Dimensional (2-D) Cursor Movement: Three of the 16 subjects also underwent training in control of 2-D cursor movement. Vertical movement was controlled by EEG from C3 and C4 with a common average reference, as in the 1D protocol (see above). Horizontal movement was controlled by the EEG amplitudes in a 3- or 5-Hz wide frequency band centered at 10 or 15 Hz from occipital electrodes PO7 and – ) PO8. The difference between these amplitudes ( was the independent variable in a second linear equation that ) controlled horizontal cursor movement (

(a)

(2) and in this equation were controlled as described for the moved the cursor 1-D protocol, and positive values of to the right. The EEG amplitudes controlling both dimensions of movement, as well as all online system parameters, were stored for later analysis. The session, run, and trial formats, and the training schedule, were comparable to those of the 1D protocol. The difference was that there were four possible targets, one occupying each corner of the screen. Thus, singletrial accuracy expected in the absence of any EEG control was 25%.

(b) Fig. 1. (a) Predicted RV accuracy (solid) and RV proportion decided (dashed) for pairs of 1-D trials versus single-trial accuracy. (b) Predicted RV accuracy (solid) and RV proportion decided (dashed) for pairs consisting of a 2-D trial (i.e., 2 possible targets) and a 1-D trial versus single-trial two-dimensional accuracy [i.e., q in (5) and (6)], when single-trial 1-D accuracy [i.e., h in (5) and (6)] is 0.90 or 0.95 (i.e., 90 or 95%).

>

p

p

B. Offline Analysis 1) One-Dimensional (1-D) Movement Data: For analysis of the 1-D data, we identified, for each subject, each pair of successive opposite trials (i.e., top/bottom or bottom/top) and each pair of successive same trials (i.e., top/top or bottom/bottom) and determined for each pair the subject’s response verification (RV) outcome: if both targets were hit, the pair was correct, if both were missed, the pair was incorrect, and if one was hit and the other missed, the pair was undecided. Every possible pair was utilized, so that the for a run total number of opposite and same pairs was for a session with runs. of trials, and The measures computed for opposite and same pairs from a set of successive sessions from each subject were: RV proportion decided (i.e., proportion of pairs that was decided), RV accuracy (i.e., proportion of decided pairs that was correct), and single-trial accuracy (i.e., proportion correct for the single trials that comprised the pairs). These results were compared to the results expected if spontaneous variation in mean amplitude did not occur, or

if any that did occur was perfectly matched by the online adjustments in the intercept value [i.e., the results expected if for there were no correlation between trial and trial from (1)]. In this case, if is the single-trial accuracy for , the predicted RV proportion opposite or same pairs, then decided for opposite or same pairs, is the sum of the probability that two single trials are correct and the probability that they are wrong (3) , the predicted RV accuracy, is the probability that two single trials are correct divided by (4) and versus Fig. 1(a) plots the predicted values of the value of . The increase in accuracy predicted for response verification is most prominent for higher values of . Thus, if is 0.55, or 55%, is expected to be only 60%; if is



329

Fig. 2. Operation of a communication system that uses the opposite-trial RV protocol to select one of four options. In this example, the user wishes to select option A. The screen shows the four options, one in each corner of the screen (i.e., one in each quadrant of the periphery). One second later, the cursor appears and moves controlled by the user’s EEG until it hits one of the four corners. The option in that corner flashes to indicate selection, and the screen asks if the selection is correct. It then displays “Yes” in the hemifield (i.e., the half of the periphery) opposite to its original position and “No” in the other hemifield. One second later, the cursor appears and moves controlled by the user’s EEG until it hits one of the two hemifields. If the “Yes” hemifield is hit, the selection is confirmed, the selected option appears on the screen, and communication, whether correct or incorrect, has occurred. If the “No” hemifield is hit, a question mark appears on the screen, and no communication has occurred. TABLE I OPERATION OF AN ACTUAL RESPONSE VERIFICATION (RV) COMMUNICATION SYSTEM (e.g., Fig. 2) AND SIMULATION OF THAT OPERATION FROM STANDARD 2-D SINGLE-TRIAL DATA. AS SHOWN HERE, THE UNIVERSE OF DATA FOR THE SIMULATION CONSISTS OF PAIRS OF TRIALS IN WHICH THE FIRST TARGET IS HIT AND THE SECOND TARGET IS OPPOSITE TO THE FIRST TARGET, AND PAIRS IN WHICH THE FIRST TARGET IS MISSED AND THE SECOND TARGET IS IN THE CORNER THAT WAS HIT BY MISTAKE. (FOR SIMULATION OF SAME-TRIAL RESPONSE VERIFICATION, THE DATA CONSIST OF PAIRS OF TRIALS IN WHICH THE FIRST TARGET IS HIT AND THE SECOND TARGET IS THE SAME AS THE FIRST TARGET, AND PAIRS IN WHICH THE FIRST TARGET IS MISSED AND THE SECOND TARGET IS IN THE CORNER OPPOSITE TO THE CORNER THAT WAS HIT BY MISTAKE)

75%, is expected to be 90%; and if is 91%, C is expected to be 99%. 2) Two-Dimensional Movement Data: Fig. 2 shows how a communication system that uses the opposite-trial RV format to select among four possible options would operate. A user who hits his/her desired option confirms the selection by hitting the opposite hemifield (i.e., the opposite half of the periphery of the screen); and a user who misses his/her desired option cancels the selection, and produces an undecided outcome, by hitting the same hemifield again. In the sametrial RV format, confirmation would be achieved by hitting the

same hemifield again, while cancellation would be achieved by hitting the opposite hemifield. Table I summarizes the sequence shown in Fig. 2 and indicates how the single-trial 2D data actually collected were used to simulate it. In the data collected, each trial had one of four possible corner targets, so that each pair of successive trials was not necessarily an opposite or same pair. If the first target of the pair was hit, the pair was an opposite pair if the next target was opposite to the first (e.g., a bottom-left target followed by a top-right target), and it was a same pair if the next target was the same as the first. Otherwise (e.g., a bottom-left target followed by a top-left


330


target), the pair was not usable for RV simulation. However, if the first target of the pair was missed, the pair was an opposite pair if the next target was the corner hit by mistake in the first trial, and it was a same pair if the next target was opposite to the corner hit by mistake in the first trial. Otherwise (e.g., a miss that hit the upper-left corner followed by an upper-right target), the pair was not usable for RV simulation. The measures computed from a set of successive sessions from each subject were: RV proportion decided (i.e., proportion of pairs that was decided) for opposite pairs and for same pairs, RV accuracy (i.e., proportion of decided pairs that was correct) for opposite pairs and for same pairs, singletrial accuracy for corner targets, and single-trial accuracy for hemifields. As noted above, no matter how many choices there are in the first trial, the second trial of a pair need only be onedimensional (i.e., confirming or cancelling the selection of the first trial). Thus, while the first trial was correct only if the corner target was hit, the second trial was correct if the cursor hit the correct hemifield (i.e., the correct half of the periphery). For example, if the second target was the top-right corner, the correct hemifield was composed of the entire top and right sides. These results were compared to the results expected if spontaneous variation in mean amplitude did not occur or if any spontaneous variation that did occur was perfectly matched by the online adjustments in the intercept values of the two linear equations that controlled horizontal and vertical movements, respectively [i.e., the results expected if there for were no correlation between trial and trial from (1) and from (2)]. In this case, if is the single-trial is the single-trial accuracy accuracy for corner targets and for hemifields, then , the predicted RV proportion decided for opposite or same pairs, is given by the equation

(a)

(5) , the RV accuracy for opposite or same pairs, is given by the equation (6) These equations, in which the first trial may have any number of targets and the second trial has only two (i.e., confirm or cancel) are the general forms of the RV equations. Equations (3) and (4) are the special forms for the case in which the first and second trials have the same accuracy (e.g., when both have the same two-target, i.e., 1-D, format). and versus the value of , when Fig. 1(b) plots is 0.90 or 0.95. The predicted RV accuracy is impressive. For is 0.50, or 50%, is equal to (i.e., is either example, if 90 or 95%). III. RESULTS Six of the 16 subjects trained on 1-D cursor control completed at least 20 training sessions and four to five later sessions from each (containing 935–1325 trial pairs) were used in the present study. The other ten completed ten sessions, and sessions nine and ten (containing 334–526 trial pairs) were used. Because the first group was well-trained and the second group was at the beginning of training, single-trial accuracy

(b) Fig. 3. (a) Actual RV accuracies for opposite-trial () and same-trial () pairs of the 16 subjects trained on 1-D cursor control, versus single-trial accuracy, with the predicted relationship [i.e., (4)] superimposed. (b) Actual RV proportions decided for opposite-trial () and same-trial () pairs of these 16 subjects, versus single-trial accuracy, with the predicted relationship [i.e., (3)] superimposed.

varied widely across subjects, ranging from 94% in the most successful to 51% in the least successful. Thus, we were able to evaluate the results of the RV method over a wide range of subject performance. The three subjects trained on two-dimensional cursor control had previously mastered onedimensional control, and had achieved a substantial degree of two-dimensional control (i.e., accuracies of 60–65% with four targets). For each, we analyzed five to six later sessions containing 322–496 usable opposite or same trial pairs. A. Response Verification Performance for One-Dimensional Data Fig. 3(a) superimposes on the predicted relationship [i.e., (4)] between (i.e., accuracy for 1-D single trials) and



(i.e., RV accuracy) their actual values for the opposite and same target pairs of each subject. Fig. 3(b) superimposes on (i.e., the predicted relationship [i.e., (3)] between and RV proportion decided) their actual values for the opposite and same target pairs of each subject. For almost all subjects, RV accuracy is higher than predicted for opposite pairs and lower than predicted for same pairs. On the other hand, for almost all subjects, RV proportion decided is lower than predicted for opposite pairs and higher than predicted for same pairs. These differences are significant. The opposite-pair RV accuracies are significantly higher than expected and the samefor pair RV accuracies are significantly lower ( each by paired -test). At the same time, the opposite-pair RV proportions decided are significantly lower than expected and the same-pair RV proportions decided are significantly higher for each by paired -test). ( Furthermore, these differences between opposite and same pairs appear to be greater for lower values of (i.e., for lower single-trial accuracies). Fig. 4(a) plots for the opposite and same trial pairs of each subject the difference between actual and predicted RV accuracies versus single-trial accuracy; and Fig. 4(b) plots for the opposite and same trial pairs of each subject the difference between actual and predicted RV proportions decided versus single-trial accuracy. Linear regressions and values are also shown. In all four and corresponding cases, the difference declines as single-trial accuracy rises, and for opposite pairs the declines are clearly significant.

331

(a)

B. Response Verification Performance for 2-D Data Table II shows for each of the three subjects trained on 2-D cursor control the predicted values of RV accuracy and RV proportion decided [i.e., from (5) and (6)] and the actual values for opposite and same trial pairs. RV accuracy is higher than predicted for the opposite pairs and lower than predicted for the same pairs in all three subjects. The RV proportion decided is lower than predicted for the opposite pairs in all three subjects, and higher than predicted for the same pairs in two of the three. IV. DISCUSSION The ultimate usefulness of EEG-based communication will depend on its accuracy and speed. At this early stage of development, we are concentrating on maximizing accuracy because, as indicated in the Introduction, the initial uses of EEG-based communication are likely to involve transmission of relatively simple commands for which accuracy is more important than speed. The starting point for the present study was the observation that, with the current protocol in which subjects control cursor movement to a peripheral target, well-trained subjects can consistently achieve one-dimensional (e.g., up/down) accuracy greater than 90%, but not greater than 99%. The goal was to develop a method for increasing accuracy above this apparent limit. A. Improvement in Accuracy The RV procedure evaluated here increases accuracy in two ways: the first is predictable, the second derives from the

(b) Fig. 4. (a) Actual RV accuracies for opposite-trial () and same-trial () pairs of the 16 subjects who trained on 1-D cursor control minus the predicted values [i.e., (4) and Fig. 1(a)], versus single-trial accuracy. (b) Actual RV proportions decided for opposite-trial () and same-trial () pairs of these 16 subjects minus the predicted values [i.e., (3) and Fig. 1(a)], versus single-trial accuracy. Linear regression lines, r 2 values, and P values are shown. TABLE II PREDICTED AND ACTUAL RV ACCURACIES AND PROPORTIONS DECIDED FOR THE 2-D DATA

characteristics of EEG control. First, assuming that perforis independent of performance on trial mance on trial , basing a decision on two trials rather than one increases


332


accuracy as predicted in (4) and (6). Second, the present results indicate that basing a decision on two opposite-target trials rather than on two same-target trials further increases accuracy by reducing the impact of the unavoidable imperfection in intercept selection. The results imply that, for most subjects, and especially for those who are less accurate, failure of the intercept to adjust perfectly for spontaneous variation in mean EEG amplitude is a significant factor in limiting accuracy. This is indicated by the fact that RV accuracy was significantly higher than predicted for opposite pairs and significantly lower for same pairs, while the RV proportion decided was significantly lower than predicted for opposite pairs and significantly higher for same pairs. The RV method using opposite pairs increases accuracy more than expected from the accuracies of the two single trials composing the pairs. Thus, a typical subject who achieves 90% single-trial accuracy may be able to achieve an RV accuracy of over 99%. The additional RV advantage appears to be most marked when single-trial accuracy is lower, as it is in the early stages of subject training. This implies that one of the effects of continued training is a decrease in the spontaneous variation of mean EEG amplitude during performance. The usefulness of the RV method is not limited to algorithms based on linear equations. Any algorithm that translates EEG amplitude into movement will need to address the problem of spontaneous variations in mean amplitude. Thus, nonlinear equations will also face the intercept problem and should be able to incorporate a comparable solution based on requiring two successive movements in opposite directions to confirm a selection. B. Application to 2-D Control with Many Targets When 2-D cursor movement is introduced and the number of targets located around the periphery of the screen exceeds two, the information content of the message increases; and, as a comparison of Fig. 1(a) and (b) indicates, the increase in accuracy provided by the RV procedure is likely to be greater than it is for 1-D movement and two targets. As illustrated in Fig. 2, in 2-D applications only the first trial of the pair need have more than two targets. The second trial of the pair simply asks the question: “Was the first response correct?” “No” is signaled by reaching the half of the periphery (i.e., the hemifield) centered on the target selected in the first trial, while “Yes” is signaled by reaching the half of the periphery centered on the point directly opposite that target. By requiring a move in the opposite direction to confirm the result of the first trial, this 2-D RV method incorporates protection against imperfect selection of the intercepts used in the algorithm that determines 2-D movement. The number of targets, , in the first trial determines the maximum information content of the message. The content is bits. At the same time, unless the first-trial equal to log2 accuracy is very low [e.g., below 30% in Fig. 1(b)], it has relatively little effect on RV accuracy, which is determined mainly by the second trial. Thus, as shown in Fig. 1(b), when first-trial accuracy is 50%, RV accuracy is equal to the accuracy of the second (i.e., 1-D) trial. Thus, for example, a

subject who achieves 50% accuracy with eight targets and 1-D accuracy of 90%, can communicate a message of 3 (i.e., log2 8) bits with an accuracy of 90%. C. The RV Procedure and Information Transfer Rate The increased accuracy provided by the RV format comes at the cost of a decrease in speed. Speed is decreased first by the addition of a second trial, and second by the occurrence of undecided trials, the number of which is usually greater for opposite pairs. While we have focused our discussion of the RV procedure on the problem of achieving maximum accuracy in a message, it is also worthwhile to consider the procedure simply in terms of information rate, without consideration of accuracy in a discrete message or fixed time period. As derived from Pierce [7] (and originally from Shannon and Weaver possible targets in which each target [22]), for a trial with is equally probable, the probability ( ) that the target will be hit is the same for each target, and each error has the same )/( )], bit rate, or bits/trial ( ), is probability [i.e., ( (7) If target number does not affect trial duration, the highest bit rate will be obtained by choosing the value of for which is greatest, and simply presenting trials with targets. Thus, for a subject who had 90% accuracy with two targets, 60% with 4, and 30% with 16, the bit rates would be 0.53, 0.40, and 0.38, respectively, and two targets would be the best of these three choices. In contrast, if the accuracies were 90, 70, and 50%, the bit rates would be 0.53, 0.64, and 1.05, respectively, and 16 targets would be the best choice. In this situation, in which the goal is to maximize bit rate rather than accuracy in a discrete message, the RV procedure as described here has no place: the best format is single trials each of which has the number of targets that gives the highest bit rate. The RV procedure would only become worthwhile in terms of information transfer rate if the second, verification trial were much faster than the first trial. This appears to be a realistic possibility. At the end of the first trial, the subject knows what will be required in the second trial, so that the 2–3 s period between the end of cursor movement in one trial and the beginning of cursor movement in the second trial might be greatly reduced. The movement gain might also be increased for the second trial. The subject might learn to produce a sequential response, in which the first part would move the cursor toward the target and the second part would move it in the opposite direction. With appropriate feedback to ensure synchronization between subject and system, an RV trial might become one complex trial in which the task is, for example, to go up and down, or down and up, as rapidly as possible, and trial duration might be little more than that of a single trial in the present format. With such improvement, the RV procedure could produce substantial gains in both accuracy for a discrete message and in bit rate. V. CONCLUSION The present study is an offline analysis of single-trial data obtained when the target for cursor movement was chosen



by the computer. It does not reveal how subjects will perform when trials are actually paired, and when the correct responses are decided by the subject rather than stipulated by the computer. Thus, it is encouraging that an initial study in which subjects used the RV procedure to answer YES/NO questions asked by the operator gave results consistent with the analyzes described here [23]. The RV procedure improved performance as expected. If this result is confirmed, and especially if the duration of the second confirmatory trial can be shortened, the RV procedure may be an important step in the development of an EEG-based communication system of significant value to those with severe motor disabilities. ACKNOWLEDGMENT The authors would like to thank T. M. Vaughan for technical assistance and J. W. Wolpaw and Dr. M. Pregenzer for valuable comments on the manuscript. REFERENCES [1] D. J. McFarland, G. W. Neat, R. F. Read, and J. R. Wolpaw, “An EEG-based method for graded cursor control,” Psychobiol., vol. 21, pp. 77–81, 1993. [2] J. R. Wolpaw, D. J. McFarland, G. W. Neat, and C. A. Forneris, “An EEG-based brain-computer interface for cursor control,” Electroencephalogr. Clin. Neurophysiol., vol. 78, pp. 252–259, 1991. [3] J. R. Wolpaw and D. J. McFarland, “Multichannel EEG-based braincomputer communication,” Electroencephalogr. Clin. Neurophysiol., vol. 90, pp. 444–449, 1994. [4] T. M. Vaughan, J. R. Wolpaw, and E. Donchin, “EEG-based communication: Prospects and problems,” IEEE Trans. Rehab. Eng., vol. 4, pp. 425–430, 1996. [5] J. R. Wolpaw and D. J. McFarland, ‘Development of an EEG-based brain-computer interface (BCI),” Rehab. Eng. Soc. N. Amer., vol. 15, pp. 645–648, 1995. [6] K. L. Kilgore, P. H. Peckham, M. W. Keith, G. B. Thrope, K. S. Wuolle, A. M. Bryden, and R. L. Hart, “An implanted upper-extremity neuroprosthesis,” J. Bone Joint Surg., vol. 79-A, pp. 533–541, 1997. [7] J. R. Pierce, An Introduction to Information Theory. New York: Dover, 1980, pp. 145–165, et passim. [8] H. Ramoser, J. R. Wolpaw, and G. Pfurtscheller, “EEG-based communication: Evaluation of alternative signal prediction methods,” Biomed. Technik, vol. 42, pp. 226–233, 1997. [9] F. Sharbrough, G. E. Chatrian, R. P. Lesser, H. Luders, M. Nuwer, and T. W. Picton, “American electroencephalographic society guidelines for standard electrode position nomenclature,” J. Clin. Neurophysiol., vol. 8, pp. 200–202, 1991. [10] D. J. McFarland, A. T. Lefkowicz, and J. R. Wolpaw, “Design and operation of an EEG-based brain-computer interface (BCI) with digital signal processing technology,” Behav. Res. Meth. Instr. Comput., vol. 29, pp. 337–345, 1997. [11] D. J. McFarland, L. M. McCane, S. V. David, and J. R. Wolpaw, “Spatial filter selection for EEG-based communication,” Electroencephalogr. Clin. Neurophysiol., vol. 103, pp. 386–394, 1997. [12] B. H. Jansen, J. R. Bourne, and J. W. Ward, “Autoregressive estimation of short segment spectra for computerized EEG analysis,” IEEE Trans. Biomed. Eng., vol. BME-28, pp. 630–638, 1981. [13] S. L. Marple, Digital Spectral Analysis with Applications. Englewood Cliffs, NJ: Prentice-Hall, 1987. [14] S. Arroyo, R. P. Lesser, B. Gordon, S. Uematsu, D. Jackson, and R. Webber, “Functional significance of the mu rhythm of human cortex: An electrophysiologic study with subdural electrodes,” Electroencephalogr. Clin. Neurophysiol., vol. 87, pp. 76–87, 1993. ´ [15] H. Gastaut, “Etude e´ lectrocorticographique de la r´eactivit´e des rythmes Rolandiques,” Rev. Neurol., vol. 87, pp. 176–182, 1952.

333

[16] J. W. Kozelka and T. A. Pedley, “Beta and mu rhythms,” J. Clin. Neurophysiol., vol. 7, pp. 191–207, 1990. [17] W. N. Kuhlman, “Functional topography of the human mu rhythm,” Electroencephalogr. Clin. Neurophysiol., vol. 44, pp. 83–93, 1978. [18] E. Niedermeyer and F. H. Lopes da Silva, Electroencephalography: Basic Principles, Clinical Applications and Related Fields. Baltimore, MD: Urban and Schwarzenberg, 1987, pp. 97–117. [19] G. Pfurtscheller, “Central beta rhythm during sensory motor activities in man,” Electroencephalogr. Clin. Neurophysiol., vol. 51, pp. 253–264, 1981. [20] G. Pfurtscheller and A. Berghold, “Patterns of cortical activation during planning of voluntary movement,” Electroencephalogr. Clin. Neurophysiol., vol. 72, pp. 250–258, 1989. [21] G. Pfurtscheller, M. Pregenzer, and C. Neuper, “Visualization of sensorimotor areas involved in preparation for hand movement based on classification of alpha and beta band activity in single EEG trials in man,” Neurosci. Lett., vol. 181, pp. 43–46, 1994. [22] C. E. Shannon and W. Weaver, The Mathematical Theory of Communication. Urbana, IL: University of Illinois Press, 1964. [23] L. M. Miner, D. J. McFarland, T. M. Vaughan, and J. R. Wolpaw, “Answering questions with an EEG-based brain-computer interface,” Arch. Phys. Med. Rehab., vol. 79, 1998, to be published.

Jonathan R. Wolpaw received the A.B. degree from Amherst College, Amherst, MA, in 1966 and the M.D. degree from Case Western Reserve University, Cleveland, OH, in 1970. He completed a residency in neurology at the University of Vermont, Burlington, and fellowship training in neurophysiological research at the National Institute of Health (NIH). He is currently Chief of the Laboratory of Nervous System Disorders and a Professor at the Wadsworth Center of the New York State Department of Health and the State University of New York, Albany. His major research interest is in developing and using operant conditioning of spinal reflexes as a new model for defining the plasticity underlying a simple form of learning in vertebrates. He is also involved in developing an EEG-based brain-computer interface as a new communication channel for those with severe motor disabilities.

Herbert Ramoser received the M.Sc. degree in telematics from the Graz University of Technology, Austria, in 1997. He has been working toward the Ph.D. degree. He is currently a Research Associate at the Graz University of Technology, Austria. His primary research interests are blind separation techniques, signal processing, and application of neural networks. Currently, his research focuses on spatial properties of EEG and sleep analysis.

Dennis J. McFarland received the B.S. and Ph.D. degrees from the University of Kentucky, Lexington, in 1971 and 1978, respectively. He is currently a Research Scientist at the Wadsworth Center of the New York State Department of Health, Albany, NY. His major research interests are in development of EEG-based communication and in auditory psychophysics.

Gert Pfurtscheller, for a biography, see this issue, p. 324.
