30th Annual International IEEE EMBS Conference Vancouver, British Columbia, Canada, August 20-24, 2008
Hilbert-Huang Spectrum as a New Field for the Identification of EEG Event Related De-/Synchronization for BCI applications Konstantinos I. Panoulas, Student Member, IEEE, Leontios J. Hadjileontiadis, Member, IEEE, and Stavros M. Panas, Member, IEEE
Abstract-Brain Computer Interfaces (BCI) usually utilize the suppression of mu-rhythm during actual or imagined motor activity. In order to create a BCI system, a signal processing method is required to extract features upon which the discrimination is based. In this article, the Empirical Mode Decomposition along with the Hilbert-Huang Spectrum (HHS) is found to contain the necessary information to be considered as an input to a discriminator. Also, since the HHS defines amplitude and instantaneous frequency for each sample, it can be used for an online BCI system. Experimental results when the HHS applied to EEG signals from an on-line database (BCI Competition III) show the potentiality of the proposed analysis to capture the imagined motor activity, contributing to a more enhanced BCI performance.
I. INTRODUCTION
W
HEN the human brain is in a calm and alert state of consciousness, the sensorimotor cortices show rhythmic electroencephalogram (EEG) oscillations, known as mu-rhythm with their frequencies ranging from 8 to 12Hz. Motor activity, both actual and imagined, as well as somatosensory stimulation, attenuate the mu-rhythm in a phenomenon known as Event Related Desynchorization (ERD). After the motor activity is stopped, the mu-rhythm rebounds and this phenomenon is known as Event Related Synchronization (ERS) [1], [2]. The mu-rhythm during ERS is usually higher than during calm states of consciousness. ERD is a usual candidate for the creation of a Brain Computer Interface (BCI) because most people can learn how to control the mu-rhythm using imagined motor activity. The simplest case of a BCI is the left and right control of a cursor and in order to achieve this, the imaginary movements of the left and the right hand have to be distinguished. Generally, a BCI consists of a signal processing part, in which some features are extracted from the signal, and a classification part in which the different states are Manuscript received April 16, 2008. Konstantinos Panoulas is a PhD candidate at the Department of Electrical and Computer Engineering, School of Engineering, Aristotle University of Thessaloniki, Greece (e-mail:
[email protected]). Leontios Hadjileontiadis is with the Department of Electrical and Computer Engineering, School of Engineering, Aristotle University of Thessaloniki, Greece. The author can be contacted at:. Aristotle University Campus, Building D GR54124, tel +302310996340 (e-mail:
[email protected]). Stavros Panas is with the Department of Electrical and Computer Engineering, School of Engineering, Aristotle University of Thessaloniki, Greece (e-mail:
[email protected]).
978-1-4244-1815-2/08/$25.00 ©2008 IEEE.
discriminated using the features acquired from the previous part. The signal processing methods reported in the state-ofthe-art for imagined motion BCI algorithms include band power estimates [3], AR modeling [4], [5], Fourier transform [6], [7], and prediction neural networks. In this paper, the EEG signal is processed using Empirical Mode Decomposition (EMD) and Hilbert-Huang Spectrum (HHS), revealing appropriate information for the discrimination of the ERD and the ERS events, contributing to a more enhanced BCI performance. II. METHODOLOGY A. Empirical Mode Decomposition The EMD is a signal-dependent decomposition of a time series into waveforms modulated in amplitude and frequency [8]. The iterative extraction of these components is based on the local representation of the signal as the sum of a local oscillating component and a local trend. The first iteration of the algorithm consists in extracting a component, referred to as Intrinsic Mode Function (IMF), which represents the oscillations of the entire signal. The difference between the original signal and the IMF time series is the residual. The IMF component is obtained by a sifting process such that it satisfies the requirement that it is zero-mean and that the number of extrema and the number of zero crossings are identical or differ by one. This same procedure is then applied to the residual to extract the second IMF. Therefore, all IMFs are iteratively extracted. The nonstationary signal x(t ) is then represented as a sum of IMFs and the residual K
ci (t ) rK (t ) , where ci (t ) denotes
component x(t ) i 1
the ith extracted empirical mode and rK (t ) the residual, which is a monotonic function without extrema and can either be the mean trend or a constant. The EMD algorithm can be summarized as follows: 1. Identify all the extrema (minima and maxima) of x(t ) . 2. Interpolate between minima, maxima using cubic spline to produce the lower emin (t ) and the upper emax (t ) envelope, respectively. 3. Compute the average
m(t )
(emax (t ) emin (t )) / 2 . 4. Extract the IMF component by c1 (t ) x(t ) m(t ). 5. Iterate on the residual m(t ).
3832
B. Hilbert-Huang Spectrum In order to calculate the HHS, the Hilbert transform is applied to each IMF using the following equation: (1) where P indicates the Cauchy principal value and . , the phase and the instantaneous The amplitude can be calculated as: frequency , , .
(2) (3) (4)
After performing the Hilbert transform on each IMF component, we can express the data in the following form: (5) Equation (5) gives both the amplitude and the frequency of each component as functions of time [8]. The variable amplitude and the instantaneous frequency have not only greatly improved the efficiency of the expansion, but also enabled the expansion to accommodate non-stationary data. With IMF expansion, the amplitude and the frequency modulations are also clearly separated. In this way, we have broken through the restriction of the constant amplitude and fixed-frequency Fourier expansion, and arrived at a variable amplitude and frequency representation. Equation (5) also enables us to represent the amplitude and the instantaneous frequency as functions of time in a three-dimensional plot, in which the amplitude can be contoured on the frequency-time plane. This frequency-time distribution of the amplitude is designated as the Hilbert Huang amplitude spectrum, , or simply HHS.
Fig. 1. The EMD of the EEG channel C3 from recording 2. The subject was instructed to move the left hand after .
III. DATASET CHARACTERISTICS The dataset used in this work is a publicly available dataset used for BCI Competition III (http://ida.first. fraunhofer.de/projects/bci/competition_iii/). This experiment contains 280 trials each 9 seconds in length. The first two , a trigger beep seconds of each trial were quiet. At indicated the beginning of the trial and a cross was displayed for one second. At , a left or a right arrow was displayed asking the subject to move a bar into the direction of the cue. Three bipolar EEG channels were recorded with a sampling rate of 128Hz for each trial and filtered between 0.5 and 30Hz. 7KH FKDQQHO SRVLWLRQV RQ WKH VXEMHFW¶V EUDLQ were C3, Cz and C4, according to the international 10\20 system. The acquired signals were further filtered with an 8th-order Butterworth band pass filter between 7 and 13Hz to isolate the mu-rhythm and the HHS was then applied. IV. RESULTS AND DISCUSSION Some indicative results from the application of the HHS to the whole available EEG dataset are shown in this section. In particular, Fig. 1 denotes the EMD of the EEG channel C3 from recording 2. Most of the signal energy is located at the first two IMFs. In that experiment, the subject was
Fig. 2. The HHS of channel C3, from the 2nd trial of the experiment. The subject was instructed to move the left hand thus the left sensorimotor cortex (which responds to right hand movements) should not show mu rhythm suppression.
instructed to move the left hand, thus the left sensorimotor cortex (which responds to right hand movements) should not show significant mu-rhythm suppression, something that is and more clearly at the reflected to the IMFs after corresponding HHS, illustrated in Fig. 2. Figure 3 presents the EMD of the EEG channel C4 from recording 2. The subject was instructed to move the left hand thus the right sensorimotor cortex (which responds to left
3833
Fig. 4. The HHS of channel C4, from the 2 nd trial of the experiment. The subject was instructed to move the bar left, thus the right sensorimotor cortex (which responds to left hand movements) should show significant mu rhythm suppression.
Fig. 3. The EMD of the EEG channel C4 from recording 2. The subject was instructed to move the left hand thus the right sensorimotor cortex (which responds to left hand movements) should show mu rhythm suppression. After the 4th second of the recording WKHPXUK\WKPVXSSUHVVLRQFDQEHVHHQDWWKHILUVW,0)¶V
hand movements) should show mu-rhythm suppression. After the 4th second of the recording the mu-rhythm suppression can be seen at the first two IMFs and the relevant HHS depicted in Fig. 4. The EMD of the EEG channel C3 from recording 10 is shown in Fig. 5. Most of the signal energy is located at the ILUVW ,0)¶V 7KH VXEMHFW ZDV LQVWUXFWHG WR PRYH WKH OHIW hand thus the right sensorimotor cortex (which responds to left hand movements) should show mu-rhythm suppression. After the 4th second, the mu-rhythm suppression is visible at the first two IMFs and the corresponding HHS illustrated in the ERS event Fig. 6. At the end of the signal around is evident (both in Figs. 5 and 6). Lastly, the EMD of the EEG channel C4 from the 10th trial is shown in Fig. 7. Again most of the signal energy is located at the first two IMFs. The subject here was instructed to move the bar right thus the right sensorimotor cortex (which responds to left hand movements) should show minimal or no mu-rhythm suppression which can easily be spotted at the first two IMFs and reflected at the relevant
Fig. 5 The EMD of the EEG channel C3 from recording 10. The subject was instructed to move the left hand. At the end of the signal ) the ERS event is notable. (
HHS of Fig. 8. The results presented in Figs. 1-8, denote that the HHS could clearly track the EDS/ERS phenomena, and could reveal information for on-line BCI implementation. The
3834
Fig. 8. The HHS of channel C4, from the 10th trial of the experiment. The subject was instructed to move the bar right, thus the right sensorimotor cortex should show minimal mu-rhythm suppression.
Fig. 6. The HHS spectrum of channel C3, from the 10 th trial of the experiment. The subject was instructed to move the bar right, thus the left sensorimotor cortex should show significant mu rhythm suppression between 4 and 7.5s, whereas the ERS event is clear for .
predefined, as in the case of wavelet transform, resulting in a more adaptive performance of the HHS. Finally, the incorporation of selective IMFs, could lead to a more focused signal processing in the area of the mu-rhythm, eliminating further effects at other frequency bands. V. CONCLUSION A new approach in the identification of the motor activity, both actual and imagined reflected in the mu-rhythm attenuation or rebound, using HHS analysis was presented here. The proposed approach seems to efficiently track the changes in the mu-rhythm, as tested in the HHS analysis of EEG signals from a BCI on-line database. This performance paves the way for adopting a new domain for EEG analysis that could foster the on-line BCI implementation. REFERENCES [1]
[2]
[3]
[4]
[5]
[6]
Fig. 7. The EMD of the EEG channel C4 from the 10 th trial. The subject was instructed to move the bar right resulting in minimal or no mu-rhythm suppression which can easily be seen at the first two IMFs.
[7]
ability of the HHS to follow the nonstationary characteristics of the EEG signal secures a new domain to act upon for a more efficient direction-identification of the XVHU¶Vintention of a movement. As the construction of the IMFs is based on the information of the signal itself, the analysis bases are not
[8]
3835
- :ROSDZ DQG ' 0F)DUODQG ³0XOWLFKDQQHO ((*-based brainFRPSXWHU FRPPXQLFDWLRQ´ Electroenceph. Clin. Neurophysiol., vol. 90, pp. 444±9, 1994. *3IXUWVFKHOOHUDQG)+/RSHVGD6LOYD³(YHQW-related EEG/MEG V\QFKURQL]DWLRQ DQG GHV\QFKURQL]DWLRQ %DVLF SULQFLSOHV´ Clin. Neurophys.,vol. 110, pp. 1842±1857, 1999. * 3IXUWVFKHOOHU & 1HXSHU ' )ORW]LQJHU DQG 0 3UHJHQ]HU ³((* based discrimination between imagination of right and left hand PRYHPHQW´Electroenceph. Clin. Neurophysiol., vol. 103, pp. 642±51, 1997. G. Dornhege, B. Blankertz, G. Curio, and K.-50OOHU³%RRVWLQJELW rates in non-invasive EEG single-trial classifications by feature combination and multi-FODVV SDUDGLJPV´ IEEE Trans. Biomed. Eng., vol. 51, no. 6, pp. 993±1002, Jun. 2004. G. Pfurtscheller, C. Neuper, G. R. Muller, B. Obermaier, and *.UDXV]³*UD]-%&, 6WDWHRIWKHDUWDQGFOLQLFDODSSOLFDWLRQV´IEEE Trans. Neural Syst. Rehab. Eng., vol. 11, no. 2, pp. 177±180, 2003. J. Lehtonen 3 -\OlQNL / .DXKDQHQ DQG 0 6DPV ³Online Classification of Single EEG TULDOV'XULQJ)LQJHU0RYHPHQWV´ IEEE Trans. Biomed. Eng., vol. 55, no. 2, pp 713-720, Feb. 2008. -.DOFKHU')ORW]LQJHU&K1HXSHU6*|OO\DQG*3IXUWVFKHOOHU ³Graz brain-computer interface II: towards communication between humans and computers based on online classification of three different EEG patterns,´ Medical and Biological Engineering and Computing, vol. 34, no. 5, pp. 382±388, Sep. 1996. N. Huang, Z. Shen, S. Long, M. Wu, H. Shih, Q. Zheng, N.-C. Yen, C. TuQJ DQG + /LX ³The empirical mode decomposition and the Hilbert spectrum for nonlinear and non-stationary time series analysis,´ Proc. R. Soc. Lond. A, no. 454, pp. 903-995, 1998.
