High-resolution SSVEP-based brain– computer interface

: Steady-state visual evoked potential based brain–computer interface (BCI) is now attracting growing attention for its fast and efficient information transfer rate. However, human beings show excellent response only in a limited range of frequencies. Due to the limitation of frequency bandwidth, it is difficult to obtain enough frequency for target encoding. This study presents a high-resolution steady-state visual evoked BCI whose frequency resolution reaches 0.1 Hz. Compared with the widely-used system with frequency resolution of 0.2 Hz, this system have doubled the number of evoked frequencies, and increased the information transfer rate by 27.89% when sampling time is >3 s.


Introduction
Brain-computer interface (BCI) is a new kind of communication and control technology, which connects the human brain to external world by converting users' intention into machine command without the cooperation of normal nerves and muscles [1][2][3][4]. Recent developments in the field of steady-state visual evoked potential (SSVEP) have led to a renewed interest in BCI [5][6][7]. When a subject gazes at a periodically flickering visual stimulus, his/her brain will produce the potential with the same fundamental and harmonic frequency as the stimulus. Therefore, if different visual stimuli flashes at different frequencies, we can analyse the frequency of electroencephalography (EEG) signal evoked by certain stimuli to determine which stimuli the subject is gazing at and get the information the user wants to convey. For example, if a visual stimuli flickering at a frequency X Hz was associated with the input command of '1', the user can easily express that he/she want to input '1' by gazing at corresponding visual stimuli [8,9].
The first approach is to enhance the system resolution to get more evoke frequency. Some researchers have focused on to enhance the resolution to realise a large number of frequencies.
Chen et al. [10] developed a high-speed BCI speller with 40 frequencies selected from 8 to 15 Hz. Using a joint frequencyphase modulation (JFPM) method, this system achieved high resolution of 0.2 Hz. Masaki et al. recently researched on whether the frequency resolution affects the classification performance of SSVEP system. Their research also chose 0.2 Hz as the minimum frequency interval. By comparing the different frequency resolutions (0.2, 0.4, 0.6, 0.8 and 1 Hz), they concluded that there was no significant difference between frequency resolutions when combining JFPM coding and template-based target identification methods [11].
The second approach is to utilise cross frequency modulation to encode one stimulus with two data. In theory, cross frequency modulation can encode 2 n subjects with n frequencies, while only n subjects can be encoded in normal method. Anette (2012) found that 5.8 and 6.2 Hz will be generated in the subjects' EEG signal, if the size of the grid changed at 0.2 Hz and the brightness of the grid changed at 6 Hz [12]. This research shows that cross frequency modulation can be used in SSVEP to increase the target. Inspired by Anette's work, Chen et al. [13] used colour and brightness as the objects of modulation. Fifteen subjects participated in this experiment and got a satisfactory result (average accuracy: 93.83%, information transmission rate (ITR): 33.8 bit/min).
However, some researchers found that the system using cross frequency modulation will lose some frequencies and lead to instability in these BCI systems [14,15]. Moreover, cross frequency modulation will complicate the system, and make the system hard to use. So this study will focus to enhance the system resolution to get more evoke frequencies.

Dataset
The EEG data used in this study was collected by 32-channel (30 measuring electrode, 1 reference electrode and 1 ground electrode) EEG signal acquisition instrument (sampling rate: 1 kHz). These active Ag/AgCl electrodes were placed according to the extended international 10-20 system (Fig. 1a). The reference electrode was set on the subject's bilateral mastoid and the ground electrode was set on the position of Fpz. The contact resistance of each electrode was guaranteed to be <500 Ω during the experiment.
Ten healthy subjects participated in this SSVEP experiment voluntarily. This stimulus interface designed for password input was used for visual stimuli in this experiment (Fig. 1b). During the experiment, 12 stimuli square (2 cm× 2 cm) were shown on a liquid-crystal display (LCD) monitor (refresh rate: 60 Hz, resolution: 1920 × 1080) within 11.5 cm× 7.7 cm. The subjects were asked to sit on a chair at a distance of 70-100 cm and gaze at the visual stimuli for 5 s followed by a 2-s short break. In order to make the data more representative, low frequencies (7.8-8.3 Hz) and medium frequencies (20.8-21.3 Hz) with frequency interval of 0.1 Hz were chosen to generate SSVEP data. Control experiments were designed with low frequencies (7.6-8.6 Hz) and medium frequencies (20.6-21.6 Hz) with frequency interval of 0.2 Hz. Each frequency was repeated three times in one experiment.

Signal preprocessing
The raw data was downsampled to 250 Hz to deduce the data. Industrial and other kind of noise signals were filtered by a fifthorder Chebyshev band-pass filter (pass band: 7-45 Hz). The baseline was corrected after filtering. Then the data was cut into 5s epochs and time-locked to the stimulation onset. Furthermore, all the data were divided into two parts. One group labelled with corresponding targets was used for channel selection and the other one was used for system verification.

Channel selection
As channels perform differently in frequency recognition, the channel with lower signal amplitude will reduce the accuracy and ITR of the system, and increase the system operation time. So selecting appropriate channels is essential for the performance of SSVEP system.
To achieve this goal, 10 experiments were arranged at each frequency. Signal acquired at each channel was analysed using Fast Fourier Transform (FFT) method. Intensity of the frequencies same as visual stimuli was put at corresponding channels in the map (Fig. 2). Those channels with high frequency intensity would be selected for signal extraction. For data analysis, images, P3, Pz, P4, O1, Oz, O2 (circled in Fig. 2), were selected.

Classification and performance evaluation
To verify the performance of SSVEP system with 0.1 Hz resolution when using different algorithms, power spectral density (PSD) and canonical correlation analysis (CCA) are selected in this study.

Power spectral density:
PSD is a classical algorithm in SSVEP signal analysing, which transforms the brain wave whose amplitude changes with time into frequency domain, so that the frequency distribution of the EEG signal can be observed intuitively.
There are two ways to realise PSD. One is to calculate the autocorrelation function of the EEG signal according to the Wiener Hinchin theorem firstly. Then calculate the Fourier transform of the autocorrelation function to obtain the PSD.
Let the preconditioned EEG signal be x(t), then the PSD can be obtained by Another method is to relate the power spectrum to the square of the amplitude-frequency characteristic. The power spectrum is the ratio of the overall average to the duration of the square of the amplitude-frequency characteristic, which is the limit value when the duration becomes infinity where x(ω) is the Fourier transform of x(t) and S x (ω) is the PSD of x(t).

Canonical correlation analysis:
The CCA algorithm is to obtain the linear relationship between two sets of variables [9,16]. The algorithm finds two linear transformations of x and y so that the correlation coefficient between the two combined variables (i.e. a b) after the linear change is the largest.
The CCA algorithm is to find two linear transformations (w x , w y ) of the variables x and y, such that the correlation coefficient between the two combined variables (w x T x, w y T ) after the linear change is maximised. Assuming that there are two sets of random variables with N samples, respectively, the linear transformation of these N samples, and two sets of data are obtained as follows: The CCA algorithm is to maximise the correlation between the two sets of data and can be expressed as: When used in SSVEP frequency recognition, one variable is a group that includes standard sine waves with frequency same as all targets, the other variable is EEG signal. If the EEG signal has the maximum correlation coefficient with a certain sinusoidal signal, we think visual stimulus is flashing at the same frequency as this sinusoidal signal.

Filter bank canonical correlation analysis:
Filter bank canonical correlation analysis (FBCCA) is a modified method to incorporate fundamental and harmonic frequency components to improve the performance of SSVEP BCI [17]. It decomposes SSVEPs into sub-band component and makes it more efficient to extract information in the harmonic components. Three steps are included in FBCCA. First, spectral analysis is performed on the EEG signals. Second, the bandpass filters with different passbands are used to filter the signals, so that the harmonics are separated independently. Finally, weighted CCA results of different harmonics multiply corresponding factors, then the results are added as the final amplify of frequency [18]. In this method, we chose three harmonic for analysis. The sub-bands are designed as shown in Fig. 3a.
The corresponding factors of different orders of harmonics are determined by the following formula ω(n) = ∑ 1 N n −(5/4) + 0.25, n = 1, 2, …, 5 where ω(n) is a factor corresponding to a certain fundamental frequency, and n is the order of sub-filter of this frequency. ρ k (n) is the value of standard CCA of kth target, and ρ k is the value of FBCCA of kth target, and Y is the final predicted result.
Since FBCCA has better performance in periodic signal analysis, we chose FBCCA as method for target recognition.

Evaluation methods
This experiment designed a 0.1 Hz frequency interval and a 0.2 Hz frequency interval test to determine whether the system with 0.1 Hz frequency interval can increase the system's information transfer rate. The collected signals are divided into two groups, one for channel selection and the other for evaluating the information transfer rate of two systems with different resolution.
The definition of information transform rate is given by where N is the number of target, p is the average classification accuracy, and ITR is the information transmission rate for the system (unit: bits/min).

Result
Detected by FBCCA method, EEG signal has clear peak at corresponding stimuli frequency, which proves that the system with a 0.1 Hz frequency interval can be used in SSVEP to provide more targets (Fig. 4a). However, results shown in Fig. 4b indicate system with a 0.1 Hz frequency interval is difficult to achieve the same accuracy as 0.2 Hz when sampling time is <3 s. This may be due to the limitations of frequency resolution of the human visual nervous system. Fig. 4c shows that although the system accuracy rate of 0.1 Hz is lower when sample time is 2 s-3 s, the ITR of the 0.1 Hz system can be >0.2 Hz due to the increase of the number of identification targets.

Conclusion
Previous studies have shown that the greater the frequency interval, the easier it is for the system to resolve different frequencies, and the higher accurate rate also can be obtained. The result of Fig. 4b does confirm this conclusion. The higher the frequency resolution is, the lower the system classification accuracy will be, but the number of targets the system can provide will be more. So the two factors form a constraint relationship. From this experiment, we can conclude that when sampling time is >3 s, the accuracy of the frequency interval of 0.1 and 0.2 Hz tends to be the same, but the 0.1 Hz ITR is increased by 27.89% compared with 0.2 Hz. Therefore, we can conclude that when the sampling time is <3 s, it is recommended to use a system with a frequency resolution of 0.2 Hz. When the experiment time is >3 s, it is recommended to use a system with a frequency resolution of 0.1 Hz. This research has thrown up an interesting question in need of further investigation: why does the system with a frequency interval of 0.1 Hz perform unsatisfying when sampling time is limited? Many factors will lead to such a result, such as the inability of the human visual nervous system to resolve small frequency changes or the low resolution of the algorithm. So, future work will answer which of the two causes such a result and focus on frequency resolution of human visual nervous system based on EEG analysis.