Brain-computer interface (BCI) is a communication link between a human’s brain and an external device. The applications of BCI technology is vast with a couple of examples listed below:
Reading brain activity and translating that activity to commands or other information understandable by computers/devices are the main components of BCI technology.
In many cases, BCI systems also include some stimulation components for presenting options to a person to choose from. These are typically presented on some display but there can be other forms of presentation such as sounds or tactile stimulations.
There are many methods for measuring brain activity. Each method differs in its invasiveness, time and spatial resolutions and other practicalities. Commonly used techniques to record brain activity are illustrated in the picture below.
LFP and ECoG techniques measure electrical activity of neurons and offer good resolution in both time and spatial domains but due to their invasive nature are mostly used for research, plus the application outside laboratory environments is limited.
fMRI measures brain activity through changes in the blood flow. It is a non-invasive technique and can study the whole volume of the brain. Spatial resolution varies greatly depending on the fMRI devices and if the whole brain is being scanned or just a particular area of interest. Again, this technique is limited to laboratory settings as the fMRI scanner itself is very large.
EEG measures electrical signals using the electrodes placed on the outside of the scalp. It measures some average response from a population of neurons mainly from the outer part of the brain - cortex. The technique suffers from spatial resolution and increased noise when compared to LFP or ECoG, however it does have many practical advantages such as being a non-invasive method with many devices being portable too.
MEG records electrical activity of the brain indirectly by measuring associated magnetic fields. It is a non-invasive method and the MEG sensors do not even touch the person’s head. It is selective to certain electrical activity and generally offers slightly better spatial resolution than EEG. However, the equipment is more expensive and bulky and usually requires well isolated rooms from the electrical noise for good quality recordings.
fNIRS technique observes hemodynamic response of the brain by measuring the oxygenation level, which increases with elevated activation of local neuron populations. The temporal resolution of fNIRS is lesser than the EEG, however it can penetrate further through the skull and measure from the neuronal structures deeper within the brain. In addition, no contact or scalp preparation is needed for the measurements to be made, although the sensor placement stability or person’s hair may have significant influence on the quality of the measurements. The fNIRS devices are becoming more portable and in some cases it is also being combined with EEG sensors offering a versatile measurement system.
Neurotechnology concentrates on using the aforementioned electroencephalography technique as a way to monitor brain activity for the development of BCI technologies. The versatility, portability and other practical aspects of EEG, makes it a top candidate for the use cases in real-life applications.
Traditionally, measurements require scalp preparation and application of conductive gel for better electrode contact. However, dry-contact electrodes that exclude the need for gel are becoming more popular, increasing the applicability of BCI outside laboratory environments. Neurotechnology offers innovative dry-contact EEG electrodes that conform to the shape of the head and make them more comfortable to wear than traditional dry-contact electrodes. BrainAccess electroencephalography acquisition hardware is also designed for use with dry-contact electrodes as these electrodes typically have larger impedances than the gel based equivalents. Care must be taken to ensure good quality recordings.
Different BCI paradigms exist within the applications of this technology. These paradigms are related to how our brains function and their response to external stimuli and/or internal mental effort. The most widely used BCI paradigms are summarized in the following sections.
P300 is a type of visually evoked potential that can be observed in EEG recording after a person is presented with visual stimuli. The potential occurrence and strength depends not on the stimulus itself but on a person’s reaction to it. If a person is interested in a particular object, their response is stronger in terms of P300 when such an object is flashed on the screen when compared to other presented objects not relevant to a person.
An example of P300 potential is shown in a figure below. The potential typically occurs approx 300-400 ms after the presentation of the stimulus, hence the P300 name. Topographically, the strongest response is measured over the parietal cerebral cortex. The ‘target’/’non-target’ waveform corresponds to the person’s response when a stimulus of interest/non-interest was presented.
The P300 paradigm can be used for applications where a person can select from multiple presented options simply by focusing on a particular option. A keyboard interface can allow a person to type letters and is particularly useful for people diagnosed with ALS syndrome that can’t use conventional keyboards.
The P300 paradigm can also be employed in a more passive way and, for example, study how a person reacts to various stimuli when using simulators, operating critical machinery or even when browsing web pages in order to optimize the presentation of the interface. The versatility of P300 even extends to the application of criminology, where a suspect can be presented with object photos from a crime scene and from a P300 response, it can be determined if a person was at the crime scene and saw these particular objects or not.
The peak of the P300 is relatively small and is typically hidden in the noisy EEG recordings and usually an average of repeated measurements has to be taken in order to reveal P300 potential as shown in the previous figure.
Here, at Neurotechnology, we train machine learning algorithms to detect P300 signals even when they are buried in the noise and would not be detectable by conventional algorithms. We also aim to minimize the number of required electrodes for the detection of P300 potential in order to minimize and reduce the costs of the required hardware.
It is worth noting that for active control, when a person is trying to choose from multiple presented options, the process can be very long when the number of options is large. For example, it can take up to tens of seconds when choosing a letter from a keyboard. Therefore, a very important research direction is trying to minimize this time with the so-called ‘fast P300’ algorithms where stimuli at smaller intervals than 300-400 ms and algorithms have to cope with overlapping P300 potentials in the recordings.
A steady state visually evoked potential is another type of visually evoked potential. Unlike in the P300 paradigm a user is presented with stimuli flashing at constant frequencies. If a person visually focuses at an object flashing at a particular frequency, the signal of that frequency will be observed in the EEG recordings. Topographically, SSVEP signals are most pronounced over occipital cortex regions. An example of EEG signals with SSVEP signals present in them are shown in the Figure below.
As with the P300 paradigm, SSVEP can be used in applications where the user can select from various options, such as letters in the keyboard, by simply focusing on a desired option.
SSVEP generated EEG signals are generally relatively strong and easier to detect than P300 and BCIs based on SSVEP can typically work faster as all the stimuli can be presented at once. However, when a number of flashing options increase, the frequency differences between different stimuli have to decrease in order to fit them in the same frequency range. This can require longer recordings to have a frequency resolution good enough for separating and determining the stimuli that a person is looking at. There are approaches where instead of constant flashing frequencies, coded sequences are used in order to have more options given the same frequency bandwidth.
Motor Imagery (MI) based BCI technology is based on the fact that neuron activity changes in the motor cortex when someone imagines a movement of their own body. The EEG response can also change when only observing the motion performed by someone else.
One of the features that change in EEG recordings associated with the imagined movement is sensorimotor rhythms or oscillations. They are typically present over the motor cortex when no movement is made/imagined but decrease (de-synchronize) over a particular motor cortex region when a particular movement is made/imagined. For example, the rhythms decrease over the left side of the motor cortex if a person imagines a movement of the right leg/arm and vice versa. With higher electrode densities over the motor cortex it is possible to separate not only left/right sides but also if the movement is associated with leg, arm or even tongue.
The changes in EEG signals associated with MI are generally more subtle when compared to visually evoked potentials and vary greatly between people. Not only are sophisticated algorithms needed to detect MI commands reliably, but sometimes user training is required to elicit MI related EEG signals.
The application of motor imagery based BCI technology ranges from object control in slow-paced computer games to allowing disabled people to control exoskeletons. It is also used in neuro-rehabilitation to help people recover lost motor functions more quickly.
Brainwaves are electrical oscillations or rhythms observed in electroencephalography recordings. Various oscillations can reflect the state of mind of the person such as tiredness, relaxation, concentration, etc. There are associated oscillations with different sleep stages so it can be used to study and determine these.
The BCIs based on the brainwaves are probably the most widely used as the associated EEG signals are relatively strong and typically do not require a large number of sensors and expensive hardware.
An example of the so-called alpha waves is shown in the figure below. The alpha rhythm can be observed over the occipital cortex region when a person closes their eyes and/or their visual activity is at rest.
The challenge when using brainwaves for the development of BCI is to find the appropriate feature of the brainwaves that best relates to the state of mind that one wants to detect or classify.
BCI technology is undoubtedly a very dynamic and rapidly evolving field. EEG is currently probably the best technique for probing the brain activity and developing BCI applications. Nonetheless, there are still many technological and practical aspects that have to be overcome in order to see a wide spread use of BCI technologies:
Comfortable EEG headware. As mentioned previously, comfortable and portable hardware is essential for applications outside laboratory environments. Not only are dry electrodes necessary but also the actual headware has to be comfortable enough for the people to wear for a reasonable length of time. If it is used to monitor sleep, for instance, then the headware and the electrodes have to be very light and comfortable.
Transferable between sessions. The EEG signal can vary greatly between sessions, when the headware is removed and put on again or different hardware is used altogether. This is due to variation in electrode fitting, their position, environmental conditions such as electrical noise sources, etc. The algorithms have to be able to cope with these changes.
Transferable between people. Not only do the algorithms of BCI technologies have to deal with variation between sessions, they have to work reliably for different people as well. As every person is generally different there are differences in neural activity too.
EEG datasets. As electroencephalography signals are very complex and as already mentioned varying between sessions and people, we believe it is necessary to use machine learning to achieve better performance than conventional algorithms. Machine learning, particularly deep learning, has shown unprecedented performance in other tasks such as biometry and computer vision and we at Neurotechnology have first hand experience with this. Nonetheless, machine learning algorithms are ‘data hungry’ and many examples are needed for the algorithm to learn. Currently, there is still a relatively small amount of publicly available EEG datasets.
