Abstract
Our research team has developed a system and methodology for measuring psycho-physiological parameters, which can be used to determine the level of fatigue and fitness of the person being measured. This article describes the electroencephalography (EEG) part of this system. This article covers the technical and mathematical background of EEG measurement, the selection and implementation of the measurement tool in the development environment, and the development of the measurement and processing algorithm. The result is a system that can detect, digitize, and process the digitized signal from the brain, and save the processed signal in an XML database.
1 Introduction [1]
Electroencephalography (EEG) is a non-invasive measurement procedure for detecting electrical signals from the brain and then processing these signals. With the help of signals from the brain, we can determine the current state of the human mind and track the change to certain external stimuli from the ground state. Fixing the baseline is a critical point for EEG measurements, since every person has a different datum elsewhere, it is not possible to add a single datum for each person. Compared to the correctly recorded baseline condition, we can draw conclusions that the subject's mental state has deteriorated or improved as a result of certain stimuli. Deeper, more comprehensive analysis requires serious neurological knowledge, but simpler, but more important analyses can be performed by simple mathematical calculations.
For example, surveying the mental health of those involved in extreme work is an area of application that has shown rapid development and need for further research. This includes performing the aforementioned resting measurement, which is important for the subject to be calm and relaxed. After resting, it is possible to re-examine the physical or mental subject to see how your brain responded to the exercise. Typically, these jobs include pilots, firefighters, air traffic controllers, ambulances.
The purpose of this research is to create a system that can quantify the fatigue and fitness status of a person and store this data in a database. The complete system includes an ECG gauge, an EEG gauge, a vascular stiffness gauge, a muscle oxygen level gauge, and a skin temperature gauge. The data processed must be credible in order to provide an expert with relevant data.
This paper is part of a larger research project, further results will be published in [18, 19].
2 Technical and mathematical basis of EEG measurement
2.1 Technical background of EEG measurement
The non-invasive EEG measurement procedure places low-resistance metal macroelectrodes on the scalp according to an international system. This system selects 4 anatomical reference points on the head and places the electrodes at 10 and 20% distance between these reference points, respectively. However, it is now more common to use the 10–10 system, which requires electrodes to be positioned 10% from the reference point. The measuring system always measures the potential difference between two points [2].
The names of the electrode locations are derived from anatomical expressions, with F beginning with the frontal, i.e., the front, P with the parietal, i.e. the back of the head, C starting with the central, i.e. the middle, T the temporal, i.e. at the ear and O at the occipital (Figs. 1 and 2).
EEG electrode mounting standard [3]
Citation: International Review of Applied Sciences and Engineering IRASE 11, 3; 10.1556/1848.2020.00151
Bipolar and unipolar EEG measurement [4]
Citation: International Review of Applied Sciences and Engineering IRASE 11, 3; 10.1556/1848.2020.00151
From the point of view of the measurement principle, two types are distinguished:
- A.Bipolar measurement is the measurement of the potential difference between two electrodes.
- B.Unipolar measurement, where the potential difference between two electrodes is also measured, but one of which is always an inactive reference electrode.
The electrodes used for measurement can be divided into two groups, distinguishing between dry and wet electrodes. For smaller portable weighing systems, a dry electrode is generally used, and for multi-channel systems, a wet electrode is preferred. The dry electrode has a gold plating which, once applied, can be used immediately and provides good contact with the skin surface. It does not require any professional qualification [5].
The signals taken from the skull using the electrodes should be filtered and amplified, usually by the measuring device on the head, but if necessary, the program can further refine the amplification and filtering. Doing these is key to signal processing because we can only process properly noise-free and amplified signals so that we can extract useful information from them [6].
Fig. 3 illustrates the structure of a classical EEG system that does not perform any pre-processing (filtering, amplification) or display. While we know that we can draw conclusions from the temporal display of brain waves, such as epilepsy, blinking, etc., we need to perform further operations on the already transformed digital signal for deeper evaluation [7].
Generic EEG system block diagram [8]
Citation: International Review of Applied Sciences and Engineering IRASE 11, 3; 10.1556/1848.2020.00151
There are many processing methods available to analyze EEG signal for different purposes, such as medical examination of Brian Computer Interface (BCI) [16]: Artificial Neural Networks (ANN), Support Vector Machines (SVN), and PCA, Principal Component Analysis. The above-mentioned methods are found to be very useful to extract special features from noisy signals. Our research aim is to find low computation and direct (simple) method to extract features, what is proportional to physical and psychological fatigue, we established our method in time domain, and statistical analysis.
2.2 Mathematical background of EEG measurement [9]
During the EEG measurement, the digitized waveform from the measuring system is processed for further evaluation for fitness/fatigue evaluation. If the signal is not properly filtered, additional noises may affect the sensitivity of the fitness/fatigue evaluation.
The Butterworth filter is the most used filtering type in this discipline because the cutting characteristics of this filter are the best to avoid distorting the signal after filtering so that no information is lost during processing. According to Butterworth transfer function (1), this type of filter rejects unwanted frequencies, and provides unified gain over the bandwidth of wanted frequencies.
The properly filtered time domain signal must be converted to a frequency domain so that each of the notable brain frequency bands can be determined. The method used for this is the Fast Fourier Transform, or FFT, which is a faster version of the discrete Fourier transform.
Butterworth filter
Butterworth filters are of the type IIR, or Infinite Impulse Response filters. Butterworth filters have the following characteristics:
Damped amplitude function at all frequencies,
The amplitude curve decreases monotonically from a given cut-off frequency,
Maximum flatness, unit response in transmission band and zero in crop band,
Half power frequency or 3 dwindling frequency is related to cutting frequency,
Butterworth filter Bode diagram with different order numbers [9]
Citation: International Review of Applied Sciences and Engineering IRASE 11, 3; 10.1556/1848.2020.00151
FFT algorithm [10]
FFT, or fast Fourier transformation, is essentially a faster version of a classic Fourier transformation. In the direct implementation of the discrete Fourier transform, we need to perform approximately N2 complex operations on N samples, which is a very time-consuming calculation operation.
The FFT and the power spectrum are very useful for measuring the frequency content of steady and transient signals. The FFT gives an average frequency content over the entire measurement range. Therefore, FFT is useful when the signal is steady state or when an average power content is required at each frequency. The frequency spectrum obtained from the EEG signal can be classified into the notable frequency bands shown in Table 1:
EEG frequency bands and their characteristics
Name | Frequency | Features |
δ-delta | Less than, 4 Hz | Fastest and strongest brain wave, very low frequency, high amplitude. Usually deep sleep or sleepless sleep. It is also found in deep meditation. |
θ-theta | 4–8 Hz | Drowsiness, light sleep, deep relaxation or meditation. |
α-alpha | 8–12 Hz | Relaxation, tranquility or peace. |
β-beta | 12–30 Hz | Normal waking awareness, high alert, active thinking, anxiety or focus. Most people work in this band during the day. |
γ-gamma | More than 30 Hz | Fastest brainwave. Hyperactivity or processing of information from different brain areas. |
3 Selecting the EEG measuring instrumentation [11]
The development procedures started with the selection of proper EEG device that is properly authenticated, can communicate wirelessly and is easy to install to patient and use by non-technical experts as well. There are many EEG metering systems on the market for sport, meditation, and medical applications, in variable price ranges. Our selection was the Muse 2016 EEG instrument, which can detect EEG signals with the same accuracy as a laboratory instrument, and comes with a manufacturer-supplied Application Programming Interface (API) for quick integration into any development environment and facilitating the development process [12].
Muse EEG is a 4-channel EEG measurement device capable of delivering real-time data of EEG signals via wireless connection (Bluetooth Low Energy 4.2.) (Fig. 5)
Muse EEG [11]
Citation: International Review of Applied Sciences and Engineering IRASE 11, 3; 10.1556/1848.2020.00151
The technical parameters of the Muse EEG are shown in Table 2.
Muse EEG technical parameters
Wireless connection | BT 4.2 BLE |
EEG channels | 4 Measurement Channels |
256 Hz sampling rate | |
12 bit/sample | |
Reference electrode position | FPz (CMS/DRL) |
Channel electrode position | TP9, AF7, AF8, TP10 (dry) |
Battery Time | Maximum 10 hours (rechargeable Li-Ion) |
Accelerometer | Three-axis 52 Hz, 16 bit resolution, range ±4 g |
Gyroscope | ±1,000 °/s |
The device is capable of real-time monitoring of brain waves on 4 EEG channels with 256 [Hz] sampling rate and 12 [bit] resolution, and a 3-axis accelerometer and gyroscope are included to estimate head pose and movement. Pose and movement information will be useful later in the measurement as motion can cause disturbance in the signal, so if the EEG waveform and the accelerometer are both jumping and disturbed at the same time. Source of disturbance signal caused by the movement and jerking.
Fig. 6 clearly shows that the device uses 4 channels, two on the forehead and two behind the ear, with the reference electrode located in the center of the forehead (Fpz). This 4-channel EEG measurement system proved to be sufficient (in technical and financial terms) for this project, and it is possible to extract the data needed to assess the subject's state of mind [13]. Bluetooth wireless connection provides a stable connection in our office and laboratory use, for our measurement, but it should be noted that it may not be suitable for all possible application areas. Wireless connection may be disturbed by other wireless devices utilizing the same 2.4 GHz ISM band. Cooperation with other Bluetooth health care devices, such as smart watches or sleep monitors, was not tested.
Muse EEG distribution of electrodes [1]
Citation: International Review of Applied Sciences and Engineering IRASE 11, 3; 10.1556/1848.2020.00151
4 Muse EEG implementation in labview development environment
4.1 Pairing Muse EEG with your computer
A utility called Muse Direct needs to be installed from the Microsoft Store to connect the device to your computer. This application communicates with the device via a Bluetooth connection, and then transmits the incoming data to the processing program as an OSC (Open Sound Control) stream, UDP (User Datagram Protocol). Please note that Bluetooth was the lower level protocol and TCP/IP protocol was the higher level protocol, where UDP was applied (Fig. 7).
Pair Muse EEG with Muse Direct [own figure]
Citation: International Review of Applied Sciences and Engineering IRASE 11, 3; 10.1556/1848.2020.00151
Clicking on the Bluetooth slider will start exploring and connecting to nearby Muse devices. After connection, you will receive information on signal quality and battery status.
Communication must be configured to provide LabVIEW with processable data (Fig. 8).
Configure an OSC stream [own figure]
Citation: International Review of Applied Sciences and Engineering IRASE 11, 3; 10.1556/1848.2020.00151
During configuration, you need to enter your computer's own IP address through any port, in this case 7,000, and then choose which data to provide.
4.2 Implementing Muse EEG in a LabVIEW development environment
Implementing Muse EEG in the LabVIEW Developer Environment by completing the OSC data stream, LabVIEW performs post-filtering of data, graphs and graphs raw and calculated values, and saves measurement and configuration parameters in a .csv and .xml file extension.
Read OSC stream
The OSC stream is transmitted through a UDP protocol to a processing program like the TCP/IP network protocol, except that the sending party does not wait for confirmation from the client whether it has received the data packet. This method provides faster communication than TCP (Transmission Control Protocol) but may result in data loss for a less resource intensive client. For this reason, the client, the processing program, must run on a powerful machine to minimize the time difference between iterations and to avoid packet loss. TCP packet waits for acknowledgment sent by receiver, so this wait may overload transmission buffers.
A feature of an OSC data stream is that it sends data in packets, with different addresses and arguments in the packets. Muse Direct sends the data according to the following encoding:
/notch_filtered_eeg: raw EEG from which 50 Hz network noise is filtered
/acc: accelerometer data
/gyro: gyroscope data
LabVIEW can read these packages using an installable directory by setting the appropriate parameters and addresses. Inbound data is merged into a waveform, transformed from a quasi-digital signal to an analog signal [14].
Fig. 9 shows that the program first connects to the appropriate UDP port, but if no data is exchanged for 3 seconds, the program will stop (and generates exception) due to timeout. Once the connection is established, the data stream starts according to the OSC address table, which provides EEG, gyroscope, and accelerometer data from the device.
EEG signal filtering by Butterworth filter
OSC streaming reader and visualization program detail [own figure]
Citation: International Review of Applied Sciences and Engineering IRASE 11, 3; 10.1556/1848.2020.00151
The implementation of the filter in the program was inevitable as only the 50 [Hz] utility network components are physically filtered by the device from the input data, so the higher frequency interfering components are still present in the signal. To filter out low frequency component disturbances, such as blinking, muscle twitching, accidental movement, the parameters of the filtering algorithm were set to pass the components between 0.5 [Hz] and 35 [Hz]. With this EEG data bandpass filter, higher frequency components can also be filtered out.
Based on the results of some research [17], the optimal number for filtering the EEG is 2, which can be used to safely filter out interfering signals without distorting the signal. 0.5 Hz was selected because our measurements were done in a quiet and calm laboratory environment (Fig. 10).
Bandwidth Butterworth Filter Program Detail [own figure]
Citation: International Review of Applied Sciences and Engineering IRASE 11, 3; 10.1556/1848.2020.00151
The input to the filter is the array read from the OSC stream, and the output is the filtered signal that can then be processed.
Calculation of spectral analysis for EEG channels
The cleaned data set is then converted from a time domain to a frequency domain using an FFT algorithm and then plotted using a graph to measure the average spectrum. According to our fitness/fatigue evaluation protocol we developed, a measurement lasts in quiet and calm environment for 6 minutes and is divided into 3 and 3 minutes. According to our investigation, it was suitable to provide information on the trend for the first and second half of the measurement [15] (Fig. 11).
FFT algorithm program snippet [own figure]
Citation: International Review of Applied Sciences and Engineering IRASE 11, 3; 10.1556/1848.2020.00151
The program calculates FFT for a total of 10 arrays, 4 channels for the first measurement, 4 channels for the second half of the measurement, and first and second half averages.
During the FFT calculation, the program also calculates the power of each frequency range, i.e. the area under the FFT graph. The area under the curve is used to determine the asymmetry between the right and left hemispheres. Obviously, the hemisphere that has higher average power at the time of measurement is the more dominant hemisphere.
Fig. 12 shows the frequency bands, which are marked with a separate color (Figs. 13 and 14).
FFT graph for EEG signal [own figure]
Citation: International Review of Applied Sciences and Engineering IRASE 11, 3; 10.1556/1848.2020.00151
Right/left hemisphere asymmetry calculation part [own figure]
Citation: International Review of Applied Sciences and Engineering IRASE 11, 3; 10.1556/1848.2020.00151
Visualization of cerebral hemispheres asymmetry for the first and second half of the measurement [own figure]
Citation: International Review of Applied Sciences and Engineering IRASE 11, 3; 10.1556/1848.2020.00151
Saving measurement data
At the end of the measurement, the measurement data is saved with headers and units in the .csv file extension. This format is selected because this extension can be interpreted by several popular data processing and display programs, so we have more options for re-measuring.
The filename composition applies the following system: “Muse_measurement_% Y-% m-% d. % H:% M:% S% 3u.csv”. Since the date and time are saved, plus the seconds to 3 decimal places, it is possible to accurately differentiate and identify each measurement file.
After the measurement is started, the program creates an XML database in which the configuration settings are stored. The most important of these are the serial numbers of the devices, the name of the person performing the measurements, the resolutions of the devices, the sampling frequency, the path of the measurement file created after the measurement, the number of channels and the minimum and maximum values. This database can be used to validate the output of the measurement in terms of configuration parameters (Fig. 15).
Configuration XML database [own figure]
Citation: International Review of Applied Sciences and Engineering IRASE 11, 3; 10.1556/1848.2020.00151
5 Test measurement
Measurement and testing of EEG signals was carried out with the assistance of a team of medical experts, including helicopter pilots. Measuring EEG signals first and foremost requires proper placement of the Muse, as this is the only way the device can provide valuable, processable signals. The device's data sheet states that it works with a dry electrode system, but for a perfect contact, it is advisable to moisten the surface of the electrode with a slightly damp or blue spirit cotton pad (Figs. 16 and 17).
Professional medical assistance: Placing Muse on the test subject. The measurement was done with ECG measurement.
Citation: International Review of Applied Sciences and Engineering IRASE 11, 3; 10.1556/1848.2020.00151
ECG measurements was done in calm and quiet laboratory to minimize external disturbance may has effect on patient's mind state
Citation: International Review of Applied Sciences and Engineering IRASE 11, 3; 10.1556/1848.2020.00151
Before starting the measurement, the subject should be asked to close his eyes and move as little as possible, as these can cause interfering signals in the collected signals, which could greatly distort the measurement output.
The installation was in all cases done by an expert person, continuously monitoring the data collected by the device during the measurement. During the measurement the experts found everything was fine, and any fine-tuning was carried out on site (Figs. 18 and 19).
Screenshot of test measurement with gyroscope on bottom graph
Citation: International Review of Applied Sciences and Engineering IRASE 11, 3; 10.1556/1848.2020.00151
Test Measurement Screenshot - EEG waveform
Citation: International Review of Applied Sciences and Engineering IRASE 11, 3; 10.1556/1848.2020.00151
During the test measurement, the gyroscope and the accelerometer proved to be useful as these sensors can be used to detect whether motion was caused by a disturbance signal (some disturbing signals such as motion, jerking) or other external sources, interference (Figs. 20 and 21).
Test measurement screenshot and EEG waveforms
Citation: International Review of Applied Sciences and Engineering IRASE 11, 3; 10.1556/1848.2020.00151
Test Measurement - EEG and accelerometer waveforms
Citation: International Review of Applied Sciences and Engineering IRASE 11, 3; 10.1556/1848.2020.00151
Measurements are tracked using graphs that transmit measured data in real-time to provide an accurate view of the measurement process. You can display more than one graph on the graph at a time, so you can look for correlations between the waveforms of some measured values (Fig. 22).
EEG figures for a 24-year-old male in the report
Citation: International Review of Applied Sciences and Engineering IRASE 11, 3; 10.1556/1848.2020.00151
The report calculates the data computed by all programs, the power, relative and absolute distributions of each frequency range, and the percentage distribution of the right and left hemispheres, all of which appear on the expert panel for the first and second half of the measurement (Figs. 23–25).
EEG spectrum of a 24-year-old male – first half measurement
Citation: International Review of Applied Sciences and Engineering IRASE 11, 3; 10.1556/1848.2020.00151
EEG spectrum of a 24-year-old male – second half of measurement
Citation: International Review of Applied Sciences and Engineering IRASE 11, 3; 10.1556/1848.2020.00151
Bar graph of 24-year-old male cerebral hemispheric rate
Citation: International Review of Applied Sciences and Engineering IRASE 11, 3; 10.1556/1848.2020.00151
For the sake of clarity, the first and second halves of the measurement were distinguished in the hemispheric diagram by two colors, so that they can be clearly distinguished. And displaying a chart helps you keep track of the trend.
6 Conclusions
In this paper, we presented a technical method and evaluation logic to measure physical and mental fitness/fatigue. Commercially available, Muse EEG device was used for EEG measurement and LabView environment was used to develop evaluation logic. Butterworth filter was applied to remove unwanted noise from signal, then FFT was applied to convert time domain to frequency domain. The signal was classified according to common frequency bandwidths of brain signals: Delta, Theta, Alpha, Beta and Gamma. The area covered by FFT signals was also measured to decide the dominant left or right hemispheres.
In the end, the system is working as expected, all required functions have been successfully implemented. The system is stable, even for long-term continuous measurements. This paper is part of a larger research project, further results will be published in [18, 19].
The system offers a number of enhancements that can be used to extract even more data from the measurement data set:
Application of Deep/machine learning algorithms to classify multiple measurements
Creating a correlation matrix for the measurement parameters
Prediction of mental illness
7 Acknowledgments
The authors would like to thank every medical and technical expert of PHARMAFLIGHT International Science and Service Center PLC for their continuous support during the research activity. The origin of the research topic was initiated by Mr. Gyula Győri.
References
- [1]↑
A. Asif, M. Majid, and S. M. Anwar, “Human stress classification using EEG signals in response to music tracks,” Comput. Biol. Med., vol. 107, pp. 182–196, 2019.
- [2]↑
M. Seeck, L. Koessler, T. Bast, F. Leijten, C. Michel, and C. Baumgartner, “The standardized EEG electrode array of the IFCN,” Clin. Neurophysiol., vol. 128, pp. 2070–2077, 2017.
- [3]↑
T. Ahmad and V. Ramachandran, “Hyperspherical manifold for EEG signals of Epileptic Seizures,” J. Appl. Math., vol. 2012, p. 22, 2012. https://doi.org/10.1155/2012/926358.
- [4]↑
J. Ahmed, Brain Machine Interface Using EEG Sci-fi to Reality Neural Interface Engineering Brain Machine Interface Using EEG 1 Brain Machine Interface Using EEG, 2016.
- [5]↑
M. A. Lopez-Gordo, D. Sanchez-Morillo, and F. P. Valle, “Dry EEG electrodes,” Sensors, vol. 14, pp. 12847–12870, 2014.
- [6]↑
I Szirmai and A. Kamondi, “Kognitív zavarok vizsgálata EEG-vel,” (Hungarian) Ideggyógyászati Szemle, vol. 64, no. 01–02 2011, pp. 14–23, 2016.
- [7]↑
M. X. Cohen, “Where does EEG come from?,” Trends Neurosci., vol. 40, no. 4, 2017. https://doi.org/10.1016/j.tins.2017.02.004.
- [11]↑
I. Inc, Muse, InteraXon Inc., [Online]. Available: http://www.choosemuse.com/. [Hozzáférés dátuma: 6. október 2018.].
- [12]↑
E. Ratti, S. Waninger, C. Berka, G. Ruffini, and A Verma, “Comparison of medical and consumer wireless EEG systems for use in clinical trials,” Front. Hum. Neurosci., vol. 11, p. 398, 2017.
- [13]↑
L Ciechanowski, M Magnuski, and P Gloor, Muse Headband: Measuring Tool or a Collaborative Gadget?, Springer International Publishing, 2018.
- [14]↑
N. Instruments, Open Sound Control Library, National Instruments, 2019, [Online]. Available: http://sine.ni.com/nips/cds/view/p/lang/hu/nid/211428[Hozzáférés dátuma: 19 October, 2019].
- [15]↑
N. Instruments, Ni Biomedical Toolkit, National Instruments, [Online]. Available: http://sine.ni.com/nips/cds/view/p/lang/hu/nid/211023[Hozzáférés dátuma: 18 October, 2018].
- [16]↑
M. R. Lakshmi, T. V. Prasad, and V. C. Prakash, “Survey on EEG signal processing methods,” IJARCSSE, Int. J. Adv. Res. Comput. Sci. Software Eng., vol. 4, no. 1, 2014, ISSN: 2277 128X.
- [17]↑
H. Higashi, and T. Tanaka, “Simultaneous design of FIR filter banks and spatial patterns for EEG signal classification,” IEEE Trans. Biomed. Eng., vol. 60, no. 4, pp. 1100–1110, April 2013. https://doi.org/10.1109/TBME.2012.2215960.
- [18]
D. Torma, Gy. Győri, and K. Sarvajcz, “Processing and analyzing psychophysiological data using NI DIAdem software,” Int. Rev. Appl. Sci. Eng., IRASE, vol. 11, no. 2, pp. 188–196, 2020. https://doi.org/10.1556/1848.2020.20128.
- [19]
D. Dezso, G. Gyori, and H. Giza Husi, “Development of ECG measurement and processing system in labview development environment,” IRASE, Under Publication.