1. Program Studi Teknik Fisika Fakultas Teknologi Industi Institut Teknologi Bandung Laboratorium Instrumentasi Medik Quantitative Brain Signal for Neurofeedback and Human Computer Interfaces Dr. Suprijanto, S.T, M.T. The 4th International Conference on Instrumentation, Control and Automation 2016 Departement of Engineering Physics Instrumentation and Control Research Group Faculty of Industrial Technology Institut Teknologi Bandung 30 August 2016 1

2. Human Brain Main part of Human brain divided by : cerebrum, cerebellum, basal ganglia, spainal cord Cerebral Cortex has 4 lobes : frontal, parietal, occipital and temporal  associated with different function such as visual, auditory, cognitive, motoric 2 Visual cortex

3. Origin Brain Electrical Signal large numbers of neurons Unit Neuron Any stimulus respones Transmit information from/to Other neurons Action potential 3 Cerebral Cortex

4. Brain Electrical Signal Acquisition Brain signal is “generated” by action potential from large number of neurons Invasive : implant electrodes under the scalp (cortical surface) Non-Invasive : surface electrodes along the scalp Electro-corticography (ECoG) Electroenchephalogram (EEG) Painless, high temporal resolution, Portable and inexpensive Required surgery procedure 4

5. EEG signal Recording Clinical International Standard EEG recording 10%-20% Scalp diameter 21 electrodes Standard placements of electrodes on the human scalp: A, auricle; C, central; F, frontal; Fp, frontal pole; O, occipital; P, parietal; T, temporal. Research EEG recording 10%-10% Scalp diameter 70 electrodes 5

6. Pattern of EEG signal : Synchronized and de-Synchronized Amplitude and frequency EEG signal influence by : -the timing of their activity action potential (AP) of each neuron and -each AP from neuron is suporposition each others  to generate EEG signal Synchronized neural activity produces larger amplitude signals and lower frequency De-synchronized neural activity produces smaller amplitude signals and higher frequency 6 Synchronized De-Synchronized Sum of EEG AP of each neuron

7. Challenge of EEG Signal Processing EEG Bio-Amplifer ref O1O1 Oz O2O2 PO 3 POz PO 4 EEG signal characteristic : Nonliniear dynamic  highly complex behaviour of neural ensembles Nonstationary  mental and emotion state Noise  altenations in electrode placement and enviromental Artifact  other electrical activity : skeletal muscle and eye movement 7

8. EEG Signal for Neurofeedback 8

9. Types of EEG Signal for Neurofeedback Rhythms (Brain Wave) Evoked Potential (EP) Event Related Potential (ERP) Steady State Evoked Potential (SSVEP) Alpha Wave Beta Wave Tetha Wave Delta Wave Gamma Wave 9 mu Wave

10. EEG Rhythms for Neurofeedback EEG rhythms correlate with patterns of behavior: such as level of attentiveness, sleeping, waking, seizures, coma. EEG Rhythms occur in distinct frequency ranges: 10 Beta : 14-20 Hz (activated cortex) Alpha: 8-13 Hz (quiet waking) Theta: 4-7 Hz (sleep stages) Delta: less than 4 Hz (deep sleep)

11. EEG Rhythms for Neurofeedback 11

12. EEG Rhythms for Neurofeedback Signal Processing of EEG Rhythms Time Domain Bandpass Digital Filter Frequency Domain Spectral Estimation with Fourier Transform   time frequency 12 Raw EEG

13. ERP for Neurofeedback Signal Processing of Evoked Respone Potential Visual Stimulation EEG Amplifer n-times Signal Averaging n-times Trials 13

14. SSVEP for Neurofeedback Steady State Visual Evoked Potential (SSVEP) arise in EEG signal if Fast repetitive external visual stimulation (>4Hz) give to subject 14

15. SSVEP for Neurofeedback Ilustration : SSVEP EEG signal in Frequency Domain power spectral 20 Hz power spectral 15

16. Quantitative EEG for Neurofeedback 16

17. QEEG for Neurofeedback Research in ITB Research in Field Neurofeedback in Medical Instrumentation Laboratory, Departement Engineering physics, Instrumentation and Control Research Group, ITB started from 2007. Research Coloboration with Physiology, Animal Development & Biomedical Science Research Group, ITB Biomedical Engineering Research Group, ITB Neurology Departement, Hasan Sadikin Hospital, Bandung, Indonesia Selected Research Topics (undergraduate and graduate students research) discussed: QEEG for ERP Identification due to Primary Color Stimulation (2008) QEEG for Brainwaves Response Related to Brain Cognitive Function Due to Blue Light (2012) QEEG for Human Computer Interface based mu-Wave (2013) QEEG for Human Computer Interface based SSVEP (2014) EEG and EMG for Bio-feedback Hand Gripping Repetitive Motion (2016) 17

18. QEEG for ERP Identification due to Primary Color Stimulation 18

19. QEEG for ERP Identification due to Primary Color Stimulation ERP Response due to Red Color 19

20. QEEG for ERP Identification due to Primary Color Stimulation 20

21. QEEG for Brainwaves Response Related to Brain Cognitive Function Due to Blue Light Non Visual Visual Circadian Cycle Biological Clock Occipital suprachiasmatic nucleus Type of photoreceptor : -cones and rods  visual biological effect -intrinsic photosensitive retinal ganglion cell (ipRGC) IpRGC regulates non visual biological effect (circadian cycle) and most sensitive to blue light =480 nm 21 the aim of this study was to examine brain response, especially the cognitive part of the brain, triggered by non-visual effect of light for people work using artificial lighting (e.g. Fluerecent Lamp)

22. QEEG for Brainwaves Response Related to Brain Cognitive Function Due to Blue Light Fluerecent Lamp (TL) 6 x 40 Watt Fluerecent Lamp (TL) + Blue Light ( =480 nm) Emotiv EEG Ts = 128 Hz wireless 5 Volunteers (Male) 20-25 Yrs not color blind Electrodes Cognitive Area : F3,F4, FC5,FC6, P7,P6 Avg[P-  ] 22

23. QEEG for Brainwaves Response Related to Brain Cognitive Function Due to Blue Light Power of  TL - Power of  TL+BL EEG segmentation @ 1 minute Frontal Lobes F3 Frontal Lobes F4 EEG segmentation @ 1 minute Power of  TL - Power of  TL+BL EEG recording each 7 minutes for each experiment scenario Average Power of  Wave each 1 minutes Frontal  Cognitive Lobes   _P_  < 0 BL   Avg[P-  TL+BL ] Attention increase  _P_  : Avg [P-  TL ] - Avg [P-  TL+BL ] Average from 5 volunteers 23

24. QEEG for Human Computer Interface based mu-Wave Design of Brain-Computer Interface Platform for Commanding Electrical Wheelchair(EWC) Simulator Movement Motoric Stimulation The µ-Rhythm is a rhythmical brainwave that relates to motor and premotor activities. In this research, actual movements are used rather than imaginary movements as stimuli. 24

25. QEEG for Human Computer Interface based mu-Wave 25 Signal processing

26. QEEG for Human Computer Interface based mu-Wave PSD for Mu Wave No Movement Frequency (Hz) Power PSD for Mu Wave Movement Right Frequency (Hz) Power No Movement = stop Hand Movement for induced mu Brain Wave Right Hand =EWC Right Left Hand =EWC Left See Movie Mu Wave 26

27. QEEG for Human Computer Interface based SSVEP SSVEP paradigm is applied in the BCI System to obtain basic movement commands based on flickering visual stimuli Visual Stimulation A.Forward : 8,57 Hz B.Stop : 10,00 Hz C.Right : 12,00 Hz D.Left : 15,00 Hz V O1O1 Oz O2O2 PO 3 POz PO 4 Before Modulation 8,57 Hz Second V After Modulation 8,57 Hz 27

28. QEEG for Human Computer Interface based SSVEP Feature Extraction based on Power Spectral Channel 1 : O1 Channel 2 : OZ Channel 3 : O2 Channel 4 : PO3 Channel 5 : POZ Channel 6 : PO4 Before Modulation 8,57 Hz After Modulation 8,57 Hz 28

29. QEEG for Human Computer Interface based SSVEP Feature Extraction based on Filter Bank Bank Filter Specification Bandwidth 1 : 7,00 - 9,00 Hz Bandwidth 2 : 9,00 - 11,00 Hz Bandwidth 3 : 11,00 - 13,00 Hz Bandwidth 4 : 14,00 - 16,00 Hz Power 29

30. QEEG for Human Computer Interface based SSVEP Feature Classification : Non Training Methods (Heuristic) Accuration Results from 8 Volunteers and 5 session experiments Non Training 30

31. EEG and EMG for Bio-feedback Hand Gripping Repetitive Motion Biofeedback is aimed to monitor the exercise in the rehabilitation program in case hand gripping repetitive motion Information from biofeeback is used for determined patient recovery rating and also for signal control Robotic Rehabilitation Biofeedback using signal from EEG and EMG EEG EMG Attention Fatigue ?? Relationship ?? 31

32. EEG and EMG for Bio-feedback Hand Gripping Repetitive Motion Five Healthy Volunteers Experiment Procedure : 1 min (relax), 5 min (exercise-section 1), 30 sec (Handgrip strength), 5 min (exercise-section 2), 1 min (relax) sesion 1 32

33. EEG and EMG for Bio-feedback Hand Gripping Repetitive Motion EMG signal EEG signal Baseline Contraction Relaxation RMS power EMG and EEG (beta wave) for each contraction and relaxation evaluated 33

34. EEG and EMG for Bio-feedback Hand Gripping Repetitive Motion RMS power of Beta Wave on Pariental (Pz) and Frontal (Fz)  Cognitive Area  Tend to decrease  indicated decreasing attention RMS power of EMG on Flexor Digitorum Superficialis  Tend to decrease  indicated fatigue on muscles 34

35. Remarks Quantitative EEG Signal for Neurofeedback has not limited for medical application but currently growing interest, such as for neuromarketing, intertaiment, human “brain” computer interface (BCI)  Requiring multi-dissipline researchers Wearable Device for capturing EEG signals become invented. Applications will be required plug and play – EEG device for several applications on near future 35

36. 36 Remarks Future BCI application is required robust translating EEG for various stimulation and various user state mental condition. Robust equipment is needed that tolerates movement and individual fluctuations of brain states. New machine learning, signal processing, and artifact removal approaches will be needed for brain state decoding and translating the brain signals into output signals in real-time.

37. Acknowledgement Dr. Augie W. Widyotriatmo, Instrumentation Research Group, ITB Dr. Lulu L. Fitri, Physiology, Animal Development & Biomedical Science Research Group, ITB and other colleagues involves on this research 37 Former Students

38. 38 Publications (2007-2015)

39. 39 Publications (2007-2015)

40. 40 Publications (2007-2015)

41. 41 Publications (2007-2015)