JAI NARAIN COLLEGE OF TECHNOLOGY BHOPAL JAI NARAIN COLLEGE OF TECHNOLOGY BHOPAL “BLUE EYE TECHNOLOGY” “BLUE EYE TECHNOLOGY” SUBMITTED TO SUBMITTED BY SUBMITTED TO SUBMITTED BY Mr. Gopal Patidar Amit Kumar(07) Mr. Gopal Patidar Amit Kumar(07) (Lecturer) (Lecturer)

Blue eye technology ( blue eyes – Human operator monitoring system )

Agenda Motivation Motivation What is BlueEye technology ? What is BlueEye technology ? What is BlueEyes ? What is BlueEyes ? System designing System designing System overview System overview DAU DAU CSU CSU Data security Data security IBM research IBM research Conclusion Conclusion Motivation What is BlueEyes ? Physiological foundations System design System implementation Future improvements Project work summary System demonstration

Motivation Human error - a frequent reason of catastrophes and ecological disasters Human error - a frequent reason of catastrophes and ecological disasters Motivation What is BlueEyes ? Physiological foundations System design System implementation Future improvements Project work summary System demonstration -temporary indisposition -weariness-oversight

What is blue eye technology ? The BLUE EYES technology aims at creating computational machines that have perceptual and sensory ability like those of human beings. The BLUE EYES technology aims at creating computational machines that have perceptual and sensory ability like those of human beings.

How can we make computers “see” and “feel” ? Blue eyes uses sensing technology to identify a user’s actions and to extract key information. Blue eyes uses sensing technology to identify a user’s actions and to extract key information. Information is then analyzed to determine the user’s physical,emotional,or informational state. Information is then analyzed to determine the user’s physical,emotional,or informational state.

What is BlueEyes ? The complex solution for human-operator monitoring: –V–V–V–Visual attention monitoring –P–P–P–Physiological condition –O–O–O–Operator’s position detection –W–W–W–Wireless data acquisition using bluetooth –R–R–R–Real time user defined alarm triggering –R–R–R–Recorded data playback Motivation What is BlueEyes ? Physiological foundations System design System implementation Future improvements Project work summary System demonstration

Doesn’t predict nor interfere with operator’s thoughts Doesn’t predict nor interfere with operator’s thoughts Cannot force directly the operator to work Cannot force directly the operator to work What is BlueEyes not?

Applications of BlueEyes Generic control rooms Generic control rooms ( System can be applied in every working environment requiring permanent operator’s attention) requiring permanent operator’s attention) –Power station –Captain bridge –Flight control centers –Operating theatres – anesthesiologists Motivation What is BlueEyes ? Physiological foundations System design System implementation Future improvements Project work summary System demonstration

BlueEyes - benefits Prevention from dangerous incidents Prevention from dangerous incidents Minimization of Minimization of –ecological consequences –financial loss –a threat to a human life The reconstruction of the course of operator’s work The reconstruction of the course of operator’s work Motivation What is BlueEyes ? Physiological foundations System design System implementation Future improvements Project work summary System demonstration

Implementation and engineering considerations -THREE groups of users OPERATOR -a person whose physiological parameters are supervised OPERATOR -a person whose physiological parameters are supervised - authorization in the system - authorization in the system - receiving alerts - receiving alerts SUPERVISOR – a person responsible for analyzing operator’s SUPERVISOR – a person responsible for analyzing operator’s condition and performance. condition and performance. SYSTEM ADMINISTRATORS- maintains the system SYSTEM ADMINISTRATORS- maintains the system deliver tools for adding deliver tools for adding - new operator’s to the database - new operator’s to the database - defining alarm conditions - defining alarm conditions - configuring logging tools - configuring logging tools - creating new analyzer modules - creating new analyzer modules

Designing A personal area network for linking all the operators and the supervising system A personal area network for linking all the operators and the supervising system Two major units Two major units - DAU (data acquisition unit ) - CSU (central system unit )

System overview Central System UnitData Acquisition Unit Atmel 89C52 microcontroller Jazz Multisensor Connection Manager Data Analysis Visualization Module Bluetooth Bluetooth technology provides means for creating a Personal Area Network linking the operators and the central system. Data Logger Motivation What is BlueEyes ? Physiological foundations System design System implementation Future improvements Project work summary System demonstration Objectives System overview DAU CSU

The eye movement sensor Off-shelf eye movement sensor – Jazz multisensor Off-shelf eye movement sensor – Jazz multisensor Supplies raw digital data regarding Supplies raw digital data regarding - Eye position - Level of blood oxygenation - Acceleration along horizontal and vertical axes - Ambient light intensity Motivation What is BlueEyes ? Physiological foundations System design System implementation Future improvements Project work summary System demonstration

Jazz multisensor Jazz Multisensor

Jazz

DAU The DAU consists of the following components ATMEL 8952 microcontroller ATMEL 8952 microcontroller BLUE TOOTH MODULE – supports synchronous voice data transmission BLUE TOOTH MODULE – supports synchronous voice data transmission PCM CODEC –used to transmit operator’s voice and central system sound feedback PCM CODEC –used to transmit operator’s voice and central system sound feedback UART –communication between bluetooth module and microcontroller ( bps) UART –communication between bluetooth module and microcontroller ( bps) MAX232 –level shifter MAX232 –level shifter ALPHAUNUMERIC LCD display ALPHAUNUMERIC LCD display LED indicators LED indicators ID CARD interface ID CARD interface

BlueEyes Data Acquisition Unit ID card DAU - components MC PCM codec ID card interface Simple keyboard Jazz Multisensor Beeper LCD display LED indicators Atmel 89C52 microcontroller UART MAX232 UART Bluetooth module microphone earphone

Microcontroller & blue tooth module

EEPROMS & the PCB

Implementation - DAU Bluetooth module PCM codec board Batteries and voltage level monitor Microcontroller board Motivation What is BlueEyes ? Physiological foundations System design System implementation Future improvements Project work summary System demonstration DAU CSU

DAU - features Lightweight Lightweight Runs on batteries - low power consumption Runs on batteries - low power consumption Easy to use - does not disturb the operator working Easy to use - does not disturb the operator working ID cards for operator authorization ID cards for operator authorization Voice transmission using hardware PCM codec Voice transmission using hardware PCM codec Data Acquisition Unit Atmel 89C52 microcontroller Jazz Multisensor Bluetooth Motivation What is BlueEyes ? Physiological foundations System design System implementation Future improvements Project work summary System demonstration Objectives System overview DAU CSU

CSU - features Central System Unit Connection Manager Data Analysis Visualization Module Bluetooth Connection management Connection management Data processing Data processing Visualization Visualization Data recording Data recording Access verification Access verification System maintenance System maintenance Connection Manager Motivation What is BlueEyes ? Physiological foundations System design System implementation Future improvements Project work summary System demonstration Objectives System overview DAU CSU Data Logger

CSU – components CONNECTION MANAGER – main task to perform low-level blue tooth communication CONNECTION MANAGER – main task to perform low-level blue tooth communication DATA ANALYSIS MODULE – performs the analysis of the raw sensor data in order to obtain information about operator’s physiological condition DATA ANALYSIS MODULE – performs the analysis of the raw sensor data in order to obtain information about operator’s physiological condition DATA LOGGER MODULE – provides support for storing the monitored data. DATA LOGGER MODULE – provides support for storing the monitored data. VISULAIZATION MODULE – provides user interface for the supervisors VISULAIZATION MODULE – provides user interface for the supervisors

CSU - Visualization Module Motivation What is BlueEyes ? Physiological foundations System design System implementation Future improvements Project work summary System demonstration Objectives System overview DAU CSU Raw and processed data visualization using: Raw and processed data visualization using: audio and video data playback audio and video data playback –VU-meters –pie-charts –time series

Data security Only registered mobile devices can connect to the system Only registered mobile devices can connect to the system Bluetooth connection authentication Bluetooth connection authentication Bluetooth connection encryption Bluetooth connection encryption Access rights restrictions Access rights restrictions Personal and physiological data encryption Personal and physiological data encryption Motivation What is BlueEyes ? Physiological foundations System design System implementation Future improvements Project work summary System demonstration Objectives System overview DAU CSU

IBM research BLUE EYE – EMOTIONAL MOUSE BLUE EYE – EMOTIONAL MOUSE sensors in the mouse,sense the physiological attributes which are correlated to emotions using correlation model sensors in the mouse,sense the physiological attributes which are correlated to emotions using correlation model -by simply touching the mouse,the computer will be able to determine a person’s emotional state. BLUE EYE enabled TELEVISION – could become active when the user makes an eye contact. BLUE EYE enabled TELEVISION – could become active when the user makes an eye contact.

Conclusion In the near future,ordinary household devices- such as television, refrigerators,ovens may be able to do their jobs when we look at them and speak to them. In the near future,ordinary household devices- such as television, refrigerators,ovens may be able to do their jobs when we look at them and speak to them. Future applications of blue eye technology is limitless Future applications of blue eye technology is limitless

References

Thank you BlueEyes Human-Operator Monitoring System Motivation What is BlueEyes ? Physiological foundations System design System implementation System demonstration Future improvements Project work summary