BLUE EYES TECHNOLOGY
CONTENTS NEED SYSTEM OVERVIEW HARDWARE SOFTWARE APPLICATIONS EMOTION MOUSE MAGIC POINTING SUITOR
NEED Technology Pilots, people who multi-task. Assistance of physically challenged Easier computer interaction Luxury & Comfort Effortless Computer interaction
BLUE EYES TECHNOLOGY The BLUE EYES technology aims at creating computational machines that have perceptual and sensory ability like those of human beings.
The term BLUE stands for Bluetooth, which enables reliable wireless communication. The term EYES is used because the eye movements that are predominantly analyzed here enables us to obtain a lot of interesting and important information.
BLUE EYES Operations can be executed more efficiently if the users emotional conditions are considered. Thus it is necessary to measure the level of the users conscious brain involvement which can be indirectly analyzed using eye motility. As BLUE EYES Technology deals predominantly with eye movements it is used to monitor and record the operators conscious brain involvement as well as his Physiological condition.
BLUE EYES uses sensing technology to identify a users actions and to extract key information. The information is then analyzed by the machine to determine what a user wants, where he is looking at, and even realize his physical, emotional or informational state
For example, in future a BLUE EYES-enabled television could become active when the user makes eye contact, at which point the user could then tell the television to "turn on".
PARTS OF BLUE EYES The major parts in the BLUE EYES system are Data Acquisition Unit Central System Unit. Data Acquisition Unit maintains Bluetooth connections. gets information from the sensor and sends it over the wireless connection delivers the alarm messages sent from the Central System Unit to the operator and handles personalized ID cards. Central System Unit maintains the other side of the Bluetooth connection buffers incoming sensor data performs on-line data analysis records the conclusions for further exploration provides visualization interface.
DATA ACQUISITION UNIT Data Acquisition Unit is the mobile part of the BLUE EYES system. Its main task is to fetch the physiological data from the sensor and send it to the central system to be processed. It must manage wireless Bluetooth connections The Data Acquisition Unit comprises several hardware modules Atmel 89C52 microcontroller - system core Bluetooth module (based on ROK101008) HD44780 - small LCD display 24C16 - I2C EEPROM (on a removable ID card) MC145483 13bit PCM codec Jazz Multisensor interface beeper and LED indicators 6 AA batteries and voltage level monitor
CENTRAL SYSTEM UNIT Central System Unit hardware contains a Bluetooth module (based on ROK101008) and a PCM codec for voice data transmission. The module is interfaced to a PC using a parallel, serial and USB cable. The audio data is accessible through standard minijack sockets.
SOFTWARESYSTEM CORE CONNECTION MANAGER VISUALISATION MODULE
DATA ANALYSIS MODULE
SACCADE DETECTOR PULSE RATE ANALYSERS CUSTOM ANALYSERS
SOFTWARE For instant reaction on the operators' condition change the software must perform real time buffering of the incoming data, real-time physiological data analysis and alarm triggering. The BLUE EYES software comprises several functional modules System core facilitates the data flow between different system modules (e.g. transfers raw data from the Connection Manager to data analyzers etc.) Connection Manager is responsible for managing the wireless communication between the mobile Data Acquisition Unit and the central system. It handles: communication with the CSU hardware searching for new devices in the covered range establishing Bluetooth connections and connection authentication incoming data buffering sending alerts
Data Analysis module performs the analysis of the raw sensor data in order to obtain information about the operators physiological condition. This module extracts different types of information.
The most important analyzers are Saccade detector - monitors eye movements in order to determine the level of operator's visual attention Pulse rate analyzer - uses blood oxygenation signal to compute operator's pulse rate Custom analyzers - recognize other behaviors than those which are built-in the system. Visualization module provides a user interface for the supervisors enabling them to watch each of the working operators physiological condition. All the incoming alarm messages are instantly signaled to the supervisor.
DISADVANTAGES Given the one-degree size of the fovea and the subconscious jittery motions that the eyes constantly produce, eye gaze is not precise enough. The eye, as one of our primary perceptual devices, has not evolved to be a control organ. Sometimes its movements are voluntarily controlled while at other times it is driven by external events. One has to be conscious of where one looks and how long one looks at an object. If one does not look at a target continuously for a set threshold (e.g., 200 ms), the target will not be successfully selected
EMOTION MOUSE Emotion Mouse is based on the six basic emotions - anger, fear, sadness, disgust, joy and surprise. Physiological measures - galvanic skin response (GSR), heart rate, skin temperature and general somatic activity (GSA) are used to distinguish various emotional states.
All four physiological variables make significant, non-redundant contributions to the functions that distinguish the six states. Also analysis indicate that these four physiological measures are sufficient to determine reliably a persons specific emotional state. The physiological factors are sensed using infrared pulse detector which can be placed inside the body of the mouse.
EMOTION MOUSEEMOTION MOUSE IMPLEMENTED ON A REAL MOUSE
MANUAL AND GAZE INPUT CASCADED (MAGIC) POINTING MAGIC pointing utilizes the eye gaze for computer input. Gaze tracking is an alternative and potentially superior pointing method for computer input. A large portion of the cursor movement is eliminated by warping the cursor to the eye gaze area, which encompasses the target. There are two different approaches in MAGIC pointing techniques Liberal approach - The cursor moves once the user decides to acquire the target he is looking at. Conservative approach - The cursor is warped to the gaze area only when a manual input device has been actuated.
SIMPLE USER INTERST TRACKER (SUITOR) SUITOR notices where the users eyes focus on the computer screen and deliver relevant information to a handheld device. The SUITOR can be more precise in determining the users topic of interest depending on how intimate to the user.
REFERENCES Shumin Zhai, Carlos Morimoto, Steven Ihde. Manual And Gaze Input Cascaded (MAGIC) Pointing; IBM Almaden Research Center Paul P. Maglio, Rob Barrett, Christopher S. Campbell, Ted Selker. SUITOR: An Attentive Information System; IBM Almaden Research Center http://www.palowireless.com/database/ericsson/rok101008_14610 6r1a