Remote Control of a Television Set via Thought Patterns

Dr. William Hunt, Project AdvisorJosh RosnerDrew PetryGary Grimm

George Melcer

Thursday, April 29, 2010

Complete System Diagram

DefinitionsEEG – Electroencephalography EAS – EmoState Analysis SoftwareSPB – Synergy Prototyping Board URC – Universal Remote Control

The MindLink System

• Manipulates television functions via user thought patterns

• Provides independence to handicapped and disabled persons

• Development costs were $62k• Per unit cost is approximately $460

Design Objectives

Objective ResultMinimize System Cost Components Cost $430

Intuitive Interface Train with Emotiv Control Panel

Manipulate Simple Commands Set of Five Commands

Real-Time Operation No Noticeable Delay

Utilize Emotiv API Invoked by EAS

Interface with Modified OTS URC 10 Wire Custom Interface

EEG Headset

Illustration

Input Propagation

Emotiv Headset

USB DongleHost PC Device Driver

EEG Signals From User

14 Sensors

2.4 GHz Wireless

Headset Issues

• Training– Training multiple actions– Cannot determine if thought was consistent– Cannot edit profile database

• Physical Headset– Sensor corrosion/replacement

Headset Acceptance Testing

0

2

4

6

8

10

12

Drop Push Left Right

Command

Tria

ls

Failures

Sucesses

Actions Tested Independently

Headset Acceptance Testing

0

2

4

6

8

10

12

Drop Push Left Right

Command

Tria

ls

Failures

Sucesses

Actions Tested Together

Host PC: EmoState Analysis Software

C++ Control Software Process

C++ Control Software Logic

EAS – Design Issues• Invoking and communicating with Emotiv API

– Included EDK.h and linked with EDK.dll– Used EE_EngineConnect for communication with

headset– Used EE_EngineRemoteConnect for communication

with EmoComposer (debugging)

• Sending commands to Synergy board– Used FTDI driver to enable serial connection over USB– Created a HANDLE and used CreateFile and WriteFile API to initialize and send commands to virtual COM port

EAS – Usability Issues• Ensure real-time reaction processing

– Processed EmoEvents soon after they occurred in order to keep the event queue clear

• Avoid false positives while improving usability– Only took action if an event had a high enough power

(confidence) level reported– Compared event timestamps to differentiate if repetition

of a command was intended by user

EAS – Acceptance Testing with EmoComposer

Synergy Prototyping Board &

Universal Remote Control

Synergy Prototyping Board

SPB Layout

SPB Implementation

Synergy Prototyping Board Firmware

SPB – Problems

• Problem: MOSFETs did not interface with URC as expected

• Solution– Utilized bi-lateral switches instead of MOSFETs in

order to short two pins instead of pulling them high or low.

• Problem: Active-Low Demux• Solution

– Directly addressed the bi-lateral switches from GPIO’s.

URC – Problems

• Problem: Soldering joints are weak on URC

• Solution: Isolate joints with Hot Glue and carefully manage wires with heat shrink.

URC – Acceptance Testing with Short-Wire Method

URC – Acceptance Testing with SPB

System Integration Issues

• EAS to SPB Serial Interface– Standard Packet Format

• URC Volume Commands did not react as expected

• Flooding SPB/URC with commands caused random behavior

System Acceptance Testing (Emo-Composer)

Demonstration Video

Budget & Cost AnalysisMindLink Venture

Future Work

• Optimize System Design– Simpler SPB– Develop In-House URC– Built-in Low Power Embedded PC– Complete Embedded Solution

• Optimize Resource Acquisition– Licensing with Emotiv– Volume Pricing with Vendors

• Develop Service Contract Platform– Initial Installation / Monthly Support & Tuning

• Register MLS with Insurance Companies– Reduce Costs to handicapped customers

Questions?