EMG-Based Human Machine Interfaceee.bradley.edu/projects/proj2018/emg/EMG_HMI_ConferencePresentation_20180428.pdfApril 28th, 2018 EMG-Based Human Machine Interface 8 Aditya Patel and

1April 28th, 2018EMG-Based Human Machine Interface

Aditya Patel and Jim Ramsay

EMG-Based Human Machine Interface


Advised by Drs. Yufeng Lu and In Soo Ahn

2EMG-Based Human Machine Interface


Conclusions

Introduction

Objectives

Gesture Detection

System

Design

Project Management

April 28th, 2018

Introduction



Conclusions

Introduction

Objectives

Gesture Detection

System

Design

Project Management

EMG Explained

• Electromyography (EMG)

o Measurement of electrical signals

produced by activity in the muscles

• Surface EMG (sEMG):

o A technique for acquiring EMG

signals by using electrodes placed

on the skin, directly above the

desired muscle

Figure 1: Myo armband and forearm

muscles [1]



Conclusions

Introduction

Objectives

Gesture Detection

System

Design

Project Management

EMG Explained

• Raw EMG signals have a peak-to-

peak voltage of up to 10 mV

• Typical frequency range is 10 Hz

to 500 Hz

• Signal strength and patterns vary

from person to person

Figure 2: Raw EMG data from eight sensors

while holding a fist



Conclusions

Introduction

Objectives

Gesture Detection

System

Design

Project Management

Applications of EMG

• Primarily for medical purposes

o Diagnosing and testing for muscle

and nerve injuries

• Control of prosthetic limbs

• Gesture control

o Drones/robots

o Computers

Figure 3: EMG test [2]



Conclusions

Introduction

Objectives

Gesture Detection

System

Design

Project Management

• Developed an EMG-based human

machine interface for a mobile robot

• Only 1 EMG sensor was used. Precise

sensor placement was required.

• Neural network was used to identify

predefined motions

• Careful calibration from the user was

needed.

Figure 4: EMG system setup from

previous year’s senior project [3]

Previous Work – EMG HMI (2016-2017)



Conclusions

Introduction

Objectives

Gesture Detection

System

Design

Project Management

April 28th, 2018

Objectives



Conclusions

Introduction

Objectives

Gesture Detection

System

Design

Project Management

Project Goals

This project aimed to design an EMG-based human

interface system which can:

1. acquire EMG data from the user

2. detect different user hand gestures in real time

3. implement gesture detection to control a system



Conclusions

Introduction

Objectives

Gesture Detection

System

Design

Project Management

Project Goals

1. Acquire EMG data from the user

a) Sufficient sampling rate

b) Wireless communication

c) Reliable connectivity

2. Detect user hand gestures in real time

a) Good user experience: quick calibration, fast and accurate

gesture recognition

b) Provide user with feedback through the console



Conclusions

Introduction

Objectives

Gesture Detection

System

Design

Project Management

Project Goals

3. System specifications

a) Motors

i. 180° rotation

ii. Smooth movement

b) Cameras

i. 30 frames per second

ii. 720p resolutionFigure 5: System hardware



Conclusions

Introduction

Objectives

Gesture Detection

System

Design

Project Management

April 28th, 2018

Gesture Detection



Conclusions

Introduction

Objectives

Gesture Detection

System

Design

Project Management

Initial Data Collection

• Objectives

o Understand the raw EMG data from Myo

armband

o Confirm that they could be used to

differentiate between hand movements

• Setup

o Record and save raw EMG data with Visual

Studio (C++)

o Plot the saved data in MATLAB

2

1 8 7

6

Figure 6: Myo Gesture Control

Armband with labeled sensors



Conclusions

Introduction

Objectives

Gesture Detection

System

Design

Project Management

Initial Data Acquisition

Figure 7: Raw data from waving wrist outward



Conclusions

Introduction

Objectives

Gesture Detection

System

Design

Project Management

Data Processing

Figure 8: Progression from raw to filtered data



Conclusions

Introduction

Objectives

Gesture Detection

System

Design

Project Management

Sensor Activation Patterns

Figure 9: Sensor histograms for different hand gestures



Conclusions

Introduction

Objectives

Gesture Detection

System

Design

Project Management

Calibration• Prompt user to hold each gesture for roughly

5 seconds

• Take averaged data from each sensor

• Grouped the data in consecutive groups of

three

o [123, 234, … 781, 812]

• Identify the three groups with the highest

activation levels

o Stored in a lookup table

User 1 1 2 3

FIST 8 1 7

IN 7 6 5

OUT 2 3 1

User 2 1 2 3

FIST 1 8 7

IN 6 5 7

OUT 2 1 3

Table 1: Comparison of calibrations



Conclusions

Introduction

Objectives

Gesture Detection

System

Design

Project Management

Data Processing

• A circular buffer is used to save 100

samples (0.5 seconds at 200 Hz) of

data from each sensor

• Compute two averages

o Average of each of the eight sensors

o Average of the whole buffer

Figure 10: Circular Buffer



Conclusions

Introduction

Objectives

Gesture Detection

System

Design

Project Management

Detection Algorithm

• Group the average value of all sets of three consecutive

sensors

o [123, 234, … 781, 812]

• Identify the top three most active groups

• Compare the top three groups to those from calibration

data



Conclusions

Introduction

Objectives

Gesture Detection

System

Design

Project Management

Detection Algorithm

• Assign weights to different matches

between calibration and real time

data

• Maximum possible confidence level:

20

• If confidence ≥ 10 for 0.75 seconds,

gesture is confirmed

Calibration

Real

Tim

e D

ata

1st 2nd 3rd

1st 10 7 3

2nd 4 6 2

3rd 2 3 4

Table 2: Confidence levels for

different pairings of top sensor

groupings



Conclusions

Introduction

Objectives

Gesture Detection

System

Design

Project Management

Detection Algorithm

Wave In Groupings

7 6 5

7 10 7 3

6 4 6 2

2 2 3 4

Wave Out Groupings

2 1 3

7 10 7 3

6 4 6 2

2 2 3 4

FIST GroupingsR

eal

Tim

e

Gro

up

ing

s8 1 7

7 10 7 3

6 4 6 2

2 2 3 4

FIST confidence = 3 Wave In confidence = 16 Wave Out confidence = 2



Conclusions

Introduction

Objectives

Gesture Detection

System

Design

Project Management

Detection Algorithm

Wave In Groupings

7 6 5

7 10 7 3

6 4 6 2

2 2 3 4

Wave Out Groupings

2 1 3

7 10 7 3

6 4 6 2

2 2 3 4

FIST GroupingsR

eal

Tim

e

Gro

up

ing

s8 1 7

7 10 7 3

6 4 6 2

2 2 3 4

FIST confidence = 3 Wave In confidence = 16 Wave Out confidence = 2



Conclusions

Introduction

Objectives

Gesture Detection

System

Design

Project Management

April 28th, 2018

System Design



Conclusions

Introduction

Objectives

Gesture Detection

System

Design

Project Management

System Flowchart

Figure 11: System flowchart



Conclusions

Introduction

Objectives

Gesture Detection

System

Design

Project Management

System Flowchart




Conclusions

Introduction

Objectives

Gesture Detection

System

Design

Project Management

System Flowchart




Conclusions

Introduction

Objectives

Gesture Detection

System

Design

Project Management

System Flowchart




Conclusions

Introduction

Objectives

Gesture Detection

System

Design

Project Management

Specifications – Myo Armband

• 8 EMG sensor electrode pairs

• Sampling rate: 200 Hz

• EMG data sent via Bluetooth 4.0 LE

o 8-bit signed integer

o Unitless, represents muscle activation

• Compatible with:

o Windows 7, 8, and 10

o OSx 10.8 and up

o Android 4.3 and up

2

1 8 7

6

Figure 12: Myo Gesture Control

Armband with labeled sensors



Conclusions

Introduction

Objectives

Gesture Detection

System

Design

Project Management

Hardware Diagram

Figure 13: System hardware diagram



Conclusions

Introduction

Objectives

Gesture Detection

System

Design

Project Management

System Hardware

Figure 14: System hardware



Conclusions

Introduction

Objectives

Gesture Detection

System

Design

Project Management

Specifications – Raspberry Pi 3B• Quad Core 1.2GHz Broadcom BCM2837 64bit CPU

• Bluetooth (BLE 4.1) and Wireless LAN

• Boots from Micro SD

• Runs Raspbian OS

• Input / Output

o 40-Pin extended GPIO

o CSI Camera port for Raspberry Pi Camera

o 4 Port USB

o HDMI out

o Ethernet

Figure 15: Raspberry Pi 3B [4]



Conclusions

Introduction

Objectives

Gesture Detection

System

Design

Project Management

Specifications – Raspberry Pi Camera

• Compatible with Raspberry Pi 3B

• 8 Megapixel Sony sensor

• Capable of video and photo

• 15 cm ribbon cableFigure 16: Raspberry Pi Camera

v2 [5]



Conclusions

Introduction

Objectives

Gesture Detection

System

Design

Project Management

Table 3: Bill of materials

Bill of Materials

Part Quantity Cost

Myo Gesture Control Armband 1 $200

Raspberry Pi 3B 3 $150

Raspberry Pi Camera v2 2 $50

Raspberry Pi Case 2 $36

16 GB micro SD card 3 $30

5-port Ethernet Switch 1 $10

Ethernet Cables (1 ft) 5 $10

Total Cost $ 486



Conclusions

Introduction

Objectives

Gesture Detection

System

Design

Project Management

Armband Communication

• Based on PyoConnect_v2.0 [6]

o Designed to function like the first

party software

o Built for a Linux environment

instead of Windows

• All code written in Python

2.7.14

Figure 17: PyoConnect_v2.0 User

Interface



Conclusions

Introduction

Objectives

Gesture Detection

System

Design

Project Management

Video Stream

• Used RPi-Cam-Web-Interface

o Firmware and API are open-

source [7]

• Each feed streams to a local

webpage

• Fully configurableFigure 18: Screenshot of the

camera’s webpage



Conclusions

Introduction

Objectives

Gesture Detection

System

Design

Project Management

Servo Control

• Controlled with PWM

o Using python module RPi.GPIO

• Servo motors

o Pros:

▪ From the ECE lab inventory

▪ Quickly integrated into the system

o Cons:

▪ Insufficient torque

▪ Jittery motion

▪ Delayed response

Figure 19: Servo motor mounted

atop Raspberry Pi



Conclusions

Introduction

Objectives

Gesture Detection

System

Design

Project Management

Demo Video



Conclusions

Introduction

Objectives

Gesture Detection

System

Design

Project Management

April 28th, 2018

Project Management



Conclusions

Introduction

Objectives

Gesture Detection

System

Design

Project Management

Tracking of Work – Trello

Figure 20: Project management Trello board



Conclusions

Introduction

Objectives

Gesture Detection

System

Design

Project Management

Division of LaborAditya Patel Jim Ramsay

Communication

• Video feed communications

• System component communication network

Programming

• Raspberry Pi 3B setup

• Pi GPIO configuration

• Development of code architecture

Deliverables

• Website

• Powerpoints

Data Collection and Analysis

• Data analysis

• Gesture detection algorithm

• Exploring neural network

Camera Hardware Design

• Case

• Mounts

• Servo motor wiring

Deliverables

• Figures

• Reports

Table 4: Division of Labor



Conclusions

Introduction

Objectives

Gesture Detection

System

Design

Project Management

Schedule

November

1. Write proposal

2. Submit parts list to Chris Mattus

3. Get raw data from armband

4. Discuss/consider filtering options

5. Make website

6. Draft project proposal presentation

7. Practice presentation

8. Revise proposal for final submission

December

1. Finalize the website design

2. Present project proposal

3. Submit all deliverables

Winter Break

January Winter Break

1. Compare gesture detection options and

choose which to continue with

2. Begin raspberry Pi development

Table 5: Schedule (part 1)



Conclusions

Introduction

Objectives

Gesture Detection

System

Design

Project Management

Schedule

February1. Configure Raspberry Pi and peripherals

2. Hardware design and building1. Code development

March 1. Code development1. Make poster

2. Refine the code and hardware

April1. Finalize all code

2. Present at Student Expo

1. Create presentation slides

2. Practice project presentation

3. Submit project report

Table 6: Schedule (part 2)



Conclusions

Introduction

Objectives

Gesture Detection

System

Design

Project Management

April 28th, 2018

Conclusions



Conclusions

Introduction

Objectives

Gesture Detection

System

Design

Project Management

Looking Forward

• Incorporate more gestures

o Fingers spread

o Pinch fingers

o Rotate a closed fist

o Add a second armband

• Explore biomedical applications such as

controlling a prosthetic hand

o Use an embedded system with more

computation power for advanced algorithms

o Requires higher precision control

Figure 21: Robotic hand [8]



Conclusions

Introduction

Objectives

Gesture Detection

System

Design

Project Management

Looking Forward

• Neural network

o Developed in MATLAB

o Tested on saved data

o 86% accuracy

o Plenty of room for

improvementTable 7: Confusion matrix from

neural network



Conclusions

Introduction

Objectives

Gesture Detection

System

Design

Project Management

Recap

This project successfully created an EMG-based human

machine interface that:

1. acquired EMG data from the user

2. can detect and recognize user hand gestures in real time

3. implemented gesture detection to control a system

4. laid a solid foundation for future development



Conclusions

Introduction

Objectives

Gesture Detection

System

Design

Project Management

April 28th, 2018

Questions?



Sources

[1] http://teachmeanatomy.info/upper-limb/muscles/posterior-forearm/

[2] https://www.healthline.com/health/electromyography

[3] 2016-2017 Bradley University EMG HMI, John Moron, John Cochrane, Thomas DiProva.

<http://ee.bradley.edu/projects/proj2017/emg/>

[4] https://www.raspberrypi.org/products/raspberry-pi-3-model-b/

[5] https://www.raspberrypi.org/blog/new-8-megapixel-camera-board-sale-25/

[6] http://www.fernandocosentino.net/pyoconnect/

[7] https://elinux.org/RPi-Cam-Web-Interface

[8] https://cdnb.artstation.com/p/assets/images/images/005/648/463/large/prashan-s-robotic-hand.jpg?1492700295

Documents

EMG-Based Human Machine Interfaceee.bradley.edu/projects/proj2018/emg/EMG_HMI_ConferencePresentation_20180428.pdfApril 28th, 2018 EMG-Based Human Machine Interface 8 Aditya Patel and