EMG-Based Human Machine Interface System By:...

EMG-Based Human Machine Interface SystemBy: Jonathan Morón, Thomas DiProva, and John CochraneAdvisors: Drs. In Soo Ahn and Yufeng Lu

Department of Electrical and Computer EngineeringBradley University5/2/2017

1

Overview

1. Introduction2. Objectives 3. System Design and Implementations4. Results and Discussion5. Project Management6. Conclusion and Future Work7. References8. Questions

2

IntroductionBackground: EMG Signals

● Electromyogram (EMG) signals are electric potential generated by muscles when activated by

the nervous system.

● Flexion of muscles produce EMG signals with a typical frequency range from 10 Hz to 500 Hz

● Raw EMG signals have a peak to peak voltage of -5 to +5 mV.

● Adhesive electrode pads are commonly used in recording EMG signals

Figure 1. Example of a recorded EMG signal 3

IntroductionRelated Work

● In 2015 Harvard made an assisted motion glove designed to give grip strength

and lifting support[1]

● Prosthetic limbs

● EMG from eye movement is used for HMI[2]

○ Path planning algorithm○ Feature detection through threshold○ Approximately 95.71% accuracy

● EMG is used to determine muscle damage [3]

● EMG is used to determine location of nerve damage [3]Figure 2. EMG soft glove design by Harvard[1]

4

Overview

1. Introduction2. Objectives3. System Design and Implementations4. Results and Discussion5. Project Management6. Conclusion and Future Work7. References8. Questions

5

ObjectivesThis project aims to develop a wearable EMG-based human machine interface system for in home assistance.

● The system shall read EMG signals from the user.● The system shall be calibrated to provide four control commands left, right,

forward, and stop based on EMG patterns from different movements.● The system shall send out commands wirelessly to control a service robot.● The system shall generate PWM signals for motor control.● The system shall provide live video for the purpose of monitoring.

6

Overview

1. Introduction2. Objectives 3. System Design and Implementations4. Results and Discussion5. Project Management6. Conclusion and Future Work7. References8. Questions

7

● Microcontroller (MCU) Board○ A low-power and small-size MCU board is preferred.○ It should have WiFi capability.○ 10 bit or above A/D converter is preferred.

● Wearable EMG sensor board○ A development muscle sensor board with documentation is preferred.○ It should provide EMG signal for interfacing.

● Microcomputer ○ It should have WiFi capability.○ It should host a webserver and provide live video feed.○ It should be able to generate Pulse Width Modulation (PWM) signals.

8

System Design and ImplementationsSystem Constraints/Design Considerations

System Design and ImplementationsSystem Block Diagram

9Figure 3. System Block Diagram

System Design and ImplementationsHardware: EMG Control Subsystem

● EMG Control Subsystem○ Particle Photon MCU Board○ MyoWare Muscle Sensor Board○ 3M Red Dot Electrodes○ Lithium ion battery

■ Output: 3.7v ■ Capacity: 2500mAH

10

System Design and ImplementationsHardware: Wheel Service Robot and Video Feedback

● Wheel Service Robot Subsystem○ Raspberry Pi 3 Model B○ Power Bank

■ Output: 5v, 2.4A ■ Capacity: 4000mAH

○ Logitech C260 webcam

● Video Feedback subsystem○ Computer

11

System Design and ImplementationsSystem Block Diagram (EMG Control System)

12Figure 4. EMG Control Subsystem

Figure 5. EMG Control Subsystem on John Cochrane 13

System Design and Implementations

MyoWare Muscle Sensor Board

Particle Photon MCU Board

Battery

System Design and ImplementationsMyoWare Muscle Sensor Board

● Output signal mode○ Raw EMG signal○ EMG signal envelope

■ Amplified■ Rectified■ Integrated

14

Figure 6. MyoWare Sensor Board

Figure 7. MyoWare Output Signals

System Design and ImplementationsMuscle Sensor Placement

● Sensor Placement○ Forearm Anterior

● Targeted muscles○ Brachioradialis○ Palmaris longus

Figure 9. Sensor Placement in Our Project (subject #1)

Figure 8.EMG Sensor Placements from Research Study[4]

15

System Design and ImplementationsParticle Photon

● STM32F205 ARM Cortex M3● 128 KB RAM● A/D converter

○ 12 bit resolution ○ 0.8 mV per level (Vref = 3.3 V)○ Maximum fadc = 30 MHz○ fadc clock Prescaler:(2 , 4 , 6 , 8)○ Minimum conversion time: 3 cycles

Figure 10. Particle Photon Board

Figure 11. ADC Timing Diagram 16

System Design and ImplementationsEMG Classification: Signal and Feature Extraction

17

Command 1

Command 2

Command 3

Command 4

Figure 12. Artificial Neural Network

Features derived from the input data

Recognized movement / command

Non-linear functions

System Design and ImplementationsEMG Classification: Artificial Neural Network

● Backpropagation

18

Figure 13. ANN Network Diagram: Backpropagation Algorithm

● Cost Function

Figure 14. Gradient Descent Graphical Representation

System Design and ImplementationsDescription of Pattern Recognition

1) In the MCU, the event of muscle movement is identified by thresholding. 2) After the occurrence of event, maximum 15000 samples (about 400 ms duration) are

captured to calculate the features for pattern recognition.3) After the training of Neural Network, the weight and bias of each neuron is determined

based on the optimization of cost function.4) The Neural Network structure with calibrated weight and bias is used for pattern

recognition.

19

System Design and ImplementationsSystem Block Diagram (Robot and Video Feedback)

20Figure 15. System Block Diagram

System Design and ImplementationsStinger Robot

● Used for System Validation○ Communication between subsystems○ PWM signal generation○ Webserver

● Previous experience

Figure 17. Service Robot: Top ViewFigure 16. Service Robot: Front View 21

Raspberry Pi 3 Transistor Circuit

Interface Board on Stinger Robot

System Design and ImplementationsRaspberry Pi 3 Model B

● Processor○ 1.2 GHZ quad-core ARM Cortex A53

● RAM○ 1 GB

● 40 GPIO Pins● 2 - PWM channels● WiFi

○ Network: 10/100 Mbps Ethernet○ 802.11n Wireless LAN

22

Figure 18. Raspberry Pi 3 Model B

System Design and ImplementationsPWM Signal Generation

● WiringPi Library (C Library)

○ GPIO interface library

○ PWM specific functions

● PWM output voltage

○ 5 V

23Figure 19. GPIO Pin Header

System Design and ImplementationsTransistor Circuit

24

System Design and ImplementationsVideo Feedback

25

Figure 21. Logitech C260

● Motion ○ Open source live video library ○ Frame rate: 30 FPS○ Resolution: 640 x 480

● Logitech C260

System Design and ImplementationsWireless Communication Chart

● Website○ Embedded Motion live video

○ User instructions

■ Movement configurations

○ Local Raspberry Pi IP address

26

Mobile Hotspot

Figure 22. Communications Network

Overview

1. Introduction2. Objectives 3. System Design and Implementations4. Results and Discussion5. Project Management6. Conclusion and Future Work7. References8. Questions

27

Results

Figure 23. EMG Signal: Wrist Movement Figure 24. EMG Signal: Fist Movement

Figure 26. EMG Signal: Middle Finger MovementFigure 25. EMG Signal: Ring Finger Movement

EMG Signals (Subject 1)

28

Results

Figure 28. EMG Signal: Fist Movement

Figure 30. EMG Signal: Middle Finger Movement

Figure 27. EMG Signal: Ring Finger Movement

Figure 29. EMG Signal: Wrist Movement29

EMG Signals (Subject 2)

ResultsPWM Output

30

Figure 31. PWM Output - Straight

Figure 32. PWM Output - Left Turn Figure 33. PWM Output - Right Turn

Results

● Fist○ Left Motor - on

● Wrist Flexion ○ Stop Motor

31

● Ring Finger○ Right Motor - on

● Middle Finger○ Straight - both motors on

Overview

1. Introduction2. Objectives3. System Design and Implementations4. Results and Discussion5. Project Management6. Conclusion and Future Work7. References8. Questions

32

Project ManagementDivision of Labor

Jon Morón1. Wheel Service Robot Subsystem 2. Video Monitoring System

a. Server

John Cochrane1. EMG-Control Subsystem

a. MyoWare Placementb. EMG DSP

Thomas DiProva1. EMG-Control Subsystem

a. EMG classificationb. Wireless Communication

33

Project ManagementSchedule

34

Project ManagementBill of Materials

35

Overview

1. Introduction2. Objectives3. System Design and Implementations4. Results and Discussion5. Project Management6. Conclusion and Future Work7. References

36

Conclusion

● A wearable EMG human machine interface system has been developed. ● The system has successfully acquired EMG signal. ● The system can calibrate and classify EMG signals from different movements.● Control commands have been sent wirelessly to control a service robot.● A web server has been implemented to provide video feedback.

37

Future Work

● A more powerful embedded system could be used to implement advanced

signal processing algorithms for pattern recognition.

● System sensitivity and accuracy could be further improved by including more

EMG sensors.

● A “real” service robot could be used to demonstrate practical applications.

38

Overview

1. Introduction2. Objectives 3. System Design and Implementations4. Results and Discussion5. Project Management6. Conclusion and Future Work7. References

39

References[1] Walsh, Connor J. Herman, Maxwell. Sanah, Siddharth. Galloway, Kevin C. Polygerinos, Panagiotis. “EMG Controlled Soft Robotic Glove For Assistance

During Activities of Daily Living.” Harvard Biodesign Laboratory, August 2015.

[2] Ferreira, Andre, Wanderley C. Celeste, Fernando A. Cheein, Teodiano F. Bastos-Filho, Mario Sarcinelli-Filho, and Ricardo Carelli. "Human-machine Interfaces

Based on EMG and EEG Applied to Robotic Systems." Journal of NeuroEngineering and Rehabilitation. BioMed Central, March 2008.

[3] Keating, Jennifer. "Relating Forearm Muscle Electrical Activity To Finger Forces." Worcester Polytechnic Institute, May 2014.

[4] Stergiou, Christos. "Neural Networks." Department of Computing. Imperial College London, September 2011.

[5]"What Is EMG (Electromyography) and How Does It Work?" IMotions, September 2016.

[6] Wu, Yunfen, María Ángeles Martínez Martínez, and Pedro Orizaola Balaguer. "Overview of the Application of EMG Recording in the Diagnosis and

Approach of Neurological Disorders."Overview of the Application of EMG Recording in the Diagnosis and Approach of Neurological Disorders | InTechOpen.

InTech, May 2013.

[7]Gil, Jamie Gomez, Israel San-Jose-Gonzalez, Luis Fernando Nicolas-Alonso, and Sergio Alonso Garcia. Steering a Tractor by Means of an EMG-Based

Human-Machine Interface. Tech. Department of Signal Theory, Communications and Telematics Engineering, University of Valladolid, July 2011.

40

Questions?

41