Real-Time Control of a Multi-Real-Time Control of a Multi-Fingered Robot Hand Using Fingered Robot Hand Using

EMG SignalsEMG SignalsMaster’s ThesisMaster’s Thesis

ByBy

Luenin BarriosLuenin Barrios

Supervisor: Marko VuskovicSupervisor: Marko Vuskovic

Department of Computer ScienceDepartment of Computer ScienceSan Diego State UniversitySan Diego State University

June 29, 2010June 29, 2010

SDSU

OutlineOutline Introduction to ResearchIntroduction to Research Multi-Fingered Robot Hands and Multi-Fingered Robot Hands and

ProsthesesProstheses Measurement of EMG SignalsMeasurement of EMG Signals Feature Extraction and ClassificationFeature Extraction and Classification Synergy and Robot Control SystemSynergy and Robot Control System Hardware DescriptionHardware Description Implementation Implementation Observations and ResultsObservations and Results SummarySummary

SDSU

IntroductionIntroduction Goal of Research:Goal of Research:

To implement a program that uses To implement a program that uses the EMG Classifier output to control the EMG Classifier output to control the grasp motions of the SDSU robot the grasp motions of the SDSU robot hand in real-timehand in real-time

Grasp modes: Grasp modes:

SDSU

Chris Miller Master’s Thesis 2008.

Prosthetic Hands Prosthetic Hands OverviewOverview

Early ModelsEarly Models Restrictions and Restrictions and

LimitationsLimitations Degrees of FreedomDegrees of Freedom EMG Signal ControlEMG Signal Control

Otto Bock Grasp Pincher TAP Version 3 Prototype SDSU Robot Hand

SDSU

Overall SchematicOverall Schematic

Prosthetic Hand

Controller

EMG Amplification

Device

A/D Converter

Classifier

Signal Processing

Time Sample Extraction

Feature Extraction

Transformation

Saksit Siriprayoonsak 2005

Chris Miller 2008

This Project

SDSU

EMG SignalsEMG Signals ElectromyographyElectromyography EMG potentials: 50 μV and up to 20 to 30 mVEMG potentials: 50 μV and up to 20 to 30 mV

Source:www.univie.ac.at/cga/courses/be522/

emg/fiber.gif

SDSU

Forearm Muscle Forearm Muscle AnatomyAnatomy

Chris Miller Master’s Thesis 2008

SDSU

EMG Amplifier DeviceEMG Amplifier Device

Saksit Siriprayoonsak Saksit Siriprayoonsak 20052005

4 Bipolar Channels4 Bipolar Channels

1 Reference Channel1 Reference Channel

Surface ElectrodesSurface Electrodes

EMG Amplifier Device EMG Amplifier Device Con’tdCon’td

SDSU

EMG Classifier ProgramEMG Classifier ProgramSignal DetectionSignal Detection

Bonato MethodBonato Method Onset of Onset of

MovementMovement

SDSU

ClassifierClassifierSignal ProcessingSignal Processing

Feature ExtractionFeature Extraction

Methods:Methods:

Waveform Length (Farry Waveform Length (Farry et alet al., ., 1996)1996)

Spectral Moments (Vuskovic Spectral Moments (Vuskovic et et alal., 2005)., 2005)

SDSU

EMG Signal Processing EMG Signal Processing Feature Extraction Method Feature Extraction Method

11Waveform Length

SDSU

EMG Signal ProcessingEMG Signal ProcessingFeature Extraction Method Feature Extraction Method

22 Spectral MomentsSpectral Moments

I-coefficientsI-coefficients

SDSU

Feature ClassificationFeature Classification Mahalanobis Distance(Mahalanobis, 1936)Mahalanobis Distance(Mahalanobis, 1936)

Sample Feature Vector Sample Feature Vector SpaceSpace

SDSU

Feature Log Feature Log TransformationTransformation

Box Cox Transformation (1964)Box Cox Transformation (1964)

SDSU

Robot Joint Control Robot Joint Control SystemSystem

PID Controller and Actuator PID Controller and Actuator

SDSU

Joint Control SystemJoint Control System Acutuator ModelAcutuator Model

SDSU

Variable Description Unit

Ra Terminal or Armature Resistance

3.38 Ohm

Ka Torque Constant 8.11 mNm/A

Jm Rotor Inertia 1.27 gcm2

Kg Gear Transmission Ratio - Thumb

Gear Transmission Ratio - Finger

1:261:19

Ga Driver Gain 1

Kb Speed or Proportionality Constant

1180 rpm/V

V0 Nominal Voltage 12 Volt

ω0 No Load Speed 13900 rpm

Joint Control SystemJoint Control System

PID ControllerPID Controller

e = qmd - qm; // Get controller errorqmdot = (qm-qmold)/_Ts; // Get derivative of errorei = eiold + e * _Ts; // Get integral of erroru = _Kp*e - _Kv*qmdot + _Ki*ei; // Control lawqmold = qm;eiold = ei;

SDSU

Synergetic MotionSynergetic Motion Synergetic MappingSynergetic Mapping

θθjj = = ffjj (m, D)(m, D) where where jj = 0, 1…5 and = 0, 1…5 and mm = = 1…4   1…4  

Approximation Function (Vuskovic and Approximation Function (Vuskovic and Marjanski)Marjanski)

am,j = γm,jcm,j = αm,j

SDSU

Synergetic TrainingSynergetic TrainingJoint Angle

θ1 (cm,1 + D1) = am,1 bm,1 - am,1 D1

SDSU

Object Shapes and SizesObject Shapes and SizesSpherical Point

Cylindrical Lateral

Calibration and TrainingCalibration and TrainingSDSU

Sample Training for Point Objects:

Sample Positions for Lateral, Cylindrical and Spherical

Robot HardwareRobot Hardware

Servo To Go BoardServo To Go Board Signal Transition Signal Transition

BoxBox

SDSU

Servo To Go Interface Servo To Go Interface BoardBoard

Encoder Input A/B signalEncoder Input A/B signal Analog Input/OutputAnalog Input/Output

SDSU

Signal Transition BoxSignal Transition Box Central hub for signals/cablesCentral hub for signals/cables Relays information Relays information Example: Joint 0

P3 DAC 2 EnIn A 14 EnIn B 17

DB50 EnOut A 35 EnOut B 34

DB25 AnalogIn 2

SDSU

EMG Robot HandEMG Robot Hand User Interface/Motion Command InterpreterUser Interface/Motion Command Interpreter Client/ServerClient/Server TCP/IPTCP/IP Real-time EMG/User CommandsReal-time EMG/User Commands Grasp modes: Cylindrical, Spherical, Point, LateralGrasp modes: Cylindrical, Spherical, Point, Lateral

SDSU

EMG Robot Hand Cont’dEMG Robot Hand Cont’dExamples: Examples:

Command: g 0 45Command: g 0 45

Command: o 3 9

SDSU

Overall Runtime Flow Overall Runtime Flow ChartChart

SDSU

Overall Runtime Flow Chart Overall Runtime Flow Chart Cont’dCont’d

SDSU

System ExecutionSystem Execution

Step 1

Step 2 Step 3

Step 4

SDSU

SummarySummary Multi-fingered Robot Hands and EMG Multi-fingered Robot Hands and EMG

SignalsSignals Collection of EMG SignalsCollection of EMG Signals Feature Extraction and ClassificationFeature Extraction and Classification PID ControllerPID Controller Synergetic MotionSynergetic Motion Overall System Diagram and Transition BoxOverall System Diagram and Transition Box Real-time control of Robot Hand using EMGReal-time control of Robot Hand using EMG SignalsSignals

SDSU

Conclusions/Future WorkConclusions/Future Work

Feasibility of EMG Signal for Hand Feasibility of EMG Signal for Hand ControlControl

Synergetic Grasp MotionsSynergetic Grasp Motions Classifier for real-time controlClassifier for real-time control Combine projects so they reside on Combine projects so they reside on

same machinesame machine Improve arm/amplifier device contactImprove arm/amplifier device contact Wireless electrodes/sensory networkWireless electrodes/sensory network Improve time delays in ClassifierImprove time delays in Classifier

Questions/Comments?Questions/Comments?

SDSU