Computational Sensory Motor Systems LabJohns Hopkins University

Computation Sensory Motor Systems Lab- Prof. Ralph Etienne-Cummings

Modeling life in silicon

Computational Sensory Motor Systems LabJohns Hopkins University

The Big Picture: Lab Motivation

Restoring function after limb amputation

Restoring locomotion after severe spinal cord injury

Developing Biomorphic Robotics

AdaptiveBiomorphic Circuits &

Systems

Computational Sensory Motor Systems LabJohns Hopkins University

Computation Sensory Motor-Systems Lab

Ralph Etienne-Cummings Lab

Towards a Spinal Neural Prosthesis Device Decoding Individual Finger Movements Using

Surface EMG Electrodes Integrate-and-Fire Array Transceiver Optimization of Neural Networks Normal Optical Flow Imager Design of Ultrasonic Imaging Arrays for Detection of

Macular Degeneration Precision Control Microsystems

Computational Sensory Motor Systems LabJohns Hopkins University

Towards a Spinal Neural Prosthesis Device

Jacob VogelsteinFrancesco Tenore

Computational Sensory Motor Systems LabJohns Hopkins University

Our Approach

Previous approaches ignore CPG and focus on controlling muscles to generate locomotion

We propose to directly control the CPG and use it to generate locomotion

Basic idea is to recreate natural neural control loop in an external artificial device (i.e. replace tonic and phasic descending inputs to the CPG with electrical stimulation)

SLP

RSMuscles

Source: Grillner, Nat Rev Neurosci, 2003

Responsibilities of Locomotion Responsibilities of Locomotion ControllerController

1. Select Gait1. Select Gait+ specify desired motor output+ specify desired motor output

-- phase relationshipsphase relationships -- joint anglesjoint angles

2. Activate CPG2. Activate CPG + tonic stimulation initiates locomotion+ tonic stimulation initiates locomotion -- epidural spinal cord stimulation (ESCS)epidural spinal cord stimulation (ESCS)

-- intraspinalintraspinal microstimulationmicrostimulation (ISMS) (ISMS)

3. Generate 3. Generate Efferent CopyEfferent Copy

+ monitor + monitor sensorimotorsensorimotor statestate -- external sensors on limbsexternal sensors on limbs

-- internal afferent recordingsinternal afferent recordings

4. Control Output 4. Control Output of CPGof CPG

+ + phasicphasic stimulationstimulation(efferent copy required for (efferent copy required for

preciselyprecisely--timed stimuli)timed stimuli) -- convert baseline CPG activityconvert baseline CPG activity

into functional motor outputinto functional motor output -- correct deviations correct deviations

-- adjust individual componentsadjust individual components -- adapt output to environmentadapt output to environment

Select gait ~ brainSelect gait ~ brainActivate CPG ~ Activate CPG ~ brainstem (MLR)brainstem (MLR)Efferent copy ~ Efferent copy ~

efferent copyefferent copyEnforce/adapt output ~ Enforce/adapt output ~

phasicphasic RS RS

Gait Control SystemGait Control System

12 pairs of IM electrodes: 3 each for left/right hip, knee, and 12 pairs of IM electrodes: 3 each for left/right hip, knee, and ankle extensors/flexorsankle extensors/flexorsTwo types of sensory data were collected for each legTwo types of sensory data were collected for each leg

Hip angle (HA) Hip angle (HA) Ground reaction force (GRF) Ground reaction force (GRF)

Source: Vogelstein et al., IEEE TBioCAS, (submitted)

Analog signal processing front-end

Spike processing back-end

Results: Results: SiCPGSiCPG Chip Controls Chip Controls Locomotion in a Paralyzed CatLocomotion in a Paralyzed Cat

Source: Vogelstein et al., IEEE TBioCAS (submitted)

Computational Sensory Motor Systems LabJohns Hopkins University

Decoding Individual Finger Movements Using Surface EMG Electrodes

Francesco Tenore

Computational Sensory Motor Systems LabJohns Hopkins University

Problem

Fast pace of development of upper-limb prostheses requires a paradigm shift in EMG-based controls

Traditional control schemes typically provide 2 degrees of freedom (DoF):

Insufficient for dexterous control of individual fingers

Surface ElectroMyoGraphy (s-EMG) electrodes placed on the forearm and upper arm of an able bodied subject and a transradial amputee

Computational Sensory Motor Systems LabJohns Hopkins University

Implemented Solution

Neural network based approach

Number of electrodes (inputs) amputation level (I-V) Level I: 32 electrodes, Level V: 12 electrodes

Computational Sensory Motor Systems LabJohns Hopkins University

Results1. High decoding accuracy:

Trained able-bodied subject, ~99%

Untrained transradial amputee, ~ 90%

2. No s.s. difference in decoding accuracy between able-bodied subjects and transradial amputee

3. No s.s. difference in decoding accuracy between networks that used different number of electrodes (12-32)

Computational Sensory Motor Systems LabJohns Hopkins University

Current/Future Work

Towards real-time control: training on rest states and movements Implementation on Virtual Integration Environment (VIE)

Independent Component Analysis (ICA) to minimize number of electrodes by choosing the ones that most contribute to the accuracy results

Computational Sensory Motor Systems LabJohns Hopkins University

Integrate-and-Fire Array Transceiver

Fopefolu Folowosele

Computational Sensory Motor Systems LabJohns Hopkins University

Motivation

The brain is capable of processing sensory information in real time, to analyze its surroundings and prescribe appropriate action

Software models run slower than real time and are unable to interactwith the environment

Silicon designs take a few months to be fabricated, after which they are constrained by limited flexibility

Computational Sensory Motor Systems LabJohns Hopkins University

IFAT

The IFAT combines the speed of dedicated hardware with the programmability of software for studying real-time operations of cortical, large-scale neural networks

Computational Sensory Motor Systems LabJohns Hopkins University

Application: Visual Processing

Computational Sensory Motor Systems LabJohns Hopkins University

Optimization of Neural Networks

Alex Russel and Garrick Orchard

Computational Sensory Motor Systems LabJohns Hopkins University

Pre Evolution Architecture

Computational Sensory Motor Systems LabJohns Hopkins University

Evolved Hip Controller

Computational Sensory Motor Systems LabJohns Hopkins University

Evolved Knee Controller

Computational Sensory Motor Systems LabJohns Hopkins University

The Final Product

Computational Sensory Motor Systems LabJohns Hopkins University

Normal Optical Flow Imager

Andre Harrison

Computational Sensory Motor Systems LabJohns Hopkins University

Normal Optical Flow Imager

Computer Vision Neuromorphic

124fig07

Computational Sensory Motor Systems LabJohns Hopkins University

Normal Optical Flow Imager

Imager that computes 2-D dense Normal Optical Flow estimates using spatio-temporal image gradients, without interfering with the imaging process

Optical Flow is the apparent motion of the image intensity

Computational Sensory Motor Systems LabJohns Hopkins University

Normal Optical Flow Imager

Computational Sensory Motor Systems LabJohns Hopkins University

Design of Ultrasonic Imaging Arrays the Detection ofMacular Degeneration

Clyde Clarke

Computational Sensory Motor Systems LabJohns Hopkins University

Design of Ultrasonic Imaging Arrays the Detection ofMacular Degeneration

www.seewithlasik.com/.../CO0077.jpg

Computational Sensory Motor Systems LabJohns Hopkins University

TooltipMountedUltrasonicMicroArray

C.NumericalModeling1) FiniteElementMethod2) FiniteDifferenceMethod

B. DeriveEquationsforWavePropagationinVitreousandRetina1) Scattering2) Absorption

L

xd

L

W

W

yd

A. CreateModelsofTransducerarrayoperatinginHomogeneousMedia

[Yakub,IEEE Trans 02]

D. ModifyDesignParametersofArraytoperformoptimallyinSurgicalEnvironment

Computational Sensory Motor Systems LabJohns Hopkins University

Adaptive and Reconfigurable Microsystems for High Precision Control

Ndubuisi Ekewe

Computational Sensory Motor Systems LabJohns Hopkins University

Adaptive and Reconfigurable Microsystems for High Precision Control

laryngoscope

base linkrotating base

distal dexterity unit (DDU)

DDU for saliva suction

DDU holder

tool manipulation unit (TMU)

fast clamping device

snake drive unitelectrical supply

/data lines

laryngoscope

base linkrotating base

distal dexterity unit (DDU)

DDU for saliva suction

DDU holder

tool manipulation unit (TMU)

fast clamping device

snake drive unitelectrical supply

/data lines

DDUholder

Parallel Manipulation UnitSnake-likeunit

enddisk

ball jointsecondarybackbone

internal wire

movingplatform lock ring

spacerdisk

basedisk

centralbackbone

DDUholder

Parallel Manipulation UnitSnake-likeunit

enddisk

ball jointsecondarybackbone

internal wire

movingplatform lock ring

spacerdisk

basedisk

centralbackbone

Simaan, 2004

EncoderG1

R2

RI

VoutMotor

D/ARs

G2 Buffer

Vcontrol

Vs

Digital position

and speed

SpeedCmd

PosMeas

SpeedMeas

SPI Interface

Microprocessor

G2-value

Digital Control(PID + FF)Position,

Velocity or Torque

Motor Setup

On-chip systems

A/DMotorFeedbk

Vifb

EncoderG1

R2

RI

VoutMotor

D/ARs

G2 Buffer

Vcontrol

Vs

Digital position

and speed

SpeedCmd

PosMeas

SpeedMeas

SPI Interface

Microprocessor

G2-value

Digital Control(PID + FF)Position,

Velocity or Torque

Motor Setup

On-chip systems

A/DMotorFeedbk

Vifb

Ekekwe et al, US Patent (Pending)

102

103

104

105

100

101

102

103

104

Encoder Frequency [Hz]

Out

put

PredictedMeasured