10
RESEARCH Open Access Decoding of grasping information from neural signals recorded using peripheral intrafascicular interfaces Silvestro Micera 1,2*, Paolo M Rossini 8,4, Jacopo Rigosa 1 , Luca Citi 1 , Jacopo Carpaneto 1 , Stanisa Raspopovic 1 , Mario Tombini 3 , Christian Cipriani 1 , Giovanni Assenza 3 , Maria C Carrozza 1 , Klaus-Peter Hoffmann 5 , Ken Yoshida 6 , Xavier Navarro 7 and Paolo Dario 1 Abstract Background: The restoration of complex hand functions by creating a novel bidirectional link between the nervous system and a dexterous hand prosthesis is currently pursued by several research groups. This connection must be fast, intuitive, with a high success rate and quite natural to allow an effective bidirectional flow of information between the users nervous system and the smart artificial device. This goal can be achieved with several approaches and among them, the use of implantable interfaces connected with the peripheral nervous system, namely intrafascicular electrodes, is considered particularly interesting. Methods: Thin-film longitudinal intra-fascicular electrodes were implanted in the median and ulnar nerves of an amputees stump during a four-week trial. The possibility of decoding motor commands suitable to control a dexterous hand prosthesis was investigated for the first time in this research field by implementing a spike sorting and classification algorithm. Results: The results showed that motor information (e.g., grip types and single finger movements) could be extracted with classification accuracy around 85% (for three classes plus rest) and that the user could improve his ability to govern motor commands over time as shown by the improved discrimination ability of our classification algorithm. Conclusions: These results open up new and promising possibilities for the development of a neuro-controlled hand prosthesis. I. Introduction The human hand is a versatile organ that is used for grasping heavy or delicate objects and for performing highly complex manipulations on the basis of fine motor control and precise sensory feedback [1]. The restoration of these sensorimotor functions after upper limb amputa- tion is particularly challenging. Two main components must be developed: (a) hand prostheses able to mimic the natural hand from both a biomechanical and sensory points of view [2]; (b) intimate interfaces for online brid- ging the users nervous system and the external prosthesis. Several solutions are possible and are currently investi- gated by independent research groups using non-invasive [3-5], and invasive [6-8] approaches. For instance, intra- cortical signals can be used to simultaneously control reaching and one degree of freedom grasping of an artifi- cial limb as recently shown in non-human primates [8]. In case of amputation, it is also possible to use the resi- dual part of amputee muscles and peripheral nerve fibers to control the artificial hand. This approach can be implemented by processing electromyographic (EMG) signals recorded using either non-invasive [9] or invasive [10,11] electrodes. The transposition of residual nerves of amputees to other muscles in or near the amputation site can be also implemented [12-14]. This approach (called Targeted Muscle Reinnervation, TMR) is probably the * Correspondence: [email protected] Contributed equally 1 BioRobotics Institute, Scuola Superiore SantAnna, Pisa (Italy Full list of author information is available at the end of the article Micera et al. Journal of NeuroEngineering and Rehabilitation 2011, 8:53 http://www.jneuroengrehab.com/content/8/1/53 JNER JOURNAL OF NEUROENGINEERING AND REHABILITATION © 2011 Micera et al; licensee BioMed Central Ltd. This is an Open Access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/2.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

Decoding of grasping information from neural …...RESEARCH Open Access Decoding of grasping information from neural signals recorded using peripheral intrafascicular interfaces Silvestro

  • Upload
    others

  • View
    7

  • Download
    0

Embed Size (px)

Citation preview

Page 1: Decoding of grasping information from neural …...RESEARCH Open Access Decoding of grasping information from neural signals recorded using peripheral intrafascicular interfaces Silvestro

RESEARCH Open Access

Decoding of grasping information from neuralsignals recorded using peripheral intrafascicularinterfacesSilvestro Micera1,2*†, Paolo M Rossini8,4†, Jacopo Rigosa1, Luca Citi1, Jacopo Carpaneto1, Stanisa Raspopovic1,Mario Tombini3, Christian Cipriani1, Giovanni Assenza3, Maria C Carrozza1, Klaus-Peter Hoffmann5, Ken Yoshida6,Xavier Navarro7 and Paolo Dario1

Abstract

Background: The restoration of complex hand functions by creating a novel bidirectional link between thenervous system and a dexterous hand prosthesis is currently pursued by several research groups. This connectionmust be fast, intuitive, with a high success rate and quite natural to allow an effective bidirectional flow ofinformation between the user’s nervous system and the smart artificial device. This goal can be achieved withseveral approaches and among them, the use of implantable interfaces connected with the peripheral nervoussystem, namely intrafascicular electrodes, is considered particularly interesting.

Methods: Thin-film longitudinal intra-fascicular electrodes were implanted in the median and ulnar nerves of anamputee’s stump during a four-week trial. The possibility of decoding motor commands suitable to control adexterous hand prosthesis was investigated for the first time in this research field by implementing a spike sortingand classification algorithm.

Results: The results showed that motor information (e.g., grip types and single finger movements) could beextracted with classification accuracy around 85% (for three classes plus rest) and that the user could improve hisability to govern motor commands over time as shown by the improved discrimination ability of our classificationalgorithm.

Conclusions: These results open up new and promising possibilities for the development of a neuro-controlledhand prosthesis.

I. IntroductionThe human hand is a versatile organ that is used forgrasping heavy or delicate objects and for performinghighly complex manipulations on the basis of fine motorcontrol and precise sensory feedback [1]. The restorationof these sensorimotor functions after upper limb amputa-tion is particularly challenging. Two main componentsmust be developed: (a) hand prostheses able to mimic thenatural hand from both a biomechanical and sensorypoints of view [2]; (b) intimate interfaces for online brid-ging the user’s nervous system and the external prosthesis.

Several solutions are possible and are currently investi-gated by independent research groups using non-invasive[3-5], and invasive [6-8] approaches. For instance, intra-cortical signals can be used to simultaneously controlreaching and one degree of freedom grasping of an artifi-cial limb as recently shown in non-human primates [8].In case of amputation, it is also possible to use the resi-

dual part of amputee muscles and peripheral nerve fibersto control the artificial hand. This approach can beimplemented by processing electromyographic (EMG)signals recorded using either non-invasive [9] or invasive[10,11] electrodes. The transposition of residual nerves ofamputees to other muscles in or near the amputation sitecan be also implemented [12-14]. This approach (called“Targeted Muscle Reinnervation”, TMR) is probably the

* Correspondence: [email protected]† Contributed equally1BioRobotics Institute, Scuola Superiore Sant’Anna, Pisa (ItalyFull list of author information is available at the end of the article

Micera et al. Journal of NeuroEngineering and Rehabilitation 2011, 8:53http://www.jneuroengrehab.com/content/8/1/53 J N E R JOURNAL OF NEUROENGINEERING

AND REHABILITATION

© 2011 Micera et al; licensee BioMed Central Ltd. This is an Open Access article distributed under the terms of the Creative CommonsAttribution License (http://creativecommons.org/licenses/by/2.0), which permits unrestricted use, distribution, and reproduction inany medium, provided the original work is properly cited.

Page 2: Decoding of grasping information from neural …...RESEARCH Open Access Decoding of grasping information from neural signals recorded using peripheral intrafascicular interfaces Silvestro

most advanced clinical solution currently available and ithas the interesting advantage that the nerve function cor-relates physiologically to the motor action controlling inthe prosthesis. Therefore, the control of the prosthesis ismore natural and easier than with other EMG-basedapproaches. However, it presents two main limitations:(i) it requires a major surgical intervention and the use ofa grid of surface electrodes. In this way, the problemsdue to the invasiveness are not totally balanced by signifi-cant advantages in terms of usability and cosmeticappearance; (ii) it is effective mainly for proximal ampu-tation levels (i.e., close to the axilla, which are lesscommon than transradial - adjacent to the elbow - ampu-tations) because of the characteristics of the surgicalprocedure.The use of invasive neural interfaces directly connected

to the peripheral nervous system (PNS) is potentiallyappealing because it is able to provide an almost “physio-logical” condition in which efferent and afferent fibers,previously connected with the natural hand, may returnto their role in controlling the prosthetic limb/hand. Thisis theoretically possible since a significant amount of per-ipheral nerve fibers, as well as spinal cord and brain con-nections dedicated to the control of the amputated limb,survive in time and remain available to restore a “physio-logical” condition also thanks to the plastic abilities ofthe CNS to reorganize for functional recovery [15-17].Several invasive PNS interfaces have been developed in

the past [18]. Although most of them were originallyused for functional electrical stimulation (FES) in spinalcord injured persons [19-21], they can also be the keycomponent of neuro-controlled hand prostheses. In thiscase, they are used to record efferent motor signals andto deliver sensory feedback [22-25].Among invasive PNS interfaces, longitudinal intra-

fascicular electrodes (LIFEs) - namely intrafascicularelectrodes inserted longitudinally into the nerve [26,27]- are interesting due to the selective contact with a lim-ited number of specific nerve fibers and the relativelylow level of invasiveness required for their implantation.In fact, after appropriate control in experimental modelsto test their biocompatibility and efficacy [18], LIFEshave been recently used to control artificial devices[6,7,22-25] showing good results during short-term trialswith human amputees. These trials showed that subjectsare able to control a one-degree of freedom hand pros-thesis through online processing of the efferent neuralsignals, and that repeatable sensory feedback may bereceived by stimulating afferent fibers [22,23] corre-sponding to the missing hand/fingers territories. More-over, as seen in animal models [28], LIFEs allow theextraction of spikes from the signals recorded signifi-cantly increasing the decoding ability for online classifi-cation. Spike shapes associated to different fibers depend

on the size and conduction speed of the fibers, on therelative distance and orientation between the fiber andthe electrode, and on the inhomogeneity of the intrafasci-cular space. Therefore, a spike sorting approach can beused to identify the activity related to different nervefibers, significantly increasing the decoding ability.Starting from these encouraging results, the aim of

this study was to verify whether “hand-related” actions(such as different grip types or movements of single fin-gers) can be decoded by processing neural motor-relatedsignals recorded by LIFEs. This result could allow con-trol of a dexterous hand prosthesis using the naturalneural “pathway” and increase the usability of actuatedartificial hands.In particular, the following issues were addressed also as

an extension of a recent study on the same case [24,25]: (i)how many degrees of freedom (or different grasping tasks)can be reliably extracted and controlled from efferentneural signals; (ii) to which extent the combined analysisof an ensemble of motor LIFE signals recorded when amotor command is dispatched to the amputated hand/fin-gers improves movement classification via the interfaceprocessor; (iii) whether there is any learning effect duringthe use of the interface; (iv) whether this kind of approachneeds frequent re-calibrations as in the case of invasivecortical neuroprostheses.A more general and clinically-oriented paper on the

same subject, concerning the technique of LIFE insertionand LIFE signal analysis, including results on output con-trol, sensory perception, clinical outcome and associatedneuroplastic changes has already been published elsewhere[24].

II. Materials and methodsA. Thin-film LIFEsA new version of LIFEs, named thin-film LIFEs (tfLIFE),was used in the experiments [29]. These electrodes weredeveloped on a micropatterned polyimide substrate, whichwas chosen because of its biocompatibility, flexibility andstructural properties. After microfabrication, this substratefilament (shown in Figure 1) was folded in half so thateach side had four active recording sites. Therefore,tfLIFEs allow recordings at eight active sites per electrode.A tungsten needle linked to the polyimide structure wasused for implanting the electrode and was removed imme-diately after insertion.

B. Experimental protocolP.P., a 26 year old male, suffered amputation of his leftarm two years before the implantation due to a car acci-dent. The surgical procedures are detailed elsewhere [24].Briefly, following epineural micro-dissection, twotfLIFE4s (Figure 1) were inserted in the ulnar and themedian nerves 45° obliquely to assure stability and to

Micera et al. Journal of NeuroEngineering and Rehabilitation 2011, 8:53http://www.jneuroengrehab.com/content/8/1/53

Page 2 of 10

Page 3: Decoding of grasping information from neural …...RESEARCH Open Access Decoding of grasping information from neural signals recorded using peripheral intrafascicular interfaces Silvestro

increase the probability of intercepting nerve fibers. Thedistal handle of the electrode was anchored to the epi-neurium. Four weeks later, tfLIFE4s were removed asrequired by the European Health Authorities. P.P.worked on the project 4-6 hours/day for 6 days/week,and did not report any complication during the following12 months. A complete, clinical report on this case canbe found elsewhere [24].During the first three weeks of experiments P.P. was

involved in several experiments addressing different bio-medical and neurophysiological issues related to the effi-cacy of this approach (e.g., sensory feedback [24], scalpEEG recordings [30], Trancranial Magnetic Stimulation(TMS-related) neural signals [31]).The trials on the recording of neural motor LIFE signals

were carried out during the last week of experiments, afterverifying an improvement in the signal-to-noise ratio ofthe LIFE signals. In particular, P.P. was specifically askedto separately and selectively dispatch the order to performthe following three movements: (a) palmar grasp, (b)pinch grasp, (c) flexion of the little finger. Pictures repre-senting these tasks were randomly presented to him on acomputer screen to provide a visual cue. The overall pro-cessing scheme is shown in Figure 2 and will be brieflydescribed in the next paragraphs.LIFE motor signals were recorded via 4-channel ampli-

fiers (Grass QP511 Quad AC; ENG amplified: X10.000, fil-tered: 100 Hz- 10 kHz; 16 bit, 1 Ms/s analogue-to-digitalconverter).When above a selected threshold, the neural signals

recorded were used to teleoperate a dexterous hand pros-thesis. The prosthetic hand used was a stand-alone ver-sion of the CyberHand prototype, already employed inseveral research scenarios [32]. This approach was usedas a “biofeedback” to increase the confidence of thepatient in this procotol.

C. Off-line Processing algorithms of LIFE signalsA wavelet denoising technique was used to improve thesignal-to-noise ratio (SNR) in the neural signals recorded.Wavelet denoising is a set of techniques used to removenoise from signals and images. The main idea is to trans-form the noisy data into an orthogonal time-frequencydomain. In that domain, thresholding is applied to thecoefficients for noise removal, and the coefficients arefinally transformed back into the original domain obtain-ing the denoised signal [33]. A decomposition schemebased on the translation-invariant wavelet transform [34]was used as shown in [28]. After the denoising, the differ-ent classes of spikes were extracted. The algorithm con-sisted of a two-step process [28]: creation of spiketemplates and then comparison of spikes in the signalwith the templates using some similarity indexes as thecorrelation coefficient or the mean square differencebetween a spike and a template and the power of thetemplate. This spike-based approach was used because italready showed to allow high classification accuracy inanimal experiments [28]. The characterization of theSNR was carried out according to the methodology pre-viously described in [35].

D. Identification of the desired motor commandsThe denoised signals were then used to identify the dis-patched motor commands by implementing the followingprocedure. For each trial, the different recording periods(’epochs’) related to the movement classes (e.g., grip typesand rest) were labeled. The three desired movements andthe rest were considered as separated classes. The perfor-mance metrics considered was the ratios between classescorrectly identified out of those presented and the leaveone out validation standard method has been used.Each epoch was an example that was used to train the

classifier or to test its generalization skills. The feature

Figure 1 Picture and unfolded overview of tfLIFE (17). Total length: 60 mm. Length without pad areas: 50 mm. Each end of the tfLIFE carriesa ground electrode (GND), a reference electrode (L0, R0) and the recording sites (L1-L4, R1-R4).

Micera et al. Journal of NeuroEngineering and Rehabilitation 2011, 8:53http://www.jneuroengrehab.com/content/8/1/53

Page 3 of 10

Page 4: Decoding of grasping information from neural …...RESEARCH Open Access Decoding of grasping information from neural signals recorded using peripheral intrafascicular interfaces Silvestro

vector was made of the ratios between the number ofspikes matching each spike template and the total num-ber of spikes in the epoch [28]. Therefore, the absolutespike rates were not used, but rather the relative spikerates of each waveform w.r.t. the others. This shouldprevent classification of the motor commands based onthe “quantity of activity” and favor the use of the “qual-ity of activity” intended in terms of different waveformsfor different stimuli.In order to infer the type of stimulus applied during a

given epoch from the feature vector F, a classifier basedon support vector machines (SVMs, [36]) was usedmaking use of the open source library LIBSVM [37]. Toallow SVMs, and other binary classifiers, to handle mul-ticlass problems, the latter must be decomposed intoseveral binary problems. In this work we used a one-against-one approach.Finally, in order to assess the inter-day robustness of

this approach the classifier was trained by using all thefeatures extracted for the first day of recordings duringthe last week of experiments and used without anyfurther change for the next days. This was done to

understand whether this approach might be used withoutany need for recalibration.

III. ResultsAn example of a LIFE motor signal before and after thewavelet denoising is given in Figure 3.The SNRs for the different channels before and after

the denoising are given in Figure 4. The quality of therecording after denoising can be considered good simi-larly to the considerations done in [35].In Figure 5, the raster plots for different classes of

spikes (different colors) for the different movements areprovided. A different modulation for different tasks canbe seen showing that the neural signals recorded and, inparticular, the classes of spikes extracted can be consid-ered related to different motor commands. It is alsointeresting that we have a double-peak spike rate espe-cially for the palmar grasp. This could allow in thefuture the use of the two peaks for the pre-shaping(opening) and closing of the prosthetic device.In Table 1, the best performance achieved with the

best combination of tfLIFE channels is shown for the

Figure 2 Scheme of the algorithms used to process the LIFE motor signals and to identify the desired class of movement. The signalsrecorded were denoised. This is necessary and quite useful for the commonly quite low signal-to-noise ratio of the LIFE signals [21]. The classesof spikes extracted from the LIFE signals were used as inputs for an SVM classifier able to identify the desired voluntary activity. Results aboutsensory feedback experiments based on electrical stimulation of afferent nerve fibers are reported in [24].

Micera et al. Journal of NeuroEngineering and Rehabilitation 2011, 8:53http://www.jneuroengrehab.com/content/8/1/53

Page 4 of 10

Page 5: Decoding of grasping information from neural …...RESEARCH Open Access Decoding of grasping information from neural signals recorded using peripheral intrafascicular interfaces Silvestro

decoding of rest plus one, two, or three movementclasses are shown in the Table 2. In most cases, a com-bination of features coming from all the three tfLIFE

channels is required to obtain the best recognition ratio(rr). Most of the channels we used were the “expectedones” from a neurophysiological viewpoint but someunexpected channels were also used. This could be dueto some inability of the user to deliver the natural ("cor-rect”) neurophysiological commands.In Table 2, the analysis of the advantages connected

with the daily retraining of the classifier is shown. Inparticular, the first row shows the best results achievedwhen the classifier was trained only during the first daywhile the second row provides performance when thealgorithm was re-trained everyday.The improvement of performance could be due not

only to a limit in the robustness of the approach (i.e., thetime-variance of the LIFE signals recorded during the dif-ferent days) but also to the learning ability shown by theuser. In fact, Figure 6 (right panel) shows the classifica-tion performance achieved for different classes duringtwo days of recording. The learning process of the sub-ject is a distinctive feature of this approach.In order to understand the importance of using multi-

channel electrodes for neural recordings, classification

Figure 3 The raw (top panel) and the denoised (bottom panel) LIFE motor signals. Superimposed with the signal (black line) are shownsignals, which represent the different tasks the subject was asked to perform (rest = yellow line; flexion of the little finger = blue line; palmargrasp = red line; pinch grasp = green line).

0 2 4 6 8 10 120

2

4

6

8

10

# of channels

)naem(

RN

S

rawfilt

Figure 4 SNR before and after the denoising for the differentchannels.

Micera et al. Journal of NeuroEngineering and Rehabilitation 2011, 8:53http://www.jneuroengrehab.com/content/8/1/53

Page 5 of 10

Page 6: Decoding of grasping information from neural …...RESEARCH Open Access Decoding of grasping information from neural signals recorded using peripheral intrafascicular interfaces Silvestro

Figure 5 Raster plots and spike rates. Top panel: raster plots for different classes of spikes during the selection of each type of movements bythe subject. Bottom panel: corresponding spike rates for the corresponding classes windows (mean = larger lines; mean ± SD = smaller linesand colourful areas; areas are colored when not overlapped). Window duration for each class has been normalized (from 0 to 100%). The redvertical lines delimit the timing of the classes triggering in both top and bottom panels. Only windows with the fixed width of the controlledexperiment have been used.

Table 1 Performance of the classifier (recognition ratio) with re-training of the classification algorithm

Task(s) identified rr M1 channel # M2 channel # U channel #

(%) 1 2 3 4 5 6 7 8 9 10 11 12

rest vs little 93 – – – – X – – – X X – X

rest vs palmar 95 X – – – – X – – X – – –

rest vs pinch 100 – X – X X – – X – X – –

rest vs little vs palmar 92 – X – – X – X – – X – X

rest vs little vs pinch 90 – X – X – X – X – – X –

rest vs pinch vs palmar 87 – – X – X – – – – – – X

rest vs little vs pinch vs palmar 85 – X – – X – X – X X – –

rest vs activity 87 – – – X X X X X – X X X

little vs pinch vs palmar – X X – – – – X – X – X

The parameters of the classifier were selected for each day using data recorded in that specific day for the training. For each classification, the max performance(rr) and the best combination of channels used are given. rr = recognition ratio of the SVM classifier; M1 an M2 indicate the two tfLIFE implanted in the mediannerve whereas U indicates the tfLIFE implanted in the ulnar nerve.

Micera et al. Journal of NeuroEngineering and Rehabilitation 2011, 8:53http://www.jneuroengrehab.com/content/8/1/53

Page 6 of 10

Page 7: Decoding of grasping information from neural …...RESEARCH Open Access Decoding of grasping information from neural signals recorded using peripheral intrafascicular interfaces Silvestro

performance was calculated and compared using differ-ent recording channels. The results shown in Figure 6(left panel) indicate that increasing the number of chan-nels used for the classification enhances the performancebecause of the increased amount of information. Above acertain limit, performance starts degrading again, giventhe classic overfitting issues of classifiers (e.g., there is anexcessive number of parameters to be optimized).

IV DiscussionAmong the different approaches for restoring the linkbetween the external world and the nervous system, theuse of implantable intrafascicular PNS interfaces is of

great potential, as confirmed in previous experiments inhumans [6,7]. The evolution of peripherally-controlledhand prostheses seems to go towards a more naturaland ideal approach which is the general evolution ofartificial devices. In fact, differently from the processingof surface EMG signals, which prevents the use of morenatural neural “channels”, the TMR approach is morenatural because of the reconnection of peripheral nervespreviously linked to the amputated limb but it still relieson surface EMG signals. To this respect, the use ofimplantable interfaces into the PNS could allow therestoration of the pre-existing neural connection com-pleting this evolution.

Table 2 Performance of the classifier (recognition ratio) without and with re-training of the classification algorithm

Grip type(s) identified and controlled (plus rest) Little, pinch, palmar Little, palmar Little, pinch Palmar, pinch Little Palmar Pinch

No Retraining 0.72 0.86 0.84 0.8 0.9 0.76 0.9

Retraining 0.85 0.92 0.9 0.87 0.93 0.95 1

In case of “re-training” the parameters of the classifier were selected for each day using data recorded in that specific day for the training. In case of “notraining”, the definition of the parameters happened only for the first day.

2 4 6 8 01 210

02

04

06

08

001

Max

Per

form

ance

(%

)

slennahc fo #1 2 3

0

02

04

06

08

001

sessalc fo #

82 yad03 yad

Figure 6 Classification performance. Right Panel: Improvement of the classification performance for one, two, or three classes during twotraining days. Left Panel: Maximum performances are shown as a function of the number of channels simultaneously acquired during the motorcommands delivered to the missing limb.

Micera et al. Journal of NeuroEngineering and Rehabilitation 2011, 8:53http://www.jneuroengrehab.com/content/8/1/53

Page 7 of 10

Page 8: Decoding of grasping information from neural …...RESEARCH Open Access Decoding of grasping information from neural signals recorded using peripheral intrafascicular interfaces Silvestro

The aim of our study was to increase the understand-ing of the basic mechanisms, as well as of the possibili-ties and limits of this approach, by processing the motorsignals recorded the nerve fibers as final output to thetarget muscles (via tfLIFE signals analysis).Two main assumptions were tested: (1) the extraction

of spike templates can allow good decoding performance;(2) it is possible to extract more than a simple one degreeof freedom control signal (as previously done by otherresearchers [7]). In particular, more complex informationrelated to grip types can be decoded from motor signalsrecorded using tfLIFEs.Concerning the first assumption we moved from pre-

vious knowledge [28] that an algorithm based on spikesorting can improve the decoding performance withrespect to other more classical approaches based on theextraction of power-related information from the signal.In particular, the spike-sorted data can effectively increasethe resolution of the information that can be extractedfrom a single electrode. This could improve the decodingperformance and to some extent reduce invasiveness sincefewer electrodes would be required to obtain the needednumber of information channels.The idea of decoding complex high-level information

(i.e., grip types) represents the main novelty of this study.It is a disruptive approach for decoding and potentiallyquite challenging because of the characteristics of the sig-nals to process. In fact, signals recorded from the periph-eral nerves are related to low-level information (i.e., thecommands to control muscular contraction). Therefore,this “high-level” decoding could be considered similar tounderstand a global picture (the grip type) with only a fewpieces (related to specific muscle commands) of it. Thisstrategy would allow an easier, more efficacious and faster(namely online) control of a dexterous hand prosthesis. Infact, this could increase the number of motor commandswhich could be dispatched to the hand without a very pre-cise decoding necessary for example for the simultaneouscontrol of the kinematics of several joints.The results of this study, in terms of rate of correct

classification of motor tasks, show that the two assump-tions were correct, based on good performance achieve-ment and a state control algorithm implementation. Thiscontrol approach is commonly used in the EMG-basedcontrol of hand prostheses [38] and has been recentlyimplemented in cortical neuroprostheses [39]. For thefirst time, three different hand movements were identi-fied with good performance. This could allow the fullusability of dexterous prostheses by the user who simplydispatches motor commands for hand movements as henormally did before amputation. Moreover, it could bepossible to use the double-peak distribution shown forthe palmar grasp (Figure 5) to separately control the pre-shaping (opening) and closing of the hand prosthesis.

Of great interest was that our amputee subject wasable to improve his performance during the consecutivetrials. This was particularly evident during the experi-ments by looking at the SNR and was also confirmed bythe classification results. However, this was achievedonly for two consecutive days and this learning abilityhas to be confirmed during longer tests.The performance also improved by using several

intrafascicular recording channels. This is due to theintrinsic blindness of the implantation procedure: alarge number of channels can increase the probability ofpicking up enough signals to reach good classificationlevels and to maintain such a rate of appropriate classi-fication stable in time. In fact, an important characteris-tic of our approach was the robustness of classificationduring the trials. This kind of interface seems to bequite stable and makes the daily re-calibration of thealgorithm - which affects other implantable neuro-prostheses - unnecessary. This is very important andcould represent a significant advantage during dailyactivities carried out outside a controlled laboratoryenvironment.Unfortunately, due to the time limitations of our experi-

ments it was not possible to increase the number of griptypes that the subject was asked to perform. However,results clearly indicate that multiple movements can begoverned, probably more than the three used in the pre-sent study. Moreover, it is also possible to combine thestate control approach presented in this paper to selectdifferent grasping tasks together with a proportional con-trol (already achieved in [6,7]). This will allow both themodulation of force during grasping and the simultaneouscontrol of hand and elbow functions.In the near future, the possibility of verifying these

findings during chronic experiments with an implantedhand prosthesis for daily activity uses, will be investi-gated. The analysis of the performance of the LIFE elec-trodes during long-term implants is very important tounderstand the potentials and shortcomings of thistechnology in terms of chronic usability, stability, etc.Moreover, particular attention will be given to charac-terize long-term changes in the cortical activation andnew approaches will be explored for characterizing thelong-term usability and for improving the efficacy ofintrafascicular electrodes. For example, as previouslyshown for the central nervous system [40], actuation ofthe active sites of the electrodes [41,42] may help toincrease the SNR and selectivity both for stimulationand recording.

AcknowledgementsThis work was partially supported by the EU within the NEUROBOTICSIntegrated Project (IST-FET Project 2003-001917: The fusion of NEUROscienceand roBOTICS).

Micera et al. Journal of NeuroEngineering and Rehabilitation 2011, 8:53http://www.jneuroengrehab.com/content/8/1/53

Page 8 of 10

Page 9: Decoding of grasping information from neural …...RESEARCH Open Access Decoding of grasping information from neural signals recorded using peripheral intrafascicular interfaces Silvestro

Author details1BioRobotics Institute, Scuola Superiore Sant’Anna, Pisa (Italy. 2Institute forAutomation, Swiss Federal Institute of Technology, Zurich (Switzerland.3Campus Biomedico University, Rome (Italy. 4Casa di Cura S. Raffaele, andIRCCS S. Raffaele-Pisana, Rome (Italy. 5Fraunhofer Institute for BiomedicalEngineering, St. Ingbert (Germany. 6Indiana University-Purdue University,Indianapolis (USA. 7Universitat Autònoma de Barcelona, and CIBERNED,Barcelona (Spain. 8Institute of Neurology, Catholic University, Po. Gemelli,Rome, Italy.

Authors’ contributionsSM and PMR contributed to all the stages of this work (i.e., design of theprotocol, following all the experiments, data interpretation and writing). JC,LC, SR, and JR developed decoding algorithms for signal processing andclassification. MT and GA contributed to the in-vivo experiments and all theclinical training. CC and MCC designed the algorithms for the control of thehand prosthesis. PD, XN, and KY contributed to the design of the protocol.KPH designed tf-LIFEs. All authors read and approved the final manuscript.

Competing interestsCC hold shares in Prensilia Srl, the company that manufactures robotichands as the one used in this work, under the license from Scuola SuperioreSant’Anna.

Received: 3 January 2011 Accepted: 5 September 2011Published: 5 September 2011

References1. Kapandji IA: The physiology of the joints, volume one: upper limb Edinburgh:

Churchill Livingstone; 1982.2. Micera S, Carrozza MC, Beccai L, Vecchi F, Dario P: Hybrid Bionic Systems

for the Replacement of Hand Function. Proc IEEE 2006, 94:1752-1762.3. Englehart K, Hudgins B: A robust, real-time control scheme for

multifunction myoelectric control. IEEE Trans Biomed Eng 2003, 50:848-854.4. Huang Y, Englehart KB, Hudgins B, Chan AD: A Gaussian mixture model

based classification scheme for myoelectric control of powered upperlimb prostheses. IEEE Trans Biomed Eng 2005, 52:1801-1811.

5. Chatterjee A, Aggarwal V, Ramos A, Acharya S, Thakor NV: A brain-computer interface with vibrotactile biofeedback for haptic information.J Neuroeng Rehabil 2007, 4:40.

6. Dhillon GS, Horch KW: Direct neural sensory feedback and control of aprosthetic arm. IEEE Trans Neural Syst Rehabil Eng 2005, 13:468-472.

7. Jia X, Koenig MA, Zhang X, Zhang J, Chen T, Chen Z: Residual motorsignal in long-term human severed peripheral nerves and feasibility ofneural signal-controlled artificial limb. J Hand Surg Am 2007, 32:657-666.

8. Velliste M, Perel S, Spalding MC, Whitford AS, Schwartz AB: Cortical controlof a prosthetic arm for self-feeding. Nature 2008, 453:1098-1101.

9. Zecca M, Micera S, Carrozza MC, Dario P: Control of multifunctionalprosthetic hands by processing the electromyographic signal. Crit RevBiomed Eng 2002, 30:459-485.

10. Weir RF, Troyk PR, DeMichele GA, Kerns DA, Schorsch JF, Maas H:Implantable myoelectric sensors (IMESs) for intramuscularelectromyogram recording. IEEE Trans Biomed Eng 2009, 56:159-171.

11. Baker JJ, Scheme E, Englehart K, Hutchinson DT, Greger B: Continuousdetection and decoding of dexterous finger flexions with implantablemyoelectric sensors. IEEE Trans Neural Syst Rehabil Eng 2010, 18:424-432.

12. Kuiken TA, Miller LA, Lipschutz RD, Lock BA, Stubblefield K, Marasco PD,Zhou P, Dumanian GA: Targeted reinnervation for enhanced prostheticarm function in a woman with a proximal amputation: a case study.Lancet 2007, 369:371-380.

13. Kuiken TA, Li G, Lock BA, Lipschutz RD, Miller LA, Stubblefield KA,Englehart KB: Targeted muscle reinnervation for real-time myoelectriccontrol of multifunction artificial arms. JAMA 2009, 301:619-628.

14. Miller LA, Stubblefield KA, Lipschutz RD, Lock BA, Kuiken TA: Improvedmyoelectric prosthesis control using targeted reinnervation surgery: acase series. IEEE Trans Neural Syst Rehabil Eng 2008, 16:46-50.

15. Schady W, Braune S, Watson S, Torebjork HE, Schmidt R: Responsiveness ofthe somatosensory system after nerve injury and amputation in thehuman hand. Ann Neurol 1994, 36:68-75.

16. Rossini PM: Implications of brain plasticity to brain-machine interfacesoperation a potential paradox? Int Rev Neurobiol 2009, 86:81-90.

17. Di Pino G, Guglielmelli E, Rossini PM: Neuroplasticity in amputees: mainimplications on bidirectional interfacing of cybernetic hand prostheses.Prog Neurobiol 2009, 88:114-126.

18. Navarro X, Krueger TB, Lago N, Micera S, Stieglitz T, Dario P: A criticalreview of interfaces with the peripheral nervous system for the controlof neuroprostheses and hybrid bionic systems. J Peripher Nerv Syst 2005,10:229-258.

19. Guiraud D, Stieglitz T, Koch KP, Divoux JL, Rabischong P: An implantableneuroprosthesis for standing and walking in paraplegia: 5-year patientfollow-up. J Neural Eng 2006, 3:268-275.

20. Fisher LE, Miller ME, Bailey SN, Davis JA Jr, Anderson JS, Rhode L, Tyler DJ,Triolo RJ: Standing after spinal cord injury with four-contact nerve-cuffelectrodes for quadriceps stimulation. IEEE Trans Neural Syst Rehabil Eng2008, 16:473-478.

21. Hoffer JA, Stein RB, Haugland MK, Sinkjaer T, Durfee WK, Schwartz AB,Loeb GE, Kantor C: Neural signals for command control and feedback infunctional neuromuscular stimulation: a review. J Rehab Res Develop 1996,33:145-157.

22. Dhillon GS, Lawrence SM, Hutchinson DT, Horch KW: Residual function inperipheral nerve stumps of amputees: implications for neural control ofartificial limbs. J Hand Surg Am 2004, 29:605-615, discussion 616-608.

23. Dhillon GS, Kruger TB, Sandhu JS, Horch KW: Effects of short-term trainingon sensory and motor function in severed nerves of long-term humanamputees. J Neurophysiol 2005, 93:2625-2633.

24. Rossini PM, Micera S, Benvenuto A, Carpaneto J, Cavallo G, Citi L, Cipriani C,Denaro L, Denaro V, Di Pino G, Ferreri F, Guglielmelli E, Hoffmann K-P,Raspopovic S, Rigosa J, Rossini L, Tombini M, Dario P: Double nerveintraneural interface implant on a human amputee for robotic handcontrol. Clin Neurophysiol 2010, 121:777-783.

25. Micera S, Citi L, Rigosa J, Carpaneto J, Raspopovic S, Di Pino G, Rossini L,Yoshida K, Denaro L, Dario P, Rossini PM: Decoding Information FromNeural Signals Recorded Using Intraneural Electrodes: Toward theDevelopment of a Neurocontrolled Hand Prosthesis. Proc IEEE 2010,98:407-417.

26. Micera S, Navarro X, Carpaneto J, Citi L, Tonet O, Rossini PM, Carrozza MC,Hoffmann KP, Vivo M, Yoshida K, Dario P: On the use of longitudinalintrafascicular peripheral interfaces for the control of cybernetic handprostheses in amputees. IEEE Trans Neural Syst Rehabil Eng 2008,16:453-472.

27. Yoshida K, Jovanovic K, Stein RB: Intrafascicular electrodes for stimulationand recording from mudpuppy spinal roots. J Neurosci. Methods 2000,96:47-55.

28. Citi L, Carpaneto J, Yoshida K, Hoffmann KP, Koch KP, Dario P, Micera S: Onthe use of wavelet denoising and spike sorting techniques to processelectroneurographic signals recorded using intraneural electrodes. JNeurosci Methods 2008, 172:294-302.

29. Hoffmann KP, Kock KP: Final report on design consideration of tLIFE2.Tech Rep IBMT; 2005.

30. Tombini M, Rigosa J, Zappasodi F, Porcaro C, Citi L, Carpaneto J, Rossini PM,Micera S: Combined Analysis of Cortical (EEG) and Nerve Stump SignalsImproves Robotic Hand Control. Neurorehabilitation Neural Repair 2011.

31. Rossini PM, Rigosa J, Micera S, Assenza G, Rossini L, Ferreri F: Stump nervesignals during transcranial magnetic motor cortex stimulation recordedin an amputee via longitudinal intrafascicular electrodes. Exp Brain Res2011, 210(1):1-11.

32. Carrozza MC, Cappiello G, Micera S, Edin BB, Beccai L, Cipriani C: Design ofa cybernetic hand for perception and action. Biol Cybern 2006,95:629-644.

33. Coifman RR, Donoho DL: Translation-invariant de-noising. Wavelets andstatistics Springer-Verlag; 1995, 125-50.

34. Diedrich A, Charoensuk W, Brychta RJ, Ertl AC, Shiavi R: Analysis of rawmicroneurographic recordings based on wavelet de-noising techniqueand classification algorithm: wavelet analysis in microneurography. IEEETrans Biomed Eng 2003, 50:41-50.

35. Suner S, Fellows MR, Vargas-Irwin C, Nakata GK, Donoghue JP: Reliability ofsignals from a chronically implanted, silicon-based electrode array innon-human primate primary motor cortex. IEEE Trans Neural Syst RehabilEng 2005, 13:524-541.

36. Cortes C, Vapnik V: Support-Vector Networks. Mach Learn 1995, 20:273-297.37. Chang C, Lin C: LIBSVM: a Library for Support Vector Machines. 2001

[http://www.csie.ntu.edu.tw/~cjlin/libsvm/].

Micera et al. Journal of NeuroEngineering and Rehabilitation 2011, 8:53http://www.jneuroengrehab.com/content/8/1/53

Page 9 of 10

Page 10: Decoding of grasping information from neural …...RESEARCH Open Access Decoding of grasping information from neural signals recorded using peripheral intrafascicular interfaces Silvestro

38. Micera S, Carpaneto J, Raspopovic S: Control of artificial limbs usingperipheral information. IEEE Rev Biomed Eng 2010, 3:48-68.

39. Kemere C, Santhanam G, Yu BM, Afshar A, Ryu SI, Meng TH, Shenoy KV:Detecting neural-state transitions using hidden Markov models formotor cortical prostheses. J Neurophysiol 2008, 100:2441-2452.

40. Cham JG, Branchaud EA, Nenadic Z, Greger B, Andersen RA, Burdick JW:Semi-chronic motorized microdrive and control algorithm forautonomously isolating and maintaining optimal extracellular actionpotentials. J Neurophysiol 2005, 93:570-579.

41. Bossi S, Menciassi A, Koch KP, Hoffmann KP, Yoshida K, Dario P, Micera S:Shape memory alloy microactuation of tf-lIFes: preliminary results. IEEETrans Biomed Eng 2007, 54:1115-1120.

42. Bossi S, Kammer S, Dorge T, Menciassi A, Hoffmann KP, Micera S: Animplantable microactuated intrafascicular electrode for peripheralnerves. IEEE Trans Biomed Eng 2009, 56:2701-2706.

doi:10.1186/1743-0003-8-53Cite this article as: Micera et al.: Decoding of grasping information fromneural signals recorded using peripheral intrafascicular interfaces.Journal of NeuroEngineering and Rehabilitation 2011 8:53.

Submit your next manuscript to BioMed Centraland take full advantage of:

• Convenient online submission

• Thorough peer review

• No space constraints or color figure charges

• Immediate publication on acceptance

• Inclusion in PubMed, CAS, Scopus and Google Scholar

• Research which is freely available for redistribution

Submit your manuscript at www.biomedcentral.com/submit

Micera et al. Journal of NeuroEngineering and Rehabilitation 2011, 8:53http://www.jneuroengrehab.com/content/8/1/53

Page 10 of 10