Improving the Performance of Hand Posture Classification by Perimeter Sensor with sEMG

Hwiyong Choi and Sangyoon Lee Department of Mechanical Design and Production Engineering

Konkuk University Seoul, Korea

{genichy, slee}@konkuk.ac.kr Abstract - This paper reports a work for improving the performance of the hand posture classifier by using perimeter of the human forearm. Misclassification occurs due to four factors: residual deformation of the muscle after the muscle activation, sensor locations, error from the voltage regulator and the transition state during posture change. In order to reduce the effect of the factors, a sensor location which gives low residual muscle deformation was selected and one channel of sEMG was employed. The least square regression was used for removing the transition state. The proposed method was verified through simulation with pre-acquired data sets. Index Terms - hand posture classification; perimeter change of the forearm; sEMG; least square regression

I. INTRODUCTION

Predicting human intention has been developed in many research areas for various purposes [1-3]. So far, among many human machine interfaces for behavior measurement or for assistive device, sEMG has been one of the most popular features. In spite of many advantages, it has disadvantages. It is affected by posture of the arm, electrode location, fatigue of the muscle and noise [4]. So it needs expertise of the sensor location and signal processing, which makes users more difficult to utilize it [5].

For these reasons, other kinds of bio-signals have been developed. Some researchers introduced optical sensor. Infrared LED injects light to the muscles and backscattered light is collected by photodiode. Depending on the muscle activation level, intensity of collected light is different [6-8]. Other researcher used stiffness of the muscles which is changed by the muscle force [9]. Kong used similar strategy but he simplified the sensor and reduced the cost for implementation [10]. And Li [11] and Seo [5] used FSR (Force sensitive resistor) to predict hand motion and leg posture respectively.

Similarly, in our previous and current research, we have measured perimeter change of the forearm due to the muscle density change during muscle activation. But our focus is classifying one of main functions of the human hand: hand grasp. First of all, the sensor was implemented to measure perimeter change of the forearm with low cost and much simplified shape. Hand postures to be classified were chosen [12]. And 24 sets were tested to classify hand postures.

However many misclassifications occurred in the last 10 sets. It was due to the remained muscle volume change after the muscle contractions (we call it ‘residual muscle deformation’) [13].

So, it is required to calibrate the perimeter sensor periodically to compensate the effect of the residual muscle deformation. However finding exact calibration time also important. In order to accomplish it, one channel of sEMG was employed to detect the calibration time. In addition, other misclassification generating factors were analyzed and minimized.

II. METHODS AND MATERIALS

A. Perimeter change of the forearm Fig.1 shows an example of the residual muscle deformation. One can see there is a large signal change by the muscle shrinkage which has larger magnitude than the signal change induced by hand postures. It lowers the performance of the classifier.

Fig. 1 An example of large signal offset induced by residual muscle deformation on the forearm muscle after the muscle activation. P1~P4 indicates voltage output of each posture from the perimeter sensor. Sampling time is 100ms.

B. Measurement of the perimeter change and selection of hand postures

Fig. 2 A perimeter sensor: (a) A strain gauge is attached on the rubber pad. (b) The strain gauge is directly connected to a single resistor. Voltage difference between node 2 and 3 varies depending on the perimeter of the forearm.

Fig. 3 Sensor locations for the perimeter sensor.

From the point of view of the classification problem, perimeter change of the forearm contains information such as hand posture and grasp force generated from the muscle [14]. To confirm whether the perimeter sensor can read such kinds of information, we first measured a perimeter change of each posture with 3 different grasp forces. Two perimeter sensors that we have used (see Fig.2) were employed.

They read the strain due to the increase or decrease of the forearm’s perimeter using a strain gauge. The signal was sent to ADC (analog to digital conversion) card (NI PXI-6259, National instrument Corp.) and recorded. For the landmark of sensor locations, anterior surface area of a subject’s (male, 24 years old, no history of injuries related to the forearm muscle) forearm was divided into five equal areas (see Fig. 3). And sensors were attached to sensor location of 4 and 5 we labeled. We preferred not to attach the sensors on the sensor location from 1 to 3. Because these sensor locations include large tendon movement. Used hand postures were relaxed muscle posture (P1), pinch grasp with three fingers closed (P2), cylindrical grasp (P3) and spherical grasp (P4) (see Fig.4).

Three different kinds of measurements were prepared. Measurement 1 is for measuring the effect of muscle force under the same posture which is P2. And in measurement 2 and 3, under the same condition, only the posture (P2) is switched to other posture (P3 and P4). To avoid the effect of

the residual deformation which are mentioned earlier, sufficient relax of muscle is needed after one measurement. Because, results with the residual deformation cannot be accepted.

• Measurement 1: Repeat following postures 5 times with grasp force: {P2F1, P2F2, P2F3}. The ‘P’ and ‘F’ stands for posture and relative grasp force respectively. F0 and F4 stands for no muscle activation and with the maximum force a subject can generate. However using the maximum force is not recommended because it cannot be repeated many times and it induces muscle fatigue fast.

• Measurement 2 and 3: Follow the same steps as measurement 1. But in measurement 2 and measurement 3, replace P2 with P3 and P4 respectively.

Measurement 1, 2 and 3 were executed independently. Experimental procedures were:

• Attach the perimeter sensors and sit on the chair. After that adopt the body posture and arm posture as illustrated in Fig.5. And gaze the pc-monitor.

• The pc-monitor displays pre-programmed commands from F0 to F3. At every 6 second, program changes the command.

• A subject makes the postures by following the commands. During that time data acquisition software programmed by LabVIEW (version 10.0.1, National Instrument Corp.) records the data at every 100ms through ADC card (same model) and carries out signal processing. Signal processing includes low pass filtering and RMS calculation.

• When 5 sets are finished, quit the measurement.

Fig. 4 Used postures: (a) Relaxed muscle posture. (b) Pinch grasp with three fingers closed. (c) Spherical grasp. (d) Cylindrical grasp.

Fig. 5 Body and arm posture.

Fig. 6 Average value with two standard deviation error bar of voltage output of the each perimeter sensor. Data during transition state was neglected. The digit after the ‘S’ stands for sensor location. And the digit after the ‘P’ and the ‘F’ stands for posture and grasp force respectively: (a) Perimeter sensor 4. (b) Perimeter sensor 5.

Fig.6 shows the result plots. Postures with relatively large grasp force give relatively large increase of perimeter. P3 generated the largest perimeter change at every grasp force followed by P2 and P4. Behavior of perimeter sensor 4 (perimeter sensor on the sensor location 4) and perimeter sensor 5 (perimeter sensor on the sensor location5) was very similar to each other (correlated). However, some classes were overlapped with each other. So, we selected postures for the future experiments: P1, P2F3, P3F3 and P4F2.

C. Misclassification generating factors and classifier Normally, to test a classification algorithm, two types of data are needed: training data (for constructing parameters of a classifier) and real data which will be classified and simulated. Moreover if there are any additional misclassification

generating factors, then it should be considered again. Following procedures are for acquisition of the training data and the real data for our algorithm simulation. Total 2 sessions were executed and 15 sets consisted 1 session:

• Attach two perimeter sensors on the sensor location of 4 and 5 respectively and one electrode is attached on the flexor digitorum superficialis which has given sEMG signal for all non-P1 postures.

• For the voltage reference of the sEMG measurement, the other electrode is attached on the back of a subject’s hand.

• Electrodes are connected to sEMG acquisition device (AM530, LAXTHA Inc.).

• And the subject adopts the body posture represented in Fig. 5 gazing the subject’s eyes to the PC monitor.

• Pre-programmed steps start when start button is pressed. The first stage is acquisition of training data. During this stage, program records perimeter change and sEMG data.

• When training step is over, it begins to display the series of names of the postures that we selected (P1 – P2F3 – P3F3 – P4F2) changing the names at every 4 second. The subject makes the postures displayed on the pc-monitor. Again, ADC card (same model) records perimeter change and sEMG.

Fig.7 shows training data of the first session and the second session. And Fig.8 is the acquired data during the sessions.

Fig. 7 Training data: 50 samples were acquired for training of each posture. Sampling time was 100ms: (a, left) First session, perimeter sensor 4. (a, right) First session, perimeter sensor 5. (b, left) Second session, perimeter sensor 4. Large residual muscle deformation occurred during training. (b, right) Second session, perimeter sensor 5.

Fig. 8 Acquired data sets with the sampling time of 100ms: (a) First session. Perimeter and sEMG data. Small residual muscle deformation occurred on the sensor location 4. (b) Second session. Perimeter and sEMG data. Large residual muscle deformation occurred on the sensor location 4 same as training data. Small offset error included in the data of perimeter sensor 5. It originated mainly from the voltage regulator error. The analysis of acquired datasets gives following results. They should be reflected to the classifier.

First, large residual muscle deformation occurred on the sensor location 4 (even during the training stage). On the other hand, on the sensor location 5, the amount of variation was much smaller than that of sensor location 4. But regardless of residual muscle deformation, magnitude of each signal change induced by the hand posture of both perimeter sensors was almost invariable and correlated with each other as shown earlier in the Fig.6. So employing perimeter sensor 4 can lower the performance of classifier (so the sensor 4 was removed).

Second, transition state at every posture change should be removed. Fig.9 is magnified view of the perimeter sensor data. At every posture change, there are samples of transition state. They are out of scope of the posture data so they can produce misclassification.

The last one which can generate misclassification is error from the voltage regulator. Since the strain gauge is directly connected to a single resistor, voltage difference between two nodes of the strain gauge reflects voltage level change of the voltage regulator. High precision voltage regulator has been used, but sometimes regulated voltage level has changed slightly.

Fig. 9 1 set of perimeter data: At every posture change, transition state exists (yellow highlighted samples).

III. SIMULATION AND RESULTS

To reduce the effect of the factors including residual deformation of the muscle, we suggested a method. Fig.10 illustrates flow chart of the method.

The key point of the method is: invariance of the perimeter sensor’s signal change induced by the hand postures in spite of the residual muscle deformation.

It repeats following steps: starting with reading the i-th data of perimeter sensor 5 and sEMG data from the data sets, it checks whether the muscle is now in the relaxed muscle state or not. Checking the muscle activation can be considered as binary classification problem between P1 and non-P1 classes. Classifier adopts maximum likelihood parameterized by the sEMG training data. If the muscle is in the relaxed muscle state, then it calibrates the signal of the perimeter sensor 5 to zero in order to compensate the offset error originated from the residual muscle deformation or voltage regulator. If not, this method classifies hand posture without calibration. During the steps residual muscle deformation and error from the voltage regulator expected to be removed.

The next step is classification of the hand postures. Different from the classification of the muscle activation, hand posture classification is multivariate classification. It also uses maximum likelihood, but this time, parameterized by the perimeter training data.

Before printing an classification result, it detects the transition state using absolute value of the slope coefficient of the least-square regression. If a current iteration is in the transition state, then it substitutes current classification result with the previous one. If not, it outputs the current classification result. However, to the comparison group, grey colored steps in the Fig.10 were neglected. Software MATLAB (version 2009b, MathWorks Inc.) was used for simulation. Training and input data for the simulation was same as we acquired.

Fig. 10 Flow chart of the suggested method: CR(i) = classification result when an iterator is an ‘i’.

Fig.11 represents simulation results and comparison of with and without the suggested method. Each perimeter data set (top of Fig.11(a) and Fig.11(b)) had different error shape. And error of the transition state was common to both of them (many misclassified dots between posture changes in the middle of Fig.11(a) and Fig.11(b)).

Almost of samples in the transition state were well corrected and so did the signal offset due to the residual muscle deformation and error of voltage regulator (bottom of Fig.11(a) and Fig.11(b)).

Time delay between the motion commands plot and the classification results plot includes the recognition time and the reaction time of the subject. But sometimes this method was not effective. At the 8th set of the first session, many misclassifications were occurred. They were originated from wrong calibration and lasted before the next calibration. It may be related to detection of the muscle activation. This issue will be covered in the future work.

Fig. 11 (a) Simulation results with the first session data. (b) Simulation results with the second session data: Top of (a) and (b) shows used perimeter data. Stair shaped ‘- -’ represents the motion commands displayed in the pc-monitor. Colored dots on the middle and bottom of (a), (b) are classification results.

IV. CONCLUSION

We presented a method which reduces the misclassification of the hand posture classifier. This classifier uses the perimeter data of the forearm. Major misclassification making factors are residual muscle deformation, sensor location, transition state during posture change and voltage regulator. The method for reducing the factors was proposed and simulated based on pre-acquired data sets. The perimeter sensor on the sensor location 4 gives large residual muscle deformation, so it was removed. The suggested method compensated the first and the fourth factor using one sEMG channel as a calibration time detector. And the third factor was corrected using the absolute value of slope coefficient of the least square regression. The simulation results show the suggested method is effective to reducing the factors. But, sometimes calibration error occurs, which will be considered in the future work.

ACKNOWLEDGMENT

This paper was supported by Konkuk University in 2012 and Leading Foreign Research Institute Recruitment Program through the National Research Foundation of Korea(NRF) funded by the Ministry of Education, Science and Technology(MEST) (2011-00260).

REFERENCES [1] P. Parker, K. Englehart, B. Hudgins, “Myoeletric signal processing for

control of powered limb prostheses”, J. ELECTROMYOGR KINES, vol. 16, no. 6, pp. 541-548, Dec. 2006.H. Simpson, Dumb Robots, 3rd ed., Springfield: UOS Press, 2004, pp.6-9.

[2] B. Blankertz, et al., “A note on brain actuated spelling with the Berlin brain-computer interface”, Universal Access in Human-Computer Interaction. Ambient Interaction, Lecture Notes in Computer Science, vol. 4555/2007, pp. 759-768, 2007.

[3] Y. Shin, Y. Kim, E. Y. Kim, “Automatic textile image annotation by predicting emotional concepts from visual features”, IMAGE VISION COMPUT, vol. 28, no.3, pp. 526-537, Mar. 2010.

[4] C. Castellini and P. Smagt, “Surface EMG in advanced hand prosthetics”, BioCybernetics, vol. 100, no.1, Jan. 2009.

[5] A. Seo, H. Jang, W. Kim, C. Han and J. Han, “Development and Verification of a volume Sensor for measuring Human Behavior”, IJPEM, vol. 13, no. 6, pp. 899-904, Jun. 2012.

[6] A. Chianura and M. E. Giardini, “An electrooptical muscle contraction sensor”, MED BIOL ENG, Vol. 48, no. 7, pp. 731-734, May. 2010.

[7] H. Han, S. Kwon, J. Kim, “Optical Muscle Activation Sensor for Bionic Applications”, IJPEM, vol. 26, no. 26, pp. 15-21, Jul. 2009.

[8] Belau M, Ninck M, Hering G, Splinelli L, Contini D, Torricelli A and Gisler T, “Noninvasive observation of skeletal muscle contraction using near-infrared time-resolved reflectance and diffusing-wave spectroscopy”, J Biomed Opt, vol.15, no.5, Sep-Oct.2010

[9] S. Moromugi, et al., “Soft Power Suit for Knee Motion Assistance”, INT J HWRS , vol.4, no.4 Dec. 2003.

[10] K. Kong and D. Jeon, "Design and Control of an Exoskeleton for the Elderly and Patients," Mechatronics, IEEE/ASME Trans. Mechatronics , vol.11, no.4, pp.428-432, Aug. 2006.

[11] N. Li, D. Yang, L. Jiang, H. Liu and H, Cai, “Combined Use of FSR Sensor Array and SVM Classifier for Finger Motion Recognition Based on Pressure Distribution Map”, J BIONIC ENG, vol.9, no.1, Mar.2012

[12] H. Choi and S. Lee, “Predicting Hand Motions Using Perimeter Change of Forearm”, presented at the KRoC 2012, Gangneung, Korea, FA-3-08, Jul. 2012.

[13] H. Choi and S. Lee, “Classification of Hand Grasp Using Perimeter Change of the Forearm”, presented at the URAI 2012, Daejeon, Korea, TP-24, Nov. 2012.

[14] S. Moromugi, et al., “Device for Assisting Grasping Function (2nd report : manuuverability Evaluation)” presented at the ICCAS 2003, Gyeongju, Korea, pp. 2665-2669, Oct. 2003.