Upload
others
View
4
Download
0
Embed Size (px)
Citation preview
Inertial Measurement Units and their applications for
Human Performance Analysis
Salvatore SESSAAssistant Professor
Graduate School of Advanced Science and Engineering
Waseda University
Adjunct Professor Dept. of Mechatronics and Robotics
Engineering
E-Just
Background
Limitations of current Skill Training Systems:Reality Trainer VR Simulator
skill evaluation subjectiveobjective
(only time, error evaluation)
objective evaluation system
noneembedded with training systems
(cannot be used for other systems)
used for operation room or clinical
treatment no no
Strong needs:
objective skill evaluation methods both for reality and VR training system
common skill evaluation methods for various medical training applications
skill evaluation methods which might be further implemented in real operation room and clinical treatment
2
Objective Skill
Training System
Application #1
……Application #3
Application #2
Objectives
Develop an Objective Skill Training System
Objective skill evaluation and quantitative feedback
Implementation in Reality training, VR training, and operation room
Assisting clinical diagnosing and treatment
Common, adaptive for multiple medical applications
3
subjective skill evaluation
subjective suggestion
training systemtrainee
expert doctor
Conventional Method (1/2)
Problems:
subjective scoring by expert, using vague criteria
No quantitative feedback
4
Scoring
specific training system
( with basic motion
analysis)trainee
Conventional Method (2/2)
Problems:
low level analysis, without detailed quantitative feedback
skill analysis system embedded with the training system, which can be used only for that specific training system
5
human motion measurement
motion analysis and skill
evaluation
quantitative feedback
various skill training systems/operation room
trainee
expert doctor
Methodology Proposal
Hypothesis:
This methodology could provide more objective and quantitative information to subjects than conventional methods
This methodology can be implemented in regular training, operation room and clinical treatment
The evaluation method is separated from training system, which can be extended to various medical fields
6
Motion Capture Techniques7
outdoor
home
training
room
operation
room
Measurement
Volume
Accuracy
Inertial-based
Electromagnetic-based
Inertial-based
Inertial-based
Camera-based
Inertial-based
Camera-based
Mechanical-based
Mechanical-based
Acoustic-based
Acoustic-based
Acoustic-based
Cost
Camera-based
Optical Capture Systems (2/2)
ProsAccurate and fast
(0.2mm and up to 10,000 fps)Not affected by metalNo electromagnetsMature technologyFlexible placement of markers
ConsNeeds dedicated spaceNot available for portable rentalLonger calibration timesIR Occlusion problemsLines of sight required3dof natively – positional only (3 marker
for achieving attitude calculation)
ExpensiveMarkers need to be reapplied each session
http://www.inition.co.uk/
8
high precisionExpensive, bulky, and
Not sufficient for measuring small object’s orientation
Inertial Measurement Units (2/2)
ProsSelf-containedCost and reliability (improving)Mature technology (sensors)Uses Earth’s magnetic field rather
than transmitterFlexible placementSimple calibration
ConsSize and weight are still critical for someapplicationsAffected by metalDriftAccuracy3dof natively – attitude only
Inertial Measurement Units (IMUs) are a very promising frontier on the wearable and reliable Motion Capture system because can be virtually used everywhere
9
Objectives
Realize an ultra-miniaturized IMU for application where size and weight of the sensor is critical
Compare the performance of the new IMU with Vicon and other commercial systems
Verify that IMU can provide precise and reliable results also in dynamic conditions (RMS error less than 2 Deg)
10
WB-3 (Waseda Bioinstrumentation ver. 3) (2/2)G
yro
scope
(X-Y
axis
)
Gyro
scope
(Z a
xis
)
Accelerometer Magnetometer
20 mm
26
mm
Y
XZ
3-Axis
accelerometer
2-axis
Gyroscope
1-axis
Gyroscope
3-axis
Magnetometer
Range ±2 g ±500 deg/s ±300 deg/s ±4 Gauss
Resolution 12 bit 12 bit 12 bit 12 bitBandwidth 40 Hz 140 Hz 88 Hz 50 HzLinearity ±2% <1% ±0.8% ±0.1%
Noise level <1 bit <1 bit <2 bit <1 bit(*) Z. Lin et al. “ Development of an Ultra-miniaturized Inertial Measurement Unit WB-3 for Human Body Motion Tracking“ SII 2010
11
WB-4 (Waseda Bioinstrumentation ver. 4)
12
Magnetometer
Accelerometer
Gyroscope
17mm
20
mm
Y
XZ
3-Axis accelerometer 3-axis Gyroscope 3-axis Magnetometer
Range ±2/±4/±8 [G]±400/±1600
[deg/s]±4 Gauss
Resolution1[mG/digit] @±2G;2[mG/digit] @±4G;3.9[mG/digit] @±8G
3.2[mV/dps]@±400deg/s;0.8[mV/dps]
@±1600deg/s
12 [bit]
Bandwidth25[Hz]@±2G50[Hz]@±4G500[Hz]@±8G
140 [Hz] 50 [Hz]
Current consumption 0.25[mA] 10.8[mA] 0.9[mA]
Two-layer configuration
WB-4 (Waseda Bioinstrumentation ver. 4)
13
WB-4 IMU Main Board
12mm
WB-4 Wireless IMU (7g)
EKF Quaternion Based (1/5)
Gyroscope
Accelerometer
PredictionRoll-Pitch
Update
kz
ku kx̂ kx
Yaw UpdateMagnetometer
D. Gebre-Egziabher, et al., "A Non-Linear, Two-Step Estimation Algorithm for Calibrating Solid-State Strapdown Magnetometers," presented at the 8th International St. Petersburg Conference on Navigation Systems (IEEE/AIAA), St. Petersburg, Russia, 2001.
D. Gebre-Egziabher, et al., "Calibration of Strapdown magnetometers in Magnetic Field Domain," Journal of Aerospace Engineering, pp. 87-101, 2006.
D. Campolo, et al., "A novel procedure for In-field Calibration of Sourceless Inertial/Magnetic Orientation Tracking Wearable Devices," in 1st IEEE/RAS-EMBS International Conference on Biomedical Robotics and Biomechatronics (BioRob 2006), 2006.
14
EKF Quaternion Based (2/5)
Gyroscope
Accelerometer
PredictionRoll-Pitch
Update
kz
ku kx̂ kx
],,,,,,[ˆ3210 zyxk bbbqqqqx
Updated State vector
],,[ zyxku
Input vector
],,[ zyxk aaaz
Measurement vector
]ˆ,ˆ,ˆ,ˆ,ˆ,ˆ,ˆ[ˆ3210 zyxk bbbqqqqx
Predicted State vector
15
Prediction step
State prediction:
Covariance matrix prediction:
EKF Quaternion Based (3/5)
Gyroscope
Accelerometer
PredictionRoll-Pitch
Update
kz
ku kx̂ kx
kkkk wuxfx ),(ˆ11
1111ˆ
k
T
kkkk QAPAP
13
012
103
230
321
0
2
1
),(
x
zz
yy
xx
b
b
b
qqq
qqq
qqq
qqq
uxf
16
EKF Quaternion Based (4/5)
Gyroscope
Accelerometer
PredictionRoll-Pitch
Update
kz
ku kx̂ kx
Correction step
Sensor model:
December 14, 2010
a
b
na
grg
vgqCGa
vbG
2
4
2
3
2
2
2
41324231
4132
2
4
2
3
2
2
2
4321
42314321
2
4
2
3
2
2
2
1
1
1
)(2)(2
)(2)(2
)(2)(2
qqqqqqqqqqqq
qqqqqqqqqqqq
qqqqqqqqqqqq
qC b
n
The acceleration acting on the body is negligible compared to the gravity acceleration
(*) A. M. Sabatini, "Quaternion-based extended Kalman filter for determining orientation by inertial
and magnetic sensing," IEEE Trans Biomed Eng, vol. 53, pp. 1346-56, Jul 2006
17
EKF Quaternion Based (5/5)
Gyroscope
Accelerometer
PredictionRoll-Pitch
Update
kz
ku kx̂ kx
Correction step
Kalman gain:
State update:
Covariance matrix update:
1)ˆ(ˆ k
T
kkk
T
kkk RHPHHPK
))ˆ((ˆkkkkk xhzKxx
December 14, 2010
kkkk PHKIP ˆ)(
18
Limitations
The acceleration sensor model is valid only when:
“the acceleration acting on the body is negligible compared to the
gravity acceleration”
The Measurement Covariance Matrix represents the noise level of
the accelerometer and is often choose as constant matrix
Hypothesis: The lack on the sensor model for the effects of
external acceleration acting on the body can be compensated
with a dynamic choice of the Measurement Covariance Matrix
1)( k
T
kkk
T
kkk RHPHHPK
December 14, 2010
R-Adaptive algorithm
19
R-Adaptive Algorithm (1/2)
Gyroscope
Accelerometer
PredictionRoll-Pitch
Update
kz
ku kx̂ kx
December 14, 2010
R-Adaptive algorithm
kR
20
R-Adaptive algorithm (2/2)
k
Nk kkk zzN
2
1
2 )(1
1
||z|
| [
m/s
2]
Time [samples]
N=10 (100 ms) Human movements less than 10Hz
M0
Module of the Acceleration
0
Kk-N
2
2
2
00
00
00
k
k
k
kR
1)ˆ(ˆ k
T
kkk
T
kkk RHPHHPK
))ˆ((ˆkkkkk xhzKxx
Adjust the Kalman gain with a term
dependant to the variance of the
acceleration module
K. C. Veluvolu, U. X. Tan, W. T. Latt, C. Y. Shee, and W. T. Ang, "Bandlimited Multiple Fourier Linear Combiner for Real-time
Tremor Compensation," in Engineering in Medicine and Biology Society, 2007. EMBS 2007. 29th Annual International Conference of
the IEEE, 2007, pp. 2847-2850.
D. O. Ibanez, F. P. Baquerin, D. Y. Choi, and C. N. Riviere, "Performance Envelope and Physiological Tremor in Microsurgery," in
Proceedings of the IEEE 32nd Annual Bioengineering Conference, 2006, 2006, pp. 121-122.
21
Experimental Setup (1/2)
Reflective Markers
WB-3Inertiacube
(Intersense )
V0 V3V1V2
X1Z1Y1
Acquisition frequency (Vicon):100 Hz
Experiments
50 sec: free rotations
100 sec: rotations around one axis (X1)
22
Data Processing Model23
EKF (R-Adaptive)
Quaternion to RPY
Data Filtering
WB-3 Raw Data
Attitude calculation
Quaternion to RPY
Data Filtering
ViconRaw Data
Resampling (100Hz)
InterSenseRaw Data
Local frame (WB-3) to Global frame (VICON)
DTW for data synchronization
DTW for data synchronization
Local frame (WB-3) to Global frame (VICON)
+ -- +WB-3 Error
InterSenseError
Sampling time comparison
0.9998
0.77
(*) All the experiment (total 150 sec)
WB-399.98% of samples is
received with the nominal
sample time of 10 ms
InertiaCubeOnly 77.00% of the samples
is received with the nominal
sample time of 25 ms
24
Data Synchronization
(*)J. Kruskall and M. Liberman The Symmetric Time Warping Problem: From Continuous to
DiscreteIn Time Warps, String Edits and Macromolecules: The Theory and Practice of
Sequence Comparison,. Massachusetts: Addison-Wesley Publishing Co., Reading, 1983
25
Free rotation26
Rotations around one axis27
Global Results
RMS Roll Error and standard
deviation[Deg]
RMS Pitch Error and standard
deviation[Deg]
InertiaCube 5.2696 (22.5080) 2.3245 (2.5740)
WB-3 2.7388 (13.2622) 0.9040 (0.9282)
50 sec - Free rotations
RMS Roll Error and standard
deviation [Deg]
RMS Pitch Error and standard
deviation [Deg]
InertiaCube 0.9183 (0.7427) 2.2640 (1.6764)
WB-3 1.0739 (0.9301) 1.2337 (0.9446)
100 sec - 45 rotation from 0 to 90
28
Results
R-Adaptive algorithm allows the WB-3 to achieve better performance than the InertiaCube, especially when the hypothesis of negligible linear accelerations in respect to the gravity is not verified
DTW techniques for data synchronization among the different motion capture systems (WB-3, Intersense, Vicon) was successfully applied
WB-3 can measure the human body movements virtually everywhere, not only in a structured environment (Vicon)
No calibration procedures
29
Single Sensor Demo30
X Y
Z
Sensor
Joint
Link
Kinematic Model (Upper Body)31
Kinematic Model Demo32
Objectiveupper limbs motion measurement
motion analysis and skill
evaluation
quantitative feedback
training devicesor operation room
trainee
expert doctor
Objectively evaluate operative skills during regular training
Provide quantitative information feedback to subjects
Further implemented in operation room for skill evaluation
33
Experimental setup with WB-3 system*
Peg-board training platform
Experimental Evaluation
Pipe-cleaner training platform
* Cooperation with Prof. Hashizume at Kyushu University
34
Box trainer: Endowork-pro II
Training task:
peg-board training
pipe-cleaner training
Subject:
Surgeon: 5
Novice: 11
Data Processing Model35
Evaluated kinematics parameters:
Joint angle
Joint angular speed
Joint angular speed frequency
Power spectrum density of joint angular speed
Feature Extraction
Feature Normalization
Principal Component Analysis
Linear Discriminant Analysis
Lm(k)
Sm(k)
Cn(k)
G (k)
Xi(t)
Feature Processing
Expertise Classification
Data Filtering
xi(t)
Pre-processing
Results
Parameter left side right side
Shoulder average angular speed ○ ○
Shoulder angular speed CDF ○ ○
Shoulder peak frequency of angular speed ○ ○
Shoulder used efficiency × ○
○ Significance
× No significance
peg-board training (significant parameters: p<0.05)
Parameter left side right side
Shoulder angle standard deviation × ○
Shoulder angle range ○ ×
Shoulder peak frequency of angular speed ○ ○
Shoulder used efficiency × ○
pipe-cleaner training (significant parameters: p<0.05)
No significance found in the cases of wrist and elbow.
The movements of shoulder are strongly related to the operative skills
36
Task Subject QUANTITYSuccessful
CaseFailed Case
Correct Rate
Peg-board
expert 5 5 0 100%novice 11 10 1 90.9%
total 16 15 1 93.75%
Task Subject QUANTITYSuccessful
CaseFailed Case
Correct Rate
Pipe-cleaner expert 5 4 1 80%novice 11 11 0 100%total 16 15 1 93.75%
TBME(submitted), ROBIO 2010
Peg-board training task
Pipe-cleaner training task
Classification37
Biomechanics Analysis38
Feature Extraction
Feature Normalization
Principal Component Analysis
Linear Discriminant Analysis
Lm(k)
Sm(k)
Cn(k)
G (k)
Xi(t)
Feature Processing
Expertise Classification
Data Filtering
xi(t)
Pre-processing
Biomechanics analysis
Biomechanics Analysis39
Feature Extraction
Feature Normalization
Principal Component Analysis
Linear Discriminant Analysis
Lm(k)
Sm(k)
Cn(k)
G (k)
Xi(t)
Feature Processing
Expertise Classification
Data Filtering
xi(t)
Pre-processing
Objectives:
analyze the biomechanics features of surgical gesture
give more insight to the operation skills
Inverse Dynamics Sequence
WB-3
measure joint angles by using WB-3 IMU on each human segment
Kinematics Data
40
Multi-Joint Dynamicsdt
d
Angles Velocities Accelerations
dt
dMusculoskeletal
Geometry
Tendon forcesMoments
Biomechanics Analysis
TricepsLong headMedialLateral
BicepsLong head Short head
Brachialis
Shoulder
Elbow
41
Multi-Joint Dynamicsdt
d
Angles Velocities Accelerations
dt
dMusculoskeletal
Geometry
Tendon forcesMoments
Biomechanics Analysis42
Time (s)50 10 2015 25 30 35
0
5
10
15
20
Jo
int
mo
me
nt
(N*m
) ShoulderElbow
Objectivehand motion measurement
motion analysis and skill
evaluation
quantitative feedback
Neurosurgery training platform
trainee
expert doctor
43
Objectively evaluate operative skills by analyzing hand motion
Provide quantitative information feedback to subjects
Further implementation in operation room for real time evaluation
Microscope
WB-3 IMU
Bipolar Forceps
Training platform
Experimental Evaluation
3 different training target
* Cooperation with Prof. Iseki at Tokyo Women Medical University
experimental setup
Basic training:pick and place task
44
Global Evaluation45
SubjN
Surgeon
SubjNParam
ParamScore
EMBC 2009, MICCAI 2009, IROS 2009
Objectively evaluate mastication pattern and skills
Provide quantitative information for better diagnosis
Assist jaw symptoms treatment
Objectivejaw motion measurement
motion analysis and skill
evaluation
quantitative feedback
mastication training diagnosis
trainee
expert doctor
46
Experimental Evaluation
marshmallow biscuit almond
WB-3
front side
Experimental setup
Types of food with different hardness
47
Experimental setup
WB-3 attached to mandible by adhesive medical tape
Subject’s head lean on wall during chewing the food
Task
Free chewing of three types of food with different hardness
Three types of foods:
marshmallow: 8.0 ± 1.0 g
biscuit: 7.0 ± 0.1 g
almond:4.3 ± 0.4 g
Results
JCS 6(8) 2010, IRIS 2010
ParametersMarshmallow
(soft)Biscuit
(intermediate)Almond(hard)
Chewing Time
Chewing Frequency
Rotation Energy(PSD of |ωx|)
Translation Energy(PSD of |a|)
Mouth Opening Angle no significant difference
Big Value Intermediate Value Small Valueωx
48
Quantitative information to the doctors for realizing a better diagnosis
Mastication performance information for jaw skill rehabilitation
marshmallow
biscuit
almond
0.5
PSD
ωx
[(deg/s
)2/H
z]
8
6
4
2
0
x 104
1 1.5 2 2.5 3Frequency [Hz]
Chewing Frequency
ωx
Chewing frequency was analyzed from the Power Spectrum Density
(PSD) of jaw’s angular speed about x-axis
Frequency of the peak as the chewing frequency for each food
Subject1 2 3 4 5 6 7 8 9
Mean P
SD
ωx
(norm
.)
1.5
1
0.5
0
2
2.5
MarshmallowBiscuitAlmond
PSD of ωx
Mean P
SD
ωx
(norm
.)
Marsh. Biscuit Almond
< 0.01p < 0.01p< 0.01p
2
1.5
1
0.5
0
ωx
Journal paper: Z.Lin, et al, JCS 6(8).
All the subjects used more rotation energy when eating hard than soft food
Subjects used significantly more rotation energy when they were eating hard
food
Z X
Y
Z X
Y
Z X
Y
Z X
Y
WB-4 WB-4
front side
Mastication Skill Evaluation51
Task:
freely chew gum (60s)
chew gum only using the teeth on right side (60s)
chew gum only using the teeth on left side (60s)Two WB-4 IMUs
one for compensating the head motion (on forehead)
one for measuring the jaw motion (on mandible)
Results52
Evaluation parameters:chewing patternchewing frequencymouth opening angleetc.
Chewing
pattern 1Chewing
pattern 2
Chewing
pattern3
My research team
Prof. Atsuo TAKANISHI ( 教授 高西 淳夫 )
Prof. Massimiliano ZECCA (准教授 ゼッカ・マッシミリアーノ)
Dr. Hiroyuki ISHII (次席研究員(研究院講師) 石井 裕之)
D3 Zhuohua LIN (林 焯华)
D1 Luca BARTOLOMEO (バルトロメオ・ルカ)M2 Yoshikazu MUKAEDA (迎田 美和)
M1 Yuto SUZUKI (鈴木 悠人)
53