[Skolkovo Robotics 2015 Day 1] Тетерюков Дмитрий | Dzmitry Tsetserukou
Preview:
Citation preview
- 1. Dzmitry Tsetserukou Assistant Professor Intelligent Space
Robotics Laboratory NurseSim: Virtual Reality Simulator for Nurse
Training with Haptic Feedback The best way to predict the future is
to invent it. Steve Jobs Dzmitry Tsetserukou, Yuuki Nakagawa and
Kazuhiko Terashima
- 2. EIIRIS: Electronics-Inspired Interdisciplinary Research
Institute Professor K.Terashima Professor J. Miura M.S. Y. Nakagawa
M.S. S. HosokawaPhD J. Sugiyama M.S. H. Sugiura
- 3. Introduction Background: Serious understaffing at the care
person Japan needs twice amount of care person next 20 years. Its
necessary to raise international human resources having nursing
care skill. Language difference causes a barrier of providing a
technical assistance and training. Purpose: Develop novel nurse
training system with haptic technology We develop nurse training
simulator with sense of force but without a language. Simulator
helps nurse training and achieve shortening of training period.
Reduction personal costs by using this simulator. Haptic: Tactile
feedback technology which takes advantage of the sense of touch.
Care person from Indonesia
- 4. NurseSim: Nurse Training Simulator Using this haptic
interface, we achieve effective and low cost training system
compared to using a human manikin or a robot. It shows an optimal
motion with sense of force, effective and easy to understand
teaching is realized. User wearing ExoInterface Virtual Care
Receiver NurseSimNursing 3D Simulator as Education System
ExoInterface: Wearable Haptic Interface Haptic Interaction Virtual
weight, 3D graphics Posture information 3D Glass 3D Projector IR
Emitter Xtion: Motion Capture Camera
- 5. Wearable Haptic Interface Elbow Unit Wrist Unit Fixing Plate
Flexible Belt Belt Shaft DC Motor Unit Base Timing Belt Pulley
Bearing Motor Holder DC Motor Bearing Connector Unit Base Motor
Shaft Elbow Unit Wrist Unit We developed wearable haptic interface
for an interaction in VR environment. The flexible belt is
connected from upper arm to wrist, and from wrist to hand. DC
motors embedded in the interface pull flexible belts, then joint
torques are generated. This device reproduces the human muscle
structure by generating extensor and flexor muscle torques at the
users arm joint. Doesnt limit users movement and can be used all
over the place.
- 6. 0 20 40 60 80 100 120 140 160 180 0.2 0.4 0.6 0.8 1 Elbow
Angle[deg] BeltLength[m] 0 20 40 60 80 100 120 140 160 180 0 0.5 1
1.5 2 2.5 Elbow Angle[deg] JointTorque[Nm] Interface Geometrical
Modeling It is necessary to prepare not only devices but also
geometric models of that for generating the arbitrary joint torques
in human arm. From this interface model, we calculated belt length
and joint torque by MATLAB. This model helps for dynamic
computation or simulation in virtual environments like bringing
sense of physical interaction. e eu bul eml fal wuF uwuF sin eud
Upper Motor Belt u Wrist Joint Elbow JointHand Upper Arm e Elbow
Unit Geometry Model eufa bueuemfa wufaeu bl ldll Fl 2 cos sin 2222
1 Elbow Joint Torque is Belt Length and Joint Torque
- 7. AR Application Experiment D/AA/D USB PC Sensor Circuit Motor
Driver Motor Flex Sensor WireWire Camera HMD VGA Wire AR Experiment
Configuration We designed an application using AR technology. The
user wears a haptic interface and HMD with a camera. When camera
recognizes AR marker, virtual object appears on it. When a virtual
ball touches surface of the hand, the interface reproduces force
feedback to elbow and wrist. AR (Augmented Reality): Technology
which combines real world and virtual world, and shows its mixed
image to the user.
- 8. Dynamics Calculation Experiment Human Model Foot Lower Leg
Upper Leg Waist Upper Body Hand Lower Arm Upper Arm Head To create
virtual nurse training simulator, we developed the dynamics
calculation space and 3D human body model with ODE. Using ODE,
accurate dynamics calculation and good visualization are
implemented. The model has 9 DOF in its arm joint and waist and its
controlled by the users posture. Xtion ScreenOpenCV OpenNI Angle
Calculation Algorithm P control RGB, Depth(m) Joint Point(x,y,z)
Joint Angle(rad) Angular Velocity (rad/s) Joint Torque(Nm)
Current(A) Virtual Object Load(N) Human Model ODE Space Interface
Model Interface RGB ODE System Block Diagram ODE(Open Dynamics
Engine): Open source physical engine which is used for simulating
the dynamic interactions between bodies in space.
- 9. Lifting Rigid Box and Multijoint Object The user can
transform posture to optimal joint angle referring to ODEs visual
information. We can design a care receiver body model with the same
method and realize interaction between human and human.
- 10. NurseSim Wrist Elbow Whole Body The high precision tracking
by optical motion capture Nurse care scenario can be changed easily
in Unity Learning by the force feedback interface Effective nurse
training can be realized PC1 PC2 Camera Hub Display Motor Driver
Trainer with Markers Skeleton data Voltage Current OptiTrack Camera
Image Force Feedback Image Interface Motors
- 11. Psychophysical experiment Experimental results y = d - a-
d( ) 1+ x / c( ) b a = -0.0020187 b = 3.05279 c = 4.30725 d =
1.05594 a = -0.0020187 b = 3.05279 c = 4.30725 d = 1.05594 a =
0.046811 b = 4.69180 c = 4.48232 d = 0.97842 Elbow unit only Wrist
unit only Both units Sigmoid Curve (4-parameter logistic curve)
Elbow Unit Wrist Unit Both Units Upper DL [g] 135.66 139.26 133.92
Lower DL [g] 78.86 94.78 88.28 0 0.25 0.5 0.75 1 0 20 40 60 80 100
Rateof"heavy"response Weight [g] Rate of heavy Sigmoid Curve 20 60
100 140 180 220 0 0.25 0.5 0.75 1 0 20 40 60 80 100
Rateof"heavy"response Weight [g] Rate of heavy Sigmoid Curve 20 60
100 140 180 220 0 0.25 0.5 0.75 1 0 20 40 60 80 100
Rateof"heavy"response weight [g] Rate of heavy Sigmoid curve 20 60
100 140 180 220
- 12. AR experiment comparing the same size ball Experiment
outline Left hand weight Right hand weight Content Virtual ball
bouncing to the left and right hand Question Which weight: in left
hand or right hand is heavier? Answers Left weight is heavy/
Slightly heavier / Same / Right is heavy / Right is heavy Number of
repetitions 25 times Simultaneous comparison Experimental
conditions Haptic interfaces at wrist and elbow joints HMDwearable
web camera EarplugsTo prevent noise from actuators Left and right
hand are bent at 90 degree Weight of ball 5 stages(80~160g)random
order
- 13. AR: experiment comparing the same size of ball Experimental
results LeftHandWeight Right Hand Weight 80g 100g 120g 140g 160g
80g 3.2 4.8 2.2 5.0 4.6 100g 2.8 4.0 4.6 4.6 5.0 120g 1.4 3.4 3.2
4.2 3.8 140g 1.4 2.6 3.4 4.0 4.0 160g 1.0 2.2 3.0 2.2 2.8
LeftHandWeight Right Hand Weight 80g 100g 120g 140g 160g 80g 0.95
0.80 0.55 1.00 0.90 100g 0.80 0.75 0.85 0.85 1.00 120g 0.85 0.65
0.95 0.95 0.95 140g 0.90 0.85 0.65 0.75 1.00 160g 1.00 0.70 0.75
0.95 0.95 LeftHandWeight Right Hand Weight 80g 100g 120g 140g 160g
80g 3 4 4 5 5 100g 2 3 4 4 5 120g 2 2 3 4 4 140g 1 2 2 3 4 160g 1 1
2 2 3 Percentage of correct answers Overall percentage of correct
answers 85 Correct value, n Experimental value, m p =1- m-n 4
- 14. NurseSim3D
- 15. Best Student Presentation Award, Oct. 24-25, Irago 2013
Awards Student Excellence Award, March 6, 2014 Best paper Award,
April 7, 2010
- 16. LinkTouch3D: a Novel Wearable Haptic Display with Parallel
Mechanism for Presentation of Three-DOF Tactile Feedback at the
Fingerpad
- 17. ! Thank you for your attention!