ISMAR 2014: Comprehensive Workspace Calibration for Visuo-Haptic Augmented Reality

University ofSouth Australia

Comprehensive Workspace Calibration for Visuo-Haptic Augmented RealityUlrich Eck1, Frieder Pankratz2, Christian Sandor3, Gudrun Klinker2, Hamid Laga1 1 PBRC, University of South Australia, Australia 2 FAR, Technische Universität München, Germany3 IMD Lab, Nara Institute of Science and Technology, Japan !10/09/2014, IEEE International Symposium on Mixed and Augmented Reality, Munich

1

University ofSouth Australia

Visuo-Haptic Augmented Reality (VHAR)

2

[Sandor C., TEDx Adelaide, 2010]

University ofSouth Australia

Contributions

3

Previous work: position calibration only Improved: with orientation calibration

Novel algorithm for orientation calibration: >40% improvement

Position calibration: ~20% improvement

Time-delay compensation between sensors: faster and better calibration procedure

University ofSouth Australia 4

Related Work

[Sandor, C. et al., IEICE 2007][Vallino, J. et al., ICMCS 1998]

Early examples of visuo-haptic co-location

University ofSouth Australia

PHANToM Haptic Device: Joint Angles

5

University ofSouth Australia

PHANToM Haptic Device: Gimbal Angles

6

University ofSouth Australia

PHANToM Haptic Device: Sensor Errors

7

University ofSouth Australia 8

Related Work

[Harders, M. et al., TVCG 2009][Ikits, M. et al., EG 2003]

Compensation of sensor errors

University ofSouth Australia

Missing: Orientation Calibration

9

Position calibration only With orientation calibration

Orientation Error in Degrees

University ofSouth Australia

Haptic Device Calibration

External tracker provides reference pose

Forward kinematics model calculates stylus pose from sensor measurements

Sensor errors are modelled as linear or quadratic function

Sensor errors are compensated by minimizing the distance to the reference

Complete coverage of sensor ranges required

Accurate pose correspondences required for good calibration results

10

External Tracker (ET)

ETorigin

HIPpose

HIPtarget

Haptic Device (HD)

Camera (C)

Ctarget

HDorigin

University ofSouth Australia

Challenge: Unsynchronized Sensor Input

Unknown delay between haptic device and external tracker

Received measurements do not match during movement

Related work

tedious point and hold process for sampling corresponding measurements

Our solution

determine time-delay between sensors first and compensate it in software to allow users to move freely during the calibration process

11

University ofSouth Australia

Calibration Procedure

Tooltip (Tuceryan et al., 1995)

Absolute Orientation (Horn, 1987)

Time-Delay Estimation (Huber et al., 2012)

Joint-Angles (Harders et al., 2009)

Reference Orientation

Gimbal-Angle Errors

12

External Tracker (ET)

ETorigin

HIPpose

HIPtarget

Haptic Device (HD)

Camera (C)

Ctarget

HDorigin

University ofSouth Australia

Calibration Procedure

Tooltip (Tuceryan et al., 1995)

Absolute Orientation (Horn, 1987)

Time-Delay Estimation (Huber et al., 2012)

Joint-Angles (Harders et al., 2009)

Reference Orientation

Gimbal-Angle Errors

13

External Tracker (ET)

ETorigin

HIPpose

HIPtarget

Haptic Device (HD)

Camera (C)

Ctarget

HDorigin

University ofSouth Australia

Time-Delay Estimation

14

External Tracker (ET)

ETorigin

HIP

Haptic Device (HD)

HDorigin

University ofSouth Australia 15

Time-delay estimation

[Huber, M et al., ISMAR 2009]

University ofSouth Australia 16

Time-aware dataflow network execution model

[Puskta, D et al., ISMAR 2007]

University ofSouth Australia

Calibration Procedure

Tooltip (Tuceryan et al., 1995)

Absolute Orientation (Horn, 1987)

Time-Delay Estimation (Huber et al., 2012)

Joint-Angles (Harders et al., 2009)

Reference Orientation

Gimbal-Angle Errors

17

External Tracker (ET)

ETorigin

HIPpose

HIPtarget

Haptic Device (HD)

Camera (C)

Ctarget

HDorigin

University ofSouth Australia

Orientation Reference Estimation

18

External Tracker (ET)

ETorigin

HIP

Haptic Device (HD)

HDorigin

University ofSouth Australia 19

Reference Orientation

x

y

z

xy

z

HIPpose

HIPtarget

REFzaxis

x

y

z

xy

z

HIPorigin

REFzaxis

University ofSouth Australia

Reference OrientationTracked markers travel on circular paths around the z-axis

Transform marker positions with inverse forward kinematic pose using only the first 5 angles

Fit circles to trajectories to find centers and fit a line through the centres and the HIP position

20

University ofSouth Australia

Calibration Procedure

Tooltip (Tuceryan et al., 1995)

Absolute Orientation (Horn, 1987)

Time-Delay Estimation (Huber et al., 2012)

Joint-Angles (Harders et al., 2009)

Reference Orientation

Gimbal-Angle Errors

21

External Tracker (ET)

ETorigin

HIPpose

HIPtarget

Haptic Device (HD)

Camera (C)

Ctarget

HDorigin

University ofSouth Australia

Gimbal-Angle Error Compensation

22

External Tracker (ET)

ETorigin

HIP

Haptic Device (HD)

HDorigin

University ofSouth Australia

Gimbal-Angle Calibration

Use reference z-axis to determine the errors of the first two gimbal sensors

Model errors as linear or quadratic function

Minimize error using Levenberg-Marquardt optimization

Exploit mechanical limits of the last gimbal sensor by finding the gaps in the circular paths to compensate the offset

23

University ofSouth Australia

Evaluation

Numerical Evaluation of calibration results:

Two VHAR workspace setups:

Low-fidelity (LF): PHANToM Omni, OptiTrack

High-fidelity (HF): PHANToM Premium, ART-Tracking

Visual Evaluation:

Unwrapped spherical heatmap of recuded errors

Visual overlays on haptic stylus in calibrated system

24

University ofSouth Australia

Numerical Evaluation

25

Reduced position errors by ~ 20%

Reduced orientation errors by > 40%

No Calib Harders Our Work No Calib Harders Our Work

No Calib Harders O u r Wo r k No Calib Harders O u r Wo r k

University ofSouth Australia

Improved Orientation Accuracy

26

Reduction of Orientation Errors in Degrees

ISMAR 2014 Live DemoYou can: - perform a calibration - paint on a fish We’re in Room 00.07.014

27

University ofSouth Australia

SummaryContributions:

Novel algorithm for orientation calibration (>40% improvement)

Improved position calibration (~20% improvement)

Compensated time-delay for faster and better calibration

28

x

y

z

xy

z

HIPpose

HIPtarget

REFzaxis

x

y

z

xy

z

HIPorigin

REFzaxis

Questions ?Thank You!!

University ofSouth Australia

ReferencesHarders, M. & Szekely, G., 2009. Calibration, Registration, and Synchronization for High Precision Augmented Reality Haptics. IEEE Transactions on Visualization and Computer Graphics, 15(1), pp.138–149. Fu, M.J. et al., 2011. Effect of Visuo-Haptic Co-location on 3D Fitts' Task Performance. In Proceedings of the International Conference on Intelligent Robots and Systems (IROS), 2011 IEEE/RSJ. San Francisco, CA, USA: IEEE, pp. 3460–3467. Rhienmora, P. et al., 2010. Augmented Reality Haptics System for Dental Surgical Skills Training. In Proceedings of the 17th ACM Symposium on Virtual Reality Software and Technology. Hong Kong: ACM, pp. 97–98. Ikits, M., Hansen, C.D. & Johnson, C.R., 2003. A Comprehensive Calibration and Registration Procedure for the Visual Haptic Workbench. In the workshop. New York, NY, USA: ACM, pp. 247–254. Vallino, J. & Brown, C., 1999. Haptics in Augmented Reality. In Proceedings of the IEEE International Conference on Multimedia Computing and Systems. pp. 195–200. Horn, B.K.P., 1987. Closed-form Solution of Absolute Orientation using Unit Quaternions. Journal of the Optical Society of America, 4(4), pp.629–642. Tuceryan, M., Greer, D.S. & Whitaker, R.T., 1995. Calibration Requirements and Procedures for a Monitor-Based Augmented Reality System. Visualization and …, 1(3), pp.255–273. Pustka, D. et al., 2006. Spatial Relationship Patterns: Elements of Reusable Tracking and Calibration Systems. In Proceedings of the International Symposium for Mixed and Augmented Reality. Santa Barbara, CA, USA, pp. 88–97. Sandor, C. & Uchiyama, S., 2007. Exploring Visuo-Haptic Mixed Reality, IEICE. Huber, M., Michael, S. & Klinker, G., 2012. Temporal Calibration in Multisensor Tracking Setups. 9. Workshop der GI-Fachgruppe Virtuelle und Erweiterte Realität (VR-AR 2012), pp.201–212.

29