Augmented Reality Visual Guidance For Spatial …mbdl.arizona.edu/publications/pdfs/Wagner2017.pdfTASKit to high fidelity virtual reality trainers like the LAP Mentor VR and the Xitact

AUGMENTED REALITY VISUAL GUIDANCE FOR SPATIAL PERCEPTION IN THE COMPUTER ASSISTED SURGICAL TRAINER

Adam Wagner Jerzy W. Rozenblit

Dept. of Electrical and Computer Engineering The University of Arizona

Dept. of Electrical and Computer Engineering Department of Surgery

1230 E Speedway Boulevard Tucson, Arizona

The University of Arizona 1230 E Speedway Boulevard

[email protected] Tucson, Arizona [email protected]

ABSTRACT

We propose several augmented reality overlays for visual guidance in training for laparoscopic surgery using a 2D display, with the goal of aiding the user’s perception of depth and space in that limiting environment. A pilot study of 30 subjects (22 male and 8 female) is presented with results showing the effect of the various overlays on subject performance of a path following task in the Computer Assisted Surgical Trainer. Deviation, economy of movement, and completion time are considered as metrics. The use of a reference indicator for the nearest point on the optimal path is found to result in significant reduction (p < 0.05) in some deviation metrics using non-parametric tests of significance. The data also indicate a reduction in subject deviation with overlays designed to provide depth information. Avenues for further investigation are presented.

Keywords: laparoscopic surgery, surgical simulation, augmented reality, visual guidance.

1 INTRODUCTION

1.1 Background

Minimally Invasive Surgery (MIS) continues to be a popular operative approach for many surgeries due to the benefits afforded to the patient, including reductions in blood-loss, post-operative pain, scarring, and recovery time. As opposed to open surgery, MIS uses cable- or rod-based camera systems and surgical tools inserted via either no incisions or small incisions. One form of MIS is laparoscopic surgery, performed using the eponymous laparoscope, which provides the surgeon a view of the surgical site via a small incision in the abdomen. Surgical instruments are also inserted via small incisions and are operated manually by the surgeon.

The benefits of MIS come at the cost of a more constrained and onerous working environment for the surgeon. In the case of conventional laparoscopy, s/he must work with a remote view of the operating site via the laparoscope, which provides only a two-dimensional (i.e. non-stereoscopic) view of the site, greatly limiting the surgeon’s sense of depth (DePaolis and Aloisio, 2012). Much research has been done on the use of stereoscopic displays (e.g. head mounted displays) for use in surgical training and procedures (van Bergen et al., 1998; Tevaearai, Mueller, and von Segesser, 2000). The results are somewhat mixed though do show advantages in some cases, but can also add further encumbrance for the surgeon (Bimber and Raskar, 2006). In addition to lacking stereoscopic information, the camera generally distorts the image from what would be perceived with the naked eye due to the wide-angle lenses that must be used to provide a SpringSim-MSM 2017, April 23-26, Virginia Beach, VA, USA©2017 Society for Modeling & Simulation International (SCS)

855

Wagner and Rozenblit

large enough view of the site. The surgeon also has limited mobility and dexterity with the surgical

instruments, and reduced tactile (haptic) feedback/sensation of the tissues being operated on.

Due to this challenging working environment surgeons are required to demonstrate a certain level of skills

with the laparoscopic instruments before attempting a real procedure. Simulation is frequently utilized in

the training process and much work has been done to develop training systems with varying degrees of

complexity and features. These range from relatively inexpensive "box" trainers such as the Ethicon

TASKit to high fidelity virtual reality trainers like the LAP Mentor VR and the Xitact LS500 VR, as

shown in Figure 1. The simple trainers allow surgeons to practice their skills but provide little to no

performance feedback. More advanced systems are designed to offer quantitative feedback to the user,

evaluating metrics such as the time taken to complete a task and economy of movement, while providing

virtual reality simulations of the procedure with realistic instruments and haptic feedback.

Augmented reality (AR) has also been investigated as another potential aid for training and even use

during surgery (Maurer et al., 2001; Wen et al., 2010). Much of the literature develops the use of AR for

overlaying medical imaging on top of the practice/operative site (Shelton, Katzky, and Stetten, 2004;

Shamir, Joskowicz, and Shoshan, 2006; Rahman, Mahmud, and Mashuk, 2013), but the authors have

found little in the literature regarding the use of 2D virtual symbology or indicators for surgical guidance.

In this paper we develop novel AR visual guidance to aid the surgical student’s perception of depth and

spatial relationships in the practice environment as well as for consideration for use during an actual

surgery.

Figure 1: Laparoscopic training systems (Simbionix, 2015; Ethicon, 2011).

1.2 The CAST-III System

The Model Based Design Lab at the University of Arizona has developed the Computer Assisted Surgical

Trainer (CAST-III) in an attempt to bridge the gap between the inexpensive but simplistic box trainers

and high-end VR training systems (Rozenblit et al., 2014). The goal of CAST is to develop a low-cost

training simulator that provides many of the same features and benefits as the more expensive systems. A

custom-built gimbal system contains real laparoscopic surgical tools placed in a box-trainer-like

environment. The surgical implements are tracked via an electromechanical system that also allows

CAST to provide haptic guidance to the user via a fuzzy logic controller developed within the lab. A web-

cam is used to stand in for the laparoscope, providing the user a view of the training environment via a 2D

monitor.

856


Figure 2: The CAST-III system when faced by the user (left) and the practice environment as seen from the monitor (right).

The OptMIS module calculates optimal paths around obstacles in the practice environment to be used for visual and haptic guidance. Currently this path planning is performed offline. OptGuide (Hong and Rozenblit, 2016) is responsible for haptic guidance and contains the fuzzy logic control algorithm. OptViz, introduced in Rozenblit et al. (2014), provides visual guidance and is the primary host of the AR guidance developed in this work. We build upon the previous foundation with new AR overlays.

1.3 Human Depth Perception

Humans perceive depth and space via a number of sensory cues. These can be divided into three broad categories: binocular, monocular, and oculomotor (Goldstein, 2009). These cues all vary in the distances at which they are effective in providing depth information. Although binocular cues (cues that come from both eyes) are usually what come to mind when considering our ability to perceive depth, a considerable amount of depth perception is gained from monocular cues - cues that come from each eye individually. The AR visual guidance developed in this work relies on some of these monocular cues, including relative height, relative size, familiar size, perspective convergence, and occlusion. Further information on these and other depth cues can be found in Goldstein.

2 AR VISUAL GUIDANCE

Currently the basic training task in CAST is for the user to move the instrument sequentially from target point to target point while following the guidance path displayed on the monitor. In this study, only the path from the previous target point to the next is displayed at any given time (as opposed to the entire path from the first point to the last), in addition to other indicators. Six total visual guidance schemes were developed and tested for this study. The AR overlays were generated using OpenGL primitives in the OptViz module. Camera calibration was performed on our web camera using MATLAB’s camera calibration function, based on the algorithm proposed by Zhang (2000). Currently the calibration only needs to be performed once as the camera is in a fixed position in the world coordinate frame. This work does not address scenarios involving camera motion which requires dynamic determination of the camera’s extrinsic parameters.

2.1 No-Reference AR Overlays

Two of the six schemes showed only the guidance path – in one case indicated with a narrow white line (“lines”), and in the other with a series of circles (“circles”) rendered in three dimensions and with constant radius, constrained to the world XZ plane. The circle path was included to investigate if there might be any benefit to giving the path itself a three dimensional structure. These two schemes are referred to as “no-reference” schemes and can be seen in Figure 3.

857


Figure 3: No-reference visual guidance: "Lines" (left) and "Circles" (right).

2.2 AR Overlays with Reference and Instrument Indicators

The remaining four of the six schemes used two indicators in addition to the path, as shown in Figure 4. These two additional indicators showed 1) the location of the instrument tip, as measured by the system, and 2) a reference point for guidance. These guidance schemes are referred to as the 2D Squares, 3D Squares, Cubes, and Post(s).

For the present study the reference point was calculated by finding the nearest point on the path (in three dimensions, by the Euclidean distance) to the current measured instrument tip location and shifting it forward by a fixed amount (in this case by 10 points along the path). The shift was added to ensure the reference point would always be slightly in front of the instrument tip so as to lead the user along the path. As an additional measure to encourage the user along the path the reference point was constrained not to move backward. That is, once the reference point had been calculated as a given index into that path segment, the reference point index could not decrease.

Figure 4: Visual guidance with references (instrument not shown). Clockwise from top left: "2D Squares", "3D Squares", "Cubes", and "Post". The red indicator shows the measured instrument tip position, the yellow indicator shows the reference position.

858


2.2.1 2D Squares Overlay

The 2D Squares use squares drawn in the image plane with a fixed side length, in this case 50 pixels (the size of the video feed was 1280x720 pixels), and are always rendered in the same order. The reference and instrument positions in the image plane were calculated using the camera calibration and determined the center point of each square, however the size of the squares did not change as a function of the depth of either point. Neither was lens distortion accounted for. The result is that the 2D Squares provide direct information about the location of each point tangent to the line of sight (LOS), but not in depth. The 2D Squares could possibly provide a depth cue via the relative height of the squares (see Goldstein), however it would be ambiguous as relative distance in depth or in the vertical plane could produce the same image. The squares are always given the same “depth” in the rendering pipeline, with the result that one (in this case the instrument tip location) is always rendered on top of the other (the reference) – removing another element of depth information that might be provided to the user via occlusion. It is expected that this will sometimes be confusing to the user as they might conclude that one position is always behind the other when in fact no such information is provided. Thus, the 2D Squares are included to represent schemes with references that are not intended or designed to provide depth information to the user.

2.2.2 3D Squares Overlay

The 3D Squares are squares constrained to the world XZ plane whose vertices are computed in the world frame before projection into the image plane using the calibration data. This is similar to how the circles in the circle path were rendered. The squares have fixed side length (1.5cm) in the world frame. Thus, the nearer or further from the camera a point was, the square would appear larger or smaller on screen, respectively, obeying the laws of perspective. The 3D Squares were also drawn accounting for lens distortion, providing additional information about how the camera affects the view of the scene.

2.2.3 Cubes Overlay

The Cubes are, naturally, cubes, rendered in three dimensions. These behave in essentially the same manner as the 3D Squares and are rendered in a similar fashion, except now have extent in depth and also have faces which are rendered with some translucence, giving them a solid appearance. The cubes can intersect each other and can provide additional depth information via occlusion. Cubes with completely transparent faces were considered but ultimately not included in the study as the number of visible edges tended to make the overlays appear very “busy”. The addition of translucent faces cut down on the busyness while emphasizing the occlusion cue.

2.2.4 Post Overlay

The Post displays a line drawn vertically (i.e. along the world Z axis) in three dimensions from a virtual “ground” plane to the position being indicated. A small dot is drawn at the point. Additionally, a black circle, constrained to the XY ground plane and with a fixed Z component, functions as a “shadow” indicator below the post. The circle does not change radius but is rendered in three dimensions as well so will obey laws of perspective. The post provides information about the height of the point via the length of the post when also taking into account the position of the “shadow”. Since the shadows are constrained to the same plane, the user can intuit both the relative position of the points in the X and Y dimensions but also their relative heights given the shadow positions and the lengths of the posts.

859


3 EXPERIMENT

3.1 Setup

A pilot study was designed to determine the potential for 2D AR visual guidance to aid the subject’s perception of depth and space. This study was approved by the Institutional Review Board of the University of Arizona and performed according to its requirements. A total of 30 subjects (22 male and 8 female) ranging from 18 to 60 years old (median age 30 yrs) were asked to use the CAST system to perform the training task with the different visual aids. A majority of the subjects (23) were novices, indicating no experience with laparoscopic surgery or training, while 7 indicated some level of experience.

A practice path built from segments generated using the OptMIS algorithm was selected to require the subject to be moving the tool in all three dimensions for a majority of the task, using a combination of straight and curved paths, as seen in Figure 5. OptMIS generates paths as a set of discrete points in three dimensions which are designed to take the subject around the obstacles in the practice environment.

Figure 5: Views of path used for study: top (left) and front (right). Target points indicated in red.

In the plots shown and in further discussion, the Y axis is approximately the depth axis (the actual depth axis ultimately depending on where the observer is looking in the image). The +Z axis is up, and +X is to the right.

3.2 Subject Task

The subject was given instructions to first move to the first target point (R1), after which a path leading to the next target point would appear. The subjects were instructed to follow the path as best as possible to the next target point, noting that the path (and the target points) existed in three dimensions.

Due to error in the sensors used to measure the position of the instrument tip, the subject was notified that the system would not track the instrument perfectly and that some “hunting” with the instrument in the vicinity of the target point might be necessary to get the system to register that the target point had been reached, even if the subject thought they were right where they needed to be. Instrument tracking was calibrated before each subject used the system by placing the instrument in a known calibration position and resetting hardware encoders. Over the typical period of time the subjects used the system and without the use of haptic feedback the drift in the tracking is considered negligible, but some initial calibration error does exist. This would have been consistent for the user across all visual aids.

-10 -8 -6 -4 -2 0 2 4 6 8 10X [cm]

0

5

10

15

Y [c

m]

-10 -8 -6 -4 -2 0 2 4 6 8 10X [cm]

2

4

6

8

10

12

14

16

18

20

Z [c

m]

860


In order to account for carry-over effects a balanced Latin square design was used in the study. The 30 subjects were divided randomly into six groups of five subjects each. Each group performed the same path-following task six times, with each group using each of the six different guidance schemes in a different order.

4 RESULTS AND ANALYSIS

Numerous performance metrics are evaluated in laparoscopic skills testing. Here we will focus on deviation, economy of movement, and completion time. Unless otherwise indicated, we rely on the balanced Latin square design and no further accounting was done for carry-over effects.

Figure 6: Sum total deviation across all subjects.

4.1 Deviation

The instantaneous deviation is defined as the Euclidean distance from the current measured instrument tip location to the nearest point on the reference path. The position of the instrument tip was captured at 20 Hz (every 50 ms). Figure 6 shows the results if we calculate the deviation at each time step and sum them up across all tasks for all subjects, then break the data out by the visual aid in use. This metric also includes the effect of time as maintaining a given deviation for longer will increase the sum. Assuming that the sampling rate was sufficient to capture all pertinent features of the subject’s performance, doubling the sampling rate would have the effect of scaling these numbers by two.

The blue (leftmost) bar in Figure 6 shows the deviation calculated as previously described (RSS of deviation in all three dimensions). The green (center) bar shows the sum total deviation as first projected along the instantaneous line of sight (LOS; vector from camera position to the point along the optimal path nearest to the current tool position). The yellow (rightmost) bar shows the RSS of the deviation perpendicular to the LOS. The results show a clear distinction between the no-reference (NR) guidance schemes and those with a reference. Additionally, one may note that the majority of the deviation in all cases, regardless of the visual aid, was along the LOS – that is, in depth – with a smaller contribution perpendicular to the LOS. This would agree with an expectation that the primary difficulty for the user is in finding the proper depth to be at due to the lack of normal stereoscopic visual cues. This data also

Sum Total Deviation by Aid for All Subjects

Lines (NR) Circles (NR) Post 2D Squares 3D Squares Cubes0

200

400

600

800

1000

1200

1400

1600

1800

[m]

All Three Axes (RSS)Along LOSPerpendicular to LOS (RSS)

861


provides the first indication that the 2D Squares resulted in the highest deviation of the four guidance schemes with references.

To evaluate deviation on a subject by subject basis across visual aids we consider the 95th percentile deviation for each subject when using a given visual aid. This value was chosen over the average as the data are not normally distributed. Figure 7 shows a MATLAB box plot of these results when using the deviation along the LOS.

Figure 7: 95th percentile deviation along LOS by visual aid.

The same trends indicated by the sum total deviation are apparent here. Of the schemes with references, the 2D Squares again show elevated deviation relative to the other three. This would seem to indicate that the other schemes are indeed effective in providing depth information to the subject.

Tests of significance were performed using the non-parametric Kruskal-Wallis test. The results for deviation along the LOS are shown in Table 1. The p-values in bold show visual aids that were significantly different at the 95% significance. In addition to the qualitative difference seen in the plots above, the reference schemes are found to be statistically different from the no-reference schemes. The results do not indicate a significant difference between any of the reference schemes at this significance level, however one can see that at the 90% significance level the Post and the 2D Squares do differ.

Table 1: Kruskal-Wallis p-values for 95th percentile deviation along LOS by visual aid.

Lines Circles Post 2D Squares 3D Squares Cubes Lines 0.99995542 0.00000012 0.01864949 0.00001141 0.00000096 Circles - 0.00000005 0.00955274 0.00000402 0.00000032 Post - - 0.09384178 0.95854230 0.99895515 2D Squares - - - 0.48756600 0.22280943 3D Squares - - - - 0.99727584 Cubes - - - - -


1

2

3

4

5

6

7

[cm

]95th Percentile Deviation Along LOS by Visual Aid

862


It is possible that the direct information provided by the Post of both the relative XY position of each position indicated as well as their relative heights (given the XY positions) was helpful to the subjects and accounts for the difference between these two overlays.

While not differing to the same significance level it is noteworthy how much the 3D Squares show reduced deviation relative to the 2D Squares. Though very similar in appearance, the additional information provided by the perspective-dependent shape of the 3D Squares as well as the relative size of the squares does seem to provide useful depth information to the subject despite being limited to a 2D display.

4.2 Economy of Movement (Path Length)

Path length was calculated as the total length of the path the instrument tip took to complete the task during the use of each visual aid. Path length is used as a measure of “economy of movement” – reducing the amount of instrument motion needed to complete a given task is associated with higher skill. Figure 8 shows a box plot with the path length results by visual aid as well as the length of the optimal path that the subject was instructed to follow.

Figure 8: Path length by visual aid.

Once again we find a distinction between the no-reference schemes and the reference schemes, as well as a negative performance impact with the 2D Squares relative to the other reference schemes. Table 2 shows the Kruskal-Wallis p-values for path length. A statistically significant difference is only seen between the no-reference schemes and some, but not all, of the reference schemes.

Table 2: Kruskal-Wallis p-values for path length by visual aid.



100

150

200

250

300

350

[cm

]

Path Length by Visual Aid

Optimal Path Length

863


4.3 Completion Time

Finally we consider the time to complete the task. Figure 9 shows the box plot for these results, and Table 3 gives the Kruskal-Wallis test results. Here the same correlations appear to be present but more weakly. The Kruskal-Wallis test did not show any difference between any of the groups in completion at the 95% significance level. A possible explanation for this weakened correlation is that when the subject was presented with a reference position to follow the subject made more deliberate or calculated movements to do so and thus moved the tool more slowly than they would have without a reference. The Cubes do seem to show a greater level of differentiation from the no-reference schemes.

Figure 9: Task completion time by visual aid.

Table 3: Kruskal-Wallis p-values for task completion time by visual aid.


5 CONCLUSION

We have presented several novel AR visual guidance schemes for use in training and simulation for laparoscopic surgery. A pilot study showed a significant (p < 0.05) reduction in deviation from the optimal path when a reference indicator was provided, as well as reductions in path length and time to complete the task. Additionally, AR overlays which were designed to provide depth and spatial information to the user showed reductions in deviation along the depth axis as well as total path length compared to visual aids that were not designed to provide depth information. A larger scale study with more participants – or a more focused study with a smaller number of visual aids – would be called for to solidify these results.


50

100

150

200

250

300

[s]

Task Completion Time by Aid

864


Several issues are worth considering in future work. It has not been determined whether the presence of the instrument tip indicator by itself (that is, without a reference indicator) would have a similar effect as the reference schemes used here, which had both present. Some initial data was collected for such a scheme and the results appeared to be similar to the no-reference schemes, but more investigation is needed. The presence of an instrument tip indicator would also presumably reduce subject deviation that results from error in the tracking hardware calibration, however this error is believed to be on the order of 1 centimeter or less and so is not expected to have contributed significantly to the results presented here.

The circle path could possibly be able to provide more depth information by exaggerating the relative size of the circles as a function of depth, which was not investigated in this work. Also, one might investigate whether merely adding occlusion to the 2D Squares (i.e. the nearer square being rendered on top of the further square) might significantly affect performance compared to the presented implementation.

Another issue is that although the overlays with depth information do indicate a positive effect on performance, some of them may not be practical for some tasks or for clinical use as the end of the tool may be occluded by the overlay. Additionally, the Post overlay, while generally indicating some of the best performance (and receiving positive feedback from subjects), does not necessarily make sense in a context without a pre-existing “ground plane” such as our practice environment has. Future work could consider how these overlays could be reworked to be more applicable in a clinical context.

Though only a pilot study, these results do seem to indicate promise in using 2D AR overlays to provide depth information to the laparoscopy student or surgeon. The use of such guidance schemes may provide an elegant solution for improving the user’s sense of depth and space without the need for a head-mounted or other form of stereoscopic display, and could also be applied as an education or collaboration tool for surgeons. Furthermore, these types of indicators could be investigated for their ability to augment spatial perception even within the context of stereoscopic displays as these technologies improve and see increased adoption.

REFERENCES

Bimber, O. and Raskar, R. 2006. “Modern approaches to Augmented Reality”. In ACM SIGGRAPH 2006 Courses (SIGGRAPH '06). ACM, New York, NY, USA, Article 1.

DePaolis, L. T. and Aloisio, G. 2012. “Augmented Reality in Minimally Invasive Surgery”. In Fifth International Conference on Advances in Computer-Human Interactions 2012 (ACHI 2012). International Academy, Research, and Industry Association (IARIA).

Ethicon, 2011. Taken from TASKit Overview. Digital image. Available from: Ethicon, www.ethiconinstitute.com/download_other_asset?fid=5421 (accessed January 3, 2017).

Goldstein, E. B. 2009. Sensation and Perception. 8th ed. Cengage Learning.

Hong, M. and J. W. Rozenblit. 2016. “A Haptic Guidance System for Computer Assisted Surgical Training Using Virtual Fixtures” In 2016 IEEE International Conference on Systems, Men, and Cybernetics (SMC), pp. 2230-2235.

Maurer, C. R., F. Sauer, B. Hu, B. Bascle, B. Geiger, F. Wenzel, F. Recchi, T. Rohlfing, C. R. Brown, R. J. Bakos, R. J. Maciunas, and A. R. Bani-Hashemi. 2001. “Augmented Reality Visualization of Brain Structures with Stereo and Kinetic Depth Cues: System Description and Initial Evaluation with Head Phantom”. In Medical Imaging 2001: Visualization, Display, and Image-Guided Procedures (SPIE Proceedings Vol. 4319).

865


Rahman, M. A., P. Mahmud and M. S. Mashuk. 2013. “Augmented and Virtual Reality Based Approaches in Minimally Invasive Surgery Training”. In 2013 International Conference on Informatics, Electronics and Vision (ICIEV), Dhaka, pp. 1-4.

Rozenblit, J. W., C. Feng, M. Riojas, L. Napalkova, A. J. Hamilton, M. Hong, P. Berthet-Rayne, P. Czapiewski, G. Hwang, J. Nikodem, and A. Shankaran. 2014. “The Computer Assisted Surgical Trainer: Design, Models, and Implementation”. In Proc. of the 2014 Summer Simulation Conference, Modeling and Simulation in Medicine, pp. 211-220.

Simbionix, 2015. LAP3-Team-Training-C-2. Digital image. Available from: Simbionix, http://simbionix.com/wp-content/uploads/2015/12/LAP3-Team-Training-C-2.jpg (accessed January 3, 2017).

Shamir, R., L. Joskowicz, and Y. Shoshan. 2006. “An augmented reality guidance probe and method for image-guided surgical navigation”. In 5th International Symposium on Robotics and Automation (ISRA 2006), pp. 25–28.

Shelton, A. D., B. R. Klatzky, and C. G. Stetten. 2004. “Method for assessing augmented reality needle guidance using a virtual biopsy task”. In 2004 2nd IEEE International Symposium on Biomedical Imaging: Nano to Macro (IEEE Cat No. 04EX821). Vol. 1, pp. 273-276.

Tevaearai, H. T., X. M. Mueller, L.K. von Segesser, 2000. “3-D vision improves performance in a pelvic trainer”. In Endoscopy, 32(6), pp. 464-68.

Van Bergen, P., W. Kunert, J. Bessell, and G. F. Buess. 1998. “Comparative study of two-dimensional and three-dimensional vision systems for minimally invasive surgery”. In Surgical Endoscopy. 12(7), pp. 948-54.

Wen, R., L. Yang, C. K. Chui, K. B. Lim, and S. Chang. 2010. “Intraoperative visual guidance and control interface for augmented reality robotic surgery”. In IEEE ICCA 2010, Xiamen, pp. 947-952.

Zhang, Z. 2000. “A flexible new technique for camera calibration”. In IEEE Transactions on Pattern Analysis and Machine Intelligence, 22(11), pp. 1330–1334.

AUTHOR BIOGRAPHIES

ADAM WAGNER is a Research Assistant in the Model-Based Design Lab at the University of Arizona Electrical and Computer Engineering (ECE) Department. He received his BS in Mechanical Engineering at the University of Arizona and recently defended his MS thesis in Electrical and Computer Engineering. His research interests include human-computer interaction, simulation, embedded systems, and machine learning. He is also a senior engineer at an engineering firm in Tucson, Arizona. His email address is [email protected].

JERZY W. ROZENBLIT is University Distinguished Professor, Raymond J. Oglethorpe Endowed Chair in the Electrical and Computer Engineering (ECE) Department, with a joint appointment as Professor of Surgery in the College of Medicine at the University of Arizona. During his tenure at the University of Arizona, he established the Model-Based Design Laboratory with major projects in design and analysis of complex, computer-based systems, hardware/software codesign, and simulation modeling. He presently serves as Director of the Life-Critical Computing Systems Initiative, a research enterprise intended to improve the reliability and safety of technology in healthcare and life-critical applications. His email address is [email protected].

866

Documents

Augmented Reality Visual Guidance For Spatial …mbdl.arizona.edu/publications/pdfs/Wagner2017.pdfTASKit to high fidelity virtual reality trainers like the LAP Mentor VR and the Xitact