4
Surgical Laser Augmented with Haptic Feedback and Visible Trajectory Peter R. Rizun * Department of Physics & Astronomy Seaman Family MR Research Centre University of Calgary Garnette R. Sutherland # Department of Clinical Neurosciences Seaman Family MR Research Centre University of Calgary Fig. 1. The prototype tactile feedback laser system contains a laser distance measurement module attached to the distal end of a 3-DOF robotic arm. This module can be manipulated by the operator’s hand relatively freely when far above a surface. However, when the focal point of the laser impinges a surface, feedback forces are generated giving the operator the impression of making contact with a solid object. Profiled surfaces can be placed on the horizontal platform for testing. ABSTRACT This application sketch describes a conceptual surgical laser system, designed for incorporation with a surgical robot, that provides haptic and visual feedback. Initial results from a prototype haptic laser system, built with a low-power (non- cutting) laser, are also presented. CR Categories: H.1.2 [Models and Principles]: User/Machine Systems—Human Factors Keywords: augmented reality, haptics, lasers, surgical robotics. 1 INTRODUCTION Humans are comfortable working with conventional hand tools such as knives and tweezers. We can feel what we are doing and we can see exactly where the tool is. Humans are less comfortable working with lasers because they do not provide the same types of sensory cues. With a laser, only light strikes the surface so there is no force feedback and visual feedback is limited to the location and size of the laser spot. Surgeons who use lasers rely on experience to compensate for the missing sensory information [1, 2]. Robots have been shown to augment surgery by scaling motion and filtering tremor [3]; however, more is possible with robotic platforms. A laser system for robotic surgery that provides a sense of touch and visually displays laser trajectory would enhance the integration of laser technology with surgery. This application sketch describes a conceptual laser system for robotic surgery that provides haptic and visual feedback. The conceptual system will generate feedback forces when the focal point of the laser impinges a surface, providing the operator the illusion of touching a solid object. The surgeon will adjust laser intensity (cutting power) by applying more or less force. The surgeon will also “see” the beam trajectory overlaying images of the surgical site, adding the last of the missing sensory cues. The haptic-feedback component of the system employs a “virtual fixture” [4] to create the sensation of a solid surface. The system converts an optical distance measurement to a haptic sensation, similar to how the SmartTool [5] converts information from its contact sensors to useful haptic feedback. Enhancing a laser with visible trajectory fits the classic stereotype of augmented reality (AR), as a real procedure is augmented by beam trajectory overlaid on the physician’s view of the surgical field. To explore touching surfaces with light, a non-teleoperable prototype tactile feedback laser system (Fig. 1) was constructed using a low-power (non-cutting) laser to avoid added complexities. The prototype consists of a simple 3-degree-of-freedom (DOF) robotic arm with a novel laser distance measurement module attached to the distal end. The operator manipulates this module while a microcontroller monitors the distance measurement and renders force feedback through the robotic arm when the focal point of the laser impinges a surface. The rest of this application sketch is organized as follows: in Section 2 we discuss some background material and related work. Section 3 provides a functional description of the conceptual system and Section 4 describes the prototype. We conclude with Section 5. 2 BACKGROUND 2.1 Augmented Reality and Haptic Transformations * email: [email protected] # email: [email protected] Overlaying images to enhance reality has been extensively researched, reported, and even applied to consumer products [6]. Touch is the next sense being synthesized to augment reality. Rosenberg introduced the concept of “virtual fixtures” [4] as haptic objects that enhance the performance of teleoperator tasks. To explain the abstract idea of virtual fixtures, he used the IEEE Virtual Reality 2005 March 12-16, Bonn, Germany 0-7803-8929-8/05/$20 ©2005 IEEE 257 Proceedings of the IEEE Virtual Reality 2005 (VR’05) 1087-8270/05 $20.00 © 2005 IEEE

[IEEE IEEE Proceedings. VR 2005. Virtual Reality, 2005. - Bonn, Germany (March 12-16, 2005)] IEEE Proceedings. VR 2005. Virtual Reality, 2005. - The XYZ virtual workspace

  • Upload
    j

  • View
    212

  • Download
    0

Embed Size (px)

Citation preview

Page 1: [IEEE IEEE Proceedings. VR 2005. Virtual Reality, 2005. - Bonn, Germany (March 12-16, 2005)] IEEE Proceedings. VR 2005. Virtual Reality, 2005. - The XYZ virtual workspace

Surgical Laser Augmented with Haptic Feedback and Visible Trajectory

Peter R. Rizun*

Department of Physics & Astronomy

Seaman Family MR Research Centre

University of Calgary

Garnette R. Sutherland#

Department of Clinical Neurosciences

Seaman Family MR Research Centre

University of Calgary

Fig. 1. The prototype tactile feedback laser system contains alaser distance measurement module attached to the distal end ofa 3-DOF robotic arm. This module can be manipulated by theoperator’s hand relatively freely when far above a surface.However, when the focal point of the laser impinges a surface, feedback forces are generated giving the operator the impressionof making contact with a solid object. Profiled surfaces can beplaced on the horizontal platform for testing.

ABSTRACT

This application sketch describes a conceptual surgical lasersystem, designed for incorporation with a surgical robot, thatprovides haptic and visual feedback. Initial results from aprototype haptic laser system, built with a low-power (non-cutting) laser, are also presented.

CR Categories: H.1.2 [Models and Principles]: User/Machine Systems—Human Factors

Keywords: augmented reality, haptics, lasers, surgical robotics.

1 INTRODUCTION

Humans are comfortable working with conventional hand tools such as knives and tweezers. We can feel what we are doing andwe can see exactly where the tool is. Humans are lesscomfortable working with lasers because they do not provide thesame types of sensory cues. With a laser, only light strikes thesurface so there is no force feedback and visual feedback islimited to the location and size of the laser spot. Surgeons who use lasers rely on experience to compensate for the missingsensory information [1, 2]. Robots have been shown to augmentsurgery by scaling motion and filtering tremor [3]; however, more is possible with robotic platforms. A laser system for robotic surgery that provides a sense of touch and visually displays lasertrajectory would enhance the integration of laser technology withsurgery.

This application sketch describes a conceptual laser system for robotic surgery that provides haptic and visual feedback. The conceptual system will generate feedback forces when the focalpoint of the laser impinges a surface, providing the operator theillusion of touching a solid object. The surgeon will adjust laserintensity (cutting power) by applying more or less force. Thesurgeon will also “see” the beam trajectory overlaying images ofthe surgical site, adding the last of the missing sensory cues. Thehaptic-feedback component of the system employs a “virtual fixture” [4] to create the sensation of a solid surface. The systemconverts an optical distance measurement to a haptic sensation,similar to how the SmartTool [5] converts information from itscontact sensors to useful haptic feedback. Enhancing a laser withvisible trajectory fits the classic stereotype of augmented reality(AR), as a real procedure is augmented by beam trajectoryoverlaid on the physician’s view of the surgical field.

To explore touching surfaces with light, a non-teleoperableprototype tactile feedback laser system (Fig. 1) was constructed

using a low-power (non-cutting) laser to avoid added complexities.The prototype consists of a simple 3-degree-of-freedom (DOF)robotic arm with a novel laser distance measurement module attached to the distal end. The operator manipulates this modulewhile a microcontroller monitors the distance measurement andrenders force feedback through the robotic arm when the focal point of the laser impinges a surface.

The rest of this application sketch is organized as follows: inSection 2 we discuss some background material and related work. Section 3 provides a functional description of the conceptual system and Section 4 describes the prototype. We conclude withSection 5.

2 BACKGROUND

2.1 Augmented Reality and Haptic Transformations * email: [email protected]# email: [email protected]

Overlaying images to enhance reality has been extensivelyresearched, reported, and even applied to consumer products [6].Touch is the next sense being synthesized to augment reality.Rosenberg introduced the concept of “virtual fixtures” [4] as haptic objects that enhance the performance of teleoperator tasks.To explain the abstract idea of virtual fixtures, he used the

IEEE Virtual Reality 2005 March 12-16, Bonn, Germany0-7803-8929-8/05/$20 ©2005 IEEE

257Proceedings of the IEEE Virtual Reality 2005 (VR’05)

1087-8270/05 $20.00 © 2005 IEEE

Page 2: [IEEE IEEE Proceedings. VR 2005. Virtual Reality, 2005. - Bonn, Germany (March 12-16, 2005)] IEEE Proceedings. VR 2005. Virtual Reality, 2005. - The XYZ virtual workspace

Fig. 2. NeuroArm in position for microsurgery: various tools canbe connected to the robotic arms and controlled by a surgeon atthe robot workstation.

Fig. 3. NeuroArm workstation: the surgeon looks through thebinoculars and sees a magnified view of the surgical field. Thearms of the robot are slaved to the motion of the haptic handcontrollers, which in turn relay the sense of touch back to thesurgeon.

analogy of a ruler as a real fixture that enables one to draw linesstraighter and faster than freehand. He proposed that virtualfixtures could be used to augment teleoperator tasks in the sameway that a ruler is used to improve line drawing. For example, aremote operator of a robot could position a virtual ruler, see it on a screen overlaying the remote environment, and actually feel theruler through his haptic hand controller. Rosenberg showed thatteleoperator performance of peg-in-hole tasks could be improved with virtual fixtures [4].

Since virtual fixtures do not obey the same laws that govern realfixtures, they can be programmed to do more. What we areconcerned with in this application sketch is how information from various non-haptic sensors can be used to generate useful haptic feedback. In a sense, we want to use the non-haptic informationto construct a virtual fixture (or perhaps more accurately toperform a “haptic transformation”). It is familiar in surgicalrobotics, for instance, to enhance a surgeon’s sense of touch byamplifying forces, but can we provide useful haptic feedback fromsensory information that was not haptic to begin with?

One such example is the SmartTool [5] developed by theUniversity of Tokyo. The SmartTool consists of a handheld device that contains various sensors, connected to a robotic arm. Control software running on a host computer analyzes the sensorreadings, commanding motors in the robotic arm to apply forcesthat guide the operator away from certain locations. Nojima et aldemonstrated cutting the white of a hard-boiled egg withoutdamaging the egg yolk with the SmartTool used as a knife [6].When sensors detected the interface between egg white and yolk,the robotic arm applied a feedback force to make it difficult forthe operator to penetrate the yolk. The implication to surgery isthat similar tools could be used to prevent the surgeon frominadvertently cutting critical structures such as arteries.

The SmartTool demonstrated that intelligent instrumentationcould be constructed to convert non-human sensory input toperformance-enhancing haptic feedback. Through sensors andcomputerized robotic systems our human senses can be extended—indiscernible information about the environment canbe translated into a form that our body can immediately understand. The tactile feedback laser is another application ofthe haptic transformation idea. It converts optically measureddistance information into haptic feedback, making the laser moreuseful because the operator can “touch” the surface with light.

2.2 Surgical Robotics

The explosion of surgical robotics is exciting for the ARcommunity because certain surgical robots are built around visual-haptic platforms ripe for AR experiments. Many robots are alsodeployed in experimental operating rooms, providing a directmethod to verify AR efficacy.

Our group at the Seaman Family Magnetic Resonance (MR) Research Centre, University of Calgary, is developing neuroArm:an MR-compatible image-guided robot for microsurgery [7, 8].NeuroArm consists of a master-slave robot (Fig. 2), a controller,and a workstation (Fig. 3). The workstation provides visual, audio, and tactile feedback, creating an immersive environmentfor the surgeon. The surgeon operates the robot using a set ofhandcontrollers to manipulate the robot end-effectors. Thesehaptic controls provide force-feedback, allowing the surgeon to feel everything the robot feels. The motion of the arms can bescaled down such that when the surgeon moves a handcontrollerby 10mm, the robot arm moves only 1mm. Similarly, the force feedback can be scaled up such that if the robot detects 10mN of force, the surgeon feels 100mN.

2.3 Tactile Feedback in Surgical Lasers

Despite the potential benefits of lasers in surgical and otherprocedures, the lack of tactile feedback has historically been animportant factor preventing their widespread adoption [1, 2]. Theproblems associated with no tactile feedback have swayed manysurgeons to continue using knives, drills and other hand tools, rather than adopting laser technology [1]. Industry has begun to capitalize on the tactile-feedback problem. There are productssuch as the SoloGrip®III by Cardio Genesis Corporation [9] and the Contact Laser Tip technology developed by Surgical LaserTechnologies Inc. [10] that claim to give tactile-feedback.However, the tactile-feedback is the consequence of a mechanicalstop contacting a surface. Furthermore, important lasers such asthe ubiquitous CO2 laser are not suitable for this type of delivery[11].

True laser tactile-feedback will provide the surgeon with atouchless sense of touch while operating a laser. However, it isphysically impossible for a laser held freehand to provide true

258Proceedings of the IEEE Virtual Reality 2005 (VR’05)

1087-8270/05 $20.00 © 2005 IEEE

Page 3: [IEEE IEEE Proceedings. VR 2005. Virtual Reality, 2005. - Bonn, Germany (March 12-16, 2005)] IEEE Proceedings. VR 2005. Virtual Reality, 2005. - The XYZ virtual workspace

Fig. 4. Through a master-slave robot, a surgical laser could be given a sense of touch. The laser used in the distance measurement modulecould be the surgical laser, and need not be physically separated.

tactile feedback. The problem arise from the fact that nothing butlight ever touches the surface—there is no way for this light to generate a detectable opposing force. But with a robotic armholding the laser instead, it is possible to create a sense of touch.

3 FUNCTIONAL DESCRIPTION OF CONCEPTUAL SYSTEM

Fig. 4 shows a conceptual diagram of the tactile feedback lasersystem integrated with a haptic master-slave robot such asneuroArm. (The surgical laser and the distance measurementmodule in Fig. 4 need not be physically distinct.) The surgeonwill direct the laser towards the surface using the standard handcontroller for moving the robot’s end-effector. As the focal pointof the laser enters a surface, the surgeon will feel an opposing force that rapidly increases with increasing penetration depth.The surgeon will experience the sensation of a solid surfacealthough nothing but light will actually be touching it. If thesurgeon maintains constant force and moves the laser laterally, itwill track the profile of the surface as shown in Fig. 5. Once thefocal point is in contact with the surface, downwards force will

control the intensity of the surgical laser as shown in Fig. 6. Thetrajectory of the laser will also appear in the workstationmicroscope and on the desk-mounted displays, providing the lastof the missing sensory cues.

Will all surfaces feel the same? It would be impossible (or atleast extremely difficult) to make the compliance of the virtualsurface felt by the operator equal to the compliance of the realsurface. However, providing haptic feedback that indicates thefraction of light absorbed by the surface is potentially more useful.It is well known that optical absorption and reflection coefficientsdepend on material and wavelength. Assume that the penetrationdepth of the light into the surface is negligible; i.e. all the lightenergy is either absorbed at the surface or scattered back into theenvironment. The prototype’s laser distance measurement module (Section 4) collects this scattered light to make a distancemeasurement. However, a second piece of information exists: theintensity of the scattered light. The amount of light absorbed bythe surface is related simply to the scattered intensity. Since it isthe absorbed laser light that performs the cutting, adjusting thecompliance and friction of the virtual surface based on scatteredlight intensity may provide important feedback to the surgeon.Surfaces that absorb very little light could be made to feel harderwith less friction than surfaces that absorb most of the light. Thecompliance of the virtual surface would then indicate to thesurgeon (albeit loosely) how quickly the real surface would ablate.This logic is not perfectly sound if you allow for the surface to be partly transparent (as in the case of biological tissue). Consider an

Fig. 5. The tactile feedback laser system tracks the profile of asurface as though it is making physical contact. This “’virtualsurface” possesses friction and compliance, which in the futurewill be dynamically adjusted based on sensor information.

Fig. 6. Operator applied force will control the intensity of the surgical laser.

259Proceedings of the IEEE Virtual Reality 2005 (VR’05)

1087-8270/05 $20.00 © 2005 IEEE

Page 4: [IEEE IEEE Proceedings. VR 2005. Virtual Reality, 2005. - Bonn, Germany (March 12-16, 2005)] IEEE Proceedings. VR 2005. Virtual Reality, 2005. - The XYZ virtual workspace

imaginary transparent tissue that transmits 95% of the lightwithout any absorption, scattering the remaining 5%. Such asurface would feel soft and suggest to the operator that it could be cut quickly, but of course it would not cut at all. Furthercomplications such as ablation energy, heat conduction, andsmoke corrupting the distance signal have not been consideredeither. Exploring the possibilities of the conceptual system ingreater detail should wait until more experimental results areavailable.

4 PROTOTYPE

The prototype was designed to permit the operator to touchsurfaces with light, without the added complication of integratinga high-power cutting laser. The prototype (Fig. 1) consists of asimple 3-DOF robotic arm with motorized actuation in only thevertical direction. The operator manipulates a novel laser distance measurement module attached to the distal end of the robotic arm.When the laser is far above an object, it can be manipulatedrelatively freely. However, when the focal point of the laserimpinges a surface, feedback forces are generated, giving theoperator the impression of touching a solid object.

The laser distance measurement module images the spot on the surface formed by the laser onto a detector array. It then uses thesize and shape of this image to estimate the focal point-to-surfacedistance. The laser distance measurement module is described indetail in [12]. The tower that connects the arm to the basecontains a linear actuator used to both generate force-feedbackand permit motion of the laser in the vertical direction. Linearactuation is achieved by coupling a 4W DC motor with integratedencoder to a miniature ball screw. Ground shafts attached to theupper portion of the tower slide inside linear bearings seated inthe lower portion of the tower, thereby preventing rotation. At thedistal end, a mechanical contact sliding on rails and connected tothe laser’s grip transfers operator-applied force to a load cellsitting in a slot near the laser. A microcontroller controls theactuator in response to reading from the distance measurement module and the force sensor. The control algorithm allows the laser to be moved relatively freely in the vertical direction whenfar above a surface. A proportional controller sets the targetvelocity of the laser proportional to the applied force, resulting in a viscous feel. Meanwhile, the microcontroller interrogates thelaser distance measurement module. When the focal pointimpinges a surface, the control loop is modified by adding forcefeedback that increases rapidly with focal point penetration depth. The surface feels solid with compliance controlled by a variable inthe microcontroller software.

5 CONCLUSION

It is our hypothesis that incorporating tactile feedback and visiblelaser trajectory will provide sensory cues that enhance the safety,performance, and acceptance of surgical laser technology. Aconceptual system was presented for use with a surgical robot such as neuroArm. To experiment with the concept of lasers withtouch, a prototype tactile feedback laser system was constructed.Although the prototype served its purpose as a proof-of-conceptdevice, it had limitations primarily related to the design of the

robotic arm. The orientation of the laser was fixed and hapticfeedback was constrained to the vertical direction, making it impossible to add friction. Also, free motion in the verticaldirection was more impeded than in the horizontal plane.Commercial haptic hand controllers permit 6-DOF motion withlittle impedance and can render a force vector in any direction.With a laser distance measurement module attached as the stylusof a haptic hand controller, the operator would be able tomanipulate the laser almost as freely as if it were held free hand.Also, using velocity feedback from the hand controller, the virtualsurface could be given friction. Thus the incorporation of thelaser distance measurement module with a haptic hand controllerwould facilitate a new level of reality for the operator whenfeeling surfaces with light. This is our next step in the initiativeto develop a tactile feedback laser system for robotic surgery.

ACKNOWLEDGEMENT

This material is based on work and infrastructure funded by theCanadian Foundation for Innovation, Canadian Institutes of Health Research, and the Alberta Heritage Foundation forMedical Research.

REFERENCES

[1] J. Garrick, and N. Kadel, “The CO2 Laser in Arthroscopy: Potential

Problems and Solutions,” Arthroscopy, 7 (2), pp. 129-37, 1991.

[2] S. S. Baker, J. M. Hunnewell, W. S. Muenzler, and G. J. Hunter,

“Laser Blepharoplastly: Diamond Laser Scalpel Compared to the

Free Beam CO2 Laser,” Dermatol Surg, 28 (2), pp. 127-131, 2002.

[3] P. D. Le Roux, H. Das, S. Esquenazi, and P. J. Kelly, “Robot-

Assisted Microsurgery: A Feasibility Study in the Rat,”

Neurosurgery, 48 (3), pp. 584-589, 2001

[4] L. Rosenberg, “Virtual Fixtures: Perceptual Tools for Telerobotic

Manipulation,” Proceedings of the IEEE Virtual Reality

International Symposium, Seattle, September 18-22, pp. 76-82, 1993.

[5] T. Nojima, D. Sekiguchi, M. Inami, and S. Tachi, “The SmartTool:

A System for Augmented Reality of Haptics,” Proceedings of the

IEEE Virtual Reality Conference, Orlando, March 24-28, 2002.

[6] R. T. Azuma, “A Survey of Augmented Reality,” Presence:

Teleoperators and Virtual Environments, 6, pp. 355-385, 1997.

[7] D. F. Louw, T. Fielding, P. B. McBeth, D. Gregoris, P. Newhook, G.

R. Sutherland, “Surgical Robotics: A Review and Neurosurgical

Prototype Development,” Neurosurgery, 54, pp. 525-537, 2004.

[8] P. R. Rizun, P. B. McBeth, D. F. Louw, and G. R. Sutherland,

“Robot-Assisted Neurosurgery,” Seminars in Laparoscopic Surgery,

11 (2) June, pp. 99-106, 2004.

[9] Cardio Genesis website, http://www.cardiogenesis.com/.

[10] Surgical Laser Technologies website, http://www.slti.com/.

[11] G. Absten, M. Kochman, H Wigdor, J. T. Cox, E. Lach, J.

McCaughan, B. Shumaker, and R. Lauzafame, “Overview of

Clinical Laser Applications,” Publication of the Professional

Medical Education Association, Marathon Shores, Florida.

[12] P. Rizun and G. Sutherland, “Tactile Feedback Laser System With

Applications to Robotic Surgery,” Proceedings of the World Haptics

Conference, Pisa, Italy, March 18-20, 2005.

.

260Proceedings of the IEEE Virtual Reality 2005 (VR’05)

1087-8270/05 $20.00 © 2005 IEEE