Abstract— The paper presented herein describes the

development of an advanced robotized system applied to in

vitro implant dentistry research. To the biomedical community

this paper shows the possibilities of industrial robots to help

the research. Robots are special suitable to the biomedical field

specially when integrated with advanced sensors and

technologies, which may facilitate both the programming tasks

and the data acquisition. Robotics researchers will find in this

paper one of the first applications of programming by

demonstration with real users with a novel explicit robot

programming technique making use of multi-camera vision

combined with speech recognition. This programming method

targets users who have minimal robot experience but aims at

‘teaching’ the robot to execute a specific task.

Keywords – programming-by-demonstration; service-oriented

architectures; dentistry

I. INTRODUCTION

esearch in implant dentistry is significantly constrained

by the availability of reliable in vivo data come from

clinical cases. The inherent differences between patients

and the high variability of the implanting procedure turns

collected data more imprecise and difficult to work with. To

overcome this limitation we developed a versatile robotic

system capable of executing several research procedures

mimicking all the process of the implant technique and

some of the living functions such as mastication.

Robotics in medical research is not a novelty. Robots

perform different tasks like automatic dispensing in

pharmaceutical labs, mechanical tests of spinal joints [1] or

stress/strain analysis in implant dentistry [2]. The choice of

the robot as the principal element of the research system

was driven by the needs of flexibility and repeatability.

Assembled with a force/torque sensor and with appropriate

software [3] this system presents high repeatability: less

than 0.01 mm in position and less than 0.1 N in force. The

global accuracy of the system enables highly reliable

Manuscript received January 31, 2012.

procedures that are being used, with minimum data, to

validate different tests regarding misalignment caused by

impression techniques, implants placing and mastication

force distribution under numerous scenarios.

It should be noticed that the complexity associated with

the research procedures promotes the interleaving between

the researcher (dentist) and the robot: the robot executes

operations when high degree of repeatability is required,

like drilling, or highly repetitive tasks like photo-shooting

p.e. On the other hand, the human (researcher) executes

tasks non-critical for the reliability of the research

procedures and also complex manipulation tasks like

screwing the prostheses p. e..

Robot programming in this scenario is mainly a two step

procedure: In the first step the position of one feature is

determined by the researcher and afterwards the robots

performs the operation with a set of different parameters.

For mastication for example the researcher determines the

position of the tool in the teeth and the robot applies the

force in that position with predefined directions and forces.

Our first experiments shown that the second step can easily

be performed independently by the researcher with the help

of appropriate software. Nonetheless, for the first step the

presence of a robotics expert is mandatory. The alternative

is the researcher program himself the robot, which

represents a great challenge since it has non expertise in the

field, is focused on other demanding tasks (research) and

have to move his attention to do extensive reprogramming

and adjustment.

Although offline programming tools have their important

(growing) market, industrial robots are still very dependent

on online teach pendant programming. This technology has

suffered a great evolution over the past years [4] but the

basic concept is still the same, and it is not well adapted to

the scenario described above. Therefore, new paradigms for

online robot programming are needed, with special focus on

explicit programming techniques that can bring the worker

or researcher closer to the robot. Recent developments

On the use of Robotics in implant dentistry research.

Germano Veiga1*, Francisco Caramelo2, Pedro Malaca3, Pedro Brito2,4 and J N Pires1,

1Mechanical Engineering Department, Faculty of Sciences and Technology, University of Coimbra, Portugal

2 Institute of Biomedical Research on Light and Image -Faculty of Medicine, University of Coimbra, Portugal

3Sarkkis Robotics Lda, Coimbra, Portugal

4Institute Health Sciences of Porto, Porto, Portugal

*Present address: INESC Porto, Porto, Portugal

R

The Fourth IEEE RAS/EMBS International Conferenceon Biomedical Robotics and BiomechatronicsRoma, Italy. June 24-27, 2012

978-1-4577-1198-5/12/$26.00 ©2012 IEEE 1596

presented by a robot manufacturer have embedded high

speed force control feedback into the robot controller [5],

which provides good perspectives in terms of advanced

sensor integration, crucial to improve the man/machine

interaction. These systems have been around in research for

a long time but their industrialization opens possibilities in

mass products with advanced teaching techniques, namely

Programming-by-demonstration [6][7].

The original contribution of the present work seeks two

audiences, the research community in the biomedical area

and the robotics researchers dealing with human-robot

interaction. To the biomedical community this paper shows

the possibilities of industrial robots to perform research

specially when combined with advanced sensors and human

robot interface technologies. Furthermore, a novel

technique for the measurement of gaps between implants

and the prosthesis is described. The robotics researchers

will find in this paper one of the first applications of

programming by demonstration with real non-expertise

users, and the presentation of a novel explicit robot

programming technique making use of multi-camera vision

fused with speech recognition.

II. ROBOTICS IN IMPLANTOLOGY RESEARCH

A. Comparative study of impression materials

We developed a method for measuring the fitness of

prosthesis and the corresponding dental implants. Since a

robot arm is use for executing the dental impressions and

part of the measuring procedure we gain high precision and

reproducibility. This “in vitro” method for evaluating

impression (methods and materials, see Fig. 1) that avoids

the operator for executing both the impression and the

measuring.

Fig. 1 Robot performing the impression procedure.

The proposed method is based on the use of a robot arm for

performing the impression. For measuring the gap, between

a realistic mandible made of a photoelastic polymer and the

prosthesis obtain from the impression, we use the robot to

move the high quality photo camera and take 6 zoom photos

of each of the implants (Fig. 2).

Fig. 2 Left: schematic representation of the photo arrangement for

measuring the gap between the plaster mandibles and the zirconia dental

prosthesis. Right: photo of the actual arrangement.

Finally the results were analyzed with specifically

developed software (Fig. 3).

Fig. 3 Photo of one implant and part of the prosthesis. The gap is well

defined as the space between the red and the blues lines. The distance

between the blue and the green line is used as reference.

B. Photoelastic analysis of implant arrangement.

The mastication force distribution inside the mandible is

decisive in the durability of dental implants. The purpose of

this study is to do a qualitative descriptive analysis of stress

patterns around four different implants embedded in photo-

elastic mandible.

The industrial robot equipped with a force torque sensor

produces force controlled movements that mimic

mastication forces that are propagated through the implants.

Photo-elastic mandible acrylic resin models were die casted

with four implants placed vertically and parallel between

mentonian hole. Zirconia prosthesis was screwed to the

implants and afterwards subjected to different loads (100N

150N and 200N) in 12 points with an axial force and 30ª

tilted (vestibular and lingual face).

Photoelastic analysis was accomplished using a reflection

polariscope. The fringe patterns produced in the

photoelastic resin for each implant and load were

photographed with a digital camera. Fringe concentrations

and the highest fringe order were recorded and described

for the apical, central, and coronal regions of all the

implants for each load scenario.

1597

Fig. 4 Left: Robot performing mastication simulation under polarized

light. Right: Results for strain stress analysis. .

III. PROGRAMMING-BY-DEMONSTRATION

The interaction technologies used in the PbD system

presented in this work are an infrared stereo vision system

and a voice recognition system. The orchestration of these

systems together with the robot is achieved using a service-

oriented platform and a high level programming

environment. The description of the methodology and

implementation is presented in sections III.A, III.B, III.D

A. Stereo Vision

The definition of trajectories is one of the crucial aspects in

robot programming. Thus, accurate detection of position in

a known frame is essential to accomplish this task.

Although, there are several ways of performing such

assignment artificial vision seems to be the easiest to be

employed. However, when just one camera is used the

problem of finding the correspondence between one point

in the image and the point in the real world is undetermined

since the dimension related to depth is lost. If other

information is employed, such as other images taken from

different angles, the 3D information can be recovered by

proper triangulation.

In a broader sense, three-dimensional reconstruction

depends on the initial information that is available. If the

extrinsic and the intrinsic parameters of the camera are

known the triangulation can be solved unambiguously.

However, if only the intrinsic parameters are known,

reconstruction is still possible but lacking one scaling

factor. Even if intrinsic and extrinsic parameters are

unknown reconstruction can be still performed based on the

known correspondences but up to an unknown global

projective transformation [8].

In the present work we were interested in recovering the

complete 3D information, which was achieved by

triangulation [8].

Figure 1 shows a schematic representation of the process:

lines l and r passing through the points in the images ( pl

and rp ) and the corresponding center of projection ( Cl

and rC ) would intersect at point P in the absence of noise.

However, the presence of noise inhibits in most cases that

the intersection exists. Therefore, P must be computed as

the point that simultaneously minimizes the distance to both

lines l and r .

pl

rp

rC

Cl

P

r

l

Fig. 1. General case of triangulation used in stereo vision in which the

rays do not intersect in space. P is the midpoint of the line that is

perpendicular to both rays l and r. (Adapted from [8]).

Point P can be compute by:

2

T

e da p T b R pP

+ += (1)

where a and b are calculated from:

( )T T

e e d da p c p R p T b R p+ × = + . (2)

R and T are defined by:

T

rR R R=l

T

rT T R T= −l

(2)

rR and Rl are the rotation matrix that represent the

orientation of right and left cameras, respectively and rT

and Tl are the position of the centers of projection.

In order to determine the parameters of the cameras we

used the method presented by [9] that was implemented in

C#. In this method, a planar calibration object is placed in

different orientations and their images are used to obtain the

camera parameters. The parameters are initially estimated

with a closed form solution and then refined using an

optimization step. Radial lens distortions are also modeled.

1598

Fig. 5 The use of the infrared pen to define positions for mastication

simulation in a prosthesis held on a photo-elastic mandible.

B. Robot Jogging and robot services

The use of one single infrared LED provides significant

advantages in terms of maneuverability in small spaces and

in the required simplicity of use, but limits the jogging of

the robot to translations. To overcome this issue we

developed a simple strategy that uses the definition of a

rotation vector making use of two points, according to the

Fig. 6. The tool center point and the two points define a

plane and an angle. The robot reorients the specified angle

around the normal vector of the plane.

Fig. 6 Reorienting the robot with two points

It should be noticed that the translation movements of the

robot can be made in either incremental or point-to-point

mode (absolute positioning). The incremental movements

permit to overcome some accuracy limitation of the stereo

vision system.

The robot services were automatically generated from the

robot program using software previously developed in the

lab [10]. This software reads properly tagged robot

programs and automatically generates network services.

C. Speech recognition system.

With the introduction of the language models, of which

the n-gram [12] is by far the most used, speech recognition

systems achieved remarkable efficiency levels.

Despite the need for further work on the recognition of

conversational speech, speech recognition technologies

regarding read speech are mature and ready to complement

other means of human-machine interaction. These advances

leaded to the development of several commercial products

that are now reaching wider markets.

However, from a practical point of view, the most robust

(speaker-independent) voice-enabled systems present in the

market are still working in command mode, which means

that the recognition is made against a predefined set of

commands defined with special purpose grammars [13]

[14].

In this work we use voice recognition grammars with a set

of commands that allowed the researcher to complement the

operation through the stereo vision system using voice

commands to change the jogging mode and activate other

functions (like photo shooting p.e.). The implementation of

the network service for the voice recognition also uses

automatic generation. Instead of robot programs this

automatic generation uses the grammar from the voice

recognition system (see [11] for details.).

D. Orchestration of services.

The concept behind the orchestration of services (or

composition in a larger sense, according to [15]) is derived

from the one introduced by distributed component-oriented

programming, which, like in many other WS*-related

technologies, was extended and standardized using XML

specifications. From the evolution process, a standard has

emerged as the most promising: Business Process

Execution Language for Web Services (BPEL4WS) [16].

The integration of the voice recognition system and the

infrared vision system was based on a service-oriented

architecture using two well know platforms (UPnP [17] and

DPWS [18]). In these platforms, network services are

composed by events and actions and their integration of the

programming-by-demonstration system is a partially

unsolved problem. In the context of constrained purpose

devices (like TV’s, p.e.) the use of such architectures is a

success, but in the flexible scenario of a robotic workcell

their major advantages is their integration with flexible

integration languages/platforms. As described in [11], in

spite of the existence of long-term consistent efforts on the

specification of languages for the orchestration of services

[19] [20], the event-driven nature of the robotic setup and

the inadequacy of business level orchestration languages

1599

(like BPEL4WS) leads us to the use of state-chart based

language and the respective environment previously

developed on our lab (SidneyChart [21]) to orchestrate our

services.

Fig. 7 SidneyChart ([21]) interface with the statechart of the PbD system.

The easy-of-use of the SidneyChart allowed the

researcher (dentist) to perform himself some orchestration

whenever a service changed, due to the change of the voice

recognition grammar p.e..

IV. RESULTS

Concerning the stereo vision system the absolute error

obtained with the present setup makes difficult the use of

the robot to the intended application in absolute mode, but

the incremental mode performance was satisfactory from

the point of view of the researcher.

The performance of the voice recognition system is very

high (>95%), due to the low number of commands used and

the quality of the microphone.

V. CONCLUSION

The evaluation of the developed equipment in this work

can be made from different perspectives. From the

implantologist/researcher point of view the equipment

provides the flexibility and repeatability required for the

research procedures: 0.01mm in position, less than 0.1 N in

force and when equipped with a high resolution camera is

able to measure down to 1 micrometer of gap between

implant and prosthesis. The global accuracy achieved with

this system enables highly reliable procedures that are

currently being used to validate different tests regarding

implant misalignment caused by different impression

techniques and mastication force distributions under

numerous implants placing scenarios.

From the Human-robot interaction perspective this paper

describes a programming-by-demonstration technique and

its evaluation with a real user (the researcher). The results

are very satisfactory. The researcher is now able to define

new research tasks independently and even perform some

orchestration of network services. This experiment shows

that with proper tools in terms of human robot interface and

advanced software tools for the integration of resources

robots can perform a wide variety of tasks in new markets.

ACKNOWLEDGMENT

REFERENCES

[1] H. de Visser, C. Rowe, and M. J. Pearcy, “Robotic

Testing Facility for the Measurement of the Mechanics

of Spinal Joints,” Proc ImechE Part H: J Eng in Med,

vol. 221, no. 3, pp. 221-227, 2007.

[2] J. N. Pires, F. J. Caramelo, P. Brito, J. Santos, and M.

F. Botelho, “Robotics in implant dentistry: stress/strain

analysis. System overview and experiments,”

Industrial Robot: An International Journal, vol. 33,

pp. 373-380, Sep. 2006.

[3] J. N. Pires and J. M. G. Sa da Costa, “Object-oriented

and distributed approach for programming robotic

manufacturing cells,” Robotics and Computer-

1600

Integrated Manufacturing, vol. 16, pp. 29-42, Feb.

2000.

[4] A. Kazi, J. Bunsendal, D. Haag, R. Baum, and R.

Bischoff, “Next Generation Teach Pendants for

Industrial Robots,” Advances in Human-Robot

Interaction, 2005. [Online]. Available:

http://www.springerlink.com/content/ac15e5b777e373

d5.

[5] ABB, “ABB IRC5 Application Manual Force Control

Machining FC,” Vasteras, 3hAC027595-001, 2007.

[6] Pires, J. N., Veiga, G., and Araujo, R., “Programming

by demonstration in the coworker scenario for SMEs,”

Industrial Robot, Emerald, vol. 36, no. 1, 2008.

[7] J. Hofschule, “Lead Through Programming,” 2008.

[Online]. Available:

http://www.smerobot.org/09_calendar/20071108_Press

etagABB-Ladenburg_LeadThrough.pdf. [Accessed:

21-May-2009].

[8] E. Trucco and A. Verri, Introductory Techniques for

3-D Computer Vision. Prentice Hall PTR, 1998.

[9] Z. Zhang, “A Flexible New Technique for Camera

Calibration,” IEEE Trans. Pattern Anal. Mach. Intell.,

vol. 22, no. 11, pp. 1330-1334, 2000.

[10] G. Veiga and J. . Pires, “Using robot programming

languages to specify service contracts,” in Proceedings

of the CONTROLO’2008 Conference, 2008.

[11] G. Veiga, P. Malaca, K. Nilsson, and J. N. Pires,

“Separation of concerns on the orchestration of

operations in flexible manufacturing,” Assembly, vol.

32, no. 1, 2011.

[12] C. Manning and H. Schütze, Foundations of

Statistical Natural Language Processing. The MIT

Press, 1999.

[13] “Voice Extensible Markup Language (VoiceXML)

Version 2.0,” 2010. [Online]. Available:

http://www.w3.org/TR/voicexml20/. [Accessed: 24-

Nov-2010].

[14] “Speech Recognition Grammar Specification Version

1.0,” 2010. [Online]. Available:

http://www.w3.org/TR/speech-grammar/. [Accessed:

23-Nov-2010].

[15] C. Pautasso and G. Alonso, “Visual composition of

web services,” 2003, pp. 92-99.

[16] “OASIS Web Services Business Process Execution

Language (WSBPEL) TC,” 2010. [Online]. Available:

http://www.oasis-

open.org/committees/tc_home.php?wg_abbrev=wsbpel

. [Accessed: 18-Dec-2008].

[17] “Universal Plug and Play.” [Online]. Available:

http://www.microsoft.com/whdc/device/media/upnp.ms

px. [Accessed: 01-Dec-2008].

[18] “OASIS Devices Profile for Web Services (DPWS),”

2010. [Online]. Available: http://docs.oasis-

open.org/ws-dd/ns/dpws/2009/01. [Accessed: 09-Mar-

2010].

[19] F. Jammes, H. Smit, J. L. M. Lastra, and I. M.

Delamer, “Orchestration of service-oriented

manufacturing processes,” presented at the Emerging

Technologies and Factory Automation, 2005. ETFA

2005. 10th IEEE Conference on, 2005, vol. 1, p. 8 pp.-

624.

[20] A. W. Colombo, F. Jammes, H. Smit, R. Harrison, J.

L. M. Lastra, and I. M. Delamer, “Service-oriented

architectures for collaborative automation,” presented

at the Industrial Electronics Society, 2005. IECON

2005. 31st Annual Conference of IEEE, 2005, p. 6 pp.

[21] P. Malaca, G. Veiga, and N. Pires, “SidneyChart: A

statechart GUI for SOA orchestration in autonomous

industrial systems,” in Industrial Informatics (INDIN),

2011 9th IEEE International Conference on, 2011, pp.

238-243.

1601