An Optimal Trajectory Planner for a Robotic Batting Task: The …wpage.unina.it/diana.serra/ICINCO_2016_104_CR.pdf · An Optimal Trajectory Planner for a Robotic Batting Task: The

An Optimal Trajectory Planner for a Robotic Batting Task: The TableTennis Example

Diana Serra1, Aykut C. Satici2, Fabio Ruggiero1, Vincenzo Lippiello1 and Bruno Siciliano1

1Department of Electrical Engineering and Information Technology, University of Naples Federico II,Via Claudio 21, 80125, Naples, Italy

2Electrical Engineering and Computer Science, Massachusetts Institute of Technology,32 Vassar Street, Cambridge, Massachusetts, U.S.A.

{diana.serra, fabio.ruggiero, vincenzo.lippiello, siciliano}@unina.it, [email protected]

Keywords: Optimal Trajectory Planning, Table Tennis Robot, Nonprehensile Manipulation.

Abstract: This paper presents an optimal trajectory planner for a robotic batting task . The specific case of a table tennisgame performed by a robot is considered. Given an estimation of the trajectory of the ball during the free flight,the method addresses the determination of the paddle configuration (pose and velocity) to return the ball at adesired position with a desired spin. The implemented algorithm takes into account the hybrid dynamic modelof the ball in free flight as well as the state transition at the impact (the reset map). An optimal trajectory thatminimizes the acceleration functional is generated for the paddle to reach the desired impact position, velocityand orientation. Simulations of different case studies further bolster the approach along with a comparisonwith state-of-the-art methods.

1 INTRODUCTION

Interest in robotic manipulation without grasping theobjects, i.e., nonprehensile manipulation, is recentlyrising. The presence of unilateral constraints in-creases the complexity in planning and control therobot that has to dynamically drive the manipulatedobject from the initial to the desired configuration.Nevertheless, as highlighted by (Lynch and Mason,1999), nonprehensile manipulation allows controllingextra degrees of freedom. The workspace is thusincreased since the manipulated object can physi-cally overcome the kinematic limitations of the end-effector by continuously breaking and creating con-tacts with the robot. Therefore, a nonprehensile ma-nipulation task is in general complex, skillful and dex-terous. Normally, it can be undertaken by splittingthe complex task in many simpler subtasks, usuallycalled primitives, such as rolling, sliding, throwing,catching, pushing and batting. The latter is the oneaddressed in this paper. Batting combines catchingand throwing in a single collision: a practical exam-ple is given by the table tennis game. Such a taskrequires so high velocities and precision that roboticcompanies take it as an example to display the highperformances of their products. For instance, Kukahas chosen the table tennis game to promote its waresin a thrilling commercial spot (Kuka, 2014), showing

the potential abilities of robots. The Omron automa-tion company has also broadcast a video showing itsparallel Delta robot playing table tennis and coachinghumans at CEATEC Japan 2015 exhibition, (Omron,2015).

The robotic batting task can be tackled with differ-ent approaches. From the artificial intelligence pointof view, for instance, the aim is to improve the per-formance of the robot through experience, while alsodealing with temporal issues; from the planning andcontrol points of view, instead, accurate models areresearched and investigated for autonomous and fastmotion execution; from the perception side, finally,fast and reliable measurements are requested. Thispaper is focused on planning and control aspects, in-herently considering the hybrid nature of the systemand real-time constraints.

Generally, a robotic system for a batting task (e.g.,table tennis) should be provided by: a vision systemto track the motion of the ball, a method to estimatethe trajectory of the ball in free flight and its spin,a decision system to choose the paddle configurationto direct the ball towards the desired position on theopposite court, and a trajectory planner for the motionof the paddle.

The scope of this paper is to provide an optimaltrajectory planner for the paddle to return the ball tothe opposite court at a desired position with an im-

posed spin. At the same time, an improvement ofthe decision system proposed by (Liu et al., 2012) tochoose the configuration of the paddle at the impacttime is here described. The proposed algorithm con-sists of two main phases: the hybrid model solution,computing the state of the paddle at the impact timesuch that the control objective is satisfied, and the tra-jectory optimization in SE(3), providing the angularand linear trajectory of the paddle up to the impacttime.

The novelties introduced by this manuscript aretwofold.• Firstly, in comparison to the state-of-the-art, the

proposed method improves the control accuracyby considering a full aerodynamic model of theball, and taking into account drag and lift forces.The computation time of the algorithm is alsofast enough to guarantee it can suitably be imple-mented in real-time.

• Secondly, rigorous methods from calculus onmanifolds are borrowed to generate an optimaltrajectory on SE(3) for the paddle to strike the ballat the impact time.

2 RELATED WORK

(Andersson, 1989a) describes a tracking system anda trajectory generation for a table tennis robot player.The method uses fifth-order polynomials to generatea trajectory for the paddle intercepting the ball. Thearchitecture is designed so that the task level controlcan monitor the torques of the robot during the move-ments of the arm, and also adjust the trajectory whilethe ball is in free flight. (Andersson, 1989b) showshow the processing time of a stereo vision systemshould be taken into account when it is applied to aquickly changing operating environment. A table ten-nis robot player prototype equipped with two levelscontrol system is described by (Acosta et al., 2003).The objective of the lower level is to move the cen-ter of the paddle to the right position to intercept theball. The high-level control defines instead the gamestrategy. This is done by orienting the paddle to returnthe ball to the desired position on the table. The pro-totype is capable of returning balls with a good suc-cess rate when their velocity is below 5 m/s and undersmall spin effects. A high-speed trajectory planner isdescribed by (Senoo et al., 2006) and it is applied tothe batting task. The authors also consider the robotdynamic model within their framework, and they relyupon a 1kHz high-speed vision system. Nevertheless,they do not go into details about the controller com-putational efforts, and they use a simplified version

of the reset map in which the spin of the ball afterthe impact is not considered. Spin is indeed relevantfor the table tennis game. Moreover, the trajectoryplanner they use is based on polynomial primitives,which has been demonstrated to demand high jointaccelerations. (Nakashima et al., 2010) show how toobtain the linear and angular ball trajectory informa-tion from the visual sensors. On the other hand, (Liuet al., 2012) propose a control method for returning atable tennis ball to a desired position with a desiredspin. The method determines the state of the paddleemploying the hybrid dynamic model of the ball infree flight as well as the state transition at the impact.In order to determine the control action, an approx-imated aerodynamic model is considered, providinga satisfactory but less accurate results. (Nakashimaet al., 2014) follow up the work of (Liu et al., 2012) byfocusing on the nonlinear aerodynamics solution. Afinite difference method and a linearization of the dis-cretized model are employed to solve the two bound-ary values problem so as to reduce the computationaltime. However, they present fewer case studies than(Liu et al., 2012), while the obtained results are notso easy to reproduce due to a lack of implementationdetails.

The artificial intelligence research community isapplying many learning techniques to the robotic ta-ble tennis task. (Matsushima et al., 2005) describean approach to perform the table tennis task basedon learning. Some impressive table tennis games be-tween two humanoid robots are performed using theadaptive trajectory prediction developed by (Zhanget al., 2012). This model involves an offline train-ing of the parameters on the base of the recorded stateof the ball. Afterwards, the model parameters are on-line adapted for estimation and prediction processes.Other approaches reproduce the movements learnedfrom human demonstrations. (Mulling et al., 2010)pursue trajectory generation for the robotic table ten-nis task from a biomimetic point of view. In the ap-proach introduced by (Mulling et al., 2013), the robotfirst learns a set of elementary table tennis hittingmovements from a human table tennis teacher, andthen generalizes those movements in a wider rangeof situations using a mixture of motor primitives ap-proach. (Huang et al., 2013) propose an active learn-ing approach where the initial parameters related tothe paddle are computed through a locally weightedregression method. A simplified hitting scenario taskis implemented by (Oubbati et al., 2013), where arobot arm hits a ball rolling on an inclined planeplaced in front of the robot. The authors propose amodel that autonomously generates and organizes se-quences of timed actions. The timing of the move-

ments is controlled by nonlinear oscillators. Their ac-tivation and deactivation are coordinated by a hierar-chical neural dynamic architecture. (Yanlong et al.,2015) find optimal striking points for a table tennisrobot. The choice is based on a reward function,measuring how well the trajectory of the ball and themovement of the paddle coincide. Given the strikingpoint, a stochastic policy over the reward is derived toevaluate prospective striking points sampled from thepredicted rebound trajectory. In that approach, the re-sulting learning method takes into account the amountof experience data and its confidence.

Recently, researchers of aerial robotics have alsobeen interested in the batting problem. A control ap-proach to juggle a ball between either a human anda quadrotor or two quadrotors is showed by (Mulleret al., 2011). (Silva et al., 2015) tackle the task ofhitting a table tennis ball with a commercial drone.The ball tracking relies only upon the onboard cam-era and not upon an external motion capture system.The estimation of the hitting point is performed us-ing a variation of regularized kernel regression, wheresamples are weighted according to their relevance tothe task. The decision phase determines the hittingmotion from a set of primitives learned from humandemonstrations. Finally, (Wei et al., 2015) presenta trajectory tracking control strategy for a ball jug-gling task on a quadrotor, based on the subspace sta-bilization approach. An optimal trajectory generationmethod is adopted to obtain a dynamically feasible,minimum jerk trajectory.

3 HYBRID DYNAMICMODELING

The hybrid dynamics of the ball consists of the freeflight aerodynamics and the impact reset map. Theformer is modeled through Newton’s equations ofmotion, while the latter is a reset of the state, updatedaccording to the impact detection. In order to ana-lytically model the ball dynamics, the work by (Liuet al., 2012) is considered. That work is supportedby simulations and experiments in several case stud-ies and follows up a deep study on the hybrid dynamicmodeling of the table tennis game, (Nakashima et al.,2010). On the other hand, a first-order dynamics forthe paddle is here introduced, assuming that it is pos-sible to directly control its velocity.

It is assumed that a point contact occurs betweenthe ball and the paddle during the impact. Moreover,as long as the paddle is made of rubber, the reboundin the direction normal to the paddle’s plane does notaffect the motion of the ball in the other directions.

Figure 1: Ball and paddle coordinate systems.

Finally, since the mass of the paddle is usually biggerthan the mass of the ball, only the velocity of the ballis considered to be affected by the impact.

According to Figure 1, let ΣW be the fixed worldframe, ΣP be the frame placed at the center of thepaddle, where the z-axis is the outward normal, andΣB be the frame placed at the center of the ball. LetpB =

[pBx pBy pBz

]T ∈ R3 be the position of theball, vB =

[vBx vBy vBz

]T ∈ R3 be the velocity ofthe ball, ωB =

[ωBx ωBy ωBz

]T ∈ R3 be the spinof the ball, assumed constant during the free flight,pP =

[pPx pPy pPz

]T ∈ R3 be the position of thepaddle, vP =

[vPx vPy vPz

]T ∈ R3 be the velocityof the paddle, ωP =

[ωPx ωPy ωPz

]T ∈ R3 be theangular velocity of the paddle, all expressed in ΣW .Finally, let RP ∈ SO(3) be the rotation matrix of ΣPwith respect to ΣW .

The continuous ball and paddle dynamics aregiven by

pB = vB, (1a)vB =−g− kd ||vB||vB + klS(ωB)vB, (1b)pP = vP, (1c)

RP = RPS(ωP), (1d)

where g =[0 0 g

]T is the gravity acceleration,||·|| is the Euclidean norm, S(·) ∈ R3×3 is the skew-symmetric matrix operator. kd and kl are drag and liftparameters, respectively, and they are typically mod-elled as

kd = kd(vB,ωB) =ρπr2(ad +bd f (vB,ωB))

2m, (2a)

kl = kl(vB,ωB) =ρ4πr3(al +bl f (vB,ωB))

m, (2b)

with

f (vB,ωB) =1√

1+(v2

Bx+v2Bx)ω

2Bz

(vBxωBy−vByωBx)2

. (3)

The meaning of other parameters, like ρ, r, ad , al ,bd , bl , and their numerical values employed in sim-ulations are depicted in Section 6, Table 1, where a

standard table tennis ball and a rubber paddle havebeen considered.

Since the paddle can modify the ball velocity onlyat the impact time, the control action, represented bythe paddle linear and angular velocities, enters the balldynamics through the reset map. Assuming that thesuperscripts − and + represent the state before andafter the impact time, respectively, the rebound equa-tions are given by

v+B = vP +RPAvvRTP(v−B −vP)+RPAvωRT

Pω−B ,(4a)

ω+B = RPAωvRT

P(v−B −vP)+RPAωωRT

Pω−B , (4b)

where the matrices of rebound coefficients are definedas

Avv = diag(1− ev,1− ev,er),Avω =−evrS(e3),(5)

Aωv = eωrS(e3),Aωω = diag(1− eωr2,1− eωr2,1),

where ei ∈R3 is the unit vector along the ith-axis, i ={1,2,3}, while ev and ew are described in Section 6,Table 1.

Notice that (1b) takes into account the spin of theball: the magnitude of the drag and lift forces andtheir coefficients change according to the spin, influ-encing the trajectory of the ball. The values of thecomponents of ωB determine the kind of the spin,namely: backspin, if ωBy > 0; topspin, if ωBy < 0;and sidespin, if ωBz > 0. The effect of the spin of theball in a table tennis game is not negligible and makesa difference between a serious table tennis player anda novice one. Serious players use spin on both theirserves and rallying shots to control the ball and toforce errors from their opponents.

4 BATTING PROBLEMSOLUTION

In order to accomplish the task of hitting the table ten-nis ball and directing it to a desired position on theopposite court with a desired spin, the paddle mustintercept the ball with a certain orientation and ve-locity. These inputs are computed employing the dy-namics of the ball in different steps. Firstly, the im-pact position pi

B =[pi

Bx piBy pi

Bz]T ∈ R3 and ve-

locity v−B are predicted by assigning the impact time ti,and the initial position p0

B =[p0

Bx p0By p0

Bz]T ∈ R3

and velocity v0B =

[v0

Bx v0By v0

Bz]T ∈ R3 of the ball

produced by the opponent’s hit. This step is accom-plished solving forward the model (1a)-(1b). Once

Figure 2: Scheme of the steps to generate the paddle trajec-tory.

the impact position is predicted, assigning the de-sired position of the ball on the opposite court pd

B =[pd

Bx pdBy pd

Bz]T ∈ R3, the desired arrival time td ,

and the desired spin of the ball after the impact,ω+dB ∈ R3, solving backward (1a)-(1b) yields the re-

quired velocity of the ball after the impact v+B ∈ R3.Afterwards, the orientation Ri

P and velocity viP of the

paddle at the impact time may be retrieved, throughthe reset map (4), once the spin and velocity of theball before and after the impact are assigned. Lastly,the paddle desired impact configuration is the inputof the optimal trajectory planner described in the nextsection. A scheme of the steps required to generate asuitable paddle trajectory is showed in Figure 2.

4.1 Aerodynamic Model Solution

In order to predict the impact state of the ball andto compute the post-impact velocity of the ball suchthat it reaches the goal, the equations (1a) and (1b) ofthe aerodynamic model are employed. However, thismodel is nonlinear and coupled, thus an analytic so-lution does not exist. In the literature (Acosta et al.,2003), (Muller et al., 2011) and others have consid-ered a linearized model, or a simplified one for control

purposes to reduce the computations.The following simplified model of (1a)-(1b) is em-

ployed by (Liu et al., 2012) since it has an analyticsolution

vBx =−cd |vBx|vBx, pBx = vBx, (6a)vBy =−cd |vBy|vBy, pBy = vBy, (6b)vBz =−g, pBz = vBz, (6c)

where cd is a constant drag parameter. In this paper,instead, a numerical solver suitable for real-time pro-cess is employed to solve the full aerodynamic model.The proposed algorithm consists in the two minimiza-tion problems shown below.

4.1.1 Aerodynamic Forward Solution

The first problem has the aim to predict the impactposition and velocity of the ball. By assigning the im-pact time, the ball impact state is predicted accordingto the initial position and velocity of the ball producedby the opponent’s hit. In detail, the algorithm solvesthe following minimization problem

minpi

B,v−B

∣∣∣∣∣∣∣∣∣∣[

p0B

v0B

]−[

p0B

v0B

]∣∣∣∣∣∣∣∣∣∣2

, (7)

where piB and v−B are the optimizing variables, p0

B =p0

B(piB,v−B ) and v0

B = v0B(p

iB,v−B ) are the position and

the velocity of the ball at the initial time, respec-tively, numerically obtained by backward integrat-ing (1a) and (1b) starting from the optimization vari-ables pi

B,v−B at time ti. In practice, the minimiza-

tion problem (7) is solved through the Levembert-Marquardt’s algorithm that is well suited for real-time computations according to (Lippiello and Rug-giero, 2012) and (Cigliano et al., 2015). To speedup the convergence of the nonlinear estimation algo-rithm (7), the initial guess for pi

B and v−B is calculatedanalytically from (6). The work by (Liu et al., 2012)assumes instead that the impact position of the pad-dle with the ball is a-priori known. Such assumptionis quite restrictive: hence, here the determination ofsuch impact position is addressed at run-time.

4.1.2 Aerodynamic Backward Solution

In order to compute the velocity v+B of the ball afterthe impact such that it reaches the goal position pd

Bat the time td , the following minimization problem issolved

minv+B‖pd

B(v+B )−pd

B‖2, (8)

where v+B is the optimization variable and pdB(v

+B ) is

the position of the ball at time td numerically obtained

by forward integrating (1a) and (1b) starting from piB

at ti computed in Section 4.1.1. The result of the min-imization problem is the value of the velocity of theball after impact so that it reaches the desired posi-tion on the opposite court at td as close as possible.The initial guess for v+B is calculated analytically from(6). This choice of the initial condition, together withthe use of the complete aerodynamic model (1a)-(1b)in (8) to compute pd

B(v+B ), guarantees that the value

of v+B found by solving (8) is more precise than theone provided by (6), since the latter employs only asimplified model.

4.2 Reset Map Solution

Given the pre-impact velocity of the ball v−B as inSection 4.1.1, and the post-impact one v+B as in Sec-tion 4.1.2, the paddle configuration can be now com-puted. Consider the YX-Euler angles (θ,φ) as a para-metric representation of the orientation of the paddle,with φ ∈ [−π/2,π/2] and θ ∈ [0,π], and define v =[vx vy vz

]T= v+B −v−B and ω=

[ωx ωy ωz

]T=

ω+dB −ω

−B . The velocity and orientation of the paddle

at the impact time are respectively computed through

viP = v−B +Ri

P(I3−Avv)−1(RiT

P v−Avωω−B ), (9a)

RiP = RY (θ)RX (φ), (9b)

where I3 ∈R3×3 is the identity matrix, Ri(·) ∈ SO(3)is the elementary rotation matrix with i= {X ,Y}, rep-resenting the rotation of an angle around the i-axis,and θ,φ are such that

vz cosφsinθ− vx cosφcosθ = ωy, (10a)

asin2φ−2bsinφ+ c = 0, (10b)

where a = e2c ||v||2, b = ece2S(v)ω, c = (e1 +

e3)||ω||2−ece2||v||2, and ec = eωr/ev. Notice that theball motions have to satisfy the proposition given by(Liu et al., 2012) (Proposition 1, Section III-A) toguarantee the existence of a solution.

5 OPTIMAL PADDLETRAJECTORY PLANNING

In this section, the problem of generating an optimaltrajectory for the end-effector of the paddle is tackled.The control design procedure from the previous sec-tion provides a desired pose and velocity of the pad-dle and corresponding desired time instant at which toachieve these. There are many different paths that thepaddle can take to fulfill these requirements. A morejudicious method would be to plan this path such that

a certain objective function is optimized. Differentialgeometry offers a way to extend the notion of differ-entiation from Euclidean space to an arbitrary mani-fold. Primarily, a short background about differentialgeometry is here reported, consequently, the theoryabout the minimum acceleration planner for the pad-dle is described.

5.1 Differential Geometry Background

In this context, trajectories for which it is possible tospecify the initial and final position and velocity of themotion are of interest. Note that, the motion may bespecified in either the joint space, which is a torus, orthe task space, which is the special Euclidean groupof three-dimensions over the reals. At this stage, itis assumed that the path is generated on an arbitraryRiemannian manifold M (do Carmo, 1992). Let γ :(a,b) −→ M be the path, and ⟪·, ·⟫ be the metric onM. Let f : (−ε,ε)× (a,b)−→M be a proper variationof γ; that is, it satisfies

f (0, t) = γ(t), ∀t ∈ (a,b)f (s,a) = γ(a), and f (s,b) = γ(b)

Two vector fields are relevant along the path γ. Thefirst one is called the variation field and is defined by

Sγ(s) :=∂ f (s, t)

∂s=

d ft(s)ds

The second vector field of importance is the velocityvector field of γ, given by

Vγ(s) :=dγ(t)

dt=

∂ f (s, t)∂t

=d fs(t)

dtIn order to perform calculus on the curves of this Rie-mannian manifold, the Levi-Civita connection ∇ is in-troduced. Given a curve γ(t) and a connection, thereexists a covariant derivative, which is denoted by D

dt .The Levi-Civita connection satisfies the following

compatibility and symmetry conditions, presented inthis order:

(11a)ddt⟪U,W⟫ = ⟪DU

dt,W⟫+ ⟪U,

DWdt⟫

(11b)∇XY − ∇Y X = [X ,Y ]

for any vector fields U and W , along the differentiablecurve γ, and any vector field X ,Y ∈ X(M).

The curvature R of a Riemannian manifold M isa correspondence that associates to every pair X ,Y ∈X(M) a mapping R(X ,Y ) : X(M)−→ X(M) given by

R(X ,Y )Z = ∇Y ∇X Z − ∇X ∇Y Z + ∇[X ,Y ]Z,where Z ∈ X(M). Some of the properties of the cur-vature that are going to be used are

D∂t

D∂s

X − D∂s

D∂t

X = R(

∂ f∂s

,∂ f∂t

)X ,

〈R(X ,Y )Z, T 〉 = 〈R(Z,Y )X ,Y 〉.

5.2 Minimum Acceleration Planning

In this work, the acceleration functional in SE(3) isminimized by following theory developed in (Zefranet al., 1998). The acceleration functional to be mini-mized by the choice of the path to be followed by thepaddle is

(12)J =∫ tb

ta〈∇V V,∇V V〉dt,

where [ta, tb] is the time interval over which the trajec-tory is planned, V = (ωP,vP) ∈ se(3) is the velocityof the paddle along a particular path, and ∇ denotesthe Levi-Civita affine connection derived from a par-ticular choice of metric on SE(3). This latter objectallows one to perform differentiation along curves onany smooth manifold. In particular, in (12), it is usedto express the inner product of the acceleration of aparticular path with itself, which may also be identi-fied by the squared norm of the acceleration of thispath at a particular point. If the metric on SE(3) istaken in the form

W =

[αI3 00 βI3

]where α,β > 0 so that for T1,T2 ∈ se(3), then⟪T1,T2⟫ = t>1 Wt2 with t1 and t2 the 6× 1 compo-nents of T1 and T2, then the Levi-Civita connectioncan be computed to be

∇XY =

{ddt

ωy +12

ωx × ωy,dvy

dt+ ωx × vy

}where ωx and ωy are the angular components and vxand vy are the linear components of the rigid bodyvelocities X ∈ se(3) and Y ∈ se(3), respectively.

Equating the first variation of the cost func-tional (12) to zero provides necessary conditions forthe path to be a minimizer of the acceleration func-tional. This necessary condition is a fourth orderboundary problem given by

(13)∇V ∇V ∇V V + R(V,∇V V)V = 0,

where R is the curvature tensor associated with theLevi-Civita affine connection (do Carmo, 1992). Thiscondition can be written down in terms of the angularand linear velocity components of the paddle as

(14a)ω(3)P + ωP × ωP = 0,

(14b)p(4)P = 0,

where (·)(n) denotes the nth derivative of (·). These or-dinary differential equations turn into a well-definedboundary value problem with the addition of the

boundary conditions. For the differential equa-tions (14a) governing the optimal rotational path, theboundary conditions are

(15a)RP(ta) = R0P ωP(ta) = ω

0P,

(15b)RP(tb) = RiP, ωP(tb) = ω

iP,

where R0P and ω0

P are the initial orientation and angu-lar velocity of the paddle, respectively. Similarly forthe differential equations (14b) governing the optimaltranslational path of the paddle, the boundary condi-tions are

(16a)pP(ta) = p0P vP(ta) = v0

P,

(16b)pP(tb) = piP vP(tb) = vi

P,

In practice, the optimal translational motion of thepaddle is found by merely solving a small scale linearsystem of equations obtained by (14b) and (16). Asa result, it may be performed very fast. On the otherhand, in order to determine the rotary motion of thepaddle, a boundary value problem needs to be solved.The boundary value problem is time invariant and nonlinear, but the forcing function (14a) is hardly compli-cated. In Section 6, the computational burden of theproposed optimal paddle trajectory planner is deeplyanalysed.

6 SIMULATIONS

This section shows the results obtained in simulation.Section 6.1 shows an exemplar simulation of the pro-posed algorithm (see Figure 2). The comparative casestudies in Section 6.2 show instead the results of thecomparison between the proposed solution to com-pute the ball post-impact velocity (see Section 4.1.2)and the one used in (Liu et al., 2012). Lastly, in Sec-tion 6.3 the optimal planner simulations are focusedon the paddle motion to highlight the properties ofthe planned trajectories.

The model parameters considered to simulate thephysical system (1) and (4) are listed in Table 1. Aprocedure to identify the aerodynamic and reboundparameters is described by (Nonomura et al., 2010).The simulations described below are implementedin the Matlab environment: the ode45 solver, withthe events option, is used for the dynamic model;the lsqcurve f it function, based on the Levembert-Marquardt’s algorithm, is employed to find the aero-dynamic forward and backward solutions; the bvp4cfunction is used for the minimum acceleration plan-ner.

Table 1: Dynamic parameters.

r Ball radius 2e-2 mrp Paddle radius 1.5e-1 mm Ball mass 2.7e-3 kgρ Air density (25◦C) 1.184 kg/m3

g Gravity constant 9.81 m/s2

ev Velocity rebound coefficient 6.15e-1eω Spin rebound coefficient 2.57e3er Linear rebound coefficient 7.3e-1ad Drag coefficient 5.05e-1bd Drag coefficient 6.5e-2al Lift coefficient 9.4e-2bl Lift coefficient -2.6e-2cd Simplified drag coefficient 5.4e-1

6.1 Batting Task Simulation

In this case study the proposed algorithm, depicted inFigure 2, is simulated. Then, supposing to have at dis-position the estimated trajectory of the ball from thevisual system and the desired final configuration ofthe ball, it is possible to compute the optimal paddletrajectory to achieve the batting task through the twominimization problems (7) and (8), and the solutionof (9) and (14).

Therefore, the visual measurement system is as-sumed to provide the initial position, linear and an-gular velocity produced by the opponent’s hit, re-spectively correspondent to p0

B =[1.2 0.7 0.9

]m,

v0B =

[−3 0.2 1.5

]m/s and ω

−B =

[0,150,0

]. The

impact time is fixed to ti = 0.5 s and the desired finalposition of the ball on the opposite court is assignedas pd

B =[1.9 0.8 0.02

]m. The desired flight time

and post-impact ball spin are assumed to be respec-tively td = 0.6 s and

[ω+By,ω

+Bz]=[−100,0

]rad/s.

The third component of the desired post-impact spinof the ball, ω

+dB , is computed from the equation

vTB ωB = 0. It is remarkable that, since the goal is

achieved when the ball hits the table on the oppositecourt, the final goal time is evaluated when the thirdcomponent of the position vector of the ball is equalto the radius of the ball.

In this case study, the aerodynamic forward solu-tion of (7) is pi

B =[−0.1394 0.7892 0.4820

]T mand v−B =

[−2.4156 0.1570 −2.9788

]T m/s, theaerodynamic backward solution of (8) is v+B =[4.0516 0.0214 2.0984

]m/s, while the solution of

reset map (9) is

Rip =

0.8614 0.0054 0.50800 0.9999 −0.0106

−0.5080 0.0092 0.8613

and vi

P =[1.4388 0.0220 −0.1131

]T m/s. The

Table 2: Ball pre-impact configuration.

v−B [m/s] ω−B [rad/s]

1[−2.5,0,0.1

] [0,150,0

]2

[−4.5,0,0.3

] [0,−150,0

]3

[−2.75,−0.8,−0.5

] [0,−50,150

]optimal trajectory is consequently planned solv-ing (14).

The time histories given by the simulation of thiscase study are depicted in Figure 3. In particular, the3D trajectories of both the ball and the paddle are rep-resented in Figure 3(a). The solid line represents themotion of the ball, while the trajectory for the paddleis depicted by a dashed line. The blue cross representsthe final desired position of the ball pd

B, while the bluecircle is the initial position of the paddle p0

P. Assum-ing that ∆ti is the difference between the planned andthe actual impact time, ∆pi

B the Euclidean norm ofthe difference between the planned impact position ofthe ball and the actual one, ∆td the difference betweenthe desired final time and the actual one, ∆pd

B the Eu-clidean norm of the difference between the desiredposition of the ball pd

B and the actual one. In this casestudy, the time and position impact and final errorsare respectively ∆ti = 4.5e−3 s, ∆pi

B = 1.73e−2 m,∆td = 1.9e− 3 s and ∆pd

B = 9.4e− 3 m. Therefore,the average of the errors is acceptably small for a ta-ble tennis robot.

A video can be found in (Serra et al., 2016) whichshows the simulation of this case study performed inMatlab, while visualization employs the V-Rep envi-ronment. A 21 degree of freedom humanoid robot isused for the simulation. The 7 degree of freedom rightarm is equipped with a parallel jaw gripper, whichfirmly grasps the paddle.

6.2 Comparative Case Studies

The purpose of this subsection is to compare theproposed planning method with the state-of-the-artapproach introduced by (Liu et al., 2012), whichpresents several case studies, with ample implemen-tation details, allowing a fair and critical compari-son. Three case studies are considered, respectively:backspin, topspin and sidespin. In order to obtaina fair comparison with the work (Liu et al., 2012),the first step of the proposed algorithm is not takeninto account, i.e. the ball pre-impact state is as-sumed to be known a-priori and then the minimiza-tion problem (7) is not solved. As a matter of fact,in each of these cases, the impact position of boththe ball and the paddle is assigned to be pi

P = piB =[

−0.15 0.70 0.25]

m, while the linear and angu-lar velocity of the ball before the impact for the back-

(a) 3D trajectories of the ball, solid line, and the paddle,dashed line, obtained with the proposed method. Theblue circle represents the initial position of the paddle,while the blue cross is the desired final position of theball.

(b) From the top to the bottom: magnitude of theplanned linear and angular velocity of the paddle, evalu-ation of the acceleration functional J in (12) between themotion plan devised using the Euler angles and the op-timal proposed one. The red star represents the impacttime ti.

Figure 3: Batting Task Simulation.

spin (1), topspin (2) and sidespin (3) case studies areshown in Table 2. On the other hand, the impact timeis ti = 0.1 s, the desired goal position for the ball ispd

B =[2.055 0.7680 0.02

]m, while the desired fi-

nal time is td = 0.6 s. Moreover, the desired spinof the ball after the impact

[ω+By,ω

+Bz]

is set for thefirst, second and third case studies to

[−100,0

]rad/s,[

100,0]

rad/s and[0,−100

]rad/s respectively, ac-

cording to the setting of the work described in (Liuet al., 2012).

The results obtained for each of the three casestudies are depicted by their time histories in Fig-ures 4, 5 and 6, respectively. In particular, the 3Dtrajectories of both the ball and the paddle are rep-resented in Figures 4(a), 5(a) and 6(a), respectively.The solid line represents the ball, while the paddleis depicted by a dashed line. The time histories ofeach component pBx, pBy and pBz related to the tra-jectory of the ball are represented in Figures 4(b), 5(b)

Table 3: Comparative case study 1 - Numerical results.

∆ti [s] ∆piB [m] ∆td [s] ∆pd

B [m] tc[ms]A 5.2e-3 1.3e-2 1.12e-2 5.47e-2 9e-3B 5.14e-3 1.29e-2 4.88e-3 6.47e-4 21







and 6(b), respectively, for both of the compared meth-ods: the solid line graphs the solution obtained by ex-ploiting the method described by (Liu et al., 2012),while the dashed curve illustrates the trajectory of theball obtained using the proposed method. The figuresindicate that the proposed method yields an improve-ment over (Liu et al., 2012) by directing the ball closerto the goal configuration.

The quantitative results for the backspin, topspinand sidespin case studies are shown in Tables 3, 4and 5, where A represents the results obtained em-ploying the Liu’s planner, whereas B the results ob-tained employing our planner. The first two columnsof each of the aforementioned tables refer to the preci-sion of the impact. In both the approaches the resultsare very similar. The gist of the comparison may becaptured by analyzing the last three columns of thesetables. The final time error is reduced by about anorder of magnitude. The Euclidean norm of the er-ror between the actual final position of the ball andthe desired one is smaller by about an order of mag-nitude, too.

Unfortunately, experiments are not yet availablefor the proposed approach since the practical set-upis under development. However, the switching fromMatlab into C++ language is already accomplished.The computational time tc required to solve at run-time both the simplified aerodynamic model and thecomplete one is show in the last column of Tables 3,4 and 5. The code is running on a computer withspecifications Intel Core 2 Quad CPU Q6600 @ 2.4GHz, Ubuntu 12.04 32-bit operating system, includ-ing the Levenberg-Marquardt C++ library (Lourakis,2004). The same numerical results obtained withMatlab have been retrieved, but the evaluation of thecomputation burden is more precise, in the sense thatit is the one that will appear during the practical exper-iments. To elaborate, compared to (Liu et al., 2012),the prosented method increases the accuracy of the fi-


(b) Comparison between ball trajectories: the solutiongiven by analytically solving (6) is depicted through asolid line, the proposed one is instead represented witha dashed line. The blue cross represents the desired finalposition of the ball.

(c) From the top to the bottom: magnitude of theplanned linear and angular velocity of the paddle, evalu-ation of the acceleration functional J in (12) between themotion plan devised using the Euler angles and the op-timal proposed one. The red star represents the impacttime ti.

Figure 4: Comparative case study 1.

nal desired position an order of magnitude for topspinand sidespin cases and two orders of magnitude forthe backspin case. Furthermore, the C++ implemen-





tation of the proposed nonlinear minimization prob-lem, which considers the full aerodynamic model ofthe ball, takes about 20 ms to give the desired veloc-





ity of the ball after the impact. This duration is graterthan what is shown in (Liu et al., 2012), but it is stillacceptable for real-time implementation.

6.3 Optimal Planner Simulations

For each case study, the paddle trajectory is plannedover the time interval [ta, tb] = [t0, ti− ε], where ε =0.02 s. The paddle trajectory is supposed to start still,from the origin of the world frame, with initial ori-entation R0

P = RY (π/2)RX (0), without loss of gener-ality. According to the proposed algorithm, the posi-tion, orientation and linear velocity of the paddle atthe impact time are given by the ball reset map solu-tion. The Euclidean norm of the linear and angularvelocities of the paddle, planned with the minimumtotal acceleration, are represented by the top and mid-dle plots in Figures 4(c), 5(c), 6(c) and 3(b), for eachsimulation.

Once the desired orientation is achieved with zeroangular velocity, the angular acceleration is set tozero so that the orientation of the paddle remains thesame until the impact occurs. As long as one hasthe control authority at the torque level, this con-trol strategy, which switches only once, is straight-forward to implement. Once the impact has occurredat time ti, notice that the linear velocity of the pad-dle is exponentially dissipated by the following termexp(−µ(t− (ti +δ))), where t is the time, µ = 50 andδ = 0.02, so that the paddle stops.

The optimal trajectory discovered by solving thetwo-point boundary value problem (14) indeed min-imizes the L2 norm of the total acceleration of thepaddle. So as to illustrate this fact, another typicaltrajectory for the orientation of the paddle is planned.This alternative plan constructs a third-order polyno-mial function for the Euler angles, φ and θ, such thatthe initial and final orientation and angular velocityconstraints are satisfied. Both the angular acceler-ation that corresponds to this motion plan and theacceleration functional J in (12) are then computed.For each case study, the bottom time histories of Fig-ures 3(b), 4(c), 5(c) and 6(c), depict the value of thedifference of the acceleration functional between themotion plan devised using the Euler angles and theoptimal motion plan. Notice that the value of this costfunctional is positive at t = ti, indicating that the opti-mal motion plan indeed yields a smaller value of theacceleration functional than a typical plan performedusing polynomials on the Euler angles, φ and θ.

About the computational burden of the proposedplanner, the code has been translated in C++ and eval-uated on the same PC as in Section 6.2. The boundaryvalue problem for the optimal paddle trajectory plan-ner takes less than 30ms. To sum up, after the high-speed vision system gives a stable trajectory estima-tion of the ball coming towards our court, it is possi-ble to compute the desired trajectory for the paddle in50ms (20ms + 30ms), hitting the ball with a proper ve-

locity to redirect it to the opposite court at the desiredposition with the imposed spin. Another possibilityis that, once the desired impact position, velocity andorientation is determined, one can immediately startcontrolling the paddle to achieve these via a PD con-troller, and revert to a trajectory following controlleronce the optimal trajectory is available and is period-ically updated.

7 CONCLUSIONS AND FUTUREWORK

The presented paper proposes an algorithm to plan arobotic batting task. In particular, a table tennis gameperformed by a robot has been considered. The pro-posed solution improves the control accuracy whiledealing with the real-time constraint. A coordinate-free, smooth, optimal motion plan, that minimizes theacceleration functional of the paddle, is proposed. Asfuture work, the proposed technique is planned to bevalidated through experimental studies. Different op-timal planners which make use of the dynamic modelof the robotic manipulator grasping the paddle mayalso be considered.

ACKNOWLEDGEMENTS

The research leading to these results has been sup-ported by the RoDyMan project, which has re-ceived funding from the European Research CouncilFP7 Ideas under Advanced Grant agreement number320992.

REFERENCES

Acosta, L., Rodrigo, J., Mendez, J., Marichal, G. N., andSigut, M. (2003). Ping-pong player prototype. IEEERobotics & Automation Magazine, 10(4):44–52.

Andersson, R. (1989a). Aggressive trajectory generator fora robot ping-pong player. IEEE Control Systems Mag-azine, 9(2):15–21.

Andersson, R. (1989b). Dynamic sensing in a ping-pongplaying robot. IEEE Transactions on Robotics andAutomation, 5(6):728–739.

Cigliano, P., Lippiello, V., Ruggiero, F., and Siciliano, B.(2015). Robotic ball catching with an eye-in-handsingle-camera system. IEEE Transactions on ControlSystems Technology, 23(5):1657–1671.

do Carmo, M. (1992). Riemannian Geometry. Mathematics(Boston, Mass.). Birkhauser.

Huang, Y., Xu, D., Tan, M., and Su, H. (2013). Addingactive learning to LWR for ping-pong playing robot.IEEE Transactions on Control Systems Technology,21(4):1489–1494.

Kuka (2014). The Duel: Timo Boll vs. KUKA Robot. [webpage] https://youtu.be/tIIJME8-au8+.

Lippiello, V. and Ruggiero, F. (2012). 3D monocularrobotic ball catching with an iterative trajectory esti-mation refinement. In IEEE International Conferenceon Robotics and Automation, pages 3950–3955, SaintPaul, MN, USA.

Liu, C., Hayakawa, Y., and Nakashima, A. (2012). Racketcontrol and its experiments for robot playing table ten-nis. In IEEE International Conference on Roboticsand Biomimetics, pages 241–246, Guangzhou, CN.

Lourakis, M. (Jul. 2004). levmar: Levenberg-Marquardtnonlinear least squares algorithms in C/C++. [webpage] http://www.ics.forth.gr/ lourakis/levmar/+. [Ac-cessed on 31 Jan. 2005.].

Lynch, K. and Mason, M. (1999). Dynamic nonprehen-sile manipulation: Controllability, planning, and ex-periments. The International Journal of Robotics Re-search, 18(64):64–92.

Matsushima, M., Hashimoto, T., Takeuchi, M., andMiyazaki, F. (2005). A learning approach torobotic table tennis. IEEE Transactions on Robotics,21(4):767–771.

Muller, M., Lupashin, S., and D’Andrea, R. (2011).Quadrocopter ball juggling. In IEEE/RSJ Interna-tional Conference on Intelligent Robots and Systems,pages 5113–5120, San Francisco, CA, USA.

Mulling, K., Kober, J., Kroemer, O., and Peters, J. (2013).Learning to select and generalize striking movementsin robot table tennis. The International Journal ofRobotics Research, 32(3):263–279.

Mulling, K., Kober, J., and Peters, J. (2010). A biomimeticapproach to robot table tennis. In IEEE/RSJ Interna-tional Conference on Intelligent Robots and Systems,pages 1921–1926, Taipei, TW.

Nakashima, A., Ito, D., and Hayakawa, Y. (2014). An on-line trajectory planning of struck ball with spin by ta-ble tennis robot. In IEEE/ASME International Con-ference on Advanced Intelligent Mechatronics, pages865–870, Besancon, F.

Nakashima, A., Ogawa, Y., Kobayashi, Y., and Hayakawa,Y. (2010). Modeling of rebound phenomenon ofa rigid ball with friction and elastic effects. InIEEE American Control Conference, pages 1410–1415, Baltimore, MD, USA.

Nonomura, J., Nakashima, A., and Hayakawa, Y. (2010).Analysis of effects of rebounds and aerodynamics fortrajectory of table tennis ball. In IEEE Society ofInstrument and Control Engineers Conference, pages1567–1572, Taipei, TW.

Omron (2015). CEATEC 2015: Omron’s Ping Pong Robot.[web page] https://youtu.be/6MRxwPHH0Fc+.

Oubbati, F., Richter, M., and Schoner, G. (2013). Au-tonomous robot hitting task using dynamical sys-tem approach. In IEEE International Conference on

Systems, Man, and Cybernetics, pages 4042–4047,Manchester, UK.

Senoo, T., Namiki, A., and Ishikawa, M. (2006). Ball con-trol in high-speed batting motion using hybrid trajec-tory generator. In IEEE International Conference onRobotics and Automation, pages 1762–1767, Orlando,FL, USA.

Serra, D., Satici, A. C., Ruggiero, F., Lippiello, V., and Si-ciliano, B. (2016). An optimal trajectory planner for arobotic batting task: The table tennis example. [webpage] https://youtu.be/GXtBvbUHu5s+.

Silva, R., Melo, F., and Veloso, M. (2015). Towards ta-ble tennis with a quadrotor autonomous learning robotand onboard vision. In IEEE/RSJ International Con-ference on Intelligent Robots and Systems, pages 649–655, Hamburg, D.

Wei, D., Guo-Ying, G., Ye, D., Xiangyang, Z., and Han,D. (2015). Ball juggling with an under-actuated flyingrobot. In IEEE/RSJ International Conference on In-telligent Robots and Systems, pages 68–73, Hamburg,D.

Yanlong, H., Bernhard, S., and Jan, P. (2015). Learningoptimal striking points for a ping-pong playing robot.In IEEE/RSJ International Conference on IntelligentRobots and Systems, pages 4587–4592, Hamburg, D.

Zefran, M., Kumar, V., and Croke, C. (1998). On the gen-eration of smooth three-dimensional rigid body mo-tions. IEEE Transactions on Robotics and Automa-tion, 14(4):576–589.

Zhang, Y., Xiong, R., Zhao, Y., and Chu, J. (2012). Anadaptive trajectory prediction method for ping-pongrobots. In Intelligent Robotics and Applications, pages448–459. Springer.

Documents

An Optimal Trajectory Planner for a Robotic Batting Task: The …wpage.unina.it/diana.serra/ICINCO_2016_104_CR.pdf · An Optimal Trajectory Planner for a Robotic Batting Task: The