Generalized Cylinders for Learning, Reproduction,Generalization, and Refinement of Robot Skills

S. Reza Ahmadzadeh, Muhammad Asif Rana, Sonia ChernovaSchool of Interactive Computing, Georgia Institute of Technology, Atlanta, GA, USA

Email: {reza.ahmadzadeh, asif.rana, chernova}@gatech.edu

Abstract—This paper presents a novel geometric ap-proach for learning and reproducing trajectory-basedskills from human demonstrations. Our approach mod-els a skill as a Generalized Cylinder, a geometric repre-sentation composed of an arbitrary space curve calledspine and a smoothly varying cross-section. While thismodel has been utilized to solve other robotics prob-lems, this is the first application of Generalized Cylin-ders to manipulation. The strengths of our approach arethe model’s ability to identify and extract the implicitcharacteristics of the demonstrated skill, support forreproduction of multiple trajectories that maintainthose characteristics, generalization to new situationsthrough nonrigid registration, and interactive humanrefinement of the resulting model through kinestheticteaching. We validate our approach through severalreal-world experiments with a Jaco 6-DOF robotic arm.

I. Introduction

Learning from Demonstration (LfD) provides the abilityto interactively teach robots new skills, eliminating theneed for manual programming of the desired behavior [4].By observing a set of human-provided demonstrations,LfD approaches learn a model and generalize the encodedskill to novel situations autonomously. These capabilitiesmake LfD a powerful approach that has the potentialto enable even non-experts to teach new skills to robotswith little effort. However, despite the existence of severaltrajectory-based LfD approaches, the vast majority of theexisting robotic platforms rely on motion-level actions thatare either hand-coded or teleoperated by experts [30],highlighting the need for further advances in this area.

Existing trajectory learning representations are limitedby a number of challenges. First, the number of parametersand the need to tune them for different types of trajectoriespresent a challenge to many users, especially non-experts.Second, most available approaches require near-optimaldemonstrations in order to perform effectively, while theredundancy and complexity of the current robotic plat-forms (i.e. high degrees of freedom) demand a significantlevel of expertise to perform near-optimal demonstra-tions. Third, because of the assumption on near-optimaltrajectories, few techniques support robust methods ofrefinement of the resulting model, assuming instead thatthe input demonstrations should be treated as the onlyinput to the system. Finally, many existing techniques fail

Fig. 1: Robot reproducing two trajectory-based skills encodedand learned using the proposed approach.

to generalize over start/end states, limiting the technique’sability to generalize to new situations.

In this paper, we present a novel LfD approach that ad-dresses the above challenges through the use of a geometricrepresentation composed of a regular curve and a surfacein 3D Cartesian space. The presented approach requiresminimal parameter tuning and can reproduce a variety ofsuccessful movements inside the boundaries of the encodedmodel by exploiting the whole demonstration space. Totackle the problem of generalization over terminal states,we use a nonrigid registration method to transfer theencoded model accordingly. This generalization approachpreserves the main characteristics of the demonstratedskill while achieving the goal of the task. To overcome theissue of sub-optimal demonstrations, our approach enablesthe user to improve the learned model through physicalmotion refinement. Unlike other existing techniques, re-finements can be applied to both the demonstrations andreproductions of the skill. Consequently, the user can startfrom a set of (sub-optimal) demonstrations and refine thelearned model interactively to reach the desired behavior.In addition to capturing the demonstrated trajectories,the constructed model extracts and represents the maincharacteristics of the demonstrated skill, which are thespatial correlations across different demonstrations. Theseunderlying characteristics are extracted from the raw dataand are not specified by the user explicitly, thereby min-imizing the effort of the user. Additionally, the proposedrepresentation is visually perceivable and can reproducethe learned skill as a trajectory using a simple geometricrule. We validate our approach in seven experiments usinga physical 6-DOF robot, as well as demonstrate its usein comparison to Dynamic Movement Primitives [16] andGaussian Mixture Models [9].

II. Related Work

Existing trajectory-based LfD approaches use a varietyof techniques to encode demonstrations and retrieve a gen-eralized form of the skill [4]. One category of approachesuses regression-based techniques to generate a probabilis-tic representation of the given demonstrations [29],[11].Grimes et. al., employed Gaussian Process (GP) regressionto learn and generalize over a set of demonstrated trajecto-ries [11]. In follow-on work, to overcome the computationalcost of GP, Schneider and Ertel used local Gaussian pro-cess regression [24]. Another approach similar to GP calledLfD by Averaging Trajectories (LAT), used only one-dimensional normal distributions [23]. Both GP and LATcannot extract constraints from the demonstrations withobjects aligned parallel to a Cartesian coordinate axis, andnone of the above approaches can deal with generalizingthe learned skill to new situations (e.g. terminal states).

Work by Calinon et. al., built a probabilistic representa-tion of the skill using a Gaussian Mixture Model (GMM)and retrieved a smooth trajectory using Gaussian MixtureRegression(GMR) [9]. To add generalization, GMM/GMRwas later extended to task-parameterized GMM [8]. Theresulting method is more robust in its execution but re-quires extensive parameter tuning for each trajectory (e.g.number of Gaussian components, scale, weight, kernel).

An alternate approach is to use dynamic systems to en-code/reproduce trajectories [16, 13]. Dynamic MovementPrimitives (DMPs) represent demonstrations as move-ments of a particle subject to a set of damped linear springsystems perturbed by an external force [16]. The shape ofthe movement is approximated using Gaussian basis func-tions and the weights are calculated using locally weightedregression. DMPs are implicitly time-dependent and thismakes the system sensitive to temporal perturbations.

Generalization to new situations is another necessaryfeature for an LfD approach. DMPs can generalize byattaching an attractor to a new terminal point. However,the implicit definition of time as a canonical system makesthe changes in the output (i.e. the speed of the movement)very small as time increases [15]. Our work overcomesthis challenge through the use of nonrigid registration [7].Finally, to maintain the shape of the movement duringgeneralization, DMPs require significant tuning of continu-ous parameters, including those of the dynamical systems,such as time constants and scaling factors [15].

Several other techniques utilize models with characteris-tics similar to generalized cylinders. The real-time motionplanning approach proposed by Majumdar and Tedrakeapproximates a boundary around a trajectory, which isvisualized as a funnel [20]. The generated funnels illustratea similar representation to our approach, however, wedo not require extensive off-line computation. Dong andWilliams proposed probabilistic flow tubes to representtrajectories by extracting covariance data [10]. The learnedflow tube consists of a spine trajectory and 2D covariance

(a) (b) (c)

Fig. 2: Three generalized cylinders with identical directrix anddifferent cross-section types. (a) is a canal surface.

data at each corresponding time-step. Although the ap-proach was applied to extract a human’s intention, theflow tube representation can be seen as a special case ofour approach in which the cross-sections are formed usingcovariance data.

Finally, regardless of the technique used for learn-ing from demonstration, the capability of improving thelearned model by refining its shape or spatial constraints,through an interface or kinesthetic teaching, is highlydesirable. There exist few approaches that enable thehuman to refine the initially given demonstrations. Argallet. al., used tactile feedback for refining a given set ofdemonstrations and reusing the modified demonstrationsto reproduce the skill through incremental learning [5].They apply this approach to teaching a robot to positionits hand for grasping of different objects. Lee and Ottproposed an incremental learning approach for iterativemotion refinement. Their approach combines kinestheticteaching with impedance control and represents the skillusing a Hidden Markov Model (HMM) [18]. Our proposedapproach goes beyond prior work by enabling the userto refine the skill interactively both during and after thelearning process.

III. Background

A Generalized Cylinder (GC) represents an elongatedobject composed of an arbitrary space curve called adirectrix (spine) and a smoothly varying cross-section [26].In other words, by moving a cross-sectional curve, definedon a plane, along the directrix while keeping the planeperpendicular to the directrix at each point, the sweptvolume represents a generalized cylinder. The shape andsize of the cross-sectional curve can vary smoothly frompoint to point. Fig. 2 illustrates three GCs with an identi-cal directrix and different cross-section types. Generalizedcylinders play a fundamental role in Differential Geometry,and in the context of Computer Aided Graphic Design,they are used for the construction of smooth blendingsurfaces, shape reconstruction and transition surfaces be-tween pipes [12]. In robotics, generalized cylinders havebeen used for finding flyable paths for unmanned aerialvehicles [27]. This is the first application we are awareof to manipulation. In this section, we first outline themathematical definition and parameterized formulation ofCanal Surfaces (CS) [14], which are a simpler form of GCs,and then extend the formulae to generalized cylinders.

A. Canal SurfacesLet R3 be Euclidean 3−space with Cartesian coordi-

nates x1,x2,x3. Let Φu be the one-parameter pencil1of regular implicit surfaces2 with real-valued parameteru. Two surfaces corresponding to different values of uintersect in a common curve. As u varies, the generatedsurface is the envelope3 of the given pencil of surfaces [1].The envelope can be defined using the following equations:

Φu : F (x1,x2,x3,u) = 0, (1)∂F (x1,x2,x3,u)

∂u= 0, (2)

where Φu consists of implicit C2−surfaces which are atleast twice continuously differentiable.

A canal surface, Cu, is defined as an envelope of theone-parameter pencil of spheres and can be written as

Cu : f(x;u) := {(c(u), r(u)) ∈ R3,1|u ∈ R}, (3)

where the spheres are centered on a regular curve Γ : x =c(u) ∈ R3 in Cartesian space. The radiuses of the spheresare given by the function r(u)∈R, which is a C1-function.The non-degeneracy condition is satisfied by assuming r >0 and |r| < ‖c‖ [12]. Γ is the directrix (spine curve) andr(u) is called the radii function. For the one-parameterpencil of spheres, (3) can be written as

Cu : f(x;u) := (x−c(u))2− r(u)2 = 0. (4)

B. Generalized CylindersSince canal surfaces are constructed using the one-

parameter pencil of spheres, the cross-sectional curve isalways a circle even though its radius can vary alongthe directrix. Generalized cylinders generalize this ideaby considering an arbitrary cross-sectional curve that canvary in both shape and size while sweeping along thedirectrix Γ.

A generalized cylinder can be defined as follows,

Gu,v : f(x;u,v) := {c(u),ρ(u,v) ∈ R3,2|u,v ∈ R}, (5)

where ρ(u,v) represents the cross-sectional curve thatis defined by two parameters, u the arc length on thedirectrix, and v the arc length on the cross-sectional curve.The dependence on u reflects the fact that the cross-section’s shape may vary along the directrix. To obtaina parametric representation of generalized cylinders, it isuseful to employ a local coordinate system defined withorigin at each point on the directrix. A convenient choiceis the Frenet-Serret (or TNB) frame which is suitable fordescribing the kinematic properties of a particle movingalong a continuous, differentiable curve in R3. TNB is an

1A pencil is a family of geometric objects sharing a commonproperty (e.g. spheres).

2An implicit surface is a surface in Euclidean space that can berepresented in F (x1(u),x2(u),x3(u)) = 0 form.

3An envelope is a curve/surface tangent to a family ofcurves/surfaces (2D or 3D).

orthonormal basis composed of three unit vectors eT , eN ,and eB , where eT is the unit tangent vector, and eN

and eB are the unit normal and unit binormal vectors,respectively4.By defining the cross-section in the TNBframe, we form a parametric representation of generalizedcylinders as follows:

Gu,v : f(u,v) = c(u) +ρx1(u,v)eN (u) +ρx2(u,v)eB(u).(6)

As mentioned earlier, canal surfaces are formed by circu-lar cross-sections even though they can vary in size. Gen-eralized cylinders, on the other hand, can be formed usingcross-sectional curves with various shapes (e.g. a closed-spline [26]). These variations make generalized cylinders apowerful candidate for modeling complicated constraintsof trajectory-based skills captured through demonstra-tions.

IV. Skill Learning using Generalized CylindersIn this section, we explain how generalized cylinders can

be used to encode, reproduce, and generalize trajectory-based skills from demonstrations. We assume that multipleexamples of a skill are demonstrated and captured as a setof trajectories in task-space. To capture demonstrationswe use kinesthetic teaching (Fig. 3), however, alternatedemonstration techniques, such as teleoperation and shad-owing, can be employed.

Given the set of captured demonstrations, our approachfirst calculates the directrix (i.e. an average form of themovements) and then extracts the main characteristicsof the set (i.e. spatial correlations across demonstrations)and forms the cross-section function by identifying itsboundaries. When the generalized cylinder is constructed,a geometric approach is used for generating new trajec-tories starting from arbitrary initial poses. We also usenonrigid registration [7] to generalize the encoded skillover terminal constraints (i.e. initial and final poses).Algorithm 1 shows a pseudo code of our approach.

A. Skill EncodingConsider n different demonstrations of a task performed

and captured in task-space. For each demonstration, the3D Cartesian position of the target (e.g. robot’s end-effector) is recorded over time as ξ

j= {ξj

1, ξj2, ξ

j3} ∈R3×T j ,

where T j is the number of data-points within the jth

demonstrated trajectory (j = 1 . . .n). Since T j can varyamong demonstrations, we use interpolation and resam-pling in order to gain a frame-by-frame correspondencemapping among the recorded demonstrations and alignthem temporally. To achieve this, a set of piecewise poly-nomials is obtained using cubic spline interpolation for

4Calculating Frenet-Serret frames for real data is prone to noise.The reason is that at some points the derivative vector deT

duvanishes

and the formulae cannot be applied anymore (i.e. eN cannot becalculated). This problem can be addressed by calculating the unitnormal vector eN as the cross product of a random vector by theunit tangent vector eT .

Fig. 3: Kinesthetic demonstration of reaching to an object(yellow box). Task-space pose of end-effector is used as input.

Fig. 4: Different types of cross-section on the same set of data.Point on the directrix and the effective points are shown in redand blue respectively.

each demonstration. Then, we generate a set of temporallyaligned trajectories by resampling N new data-points fromeach obtained polynomial. This process provides us withthe set of n resampled demonstrations ξ ∈ R3×N×n, eachof which consists of N data-points. An advantage of thistechnique is that when the velocity and acceleration dataare unavailable, the smoothed first and second derivativesof the obtained piecewise polynomials can be used instead.An alternate solution is to employ Dynamic Time Warp-ing [21].

Estimating the directrix: In order to estimate thedirectrix, Γ, we calculate the directional mean (axis-wisearithmetic mean) value for the given set of demonstrations.Let m∈R3×N be the arithmetic mean of ξ. Note that m isthe space curve that all the cross-sections are centered onto form a generalized cylinder (Line 4 in Algorithm 1). Al-ternatively, the directrix can be produced using GMR [9].In that case, GMR generates the directrix by samplingfrom the learned statistical model using GMM. However,using GMR requires defining a time vector explicitly. Inthis work, we use the directional mean.

Estimating the cross-section function: Given ndemonstrations and the estimated directrix, in this step,we explain methods for calculating the cross-section func-tion ρ(u,v). Fig. 4 illustrates three different types of cross-sections calculated for the same set of data. For eachpoint u on the directrix, we gather one correspondingpoint (aligned with u on the same cross-sectional plane)from each demonstration; we call this set the effectivepoints (the blue points in Fig. 4). We use the effectivepoints to calculate the cross-section at each step with asmooth closed curve. The circumference of a cross-sectionrepresents the implicit local constraints of the task (i.e.boundaries) imposed by the set of demonstrations. In itssimplest form, we can employ (4) and construct a canalsurface which has a circular cross-section. In this case, wecalculate the distances from the point on the directrix tothe effective points and use the maximum distance as theradius of the circle. As shown in Fig. 4(left), the estimated

Algorithm 1 Skill Learning using Generalized Cylinders1: procedure Encoding demonstrations2: Input:set of n demonstrations ξ ∈ R3×N×n

3: Output:Generalized cylinder Gu,v, TNB frames F(u)4: m(u)←mean(ξ)5: P(u,v)← estimateCSpline(ξ)6: Gu,v,F ←makeGeneralizedCylinder(m(u),P(u,v))

7: procedure Reproducing Trajectory8: Input:initial point p0 ∈ R3, Gu,v, F9: Output:New trajectory ρ ∈ R3×N

10: η← ‖p0−c0‖‖g0−c0‖

11: pi← p0 , ρ← p012: for each frameFi do13: pi+1← project(pi,η,Fi+1,Fi)14: ρ← pi+115: i← i+ 1

Algorithm 2 Generating GC with arbitrary cross-section1: procedure makeGeneralizedCylinder2: Input:directrix m(u), boundary function P(u,v)3: Output:Generalized cylinder Gu,v, TNB frames Fu

4: for each ui do5: {eT (ui),eN (ui),eB(ui)}← estimateFrame(m(ui))6: F ← {eT (ui),eN (ui),eB(ui)}7: Gu,v←m(ui) +Px(ui,v)eN (ui) +Py(ui,v)eB(ui)

cross-section bounds other points as well and therefore theformed canal surface encloses all the demonstrations. Theradii function r(u) ∈ R produces a radius for each pointu, and v parameterizes the circumference of the circularcross-section (e.g. v = [0 2π]). More detail on encodingskills using canal surfaces can be found in [2]. To cover thecross-sectional area more effectively and precisely whilemaintaining the implicit local constraints of the task, wecan also construct generalized cylinders with ellipticalcross-sections (see Fig. 4(middle)). The radii function forelliptical cross-section r(u) : R 7→ R3 produces the majorand minor axes and the rotation angle of the ellipse foreach step u.

In a more general form, we generate cross-sections byinterpolating closed splines to the data. Given a set ofbreak points vj , (j = 1, . . . ,n) on the interval [v0,vn]such that v0 < v1 < .. . < vn−1 < vn, we can fit a cubicpolynomial to each interval.

p(v) = a0 +a1(v−v0) +a2(v−v0)2 +a3(v−v0)3. (7)

Each polynomial p(v) is described with four coefficientsa0,a1,a2,a3. The accumulated square root of chord lengthis used to find the breaks and the number of polynomials.Since each polynomial is C2−continuous, by applyingthe boundary condition p

′′(t0) = p′′(tn) = 0 and joining

the polynomials we construct a smooth piecewise poly-nomial curve called a closed cubic spline. We denote the

closed-spline as P(u,v). Therefore, the obtained spline isC2−continuous within each interval and at each inter-polating nodes. Fig. 4(right) shows a closed-spline cross-section constructed on the same set of effective points.

B. Skill ReproductionDuring the reproduction phase, the initial position of

the end-effector p0 in the cross-sectional plane S0 (per-pendicular to the directrix at c0) is used as input. Bydrawing a ray starting from c0 and passing through p0, wefind g0, the intersection of the ray and the cross-sectionalcurve (See Fig. 5). The distance between the given pointp0 to g0 reflects the similarity of the movement we want toreproduce to the nearest neighbor on the GC. We measurethe ratio η (Line 10) by calculating the distance ‖p0c0‖ andthe distance ‖g0c0‖. We use η to generate the next pose ofthe end-effector by transforming p0 from the current TNBframe F0 to the next (Line 13). The pose on the secondframe is then adjusted by applying the measured ratio as:

p1 = η‖g1c1‖TF1,F0 . (8)

Since the ratio η is kept fixed throughout the process,thus we call this reproduction method the ratio rule. Theratio rule can generate new trajectories from any pointinside the generalized cylinder and serves to ensure thatthe essential characteristics of the demonstrated skill arepreserved. An illustration of a single-step reproductionprocess using the ratio rule can be seen in Fig. 5.

Si Si+1

x(u)

pi

pi+1

ci

ci+1

gi

eN

i

eB

i

eT

i

eB

i+1

eN

i+1

eT

i+1

gi+1

Tii+1

Fig. 5: Reproduction from a random initial pose pi on the ith

cross-section Si.

Finally, as mentioned earlier, the presented approachrequires minimal parameter tuning in that only the shapeof the cross-section needs to be defined. However, we havefound the closed-spline cross-section to be most effectivein encoding a wide range of trajectories, thus serving as auseful default for this single parameter.

0.150.1

x2

0.050

x1

0 00.05 0.1

0.1

(c) closed-spline

x 3

0.2

0.3

0.150.1

x2

0.050

x1

0 00.05 0.1

0.1

(b) elliptical

x 3

0.2

0.3

0.150.1

x2

0.050

x1

0 00.05 0.1

0.1

(a) circular

x 3

0.2

0.3

Fig. 6: A reaching skill encoded using three generalized cylin-ders with different cross-section types.

C. GeneralizationThe approach described thus far enables a robot to

reproduce the skill under similar conditions (i.e. startand goal states within the GC cross-section). However,to have a robust model we must ensure it generalizesto novel start and terminal states. We use a nonrigidregistration technique to achieve this goal. Given a setof points in source geometry and a corresponding set ofpoints in target geometry, nonrigid registration computesa spatial deformation function. Nonrigid registration tech-niques have been widely used in medical imaging [19],computer vision [6] and 3D modeling communities [22].Recently, Schulman et al. [25] demonstrated the useful-ness of nonrigid registration in LfD by employing it forautonomous knot tying. Their proposed trajectory transfermethod is based on the classic Thin Plate Splines (TPS) [7]registration algorithm extended to 3D Cartesian space,which we also utilize here.

Consider a source geometry composed of a set of Nlandmark points in 3D Cartesian space, L= {Ln ∈R3|n=1,2, ...,N}, and a target geometry composed of the cor-responding set of landmark points, L′ = {L′

n ∈ R3|n =1,2, ...,N}. The nonrigid registration problem then is tofind an interpolation function z : R3 7→ R3, constrainedto map the points in L to the points in L′ . However,there are infinitely many such interpolation functions. Toaddress this issue, TPS finds an interpolation function thatachieves the optimal trade-off between minimizing the dis-tance between the landmarks and minimizing the so-calledbending energy, in effect finding a smooth interpolator. TheTPS formulation is as follows,

minz

N∑

n=1‖L

′n−z(Ln)‖2 +λ

∫R3dx

∑i∈{1,2,3}

‖52 zi(x)‖2F

(9)

where, 52zi represents the Hessian matrix of the ith

dimension of the image of z, λ is a regularization param-eter, and ‖.‖F is the Frobenius norm. The integral termrepresents the bending energy.

The required function z which solves (9) is known toconstitute of two parts: an affine part and a non-affinepart. The affine part approximates the overall deformationof the geometry acting globally, while the non-affine partrepresents the local residual adjustments forced by individ-ual landmark points. With the non-affine part expandedin terms of the basis function, z can be represented as,

z(x) = b+Ax +N∑

n=1wnφ(Ln,x) (10)

where, b∈R3, A∈R3×3 and wn ∈R3 are the unknown pa-rameters while φ is the basis function defined as φ(Ln,x) =‖Ln−x‖ for all x ∈ R3. The unknown parameters in (10)can be found using matrix manipulation [7].

The generalization procedure is detailed in Algorithm 3.The source geometry is composed of the locations of

the important landmarks in the workspace during thedemonstration. The corresponding target geometry is com-posed of the new locations of the landmark points in theenvironment. We first find a mapping z of the form in (10)using the nonrigid registration method (line 4). We thenuse the resulting function to map the directrix m to m′ ,and the cross-sectional function P to P ′ (lines 5 and 6).The new generalized cylinder G′

u,v is then found usingthe mapped parameters (line 7). The ratio rule can beemployed to perform reproduction in G′

u,v.

Algorithm 3 Generalization of GC1: procedure GeneralizeGC2: Input:Gu,v, frames Fu, source & target landmarks L, L

′

3: Output:target G′u,v, target frames F

′u

4: z← findTPS(L,L′) UsingEq. 95: m′(u)← z

(m(u)

)6: P ′(u,v)← z

(P(u,v)

)7: G′

u,v,F′ ←makeGeneralizedCylinder(m′(u),P ′(u,v))

Fig. 7: Generalization of generalized cylinders over initial andfinal poses using nonrigid registration.

V. Experimental ResultsWe conducted eight experiments to demonstrate the

GC model, as well as its reproduction and generalizationcapabilities, on multiple trajectory-based skills. For eachexperiment, we gathered a set of demonstrations throughkinesthetic teaching using a 6-DOF Kinova Jaco2 roboticarm (Fig. 3). The data was recorded at 100Hz.

A. Learning and ReproductionIn this section, we present examples of five trajectory-

based skills encoded using the generalized cylinder model.In the first experiment, we performed a reaching skilltowards an object (green sphere) from above (Fig. 6)5. Wepresent circular, elliptical and closed-spline cross-sectionsto showcase how GCs with different cross-section typesencode the boundaries of the movement.

The demonstrations recorded for the second experiment(Fig. 8(a)) show an example of a movement that can be

5In all of the figures, the demonstrations, directrix, and reproduc-tions are plotted in red, blue, and magenta, respectively.

started and ended in a wide task-space but in the middle isconstrained to pass through a narrow area. This movementresembles threading a needle or picking up an object inthe middle of the movement. The obtained GC extractsand preserves the important characteristics of the demon-strated skill, i.e. the precision and shape of the trajectorythroughout the movement. Given an arbitrary initial poseof the end-effector, the robot successfully reproduces thelearned skill using the ratio rule.

The third experiment (Fig. 8(b)) shows a reach-ing/placing skill similar to the first experiment with acurved trajectory. The robot learns to exploit a widerspace while reaching the object while maintaining trajec-tories with precision near the object. The accompanyingvideo shows the execution of this task [28].

The fourth experiment (Fig. 8(c)) shows a circularmovement around an obstacle, which is unknown to therobot. Since the given demonstrations avoid the obstacle,and the encoded GC guarantees that all the reproductionsof the task remain inside the cylinder, the reproduced pathis guaranteed to be collision-free. Fig. 1 (right) shows asnapshot during the reproduction of the skill.

The fifth task represents a pick-and-place movement, inwhich the robot picks up an object and places it in a box(Fig. 8(d)). The encoded GC shows that the initial andfinal poses of the movement are the main constraints ofthe tasks while in the middle of the trajectory, the end-effector can pass through a wider space while preservingthe shape of the movement. Fig. 1 (left) shows a snapshotduring the reproduction of this skill.

B. GeneralizationIn this section, we demonstrate the generalization ca-

pability of the proposed approach using data from thethird and fifth experiments (Fig. 7). After encoding theskill in the third experiment, we relocated the targetthree times and each time used the nonrigid registrationapproach to adapt the encoded model to the new situationas explained in Section IV-C. In the fifth experiment, werelocated the initial pose of the object four times andused nonrigid registration to adapt the encoded model tothe new situations. As noticeable in both generalizationexperiments, the overall shape of the generalized cylinderis preserved while accordingly expanding or contracting fordifferent initial and final poses. As a direct consequence ofthe ratio rule for reproduction, this successfully enablesshape preservation of the reproduced trajectories. Non-rigid registration can also be applied in the case that theinitial and target points both differ.

C. Comparison to DMP and GMM/GMRIn this section, we compare the presented approach to

two widely-used LfD techniques, DMPs and GMM/GMR.Fig. 9 shows a comparison of both results on the datafrom experiment four. For the GC, we encoded the demon-strations using a generalized cylinder with a closed-spline

0.20.4

x 3-0.5

0.6

(a) picking

x2

0 -0.6-0.4

x1

-0.20.5 0 0.4

(b) reaching

0

0.2

x1

0.2x 3

0.4

x2

0 0-0.2 -0.4

-0.2-0.3 x 2

-0.4

(c) obstacle avoidance

x1

0.4

x 3 0.6

-0.4 -0.2 0 0.2

0.8

0

-0.4

0.2x 3

-0.2

0.4

x2

0

(d) pick & place

-0.60.2

x1

-0.5-0.40.4

Fig. 8: Four real-world experiments performed using our approach. The demonstrations (red) are used to extract the maincharacteristics of the skills. The obtained directrix (blue) and generalized cylinders (gray) represent the skills.

cross-section and generated five reproductions from var-ious initial poses (Fig. 9(b)). As mentioned before, ourapproach requires no parameter tuning beyond specifyingthe cross-section type, and by extracting the characteris-tics of the movement it learns to avoid the obstacle. ForGMM/GMR we tuned the number of Gaussian compo-nents (5 and 10, in figures (c) and (d), respectively), andfor DMPs, we tuned the number of attractors (5 and 10,in figures (e) and (f), respectively). As can be seen in theresults, both DMPs and GMM/GMR can learn the skill.In (c), and to a lesser degree in (d), GMM/GMR producesa more angular trajectory than seen in the demonstrations.In (e) we see the DMP deviating from the demonstrationsand colliding with the object. In all four GMM/GMR andDMP examples, we also observe that the five differentstart locations all converge to a single trajectory path.In contrast, the trajectories reproduced by the generalizedcylinder produce a more natural set of motions that are notidentical and exploit the whole cylinder while maintainingthe important characteristics of the skill. It has to be notedthat the GC can reproduce analogous trajectories by usingthe directrix directly if that behavior is desirable.

Fig. 9: Results of the comparison between GMM, DMP, andGC. (a) five demonstrations, (b) GC and five reproductions, (c)GMM with five components, (d) GMM with ten components,(e) DMP with five attractors, and (f) DMP with ten attractors.

VI. Skill RefinementThe above examples show that generalized cylinders can

be used to extract, generalize and reproduce skills fromreliable human demonstrations. In practice, however, user-provided demonstrations of a task are usually sub-optimal.Multiple solutions to this problem have been proposed.

Argall et al. showed that behavior can be corrected byhaving the teacher assign weights to each demonstrationbased on its quality [3, 5]. It has also been shown that inte-grating LfD with Reinforcement Learning can be used forrefinement [17]. However, assigning weights to demonstra-tions is not trivial and exploring the state-action to findan improved solution requires extensive trial and error. Analternate approach is to refine the skill through physicalhuman-robot interaction [5]. In this work, we differentiatetwo types of refinement. Incremental refinement occursduring the learning process, in which the robot replays ademonstrated trajectory (or a reproduction), the user canmodify it in some way, and then the model is retrainedon the updated data. Once a model is learned, constraint-based refinement can be used to refine the model furtherby applying new constraints. In this section, we show thatboth approaches can be applied to generalized cylinders.Note that, we have selected simple movements for analysisand illustrative purpose.

A. Incremental RefinementIn its first form, skill refinement can be performed

during the learning process incrementally. After gatheringthe set of demonstrations and encoding the skill usinggeneralized cylinders, the user identifies a target trajectory(either a demonstration or a reproduction) that needsto be modified. The user executes the target trajectorywith the robot in compliant mode, allowing the joints andthe end-effector position to be adjusted while the robotis moving. While the robot is replaying the trajectory,the teacher can reshape the target trajectory throughkinesthetic correction. The obtained trajectory is added tothe set and replaces the initial demonstration. Given thenew set, the algorithm updates the model and reproducesnew trajectories that inherit the applied corrections.

The following experiment illustrates this approach onGCs. Initially, we demonstrated three simple trajecto-ries, encoded the skill as a GC with spline cross-section(Fig. 10(a)), and reproduced the skill using the ratio rule(Fig. 10(b)). Now, we assume we would like the arm of therobot to dip downwards in the middle of the first (top)demonstration. While the robot is replaying the targetdemonstration, the teacher reshapes the demonstrationthrough kinesthetic correction in the middle of the move-ment. Fig. 10(c) illustrates the original and refined demon-strations. Fig. 10(d) shows the updated GC and directrix

Fig. 10: Incremental refinement of a skill by correcting a demonstration. (a) demonstrations (red), directrix (blue) and theobtained GC, (b) reproduction from a random pose (magenta), (c) first demonstration was refined by the teacher (red), (d)updated GC, directrix, and a new reproduction.

Fig. 11: Constraint-based refinement of a skill by correcting a reproduction. (a) demonstrations, (b) reproduction (c) reproductionrefined by the teacher (green), (d) two new reproductions; the upper one is affected by the refinement, while the lower is not.

after replacing the target with the refined demonstration.Given an arbitrary initial pose, the algorithm reproducesa new trajectory that reflects the performed refinements.This experiment shows that our approach can deal withthe refinements incrementally. Although many approachescan benefit from a similar process [5], our representationis visually perceivable and enables even non-experts toobserve and interpret the effect of the refinements on themodel. The accompanying video shows the execution ofthis task [28].

Original reproduction

Refined reproductionvivi-1

Fig. 12: Formation of the refinement matrix from the originaland modified reproductions.

B. Constraint-based RefinementIn this section, we show that skill refinement can also

be performed after the model has been encoded by apply-ing new constraints. During the compliant reproductionphase, the user observes and kinesthetically modifies thereproduced trajectory. When this occurs, we compare theoriginal and the modified reproductions, calculate point-to-point translation vectors, vi, and form a refinementmatrix, V by concatenating the vectors. Fig. 12 depictsthe formation of the refinement matrix from the originaland the modified trajectories. The refinement matrix actsas a geometric constraint on the GC that would affectfuture reproductions.

Fig. 11 presents an example of this technique. Thegreen trajectory in Fig. 11(c) shows how the originalreproduction is refined by the teacher through kinestheticcorrection; the teacher has applied downward forces (−x3

direction) to keep the end-effector at a certain level. Wecalculate the refinement matrix V = [v1, . . . ,vn] ∈ R3×n

and apply it as a constraint to our ratio rule. In otherwords, a reproduction remains unaffected if it is generatedbelow the constraining plane. This case can be seen as thelower reproduction in Fig. 11(d). On the other hand, ifa reproduction intersects with the constraining plane, therefinement matrix applies to it. The upper reproductionin Fig. 11(d) shows the effect of the constraint while thedashed line shows reproduction without applying the con-straint. This experiment indicates that using constraint-based refinement, the user can apply new constraints tothe model without modifying it. One of the advantagesof this approach is that the imposed constraint later canbe removed or combined with other constraints withoutupdating the encoded model. To our knowledge, there isno other LfD approach with similar capabilities.

VII. ConclusionWe have presented a novel LfD approach for learn-

ing and reproducing trajectory-based skills. Our geomet-ric representation maintains the important characteristicsand implicit boundaries of the skill and generalizes itover the initial condition of the movement. By exploitingthe whole demonstration space, it reproduces a variety ofsuccessful movements. In addition, the proposed approachrequires minimal parameter tuning that not only simplifiesthe usage of the algorithm and makes the result consistentbut also can make the approach more convenient fornon-expert users. We also have shown that our approachenables users to refine a learned skill both through incre-mental and constraint-based refinement interactively.

AcknowledgmentsThis work is supported in part by the Office of Naval

Research Award N000141410795.

References

[1] Elsa Abbena, Simon Salamon, and Alfred Gray. Mod-ern differential geometry of curves and surfaces withMathematica. CRC press, 2006.

[2] S. Reza Ahmadzadeh, Roshni Kaushik, and SoniaChernova. Trajectory learning from demonstrationwith canal surfaces: A parameter-free approach. InHumanoid Robots (Humanoids), IEEE-RAS Interna-tional Conference on, pages 544–549. IEEE, 2016.

[3] Brenna D Argall, Brett Browning, and ManuelaVeloso. Automatic weight learning for multipledata sources when learning from demonstration. InRobotics and Automation (ICRA), IEEE Interna-tional Conference on, pages 226–231. IEEE, 2009.

[4] Brenna D. Argall, Sonia Chernova, Manuela Veloso,and Brett Browning. A survey of robot learning fromdemonstration. Robotics and autonomous systems, 57(5):469–483, 2009.

[5] Brenna D Argall, Eric L Sauser, and Aude G Billard.Tactile guidance for policy refinement and reuse. InDevelopment and Learning (ICDL), IEEE 9th Inter-national Conference on, pages 7–12. IEEE, 2010.

[6] Serge Belongie, Jitendra Malik, and Jan Puzicha.Shape matching and object recognition using shapecontexts. IEEE transactions on pattern analysis andmachine intelligence, 24(4):509–522, 2002.

[7] Fred L. Bookstein. Principal warps: Thin-platesplines and the decomposition of deformations. IEEETransactions on pattern analysis and machine intelli-gence, 11(6):567–585, 1989.

[8] Sylvain Calinon. A tutorial on task-parameterizedmovement learning and retrieval. Intelligent ServiceRobotics, 9(1):1–29, 2016.

[9] Sylvain Calinon, Florent Guenter, and Aude Billard.On learning, representing, and generalizing a task ina humanoid robot. IEEE Transactions on Systems,Man, and Cybernetics, Part B (Cybernetics), 37(2):286–298, 2007.

[10] Shuonan Dong and Brian Williams. Learning andrecognition of hybrid manipulation motions in vari-able environments using probabilistic flow tubes. In-ternational Journal of Social Robotics, 4(4):357–368,2012.

[11] David B. Grimes, Rawichote Chalodhorn, and Ra-jesh PN Rao. Dynamic imitation in a humanoidrobot through nonparametric probabilistic inference.In Robotics: Science and Systems (RSS), pages 199–206, 2006.

[12] Erich Hartmann. Geometry and algorithms for com-puter aided design. Darmstadt University of Technol-ogy, 2003.

[13] Micha Hersch, Florent Guenter, Sylvain Calinon, andAude G Billard. Learning dynamical system modu-lation for constrained reaching tasks. In HumanoidRobots (Humanoids), 6th IEEE-RAS International

Conference on, pages 444–449. IEEE, 2006.[14] David Hilbert and Stephan Cohn-Vossen. Geometry

and the Imagination, volume 87. American Mathe-matical Society, 1952.

[15] Auke J. Ijspeert, Jun Nakanishi, and Stefan Schaal.Movement imitation with nonlinear dynamical sys-tems in humanoid robots. In Robotics and Automation(ICRA), IEEE International Conference on, pages1398–1403. IEEE, 2002.

[16] Auke J. Ijspeert, Jun Nakanishi, Heiko Hoffmann,Peter Pastor, and Stefan Schaal. Dynamical move-ment primitives: learning attractor models for motorbehaviors. Neural computation, 25(2):328–373, 2013.

[17] Petar Kormushev, Sylvain Calinon, and Darwin G.Caldwell. Robot motor skill coordination with EM-based reinforcement learning. In Intelligent Robotsand Systems (IROS), IEEE/RSJ International Con-ference on, pages 3232–3237. IEEE, 2010.

[18] Dongheui Lee and Christian Ott. Incremental kines-thetic teaching of motion primitives using the motionrefinement tube. Autonomous Robots, 31(2-3):115–131, 2011.

[19] JB Antoine Maintz and Max A Viergever. A survey ofmedical image registration. Medical Image Analysis,2(1):1–36, 1998.

[20] Anirudha Majumdar and Russ Tedrake. Funnel li-braries for real-time robust feedback motion planning.arXiv:1601.04037, 2016.

[21] Cory Myers, Lawrence Rabiner, and Aaron Rosen-berg. Performance tradeoffs in dynamic time warpingalgorithms for isolated word recognition. IEEE Trans-actions on Acoustics, Speech, and Signal Processing,28(6):623–635, 1980.

[22] Mark Pauly, Niloy J Mitra, Joachim Giesen,Markus H Gross, and Leonidas J Guibas. Example-based 3d scan completion. In Symposium on Geome-try Processing, pages 23–32, 2005.

[23] Benjamin Reiner, Wolfgang Ertel, Heiko Posenauer,and Markus Schneider. Lat: A simple learning fromdemonstration method. In Intelligent Robots and Sys-tems (IROS), IEEE/RSJ International Conferenceon, pages 4436–4441. IEEE, 2014.

[24] Markus Schneider and Wolfgang Ertel. Robot learn-ing by demonstration with local gaussian processregression. In Intelligent Robots and Systems (IROS),IEEE/RSJ International Conference on, pages 255–260. IEEE, 2010.

[25] John Schulman, Jonathan Ho, Cameron Lee, andPieter Abbeel. Learning from demonstrationsthrough the use of non-rigid registration. In RoboticsResearch, pages 339–354. Springer, 2016.

[26] Uri Shani and Dana H Ballard. Splines as embeddingsfor generalized cylinders. Computer Vision, Graphics,and Image Processing, 27(2):129–156, 1984.

[27] Madhavan Shanmugavel, Antonios Tsourdos, Rafa lZbikowski, and Brian A White. 3D path planning for

multiple UAVs using pythagorean hodograph curves.In AIAA Guidance, Navigation, and Control Confer-ence, pages 20–23, 2007.

[28] Video. video accompanying this paper, availableonline. https://youtu.be/0ANvgLkqdaM, 2017.

[29] Sethu Vijayakumar, Aaron D’souza, and StefanSchaal. Incremental online learning in high dimen-sions. Neural computation, 17(12):2602–2634, 2005.

[30] Holly Yanco, Adam Norton, Willard Ober, DavidShane, Anna Skinner, and Jack Vice. Analysisof human-robot interaction at the DARPA roboticschallenge trials. Journal of Field Robotics, 32(3):420–444, 2015.