Learning Haptic Exploration Schemes for Adaptive Task Execution

Thomas Eiband1, Matteo Saveriano1, and Dongheui Lee1,2

Abstract— The recent generation of compliant robots enableskinesthetic teaching of novel skills by human demonstration.This enables strategies to transfer tasks to the robot in a moreintuitive way than conventional programming interfaces. Pro-gramming physical interactions can be achieved by manuallyguiding the robot to learn the behavior from the motion andforce data. To let the robot react to changes in the environment,force sensing can be used to identify constraints and actaccordingly. While autonomous exploration strategies in thewhole workspace are time consuming, we propose a way to learnthese schemes from human demonstrations in an object targetedmanner. The presented teaching strategy and the learningframework allow to generate adaptive robot behaviors relyingon the robot’s sense of touch in a systematically changingenvironment. A generated behavior consists of a hierarchicalrepresentation of skills, where haptic exploration skills areused to touch the environment with the end effector, andrelative manipulation skills, which are parameterized accordingto previous exploration events. The effectiveness of the approachhas been proven in a manipulation task, where the adaptivetask structure is able to generalize to unseen object locations.The robot autonomously manipulates objects without relyingon visual feedback.

I. INTRODUCTION

In human daily life scenarios, interaction forces are animportant means of detecting and exploring the surround-ings. Besides visual observations, haptic feedback is oftenused as an additional source of information when exploringor manipulating an uncertain environment [1]–[4]. This isachieved by relying on the sense of touch in a region of thehuman’s spatial imagery where a physical contact is expected[5]. Especially when a human cannot rely on vision alone,haptic sensing is purposeful to gain feedback about objectlocations. In contrast to humans, technical systems mightnot be equipped with a vision system. Moreover, vision isnot always applicable due to occlusions, adverse cameraangle, weak illumination, or small color gradients. In servicerobotics or in human-robot shared workspaces, there is ademand to act in unstructured or uncertain environments.Examples are variable pick/place positions, a manually filledstack with mutable structure or a staple which varies inheight.

Programming a robot to use exploration motions by con-ventional interfaces is elaborate and requires expert knowl-edge. To ease this procedure, learning from demonstration[6] is employed as teaching interface to directly extract thedesired behavior. Hereby, the user demonstrates the task

1 German Aerospace Center (DLR), Institute of Robotics and Mecha-tronics, Wessling, Germany

2 Technical University of Munich, Chair of Human-centered AssistiveRobotics, Munich, Germany{thomas.eiband, matteo.saveriano, dongheui.lee}@dlr.de

Dem

onst

rati

ons

Demonstrated

Trajectories

Distribution of

contact key-points

and its 1st principal

component

Learned exploration

trajectory with

extended exploration

segment

𝑥sp

switching

of contact

observer

𝑥ep

Explore until

contact

Update

transformation

Compute new

motion

Execute

adapted skill

Gripper open (GO)

Gripper close (GC)

Movement, free (MF)

Haptic exploration (HE)

𝑥s

𝑥g

𝑥s

𝑥g

𝑥g′ 𝑥s′

Learning Execution

Demonstration

Execution

Identify skills

Learn skill

relations

Learn DMP

weights

Segment

Fig. 1. The skill demonstration, learning and execution framework toexecute parameterized motions. Motion start and end points are adapted bypreviously observed constraints during exploration of the environment. Allbehaviors are extracted from human demonstrations via kinesthetic teaching.

consisting of exploration and manipulation motions multipletimes by kinesthetic teaching. The locations of the involvedobjects are changed in a region where they are expected to bepositioned during execution. This is an intuitive input methodfor a novice user, where also pick and place actions can beeasily demonstrated by controlling the gripper with a footpedal or a button.

Having this in mind, we propose a strategy to segmentmultiple demonstrations of a task into parametrizable roboticskills, considering intended variations of the environment.We developed a method, which extracts position-based re-lations between skills grounded on the observed varianceduring the demonstrations. Further, we propose a strategyto learn exploration paths from demonstrations to enableadaptive movements in an uncertain environment.

Our work is outlined as follows: Section II summarizesrelated work in the domain of haptic exploration and contact-based localization. Section III describes our approach forsegmentation and skill identification and Sec. IV describesour method for learning skill relations. The experiments arepresented in Sec. V. Finally, conclusions and future work arestated in Sec. VI.

II. RELATED WORK

Learning of force-based robot skills from demonstrationshas been shown in a variety of applications like door-openingand ironing [7], pouring from a bottle [8], screw driving [9],grasping [10], engraving [11], wood planing [12] or writing[13]. Further, in collaborative setups such as assembly [14]

or transportation tasks [15], [16]. However, none of theseapproaches addressed sensing of object locations in theenvironment. A force-based learning framework is proposedin [17], where the robot continuously reacts to force variableswhich have been identified by Mutual Information analy-sis. The framework does not consider motion segmentationrequired to reach absolute positions alongside with severalrobot skills, such as pick and place. Continuous onlinemovement adaption by force perceptions is also presented in[10], [17] and [18] but they are again not suitable in a taskrequiring both fixed and adaptive positions. Learning force-based tasks combined with motion segmentation in differentframes of reference is proposed in [19], which requiresvision-tracked object positions, where our approach extractsimportant points vision-free during demonstration. Anotherlearning scheme pairs vision with tactile sensing [20] andfocuses on material and object detection. The system is notbootstrapped from demonstration and the inferred class isnot used to adapt the robot’s behavior. In the context ofsegmenting and sequencing, [21] proposes the prediction ofmanipulation phases and [22] shows learning of hierarchicalskills by reinforcement learning, where both approachesrequire object tracking if the object is not initially in thehand. Learning of complex tasks using kinematic and videodata is proposed in [23]. In [24], compliant manipulation isachieved by interaction-based phase transitions in a HiddenMarkov Model, where non-linear motions in free space andexact goal points in free space were not addressed. In [25],autonomous exploration of the whole workspace is combinedwith tactile-based object discrimination. In [26], the searchpolicy is learned from human behavior to act in the wholeworkspace, where we instead focus on the transfer of spe-cialized skills to rapidly program and execute a novel task.The approach in [27] combines in-hand object localizationusing a tactile sensor array with tactile based manipulation.During execution, a DMP approach is used similarly to [28],which reproduces both desired motion and tactile trajectory.More elaborate sensing techniques make use of embeddedsensors in gripper fingers [29], or tactile sensor arrays forobject recognition [30]. The latter authors use sampling-based motion planning [31], which requires a model of theenvironment or a huge number of real robot executions.Learning from human hand motion observation is presentedin [32], with the goal to correct wrongly observed handpostures with the sensed contact information while grasping.The authors of [33] present teaching of stiffness profiles,where a haptic interaction lies between robot and humanin order to adapt the compliance to the task requirements.A large review about tactile sensing in [34] shows that avariety of sensors is available but the major challenge lies indevelopment of novel manipulation algorithms.

III. SKILL IDENTIFICATION FROM HUMANDEMONSTRATIONS

We propose a learning from demonstration framework,which extracts skills from a demonstrated task. A skillis a predefined robot behavior [35] parameterized by thedemonstrations. First, we introduce our Task Demonstration

Gripper

control

button

Next

demo

button

Hand

hold

position

FT

sensor

Sensing

plateGripper

Segments

MF IC

Skills

HE

GC GC

GO GO

MF MF

a)

b)

c)

Fig. 2. Kinesthetic teaching by manually guiding the robot (a) andcontrolling gripper actions and demo recording by a foot pedal (b). Labeledsegments and identified skills (c)

System and explain how the user transfers the sequenceof skills to the robot (starting in Fig. 1 at the top left).The Task Segmentation produces labeled segments of thedemonstration data. The next step in our learning frameworkis the Skill Identification, where we construct predefinedskills from the previously labeled segments.

A. Task Demonstration System

The system is able to identify a sequence of predefinedrobot skills, which can be motion in free space (MF), hapticexploration (HE), and motions ending with gripper open(GO) or gripper close (GC). The software framework is im-plemented in Python, communicating via a realtime-capableC++ middleware with Matlab/Simulink models connectedto the robot. In order to demonstrate a task consisting ofa sequence of skills, the robot is guided by kinestheticteaching. Hereby, the user guides the robot using a holdclose to the tip of the robot (see Fig. 2a). Whenever theuser wants the robot to explore a part of the environment,the user lets the gripper fingers or other parts of the grippertouch the desired object while the gripper state can be openor closed. The interaction forces are recorded by a force-torque (FT) sensor, mounted between robot tip and gripper.Gripper actions can be triggered by a foot pedal (see Fig. 2b).During teaching, the robot position xp ∈ R3 and orientationquaternions elements xo ∈ R4, interaction FT measurementf ∈ R6 and gripper state g are recorded. A FT measurementconsists of f = [fx, fy, fz, tx, ty, tz]

T and the gripper stateis defined by g = {1 if gripper open;−1 if gripper closed}.The whole input vector is denoted as x = [xp,xo,f , g]

T .Each sample of user demonstration i is stacked in a matrixXi ∈ RNd×Ni

, where Nd is the number of input dimen-sions and N i the number of samples in demo i. The setT = {X1, ...,XI} consists of I demonstrations, where ourmethod requires a minimum number of I = 2. Ideally, theuser covers significant changes in object locations to extractthe variance in the data. In fact, the length of an explorationpath is derived from the exposed variance during teaching.

B. Task Segmentation

a) Pre-processing: All demonstrations recorded for onetask are preprocessed to allow a robust segmentation intoskills. First, the FT measurements are filtered by a 1storder low-pass Butterworth filter with a cutoff frequency offco = 1Hz. Periods in which the robot movement does notexceed a defined threshold in translation or rotation over timeare removed.

Dynamic Time Warping (DTW) [36] is used to align eachdemo in the set T with a medoid demo. A medoid demo canbe found by pairwise computing DTW distances betweenall demos and selecting the demo with minimum sum ofsquared distances to all other demos. The output is stored inC = [Xp, Xo, F , G]T ∈ RI×Nd×N , with forces and torquesin F = [Ff,Ft]

T and with N samples.The average of Ff and Ft over demonstrations is calculated

as Fm,f =1N

∑Ii=1 Ff and Fm,t =

1N

∑Ii=1 Ft, which reduces

artifacts when segmenting the contact state. Hereby, the num-ber and quality of demonstrations influence the robustness.Possible artifacts are multiple bumps when the user touches arather stiff environment, accidentally touching the tool belowthe FT sensor with a part of the hand, or unintended forceinteractions while picking and placing an object. The goalof the subsequent segmentation is to separate contact statesand free movement as well as to split segments by the binarygripper state.

b) Segmentation of contact states: The in-contact seg-mentation relies on predefined force and torque thresholdsδf and δt compared with the Euclidean norm of force andtorque measurements respectively. Algorithm 1 detects risingand falling edges in the force and torque domain. Whenevera rising edge is detected in one domain, the subsequentsegment is labeled as IC. For every falling edge, the subse-quent segment is labeled as MF. The independent comparisonof force and torque increases segmentation robustness asthese signals can occur independently when in contact withthe environment. Reaching the threshold in one modality issufficient to trigger an IC state while both modalities needto be below their thresholds to reset it, leading to the start ofan MF segment. For instance, force and torque can alternatewhile the tool is continuously in contact, which leads to onlyone IC segment.

c) Segmentation by gripper state: The MF segmentscoming from the previous segmentation are further splitwhen a gripper action occurs in the task. All indexes forgripper open (GO) ni

GO and gripper close (GC) niGC are

extracted for each demo i and then averaged over demos,resulting in the averaged indexes nGO = 1

I

∑Ii=1 n

iGO and

nGC = 1I

∑Ii=1 n

iGC respectively. Segments before these

indexes are labeled as gripper open (GO) or gripper close(GC) accordingly.

A segment s is stored in the matrixSs = [Xp,s,Xo,s,Fs]

T ∈ RI×13×Ns , with Ns samplesfrom I demos. It contains position Xi

p,s, orientation Xio,s,

and force and torque F is for each demonstration i. The

result of the segmentation is the set of all segments, givenby S = {S1, ...,SNs}.

Algorithm 1 IC segmentationInput: Fm

1: Initialization :2: c← 13: IC ← false4: for n = 1 . . . Nn do5: Detect rising edge:6: if (‖Fm,f(n− 1)‖ < δf and ‖Fm,f(n)‖ > δf ) or7: (‖Fm,t(n− 1)‖ < δt and ‖Fm,t(n)‖ > δt) and not8: IC then9: ICs(c)← n . add segmentation index to list

10: IC ← true . set in-contact flag11: c← c+ 112: end if13: Detect falling edge:14: if (‖Fm,f(n)‖ < δf and ‖Fm,t(n)‖ < δt) and15: IC then16: ICe(c)← n . add segmentation index to list17: IC ← false . reset in-contact flag18: end if19: end for20: return ICc, ICe

C. Skill Identification

In our framework, each contact with the tool is consideredas haptic exploration action intended by the user. Further-more, for object manipulation, skills for robot motion, pick-ing, and placing are required. To identify a haptic exploration(HE) skill, an IC segment preceded by a MF segment needsto be found. Figure 2c) shows the mapping between segmentsand skills. If a MF segment occurs before an IC segment,a HE skill r is generated, provided with the stacked dataRr = [SMF,SIC] of both segments. The remaining segments,such as MF, GO and GC are transformed into their accordingskills. This leads to a number of skills Nr, less than or equalto the original number of segments Ns. The identified skillsare stored in a set R = {R1, ...,RNr}.

IV. LEARNING SKILL RELATIONS

A. Relation Learning and Representation

After the Skill Identification, our goal is to infer whethera skill motion shall be executed relative to another skill orrelative to the origin (root). Therefore, a metric is developedwhich pairwise compares all skills to identify how strongthey are related to each other. First, the goal position xi

g,r ∈R3 of a skill r is extracted for each demonstration i. Thisleads to I goal points for skill r, stored in a key-points matrixKr = [x1

g,r, . . . ,xIg,r]

T ∈ RI×3. All key-points are added toa set of available key-points given by L = {K1, ...,KNr}.

Next, a dimension-wise metric for the relation distancebetween skills m and n with their key-points Km and Kn

in dimension d over all demonstrations is given by

vdm,n = Var

Km(1, d)−Kn(1, d)

...Km(I, d)−Kn(I, d)

for m 6= n. (1)

Similarly, the dimension-wise relation distance between asingle skill m with key-point Km and the coordinate origin

Algorithm 2 Constructing the skill relation treeInput: R,Rrel

1: for each child parent relation c, p in Rrel do2: T

µp

Kp← [µp, . . . ,µp]

T −Kp . parent key-pointtranslations

3: Kc ←Kc + Tµp

Kp. transform child keypoints

4: Rp,c ← Rp,c +[T

µp

Kp, . . . ,T

µp

Kp

]. transform skill data

5: tcp ← mean(Kc −Kp) . parent child translation6: connect nodes with trafo(c, p, tcp)7: end for8: return R

O is

vdm,n = Var

Km(1, d)

...Km(I, d)

for m = n. (2)

The relation distance over all dimensions for all m,n iscomputed using the root mean squared error

V (m,n) =

√√√√1

3

3∑d=1

vdm,n2 for ∀ m,n. (3)

For each skill m, in row m of the relation distance matrixV ∈ RNr×Nr , the skill which has minimum relation distanceto skill m is found by

k = argmink∈{1,...,m}

{V (m, k)} with k ≤ m, (4)

considering all skills in the demonstrated sequence until skillm to exclude non-causal relations to future skills. The skill kfound with minimum distance is stored in a relation matrixRrel ∈ RNr×Nr by Rrel(m, k) = 1. In the case m = k,the skill is relative to the coordinate origin (root) by settingRrel(m,m) = 1, as it does not depend on any other skills.All other entries in Rrel are set to 0.

A hierarchical structure of skills in form of a relationtree is constructed from the relation matrix Rrel to representtranslations between skill goal points (Fig. 3), where eachnode is a skill. The translations are directed from a parentskill goal point to a child skill goal point and are updatedduring the execution by the previously unknown contactpoints of the HE skills. Skills which are directly attached tothe root node (acting as the coordinate origin O) are executedwith absolute motions, such as HE0 and GO3. The first skillof a task is always attached to root as it cannot be influencedby any preceding skill. Skills attached in deeper layers areexecuted with relative motions to their parent skills. Forinstance, the goal point of GC2 is defined relative to thereached contact point of HE1. As a consequence, a singleHE skill is able to affect multiple child skills. In addition, achild skills can be executed directly after its parent skill butalso at any later time.

The relation tree is constructed by Algorithm 2, whichuses a mean key-point µp = 1

I

∑Ii=1 K

ip of a parent skill p

to transform the position data Rp,c and key-points Kc of thechild skill c.

root

HE0

HE1

GC2

GO3

MF4

Fig. 3. Relative skill tree from the box experiment. The goal pointtransformation between two skills is a directed relationship from parentto child skill.

B. Skill LearningThe skills to execute free motions only, such as MF,

GO, and GC are directly learned from the data of multipledemonstrations. In contrast, a HE skill needs to handleuncertain environments where the contact point is unknownbefore execution. Therefore, a collision aware motion and amodification of the skill data are necessary

To allow the robot to start exploring outside of the demon-strated region, the motion learning considers an approachingpath (AP) to reach the exploration area and an extendedexploration path (EEP) inside the exploration area (Fig.4).In the following, the generation of the full HE motion isexplained, where the subscript HE is used for referring toany HE skill.

a) Modeling the exploration motion: First, the distri-bution of HE key-points is modeled as normal distributionKHE ∼ N (µHE,ΣHE) with mean µHE = 1

I

∑Ii=1 K

iHE and

covariance ΣHE = Cov(KHE). Note that at least two demon-strations need to be provided to exploit the variance in thedata, however more demonstrations might cover a widerrange of key-points. It is further assumed that explorationtakes place in a defined region of this distribution and along alinear exploration path to find an object. To identify this path,a principal component analysis is applied, using the covari-ance matrix ΣHE. Its principal eigenvector eHE is termed asthe main exploration vector (Fig. 4 middle). However, its signmight be inverse to the desired linear exploration direction.We use the maximum force vector of the skill while incontact F i

f,IC of each demo i (see Fig. 7 for an example) to getthe average force vector Ff,IC = 1

I

∑Ii=1 F

if,IC. The alignment

with the main exploration vector eHE is ensured using thesign of the dot product Ff,IC · eHE. If positive, the mainexploration vector eHE needs to be inverted to be alignedwith the approaching path. The position trajectory Rp,EEPof the EEP is constructed by connecting the explorationstart point xsp = µHE − keeHE and the exploration end pointxep = µHE + keeHE with linearly spaced points, where ke isa parameter to scale the exploration region (Fig. 4 right). Incontrast to the position, the orientation is kept fixed duringthe whole EEP, taken from the quaternions average [37] atthe contact points.

The motion of the HE skill consists of two phases. In thefirst phase, the robot follows the AP until the explorationregion is reached. The data Rp,AP for AP is constructed fromsamples from the beginning of the corresponding segmentuntil the index where a sample falls below a distance to theexploration start point xsp (Fig. 4 right). In the second phase,we want the robot to follow the EEP where xsp is a desiredvia-point before the robot reaches the exploration end-point

Demonstrated

trajectories (blue)

and objects (grey)

Distribution of contact

key-points 𝐾𝑖 (×) and

principal eigenvector 𝑒HE

Learned exploration

trajectory

𝑥sp

𝑥ep

𝐾1

𝐾2

𝐾3

Extended

Exploration

Path (EEP)

Approaching

path (AP)

𝜇HE

𝑒HE

𝑅p,AP

𝑅p,EEP

Fig. 4. Schema of multiple demonstrations in a changing environment(left), the distribution over extracted key-points with principal componenteigenvector (middle) and learned exploration trajectory (right).

xep. The final definition of the input data of a HE skill is thenRp,HE = [Rp,AP,Rp,EEP] with an example given in Fig. 7.

b) Motion Encoding: For any skill r, a Gaussian Mix-ture Model [38] is used to encode position Rp,r, orientationRo,r, and a time vector t ∈ RNt from multiple demonstra-tions in a model MGMM. Expectation Maximization is usedto fit a number of Gaussian distributions on the data fromall stacked demonstrations in

Rinr =

R1p,r . . . RI

p,r

R1o,r . . . RI

o,r

t . . . t

; MGMM = GMM(Rinr ). (5)

Gaussian Mixture Regression is applied to obtain a general-ized trajectory Rout

r ∈ R7×Nt conditioned on the time vectort and denoted as

Routr =

[Rout

p,r

Routo,r

]= GMR(MGMM, t) = P (MGMM|t). (6)

The quaternions in Routo,r are normalized to unit quaternions

after GMR. The computations of the GMM and GMRare carried out using pbdlib [39]. We decided to encodethe motion first in a probabilistic model before learninga dynamical system for execution. Even so, it is possibleto encode multiple demonstrations directly in a DynamicMovement Primitive (DMP). DMPs are employed to learna stable motion which is guaranteed to converge to a desiredgoal point [40]. In this work, the approach from [41] has beenadopted to learn DMPs for position and quaternion basedDMPs by using the quaternion logarithmic and exponentialmap. The DMPs enable adaption of starting point and goalgiven the transformations in the relation tree. The DMPweights for a skill r are learned from its motion data Rout

r

generated by the GMR.

C. Skill Sequencing and Execution

The adaptive execution scheme is implemented in Al-gorithm 3, generating motions with parameterized start xsand goal point xg by the DMPs learned beforehand. Thecoordinate origin is termed as O. During execution of aHE skill, a contact is detected when the exploration forceor torque exceeds a defined threshold. Whenever a contactis registered, the controller stops the current motion. The

Algorithm 3 Adaptive task execution1: for r = 1 . . . Nr do2: p← get parent skill(r)3: if skill r is HE skill then4: xg ← get trafo(O, p) +Rout

p,r(Nn,r)−Routp,r(1)

5: else6: xg ← get trafo(O, r)7: end if8: xs ← get robot pos()9: execute skill(r,xs,xg)

10: if skill r is HE skill then11: xc ← get contact pos()12: tcp ← xc − get trafo(O, p)13: update trafo(r, p, tcp)14: end if15: end for

contact point xc is extracted and the goal point translationbetween this skill and its parent skill is updated in the relationtree (Alg. 3, line 13).

If required, object positions can be explored in multipledimension, e.g. an arrangement of staples of objects or asingle object in a plane. As relative skill’s motions aretransformed by their parent skills, multiple HE skills can bechained to allow more complex behaviors, which is shownin the following Experiments section.

V. EXPERIMENTS

The robot task is to pick boxes from an uncertain locationand place them on a demonstrated position. We use aDLR LWR4 [42], mounted on a linear axis, equipped witha two-finger Robotiq 85 gripper and a JR3 force-torquesensor mounted between robot tip and gripper. The FTmeasurements caused by the gripper mass are compensated.The robot is operated in Cartesian impedance control withconstant stiffness.

As shown in Fig. 8 (right), we use a 2-dimensionalarrangement of boxes with 4 possible box locations. Thenumber of boxes can vary from 1 to 4 and the boxes can bestapled in different configurations. The experimenter givestwo demonstrations to keep the teaching effort low, wherea subset of 2 box configurations has been used. In the firstdemo, boxes are set at locations p11, p21, p22, while in thesecond demo, one box is located at p11. As shown in Fig. 5,during each demonstration one box is touched in horizontaland vertical direction in order to identify its position. Thedemonstrator starts exploring horizontally at the bottom rowbecause it is guaranteed to find a box when one is availableinstead of starting at the top row. Starting in vertical directionto explore a column would not succeed whenever this columncontains no boxes.

After gathering two demonstrations, the task is segmentedinto contact states, free movements, and gripper states. Theresult of the segmentation with according labels can be foundin Fig. 6. From the labeled segments, the skills HE0, HE1,GC2, GO3, MF4 are identified, where the trailing numberdenotes the sequential order. Skill relations are learned basedon the relation distance matrix V (Fig. 8 left) and representedin the tree structure in Fig. 3. The position data Rp,r and

1A 2A 3A 4A

1B 2B 3B 4B

Fig. 5. Demonstration of the adaptive task. In demonstration A, thedemonstrator touches the bottom row of boxes in horizontal (1A) and theupper box of the first staple in vertical direction (2A). The box is picked(3A) and placed (4A) on a fixed position on the table. After rearrangingboxes for demonstration B, the demonstrator touches the residual box inhorizontal (1B) and vertical (2B) direction, picks it (3B) and places it (4B)on the same position on the table.

50 100 150 200 250 300 350 400 450-0.8-0.6-0.4-0.2

00.20.4

pos.

(m

)

MF IC MF IC GC GO MF

x y z

50 100 150 200 250 300 350 400 450

0

10

20

F (

N)

fx

fy

fz

50 100 150 200 250 300 350 400 450n (sample)

-1

0

1

GO

GC

g

Fig. 6. Segmented task with labeled segments.

learned motion of both HE skills are shown in Fig. 7, wherethe motion and according key-points of HE1 are alreadytransformed by Algorithm 2. The system is able to generalizeto unseen box arrangements, showing the motion and contactforces in Fig. 9 for a new box configuration {p11, p12, p21}.See the accompanying video for the task demonstration andgeneralization capability of the execution for all seen andunseen box configurations.

The experiments show that no prior knowledge about theinitial configuration is needed, as long as the same objects1

are consistently used in demonstration and execution. Al-though our approach works only for a bounded region inthe workspace and does not explore object orientations, itis tailored to the task requirements of the demonstrator andable to parameterize subsequent motions.

VI. CONCLUSIONS AND FUTURE WORK

We introduced a framework consisting of a demonstrationsystem, a segmentation method and a skill learning approachto execute adaptive tasks relying on haptic exploration skills.We showed that learning of an adaptive task structure ispossible within only two demonstrations. Segmentation isapplied on averaged demonstration data with the aim toincrease robustness. The haptic exploration skill learningconsiders environmental constraints such that the explorationpath starts at a point outside of the demonstrated contactpoints to avoid unexpected collisions. We validate that theHE behaviors can be extended to more than one dimension

1In theory, each object might be manipulated which is touchable and gras-pable at the same points as the corresponding object used in demonstration.

-0.8 -0.6 -0.4 -0.2y

-0.1

0

0.1

0.2

0.3

0.4

z

GMM

-0.8 -0.6 -0.4 -0.2y

-0.1

0

0.1

0.2

0.3

0.4

z

GMR

-0.7 -0.6 -0.5 -0.4y

-0.1

0

0.1

0.2

0.3

z

GMM

-0.7 -0.6 -0.5 -0.4y

-0.1

0

0.1

0.2

0.3

z

GMR

Fig. 7. Top: HE0, with skill’s position samples Rp,r (grey dots), GMM(blue), key-points (×) and learned GMR motion (green). Black arrows showmagnitude and direction of F i

f,IC. Exploration takes place in approximatelyhorizontal direction. Bottom: HE1, which is executed relative to HE0,exploring in approximately vertical direction.

HE0 HE1 GC2 GO3 MF4

HE0

HE1

GC2

GO3

MF40

0.02

p11 p21

p12 p22

Fig. 8. Left: Relation distance matrix V with extracted relations, where thered boxes mark which skill (in each row) is relative to a skill in the column.Right: Arrangement of boxes at locations p11,p12,p21,p22 and approachingrobot at first skill HE0.

-0.6-0.4-0.2

00.20.4

pos.

(m

)

HE0 HE1 GC3 GO4 MF5

x y z

0 5 10 15 20 25 30 35t (s)

0

5

10

15

F (

N)

fx

fy

fz

Fig. 9. Measured position and forces for an unseen box configuration {p11,p12, p21}. Dashed lines show where each skill terminated. Detected contactsby the HE skills stop the robot movement and adapt the pick position atGC3. Place position at GO4 is reached with a fixed goal.

and to adapt skills online in a systematically changing en-vironment. The proposed system does not require computervision and purely relies on the robots sense of touch. Futureresearch addresses the system’s capability in situations withuncertain object orientations and previously unseen objects.

ACKNOWLEDGMENTS

The first author would like to thank Roman Weitschat forthe advice in the application and experiments. This work hasbeen partially funded by the Helmholtz Association.

REFERENCES

[1] S. J. Lederman and R. L. Klatzky, “Haptic perception: A tutorial,”Attention, Perception, & Psychophysics, vol. 71, no. 7, pp. 1439–1459,2009.

[2] G. Robles-De-La-Torre, “The importance of the sense of touch invirtual and real environments,” IEE Multimedia, vol. 13, no. 3, pp.24–30, 2006.

[3] M. O. Ernst and M. S. Banks, “Humans integrate visual and hapticinformation in a statistically optimal fashion,” Nature, vol. 415, no.6870, p. 429, 2002.

[4] G. Metta and P. Fitzpatrick, “Better vision through manipulation,”Adaptive Behavior, vol. 11, no. 2, pp. 109–128, 2003.

[5] Y. Hatwell, A. Streri, and E. Gentaz, Touching for knowing: cognitivepsychology of haptic manual perception. John Benjamins Publishing,2003, vol. 53.

[6] S. Calinon and D. Lee, “Learning control,” in Humanoid Robotics: aReference, P. Vadakkepat and A. Goswami, Eds. Springer, 2018.

[7] P. Kormushev, S. Calinon, and D. G. Caldwell, “Imitation learning ofpositional and force skills demonstrated via kinesthetic teaching andhaptic input,” Advanced Robotics, vol. 25, no. 5, pp. 581–603, 2011.

[8] L. Rozo, P. Jimenez, and C. Torras, “Force-based robot learningof pouring skills using parametric hidden markov models,” in 9thWorkshop on Robot Motion and Control (RoMoCo). IEEE, 2013,pp. 227–232.

[9] L. Peternel, T. Petric, and J. Babic, “Robotic assembly solutionby human-in-the-loop teaching method based on real-time stiffnessmodulation,” Autonomous Robots, pp. 1–17, 2017.

[10] A. M. Schmidts, D. Lee, and A. Peer, “Imitation learning of humangrasping skills from motion and force data,” in IEEE/RSJ InternationalConference on Intelligent Robots and Systems (IROS), 2011, pp. 1002–1007.

[11] V. Koropouli, D. Lee, and S. Hirche, “Learning interaction controlpolicies by demonstration,” in IEEE/RSJ International Conference onIntelligent Robots and Systems (IROS), 2011, pp. 344–349.

[12] A. Montebelli, F. Steinmetz, and V. Kyrki, “On handing down our toolsto robots: Single-phase kinesthetic teaching for dynamic in-contacttasks,” in IEEE International Conference on Robotics and Automation(ICRA), 2015, pp. 5628–5634.

[13] F. Steinmetz, A. Montebelli, and V. Kyrki, “Simultaneous kinestheticteaching of positional and force requirements for sequential in-contacttasks,” in IEEE-RAS 15th International Conference on HumanoidRobots (Humanoids), 2015, pp. 202–209.

[14] L. Rozo, S. Calinon, D. Caldwell, P. Jimenez, and C. Torras, “Learningcollaborative impedance-based robot behaviors,” Proceedings of theTwenty-Seventh AAAI Conference on Artificial Intelligence, vol. 1,no. 1, p. 1, 2013.

[15] J. R. Medina, M. Lawitzky, A. Mortl, D. Lee, and S. Hirche, “Anexperience-driven robotic assistant acquiring human knowledge toimprove haptic cooperation,” in IEEE/RSJ International Conferenceon Intelligent Robots and Systems (IROS), 2011, pp. 2416–2422.

[16] L. Rozo, S. Calinon, D. G. Caldwell, P. Jimenez, and C. Torras,“Learning physical collaborative robot behaviors from human demon-strations,” IEEE Transactions on Robotics, vol. 32, no. 3, pp. 513–527,2016.

[17] L. Rozo, P. Jimenez, and C. Torras, “A robot learning from demonstra-tion framework to perform force-based manipulation tasks,” Intelligentservice robotics, vol. 6, no. 1, pp. 33–51, 2013.

[18] P. Pastor, L. Righetti, M. Kalakrishnan, and S. Schaal, “Online move-ment adaptation based on previous sensor experiences,” in IEEE/RSJInternational Conference on Intelligent Robots and Systems (IROS),2011, pp. 365–371.

[19] A. L. Pais, K. Umezawa, Y. Nakamura, and A. Billard, “Taskparameterization using continuous constraints extracted from humandemonstrations,” IEEE Transactions on Robotics, vol. 31, no. 6, pp.1458–1471, 2015.

[20] O. Kroemer, C. H. Lampert, and J. Peters, “Learning dynamic tactilesensing with robust vision-based training,” IEEE Transactions onRobotics, vol. 27, no. 3, pp. 545–557, 2011.

[21] O. Kroemer, H. Van Hoof, G. Neumann, and J. Peters, “Learningto predict phases of manipulation tasks as hidden states,” in IEEEInternational Conference on Robotics and Automation (ICRA), 2014,pp. 4009–4014.

[22] O. Kroemer, C. Daniel, G. Neumann, H. Van Hoof, and J. Peters,“Towards learning hierarchical skills for multi-phase manipulationtasks,” in IEEE International Conference on Robotics and Automation(ICRA), 2015, pp. 1503–1510.

[23] S. Niekum, S. Osentoski, G. Konidaris, and A. G. Barto, “Learning andgeneralization of complex tasks from unstructured demonstrations,” inIEEE/RSJ International Conference on Intelligent Robots and Systems(IROS), 2012, pp. 5239–5246.

[24] T. M. Hagos, M. Suomalainen, and V. Kyrki, “Segmenting andsequencing of compliant motions,” in IEEE/RSJ International Con-ference on Intelligent Robots and Systems (IROS), 2018, pp. 1–9.

[25] M. Kaboli, K. Yao, D. Feng, and G. Cheng, “Tactile-based active ob-ject discrimination and target object search in an unknown workspace,”Autonomous Robots, pp. 1–30, 2018.

[26] G. De Chambrier and A. Billard, “Learning search polices fromhumans in a partially observable context,” Robotics and Biomimetics,vol. 1, no. 1, p. 8, 2014.

[27] Y. Chebotar, O. Kroemer, and J. Peters, “Learning robot tactile sensingfor object manipulation,” in IEEE/RSJ International Conference onIntelligent Robots and Systems (IROS), 2014, pp. 3368–3375.

[28] P. Pastor, M. Kalakrishnan, L. Righetti, and S. Schaal, “Towards asso-ciative skill memories,” in 12th IEEE-RAS International Conferenceon Humanoid Robots (Humanoids). IEEE, 2012, pp. 309–315.

[29] P. A. Schmidt, E. Mael, and R. P. Wurtz, “A sensor for dynamic tactileinformation with applications in human–robot interaction and objectexploration,” Robotics and Autonomous Systems, vol. 54, no. 12, pp.1005–1014, 2006.

[30] Z. Pezzementi, E. Plaku, C. Reyda, and G. D. Hager, “Tactile-object recognition from appearance information,” IEEE Transactionson Robotics, vol. 27, no. 3, pp. 473–487, 2011.

[31] H. M. Choset, S. Hutchinson, K. M. Lynch, G. Kantor, W. Burgard,L. E. Kavraki, and S. Thrun, Principles of robot motion: theory,algorithms, and implementation. MIT press, 2005.

[32] P. Falco, R. Jakel, C. Natale, and R. Dillmann, “Improvement of humanhand motion observation by exploiting contact force measurements.”in Humanoids, 2011, pp. 141–146.

[33] K. Kronander and A. Billard, “Learning compliant manipulationthrough kinesthetic and tactile human-robot interaction,” IEEE Trans-actions on Haptics, vol. 7, no. 3, pp. 367–380, 2014.

[34] J. Tegin and J. Wikander, “Tactile sensing in intelligent roboticmanipulation–a review,” Industrial Robot: An International Journal,vol. 32, no. 1, pp. 64–70, 2005.

[35] M. R. Pedersen, L. Nalpantidis, R. S. Andersen, C. Schou, S. Bøgh,V. Kruger, and O. Madsen, “Robot skills for manufacturing: Fromconcept to industrial deployment,” Robotics and Computer-IntegratedManufacturing, vol. 37, pp. 282–291, 2016.

[36] H. Sakoe and S. Chiba, “Dynamic programming algorithm optimiza-tion for spoken word recognition,” IEEE transactions on acoustics,speech, and signal processing, vol. 26, no. 1, pp. 43–49, 1978.

[37] F. L. Markley, Y. Cheng, J. L. Crassidis, and Y. Oshman, “Averagingquaternions,” Journal of Guidance, Control, and Dynamics, vol. 30,no. 4, pp. 1193–1197, 2007.

[38] S. Calinon, F. Guenter, and A. Billard, “On learning, representing,and generalizing a task in a humanoid robot,” IEEE Transactions onSystems, Man, and Cybernetics, Part B (Cybernetics), vol. 37, no. 2,pp. 286–298, 2007.

[39] S. Calinon, “A tutorial on task-parameterized movement learning andretrieval,” Intelligent Service Robotics, vol. 9, no. 1, pp. 1–29, 2016.

[40] A. J. Ijspeert, J. Nakanishi, H. Hoffmann, P. Pastor, and S. Schaal,“Dynamical movement primitives: learning attractor models for motorbehaviors,” Neural Computation, vol. 25, no. 2, pp. 328–373, 2013.

[41] A. Ude, B. Nemec, T. Petric, and J. Morimoto, “Orientation incartesian space dynamic movement primitives,” in IEEE InternationalConference on Robotics and Automation (ICRA), 2014, pp. 2997–3004.

[42] A. Albu-Schaffer, S. Haddadin, C. Ott, A. Stemmer, T. Wimbock, andG. Hirzinger, “The dlr lightweight robot: design and control conceptsfor robots in human environments,” Industrial Robot: an internationaljournal, vol. 34, no. 5, pp. 376–385, 2007.