D7.3 Second Project Demonstrator and its Validationclopemaweb.felk.cvut.cz/wp-content/uploads/sites/2/... · Clopema - 288553 D7.3:Second Project Demonstrator and its Validation The

Clothes Perception and Manipulation

D7.3Second Project Demonstrator and its

Validation

Version: 0.1Last Update: 31 - 1 -2014

Distribution Level: PU (Public)

The research leading to these results has received funding from the European Community’sSeventh Framework Programme (FP7/2007-2013) under grant agreement no 288553

Clopema - 288553 D7.3:Second Project Demonstrator and its Validation

Project Number : FP7-288553Project Title : Clothes Perception and Manipulation

Deliverable Number : D7.3Title of Deliverable : Second Project Demonstrator and its ValidationNature of Deliverable : ReportDissemination Level : PublicVersion : 0.1Contractual Delivery Date : 31 - 1 - 2014Actual Delivery Date : 31 - 1 -2014Contributing WP : WP7Author(s) : Jiri Trdlicka (NEO), Gerardo Aragon-Camarasa,

Paul Cockshott, Susanne Oehler, J. Paul Siebert(UG), G. Cannata, S. Denei, M. Jilich, A. Lan-dini, L. Le, P. Maiolino, R. Molfino, D. Zlatanov,M. Zoppi (UNIGE), Vaclav Hlavac, Libor Wagner,Vladimir Smutny, Pavek Krsek, Vladimir Petrik,Martin Meloun (CVUT), Andreas Doumanoglou,Dimitra Triantafyllou, Georgia Peleka, Andreas Kar-gakos, Christos Kampouris, Ioannis Mariolis, SotirisMalassiotis (CERTH)

Reviewer(s) : Sotiris Malassiotis (CERTH), Libor Spacek (CVUT)

AbstractThis deliverable describes the second demonstrator of the CloPeMa project. Its pur-pose is to present the state of the project and to make a preliminary evaluation of thesystem performance and function. The demonstrator is focused on a presentation ofthe individual modules which have been integrated into one system and work side-by-side. The modules share the same hardware, data, computational results and servicesto perform their tasks. Moreover, several modules which are supposed to be integratedin the third year are presented separately.

KeywordsDemonstrator, Validation

1


Revision History

Version Description Author(s) Date0.1 Document seed Jiri Trdlicka 8 Jan 20140.2 Ready for the proofreading Jiri Trdlicka 10 Feb 20141.0 Final Document S. Malassiotis, L Spacek 12 Feb 2014

2


Contents1 Introduction 5

2 Two Cooperating Manipulators 62.1 Description . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 62.2 State of the realization . . . . . . . . . . . . . . . . . . . . . . . . . . . 62.3 Integration, Performance . . . . . . . . . . . . . . . . . . . . . . . . . . 7

2.3.1 Robot Kinematics Accuracy . . . . . . . . . . . . . . . . . . . . 72.3.2 Xtion sensor on the robot arm . . . . . . . . . . . . . . . . . . . 72.3.3 Stereo head . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 92.3.4 Xtion sensor mounted on the torso platform . . . . . . . . . . . . 92.3.5 Tables . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9

2.4 Demonstration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9

3 Clopema hand and tactile sensing 103.1 Description . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 103.2 State of the realization . . . . . . . . . . . . . . . . . . . . . . . . . . . 113.3 Integration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 113.4 Demonstration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 113.5 Performance . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11

4 Binocular Robot Head, Computer Vision and Depth Sensing 134.1 Description . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 134.2 Dynamic calibration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 134.3 Point cloud acceleration . . . . . . . . . . . . . . . . . . . . . . . . . . . 154.4 Feature Extraction for Deformable Clothes Manipulation . . . . . . . . . 16

4.4.1 Clothing Surface Topology-Based Representation . . . . . . . . . 164.5 Contour Localisation Based on Matching Dense HexHoG Descriptors . . 184.6 Robot head software . . . . . . . . . . . . . . . . . . . . . . . . . . . . 184.7 GPU Stereo matcher . . . . . . . . . . . . . . . . . . . . . . . . . . . . 194.8 Demonstration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 194.9 Performance . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20

4.9.1 Dynamic calibration . . . . . . . . . . . . . . . . . . . . . . . . 204.9.2 Stereo GPU matcher . . . . . . . . . . . . . . . . . . . . . . . . 214.9.3 Simulated Flattening Experiments . . . . . . . . . . . . . . . . . 23

5 Photometric Stereo System 255.1 Description . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 255.2 State of the realization . . . . . . . . . . . . . . . . . . . . . . . . . . . 255.3 Integration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 255.4 Demonstration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 255.5 Performance . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26

3


6 World Representation, Perception and Action Planning 276.1 Description . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 276.2 State of realization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 276.3 Integration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 276.4 Demostration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 276.5 Performance . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 28

7 Recognition, learning and active exploration 297.1 Description . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 297.2 State of the realization . . . . . . . . . . . . . . . . . . . . . . . . . . . 297.3 Integration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 297.4 Demonstration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 297.5 Performance . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 30

8 System Integration 328.1 Integration of the Stereo-head to the CVUT CloPeMa system . . . . . . . 328.2 Integration of the CERTH ROS package for garment unfolding to the

CVUT CloPeMa system . . . . . . . . . . . . . . . . . . . . . . . . . . 338.3 Integration of the Photometric stereo sensor . . . . . . . . . . . . . . . . 338.4 Integration of the CloPeMa hand and tactile sensing . . . . . . . . . . . . 33

9 Conclusion 34

4


1 IntroductionThe deliverable D7.3 describes the the second demonstrator of the CloPeMa project. Itdemonstrates the individual components integrated into one system, with shared hardwareand software resources, data and tasks. The basic structure of the deliverable follows theDescription of Work [1] of the project.

The purpose of the deliverable is to present the state of the project and to make apreliminary evaluation of the system performance and function in accordance with thepre-agreed scenario of operation which has been described in the deliverable D1.1 [2].

The demonstrator is focused on a presentation of the individual modules which havebeen integrated into one system and work side-by-side. The modules share the same hard-ware, data, computational results and services to perform their tasks. Moreover, severalmodules which are supposed to be integrated in the third year are presented separately.

This document is divided into several sections according to the DoW [1]. Each sectionconsists of a short description of the subsystem, state of its realization, integration aspect,demonstration of the function and performance evaluation. Some of the subsystems aredescribed in more detail in special deliverables. A description of the integrated platformand demonstrators follows the description of the individual components.

The demonstrator is presented as a live-demo on the CVUT robot system in Prague.

5


2 Two Cooperating ManipulatorsThis chapter describes state of the HW and elementary SW functionality of the test bed.

2.1 DescriptionThe test bed consist of a manipulator with 13 DOF. The manipulator is composed of twostandard robot arms with 6 DOF sitting on the torso which allows to rotate around verticalaxis. This torso arrangement allows to significantly extend the operating space so robotcould operate on three tables as well as above the floor.

The robot arms are equipped with the grippers, where the first year gripper has two de-grees of freedom, one controlled by an industrial actuator allows opening and closing twofingers and the second DOF allows to move fingers under the external force sideways, inthe direction of finger opening/closing. The second year gripper is significantly advancedversion where the second degree of freedom is controlled by a hydraulic actuator. Thesecond year gripper is described in more detail in [4].

The robot is equipped with the stereo head mounted on the pole which is attached tothe platform moving with the torso. The stereo head cameras are each on pan-tilt unitwhich allow to control the direction of the camera view.

Each robot arm is equipped with the ASUS Xtion color camera and depth sensor.Another Xtion sensor is mounted on the platform.

The robot gripper is further equipped with the photometric stereo sensor. This sensoris used for recognition of the type of the fabric.

To allow force and torque feedback, CVUT robot arms are equipped with the forceand torque sensors mounted between robot arm flange and gripper.

The Xtion sensors on the CVUT test bed were equipped with the electronically con-trolled shutter which allows to use several structured light range finders time multiplexedwith the frame rate about 5 fps. The details could be found in [10].

The computers processing the data are placed on various places, some of them in theroom, some of them on the torso platform, some of them on the robot shoulder .

2.2 State of the realizationThe three test beds located in CERTH, UG, and CVUT are slightly different.

The fourth installation in UniGe differs significantly from the rest, as it has only onearm without torso and misses most of the sensors, particularly stereo head and Xtionsensor. It has also second year gripper and second year tactile sensor.

The CERTH, UG, and CVUT have two arms with torso, stereo heads, first year grip-pers, Xtion sensors.

The CERTH and CVUT have on one first year gripper mounted a photometric stereocamera.

The CVUT installation has two force/torque sensors. The second year gripper andtactile sensor is going to be installed on the CVUT test bed soon. The CVUT Xtionsensors are equipped with the shutter allowing time multiplexed depth images capturing.

6


The CVUT test bed was during calibration experiments equipped with further camerasand calibration targets, RedCam calibration mechanism, and Leica Laser Tracker system.

2.3 Integration, PerformanceAll the HW components listed above are fully integrated both mechanically and in ROSenvironment. The various subsets of individual components are used in applicationsdemonstrating different functionality.

Important part of the integration is the geometrical calibration of all components in-volved in geometry:

1. Robot

2. Xtion sensor on the robot arm

3. Stereo head

4. Xtion sensor on the torso

5. Tables in front of the robot

The following geometry evaluation data were measured on the CVUT instance of thetest bed. We suppose that similar numbers hold for the UG and CERTH test beds.

2.3.1 Robot Kinematics Accuracy

The robot accuracy was evaluated by the linear probe measuring the distance of the grip-pers in the set of predefined position where according kinematics model, the distancesshould be constant (see Fig. 2. The offset of the linear probe was subtracted as it wasunknown parameter. The standard deviation of the distance between left and right gripperfor uncalibrated robot, that is using only design (CAD) parameters measured on the set ofreachable positions in rectangular operating space in front of the robot was 0.7 mm. Theaccuracy performance improved after Leica Laser Tracker calibration described in [9] toabout 0.5 mm. The calibration is visible on the Fig. 1. Alternative calibration was per-formed by a RedCam tool [8, 13, 15]. The method and results of calibration is describedin [14].

We further tested a robot repeatability via repeatable touching the metal table andmeasuring again the distance to the table using linear probe. The standard deviation wasfound to be 0.01 mm which is in good agreement with the manufacturer claimed repeata-bility ±0.08 mm for individual arm having in mind that the robot is almost new. Moredetails can be found in [12].

2.3.2 Xtion sensor on the robot arm

The relative pose of the Xtion sensor located on the next to last link of the robot armwas estimated using Hand-Eye calibration procedure. The Xtion sensor observed squarecalibration target 800 by 800 mm large with automatically recognized dot pattern from

7


Figure 1: Calibrating robot with the Leica Laser Tracker

Figure 2: Verification of the robot calibration via measuring relative position of the leftand right gripper.

8


several different positions. The residuals of the individual dot positions in camera hadstandard deviation about 0.25 mm and including the robot position the standard deviationsare about 0.5 mm.

2.3.3 Stereo head

The stereo head camera position on the robot was measured by the calibrating the in-dividual cameras using robot as a source of reference calibration target positions. Theaccuracy of individual camera calibration is characterized by the mean of the residualswhich is roughly 0.8 mm.

2.3.4 Xtion sensor mounted on the torso platform

The same procedure used for calibration of the stereo head was used for calibrating torsoXtion sensor.

2.3.5 Tables

As the tables used for folding move frequently back and forth, we have developed theprocedure which allows to measure the position of the table from the Xtion sensor data.The estimated error could be at most 10 mm.

2.4 DemonstrationThe two cooperating manipulators are a core technology used in all other demonstrations.

9


3 Clopema hand and tactile sensing

3.1 DescriptionThe second year dual arm demonstrator uses two different hands: left hand as in the firstyear demonstrator with passive rubbing motion and unloaded configuration determined bya beam spring; right hand with advanced design, variable stiffness actuation of the rubbingmotion, advanced multisensorial fingertip, advanced and completed control electronics.

The experiments are aimed to demonstrate the improved performance of grasping andsensing in single hand operation and the dual arm manipulation procedures involving theadaptability and compliance of the hand.

The characteristics highlighted by the experiments are:

• Hand ability to accommodate picking of a garment from a hard surface with coarseinformation on the relative position hand-surface. This feature is available in thehand design with passive rubbing motion; in the hand design with VS rubbing ac-tuation, the compliance of the lower finger of the hand can be adjusted from verylow (at grasping) to high (when the garment is lift, for accurate positioning of thefingertip).

• Hand ability to recognize the presence of material between the fingertips. A dedi-cated sensor is present in the multisensorial fingertip: it detects an obstruction dueto presence of material between the fingertip at the tip of the fingers.

• Hand ability to adjust the holding force at the fingertip by adjusting the pressurebetween the fingertips. The tactile pad in the multisensorial fingertip is used totransduce information on the pressure; the stepper motor in the driver unit of thehand is controlled to reach a desired pressure.

• Hand ability to rub a garment. Four cases are considered: rubbing over an innerportion (absence of discontinuities); rubbing over an edge (presence of a pressurediscontinuity due to the variation of thickness; typical shape in the pressure image);rubbing over a buttonhole (presence of a pressure discontinuity due to variation ofthickness; typical shape in the pressure image); rubbing over a button (presence ofa pressure discontinuity due to variation of thickness; typical shape in the pressureimage).

• Hand-arm ability to spread a fold in a garment laying on the surface of a table.The side plate of the lower finger of the right hand is used to flatten the fold; theactuation of the rubbing motion is set to low stiffness for the plate to follow theprofile of the garment.

• Dual arm ability to stretch a garment hold by both arms, with direction of stretch-ing orthogonal to the surface of the fingers. The fingers in the hand with passiverubbing motion bend of an angle proportional to the tension in the garment; thefingers in the hand with VS actuation of the rubbing bend of an angle depending ongarment tension and on the finger stiffness set. The tension in the garment can be

10


maintained in a safe range without knowing accurately the length of fabric betweenthe fingertips of the two hands.

3.2 State of the realizationBoth hands have been realized. The advanced hand with VS rubbing actuation has beentested stand alone. The integration of the VS hand on the dual arm is ongoing.

3.3 IntegrationThe hands comprise all electronics for their operation embedded in the mechanics. Thewrist power and signal interface required is limited to a DC electrical power supply andto a CAN bus. A detailed description is provided in D2.2.

The hand is operated through a CAN card present in the computer of the dual arm,converting ROS commands into CAN. All required software has been implemented.

3.4 DemonstrationAn experiment will be implemented comprising multiple actions to show all characteris-tics listed above. This combined test might comprise the picking of a garment with theVS hand; its repicking and hanging with the left hand; the localization of relevant regionswhere performing rubbing over a region, over an edge, over a buttonhole, over a button;the repicking with both hands to show the stretching performance.

3.5 PerformanceGeometry related

• Range of relative displacement at fingertips: 32 mm

• Span of the rubbing (for compensation of contacts between lower finger and table):34 deg

• Grasping opening range: 50 mm

Force related

• Range of stiffness variation: 1 - 18 N/mm

• Max grasping force at fingertips: 30 - 35 N

• Max rubbing force at fingertips (tangential to the tactile pad): 75 N

Sensing related

• Spatial Resolution of contact pressure: 10 dpi

11


• Capacitance to digital conversion range: 1fF to 20pF.

• Capacitance to digital conversion resolution: 16bit.

• Maximum Sampling rate: 200KHz

• Microphone frequency range: 100Hz to 15KHz.

• Microphone sensitivity: -42dB.

• Microphone sample resolution: 10bit.

• Microphone sampling frequency: up to 1Msps

• Proximity sensor range: 1mm to 200mm.

• Proximity sensor resolution: 16bit.

• Ambient light sensor range: 0.25 lx to 16 klx.

• Ambient light sensor resolution: 16bit.

• Proximity and Ambient Light sampling frequency: 10Hz to 50Hz.

12


4 Binocular Robot Head, Computer Vision and DepthSensing

4.1 DescriptionUniversity of Glasgow (UG) developed in the first year three binocular robot heads com-prising a stereo-pair of actuated cameras (suitable for photogrammetric measurements), asreported in deliverable [3]. The aim of the CloPeMa binocular robot head is to integratehigh resolution imaging for 3D mapping and range images and low resolution imagingfor real-time gaze control and feature extraction. These operational modes will enablea high-level reasoning engine working in conjunction with the CloPeMa dual-robot armand tactile and RGBD sensors to exploit sensed visual information for the tasks of manip-ulation and interaction with garments.

For this demonstrator, we have improved range capture by means of accelerated codesin order to reduce range map acquisition and processing times close to real-time per-formance on 16 Mega-Pixels images. Camera calibration has been also improved andautomated using the dual-arm robot in order to reduce the interaction of the user whileperforming camera and hand-eye calibration. Likewise, UG has released the integrationof camera calibration under dynamic camera motion using standard OpenCV camera cal-ibration routines in order to maintain the calibration of the camera system under extrinsiccamera movement.

An investigation into accelerated feature extraction for clothing manipulation appli-cations is currently in progress. At the time of this report, the freely-available GPU SIFTimplementation[16] has been integrated within ROS and its functionality has been en-abled. Specifically, the feature extraction front-end has been redesigned to accommodatedifferent feature extraction techniques in order to reduce the integration time. SIFT fea-tures extracted from each camera of the stereo-pair are employed to maintain cameravergence using inter-camera correspondences. Under the remit of this deliverable, UGreports advances on the dynamic calibration of the CloPeMa stereo head (Section 4.2),a strategy for accelerated point cloud computation (Section 4.3) and feature extractiontechniques used to describe the topology of garments (Section 4.4 and 4.5). Likewise, wehave investigated accelerated codes for range map construction under Task 4.2 Acceler-ated Range Sensing (Section 4.7).

4.2 Dynamic calibrationThe aim of the dynamic calibration routines is to provide ready-to-use calibration rou-tines based on OpenCV and ROS implementations that allow the binocular stereo-visionsystem to recover metrically accurate range images under dynamic gaze control. Cameracalibration is provided to recover intrinsic and extrinsic camera calibration parameters foreach camera by sampling the camera’s parameter space using a check-board target (Figure3).

The complete camera-hand eye calibration routine is depicted in Figure 5. The strat-egy adopted consists of capturing images of the chessboard target with both cameras ina static position. The target is presented to the cameras at different positions and orien-

13


Figure 3: OpenCV’s chessboard target used for calibrating the CloPeMa binocular robothead.

Figure 4: Coordinate frames in the HOME position of the CloPeMa robot head.

14


Figure 5: Flow diagram of the camera-hand eye calibration ROS node.

tations in order to sample the camera parameter space adequately. For this process, wehave used the dual-arm robot to automate this process. After capturing these images, theROS node then calibrates each camera in isolation and outputs the calibration parameters.Thereafter, the node starts capturing images again over different camera positions whilethe chessboard target is held static . The forward kinematic chain is then computed fromthe base of the PTU to the tilt frame. Finally, Tsai’s hand-eye calibration is carried outfor each camera in order to obtain the geometric transformation that relates the tilt andcamera reference frame. Hence, by means of the above process the kinematic chain of thestereo head is described completely.

4.3 Point cloud accelerationPreviously, a point cloud was reconstructed using a Singular Value Decomposition (SVD)solution. However, the computation time incurred using this approach was of the order of10 minutes for 16 MPixel images. To accelerate this, we obtained simplified and directsolutions for the reconstruction problem in a least-squares fashion which is equivalent tothe SVD solution. The above has been implemented in ROS and the average execution istime is 500ms for all disparity points in a 16 Mega-Pixel range image.

15


4.4 Feature Extraction for Deformable Clothes ManipulationCurrently, the group at Glasgow are investigating and formulating methods for extractingrange-based and intensity-based features primarily suitable for clothing and limp materialmanipulation. It is anticipated that the core set of features extraction methods and imageinterpretation techniques developed in this task will also be of wider general applicability.We describe below our pipeline for representing and interpreting range images depict-ing clothing scenes, however, we believe that this could be extended to provide a 2.5D“primal sketch” of any arbitrary range scene. Similarly, we are investigating hierarchicalgrouping methods for representing and matching the robot gripper and edge contours inclothing that could also be extended for general purpose robot vision. Having finalisedthe development of these feature extraction systems, the second phase of this work will beto accelerate the computation of these features based on the same strategy that has beensuccessfully adopted for accelerated range map recovery, namely GPU/multi-processorimplementation of the codes in combination with foveated image representations.

4.4.1 Clothing Surface Topology-Based Representation

We have developed an accurate topology-based wrinkle descriptor for deformable cloth-ing manipulation. Detecting wrinkles is of critical significance in clothing manipulation.Wrinkles can be used as potential grasping locations and therefore understanding theirgeometry is essential to the task of designing folding and flattening strategies. In ourapproach, range map images of clothing are interpreted by means of high level wrinkledescriptors which are constructed by analysing the underlying cloth surface structure us-ing low level curvature based features, e.g. ridges, contours, shape index, etc. In thatrespect, we defined a wrinkle as a set of triplets, each triplet comprising one ridge pointand two corresponding contour points. Triplets are used to explore potential graspinglocations, while their grouping into wrinkles are used to design flattening strategies.

The feature extraction process includes 4 steps: B-Spline surface fitting, clothes topol-ogy and shape analysis, triplet matching, and wrinkle construction. As geometry-basedfeatures such as curvature and shape index are susceptible to high frequency noise, wefirstly employ a B-Spline surface fitting approach to suppress this noise. We then detectclothing surface topographic features (including ridges, contours, and surface shape types)by computing the shape index of the clothing surface. Thereafter, triplets are formed bymatching ridge points with contour points, and grasping candidates are then ranked andselected. Finally, wrinkles are constructed and represented. We have a Matlab imple-mentation that has been integrated into ROS and can be applied to range maps generatedby robot head to demonstrate grasp point selection using the robot. This topology-basedrepresentation can also on range images generated by our simulator to demonstrate au-tonomous clothing flattening.

16


Figure 6: Clothes topology. Here, ridge points are shown in green, and contour pointsred.

Figure 7: The shape index feature on clothes surface.

17


Figure 8: The triplets and top-ranked grasping candidates.

4.5 Contour Localisation Based on Matching Dense HexHoG De-scriptors

We have investigated the potential of our robot vision system to detect and localise an ob-ject of interest from captured images containing a cluttered background. We have devel-oped a new feature descriptor, the HexHoG, based on a hexagonal, hierarchical groupingmechanism for dense local edge contour matching, edge labelling and pose estimationrefinement, as a post detection and classification process. Our initial experiments usingthe ALOL image database indicate that our HexHoG-based pose estimation refinementprocess enables the contours of a detected object to be labelled with sufficient accuracyto support tasks such as grasping[11], Figure9.

This work is currently being progressed within the CloPeMa context to support taskssuch as button detection and localisation, edge grasping points in garments, etc., andalso for representing the Clopema robot grippers/arms in order to facilitate their poseestimation and segmentation.

4.6 Robot head softwareFor the second year demonstrator, the CloPeMa robot head stack consists of the followingROS packages:

• Feature Extraction: SIFT features in the Graphics Processor Unit (GPU) [16] forreal-time operation for gaze control, object recognition and binocular vergence.This package also includes range surface topology-based features, and their groupedrepresentation into wrinkle structures.

• Cameras drivers: these provide the required image capture capabilities and alsodynamically update the extrinsic parameters of each camera.

• Pan and tilt drivers: These units are interfaced and controlled by a C++ ROS node.This package also includes the robot head URDF model description.

• Binocular vergence: C++ ROS node in order to verge the cameras towards a point.The vergence algorithm is described in [6].

18


(a) Level1 (b) Level2 (c) Level3

(d) Level1 (e) Level2 (f) Level3

(g) Level1 (h) Level2 (i) Level3

(j) Level1 (k) Level2 (l) Level3

(m) Level1 (n) Level2 (o) Level3Figure 10: Object edge contour labelling results: from the first column to the third column, edge labelling results by usingL1 HexHoG, L2 HexHoG, L2 HexHoG, are shown respectively.

Figure 9: Edge labelling examples using 3 different level of HexHoG descriptor hierar-chical grouping.

• Graphic User Interface (GUI): C++ ROS node that allows a user to control the gazeof the robot head and provides a graphical front end for demonstration purposes.

• Stereo Processing: C++ and Python nodes that includes the GPU stereo matcher(full resolution and foveated) and the point cloud generation.

• Robot Interface: This provides the SMACH functions to control the robot arm whileperforming camera and hand-eye calibration.

4.7 GPU Stereo matcherThe aim of the Stereo Matching algorithm is to compute a disparity map from a pair ofstereo images, from which a rnage map is computed by means of photogrammetry. Thematching algorithm is implemented on a GPU to exploit its parallel computing capabili-ties. The most popular GPU Architecture is CUDA (Compute Unified Device Architec-ture), which is the software architecture developed by NVIDIA GPGPU (General-PurposeComputing on Graphics Processing Units). This high-resolution range capture module isfully integrated as a ROS node and supports both full-resolution and foveated operation.

4.8 DemonstrationUniversity of Glasgow has designed and delivered 3 working binocular robot heads andprovided an graphical user interface that supports, stereo-pair image capture, camera

19


pan/tilt control and automatic vergence control either by physically converging or di-verging the cameras or by shifting the offset between the captured stereo-pair images.Similarly, we integrated range capture based on the University of Glasgow C3D technol-ogy and this was integrated in ROS in order to provide high resolution full field-of-view(FOV) range maps.

During the M24 demonstrator, University of Glasgow will therefore present:

• Demo 1: Camera and hand-eye calibration of the binocular stereo robot head. Avideo will present the main idea of the calibration procedure where the robot cal-ibrates itself and demonstrates that the RH is calibrated by means of a range mapand back-projection of the calibration target markers.

• Demo 2: Live demonstration of the visual range sensing system. The workspacetable will be draped with an item of clothing and the robot head will then vergethe cameras on the cloth, generate and display a range map and surface topologyfeatures. The cameras will then saccade to a new fixation point on the clothing,under manual or automatic control (driven by salient information).

• Demo 3: Live grasping demo. This demonstration is essentially an extension ofdemo 2 above. The cameras will complete a saccade to look for the best graspingpoints; after selecting the best candidate, the robot will attempts to grasp at thatlocation.

• Demo 4: Flattening demo: a video will show the new feature extraction method-ology and wrinkle representation methodology detect wrinkles and apply gripperforces to flatten these in simulation.

4.9 Performance4.9.1 Dynamic calibration

The ability of the system to maintain dynamic calibration is expressed in terms of theaccuracy of 3D reconstruction. The cameras, in this case, can be positioned in any anglerange in tilt and pan axes (within the hardware range limits of the PTU); therefore, theextrinsic properties of the stereo configuration need to be updated as explained in section4.2. The objective of this experiment is thus to test the accuracy of the binocular robothead to recover and reconstruct an accurate point cloud despite any camera motion. Fig-ure 10 shows a simplified example of the coordinate frames defined for describing thekinematic structure of the robot head.

For this purpose, we employed an object model with known 3D structure. The objectmodel used in this experiments is the check-board calibration target shown in Figure 3.This planar calibration target allow us to:

1. compare the reconstruction residuals between a calibrated stereo system and theupdated extrinsic camera parameters, and,

20


Figure 10: Simplified coordinate frames of the stereo head. This consists of the world,camera and calibration target coordinate systems.

2. verify the accuracy of the reconstructed geometry by measuring the RMSE betweena known 3D (ground truth) measurement on the real object and the 3D reconstructedmodel.

To measure the accuracy of the system, we measure the 3D euclidean distance fromone point of the calibration target to next in line. The ground truth distance betweensquares on the check-board is 24mm. Therefore, the RMSE between reconstructed pointsof the check-board while using the dynamic calibration parameters is compared with re-spect to the true value. Table 2 shows the RMSE (in millimetres). From the results inthe table, we can conclude that the system is able to recover the 3D geometry with sub-millimetre accuracy.

4.9.2 Stereo GPU matcher

Experiments Setup The experiment was carried out in a Intel(R) Core(TM) i5-2400CPU with 3.10GHz, which contains 4 cores, and 8 GB of RAM running under Ubuntu12.04. The GPU model used during these experiments is an Nvidia GeForce GTX 770,which has 1536 processors, with a 4GB total memory of RAM. CUDA C is the languageused for coding acceleration on the GPU. The test data originates from a pair of stereocameras of the Glasgow’s Stereo Image Database of Garments [7], and every single stereoimage have 4928 × 3264 pixels. They are represented by three layers and each pixel isstored as 32 bits.

Results The total running time of the GPU stereo matching is 13.34 seconds (imagepyramid costs 0.46 second and matching costs 12.88 seconds). For a foveated matching,this time decreases to 4.13 seconds in total (generating the foveated image pyramid costs0.47 second and matching costs 3.66 seconds). Range image examples while matching

21


Table 1: Residual errors (in millimetres) between optimal stereo calibration and dynamiccalibration of the extrinsic camera parameters. 3D coordinates are expressed with respectto the world reference frame of the robot head.

Pose # X Y Z1 0.17 -1.03E-02 -3.02E-022 1.50E-02 6.37E-02 -4.68E-023 2.04E-02 5.14E-02 3.76E-024 0.671 -6.69E-02 -0.2015 0.792 -8.60E-02 -0.1116 7.82E-02 1.85E-02 -3.47E-027 0.427 -3.67E-02 -0.1198 6.95E-02 1.16E-02 -6.94E-039 0.381 -3.22E-02 -6.90E-02

10 0.322 -2.60E-02 1.50E-02Mean 0.294 -0.0113 -0.0561 Std 0.275 0.0257 0.06

Table 2: RMSE (in millimetres) between estimated 3D reconstructed points and groundtruth.

Pose # Mean 1 Std1 0.1008 0.07722 0.0904 0.03493 0.1279 0.06744 0.3171 0.07255 0.0567 0.06016 0.1257 0.06797 0.2292 0.07768 0.2204 0.07789 0.0570 0.0577

10 0.3463 0.0699Overall 0.1671 0.1050

22


Left image Right image Range map image

Figure 11: Input Left and Right Images and Range Image

Pyramid level Full Matching Foveated Matching1 0.0003251 0.000185122 0.00060787 0.000698723 0.00091098 0.000830834 0.0013234 0.00120595 0.002095 0.00348166 0.0025911 0.00928347 0.003664 0.022397

Table 3: Mean Square Error between full matching and foveated matching point cloud

the left and right images in our dataset [7] are shown in Figure 11. The Mean SquareError (MSE) between the generated point cloud from a full and a foveated matching areshown in Table 3. The table demonstrates that the MSE increases from the central area toborder; this is because the image resolution close to the image periphery is coarser withrespect to the central area.

4.9.3 Simulated Flattening Experiments

In order to demonstrate the effectiveness of our feature in representing clothes surfaces,we employ the descriptor described in Section 4.4 in cloth flattening task in a physicalcloth simulator. Our flattening strategy assumes a single robotic arm which consists ofeliminating the largest wrinkle iteratively. Figure 12 depicts a flattening experiment.

23


State 0 Force 1

State 1 Force 2

State 2 Force 3

State 3 Force 4

State 4 Force 5

State 5 Force 6

State 6 Force 7

State 7 Force 8

State 8 Finished

50 100 150 200 250 300 350 400

50

100

150

200

50 100 150 200 250 300 350 400

50

100

150

200

50 100 150 200 250 300 350

50

100

150

200

50 100 150 200 250 300 350 400

50

100

150

200

50 100 150 200 250 300 350 400

50

100

150

200

50 100 150 200 250 300 350

50

100

150

200

50 100 150 200 250 300 350

50

100

150

200

50 100 150 200 250 300 350 400

50

100

150

200

50 100 150 200 250 300 350 400

20

40

60

80

100

120

140

160

180

200

220

Figure 12: Flattening demo.

24


5 Photometric Stereo System

5.1 DescriptionThe photometric stereo system captures the 3D microgeometry of a garment and usesthis information to assess its surface roughness and infer its material properties. Thesystem consists of a small camera synchronized with light sources and software for 3Dreconstruction and classification.

5.2 State of the realizationThe development of the photometric stereo sensor has been completed. The sensor con-sists of a camera, 4 LED arrays as light sources and a control system to synchronise thelight sources with the camera. All components are enclosed in a 3D printed cylindricalcase with diameter of 39 mm and height 40 mm, figure 13. The sensor fits inside thegripper of the robot and connects to the CloPeMa server using the USB interface. Thephotometric stereo software has been written in C++ and its main functionality has beenimplemented as a ROS service.

Figure 13: The photometric stereo sensor (left) and installed on the robot (right)

5.3 IntegrationThe photometric stereo system is fully integrated into CERTH and CVUT testbeds. Thesensor has been mounted on the gripper of the robot and the whole system has beencalibrated and tested under ROS Hydro.

5.4 DemonstrationDuring the M24 demonstration the main functionality of the photometric stereo systemwill be shown. The sensor will be mounted on the gripper and a garment will be lyingflat on a table in front of the robot. When the photometric stereo service is called, therobot hand will approach the table, the sensor will capture 4 images of a small patchof the garment surface under different illumination conditions and it will return its 3D

25


reconstruction, figure 14. Next, an assessment of the surface roughness and will be madeas well as an estimation of the garment material (cotton, wool etc). The same procedurewill be repeated for different clothes.

Figure 14: Capturing setup (left) and 3D reconstruction (right)

5.5 PerformanceThe photometric stereo system can recover the 3D microgeometry of a small patch of theinspected surface in about 6 seconds. The accuracy of the 3D reconstruction is adequatefor the purposes of assessing the roughness of a garment and since this is not a gauging de-vice, it has not been evaluated extensively. The classification system classifies garments toone of the following 5 categories: cotton pants, cotton shirts, cotton towels, wool clothesand polyester clothes. The overall classification rate using only 3D shape features is over65%, with most errors observed in distinguishing cotton shirts and polyester clothes dueto the low inter-class variation. When we consider more categories (viscose, silk etc) therate drops to about 50%.

26


6 World Representation, Perception and Action Planning

6.1 DescriptionThe demonstrator consists of autonomous unfolding of an unknown piece of garmenttaken from a heap. This includes vision algorithms and planning pipeline for autonomousgarment grasping from a heap and subsequent unfolding using a dual arm percpetion-action loop described in detail in deliverable D6.1 [5].

6.2 State of realizationA working prototype is realized. Several improvements are though planned towards thefinal demonstrator. These include a) improvement of grasping by means of collision awaregrasp search, b) improvement of unfolding planning by the introduction of new actionsin the perception-action look and better visual features, c) realization of an alternativegeometric-based unfolding algorithm.

6.3 IntegrationThe demonstrator has been fully intergrated into CERTH testbed, nd evaluated. The codeand sensor configuration was duplicated on the CVUT testbed.

6.4 DemostrationThe demonstration will be on the testbed. The heap will be on a table located on theside of the robot. The first step is to use vision from an eye-on-hand camera to locatethe garment heap, find and evaluate potential grasping points, pick the garment from oneof the selected points and evaluate the success of the grasping. If the grasping is notsuccessful the process is repeated a number of times until successful. Then the robottorso will rotate 45 degrees and the holding arm will bring the item in front of the robot.An Xtion sensor placed close to the central stereo-head column will be used to guide thesubsequent unfolding procedure. The free arm will move to grasp the cloth from its lowestpoint as determined by the sensor and will bring it in a configuration that the wrist isvertical to the floor and the hanging cloth in the viewing frustum of the camera. The wristwill rotate while images of the hanging cloth will be captured. The unfolding algorithminteractively rotates the wrist of holding robot to classify the garment in one of the knowntypes and locate the next potential grasping point. If the point is correctly located the freearm will move to grasp it. The procedure is repeated with the other arm. Once the secondgrasp point has been grasped the arms are spread so that cloth unfolds due to gravity.The above procedure is safeguarded by grasp stability evaluation using vision. In case ofgrasping failure the corresponding step is repeated. Flipping hands is performed by dualarm planning with closure constraints to avoid having the cloth oscillating.

27


Figure 15: Unfolding pipeline (from top-left to bottom right): 1) Approaching table anddetecting garment. 2) Approaching garment detecting grasping candidates. 3) Movinginto pre-grasping position. 4) Grasping garment. 5) Confirm grasping. 6) Picking fromtable completed. 7) Moving arms for unfolding and lowest point detection. 8) Graspinglowest point. 9) Confirm grasping. 10) Switching arms and detection of first graspingpoint for unfolding. 11) Rotating cloth to that the grasping point could be graspable fromthe arm. 12) Grasp garment. 13) Confirm grasping. 14) Switching arms and detection offirst grasping point for unfolding. 15) Grasp garment. 17) Unfolding Completed

6.5 PerformanceGarment pick-up from heap was evaluated on the CERTH testbed. Three randomly se-lected garment items were stacked to make a heap resembling clothes coming out from thelaundry machine. The grasping procedure is executed and success or failure is recorded.A failure consists of failing to pick-up the garment after 3 tries or picking up two piecestogether. Then, the heap is manually shuffled and the process is repeated. Three suchheaps (with different garments) were used giving a total of about 80 experiments. A 95%success rate was recorded. The running time is about 20 secs with about 18 secs con-sumed in robot planning and movement (in safety speed) and less than a second for visualscene analysis and image acquisition.

Garment unfolding has been also evaluated on the robot using a small dataset of gar-ments from a few types (t-shirts, pants, long sleeved shirts, shorts). Unfolding has beensuccessful in about 93% of the cases. The majority of failures was due to collisions ofthe gripper with the cloth thus failing to grasp it. The overall running time is about 2mins but most of this time is consumed by the robot movement (in safety speed) while theclassification time consumes totally a couple of seconds.

28


7 Recognition, learning and active exploration

7.1 DescriptionWe developed recognition algorithms which are used for solving tasks corresponding toproject aim. The recognition and learning algorithms are described in detail in deliver-ables [5]. We are solving following tasks:

• Unfolding process uses active recognition method using random decision forests.The random forests and probabilistic planning are utilized for action planing. Themethods ensures, that hanging garment is recognized and unfolded. The randomforest representation and probabilistic planning corresponds to word representation.Therefore the demonstration of the method is described in section 6.

• Folding process utilizes polygonal model of the garment lying on a table. Thelying unfolded garment is segmented based on RGB image and learned model of abackground. The polygonal model is represented by significant points. Positions ofthe points are estimated according to observed garment contour. The model is usedin process of fold planning.

7.2 State of the realizationThe folding based on polygonal model is realized and fully working. The polygonalmodel fitting algorithm was extended by dynamic model modification. The model mod-ification is based on known folds witch were applied on the garment. The modificationallows to use the model in each step of folding. We focused our work on using of themodel for garment recognition and verification of executed folds.

7.3 IntegrationThe polygonal model algorithm is implemented in Matlab environment. The code isintegrated into ROS through Matlab interface, which was developed in previous year. Thepolygonal model is utilized by folding procedure. The whole folding process is workingand allows to fold towel, T-shirts and trousers which are lying on working table.

Unfolding procedure were realized and is demonstrated as mentioned in section 6.The realized prototype of the process grasps garment from heap and unfolds the hang-ing garment. The unfolding and folding procedure represents whole process of foldinggarment from heap. The methods are not fully connected yet. Now, we are working onintegration of unfolding and folding. Between the operations must be inserted procedurewhich put the hanging garment down to the table.

7.4 DemonstrationWe demonstrate the polygonal model method by fitting the model on the image of piece ofgarment (towels, T-shirts, trousers) lying on table. The fitting of the polygonal model wastested on our image dataset. The algorithm is working properly and is utilized in folding

29


process. The folding process is driven by preprogrammed scheme for the specific type ofgarment. The model is fitted on actual image of the garment. In each step of folding, themodel is updated by known realized fold.

The folding process is demonstrated on the test bed. The demonstration starts witha piece of garment lying on working table in front of the manipulator. The manipulatoruse one of the Xtion sensors on the arms to obtain image of the garment from to ofthe table. The sensor is placed in proper place by manipulator. The polygonal modelposition is estimated and first fold is planed. Than the fold is realized by method called G-folding. Now actual image of the garment is captured. Than polygonal model is updatedby knowledge about realized fold and fitted based on the actual image. After that, nextfold can be planed and realized. The demo finish when all preprogrammed folds for thetype of garment are realized or when the path, which realize next fold is not realizable.The folding process of T-shirt is documented on figure 16.

Figure 16: Process of T-shirt folding. The folding is based on polygonal model.

7.5 PerformanceThe polygonal model offers localization of the garment with sufficient precision. Theprecision of significant points positions is up to 17.9 mm (standard deviation). In thatcase resolution of processed images is 320×256 pixels. Images from camera with orig-

30


inal resolution 1280× 1024, are down-sampled. Edge of pixel in down-sampled imagerepresents approximately 4mm in the scene (garment lying on table).

The down-sampling ensure speed of processing with suitable precision. The segmen-tation, contour extraction and model fitting takes around 5 seconds in worst case. Mostof the time is consumed by contour extraction and simplification. The steps are highlydependent on the length of the contour. The model matching algorithm works with sim-plified contour and thus it is very fast. Details of the algorithm performance could befound in deliverables [5].

The manipulator driven by the polygonal model is able to fold towel, T-shirt andtrousers of appropriate size. The folding process itself is successful each time, when thegarment lies in working place of the robot and each step of folding procedure can berealized in the working place. Problem is that quality of realized folds differs. The resultof one step of folding algorithm is determined by position of the garment and limited byrobot dexterity. Therefore some folds are not perfect.

Folding T-shirt take 4 minutes. All the computation (segmentation, model fitting, pathplanning) takes only 15 seconds. The time is limited by speed of the robot movement.Due to security reasons and robot safety, we must ensure, that operator can stop the robotbefore the robot destroy itself or other equipment. Therefore we set speed of the robot to2% of maximal speed.

31


8 System IntegrationIn the second demonstrator, the individual modules have been integrated to the CVUTCloPeMa system. The modules share the hardware resources, sensors, mechanics, ROSmodules etc. They have to work side-by-side on the same system without disturbing thefunction of the other modules. The shared modules have to be managed to avoid controland usage conflicts such as simultaneous usage of the robot mechanics, sensor setting,etc. The same interface and data representation have to be used for the shared modulesand data.

Moreover, as the hardware and software setup of the individual testbeds differs be-tween partners, the modules have been modified to work on the CVUT CloPeMa system.E.g. the CVUT CloPeMa system uses more computers while the other partners use one.The first computer is used for the main computation and control. The other computersare used for the sensors and some subsystems control and their are placed directly on therobot to decrease the distance. The modules have been modified to work in the distributedsystem.

The demonstrator is mainly designed as a live robot demonstration to present the func-tion of the system under the real-word conditions. For the demonstrator, the individualmodules which have been integrated to the CVUT CloPeMa testbed will be presentedseparately to demonstrate their functions and performance. The description of the indi-vidual modules can be found in the previous sections of this document and in the modulespecific deliverables.

8.1 Integration of the Stereo-head to the CVUT CloPeMa systemNEO has integrated the Stereo-head camera system, which has been developed at UG,to the CVUT CloPeMa system. The integration has mainly consisted of the hardwareadjustment, software modification and testing and new modules implementation.

The hardware adjustment consists of the stereo-head stand modification to adjust theview angle, camera distances and position and from the cameras power supply and datawiring.

The software of the stereo-head system has been tested and updated in cooperationwith the UG for the integration to the CVUT CloPeMa system. As the UG system usesone PC and the CVUT CloPeMa system uses two PCs the software has had to be modi-fied. The CVUT uses one PC for the main computation and robot control and the secondmini-PC to acquire the data. The software of the stereo-head system has been dividedin two parts for the data acquisition and for the point-cloud computation. The commu-nication and interface between these two parts has been implemented. Several modulesand scripts has been implemented for the integration such as cameras detection and reset,USB handling, data under-sampling, etc. The robot auto calibration of the stereo-headsystem has been adapted for the CVUT CloPeMa system and tested.

Part of the Stereo-head integration to the CloPeMa system has been a calibration ofthe relation between the stereo-head and the robot coordinates systems. The calibrationprocedure has been implemented by NEO. In the firs stage the computation has been im-plemented in Matlab and after the verification and accuracy evaluation it has been reim-

32


plemented in C++ for better integration to the CloPeMa system. The calibration is runningin two phases. The first phase automatically acquires the image data of a calibration tar-get, which is attached to the robot gripper. The second phase computes the coordinatesprojection.

8.2 Integration of the CERTH ROS package for garment unfoldingto the CVUT CloPeMa system

A ROS package for garment unfolding which has been developed at the CERTH has beenintegrated to the CVUT CloPeMa system. The integration has been performed by NEO.The 3th Xtion sensor has been integrated to the system for this task. Several modificationsand software updates have been performed in cooperation with the CERTH. The packagefunction and compatibility with the rest of the system have been tested.

8.3 Integration of the Photometric stereo sensorThe Photometric stereo sensor has been integrated to the first year gripper of the CloPeMasystem. The function control and data have been provided to the ROS system and to theother modules for the future usage. More details can be found in Section 5

8.4 Integration of the CloPeMa hand and tactile sensingThe second year CloPeMa hand is going to be integrated to the CVUT system during theFebruary. More details can be found in Section 3

33


9 ConclusionAccording to the Description of Work [1], the second demonstrator deliverable describesthe individual components of the integrated system, the state of their realization and fu-ture vision. The demonstration of the current state of the system integration has beenpresented.

The main part of the demonstrator is the live robot demonstration of the integratedsystem at the CVUT CloPeMa testbed.

34


References[1] DoW - Description of Work, European Communitys Seventh Framework Pro-

gramme (FP7/2007-2013) under grant agreement no 288553, CloPeMa, August2011.

[2] Deliverable D1.1, European Communitys Seventh Framework Programme(FP7/2007-2013) under grant agreement no 288553, CloPeMa, May 2012.

[3] Deliverable D4.1, European Communitys Seventh Framework Programme(FP7/2007-2013) under grant agreement no 288553, CloPeMa, November 2012.

[4] Deliverable D2.2 - Experimental testbed with dual multifinger hands and its con-trol using tactile feedback, European Communitys Seventh Framework Programme(FP7/2007-2013) under grant agreement no 288553, CloPeMa, January 2014.

[5] Deliverable D6.1 - Visual patterns, European Communitys Seventh Framework Pro-gramme (FP7/2007-2013) under grant agreement no 288553, CloPeMa, January2014.

[6] Gerardo Aragon-Camarasa, Haitham Fattah, and J Paul Siebert. Towards a uni-fied visual framework in a binocular active robot vision system. Robotics and Au-tonomous Systems, 58(3):276–286, March 2010.

[7] Gerardo Aragon-Camarasa, Susanne Oehler, Yuan Liu, Sun Li, W. Paul Cock-shott, and J. Paul Siebert. Glasgow’s stereo image database of garments. CoRR,abs/1311.7295, 2013.

[8] P. Benes, M. Valasek, and J. Sveda. Measurement and calibration of machine toolsin 6 dofs in large workspace. Journal of Machine Engineering, 9(3):77 – 87, 2010.

[9] Petr Benes, Vladimır Petrık, Zbynek Sika, and Michael Valasek. Calibration us-ing two laser trackers leica AT901 MR. Technical report, Czech Technical Univer-sity in Prague, 2014. http://clopema.felk.cvut.cz/files/reports/Benes_CalibrationTR2014.pdf.

[10] Petr Jerabek. Computer control of the projector of the range finder, July 2013.http://cyber.felk.cvut.cz/research/theses/detail.phtml?id=429.

[11] Yuan Liu and J. Paul Siebert. Contour localization based on matching dense hex-hog descriptors. In VISAPP 9th International Joint Conference on Computer VisionTheory and Applications, Lisbon, 5-8 January 2014.

[12] Vladimır Petrık and Vladimır Smutny. Comparison of calibrations for theCloPeMa robotcomparison of calibrations for the CloPeMa robot. Technicalreport, Center for Machine Perception, Department of Cybernetics FEE CTUin Prague, 2014. http://clopema.felk.cvut.cz/files/reports/petrik_calibration_verification.pdf.

35

http://clopema.felk.cvut.cz/files/reports/Benes_CalibrationTR2014.pdf

http://clopema.felk.cvut.cz/files/reports/Benes_CalibrationTR2014.pdf

http://cyber.felk.cvut.cz/research/theses/detail.phtml?id=429

http://clopema.felk.cvut.cz/files/reports/petrik_calibration_verification.pdf

http://clopema.felk.cvut.cz/files/reports/petrik_calibration_verification.pdf


[13] Z. Sika, V. Hamrle, M. Valasek, and P. Benes. Calibrability as additional designcriterion of parallel kinematic machines. Mechanism and Machine Theory, 50(0):48– 63, 2012.

[14] Zbynek Sika, Petr Benes, Michael Valasek, Vladimır Smutny, and Pavel Krsek. Cal-ibrations of robots within CloPeMa project using RedCam arm. Technical report,Czech Technical University in Prague, 2013. http://clopema.felk.cvut.cz/files/reports/sika_redcam.pdf.

[15] M. Valasek, F. Petru, Z. Sika, V. Bauma, and R. Smıd. Method and appa-ratus for measurement and/or calibration of position of an object in space, eu-ropean patent ep1968773, 2012/06/13. http://www.freepatentsonline.com/EP1968773.html.

[16] Changchang Wu. SiftGPU: A GPU implementation of scale invariant feature trans-form (SIFT). http://cs.unc.edu/ ccwu/siftgpu, 2007.

36

http://clopema.felk.cvut.cz/files/reports/sika_redcam.pdf

http://clopema.felk.cvut.cz/files/reports/sika_redcam.pdf

http://www.freepatentsonline.com/EP1968773.html

http://www.freepatentsonline.com/EP1968773.html

Documents

D7.3 Second Project Demonstrator and its Validationclopemaweb.felk.cvut.cz/wp-content/uploads/sites/2/... · Clopema - 288553 D7.3:Second Project Demonstrator and its Validation The