Automation in Construction 35 (2013) 579–586

Contents lists available at ScienceDirect

Automation in Construction

j ourna l homepage: www.e lsev ie r .com/ locate /autcon

Installation of heavy duty glass using an intuitive manipulation device

Myeong-Su Gil a, Min-Sung Kang a, Seunghoon Lee b, Hee-Don Lee b, Kyoo-sik Shin b,Ji-Yeong Lee b, Chang-Soo Han b,⁎a Dept. of Mechatronic, Eng., Hanyang University, 1271 Sa-1 dong, Sangnok-gu, Ansan, Kyeonggi-do 426-791, Republic of Koreab Dept. of Mechanical, Eng., Hanyang University, 1271 Sa-1 dong, Sangnok-gu, Ansan, Kyeonggi-do 426-791, Republic of Korea

⁎ Corresponding author. Tel.: +82 31 400 5247; fax:E-mail address: cshan@hanyang.ac.kr (C.-S. Han).

0926-5805/$ – see front matter. Crown Copyright © 20http://dx.doi.org/10.1016/j.autcon.2013.01.008

a b s t r a c t

a r t i c l e i n f o

Article history:Accepted 6 December 2012Available online 16 February 2013

Keywords:Human–robot cooperationConstruction robotIntuitive manipulation deviceHeavy duty glassVirtual axis

Glass is widely used as finishing material to improve both building interiors and exteriors. As buildings are be-coming larger, thematerials used are also increasing in size to reduce the time required for construction. Toma-nipulate a heavy duty panel glass intuitively, we proposed an installation methodology based on human–robotcooperation for a glazing robot. Here, we classified all types of glass installation robots, and then analyzed the in-stallation methods compared to construction workers. We propose a human–robot cooperation method that issimilar to installation method of construction worker and detail the requirements. In addition, a control systemfor using intuitive manipulation device is implemented. We confirmed that the glass installation work using in-tuitive manipulation device is more efficient than existing methods through a simple experimental setup. Thismethod combines operator's control ability and robot's power. It is expected that glass installation work willnot only be more efficient but also safer.

Crown Copyright © 2013 Published by Elsevier B.V. All rights reserved.

1. Introduction

Currently, during the process of installing heavymaterials inside andoutside of buildings, handling materials has been mostly executed byusing cranes or pulleys, whereas assembly and installation has mostlybeen done by construction workers. That is, under a changeable envi-ronment, the construction industry has a limit to wholly automatizeor semi-automatize the required capacities (such as accurate approachand alignment) on assembly and installation [1]. However, these sim-ple, repeated processes have yet to be automatized and the develop-ment of automated systems or semi-automated systems based onhuman–robot cooperation has been attempted to aid this situation [2].

Gonzalez designed a power assistance device for the safety of theoperator while installing plaster panels on construction sites [3]. Yu etal. developed a curtain wall installation robot for the safety of the oper-ator and to increase work efficiency at construction sites [4]. KAJIMACorp. developed a ‘Mighty Hand’ for handling concrete and glass mate-rials [5]. This corporation also developed amulti-jointed handling robotto install heavy materials on both inside and outside of the buildings.Likewise, to install construction materials on the ceiling of buildings,an interior finishing robot was developed by Shimizu Corp. Lee et al.designed a ceiling glass installation robot to install heavy glass on

+82 31 406 6398.

13 Published by Elsevier B.V. All rig

buildings inside and out [6]. These heavy material handling robots par-ticularly showed the possibility of improving the operator's safety,workefficiency, and protecting construction materials at the same time. Still,these existing robots have only demonstrated optimal performance forlimited target tasks. On a practical level, it is realistically difficult forthese robots to be applied because of their complex systems, high-costand difficulty in mass production. Exceptionally, OKTOPUS developedbyMaterials Handling Corp. in Australia, Mobile Ergonomic Handler de-veloped by Arlington Equipment Corp. in USA, Geko & Glass Robot Hiredeveloped by GGR Corp. in UK, and Spider Crane developed by PeterHird & Sons Ltd. in UK have been commercialized on constructionsites [7–10]. However, these robots have limited capacities andmaneuver.

Considering all these robots, we reached a provisional conclusion: arobot manipulation device could serve as an alternative since it is notonly portable, but also enables the operator to directly handle themate-rials. This paper presents several techniques to realize concepts such asthe virtual axis coordinates and the input force treatment to generaterobot motion commands. Finally, we verified the feasibility of the pro-posed system through simple comparison experiments.

2. Proposal of a new method

2.1. Analysis of existing method

In order to analyze the existing methods, we need to classify theexisting robots. Depending on the input signal of robots, they are

hts reserved.

580 M.-S. Gil et al. / Automation in Construction 35 (2013) 579–586

classified as robots driven in Joint space and Cartesian space. Based onthe carrier, this paper classified installation methods by operators androbots.

In the case of a construction worker, first, the glass is transportednear the contact point — A. At this time Fx, Fy, Fz are generated ontothe instantaneous axis of rotation as shown in Fig. 1(a). Then Ty, andTz are acted onto the instantaneous point of rotation (b). After that,the glass is rotated on contact point A which is put on the glass.Next, Tx is generated onto the instantaneous axis of rotation (c).Then the glass is rotated on the rotational axis which is the vector de-rived from contact point A to contact point B. Finally, the installationwork is finished.

Fig. 2 presents the installation method by robot driven in Jointspace. First, glass is closely transported to the target location with alarge linear motion. At this time, Fx, Fy and Fz are acted onto the in-stantaneous axis of rotation within the glass. Second, the glass is ro-tated near the intended install location. At this time, Tx, Ty and Tzare acted on the rotational axis for fitting the glass on the installationplane on a parallel level. Third, the glass is fitted onto the destinedplace with a slight linear motion. After repeated motions shown inFig. 2(b) and (c), the installation is finished. For installing a fragilematerial, an operator has to pay careful attention to the installationprocess. Also, a fixed instantaneous axis of rotation, such as Fig. 3, isthe cause of increasing the repeated number of rotational and trans-lational motions for the robot driven in Joint space. Therefore, the in-stallation work by robot driven in Joint space requires many moremotions than work done by a construction worker. For these reasons,the work efficiency of this method is lower than that of a constructionworker.

To solve this problem, we developed a ceiling glass installationrobot, which is driven in Cartesian space. Theoretically, the installa-tion method using a robot driven in Cartesian space is more efficientthan a method using a robot in Joint space. However, this method isnot efficient because of the following issues. Fig. 4 displays the prob-lem of the installation method by using a robot driven in Cartesianspace. This robot system uses a 6DOF F/T sensor for inputting theoperator's force. In order to work more intuitively, the location ofthis sensor is set on the robot EEF. Then this sensor inputs theoperator's force and torque. This robot system needs more than twopoints of action as shown in Fig. 4. With these points of action, thesensor not only receives the operator's torque but also the torquegenerated between the operator's force and the moment arm. There-fore, the operator's force and torque are not inputted to the sensoraccurately and the robot is driven in unexpected motions. In thecase of the robot method, exact simultaneous force and torque are re-quired to act on the glass in order to install the glass with as little mo-tion as possible. However, in the case of the operator method, only

a) Linear-motion b) Rotational motion-Tz, T

Fig. 1. Installation method b

the linear force needs to act on the glass such as Fig. 4. This problemof robot method is a cause of repeated motion. Consequently, theseunexpected motions and repeated motions cause inefficiency.

Assuming that the installation method by an operator is ideal, theproblems of each robot method are presented in Table 1.

2.2. Methodology of the proposed method

The proposed method in this paper is to carry the process out inthe same way as the ideal operator's method. First, the instantaneousaxis of rotation is located on the center of the panel glass. Second, theinstantaneous point of rotation is located on the edge of the glass.Third, the instantaneous axis of rotation is located on the contactline. However, it is difficult for a robot to find these changeable in-stantaneous axes of rotation. Also, the size and the location of theglass attached to the robots are subject to change. Therefore, in thispaper, an operator decides the instantaneous axis of rotation, andthe robot detects these axes. At this time, the points of action are lo-cated on these axes. There are 2 points of action specified in thispaper. The first point (we assume this point to be the virtual axis)plays the role of the instantaneous axis of rotation and decides thelinear motion of the glass. The second point decides the rotationalmotion of the glass. The roles of these points are changeable by theoperator. In other words, these points receive the operator's forceand create the trajectory of the robot EEF for handling the panel glass.

Fig. 5 presents the methodology of the proposed method asfollows:

1. An operator decides on the linear and rotational motion of theglass while the robot handles the mass of the glass.

2. The proposed method is carried out in the same method done by aconstruction worker.

3. The operator decides the instantaneous axis of rotation.4. The first point defines the instantaneous axis of rotation and de-

cides the linear motion of glass.5. This paper assumes that the first point is the virtual axis.6. The second point decides the rotational motion of the glass.7. These points receive the operator's force and generate the trajectory

of the robot EEF.

3. Operation strategy

3.1. Detection method of the points of action

In this paper, we assume that the instantaneous axis of rotation iswithin the intuitive manipulation device (IMD). In other words, the

y c) Rotational motion-Tx d) Finished work

y construction worker.

a) Linear motion-large motion b) Rotational motion c) Lenear motion-little motion d) Finished work

Fig. 2. Installation method by robot driven in joint space.

581M.-S.G

iletal./

Autom

ationin

Construction35

(2013)579

–586

Fig. 3. Problem of installation method by robot (joint space).

Table 1Limitations of robot methods.

Installation method Problem

By an operator No problem (ideal)Using robots driven in Joint space Repeated motionsUsing robots driven in Cartesian space Unexpected motions & repeated motions

Fig. 5. Methodology of proposed method.

582 M.-S. Gil et al. / Automation in Construction 35 (2013) 579–586

center axis of the IMD corresponds to the instantaneous axis of rota-tion. Therefore, detecting the points of action means definingthe position and the orientation of the IMD with respect to therobot EEF.

Due to various installation forms, working methods and irregularsizes of glass, the position and orientation of the IMD put on theglass is not fixed. Additionally, the operator determines its positionand orientation arbitrarily. Thus in this study, we used the IR sensormodule and acceleration sensor to detect the position and orientationof the IMD. The IR sensor module is composed of an infrared sensorand a radio control motor. This sensor module is attached on therobot EEF. The IR sensor measures distance of the IMD's contour withrespect to the robot EEF while the RC motor is rotating. Fig. 6 showsthe detection method of the IMDs.

Fig. 6. Detection method of the IMDs.

3.2. Kinetic relationship of forces

In Cartesian space, the glass's position and orientation changedepending on the force and torque generated on the glass, and theoperator's forces inputted from IMDs generate the trajectory ofrobot EEF. Therefore, to define this relationship, we considered 2cases: when the virtual axis is at the robot EEF and when the virtualaxis is at the IMD.

Fig. 4. Problem of glass installation by robot (Cartesian space).

Fig. 7. Force and torque acted on robot.

Fig. 8. Force and torque acted on virtual axis.

Fig. 10. Calculation method for center of IMD.

583M.-S. Gil et al. / Automation in Construction 35 (2013) 579–586

If the virtual axis is placed at the robot EEF (Fig. 7), the force andtorque are determined by the combination of the forces on eachIMDs (left, right) as in Eqs. (1) & (2).

Fe ¼ eLRF1 þ e

RRF2 ð1Þ

Fig. 9. Algorithm for detecti

τe ¼ ePL � eLRF1 þ ePR � e

RRF2: ð2Þ

At this moment, Ze, ZL and ZR are parallel and perpendicular tothe panel glass because the IMDs and the robot EEF are put on a flatglass. The robot EEF's orientation, e1 and e2 are inputted from the ac-celeration sensors. 3-axes force vector acted on the left IMD is F1 andthe force vector acted on the right IMD is F2. ePL is the position vectorfrom the robot EEF to the left IMD and ePR is the position vector fromthe robot EEF to the right IMD.

Second, if the virtual axis is on the left IMD and the point of actionis on the right IMD (Fig. 8), the worker's force on the virtual axis(IMD-left) determines the force of the robot EEF as in Eq. (3). Andthe force on the point of action determines the torque of the robotEEF. At this moment, the torque on the robot EEF is the same as thatof the virtual axis as in Eq. (4)

Fe ¼ eLRF1 ð3Þ

τe ¼ τ1 ¼ LPR � LRRF2: ð4Þ

ng of the IMD's center.

Fig. 12. Prototype of the IMD.

Fig. 13. Operator force and torque acted on ro

Fig. 11. System configuration for the proposed method.

584 M.-S. Gil et al. / Automation in Construction 35 (2013) 579–586

4. Performance evaluation

We will describe, through a simple experimental setup, how thismethodology can be applied. The performance evaluation is com-posed of two parts. One is to detect the position of the IMDs, whichare arbitrarily attached on the panel by an operator. Another is to per-form the test and deliver results in comparison with existing methodsof Cartesian space.

4.1. Detecting the position of the IMDs

To detect the position of the IMDs, the center of the virtual axis andthe point of action is located. That is, the position vector of the IMD'scenter is derived from the IMD's contour. Therefore, to find a positionvector, an algorithm is shown in Fig. 9. This algorithm is as follows.

First, we select the best linearity range of the PSD (position sensi-tive device) sensor as in Fig. 9(a). This paper limits a sensing rangefrom 200 mm to 700 mm and this sensing range is divided into 5parts. Second, while the RC motor rotates, the angle and distancedata of the RC motor and the PSD sensor are inputted in thePolar coordinates as in Fig. 9(b). These data are converted into po-sition vectors in Cartesian coordinates like Fig. 9(c). Third, to dis-tinguish the IMD's contour signals, we divided the sensing rangeinto several groups with 100 mm intervals. Fig. 9(c) shows thesegroups. We selected a data group such as Fig. 9(c) which hadmore than a certain number of data in the group.Lastly, for linearization of these data in a selected group, we used ahigh-order interpolation method. We found the center of the IMDusing the following calculation method as shown in Fig. 10. Vec-tors AD and CD are perpendicular and the mean of the vector ACis r since the IMD is a circle. Therefore, the following equationsare derived.

AD→

⊥ CD→ ¼ AD

→ T⋅ CD

→ ¼ 0 ð5Þ

AC→ ¼ r: ð6Þ

bot EEF (robot driven in Cartesian space).

585M.-S. Gil et al. / Automation in Construction 35 (2013) 579–586

4.2. System configuration

To demonstrate the performance of the proposed method, the sys-tem configuration is shown in Fig. 11. We used a 6-DOF serial manip-ulator construction robot including a suction device and used a plasticpanel (1200 mm by 850 mm) instead of a panel glass assuming thatthe installation location is a vertical wall with a spring-damper sys-tem for safety.

The IMD, which receives the operator's force, is composed of a han-dle, a 3-axis load cell and a suction device shown in Fig. 12. To grip theIMD, the handle part is designed to input the 3-axis force of the operatorindependently. Therefore, to easily put the IMD on the panel glass, asuction device was made up of a lever and a sucking disk with a screwtype sucking method. That is, while the lever rotates, the sucking diskexpands. As a result, the IMD is attached to the surface of the panel.Tofind the IMDby the IR sensor easily, a detecting sectionwas designedin the shape of a circle with a 100 mm diameter. The height of the IMDis 150 mm and the allowed weight of the sucking disk is 80 kg.

4.3. Experiment and results

Fig. 13 shows the operator's forces acting on the robot EEF drivenin a Cartesian space such as ceiling glass installation work [6]. In thiscase, the working method is divided into 6 parts and requires a

Fig. 14. Operator forces and torque acted on

minimum of two operators. The work processes from (b) to (c) arelarge motions, part (d) is slight motion and part (e) is a work motionafter the installation. The total work time for the material installationwas about 250 s. The time of large motion (part-(b)) was 160 s, slightmotion (part-(d)) as 80 s and the rest (part-(e)) was 10 s.

Fig. 14 shows the operator's forces and torque acting on the IMDsand the robot EEF. In this case, the working method and the number ofrequired operators are the same as the method shown in Fig. 13. Thetotal work time was about 62 s. The time of large motion (part-(b))was 45 s, little motion (part-(d)) was 10 s, and the rest (part-(e)) was7 s.

When comparing these twoworkingmethods, it is apparent that thetotal work time is reduced using the proposedmethod. Considering theadditional installation work, these working times are similar to the re-sults of the performance evaluation of the ceiling glass installationrobot [6]. As in Fig. 13, the torque was generated (section A′) beforethe continuous linear force (section A) was inputted because it is diffi-cult for two operators to generate the same force vectors at the sametime. And in the slight motion, many repeated forces and torques(sections B and B′) are needed to fit the material on the installationplane because it is difficult for the operator to fit the glass on the firstattempt. In Fig. 14, the right IMD part (operator A) is an instantaneousaxis of rotation and the left IMD part (operator B) is the point of action.In the largemotion, as in Fig. 14, the force summation vector (sectionD′)

IMDs and robot EEF (proposed method).

Table 2Comparison of the proposed and existing methods.

Existing method Proposed method

Large motion work time 160 (s) 45 (s)Little motion work time 80 (s) 10 (s)Total work time 250 (s) 62 (seForce/torque interference Mutually dependent Mutually independent

586 M.-S. Gil et al. / Automation in Construction 35 (2013) 579–586

acted on the robot EEF is dependent on the force vector acted on theright IMD (section D), while the torque summation vector (section C′)acted on the robot EEF is dependent on the force vector acted on theleft IMD (section C). Likewise, in the slight motion, the torque summa-tion vector is dependent on the left IMD. Also, in the little motion, asmall force summation vector (section E′) is generated. This force vector(section E′) is a compensation vector for error that occurred from thedifference between the corner of thematerial (the actual instantaneousaxis of rotation) and the center axis of the right IMD. Operator A decidesthis vector during the slight motion work. Table 2 presents the charac-teristics of the two methods.

5. Conclusion

Currently, to protect glass materials and construction workers andincrease work efficiency, the study of heavy duty glass installation ro-bots have continued and these variable installation robots have beendeveloped and applied at construction sites. This paper investigatedand classified these many types of robot systems, and analyzed themethodology of glass installation compared to the human operators.

Based on this analysis, a new methodology of installing heavy dutyglass is proposed. Ourmethod detects the instantaneous axis of rotationand axis of the action point and controls the robot system from the in-putted force to these axes. At this time, the position and orientation ofthese axes are defined by the operator changeably and correspondedto the center axis of the IMD. This paper defined that the instantaneousaxis of rotation of the IMDwas to virtual axis. To detect the center axis ofthe IMD, this paper used an IR sensor module and an acceleration sen-sor. To understand the force relationship between the action pointand the robot EEF, the kinetic relationship was analyzed. To verify the

proposed methodology, a simple test using 6-DOF serial manipulatorwas compared with the existing method.

The results showed that the proposed method was faster thanexisting methods and comparable to one of operator in time [11].

Acknowledgment

This research was supported by Basic Science Research Programthrough the NRF funded by the Ministry of Education, Science andTechnology (2010-0025255), the MKE, under the Advanced RobotManipulation Research Center support program supervised by theNIPA (2011-C7000-1001-0002), the Happy tech. program through theNRF funded by the Ministry of Education, Science and Technology(No. 2011-0020938), a grant from the Construction Technology Innova-tion Program (CTIP) funded by the MLTM, and the Industrial Strategictechnology development program (10040180) funded by the Ministryof Knowledge Economy, Korea

References

[1] Carlos Balaguer, Mohamed Abderrahim, Trends in robotics and automation inconstruction, 978-953-7619-13-8, 2008. (InTech).

[2] T. Fukuda, Y. Fujisawa, K. Kosuge, F. Arai, E. Muro, H. Hoshino, T. Miyazaki, K. Outbo,K. Uehara, Manipulator/vehicle system for man–robot cooperation, Proceeding ofthe 1992 IEEE International Conference on Robotics and Automation Nice, 1992.

[3] P. Gonzalez de Santos, J. Estremera, E. Garcia, M. Armada, Power assist device forinstalling plaster panels in construction, Automation in Construction 17 (2008)459–466.

[4] Seung-Nam Yu, Seung-Yel Lee, Chang-Soo Han, Kye-Young Lee, Sang-Heon Lee,Development of the curtain wall installation robot: performance and efficiencytests at a construction site, Autonomous Robots (2007) 281–291.

[5] Mighty Hand, KAJIMA Co., http://www.Kajima.co.jp/topics/perspect/vol_15_1/luke/water/index.html.

[6] Seungyeol Lee, Myeongsu Gil, Kyeyoung Lee, Sangheon Lee, Changsoo Han, Designof a ceiling glass installation robot, International Symposium on Automation andRobotics in Construction (ISARC 2007), 2007, pp. 247–252.

[7] OKTOPUS, www.materialshandling.com.au/, Materials Handling Co., Australia.[8] Mobile Ergonomic Handler, www.gotoartech.com, Arlington Equipment Co., U.S.A.[9] Geko & Glass Robot Hire, www.ggrglass.co.uk, GGR Co., England.

[10] Spider Cranes, www.peter-hird.co.uk, Peter Hird & Sons Ltd., England.[11] Myeong-Su Gil, Byung-Gab Ryu, ChengJie Li, Min-Sung Kang, Seung-Nam Yu,

Chang-Soo Han, Development and performance evaluation of robot for the installa-tion of heavy ceiling glass using the human robot cooperation, 2009 KSME AutumnAnnual Conference, 2009, pp. 835–840.