CyberLab – Laboratory training over the internet

D. Iakovou, J. Meijer University of Twente, Mechanical Automation Lab., Horst CTW-WA,

P.O.Box 217, 7500AE Enschede, The Netherlands d.iakovou@ctw.utwente.nl, j.meijer@ctw.utwente.nl

G. Wiklund, A. Kaplan

Luleå University of Technology, Division of Manufacturing Systems Engineering SE-971 87 Luleå, Sweden

greger.wiklund@ltu.se; alexander.kaplan@ltu.se

Abstract

The advancement of internet technology has allowed the implementation of web based lectures and conferences. At these types of communication, knowledge and information can be exchanged instantly over great distances. The question that arises is whether hands on experience can also be passed on in the same manner. Such a task would allow trainees to learn and obtain experience on how to use equipment that is not directly available to them over the internet. In this paper it is presented how such a cybernetic laboratory environment can be implemented where students can learn how to use and gain hands on experience of the equipment of a real laboratory over the internet. Keywords: web lectures, internet laboratory, web meetings

1) Introduction Nowadays there is almost unlimited access to any type of information that might be required by companies of educational institutes. The development of internet and internet based applications has allowed instant sharing of knowledge anywhere in the world. Complete lectures from universities and video conferences can be transmitted and viewed all over the world from the interested students. Companies hold meetings over the internet, thus avoiding the loss of time and expenses of the trips. In all these situations the involved members are allowed to interact with each other by voice and video in the way that emulates a class or a meeting room. It is interesting to see how such an interaction could be extended in the laboratory environment (Fig.1.1).

Fig.1.1: Laboratory training over the internet

Such an interacting platform will be useful for both the educational institutes but also the industry. The universities will have the ability to provide laboratory courses to students from any part of the world. Additional the university could also provide to its own students laboratory courses with equipment that exist on some other institute or company. On the industry part, the companies will be able to train their employees and provide them with hands on experience on equipment at their work place, without having to move them to a specified training center. In this paper an implementation of such a laboratory course is presented.

2) Objectives In a real laboratory environment, the trainee is free to move around and have different views of the equipment, the process and the result. Additionally, the trainee has immediate access to the instructor for questioning, and to the equipment that is required to be trained on. The objective of this project is to allow the participant to have the same freedom on exploring the laboratory environment and on using the laboratory equipment over the internet. Furthermore, the participant must have immediate access to the instructor by voice and if possible by video. In a real course the instructor sets a number of behaving rules like: what students are allowed to do or not, how and when can the student address questions, etc. The same holds for this virtual laboratory, and the presented system is designed to allow the instructors to be able to implement their own rules. Finally, the instructor should be able to have full control over the actions of the student due to safety reasons. The system provides that possibility and allows only designated attendants by the instructor to take control of the laboratory equipment, and also enables the instructor to interrupt whenever it is required.

3) Equipment The equipment that is used for this internet course is chosen according to the requirements of the laboratory and the student learning environment. Even though variations to the required equipment might be required for different type of laboratory work cells, the main core can be used for almost every application. The courses are designed to accommodate two different alternatives for the student attendance. In the first alternative there is one lecture room where the students are gathered to attend the laboratory course through the internet. The second alternative is for the students to connect from their own home PC to the lecture. The main core of equipment for both alternatives is the same, though there are a number of differences on the implementation. The required equipment are divided in four major parts: the equipment necessary on the side on the course instructor in the laboratory for voice and video communication, the equipment that is required on the attendant’s part, the equipment required in the laboratory for the course implementation (robots, microscopes, etc) and the software that is required for the course and communication. a) Laboratory audio/video equipment The core equipment that is required in the laboratory is a couple of network pan/tilt/zoom camera, a video web server, a computer, a web camera and a headphone set. The course that is given is on robotized laser welding and the following equipment is used: A Sony SNC-RZ30P network camera, a Vivotek PZ6112 network camera, an Axis 241Q video web server, a Pentium 4 computer, a Philips SPC 900NC web camera and a GN Netcom 9120 wireless headset (Fig.3.1).

a) Sony SNC RZ30P b) Vivotek PZ6112 c) Axis 241Q

d) Philips SPC 900NC e) GN Netcom 9120

Fig.3.1: Laboratory audio/video equipment b) Attendant’s audio/video equipment The equipment that is required on the attendant’s side depends on the alternative that is chosen for the implementation of their participation. If all the attendants are to be gathered in the same room then the core of the required equipment can be summarized to one network camera with voice over IP capability, a computer, a projector with resolution of at least 1024x800 pixels, a microphone and one of more speakers. If the students are to participate at the course from their own place, then they should have a computer with a resolution of at least 1024x800 pixels, a microphone, speakers and a web camera. The web camera, speakers and microphone on the attendant’s side are not necessary for the conduction of the course in this alternative. Other means of communicating are provided to the attendants in this solution. But the existence of this equipment adds to the easiness of communication between the instructor and the attendant, and emulates in a more realistic way the real life course. In the course described in this paper, for the first alternative the following equipment is used: a Vivotek PZ6112 network camera, a simple Pentium 3 computer, an Optoma 739H LCD projector, a ClearOne Accumic PC microphone, and speakers (Fig.3.2). For the second alternative a laptop with a build in microphone was used, a pair of earphones and a Philips SPC 900NC web camera. The implementation of this alternative is not depended on the types of the equipment that is used, as long as there is an internet connection and a required screen resolution.

Fig.3.1: Attendant’s audio/video equipment, Optoma 739H LCD projector (left),

a ClearOne Accumic PC microphone (right) c) Laboratory course equipment There are three courses that are developed for this project from three different universities. The first course is on Robotics from the University of Nottingham, the second is on Robotic Laser Welding from the University of Twente and the third is on Metallurgic Microscopy and

is provided by Budapest University. In this paper only one course will be presented and it involves the training of robotic seam teaching and laser welding. During this course the participants will have the ability to control an industrial robot and use special sensors to teach a seam and weld it with the use of laser radiation. For this purpose the robot cell of the University of Twente is used in combination with Staubli RX130 robot. For the purposes of the course also seam detection sensors will be used. d) Software requirements The software involves the websites and applications of the equipment that is being used as well as internet applications that help at the transmission of video and audio and the sharing of applications. Such an internet application is Breese [1], which is developed with Macromedia and is provided to this implementation by SurfNET [2]. Breese is a virtual room where participants can view and hear each other, exchange chat messages, share a virtual white board, view presentations and share applications that run on their own PCs. Additionally to Breeze extra software is required for the handling of the robot and the control of the seam detection sensor. Two applications have been developed within the Applied Laser group of the University of Twente. The first is 24-Laser which allows the control of the robot movement and the use of the sensor data to create a path. Furthermore, 24-Laser allows the implementation of laser jobs, where a taught path is used for laser welding. The second application is IntegLas, which is the application that is developed to monitor the functionality of the seam detection sensor and the laser welding head. 4) Connections and signals The next issue is the connections between the equipment and what signals are required to be transferred among the attendants. As mentioned, there are two different implementations on the attendant side of connection. The optimum solution is a common interface on the instructor’s side where the different attendant modules can be connected. The objective is to provide the attendants with as much as possible freedom in the virtual environment, that will allow the to act similarly as if they were present to the real laboratory. For this purpose, the attendants should be able to look around in that laboratory and have immediate voice communication with the instructor (according to the rules that the instructor has set). In Fig.4.1 the implementation of the instructor’s module is presented. The instructor’s computer is running the Breeze, 24-Laser and IntegLas applications. On this computer, the webcam and the headset are connected which, depending on the attendant’s module, can be used by Breeze to transfer the voice and the image of the instructor to the attendants. The instructor’s computer is connected through local network to the industrial robot and the required sensors. The sharing function of Breeze allows the instructor to give the attendants access to the 24-Laser and IntegLas applications.

Fig.4.1: Connection diagram among the laboratory equipment

Furthermore there are two network cameras in and out of the robot cell. And they both provide their video signals to the web video server. The seam detection sensor also provides the video server with the sensor’s view. One more video channel is provided to the web video server by the instructor’s computer. This view is displaying the instructor’s monitor, which is useful during his instruction on the use of the software. The web video server provides a website where all these four views are presented simultaneously (Fig.4.2). The two network cameras are also connected to the internet, and have a interface where the students can change the view of the cameras (pan, tilt, zoom) and can look around the laboratory and the robot cell.

Fig.4.2: Web video server view

On the attendant connection module there are two alternatives. One is that the attendants are connected from their own place. With this alternative connection scheme is shown in Fig.4.3. The attendants have a computer with monitor a fast internet connection, speakers and optionally a microphone and webcam. The microphone is optional since Breeze gives the attendants the capability to post typed questions. The webcam is only required if the instructor wants to view the attendants. But speakers are required since voice is the fastest way for the instructor to communicate freely.

Fig.4.3: Connection diagram of individual student module

The second alternative is the one where all attendants are gathered in one room. The connection scheme of this alternative is presented in Fig.4.4. This alternative uses the lecture room computer mostly for the visualization part of the connection. The lecture room computer is connected with a LCD projector and the internet, and through it the student can have access to the websites of the laboratory cameras and web video server. For this implementation, the use of the application Breeze is optional, since it is not practical to gather the attendants around one computer in order to give access to the lab equipment. Without Breeze or any other sharing application the student can not have control over the equipment in the laboratory. Finally, there is a network camera with voice over IP present in the lecture room. This camera allows the instructor to focus on the attendant that communicates with him. A microphone and speakers allow the transfer of voice from both sides.

Fig.4.4: Connection diagram of lecture room module

5) Implementation In this section the implementation of the laboratory and lecture room will be analyzed. The individual attendant implementation depends on the point from where the user is connected and on his/hers personal preferences.

a) Laboratory implementation A top view of the laboratory robot cell configuration is displayed in Fig.5.1. One of the network cameras is placed inside the robot cell. We chose to use the Sony camera inside the robot cell. It has a good zooming lens which is required to view the welded work piece in detail. The position of the Sony allows the attendant to observe almost every part of the robot sell. A second camera is placed outside the robot cell, which forms an angle of about 90o degrees with the Sony camera in relation to the position of the robot. For this purpose we used the Vivotek which suffices for the positioning purposes of the robot. Furthermore, a webcam was placed on the monitor of the instructor and a wireless headset to allow the instructor to hear and talk with the attendants. The position of the web video server is not of importance as long as the four video channels are connected to it.

Fig.5.1: Actual laboratory implementation scheme

b) Lecture room implementation A view of the lecture room configuration is displayed in Fig.5.2. A LCD projector is used to display the computer screen on a whiteboard. The network camera is placed in a position that allows the instructor to view any of the attendants. The speakers and the microphone are connected to the network camera, when it is used for transferring of the audio signals. If Breeze is used for that audio, then the microphone and the speakers have to be connected to the lecture room’s computer. A Vivotek network camera is being used for the lecture room as it provides that ability to transmit and receive audio over the internet, and its pan/tilt/zoom capabilities are sufficient for this purpose.

Robot Controller Robot Work

Envelope

Robot Work Table

ROBOT

Sony Camera

Teacher Computer

Vivotek Camera

Detection Sensor

Fig.5.2: Actual lecture room implementation scheme

For the software implementation of this scheme, the lecture room computer can be connected to Breeze if application sharing is required for the course. The attendants can only use software that is shared by the instructor. 6) Results There are several courses that are developed within this project (University of Nottingham, University of Twente, Budapest University). Each course has been implemented in a different laboratory and on different laboratory equipment. Each of the courses is using a similar communication scheme. The results that are presented in this paper were produced in experiments that were conducted mainly between the University of Twente (Netherlands) and the University of Luleå (Sweden). During these experiments the University of Twente implemented the course (instructor) and the University of Luleå was the student. The experiments on individual students were conducted within the University of Twente. No extra care was taken on the reliability of the network, or the time of connection. The lectures took place with the available Ethernet networks at the institutes. The University of Twente has a 2Gbit local network with external connection of about 400Mbit. The University of Luleå has … The course from Twente U received by Luleå TU is based on teaching of robot manipulation, in particular seam tracking for a robotic laser welding process. By computer sharing the students in Luleå get access to the robot control as well as to the seam tracking control of Twente U. Safety aspects are carefully considered and emphasized. The teacher at Twente U introduces to the hardware and software and teaches the programming and use of the equipment. Each student can by distance program the robot and seam tracking and at the same time observe corresponding robot motions. At the end of the training some test welds will be performed, observed and analysed by looking at the resulting weld. This type of course not only succeeded in replacing an on-site course in the lab with this expensive laser-robot-sensor equipment but even opened a new way of learning, by simultaneously looking at different camera pictures/screens, in the group. Teaching can become more efficient and improved in terms of pedagogical aspects. Although by distance, a clear learning impact could be achieved.

LCD Projector

Lecture Room Computer

Vivotek Camera

Microphone

WHITEBOARD

Speakers

Attendants

Connectivity experiments were performed from various parts over the world also with the use of wireless networks. Connections from environments with wide bandwidth (Paris University, Budapest University) allow good audio signals and a satisfactory frame rate from the network cameras and web video server (minimum 16 frames/sec). Such environments are ideal for the performance of such a course. The wireless connectivity experiment was performed from hot-spot in Miami. Even though the signal was not very strong, connection with the laboratory equipment was achieved. The voice signal was reasonably clear, but the frame rate of the network cameras and web video server was slow (1 frame/sec). Moreover, the control commands towards that equipment also had such a delay. It is obvious that such networks should be avoided for a laboratory course. The delays caused to the transfer of data will slow down the course even for the attendants that are connected though fast networks. 7) Conclusions The use of of-the-self product has allowed the implementation of an internet laboratory course. This course has been conducted with the existing networks of the participating partners. Nevertheless, it is suggested that high bandwidth networks should be used especially when the individual attendant scheme is being used. The equipment that is used allows the attendants to perform most of the actions that they would if they were in the laboratory. It could be possible to connect directly the robot and the sensor to the network. This option is on hold until a safety procedure can be defined that will allow the instructor to oversee the actions of the attendants. Acknowledgements This project has been financed by the EU Leonardo Da Vinci program and has been carried out with the participation of various educational institutes (Luleå University of Technology (Sweden), Université René Descartes-Paris (France), Budapest University of Technology and Economics (Hungary), The University of Nottingham (UK), University of Twente (The Netherlands)) and industrial partners (IVF (Sweden), NedClad (The Netherlands), Ericsson Microelectronics AB (Sweden), Volvo Car - Car Body Components (Sweden), Laser Expertise Ltd (UK)). References 1 – Breeze, http://www.adobe.com/products/breeze/ 2 – SurfNET, http://www.surfnet.nl/info/home.jsp 3 – CyberLab Breeze group