Machine Vision Applications Within A Manufacturing ... · PDF fileMachine Vision Applications within a Manufacturing Engineering ... with pneumatics, hydraulic and ... Within A Manufacturing

Proceedings of the 2002 American Society for Engineering Education Annual Conference and Exposition Copyright Ó 2002, American Society for Engineering Education

Machine Vision Applications within a Manufacturing Engineering

Technology Program.

Andrew W. Otieno and Clifford R. Mirman

Department of Technology, Northern Illinois University,

Dekalb, IL 60115, USA Email: [email protected] or [email protected]

Abstract

The implementation and usage of industrial automation is undergoing major and rapid changes. This change is driven by the need for industry both remain competitive in their cost structure and to increase the levels of quality and consistency in the products that are produced. Today, companies can implement automation at a reasonable price, through advances in sensor technology, networking capabilities, microprocessor design, open architecture for machine controls, Internet applications in machine control, and standardized software. During the past few years, the acquisition cost of vision inspection systems has dropped to levels that permit most companies to purchase and implement the systems. In addition, with the advent of faster computer processors, the vision system is software controlled, and thus the applications are increased. Vision systems provide means by which continuous and total autonomous inspection can be achieved during production. Today's vision systems can easily control guidance of automated manufacturing support components such as robots, as well as interface to sensors and output to auxiliary devices.

The Department of Technology at Northern Illinois University has recognized these

needs and challenges and has responded by strengthening its curriculum and adding new relevant areas in its automation courses such as machine vision. Within our automation course, basics principles of vision are covered, including camera systems, basic optics, lighting, and image capturing and processing. A key component in this section of the automation course is the hands on experience in which student teams use and apply the vision systems components and software in an automation cell. In addition, the students are taught the principles of vision integration with other control devices, such as PLC’s and robotics. From this level of automation instruction, the department has generated much interest in the students, as well as much industrial collaboration with companies in the Northern Illinois region.

Introduction

In order to prepare the next generation of manufacturing engineers to perform in the fast changing industrial environment, educators must provide training that mirrors this environment that students seek to enter. This means that future manufacturing engineers must be comfortable

Session number: 3548

Page 7.824.1


in a diverse range of manufacturing contexts, and must be well versed in latest developments in manufacturing technology, especially integrated product development and automation. The key to manufacturing productivity is maintaining high levels of automation cost effectively. By far, machine vision has continued to play a major role in the integration of automated manufacturing, both from a quality and inspection perspective to more advanced applications such as motion control and robot guidance. Indeed, both researchers and engineers in industry concur about the importance of machine vision in manufacturing.1,2 In reality, however, most college manufacturing courses still tend to be limited to traditional techniques of inspection with few applications of machine vision. Students also rarely get a chance to gain the necessary experience of the entire process of vision applications and integration into an automation environment. The Department of Technology at Northern Illinois University (NIU), having realized the need to integrate such areas into the curriculum, has embarked on major curricular reforms. One of the main goals of this curricular improvement is therefore to incorporate some important areas of automation such as machine vision within our Manufacturing Engineering Technology (MET) curriculum.

Over the past two years, we have been restructuring our curriculum to meet challenges posed by a fast changing technological world of manufacturing. Our curricular restructuring efforts have been geared mainly towards maintaining technological currency, in addition to strengthening our hands on experiences for students, as required by ABET3, NAIT4 and other accreditation bodies. As observed by the SME, “…Students with a solid grounding in science and math, strong hands-on project experience and teamwork skills make the best manufacturing engineers.”* Our curricular reform efforts are also geared towards renewing the Department’s NAIT accreditation this year for it Industrial Technology programs, and also to seek ABET accreditation for its Engineering Technology programs in the next two years. Through partnership with industry and our advisory boards, several areas were identified for improvement or incorporation into the curriculum. Within the automation course, one major area that was identified as either lacking or weak was the applications of machine vision in manufacturing. As a result this structural reform sought to examine this area critically and incorporate it into the curriculum. In addition, this restructuring has led to major laboratory developments and to strengthen the vision area, several state of the art vision systems were acquired through partnership with DVTSensorsä.

Another major motivation for the curriculum reform activities has been the regional

impetus. NIU is strategically located between two major industrial metropolitan regions of Northern Illinois, namely Chicago and Rockford, and for several years, has played a major role in providing workforce for this diverse region. Currently, employment projections for the state of Illinois, and this region look fairly attractive. According to the Department of Employment Security, the state would require nearly 7 million new workers by the year 20085. The majority of these will go into manufacturing. The projected percentage increase in jobs in the manufacturing sector for the Chicago and Rockford areas by the year 2010 will have increased by 2.18% and 0.75% respectively, compared to a state average of 1.8% and a national average of 3.1%6. To meet these manufacturing demands, institutions at all levels must respond by preparing highly qualified and competitive professionals. In addition, NIU has recently been admitted to NASULGC (National Association of State Universities and Land Grant Colleges), * SME Manufacturing Education Plan goals; http://www.sme.org/cgi-bin/eduhtml.pl?/mep/intro.htm&&&SME&

Page 7.824.2


and with this status an increased volume of regional outreach has been earmarked as the next logical step in the growth of the university§. The Department of Technology at NIU is preparing to meet this regional manufacturing related challenge by redesigning the MET curricula and improving laboratories. The department is already seeing increases in student enrollment, and by stressing "quality" of educational experiences, we hope to attract even more potential students. Some of the major companies that actively recruit our graduates include Ingersoll Cutting tools (Rockford), RR Floody Company (Rockford), Motorola (Rockford, Aurora), Sunstrand Corp. (Rockford), Chrysler (Rockford), Siemens (Chicago), Nissan Forklift (Marengo), Caterpillar (Aurora, Peoria) and Underwriters Laboratories (Northbrook), just to name a few. In this paper, we shall describe how we have incorporated this important aspect of automation into our Manufacturing Engineering Technology (MET) curriculum.

Overview of the Curriculum and Current state of Laboratory facilities.

The MET program is a comprehensive inclusion of manufacturing techniques comprising of “old” and “new” technology components. The older technology includes components in metal forming and mechanics, while the new technology includes robotics, PLC’s, vision applications and NC/CNC. A full description of how these courses are taken by students has been discussed elsewhere7. Very significant changes have taken place in the structuring of the courses especially in the “new” technology areas. For instance, it was determined, through discussions with graduates, employers, and the MET advisory board, that the Programmable Logic Controllers (PLC) curriculum must be altered8. The newly designed PLC course now includes ladder-logic programming, sensor interface, component selection (voltages, current, and compatibility), component manufacturer literature search, pneumatics and other motion control systems, and user interfaces. The same approach was used for the Manufacturing Automation course, in which vision systems is taught, and it now includes the following:

Ø Sensors and actuators Ø Computer based and PLC based controls Ø Motion control with pneumatics, hydraulic and electrical drives Ø Robotics Ø Vision systems Ø Automatic identification and tracking Ø Systems integration

These components were determined based upon the needs of industry, upper level classes,

and the senior project course. This course is taught in the Spring semester to approximately 20 MET students. Other students may enroll if they meet the pre-requisites, which currently include a programming language such as Visual Basic or C, NC machines, and PLC’s. The automation course is taught at the senior level, and is designed to synthesize theory and operation with an integrated laboratory component. Students meet together for the lecture/discussion portion for 2 hours a week. The laboratory component is carried out in groups, usually of 2 to 4 students and each lasts between 2-3 hours. It is typical that extra laboratory time is arranged during lab open hours. The labs are divided into six sections as follows: Pneumatics and hydraulic motion

§ Inaugural State of the University Address by Dr. John G. Peters, Eleventh President of Northern Illinois University, October 12, 2000; http://www.niu.edu/president/sofu00.html

Page 7.824.3


systems; Sensors, Actuators and PLC’s; Robotics; Automatic Data Capture and Identification; Vision systems; and Integration project covering all the above.

In order to accomplish this level of practical experience the College and the department have recently invested in a rigorous laboratory development program that led to the acquisition of some significant state of the art equipment and facilities. The laboratory currently has installed five modular training units shown in figure 1. Each of this comprises of a mini-conveyor, a DVT vision system, a 14 point GE Fanuc VersaMax Micro PLC, a pneumatic air supply system, and a PC. The lab is also equipped with a carousel Gilman link conveyor to which are installed two 4-axis DENSO robots and one 6-axis DENSO robot shown in figure 2. This is mainly used for instruction in robot programming and also for senior design projects. There various sensors and actuators also utilized alongside these facilities. The lab also features a Z-Corp rapid prototyping machine and 14 networked PC’s that are also used for instruction mainly in AutoCAD, SurfCam and Visual Basic.

Figure 1. Modular Conveyor Unit with PLC and Control Board, and Pneumatic Actuator.

Figure 2: Conveyor loop with DVT camera and Denso 6-axis robot.

Page 7.824.4


Machine Vision in the Automation Course

At the start, students learn basics of machine vision, beginning with an exploratory discussion on image acquisition and digitization. The concept of dividing a viewing area into small pixels is introduced, followed by techniques of video capture and image storage for subsequent analysis. The students learn about how each pixel has a value that is proportional to the light intensity at that portion of the scene. They are also taught how the intensity of each pixel is converted into its equivalent digital value by an A/D converter, that is, the whole concept of a frame grabber. The students are then taught the differences between expressing pixel intensities in binary vision, gray-scale levels and color. In this section the importance of lighting to obtain the best results is also emphasized through varying the light sources and directions.

In the next part the students then learn some basics about image processing algorithms.

First they learn about filtering techniques such as thresholding, as applied to simple algorithms such as edge detection where the edges are regarded as a boundary between two dissimilar regions in an image. The students are shown how an already filtered image can be checked to find a sharp edge between the foreground and background intensities. The edge detection algorithm looks for points which have a sharp gradients (or discontinuities in gradients) in pixel values and links them together as a line or curve. Other algorithms such as segmentation and form fitting are introduced. Since the ultimate goal of many vision systems is to be able to recognize objects and extract features the students are shown how these algorithms are used to recognize different geometrical features such as circles, squares, e.t.c. The students also learn how features can be related to area or length or even aspect ratio, and how these can be measured in pixels, for purposes of inspection. These are also later used in measurement tools of the vision system. Pattern recognition is also discussed at length. The students learn how to obtain a template match and use it for inspection purposes. In the basic template match, the image is compared pixel by pixel with a known model.

The classroom discussion is complemented by five lab activities. In the first four,

students apply techniques for specific machine vision tasks, and the final lab involves integration. In the first lab exercise the students learn the basics of the series 630 DVT cameras that are installed in the lab, the camera accessories, the camera software (Frameworkä) and the different modes of communication with the camera. Students also learn to set up and carry out a simple inspection using edge count soft sensors to inspect tabs on a widget. In the next lab activity, the students use a feature count soft sensor to inspect the same widget, where, the students must not only be able to distinguish between a dark and a bright feature, but also sizes, such that each required feature in an inspection can be recognized. The application of an edge and feature count is shown in figure 3, where, the round sensor is the edge count tool. Figure 3 shows the good widget, where all of the tabs are present and their widths meet requirements. In the third lab exercise, the students apply their knowledge of pixel intensities to detect the presence of a flaw. Basically this sensor is configured to identify a feature in terms of the total amount of pixels meeting a certain amount of minimum or maximum intensity (i.e. the percentage of bright or dark area). For instance in the widget shown in figure 4 below, the circle in the center is used to emulate a flaw and since there is about 5% area with pixels whose intensity is greater than the threshold (bright pixels), the soft sensor will recognize the flaw. The final lab focuses on using vision for measurement inspection. Here the students learn how to set

Page 7.824.5


up a simple measurement tool, how to obtain the measurement of a part in pixels and how to set up a scaling factor so as to get a direct measurement in the required units. An example of this is shown in figure 5, where the radius of a circular part is being inspected.

The integration portion of the course involves more advanced applications. The students learn how to use other tools such as template matches for inspection, and the use of the camera for reading barcodes and data matrices. In these applications the students typically set up different sensors for the applications, basically in a way similar to that of diameter measurement or pixel counter in the intensity tool. But one of the most important things that the students learn in this section is how to integrate the results of the inspections with other areas of automation. They are first introduces to the basics of the camera’s digital input and output (I/O) setup and how to configure them. For instance, in the application illustrated in figure 1 involves using a sensor to trigger inspection, and using inspection results as inputs to PLC. The PLC controls the opening and closing of a solenoid valve so that bad parts can be rejected by the action of extracting a pneumatic actuator across the conveyor, and good parts are left to travel to the end of the conveyor. In this application, they not only learn how to wire the sensors to the camera control board and also how to wire the outputs from the camera board to the PLC, but they also familiarize themselves with toggling through the camera I/O setup and entering the correct parameters. Figure 6 illustrates the drop down menu to setup and configure digital I/O’s of the camera. This section is treated as a mini-project where the students must set-up and implement the simple parts-sorting system on the conveyor. PLC ladder programming is required for the pneumatic actuators. In a more advanced application, the inspection results are used to activate pneumatic cylinders through PLC control for part-stop, and a robotic arm to sort the parts onto a bin depending on whether they are good or bad.

Figure 3. Application of Edge and Feature count tools for vision inspection.

Page 7.824.6


Figure 4. Intensity sensor for detecting a flaw

Figure 5. Measurement tool being used to measure radius of circular part.

Page 7.824.7


Figure 6. Digital I/O Parameter set-up tabs for the vision system. Conclusion

Exposing students to automation techniques that are currently used in industry in an effective way of preparing students for their future careers. This not only reduces the amount of time invested in training by companies that employ them but it also contributes significantly to productivity. NIU’s Technology Department has just updated the Automation courses with

Page 7.824.8


emphasis on introducing some of the current techniques such as vision systems. The Department has realized an increase in interest from the students and also from industry. For instance, our automation laboratory has hosted DVTSensors’ Advanced Vision systems training seminar that was attended by participants from Illinois, Wisconsin, Iowa, Missouri and Georgia. We are working closely with DVT to make this seminar an annual event. Apart from this, the department has continued to maintain active partnerships with other companies in the DeKalb area such as Eaton Inc., and General Electric through internships and senior design projects. Some of these senior design projects have culminated into the development of automated vision guided materials handling system. As we continue to restructure our curriculum we plan to widen the scope to include more advance applications with vision systems such as robot guidance. The inclusion of vision systems in the curriculum not only benefits students in the automation course but also the knowledge learned in this course is used in other courses such as senior design projects. We hope that our efforts to restructure the curriculum will continue to produce high quality MET students who are sought after by the regional industry base.

Bibliography 1. Dechow, D., “Key issues in integrating machine vision”, Robotics World, 15(2), 1997, p 34-39. 2. Zuech, N., “Are You Ready for Machine Vision? Is Machine Vision Ready for You? Tools Available for

Someone Buying a Machine Vision System”, Society of Manufacturing Engineers, Technical Paper No. MS95-100, 1994.

3. Accreditation Policy and Procedure Manual, http://www.abet.org/images/2001-02APPM.pdf, Accreditation Board for Engineering and Technology, December 15, 2001.

4. Industrial Technology Accreditation Handbook – 2000, http://www.nait.org/accred/accreditationhandbook.html, National Association of Industrial Technology, December 15, 2001.

5. Illinois Department of Employment Security, http://lmi.ides.state.il.us, December 20, 2001. 6. Bureau of Labor Statistics, US Department of Labor, http://stats.bls.gov/emp/home.htm, December 20, 2001. 7. Balamuralikrishna, R., Mirman, C. and Otieno, A., “Redesigning a Manufacturing Engineering Technology

Program: Standards, Challenges and Opportunities” Proc. ASEE IL-IN Sectional Conference, Purdue University, IN, March 2001, pp. 53-57.

8. Otieno, A., Urbanowicz, K. and Mirman, C., “A Laboratory Based Programmable Logic Controller (PLC) Course for a Manufacturing Curriculum”, to be presented at the 2002 ASEE IL-IN Sectional Conference, Illinois Institute of Technology, Chicago, IL, April 11-12, 2002.

Biographical Information Dr. ANDREW W. OTIENO received his Ph.D. from Leeds University, UK in 1994 and has been at NIU since August 2000. He has been actively involved in the restructuring of the MET curriculum and development of the Automation laboratory. His research is in the area of finite element modeling, machining processes and structural health monitoring. He has experience in hardware/software interfacing with special applications in machine vision. Dr. CLIFFORD R. MIRMAN received his Ph.D. degree from the University of Illinois at Chicago in 1991. From 1991 until 1999, he was at Wilkes University’s Mechanical Engineering Department. He is currently the Chair of the Department of Technology at NIU. His research areas are CAD, Finite-Element-Analysis, and kinematics, both securing grants and writing publications. Dr. Mirman is actively involved in ASEE and SME. P

age 7.824.9

Documents

Machine Vision Applications Within A Manufacturing ... · PDF fileMachine Vision Applications within a Manufacturing Engineering ... with pneumatics, hydraulic and ... Within A Manufacturing