Vision-Guided Robot Position Control SKYNET Tony BaumgartnerBrock Shepard Jeff Clements Norm Pond Nicholas Vidovich Advisors: Dr. Juliet Hurtig & Dr. J.D. Yoder November 12, 2003 Problem Identification Goal: Develop a vision guided robot positioning control system Current systems are limited to Teach- Repeat style On completion the robot will automatically perform tasks specified by the software Budget/Purchasing A budget of approximately $20,000 has been provided by Ohio Northern University Robot Gripper Desktop computer Two CCD (Charge-Coupled Device ) cameras Background Information Articulated SCARA Selectively Compliant Assembly Robot Arm Vision Systems Eye-On-Hand Configuration External Stereo Camera Configuration Prototype Final prototype will include: CCD camera(s) Desktop computer Robot Control Unit Robot Gripper Specifications Robots are measured according to two specifications: Repeatability Absolute Accuracy Object Recognition Algorithms Normalized Cross-Correlation Slower Less Sensitive To Light Shape-Based Matching Faster Similarity Measure The Sum of Absolute Differences Difference Algorithm Visuals Design Deliverables PurchasingTarget Date Robot09/12/ /21/03 Gripper09/12/ /21/03 Image Processing Camera pan/tilt/zoom09/30/ /20/04 CAD11/03/ /09/04 Difference Algorithm11/04/ /19/03 Object Recognition01/09/ /02/04 User Interface02/02/ /20/04 General Controller Integration12/01/ /02/04 Gripper Implementation12/01/ /02/04 Testing03/08/ /01/04 Overall Block Diagram ControllerCPURobot Gripper Cameras - Pan - Tilt - Zoom Image Processing C++/C# UI 3D Model - Laser - Virtual CAD Algorithm Software Block Diagram Capture Initial Capture Current Compare Using Difference Read y? Identify Object / Find Coordinates Send Instructions Controller Commands Robot Check Actions Done ? Robot Decision Matrix 0 Unacceptable 1 Acceptable 2 Average 3 Good 4 Excellent 5 Best Case QUESTIONS ? References ABB Product Specification Sheet (2003) Lin, C.T., Tsai D.M. (2002) Fast normalized cross correlation for defect detection. Machine Vision. Yuan-Ze University,1-5. Phil Baratti Robot Precision (personal communication, November ) Stegar, C.., Similarity measures for occlusion, clutter, and illumination invariant object recognition. In: B. Radig and S. Florczyk(eds), Mustererkennung 2001, Springer Munchen, pp Stegar, C., Ulrich, M Performance Evaluation of 2D Object Recognition Techniques. Technical Report: Technische Universitat Munchen, Robots and Manufacturing Automation, pg Software Flow Description Take Initial Picture: Initialize cameras Capture image Store image to hard disk or in RAM (will depend on speed) Take Current Picture: Start current image capture loop Capture image and store in memory Software Flow Description Compare Current Picture With Initial : Use difference algorithm to compare initial image and current image this yields a bitmap of the pixels that have changed based on a specified error factor Store difference image in memory Software Flow Description Ready to Manipulate Object? : Check difference image to see that an object has entered the area using Boolean to keep track If object is already present check to see if it has stopped moving If done moving, act on object, else return to Take Current Picture Block Software Flow Description Identify and Find Object Coordinates : Take pictures of scene using both cameras focused on object Compare the images received from the cameras using the Stereo Vision Algorithm to determine object type and relevant points relative to the position of the robot arm Send Coordinates & Instructions to Robot Controller : Communicate with robot controller sending incremental instructions based on object type and coordinates Software Flow Description Use Stereo Vision Algorithm to make sure desired actions : Take pictures of scene using both cameras focused on object Compare the images received from the cameras using the Stereo Vision Algorithm to determine whether the robot arm has moved correctly. After adjusting to mistakes by the robot, send more coordinates and instructions to robot controller until it has performed the correct task