Piedmont Automation, Inc. · Web viewe vision still guides us forward, but above all else, a friend

Robotics and Vision: Forward for Piedmont Automation Inc.

John C. Glisson

Piedmont Automation Inc.

Piedmont Automation, Inc. V1.0

Dedicated to Christopher Hugh O’Reilly

A man whose vision still guides us forward, but above all else, a friend.

1


Table of ContentsVersion History............................................................................................................................................3

Introduction.................................................................................................................................................4

Background..................................................................................................................................................5

Overview.................................................................................................................................................5

B&R Automation......................................................................................................................................5

Integrated Development Environment- Automation Studio................................................................6

Servomotor Control.............................................................................................................................6

PLC based machine controller.............................................................................................................6

MAPP...................................................................................................................................................6

Safety...................................................................................................................................................8

Machine Vision........................................................................................................................................9

Image Capture.....................................................................................................................................9

Image Processing.................................................................................................................................9

Intel VT-X...............................................................................................................................................10

Computer Numerical Control (CNC)......................................................................................................10

Solution.....................................................................................................................................................11

Robot.....................................................................................................................................................11

Real-Time Controls................................................................................................................................11

Image Capture & Processing..................................................................................................................13

Image Capture...................................................................................................................................13

Point Grey Research..........................................................................................................................14

Conclusion.................................................................................................................................................15

Controls.................................................................................................................................................15

Mechatronics.........................................................................................................................................16

Vision.....................................................................................................................................................16

Overall...................................................................................................................................................16

Other Points-.............................................................................................................................................17

Citations....................................................................................................................................................17

2


3


Version HistoryVersion Author Date Notes

V1.0 John Glisson 24. January 2016 First Release

4


IntroductionPiedmont Automation, Inc. (PAI) represents the Austrian industrial supplies company

Bernecker & Rainer (B&R) in the states of Georgia and Florida. PAI currently supplies hundreds of companies ranging from Original Equipment Manufacturers to machine retrofitters with the hardware and software required for motion control, Human Machine Interfaces (HMI), materials processing and separation, payroll calculation, and process control, though our main source of revenue is hardware sales. Through investments of the combined resources of PAI, B&R, and other associated entities, a decent amount of capital has been expended on solutions that PAI can offer its customers, specifically in the fields of Computer Vision, Industrial Robotics, and Computer Numerical Control (CNC). More so than just being able to offer customers individual Programmable Logic Controllers (PLCs, servo-motors and their associated drive implements, and machine-vision cameras; PAI would like to be able to offer an entire integrated mechatronics solution. This requires bridging complicated and specialized fields of Information Technology, Hard Real-Time Operating Systems, and motion control, as well as other cutting-edge industrial technologies. Fortunately, certain packages made available by B&R as well as open-source initiatives undertaken by IBM and Intel have forged a clear path to making this initiative become a reality. The purpose of this paper is to document these technologies paying specific attention to how they can communicate with one another in an industrial setting as well as document the costs and feasibility of the solution.

5


Background

Overview The requirements for a robotics solution- be it supplying a single part of an immensely

complex system to an entire turnkey factory installation- vary widely. The immense complexity of the entirety of a single industrial operation is beyond the volume of understanding that one person is capable of, and as such, the scope of this document is confined to the following-

1. Hardware Implementsa. CPU-based machine controlsb. Servomotors, servodrives, and feedback sensorsc. Mechatronicsd. Vision systeme. Cabling

2. Softwarea. Real-time OSb. Peripheral firmwarec. Machine vision processingd. System supervision and data collection

3. Networkinga. Real-time network communicationb. Pseudo-real-time image processing communicationc. Human-Machine Interfacing

B&R AutomationRobotics combines the fields of computer science, mathematics, mechanics and mechanical

engineering, and electrical engineering; and as such, designing and commissioning a robotics system is a daunting endeavor. The most effective approach to lowering the knowledge barrier required to work with these systems is “software abstraction”- the use of confined software containers with clean interfaces where each software object included on the PLC represents a physical object on the machine. This method has the combined advantage of handling all tasks related to the hardware as well as keeping the code organized in a transparent and portable fashion.

Founded in 1979 and reporting Euro 535 Million in sales (2014), B&R offers industrial PC’s, Human-Machine interfaces, Network-based IO, ISO- and UL-conforming safety systems, and motion control; as well as the software required to develop, commission, and maintain their products. As their partner and exclusive distributor representing Georgia and Florida, Piedmont Automation Inc. has forged a relationship that puts us in a position to take advantage of evolutions in the industrial automation market.

6


At its advent, industrial motion control applications like CNC machining, robotics, and coordinated multi-axis motion control were applications that required advanced engineering degrees to utilize effectively; and off-the-shelf solutions required dedicated controllers that had limited functionality. B&R has significantly reduced the expertise required to design and commission a multi-axis motion control system by abstracting physical entities (servomotors and drives) into software entities (axes) and further into complete robots (axis systems). This suite of controls software, integrated as libraries into B&R’s development environment Automation Studio, is called MAPP Motion.

Integrated Development Environment- Automation StudioB&R’s Integrated Development Environment (IDE) Automation Studio is a one-stop Windows-

based solution for designing hardware systems, developing controls software, managing different software deployments, cataloging B&R’s hardware offerings, managing engineering calculations such as motor sizing or network speed requirements, Connecting to deployment targets for initial machine commissioning and iterative development changes, documenting network topologies, amongst other functionality.

Servomotor ControlThe basis of any robotics operations at its core is precise, accurate, and programmable motion

control. The means to which this end is accomplished is with the use of Servomotors and Servomotor drives. B&R has a comprehensive offering of servomotors and drives at a competitive price point. In addition, full integration into Automation Studio simplifies the task of commissioning a servomotor system by circumventing the need to manually enter motor parameters and automatic control loop gain parameter calculation, for example. B&R also offers accessory mechanical hardware such as gearboxes and cabling to ensure the solution meets all applicable quality and safety standards.

PLC based machine controllerB&R’s PLCs represent the state-of-the-art in digital electronics and networking. Though B&R

offers a wide assortment of controllers, the process of migrating from one platform to another is greatly simplified by the fact that every PLC runs the same real-time operating system, Automation Runtime, which is responsible for managing the interrupts required to ensure the hard real-time performance that is to be expected out of an enterprise-grade solution.

MAPPB&R offers the following templates for turnkey robotics control software-

1. 3-Axis (Cartesian) CNC2. 4-Axis (Cartesian + Rotation) CNC3. 2-, 3-, and 4-Axis Delta robot4. 4- and 6-Axis Serial-manipulator (Articulated Arm)5. 2- and 4- Axis SCARA6. Arbitrary (user-defined) kinematics

7


For example, an operation consisting of 3 distinct processes (Filling, labeling, and packaging of bottles) could set up controls in the following manner [ (B&R Automation)Figure 1.- Typical MAPP functionality Group. (B&R Automation)]

Figure 1.- Typical MAPP functionality Group. (B&R Automation)

Further, configuring the individual axes to work in tandem for robotic arm control requires only the following configurating [ (B&R Automation)Figure 2.- Sample MAPP robotics configuration (B&R Automation)]

8


Figure 2.- Sample MAPP robotics configuration (B&R Automation)

SafetyNo industrial system would dare to operate without paying the utmost attention to operator

and equipment safety. B&R’s position as a worldwide supplier of industrial automation and controls has continuously sharpened its revisions of its safety hardware and software with operation under every safety protocol imaginable. B&R has also revolutionized the concept of safety sensors by carrying safety data over the network bus, negating the need for complicated wiring of safety implements that can be prone to short-circuits and end-user safety bypasses. As any remotely marketable solution would inevitably involve a complex motion system operating in the immediate vicinity of people, developing not only a safe solution but a smart one as well will be required if our services are to be marketable.

Machine Vision

Image CaptureMachine vision involves the use of 1D (line scan cameras), 2D (Area scan cameras), and pseudo-

3D (stereoscopic vision) to inform a process controller about the current state of its working envelope. The solution should include not only the ability of the machine to identify, locate, and track the movement of pre-defined image templates, but also the ability for the end-user to configure and optimize the solution.

9


Machine vision hardware suppliers are as diverse as the applications they attempts to solve. This potentially complicates the software development aspect of the solution, if software must be rewritten in response to a change in camera vendor or technology. Fortunately, compliance standards such as GenICam and GigEVision abstract the physical camera as a class such that software does not need to be rewritten nor recompiled to compensate for a change in machine vision hardware. The camera as well as lensing, Iris control, and aperture hardware can change based on the application with no adverse effects on the software.

Image ProcessingOriginally an Intel Research initiative, OpenCV has become the industry standard in computer

vision software due to its platform-independence (portable C++ code) and open-source licensing. In addition to offering libraries capable of object identification, motion tracking, image segmentation & recognition, and structure-from-motion; OpenCV offers a comprehensive set of algorithms in artificial intelligence, machine learning, and algorithms capable of “teaching” a robot the correct actions to take. In addition, open-source Application Programming Interfaces (API) are included for standard camera communication protocols (GenICam, GigEVision, CameraLink, USB3Vision, IEEE1394, etc.). (Andy, 2011)

While open-source products in general are of questionable applicability in industrial- and enterprise-grade applications due to their lack of a dedicated support team and bleeding-edge nightly builds, OpenCV is an accepted industrial-grade solution [todo: citation]. OpenCV has 10+ years of revisions and testing since Intel let it loose. Open-source licensing, in addition to being free, also allows PAI and its customers full control over the entire software environment. While it has the drawback of only being programmable at a low-level C/C++ interface, PAI would be able to provide field support and upgrades.

A big impediment to offering a marketable machine vision platform is the requirement for the real-time networking and controls to operate harmoniously with the auxiliary services that do not exist on real-time platforms (machine vision, remote desktop, SQL- Data-layer IP based communications, etc.) While B&R and its competitors offer real-time critical task scheduling for mechatronics and safety-related programs, there currently does not exist a library for operation of a machine vision application that coexists alongside a hard-real-time operating system. This problem can be circumvented using a separate processing unit in what is considered “soft-real-time” operation where calculations are to be performed synchronously to the real-time application but failure to return in the specified time frame does not cause a thread exception. In this case, a “heartbeat” communication protocol would be used to ensure new data populates the registers between CPU’s, and in failure of the vision system to operate synchronously the robot would go into a service position until the communications and processing has been sufficiently restored. In practice, however, sufficient resources allocated to the vision system should make this a rarity, and intermittent interruptions could be ignored. This would preclude the vision platform from being used in certain fields of ISO compliance (e.g. SKU monitoring and safety-critical tasks) but would not be an issue in applications such as this, where the robot is tasked with image segmentation and motion tracking. [todo: citation of ISO compliance guidelines]

10


Intel VT-XThe entire solution would collapse without a platform to run on. In the past, software

developers were constrained to developing their software for a single hardware platform. Now, Intel VT-X allows the deployment of software in a portable virtual machine environment which can share CPU and memory resources on the host machine with minimal overhead associated with virtualization. This also allows a true RTOS like Automation Runtime to run on the same physical unit as a Windows or Linux deployment without sacrifice of true real-time operation capabilities. Further, Intel’s new VT-D technology extends VT-X to allow virtual guest operating systems to share and interrupt the internal buses, furthering the capabilities of a virtual machine within the host operating system. (Smith, 2005)

In the past, rapidly evolving hardware platforms and their associated operating systems required constant software updates to remain compatible. With the advent of Intel’s VT-X , software is no longer required to be deployed in a package compatible with the operating system on which it is run, as it is now possible to deploy multiple guest operating systems in one host operating system. Intel’s new VT-D technology in its fourth generation and higher Core Series processor line takes this one step further, by allowing guest OS’s to communicate directly with their associated peripherals, bypassing the overhead that has been traditionally required and negating compatibility concerns between the host OS and peripherals (specifically, offering real-time deterministic behavior on a Windows-based platform)

Computer Numerical Control (CNC)CNC as a technology refers to using a single program to coordinate movement between multiple

axes simultaneously, with the end result being the end effector (usually a cutting tool) moving along a controlled path, usually with the result being a part being cut out of a stock piece of material. Computer Aided Manufacturing (CAM) software provides these paths in the form of a standard syntax program called G-Code, which is generated on a development machine (generally Windows) and transferred to the real-time operating system target, which parses the program and turns it into coordinated machine setpoints. Veterans of industrial machining and control are generally familiar with the XYZ programming coordinates, where each axis movement corresponds directly to the absolute-referenced position of a single motor; but that is not always the case- for more complex robots geometries consisting of multiple sets of serial and parallel manipulated bodies, operations referred to as “axis transformations” are performed that transform the machine-agnostic path programs in the G-Code into motor setpoints in robot space. This way, the generation of movement paths is kept in a coordinate system easy for the user to understand and there is no need to rewrite or regenerate path programs when the robot hardware changes (“hardware-agnostic programming”).

11


Solution

RobotAn ABB Flexpicker IRB340 unit was procured to demonstrate operation of the system. 3x

Acopos 1045 series drives (potentially replaceable by the P3 drive as an even more cost effective solution) are used to drive the arms through a gearbox with a 30:1 reduction. The drives are controlled over the Ethernet-Powerlink (EPL) network by a B&R X20CP1584 Atom-based controller running B&R’s Automation Runtime operating system in real-time. The X20CP1584 interfaces to the other hardware controlling machine vision via standard Ethernet communication protocols, specifically ModBus TCP (machine vision) and UDP (database communication). [Figure 3- Real-Time Network Topology]

Figure 3- Real-Time Network Topology

Real-Time ControlsB&R’s function block MpDelta3Axis in the library MpDelta is used as the sole programming

interface for the robot to the rest of the application. The following code is responsible for initializing the robot’s operation [ (B&R Automation)Figure 4.- Example robot component initialization (B&R Automation)]

12


Figure 4.- Example robot component initialization (B&R Automation)

In addition, B&R offers a single interface for configuring the parameters specific to this delta robot [ (B&R Automation)Figure 5. Example robot configuration (B&R Automation)]

Figure 5. Example robot configuration (B&R Automation)

As can be surmised, the code responsible for the coordinate transformations between the joints (“joint space”) and the machine XYZ path (“machine space” or “Cartesian space”) has been “wrapped” into these function blocks. Though this can obscure certain functionalities of the operation from the end user of the software blocks, it dramatically simplifies operation and commissioning into ~20 lines of programming code required by the end user to operate the robot, compared to thousands on other platforms. This negates the need to write application-specific drivers between modules, which

13


significantly decreases the cost of deploying a system. The code required to run the delta robot is trivial, as it only requires a call to a single function block to manage the entire mechatronics system.

Image Capture & ProcessingGigEVision (GEV) represents the state-of-the-art framework in industrial camera communication

(Thryft, 2010). Based on the UDP protocol, GEV completely decouples the camera system from the networking protocol required to transmit the images in real time. As such, the solution would maintain the freedom to mix-and-match imaging hardware by different manufacturers without requiring a different software deployment of the computer vision application- a requirement that is very important to maintain a broad appeal of the overall robotics solution, as imaging hardware can vary drastically based on the requirements. GEV supports 125MB/s streaming rates, which translates into 75 frames per second at 1288x964 resolution [todo: citation of calculation] on a dedicated network.

Image CaptureTo capture an image for processing, the following must be considered (Machine Vision Lens

Calculator)-

1. Field of View size (Width and Height)2. Working distance (distance from lens to object)3. Depth of field (distance, near and far, from the lens where image can be resolved acceptably)4. Frequency of image capture for processing5. Amount of light & variability in lighting6. Sensor type (CCD vs. CMOS)7. Minimum resolvable feature or aspect ratio8. Triggering method (Digital Input pin, Software, clock-based)

From the previous, the following parameters must be determined-

1. Lens F-stop2. Sensor format3. Back Focal Distance4. Communication Interface5. ADC resolution (8-32 bit)

A typical machine vision application as well as the criteria for selecting the hardware [ (vision-doctor)Figure 6- llustration of standard industrial machine vision application (vision-doctor)]

14


Figure 6- llustration of standard industrial machine vision application (vision-doctor)

Point Grey ResearchPoint Grey Research Inc. out of BC, Canada represents a large manufacturer of industrial

machine vision systems conforming to the GenICam communications protocol. The following sources represent usage of Point Grey hardware in similar applications-

1. “Vision Automates Jar Inspection”, http://www.vision-systems.com/articles/print/volume-18/issue-7/features/vision-automates-jar-inspection.html

2. “Simplifying Automotive Assembly”, https://www.ptgrey.com/case-study/id/10397, http://www.vision-systems.com/articles/print/volume-15/issue-5/Features/Simplifying_Automotive_Assembly.html

3. “Vision System uses Flea2 Camera for Case Pack Inspection”, https://www.ptgrey.com/case-study/id/10387

4. “Granule and Powder Inspection with Flea3 GigE Cameras”, https://www.ptgrey.com/case-study/id/10419

5. “Complete off the shelf 3D system”, https://www.ptgrey.com/case-study/id/104106. “Quality Kegs”, https://www.ptgrey.com/case-study/id/10401

15

https://www.ptgrey.com/case-study/id/10401





http://www.vision-systems.com/articles/print/volume-18/issue-7/features/vision-automates-jar-inspection.html

http://www.vision-systems.com/articles/print/volume-18/issue-7/features/vision-automates-jar-inspection.html


7. “Sweet Success”, https://www.ptgrey.com/case-study/id/10400

For example, SKODA Automotive used the following schematic for development of their Yeti SUV quality control system alongside a Siemens S7 PLC [Figure 7. Example architecture of Industrial PLC, image acquisition, and machine vision processing]

Figure 7. Example architecture of Industrial PLC, image acquisition, and machine vision processing

The key similarity between this system and the general solution being proposed in this paper is that two distinct units are being used- one PLC and an industrial computer. While GigEVision and IEEE1394 (FireWire) are comparable in terms of performance, GigEVision has the added benefit of being able to take advantage of commonly available networking hardware (Cat5 or Cat6 cabling, RJ45 ends), which can be a big selling point over IEEE1394. In addition, GigEVision can communicate 100+ meters without extra hardware (a switch or repeater would be required over 100m.), whereas IEEE1394 is constrained to ~5m.

Conclusion

ControlsThrough B&R, Piedmont has the ability to offer enterprise-grade software solutions for

mechatronics control and integration to current and potential clients. As demonstrated previously, mechatronics is a cross-discipline application and requires a very skilled sales and marketing approach to

16



prove the MAPP platform as a viable, workable solution. B&R is also cost-competitive with the other players in the market in this domain.

MechatronicsPAI does not currently have the resources to build industrial-grade robots, nor does it believe

such an endeavor would be a worthwhile expansion of its operations. Through a partner supplier, Codian Robotics, we have the ability to buy “headless” robots (without motors, gearboxes, drives, and controls; all of which we do offer).

VisionTo offer a complete workable, industrial-grade vision solution is definitely within the capabilities

of PAI. Though we would not be able to offer the same intuitive UI for vision controller programming as the other market players (Cognex, Keyence), the collective brain trust has the capacity to analyze a problem and conceive a solution consistent with the quality that Piedmont Automation Inc. is known to deliver.

OverallSince our clients have definitely displayed a desire for more advanced robotics integration into

their current product platforms, Piedmont is at the stage where we could accept a “pilot” project to prove each individual component of the mechatronics solution, as well as their integration together functioning as a single unit.

17


Other Points-

1. Advantech machine vision using Intel CPU’s- (catalog)2. Interesting “philosophical” article on computer evolution- (wetware, 2012)3. Machine Vision Application Brief- (AMD)4. Blackfly POE imaging performance- (Grey)

Citations

AMD. (n.d.). Data Handling, Throughput, and Image Processing. Retrieved 1 26, 2016, from AMD: https://www.amd.com/Documents/Machine_Vision_Application_Brief.pdf

Andy. (2011, 8 25). Integrating the FlyCapture SDK for use with OpenCV. Retrieved 1 26, 2016, from Technical Recipes: http://www.technical-recipes.com/2011/integrating-the-flycapture-sdk-for-use-with-opencv/

B&R Automation. (n.d.). MAPP in Machine Groups.

catalog, e. (n.d.). New, Intelligent System for Machine Vision. Retrieved from http://eecatalog.com/intel/2014/02/13/new-intelligent-system-for-machine-vision/

Grey, P. (n.d.). Blackfly GigE performance. Retrieved from https://www.ptgrey.com/support/downloads/10107

Machine Vision Lens Calculator. (n.d.). Retrieved 1 26, 2016, from http://machinevisionstore.com/design/calculator

Smith, J. E. (2005). The Architecture of Virtual Machines. Computer, 7.

Thryft, A. (2010, 8 1). GigE Vision boosts inspection networks. Retrieved 1 26, 2016, from EDN Network: http://www.edn.com/electronics-news/4388256/GigE-Vision-boosts-inspection-networks

vision-doctor. (n.d.). Specifiation of a Machine Vision System. Retrieved from Vision Doctor: http://www.vision-doctor.co.uk/systems.html

wetware. (2012, 8 11). What "Worse is Better vs The Right Thing" is really about. Retrieved 1 26, 2016, from Proper Fixation: http://yosefk.com/blog/what-worse-is-better-vs-the-right-thing-is-really-about.html

18


19