111
Robot Vision slide1

Robot Vision slide1. Robot Vision Exercise Robot Vision slide3 The task To locate the picking point of the object = Centre of Gravity of the object To

  • View
    225

  • Download
    1

Embed Size (px)

Citation preview

Page 1: Robot Vision slide1. Robot Vision Exercise Robot Vision slide3 The task To locate the picking point of the object = Centre of Gravity of the object To

Robot Vision slide1

Page 2: Robot Vision slide1. Robot Vision Exercise Robot Vision slide3 The task To locate the picking point of the object = Centre of Gravity of the object To

Robot Vision

Exercise

Page 3: Robot Vision slide1. Robot Vision Exercise Robot Vision slide3 The task To locate the picking point of the object = Centre of Gravity of the object To

Robot Vision slide3

The task

• To locate the picking point of the object = Centre of Gravity of the object

• To measure the rotation of the object = the angle between Y axes and the line going from centre point of mid hole and to centre point outer hole

• To send the picking point co-ordinates (x,y) and rotation (angle) in the robot co-ordinate system to the Robot system using RS232

Robot co-ordinate system

X

Y

Origo

Picking pointAngle

Page 4: Robot Vision slide1. Robot Vision Exercise Robot Vision slide3 The task To locate the picking point of the object = Centre of Gravity of the object To

Robot Vision slide4

Configuring of Scorpion Robot vision system

1. Create a new Scorpion Profile2. Configure the image source - camera/file3. Calibration of images4. Locate the object and find the pick-up x,y

coordinates5. Calibration in Robot co-ordinates6. Establish reference system to measure rotation7. Configure the User Interface8. Configure communication to external systems

Page 5: Robot Vision slide1. Robot Vision Exercise Robot Vision slide3 The task To locate the picking point of the object = Centre of Gravity of the object To

Robot Vision slide5

Restore Robot Vision Start profile

1. Open Scorpion and click on restore(if the menu is not visible right click on the mouse, click on “Show Buttons” for permanent display)

2. Browse to the catalogue “unsupportedprofiles”on the Scorpion CD

3. Select RobotVision_Start.zipan open the file

Page 6: Robot Vision slide1. Robot Vision Exercise Robot Vision slide3 The task To locate the picking point of the object = Centre of Gravity of the object To

Robot Vision slide6

Create MyRobotVision profile

• Step 1 – Create a new Profile

1. Select ”New”

2. Give it the name”MyRobotVision”

Page 7: Robot Vision slide1. Robot Vision Exercise Robot Vision slide3 The task To locate the picking point of the object = Centre of Gravity of the object To

Robot Vision slide7

Copy the content of RobotVision_Start to the new profile1. You can copy the content of

an existing profile (including images) to automatically add functionality to the new profile

2. Mark the profile called ”RobotVision_start” and select copy

3. Select copy to the new profile”MyRobotVision”

4. Start MyRobotVision profile

Page 8: Robot Vision slide1. Robot Vision Exercise Robot Vision slide3 The task To locate the picking point of the object = Centre of Gravity of the object To

Robot Vision slide8

Starting point

• Operation mode: (No password)– For operators– Can start and stop the inspection– Read history list, curves and

statistics– No access to any configuration

• Settings mode: (Password: 1234)– For operators certified to adjust

certain parameters, alarm limits and logic states

• Service mode: (Password: 911)– Full access to all functions

• Enter Service mode by pressing the service button and enter the password

Page 9: Robot Vision slide1. Robot Vision Exercise Robot Vision slide3 The task To locate the picking point of the object = Centre of Gravity of the object To

Robot Vision slide9

Configure the image source1. Select Service Tab2. We will use stored images

therefore activate Simulation ON

3. Click on Snapshot4. If no image the image directory

must be checked5. Click on the image setting

source6. Browse to find the selected

image directory7. Assure that the directory for

images are set to

\Scorpion_200\MyRobotVision\Images

1

2

3

4

5

6

Page 10: Robot Vision slide1. Robot Vision Exercise Robot Vision slide3 The task To locate the picking point of the object = Centre of Gravity of the object To

Robot Vision slide10

Vision Strategy

• Calibration – for correcting lens distortion to get high precision

• 4 point robot calibration to synchronize image and robot co-ordinate system

• Use blob to find centre of gravity of the object

• Use the big outer hole centre and mid hole centre to establish a local reference system handling rotation

Robot co-ordinate system

X

Y

Origo

Picking pointAngle

Page 11: Robot Vision slide1. Robot Vision Exercise Robot Vision slide3 The task To locate the picking point of the object = Centre of Gravity of the object To

Robot Vision slide11

Lens calibration

• In robot vision application the camera often have a large Field Of View (FOW) and a relative short distance to the object

• This require a wide angle lens and consequently large lens distortion

• Without correcting the distortion, the inaccuracy in the outer part of the FOW might cause the robot to fail picking

the object correctly

Page 12: Robot Vision slide1. Robot Vision Exercise Robot Vision slide3 The task To locate the picking point of the object = Centre of Gravity of the object To

Robot Vision slide12

4 point robot calibration

• By using 4 point calibration you can establish a co-ordinate system equal to the one used by the robot

• By moving the robot to the centre of each point the robot co-ordinates are found

• In Scorpion the centre of each is found in the Scorpion reference system

• This enable you to communicate pick up points and rotation in the robot co-ordinate system

Page 13: Robot Vision slide1. Robot Vision Exercise Robot Vision slide3 The task To locate the picking point of the object = Centre of Gravity of the object To

Robot Vision slide13

Understanding lens correction and robot calibration

• Correction of lens distortion is independent of distance to the object

• The lens correction is done in one planeand valid for allrobot planes

• Robot calibration is only valid for a specific distance to the object (Robot plane)

• For each different robot picking plane a individual robot calibration must be done

• Scorpion can handle multiple robot planes

Robot plane 1

Robot plane 2

Robot plane 3

Page 14: Robot Vision slide1. Robot Vision Exercise Robot Vision slide3 The task To locate the picking point of the object = Centre of Gravity of the object To

Robot Vision slide14

Pixel co-ordinate system

• Default co-ordinate system with Origo in upper left corner

• X and Y pixels according to camera resolution– VGA 760 x 480

– XGA 1024 x 768(used in the exercise)

X

YOrigo

X=0, Y=0

X=768, Y=1024

X=0, Y=1024

X=768, Y=0

Page 15: Robot Vision slide1. Robot Vision Exercise Robot Vision slide3 The task To locate the picking point of the object = Centre of Gravity of the object To

Robot Vision slide15

Show Info on the image

• With the cursor on the image right click the mouse and select ”Show info” from the menu

• X and Y number are pixel position, origo (X=0 and Y=0) is in the upper left corner

• Pixel value is the greyscale value (0 = Black, 256 = White)

Page 16: Robot Vision slide1. Robot Vision Exercise Robot Vision slide3 The task To locate the picking point of the object = Centre of Gravity of the object To

Robot Vision slide16

Zoom the image

• Press the left button on the mouse and mark the zoom area

• Release the button and the image is zoomed in

• Click on the image to zoom out

Page 17: Robot Vision slide1. Robot Vision Exercise Robot Vision slide3 The task To locate the picking point of the object = Centre of Gravity of the object To

Robot Vision slide17

Operating the Tool Box

1. Go to the Toolbox and click on New

2. Give the Tool a NameFor easy understanding of the task for this specific tool – use names

3. Select a Tool category and then a Tool

4. Use help to get information about the tool

1

2

3

4

Page 18: Robot Vision slide1. Robot Vision Exercise Robot Vision slide3 The task To locate the picking point of the object = Centre of Gravity of the object To

Robot Vision slide18

Using a Tool – step 1

• GENERALShow the name and type of tool. Comments can be entered in the description field

• SETUPFor entering parameters controlling the tool

Page 19: Robot Vision slide1. Robot Vision Exercise Robot Vision slide3 The task To locate the picking point of the object = Centre of Gravity of the object To

Robot Vision slide19

Using a Tool – step 2

• VISUALISATIONFor adding graphics on the image, visualising the operation of the toolAll or this tool only visualisation, can be selected

• RESULTDisplaying all the results from the operation of the tool

Page 20: Robot Vision slide1. Robot Vision Exercise Robot Vision slide3 The task To locate the picking point of the object = Centre of Gravity of the object To

Robot Vision slide20

Create a tool - Calibration

1. Click on the button “New”

2. Name the tool - Calibration

3. Select the tool category Advanced

4. Select “Calibrator”Use help if you want information about the tool

5. Click OK

1

2

3 4

5

Page 21: Robot Vision slide1. Robot Vision Exercise Robot Vision slide3 The task To locate the picking point of the object = Centre of Gravity of the object To

Robot Vision slide21

Selecting the image for calibration

• Open the tool named Calibration (double click on it) and click on the Set-up tab

• Browse to find the calibration image in the “MyRobotVision/References” directory

• Select calib1.bmp image and open

1

2

3

Page 22: Robot Vision slide1. Robot Vision Exercise Robot Vision slide3 The task To locate the picking point of the object = Centre of Gravity of the object To

Robot Vision slide22

Configure the Calibration tool

• Set number of rows and column – count from image

• Set row and column spacing to 8 mm, and the unit to mm (millimetre)

• Click on compute

1 2

3

Page 23: Robot Vision slide1. Robot Vision Exercise Robot Vision slide3 The task To locate the picking point of the object = Centre of Gravity of the object To

Robot Vision slide23

Results from Calibration

• Scaling– Under the Result tab you

will find the scaling factor– The scaling factor is only

valid in the grid plane

• Lens correction– On the image the actual

correction is visualised as yellow arrows

– Lens correction is valid in any plan (not only in the grid plane)

Page 24: Robot Vision slide1. Robot Vision Exercise Robot Vision slide3 The task To locate the picking point of the object = Centre of Gravity of the object To

Robot Vision slide24

Multi co-ordinate system

• Pixel co-ordinate system (red)– the default coordinate

system– Scaling is in pixels

• Calibration co-ordinate system (blue)– Generated by the

Calibration tool– Scaling is in mm

X

Y

X

Y

Page 25: Robot Vision slide1. Robot Vision Exercise Robot Vision slide3 The task To locate the picking point of the object = Centre of Gravity of the object To

Robot Vision slide25

Cursor information on the image

• With the cursor you can now display the x and y values in both co-ordinate system

1. Position the mouse over the image and right click

2. Highlight “Referencesystem”

3. Select the co-ordinate system

Centre hole co-ordinates:

Pixel co-ordinates

Calibrationco-ordinates

1

2 3

Page 26: Robot Vision slide1. Robot Vision Exercise Robot Vision slide3 The task To locate the picking point of the object = Centre of Gravity of the object To

Robot Vision slide26

Manual measurements on the image

• Position the mouse on the image and right click to get the menu (be sure that “Show info is activated first”

1. Select “Measure”2. With the cursor you now

can measure the distance from the default pixel co-ordinate origo and the angle of the line

1

2

Page 27: Robot Vision slide1. Robot Vision Exercise Robot Vision slide3 The task To locate the picking point of the object = Centre of Gravity of the object To

Robot Vision slide27

To measure from an arbitrary point

1. To set a new origo position the mouse where you want the new origo (i.e. centre of mid hole) and right click on the mouse

2. Select “Set Origo” and the cursor display distance from the new origo

1

2

Page 28: Robot Vision slide1. Robot Vision Exercise Robot Vision slide3 The task To locate the picking point of the object = Centre of Gravity of the object To

Robot Vision slide28

Saving and storing the configuration

• Saving configuration– Pressing Profile button

– Every time you press the Start button

– Every time you close the application

• Archiving– The archive file is a zip

file containing all configuration and images

– Used for storing your profile

– Ideal for sending to a Scorpion support centre for remote support

Page 29: Robot Vision slide1. Robot Vision Exercise Robot Vision slide3 The task To locate the picking point of the object = Centre of Gravity of the object To

Robot Vision slide29

Archiving a profile

• Enter to Service mode• Press the Maintenance tab• Check that the Archive

directory is selected• Press the Backup button• Scorpion suggest a default

name• You can add comments

before the zip file is generated

1

23

3

4

Page 30: Robot Vision slide1. Robot Vision Exercise Robot Vision slide3 The task To locate the picking point of the object = Centre of Gravity of the object To

Robot Vision slide30

Trigger the camera

• Hardwire trigger– Direct on camera from sensor

• Soft trigger– External from PLC

• RS232, TCP/IP, I/O

– Internal - used in this exercise• Using Scheduler (pressing Start button)• Manual user input (pressing Snapshot button)

Page 31: Robot Vision slide1. Robot Vision Exercise Robot Vision slide3 The task To locate the picking point of the object = Centre of Gravity of the object To

Robot Vision slide31

Scheduler

• For setting up tasks running at scheduled intervals

• The task ”Trigger” is configured to grab an image every 1000 ms

• When ”Start” button is pressed a new image according to the Trigger period

• By pressing edit you can adjust the time interval

1

2

4

3

Page 32: Robot Vision slide1. Robot Vision Exercise Robot Vision slide3 The task To locate the picking point of the object = Centre of Gravity of the object To

Robot Vision slide32

Start and Stop of Inspections

• Automatic inspection• Pressing ”Start” button set

Scorpion in Running mode

• For every new image an inspection is automatically executed

• The image can either be triggered from an external system or from an internal scheduler

• Pressing the ”Stop” button stop the process

• Manual inspection– Pressing ”Snapshot” button grab a

new picture

– Pressing ”Inspection” button execute the inspection

– Pressing ”Snapshot” button again will grab another new picture.

Page 33: Robot Vision slide1. Robot Vision Exercise Robot Vision slide3 The task To locate the picking point of the object = Centre of Gravity of the object To

Robot Vision slide33

Using a Blob to locate the object

• Add a new tool in the toolbox:

– Name= FindObject

– Tool = BlobTool

1

2

34

Page 34: Robot Vision slide1. Robot Vision Exercise Robot Vision slide3 The task To locate the picking point of the object = Centre of Gravity of the object To

Robot Vision slide34

Tool references

• All tools can have a Reference

• To relate Search area to a specific point (i.e. Centre of Gravity of an area)

• To relate Search area to a specific co-ordinate system, Calibration in this exercise

Page 35: Robot Vision slide1. Robot Vision Exercise Robot Vision slide3 The task To locate the picking point of the object = Centre of Gravity of the object To

Robot Vision slide35

ROI – Region of Interest

• Each tool can must be configure to search in a specific area on the image (Range of Interest – ROI)

• Move the mouse to the centre of the image and read the X and Y coordinates.

• Enter the co-ordinates and specify the delta X and delta Y values

• The ROI visualisation is shown on the picture

Page 36: Robot Vision slide1. Robot Vision Exercise Robot Vision slide3 The task To locate the picking point of the object = Centre of Gravity of the object To

Robot Vision slide36

Understanding the Blob tool

• A blob search for areas in the specified ROI with greyscale values within a given range

• Place the mouse on the object and read the greyscale value

• If the maximum Threshold is set to higher value and the minimum to a lower, the Blob tool will find the object

Page 37: Robot Vision slide1. Robot Vision Exercise Robot Vision slide3 The task To locate the picking point of the object = Centre of Gravity of the object To

Robot Vision slide37

Apply your configuration

• To apply a new parameter set by you, the button apply must be pushed

• You can also set minimum and/or maximum requirements to the blob area to be located

Page 38: Robot Vision slide1. Robot Vision Exercise Robot Vision slide3 The task To locate the picking point of the object = Centre of Gravity of the object To

Robot Vision slide38

Blob visualisation

• Through the visualisation you can see how the tool works on the image

• Activate only CenterofGravity, MaxContour and ROI

• Remember to push ”Apply” to see the result!

Page 39: Robot Vision slide1. Robot Vision Exercise Robot Vision slide3 The task To locate the picking point of the object = Centre of Gravity of the object To

Robot Vision slide39

Blob result

• In the result tab you find the different results of the blob tool– Number of blobs (1)– total blob area (2)– centre of gravity of the

largest blob, blob[0] (3)– Number of ”Holes” in

blob[0](4)– If there is more than 1 blob

you will find values for blob[1-n] also

1

2

3

4

Page 40: Robot Vision slide1. Robot Vision Exercise Robot Vision slide3 The task To locate the picking point of the object = Centre of Gravity of the object To

Robot Vision slide40

Holes in a Blob

• A Hole is an area inside a blob with greyscale values outside the max and min values set in the blob tool

Page 41: Robot Vision slide1. Robot Vision Exercise Robot Vision slide3 The task To locate the picking point of the object = Centre of Gravity of the object To

Robot Vision slide41

Picking location

• In this application the pick-up position is the centre of the object

• This is equal to the centre of gravity co-ordinates found for the blob

Page 42: Robot Vision slide1. Robot Vision Exercise Robot Vision slide3 The task To locate the picking point of the object = Centre of Gravity of the object To

Robot Vision slide42

Strategies to find a picking location

• Geometrical characteristic of the object is used to find the correct picking localisation. In this example the centre of gravity was the correct picking location. However other localisations might very well be used as well. Below are some examples

1. Mid point short edge

2. One corner of the long edge

3. Centre point of the small hole on the short edge

Page 43: Robot Vision slide1. Robot Vision Exercise Robot Vision slide3 The task To locate the picking point of the object = Centre of Gravity of the object To

Robot Vision slide43

Co-ordinate systems

• The picking point is the centre of gravity of the blob. However the x,y co-ordinates we found in the blob is according to the “Calibration” co-ordinate system (Blue) used as the reference in the blob tool

• We now need to recalibrate according to the Robot Calibration system

Page 44: Robot Vision slide1. Robot Vision Exercise Robot Vision slide3 The task To locate the picking point of the object = Centre of Gravity of the object To

Robot Vision slide44

4 point calibration steps

1. By moving the robot to the centre of each point the robot co-ordinates are found

2. In Scorpion the centre of each point is found in the Scorpion reference system

3. By using the “ExternalReference” tool Scorpion can calibrate the image in robot co-ordinates

Page 45: Robot Vision slide1. Robot Vision Exercise Robot Vision slide3 The task To locate the picking point of the object = Centre of Gravity of the object To

Robot Vision slide45

4 point Robot calibration

• First we need to find the four points and their centre of gravity in the robot calibration grid

• Add a new tool in the toolbox:

– Name= FindRefPoints

– Tool = BlobTool

Page 46: Robot Vision slide1. Robot Vision Exercise Robot Vision slide3 The task To locate the picking point of the object = Centre of Gravity of the object To

Robot Vision slide46

Find Ref Point image

• The image of the 4 point calibration grid is located in the directoryMyRobotVision/References

• Follow the steps (1 – 5) and push the Snapshot button (6) until you get the right image

1

2

3

4

5

6

Page 47: Robot Vision slide1. Robot Vision Exercise Robot Vision slide3 The task To locate the picking point of the object = Centre of Gravity of the object To

Robot Vision slide47

Configure FindRefPoints tool

• Follow the steps to set reference to “Calibration” (1), the correct search area (2) and threshold levels (3). To eliminate disturbing blobs from the text “5cm” activate Smallest area = 10

1

2

3

4

Page 48: Robot Vision slide1. Robot Vision Exercise Robot Vision slide3 The task To locate the picking point of the object = Centre of Gravity of the object To

Robot Vision slide48

RefPoints co-ordinates

• In the Result tab you find the centre of gravity x,y co-ordinates

• Use the cursor in the image to check that you write the correct x and y values for point 1 – 4

1 2 3 4

X 8.12 8.63 58.05 58.55

Y 9.48 59.78 8.92 59.22

Point 1 Point 2

Point 3 Point 4

Page 49: Robot Vision slide1. Robot Vision Exercise Robot Vision slide3 The task To locate the picking point of the object = Centre of Gravity of the object To

Robot Vision slide49

Robot co-ordinate system

• By moving the robot to the four points you get the equivalent robot co-ordinates

1 2 3 4

X 0 0 50 50

Y 0 50 0 50

Point 1 Point 2

Point 3 Point 4

Origo

Page 50: Robot Vision slide1. Robot Vision Exercise Robot Vision slide3 The task To locate the picking point of the object = Centre of Gravity of the object To

Robot Vision slide50

External Reference tool

• Next we need to co-ordinate the image and robot co-ordinate systems

• Add a new tool in the toolbox:

– Name= ExternalRefSystem

– Tool = ExternalReference

Page 51: Robot Vision slide1. Robot Vision Exercise Robot Vision slide3 The task To locate the picking point of the object = Centre of Gravity of the object To

Robot Vision slide51

Configure ExternalRefSystem tool

• Select Calibration as a reference (1)

• Fill inn the local co-ordinates for all 4 points (2)

• Fill in the external coordinates for all 4 points (3)

1

2 3

Page 52: Robot Vision slide1. Robot Vision Exercise Robot Vision slide3 The task To locate the picking point of the object = Centre of Gravity of the object To

Robot Vision slide52

The different Reference system

• In the image you now have 3 different co-ordinate systems

1. The original Pixel systems (green)

2. The Calibration co-ordinate system (blue)

3. The Robot co-ordinate system (red)

Page 53: Robot Vision slide1. Robot Vision Exercise Robot Vision slide3 The task To locate the picking point of the object = Centre of Gravity of the object To

Robot Vision slide53

Deactivate/active a tool

• The Blob tool “FindRefPoints” is only used to find the centres of each point during robot calibration

• During normal operation this tool has no task and should be deactivated (1) or deleted

• Highlight the tool “FindRefPOints” and click the button “Deactivate”

Page 54: Robot Vision slide1. Robot Vision Exercise Robot Vision slide3 The task To locate the picking point of the object = Centre of Gravity of the object To

Robot Vision slide54

Changes back to normal images

• To continuo we need to change back to the original images found in the directory:MyRobotVision/Images

• Follow the steps (1 – 5) and push the Snapshot button (6) to check that you get the right image

1

2

3

4

5

6

Page 55: Robot Vision slide1. Robot Vision Exercise Robot Vision slide3 The task To locate the picking point of the object = Centre of Gravity of the object To

Robot Vision slide55

Changing reference to the FindObject Tool

• To be able to find the centre of gravity x and y values we need to change the reference of the “FindObject” blob tool to the new “ExternalRefSystem”

• Open the tool named “FindObject” select Setup tab and push the button (1)

• Here you find the available references, but ExternalRefSystem is not THERE

1

Page 56: Robot Vision slide1. Robot Vision Exercise Robot Vision slide3 The task To locate the picking point of the object = Centre of Gravity of the object To

Robot Vision slide56

Understanding the tool sequence

• All tools are executed in a sequence starting from the top of the toolbox

• Only results from tool being executed before can be used

• Therefore results from ExternalRefSystem cannot be accessed by “FindObject

Page 57: Robot Vision slide1. Robot Vision Exercise Robot Vision slide3 The task To locate the picking point of the object = Centre of Gravity of the object To

Robot Vision slide57

Editing the tool sequence

• By highlighting the tool and then using the Up and Down buttons, any tool can be moved

• Move the FindRefPoints to number 2 and ExternalRefSystem to number 3.

Page 58: Robot Vision slide1. Robot Vision Exercise Robot Vision slide3 The task To locate the picking point of the object = Centre of Gravity of the object To

Robot Vision slide58

Changing the reference toRobot co-ordinates

• Now you can change the reference of the “FindObject” blob tool to the new “ExternalRefSystem”

• Open the tool named “FindObject” select Setup tab and push the button (1)

• Select the ExternalRefSystem

1

Page 59: Robot Vision slide1. Robot Vision Exercise Robot Vision slide3 The task To locate the picking point of the object = Centre of Gravity of the object To

Robot Vision slide59

Adjusting the search area according to the new reference system

• Since the reference system is changed, the parameter values must be changed to give the correct search area

Page 60: Robot Vision slide1. Robot Vision Exercise Robot Vision slide3 The task To locate the picking point of the object = Centre of Gravity of the object To

Robot Vision slide60

Scorpion logical conditions

• In an industrial application a number of situations may occur

• In Scorpion you can define a set of conditions that will help you to handle the different situations during robot picking or inspection

• The worst scenario is that a situation occur and neither the vision system nor the robot knows how to handle it

• Then the system will fail and the process stop

Page 61: Robot Vision slide1. Robot Vision Exercise Robot Vision slide3 The task To locate the picking point of the object = Centre of Gravity of the object To

Robot Vision slide61

Potential logical conditions inRobot Vision

• OK– Object present– Can measure rotation

• No Object– Object NOT present

• Cannot measure rotation– Object present– Cannot measure rotation

(missing the outer hole used to define rotation)

Page 62: Robot Vision slide1. Robot Vision Exercise Robot Vision slide3 The task To locate the picking point of the object = Centre of Gravity of the object To

Robot Vision slide62

Understanding Logical Tool

• In a Logical tool any results in all prior tools can be tested against max and min conditions

• Typical applications are:– Establish OK and Fail conditions

– Testing measurements against tolerances

– Verify presence of object

– Validate type of object

– Validate functionality of a tool

Page 63: Robot Vision slide1. Robot Vision Exercise Robot Vision slide3 The task To locate the picking point of the object = Centre of Gravity of the object To

Robot Vision slide63

Object Present

• For testing whether an object is present we could choose different parameter

1. Number of blobs(could be sensitive to dirt and small particles that may be identified as blobs)

2. Size of Blob area – a known parameter for the specified product

3. We will use Blob area

Page 64: Robot Vision slide1. Robot Vision Exercise Robot Vision slide3 The task To locate the picking point of the object = Centre of Gravity of the object To

Robot Vision slide64

Object Present logical condition

• Add a new tool (1), name it ObjectPresent (2) and select the LogicTool (3)

• Open it and select the measured value from the tool FindObject (4) and the parameter blob area[0] (5) and set the minimum conditions to 500

• Normal value for the blob area is around 1100

1

2

3

45

6

Page 65: Robot Vision slide1. Robot Vision Exercise Robot Vision slide3 The task To locate the picking point of the object = Centre of Gravity of the object To

Robot Vision slide65

Checking the ObjectPresent tool

• Select the result panel and go through all images and check that the result is correct

• Object present should give Value = 1

• Object absent should give Value = 0

Page 66: Robot Vision slide1. Robot Vision Exercise Robot Vision slide3 The task To locate the picking point of the object = Centre of Gravity of the object To

Robot Vision slide66

Controlling the orientation

• The task is to establish a new reference system that will give the rotation/angle relative to the external robot co-ordinate system (red)

• By finding the centre point of the outer big hole a reference line can be designed and using the centre of gravity of the blob as origo we can generate the new reference system (blue)

Y

X

Y

X

Page 67: Robot Vision slide1. Robot Vision Exercise Robot Vision slide3 The task To locate the picking point of the object = Centre of Gravity of the object To

Robot Vision slide67

Find the outer big hole

• Add a new tool, name it “FindHole” and select the Blob2 tool

• Open the tool and select FindObject as the reference and “Apply”

• When FindObject is used as reference, the Centre of Gravity of the object represent centre X = 0 and Y = 0 wherever the object is located on the image

Page 68: Robot Vision slide1. Robot Vision Exercise Robot Vision slide3 The task To locate the picking point of the object = Centre of Gravity of the object To

Robot Vision slide68

Multi Polygon and Multi ROI’s

• Multi polygon ROI– An ROI constructed from

arbitrary points

– The ROI is constructed by clicking the mouse while the “Ctrl” button is pressed on the points defining the ROI

• Multi ROI’s– One tool can operate in

several ROI’s

ROI-1

ROI-2

Mouse clicking points

Point co-ordinatesdefining the ROI

Page 69: Robot Vision slide1. Robot Vision Exercise Robot Vision slide3 The task To locate the picking point of the object = Centre of Gravity of the object To

Robot Vision slide69

Difference ROI in Blob2

• Using two ROI’s where ROI-2 is a segment inside ROI-1, the difference area will be the final ROI for the tool

• The red circle is ROI-1• The blue circle is the ROI-2• The search are is the

difference, the grey area, and we can locate the the white outer hole

Page 70: Robot Vision slide1. Robot Vision Exercise Robot Vision slide3 The task To locate the picking point of the object = Centre of Gravity of the object To

Robot Vision slide70

Define ROI-1

1. Select “Add polygon” tab2. Select “Circle” polygon3. Enter 0 for centre X and Y4. Enter Radius 16 (mm). You

can use the cursor to find the correct radius to use

5. Set Minimum threshold to 100 and maximum to 256 since we are looking for white holes

6. Click on “Add”7. Click on “Apply” and the

ROI is active

12

34

5

6

7

Page 71: Robot Vision slide1. Robot Vision Exercise Robot Vision slide3 The task To locate the picking point of the object = Centre of Gravity of the object To

Robot Vision slide71

Editing multiple ROI’s

• Select the tab “Polygon” and you have an overview of the ROI’s

• Number 1 is a default ROI and number 2 is the polygon you added

• You can highlight the individual ROI’s by selecting “Highlight selection”

• To delete the default ROI highlight 1 and click on the “Delete” button

• Finally click on “Apply”

1

2

3

Page 72: Robot Vision slide1. Robot Vision Exercise Robot Vision slide3 The task To locate the picking point of the object = Centre of Gravity of the object To

Robot Vision slide72

Constraints

• In many tools you can set a number of constraints to eliminate unwanted results

• In this case you may detect small blobs that have no interest

• They can be eliminated by setting constraints for the blob area

• Setting smallest blob area to 10 will eliminate these blobs

Page 73: Robot Vision slide1. Robot Vision Exercise Robot Vision slide3 The task To locate the picking point of the object = Centre of Gravity of the object To

Robot Vision slide73

Define ROI-2

• Define the ROI-2 with the following parameters– Centre X and Y = 0

– Radius = 10

• Add the polygon and then Apply

• You now have to polygons and you see the two circles visualised in the image

Page 74: Robot Vision slide1. Robot Vision Exercise Robot Vision slide3 The task To locate the picking point of the object = Centre of Gravity of the object To

Robot Vision slide74

FindHole results

1. Click on “Visualisation this only” to remove visualisation from the other tools

2. Select the “Result” tab and you will find all the results

3. Number of Blobs

4. Centre of Gravity

1

2

3

4

Page 75: Robot Vision slide1. Robot Vision Exercise Robot Vision slide3 The task To locate the picking point of the object = Centre of Gravity of the object To

Robot Vision slide75

The Guard function

• Used to activate tools only for a specific condition

• Typical applications:– Selecting a set of tools tailored to a specific

product (used when Scorpion is set to handle multiple products)

– To prevent tools from giving false results when the condition is not fulfilled

Page 76: Robot Vision slide1. Robot Vision Exercise Robot Vision slide3 The task To locate the picking point of the object = Centre of Gravity of the object To

Robot Vision slide76

Configure the Guard function

• When the condition ObjectPresent is False (no object in the image), the FindHole tool have no meaning and should be guarded

1. Select the General tab

2. Click to the list of possible guard functions

3. Select “ObjectPresent”

1

23

Page 77: Robot Vision slide1. Robot Vision Exercise Robot Vision slide3 The task To locate the picking point of the object = Centre of Gravity of the object To

Robot Vision slide77

• In the Status column or in the Result tab you can see the status of the tool– Status = 1 - tool is OK– Status = 3 - there is an error

in the tool– Tool is blocked by guard or

reference– Tool is deactivated– Tool is not run– When tool is either blocked,

deactivated or not run the Status in the result tab = 0

Status for tools

Run through all images andsee how status changeswhen there is no object

Page 78: Robot Vision slide1. Robot Vision Exercise Robot Vision slide3 The task To locate the picking point of the object = Centre of Gravity of the object To

Robot Vision slide78

Toolbox overview

• In the toolbox overview you will find the following columns with info– 1 = Status– 2 = Processing time for the

tool– 3 = The name you gave the

tool– 4 = The type of tool– 5 = Image number reference– 6 = The reference used by

the tool– 7 = The guarding tool

1 2 3 4 5 6 7

Hint: to see all columns you mayhave to adjust the column widths

Page 79: Robot Vision slide1. Robot Vision Exercise Robot Vision slide3 The task To locate the picking point of the object = Centre of Gravity of the object To

Robot Vision slide79

Understanding Geometry tools

• The Geometry tools are used to create new points and lines based on results from previous tools

• Typical applications:– To construct geometrical structures without image

processing techniques

– Do measurements• Distances between lines and points

• Length and angle of lines

Page 80: Robot Vision slide1. Robot Vision Exercise Robot Vision slide3 The task To locate the picking point of the object = Centre of Gravity of the object To

Robot Vision slide80

Make the reference line

• Add a new tool named “ReferenceLine” and select the tool “LineFromPoints”

• Guard with “ObjectPresent” condition

• Use ExternalRefSystem (Robot co-ordinates) as reference and select centre of gravity of the object as point 1 and centre of gravity of the outer hole as point 2

1

2 3

4

5

6

7

Page 81: Robot Vision slide1. Robot Vision Exercise Robot Vision slide3 The task To locate the picking point of the object = Centre of Gravity of the object To

Robot Vision slide81

Reference Line check

• Scorpion visualise a line between the two points and a direction (from point 1 to point 2)

• In the result tab you will find results as length and angle

• If one of the points are missing the tool will have an error (status = 3)

Page 82: Robot Vision slide1. Robot Vision Exercise Robot Vision slide3 The task To locate the picking point of the object = Centre of Gravity of the object To

Robot Vision slide82

Configure the ReferenceLineOK logic tool

• We can use a logical tool to establish a condition validating the ReferenceLine

• The procedure is:– Name: “ReferenceLineOK”– Tool: Logical tool– Guard: “ObjectPresent”– Add new logical parameter– Select Tool: ReferenceLine– Parameter: Status– Min and Max condition = 1

• Check the result

1

2

3

4

5

6

Page 83: Robot Vision slide1. Robot Vision Exercise Robot Vision slide3 The task To locate the picking point of the object = Centre of Gravity of the object To

Robot Vision slide83

Create the local co-ordinate system

• Based on the Reference Line and Centre of Gravity of the object we can create e new co-ordinate system.– Name: RotationRefSystem

– Tool: PointLine Reference

– Guard: ReferenceLineOK

1

2 3

4

Page 84: Robot Vision slide1. Robot Vision Exercise Robot Vision slide3 The task To locate the picking point of the object = Centre of Gravity of the object To

Robot Vision slide84

Define the co-ordinates

• Select “ExternalRefSystem” to link this co-ordinate system to the robot system

• Select Centre of Gravity as the origo point

• Select ReferenceLine as the Line and the co-ordinate system will follow the rotation of the object

1

2

34

Page 85: Robot Vision slide1. Robot Vision Exercise Robot Vision slide3 The task To locate the picking point of the object = Centre of Gravity of the object To

Robot Vision slide85

Scorpion co-ordinate systems

• We now have 4co-ordinate systems

1. The Pixel system (green)2. The Calibration system

(blue)3. The Robot system

(ExtrenalRefSystem) (red)

4. The Rotation system(light blue)

– With the cursor you can toggle between the systems

1

23

4

Page 86: Robot Vision slide1. Robot Vision Exercise Robot Vision slide3 The task To locate the picking point of the object = Centre of Gravity of the object To

Robot Vision slide86

OK condition - Object Present

• Double click on the existing OK state

• Now you can enter a name, description and select the colour for display in the result panel

Page 87: Robot Vision slide1. Robot Vision Exercise Robot Vision slide3 The task To locate the picking point of the object = Centre of Gravity of the object To

Robot Vision slide87

Defining the OK (Present) condition

• Click on ”Constrains” and ”New”

• The box shows a list of all available logical classifications in the Toolbox

• Select ”ObjectPresent”• The OK condition is now

true when the logical classification ”ObjectPresent” is true (=1)

1

23

4

Page 88: Robot Vision slide1. Robot Vision Exercise Robot Vision slide3 The task To locate the picking point of the object = Centre of Gravity of the object To

Robot Vision slide88

Including ReferenceLineOK

• The OK condition is also dependent of that correct angle can be measures

• Correct angle can only be measured when the logical statement “ReferenceLineOK” is true

• Add “ReferenceLineOK” using the same procedure as for “ObjectPresent”

Page 89: Robot Vision slide1. Robot Vision Exercise Robot Vision slide3 The task To locate the picking point of the object = Centre of Gravity of the object To

Robot Vision slide89

Testing the OK condition

• Run through all pictures and check the result of the condition (true or false) and the display on the result panel

• When no object and the object missing a hole occur, the result panel display ”Undefined Condition” since no condition for No Object and No Reference line is yet defined

Page 90: Robot Vision slide1. Robot Vision Exercise Robot Vision slide3 The task To locate the picking point of the object = Centre of Gravity of the object To

Robot Vision slide90

Adding a NoObject Condition

• Click on ”New” for adding a new condition

• Name it ”No Object”• Select Yellow as

background colour (when this condition is true the result panel will be yellow)

1

2

3

Page 91: Robot Vision slide1. Robot Vision Exercise Robot Vision slide3 The task To locate the picking point of the object = Centre of Gravity of the object To

Robot Vision slide91

Configuring the No Object condition

• To define the No Object condition we can use ObjectPresent Logical classification

• Click on ”New” and select ”ObjectPresent”

• To make the No Object condition true when the logical classification ObjectPresent is false, we invert the statement by pushing the invert button

Page 92: Robot Vision slide1. Robot Vision Exercise Robot Vision slide3 The task To locate the picking point of the object = Centre of Gravity of the object To

Robot Vision slide92

Testing No Object State

• Run through the pictures and check:– the result of the both

conditions (true or false)

– The colour and text in the result panel

Page 93: Robot Vision slide1. Robot Vision Exercise Robot Vision slide3 The task To locate the picking point of the object = Centre of Gravity of the object To

Robot Vision slide93

Adding Cannot Measure Rotation logic condition

• Add a new state– Name it “Cannot

Measure Rotation”

– Select background colour “Red”

Page 94: Robot Vision slide1. Robot Vision Exercise Robot Vision slide3 The task To locate the picking point of the object = Centre of Gravity of the object To

Robot Vision slide94

Configure Cannot Measure Rotation

• The constrains needed are:– Object present– ReferenceLineOK

• “ReferenceLineOK” must be inverted to give the result true (=1 in the result) when the line is failing

Page 95: Robot Vision slide1. Robot Vision Exercise Robot Vision slide3 The task To locate the picking point of the object = Centre of Gravity of the object To

Robot Vision slide95

Testing the logic conditions

• Run through all images and check that all logical conditions are working correctly

Page 96: Robot Vision slide1. Robot Vision Exercise Robot Vision slide3 The task To locate the picking point of the object = Centre of Gravity of the object To

Robot Vision slide96

Displaying measurement values

• Any results from any tool can be displayed in the result panel

• We want to display the inner and outer radius

• Position the mouse inside the result panel and right click, select ”General”, ”No of measumentvalues” and enter 3 for 3 values

• Check the result panel

1 2

3

4

Page 97: Robot Vision slide1. Robot Vision Exercise Robot Vision slide3 The task To locate the picking point of the object = Centre of Gravity of the object To

Robot Vision slide97

Selecting parameters to be displayed

• Position the mouse over Tagvalue 1 and right click, select Measured Values, Parameter.

• In the new dialog box click the ”..” button.

• In the new dialog box you can select the tool and the parameter you want

• Select the tool ReferenceSystem and the parameter Origo_x

1

23

4

5

Page 98: Robot Vision slide1. Robot Vision Exercise Robot Vision slide3 The task To locate the picking point of the object = Centre of Gravity of the object To

Robot Vision slide98

Add name and unit to the value

• Position the mouse over Tag Value 2, right click, select Measured Values and Caption.

• Enter the name ”X value:”

• Through the same procedure you can add unit (mm) and change the number of decimals

• Repeat the procedure for displaying the outer radius as Tag Value 2 and name it ”Y value”

1 2

Page 99: Robot Vision slide1. Robot Vision Exercise Robot Vision slide3 The task To locate the picking point of the object = Centre of Gravity of the object To

Robot Vision slide99

Adding angle value

• Finnaly select Tagvalue 3 and name it “Angle”

• In the tool RotationRefSystem you will find the parameter Angle

• Since the reference to this tool is the robot system (ExternalRefSystem) this value is relative to the robot co-ordinate system

Page 100: Robot Vision slide1. Robot Vision Exercise Robot Vision slide3 The task To locate the picking point of the object = Centre of Gravity of the object To

Robot Vision slide100

Check all conditions

Page 101: Robot Vision slide1. Robot Vision Exercise Robot Vision slide3 The task To locate the picking point of the object = Centre of Gravity of the object To

Robot Vision slide101

Sending data to a Robot or PLC

• We want to send status data (OK, No Object and Cannot Measure Rotation) using the RS232 interface

• When the status is OK we want to send the data x, y and angle

• We will link the timing of sending the data to the event ”Inspection”

• Initially the event inspection activate the command ”Inspect” (running the tools)

• After that we want the system to send status data

Page 102: Robot Vision slide1. Robot Vision Exercise Robot Vision slide3 The task To locate the picking point of the object = Centre of Gravity of the object To

Robot Vision slide102

Adding a RS232 command

• In command sequence for Inspection click on ”New”

• Select ”RS232Cmd”

• Name the command ”SendStatus”

1

23

4

5

Page 103: Robot Vision slide1. Robot Vision Exercise Robot Vision slide3 The task To locate the picking point of the object = Centre of Gravity of the object To

Robot Vision slide103

The command protocol• Syntax

%dparameter– %d = format instruction for

the result value (Integer number format)

– Parameter = The parameter name from any of the tools in the toolbox

• To find the parameter description of the result push the ”Parameters” button on the top toolbar

• Select ObjectPresent and Value and click on the copy button

Page 104: Robot Vision slide1. Robot Vision Exercise Robot Vision slide3 The task To locate the picking point of the object = Centre of Gravity of the object To

Robot Vision slide104

Writing the command protocol

• Open the SendStatus command

• In the Parameter field write the following– Status=%d

• Place the cursor after the d and paste (right click the mouse)

• Status=%dObjectPresent.Value

Page 105: Robot Vision slide1. Robot Vision Exercise Robot Vision slide3 The task To locate the picking point of the object = Centre of Gravity of the object To

Robot Vision slide105

Adding Cannot Measure Rotation

• We can make a 2 bit code where the first bit is Object Present and second Cannot Measure Rotation

• Copy the parameter Value from “ReferenceLineOK” tool

• The parameters must be separated by a comma

• Add %d to format the number

• Paste the parameter

1

2

3

4

Page 106: Robot Vision slide1. Robot Vision Exercise Robot Vision slide3 The task To locate the picking point of the object = Centre of Gravity of the object To

Robot Vision slide106

Testing the communication

• Mark the SendStatus command

• Push the Execute button – Now the command is run once

• In ”Communication” select RS232 - here the out messages are logged

• Check that the correct status value is sent

1

23

4

5

67

8

Page 107: Robot Vision slide1. Robot Vision Exercise Robot Vision slide3 The task To locate the picking point of the object = Centre of Gravity of the object To

Robot Vision slide107

Testing the Scorpion Profile

• Activate automatic inspection (push Start) and check that the status value sent over RS232 is correct for each inspection

Page 108: Robot Vision slide1. Robot Vision Exercise Robot Vision slide3 The task To locate the picking point of the object = Centre of Gravity of the object To

Robot Vision slide108

New command for sending data

• When the logical condition is OK x, y and angle values should be sent also over RS232

• Add a new command after SendStatus– Name: SendData

– Command: RS232

– Guard: ReferenceLineOK

Page 109: Robot Vision slide1. Robot Vision Exercise Robot Vision slide3 The task To locate the picking point of the object = Centre of Gravity of the object To

Robot Vision slide109

Configure the parameters

• %f for floating point number format

• Tool: RotationRefSystem

• Parameters– Origo.x

– Origo.y

– Angle

• Remember to format each parameter and to separate them with a comma

Page 110: Robot Vision slide1. Robot Vision Exercise Robot Vision slide3 The task To locate the picking point of the object = Centre of Gravity of the object To

Robot Vision slide110

Testing the communication

• Mark the SendData command

• Push the Execute button – Now the command is run once

• In ”Communication” select RS232 - here the out messages are logged

• Check that the correct data values are sent

1

23

4

5

Page 111: Robot Vision slide1. Robot Vision Exercise Robot Vision slide3 The task To locate the picking point of the object = Centre of Gravity of the object To

Robot Vision slide111

End of Robot Vision exercise

Congratulations!

You have made your firstRobot Vision system usingScorpion Vision Software