30
Robot Vision Module Dott. Emanuele Menegatti ([email protected]) Intelligent Autonomous Systems Lab University of Padua ITALY Based on course notes of Prof. Bob Fisher Edinburgh University - UK

Robot Vision Module Dott. Emanuele Menegatti ([email protected]) Intelligent Autonomous Systems Lab University of Padua ITALY Based on course notes of

Embed Size (px)

Citation preview

Page 1: Robot Vision Module Dott. Emanuele Menegatti (emg@dei.unipd.it) Intelligent Autonomous Systems Lab University of Padua ITALY Based on course notes of

Robot Vision Module

Dott. Emanuele Menegatti([email protected])

Intelligent Autonomous Systems LabUniversity of Padua

ITALY

Based on course notes of Prof. Bob Fisher Edinburgh University - UK

Page 2: Robot Vision Module Dott. Emanuele Menegatti (emg@dei.unipd.it) Intelligent Autonomous Systems Lab University of Padua ITALY Based on course notes of

…a Computer Vision example...

Sorting parts on a conveyor belt

Page 3: Robot Vision Module Dott. Emanuele Menegatti (emg@dei.unipd.it) Intelligent Autonomous Systems Lab University of Padua ITALY Based on course notes of

... A Robot Vision example... Navigation and Obstable avoidance

?

!! Real Time – Real World !!

Page 4: Robot Vision Module Dott. Emanuele Menegatti (emg@dei.unipd.it) Intelligent Autonomous Systems Lab University of Padua ITALY Based on course notes of

Physics of Vision

Page 5: Robot Vision Module Dott. Emanuele Menegatti (emg@dei.unipd.it) Intelligent Autonomous Systems Lab University of Padua ITALY Based on course notes of

Illumination

Page 6: Robot Vision Module Dott. Emanuele Menegatti (emg@dei.unipd.it) Intelligent Autonomous Systems Lab University of Padua ITALY Based on course notes of

The sensors…

Humans Eyelid Iris Lens Retina Optical Nerve

Robots Shutter Iris Lens CCD TV Cable

Page 7: Robot Vision Module Dott. Emanuele Menegatti (emg@dei.unipd.it) Intelligent Autonomous Systems Lab University of Padua ITALY Based on course notes of

The retina…

Horizontal cell

Bipolar cellAmacrine cell

Ganglion cell

Optical Nerve

Light

http://webvision.med.utah.edu/

Page 8: Robot Vision Module Dott. Emanuele Menegatti (emg@dei.unipd.it) Intelligent Autonomous Systems Lab University of Padua ITALY Based on course notes of

QuickTime™ and aTIFF (Uncompressed) decompressor

are needed to see this picture.

Page 9: Robot Vision Module Dott. Emanuele Menegatti (emg@dei.unipd.it) Intelligent Autonomous Systems Lab University of Padua ITALY Based on course notes of

Camera Evolution

Pinhole Camera

http://www.kodak.com/global/en/consumer/education//lessonPlans/pinholeCamera/http://www.pinhole.org/about/index.cfm

Page 10: Robot Vision Module Dott. Emanuele Menegatti (emg@dei.unipd.it) Intelligent Autonomous Systems Lab University of Padua ITALY Based on course notes of

QuickTime™ and aTIFF (Uncompressed) decompressor

are needed to see this picture.

Page 11: Robot Vision Module Dott. Emanuele Menegatti (emg@dei.unipd.it) Intelligent Autonomous Systems Lab University of Padua ITALY Based on course notes of

Camera Evolution The lens Introduced to collects more

lights

The film Introduced to store the image The CCD Introduced to acquire directly a

digital image (higher performances for certain applications)

Page 12: Robot Vision Module Dott. Emanuele Menegatti (emg@dei.unipd.it) Intelligent Autonomous Systems Lab University of Padua ITALY Based on course notes of

How to capture colours

The original (left) image

is split in a beam splitter

Page 13: Robot Vision Module Dott. Emanuele Menegatti (emg@dei.unipd.it) Intelligent Autonomous Systems Lab University of Padua ITALY Based on course notes of

Thin Lens terminology

Optical Axis

Lens Axis

Focal point(Secondary)

Focal length

(in a real camera: Focal length = the distance between the equivalent center of the lens

and the image plane)

Page 14: Robot Vision Module Dott. Emanuele Menegatti (emg@dei.unipd.it) Intelligent Autonomous Systems Lab University of Padua ITALY Based on course notes of

Ray TracingWe will consider a simple system composed of a single thin lens,

there are simple rules we have to follow to ray-trace:1. Rays travel in straight lines and change directions only when

they encounter a discontinuity of the refractive index

2. It is conventional to have the object on the left of the optical system and the image will form on the right

3. All rays emanating from a single point in space must converge on a single point in the image plane (definition of focus)

4. Any ray entering the lens parallel to the axis on one side goes through the focus point on the other side

5. Any ray entering the lens from the focus point on one side emerges parallel to the axis on the other side

 

Page 15: Robot Vision Module Dott. Emanuele Menegatti (emg@dei.unipd.it) Intelligent Autonomous Systems Lab University of Padua ITALY Based on course notes of

How to look at Vision? Low Level Vision:

Middle Level Vision:

High Level Vision:

Image Level Properties

Properties of World

Properties of Actual Objects

–Feature detection–Lightness–Geometry & Shape

–Stereo & 2½D sketch–Motion & Optical Flow–Shape from Shading

–Object Representation–Object Recognition–Geometric Invariance

Page 16: Robot Vision Module Dott. Emanuele Menegatti (emg@dei.unipd.it) Intelligent Autonomous Systems Lab University of Padua ITALY Based on course notes of

Basic Optics

Page 17: Robot Vision Module Dott. Emanuele Menegatti (emg@dei.unipd.it) Intelligent Autonomous Systems Lab University of Padua ITALY Based on course notes of

Images’ FileComposed of: Image File Header

Self-description of the image Image dimensions Image type Date of creation Program that created the image

Image data Some image formats can handle only limited types of images (bynary or monocrome), but the current formats are evolving toward Multimedia contents

Page 18: Robot Vision Module Dott. Emanuele Menegatti (emg@dei.unipd.it) Intelligent Autonomous Systems Lab University of Padua ITALY Based on course notes of

Formati di ImmagineDue tipi fondamentali di immagini o meglio due modalità grafiche per rappresentare le immagini:

Bitmap L’immagine viene descritta dando il colore pixel per pixel

Vettoriale Oggetti presenti nell’immagine descritti in termini matematici

Page 19: Robot Vision Module Dott. Emanuele Menegatti (emg@dei.unipd.it) Intelligent Autonomous Systems Lab University of Padua ITALY Based on course notes of

BitmapCaratteristiche: Risoluzione (ppi) Profondità di colore

Compressione Lossy (JPEG) Lossless (GIF)

Adatto per sfumature di colore

Page 20: Robot Vision Module Dott. Emanuele Menegatti (emg@dei.unipd.it) Intelligent Autonomous Systems Lab University of Padua ITALY Based on course notes of

Bitmap (2)

GIF (Lossless)Graphics Interchange Format

JPEG (Lossy)Joint Photographic Expert Group

Page 21: Robot Vision Module Dott. Emanuele Menegatti (emg@dei.unipd.it) Intelligent Autonomous Systems Lab University of Padua ITALY Based on course notes of

Bitmap (3)GIF

GIF87 256 colori

GIF89 Trasparenza Interlacciamento Immagini multiple (gif animate)

Created by CompuServe Inc.

JPEG Non ha limiti di colore

Adatto per foto Una sola immagine per file

Header può contenere preview

Page 22: Robot Vision Module Dott. Emanuele Menegatti (emg@dei.unipd.it) Intelligent Autonomous Systems Lab University of Padua ITALY Based on course notes of

Immagini Vettoriali

Caratteristiche: Dimensione immagine Descrizione matematica

Compatta Non adatta per foto o immagini ricche di dettagliImmagine vettoriale Bitmap (FACILE)

Bitmap Immagine Vettoriale (DIFFICILE)

Page 23: Robot Vision Module Dott. Emanuele Menegatti (emg@dei.unipd.it) Intelligent Autonomous Systems Lab University of Padua ITALY Based on course notes of

PBM (Portable Bit Map) Family of formats:

PBM Supports monochrome bitmaps (1 bit per pixel). PGM Supports greyscale images PPM Supports full-color images PNM Supports content-independent manipulations on

any of the three formats listed above

P3 # magic number# example from the man page 4 4 # cols & rows15 # maxval0 7 0 015 0 15 0 0 0 0 15

Page 24: Robot Vision Module Dott. Emanuele Menegatti (emg@dei.unipd.it) Intelligent Autonomous Systems Lab University of Padua ITALY Based on course notes of

TIFF (Tag Image File Format) Supports multiple images with 1 to 214 bit depth

Can be lossy or lossless Very general and very complex Used by scanners Created by Aldus Corp.

Page 25: Robot Vision Module Dott. Emanuele Menegatti (emg@dei.unipd.it) Intelligent Autonomous Systems Lab University of Padua ITALY Based on course notes of

PostScript (ps, eps) Store image data using ASCII characters (7-bit ASCII code)

Used for graphics displays and printer

Newer versions include JPEG compression

Used to include graphics or images in a document

Page 26: Robot Vision Module Dott. Emanuele Menegatti (emg@dei.unipd.it) Intelligent Autonomous Systems Lab University of Padua ITALY Based on course notes of

MPEG (Motion Picture Expert Group)

Stream-oriented encoding for video Contains video, text, graphics Created by an international group of industry and governments

Uses Spatial and Temporal Redundancy It is evolving:

MPEG-1: 0.25Mbps (audio) 1.25Mbps (video) MPEG-2: 15Mbps good for TV Future version will recognizes objects and generate their images

Page 27: Robot Vision Module Dott. Emanuele Menegatti (emg@dei.unipd.it) Intelligent Autonomous Systems Lab University of Padua ITALY Based on course notes of

The Difference Between CCD and CMOS CCD high-quality, low-noise images. CMOS more flexible (every pixel can be read independently) The light sensitivity of a CMOS chip is lower (Many of the

photons hitting the chip hit the transistors instead of the photodiode)

CCDs consume 100 times more power than an equivalent CMOS sensor.

CMOS extremely inexpensive compared to CCD sensors (Chips can be fabricated on any standard silicon production line)

CCD sensors produced for a longer period of time, so they are more mature. They tend to have higher quality pixels, and more of them.

Page 28: Robot Vision Module Dott. Emanuele Menegatti (emg@dei.unipd.it) Intelligent Autonomous Systems Lab University of Padua ITALY Based on course notes of

Depth of Field...

… depends on: Shutter Opening Sensitive Element

Focal Length of the lens

Distance of the object

Region of confusion

Hyperfocal distance

Page 29: Robot Vision Module Dott. Emanuele Menegatti (emg@dei.unipd.it) Intelligent Autonomous Systems Lab University of Padua ITALY Based on course notes of

Depth of Field

Page 30: Robot Vision Module Dott. Emanuele Menegatti (emg@dei.unipd.it) Intelligent Autonomous Systems Lab University of Padua ITALY Based on course notes of

Task: Sort Parts Vision Goal: Describe and Identify

Constraints: Flat Shapes Sit flat Different Areas Opacity Serially delivered

Camera

Synch

Silhouette

Detector

Counter

Size Comparator

Threshold

Threshold