Practical Computer Vision: Theory & Applications

Practical Computer Vision:Theory & Applications

Carmen Alonso Montes

23rd-27th November 2015

Carmen Alonso Montes 2

Wrap up

Today, we are here


Learned concepts

Hough Transform Distance mapping

Active contoursWatershed


Contents

Wrap up

Object Recognition

Practical application in industry: code target recognition

Biometrics Recognition

Practical application in authentication applications using medical images

Summary


Object Recognition


Object Recognition

Object recognition is a process for identifying a specific object in a digital image or video.

Object recognition algorithms rely on matching, learning, or pattern recognition algorithms using appearance-based or feature-based techniques

Humans recognize a lot of objects in images with little effort even when

the image may vary in view points, sizes and scales, translated or rotated.

partially obstruction

http://www.engadget.com/2014/09/08/google-details-object-recognition-tech/


Challenges

The main challenges is:

Images are big

Viewing conditions are infinite

Computers are finite

Objects are surrounded by other objects


How does the machine “recognise”?

Once the objects are detected (segmented, edges, skeletons, etc) it is the moment to recognise

Highly dependent on the application

Real-time?

Offline processing

Do we have a database with reference images?

How much is the variability of the images? Do I need any similarity measure?

As engineers, we can go for sophisticated approaches (neural networks) or more simple ones template matching or image registration

Usually techniques from pattern recognition are useful in this area.

https://encrypted-tbn3.gstatic.com/images?q=tbn:ANd9GcTdhVMLS7x7b5avBWy_aZNaF8KD5BN9PWvO6Dq2XHIDhP5HvumiAA



Template matching

Template matching is a technique in digital image processing for finding small parts of an image which match a template.

It can be used in manufacturing as a part of quality control, a way to navigate a mobile robot, or as a way to detect edges in images.

http://goo.gl/mj3Mei

http://goo.gl/7VfQRM


Optical Character Recognition (OCR)

Optical character recognition(optical character reader) (OCR) is a classical example of pattern recognition. Hand made characters shall be recognised by the computer and provided a digital version. First test and approaches were developed to recognise Chinese characters due to its singularity

Nowadays, it is a popular technique thanks to initiatives to put in digitall format old printed books

OCR is a field of research in pattern recognition, artificial intelligence and computer vision.

Resources: Stanford data set: http://ai.stanford.edu/~btaskar/ocr/

Some popular techniques

Matrix matching involves comparing an image to a stored glyph on a pixel-by-pixel basis; it is also known as "pattern matching", "pattern recognition", or "image correlation".

Feature extraction decomposes glyphs into "features" like lines, closed loops, line direction, and line intersections. These are compared with an abstract vector-like representation of a character, which might reduce to one or more glyph prototypes. (e.g. k-nearest neighbors algorithm)

Software such as Cuneiform and Tesseract use a two-pass approach to character recognition. The second pass is known as "adaptive recognition" and uses the letter shapes recognized with high confidence on the first pass to recognize better the remaining letters on the second pass.


Robotics: Obstacle Avoidance

Obstacle avoidance: non-intersection or non-collision

The willow garage project

ROS: Robot operating system

R3-COP - Resilient Reasoning Robotic Co-operating Systems


Collision Avoidance


Practical case in industry


Practical Industrial Case

Problem: Automatic raw part alignment for machining process by means of artificial Coded Targets (CT) within a Phogrammetry-based Application

Published: Miguel Fernandez-Fernandez, Carmen Alonso-Montes, et al.: Industrial Non-intrusive Coded-Target Identification and Decoding Application. IbPRIA 2013: 790-797

http://goo.gl/bwDtVg


What do we need to do as CV Engineers?

Analyze the environment

Where will the camera be placed?

Light conditions?

Pollution?

External factors that can affect my system?

Analyze the elements you have

The camera properties and capacities

Crucial: Calibration phase

The images

Do I have a set of training images ?

Is this training set realistic?

Must I have real time processing?

How do I integrate with the other systems?


Our environment: Schema

Wireless communication among camera & PC

Processing must be done in the camera or the image must have lower resolution


Coded Target (CT)

The reliability of CT for industrial applications comes from its structure

invariant to rotation, scale, and distortion

The central circle target is surrounded by the code band, which is divided into equally spaced angular intervals corresponding to bit positions (meaning 1 for a white interval and 0 for a black one).

The code band is read anticlockwise from a predefined starting point extracting the associated bit pattern, and consequently the associated number.

The bit pattern is 0010111101 which corresponds to the associated number 190.

The exact raw part geometry correspondence is guaranteed thanks to the uniquity of the CT decoded numbers.


Let's start

We already have an black and white image

High resolution image

Objective: read it and get the associated number

Challenges:

Light conditions

Pollution and dust

Occlusions and changes of perspective depending on where is placed


The algorithm

We have split up the system in 2 phases

Phase 1: aims to locate approximately the valid Cts

Phase 2: To read the selected CT from Phase 1


Phase 1: Where are my Cts?

The bilateral filtering is an edge preserving and noise reducing smoothing filter, which performs specifically well in poor illumination conditions and the pollution type of this industrial raw part images.

An adaptive threshold algorithm (mean value of all pixels in its 5x5 neighbourhood) is applied to binarise the image.

Then, erosion and dilation operations are performed in order to remove the salt-and-pepper noise.


Identification of the potential CT

Search for the CT and to compute the surrounding area (ROI). The CT identification is done based on the analysis of the detected contour structures.

valid or non-valid CT structures according to their conformance with an ideal CT pattern

Circularity factor : minimum enclosing technique to distinguish those contours that are actually circles

Three concentric circles

Ratio descriptors: distance ratio among circles and diameter ratio of the central target


Phase 2: My identified CTs, let's read them

Step 2. Image denoising

Noise will be removed by means of the bilateral filter.

Image is binarised by the adaptive threshold and then,

Erosion and dilation operations are performed to remove noise.

Step 3 performs a contour identification.

Minimum enclosing circle technique.

The CT center coordinates must be the same for all the circles.

In order to compute the exact center, the momentum of the image is used, in particular, the centroid or center of mass of the CT

The Step 4 will perform the analysis of the CT structure for its number decoding.


Examples


Biometrics


Biometrics

Biometrics refers to metrics related to human characteristics.

Biometrics authentication (or realistic authentication) is used in computer science as a form of identification and access contro, to identify individuals in groups that are under surveillance.

Biometric identifiers are the distinctive, measurable characteristics used to label and describe individuals

fingerprint, palm veins, face recognition, DNA, palm print, hand geometry, iris recognition, retina and odour/scent.

biometric identifiers

unique to individuals

more reliable in verifying identity

Challenge: personal privacy concerns about the ultimate use of this information.


Fingerprint Recognition

http://www.nitgen.com/eng/technology/img/02_img_01.jpg


Face Recognition

http://news.bbc.co.uk/nol/shared/spl/hi/guides/456900/456993/img/noflash/facial_scan_416_dc.gif


Example

http://biometrics.mainguet.org/types/face/face_visualskinprint.jpg


Iris Recognition

http://4.bp.blogspot.com/-Yb-isbxtQoQ/VPqDNJlcf5I/AAAAAAAAAMQ/OSq7mozt11o/s1600/iris%2Brecognition%2Bsystem%2Balgorithm.gif


Practical case: biomedical authentication based on retina pattern


Retinal Authentication


Practical case: Retinal AuthenticationMinority Report → Future or reality?


The problem

We want to use the retinal vessel tree to identify persons using the retina vessel pattern.


Thinking about the problem

Images

Segmentation issues

Image acquisition

Vessel

Diseases

Central reflex

How do I get the minutae?

How do I register and get the authentication?

Validation of the system.

How well do I perform?


My schema


Processing the retinal images

Retinal images can be obtained by several methods

We are only interested in those points of the vessels corresponding to junctions

Authentication using images → the same philosophy than fingerprints

We only need crossing points


Challenges (I)

Central reflex

Small vessels


Challenges (II)

Pathologies can make that the images under study does not comply with our minimum requirements

Optic disk usually is not considered in global retinal process due to its particular brigthness


Our specific needs


Vessel skeleton

Stage 1 is devoted to clean the vessels from noise, and to homogeneize vessel locations

The results will be used to compute the initial contours and the external energy needed by the active contours

Pixel Level Snakes are a specific active contours technique were all the pixels of the contours evolve

No nodes are needed in this case


Phase 1. Vessel pre-estimation

Diffusion acts as a kind of smoothing kernel

Goal: remove noise due to the acquisition device

The thresholded image is obtained through adaptive segmentation due to the high variability in the different areas of the retina


Phase 2. Computing the input for the Active Contours

Active contours need

Initial contour

External Potential

Initial contour must be close to vessel locations

External Potential must guide the contours towards the real edges


Computing the External Energy

Simple operations were used since only addition, substraction, multiplication and neighbourhood operations were available in the SCAMP device

Distance mapping to the real edges was computed with several dilations over the detected edges


The evolution of the active contours was splitted in 2 steps

Fast evolution (mainly guided by external energy only)

Hole filling to remove noisy regions

Slow evolution to fit the details


Retinal vessel skeleton


Computing the Skeleton


Minutae in Retina

Skeletonised image barely is optimal

Discontinuities usually appears

Two types discontinuities

Union of the same vessel

Junctions


Results

No possibility of fraud

No false positives


Conclusion

Segmentation techniques are in the core of a good range of computer vision applications, and in particular for object recognition

Different images from industrial and from medical domain benefit from segmentation for

Normalizing intensities

Segmenting important objects for their applications

Usually, computer vision engineers needs to thing in the big picture before start doing

Think in your needs and in what you have

Go for a simple approach, and then try to improve it moving towards more sophisticated algorithms to fulfill your needs

After this course, you have a minimum knowledge about classical techniques used in the literature

Try to play with the classical ones and move forward to more exciting approaches

Documents

Practical Computer Vision: Theory & Applications