Computer Vision Set: Applications Slides by D.A. Forsyth, C.F. Olson, J. Ponce, L.G. Shapiro Computer Vision and Applications

Computer VisionSet: Applications

Slides by D.A. Forsyth , C.F. Olson, J. Ponce, L.G. Shapiro

Computer Vision and Applications



Computer Vision

Computer vision is • the study of extracting content from digital image data

(my definition)• the analysis of digital images by a computer

(Shapiro’s definition)• the science and technology of machines that see

(Wikipedia definition)

One textbook says:“The goal of computer vision is to make useful decisions about real physical objects and scenes based on sensed images.”



Applications

Image databases



More applications

Robot vision

Mars exploration roverStanley – winner of 2005 DARPAGrand Challenge

http://en.wikipedia.org/wiki/Image:Stanleyrobot.jpg



More applications

Face detection and recognition



More Applications

Surveillance

Modeling for graphics and animation



More Applications

liver

kidney kidney

Medical imagingDocument analysis



More Applications

Photo tourism



More ApplicationsGames

• Industrial inspection• Biometrics (faces, fingerprints)• Motion analysis (including gestures and actions)• Road / traffic analysis• Real-time tracking• Augmented reality• Human-computer interaction• Visual navigation• Image / video indexing and retrieval• Motion capture and entertainment• …

More Applications

Related fields

• Machine vision– Sometimes refers to industrial applications

• Image processing– Transforming one image into another

• Pattern recognition– Concerned with classification or description of observations– Data could be anything (not necessarily images)

• Photogrammetry– Science of obtaining accurate measurements and maps from

photographs (images)





Why Study Computer Vision?

• Images and movies are everywhere• Fast-growing collection of useful applications• Interesting scientific mysteries

– how does object recognition work?

• Better understanding of human vision



Complexity

• Computer vision is far from a solved problem

• Successful systems exist– Usually for controlled situations– Often dependent on parameter settings

• There are many visual tasks that people perform better than computers



Imaging Geometry



Pinhole Cameras

• Abstract camera model - box with a small hole in it

• Pinhole cameras work in practice



Distant Objects Are Smaller



Parallel Lines Meet

Common to draw film planein front of the focal point.Moving the film plane merelyscales the image.



• Cartesian coordinates:– We have, by similar triangles, that:

– (X, Y, Z) ~ (f X/Z, f Y/Z, f)– f is called the focal length.

),(),,(Z

Yf

Z

XfZYX

The equation of projection

[X, Y, Z]

[fX/Z, fY/Z]



The reason for lenses

We won’t worry much about lenses in this class.



Lens distortion

• “Barrel distortion” of rectangular grid is common for inexpensive lenses

• Precision lenses can be expensive

• Zoom lenses often show severe distortion

• Fish-eye lenses also have severe distortion



Image capture• Images are not continuous• Typically captured with a CCD camera (charge-coupled-device)• The amount of light striking each location on a grid is integrated over

some time period• Rows are read out one at a time• For color images, successive

pixels usually correspond to different colors

• High quality color cameras usea beam splitter and 3 separateCCD chips



Resolution

• Resolution often (but not always) refers to the number of pixels in the image.

• Lower resolution has fewer pixels.

• Interestingly, faces of people you know can usually be recognized at 64x64 (or less) pixels.

• Squint and look at the lowest resolution image.

Programming in OpenCV

In OpenCV, images are represented as matrices (as in linear algebra).

Mat image = imread("photo.jpg"); // Most generic declaration

The image could have a number of underlying data types for each pixel:• uchar – unsigned byte (greyscale image)• Vec3b – vector of 3 bytes (color image)• Point2f – point in two dimensions, float• many others…



Creating images

Images can be created using a number of methods:• using namespace cv; // all my code assumes this• Mat image; // creates 0x0 image• Mat image = … // uses copy constructor• Mat image(rows, cols, type); // type is CV_8U, for example• Mat image(rows, cols, type, scalarValue);

– Example: Mat allBlue(360, 480, CV_8UC3, Scalar(255, 0, 0));• Mat_<Vec3b> colorImage = imread(“color.jpg”);

// Can be convenient, but now limited to Vec3b images (matrices)// Also, must declare as a similar parameter type when passed



Copying images

Be careful to remember that most image copy and pass by value methods do NOT perform a deep copy.

image2 = image1; // shallow copyvoid someMethod(Mat imageParam); // shallow copy

If you want a deep copy, then use clone (or copyTo):

image2 = image1.clone(); // deep copyimage1.copyTo(image2); // deep copy



Memory management

Memory management is handled by the Mat class.– This is different from the IplImage class in OpenCV 1– This works correctly even if multiple images share the same data– A reference count is kept for each image– The data is deallocated only when the reference count goes to zero– However, this can allow privacy leaks unless you are careful



Backwards compatibility

OpenCV 2 is backwards compatible with OpenCV 1.

IplImage *iplIm = cvLoadImage("photo.jpg");// Do work with image herecvReleaseImage(&iplIm); // necessary to prevent memory leak

Can convert to Mat simply:

Mat converted(iplIm); // do not release image until finished



Image manipulation

OpenCV provides many methods to manipulate entire images:• Filtering: blur, smooth, median, gradient, laplacian• Transformations: resize, affine, perspective, color space,

threshold, flood fill• Segmentation: grabCut, watershed• Feature detection: edges, corners, lines, circles, template

matching, SIFT



Documents

Computer Vision Set: Applications Slides by D.A. Forsyth, C.F. Olson, J. Ponce, L.G. Shapiro Computer Vision and Applications