Upload
brian-thorne
View
19.832
Download
4
Tags:
Embed Size (px)
DESCRIPTION
An introduction to computer vision in Python, from the general concept to its implementation with some current open-source libraries. Demonstrates a selection of basic computer vision examples using SciPy, OpenCV and Pygame.
Citation preview
Motivation & BackgroundComputer Vision in Python
More InformationSummary
Introduction to using Python in Computer VisionKiwi PyCon, Christchurch, 2009
Brian Thorne
University of Canterbury
6th November 2009
Brian Thorne Computer Vision in Python
Motivation & BackgroundComputer Vision in Python
More InformationSummary
Outline
1 Motivation & BackgroundWhat is Computer Vision?Uses & Examples
2 Computer Vision in PythonToolsImage AcquisitionImage FilteringBackground SubtractionFeature Point Detection
3 More InformationDifferent platformsAdditional Tools
Brian Thorne Computer Vision in Python
Motivation & BackgroundComputer Vision in Python
More InformationSummary
Computer VisionUses & Examples
Outline
1 Motivation & BackgroundWhat is Computer Vision?Uses & Examples
2 Computer Vision in PythonToolsImage AcquisitionImage FilteringBackground SubtractionFeature Point Detection
3 More InformationDifferent platformsAdditional Tools
Brian Thorne Computer Vision in Python
Motivation & BackgroundComputer Vision in Python
More InformationSummary
Computer VisionUses & Examples
Vision
25% of the whole brain is for vision. Around 50% of cerebralcortex is for vision, 80% of the brain is associated with visionin some manner.
Brian Thorne Computer Vision in Python
Computer Vision
DefinitionThe goal of computer vision is to recognize objects and their motion
What is it used for?Scene reconstructionEvent detectionVideo trackingObject recognitionLearningIndexingMotion estimationImage restoration
Computer Vision
DefinitionThe goal of computer vision is to recognize objects and their motion
What is it used for?Scene reconstructionEvent detectionVideo trackingObject recognitionLearningIndexingMotion estimationImage restoration
Motivation & BackgroundComputer Vision in Python
More InformationSummary
Computer VisionUses & Examples
Computer Vision crosses over with many domains
Brian Thorne Computer Vision in Python
Motivation & BackgroundComputer Vision in Python
More InformationSummary
Computer VisionUses & Examples
What makes it hard?
What we see What the computer sees
Brian Thorne Computer Vision in Python
Motivation & BackgroundComputer Vision in Python
More InformationSummary
Computer VisionUses & Examples
Vision is inferential
http://web.mit.edu/persci/people/adelson/checkershadow_illusion.html
Brian Thorne Computer Vision in Python
Motivation & BackgroundComputer Vision in Python
More InformationSummary
Computer VisionUses & Examples
Outline
1 Motivation & BackgroundWhat is Computer Vision?Uses & Examples
2 Computer Vision in PythonToolsImage AcquisitionImage FilteringBackground SubtractionFeature Point Detection
3 More InformationDifferent platformsAdditional Tools
Brian Thorne Computer Vision in Python
Motivation & BackgroundComputer Vision in Python
More InformationSummary
Computer VisionUses & Examples
Visual Object Classes Challenge 09
http://www.pascal-network.org/challenges/VOC/voc2009
Brian Thorne Computer Vision in Python
Motivation & BackgroundComputer Vision in Python
More InformationSummary
Computer VisionUses & Examples
Object Recognition and Segmentation - Texture
−−−−−−−→
(Sharon, Balun, Brandt, Basri)
Brian Thorne Computer Vision in Python
Motivation & BackgroundComputer Vision in Python
More InformationSummary
Computer VisionUses & Examples
Object Recognition and Segmentation - Edges
http://www.robots.ox.ac.uk/~vdg/dynamics.html
Brian Thorne Computer Vision in Python
Motivation & BackgroundComputer Vision in Python
More InformationSummary
Computer VisionUses & Examples
Traffic Monitoring
Brian Thorne Computer Vision in Python
Motivation & BackgroundComputer Vision in Python
More InformationSummary
Computer VisionUses & Examples
Augented Reality - Sixth Sense
’SixthSense’ is a wearable gestural interface that augments thephysical world around us with digital information and lets us use
natural hand gestures to interact with that information.
http://www.pranavmistry.com/projects/sixthsense/
Brian Thorne Computer Vision in Python
Motivation & BackgroundComputer Vision in Python
More InformationSummary
ToolsImage AcquisitionImage FilteringBackground SubtractionFeature Point Detection
Outline
1 Motivation & BackgroundWhat is Computer Vision?Uses & Examples
2 Computer Vision in PythonToolsImage AcquisitionImage FilteringBackground SubtractionFeature Point Detection
3 More InformationDifferent platformsAdditional Tools
Brian Thorne Computer Vision in Python
Python In Computer Vision: OpenCV
Provides well tested, optimized andopen source code for image processingand computer visionWritten in C, ensuring both fast andportable.Has been compiled for manyembedded platformsHas multiple language wrappersincluding 3 for PythonTools have been made to use graphicshardware to accelerate CVperformance on the GPU
Project home page and documentation is at:http://opencv.willowgarage.com
Python In Computer Vision: OpenCV
Provides well tested, optimized andopen source code for image processingand computer visionWritten in C, ensuring both fast andportable.Has been compiled for manyembedded platformsHas multiple language wrappersincluding 3 for PythonTools have been made to use graphicshardware to accelerate CVperformance on the GPU
Project home page and documentation is at:http://opencv.willowgarage.com
Python In Computer Vision: OpenCV
Provides well tested, optimized andopen source code for image processingand computer visionWritten in C, ensuring both fast andportable.Has been compiled for manyembedded platformsHas multiple language wrappersincluding 3 for PythonTools have been made to use graphicshardware to accelerate CVperformance on the GPU
Project home page and documentation is at:http://opencv.willowgarage.com
Python In Computer Vision: OpenCV
Provides well tested, optimized andopen source code for image processingand computer visionWritten in C, ensuring both fast andportable.Has been compiled for manyembedded platformsHas multiple language wrappersincluding 3 for PythonTools have been made to use graphicshardware to accelerate CVperformance on the GPU
Project home page and documentation is at:http://opencv.willowgarage.com
Python In Computer Vision: OpenCV
Provides well tested, optimized andopen source code for image processingand computer visionWritten in C, ensuring both fast andportable.Has been compiled for manyembedded platformsHas multiple language wrappersincluding 3 for PythonTools have been made to use graphicshardware to accelerate CVperformance on the GPU
Project home page and documentation is at:http://opencv.willowgarage.com
Motivation & BackgroundComputer Vision in Python
More InformationSummary
ToolsImage AcquisitionImage FilteringBackground SubtractionFeature Point Detection
Python In Computer Vision: Numpy & Scipy
Gives strongly typed N-dimensionalarrays to PythonWell used and tested libraries forscientific computingIncludes lots of handy tools such asoptimisation and signal processingused often in computer vision.Usually used with iPython andmatplotlib
SciPy can be downloaded from: http://www.scipy.org
Brian Thorne Computer Vision in Python
Motivation & BackgroundComputer Vision in Python
More InformationSummary
ToolsImage AcquisitionImage FilteringBackground SubtractionFeature Point Detection
Python In Computer Vision: Numpy & Scipy
Gives strongly typed N-dimensionalarrays to PythonWell used and tested libraries forscientific computingIncludes lots of handy tools such asoptimisation and signal processingused often in computer vision.Usually used with iPython andmatplotlib
SciPy can be downloaded from: http://www.scipy.org
Brian Thorne Computer Vision in Python
Motivation & BackgroundComputer Vision in Python
More InformationSummary
ToolsImage AcquisitionImage FilteringBackground SubtractionFeature Point Detection
Python In Computer Vision: Numpy & Scipy
Gives strongly typed N-dimensionalarrays to PythonWell used and tested libraries forscientific computingIncludes lots of handy tools such asoptimisation and signal processingused often in computer vision.Usually used with iPython andmatplotlib
SciPy can be downloaded from: http://www.scipy.org
Brian Thorne Computer Vision in Python
Motivation & BackgroundComputer Vision in Python
More InformationSummary
ToolsImage AcquisitionImage FilteringBackground SubtractionFeature Point Detection
Python In Computer Vision: Numpy & Scipy
Gives strongly typed N-dimensionalarrays to PythonWell used and tested libraries forscientific computingIncludes lots of handy tools such asoptimisation and signal processingused often in computer vision.Usually used with iPython andmatplotlib
SciPy can be downloaded from: http://www.scipy.org
Brian Thorne Computer Vision in Python
Motivation & BackgroundComputer Vision in Python
More InformationSummary
ToolsImage AcquisitionImage FilteringBackground SubtractionFeature Point Detection
Python In Computer Vision: Numpy & Scipy
Gives strongly typed N-dimensionalarrays to PythonWell used and tested libraries forscientific computingIncludes lots of handy tools such asoptimisation and signal processingused often in computer vision.Usually used with iPython andmatplotlib
SciPy can be downloaded from: http://www.scipy.org
Brian Thorne Computer Vision in Python
Motivation & BackgroundComputer Vision in Python
More InformationSummary
ToolsImage AcquisitionImage FilteringBackground SubtractionFeature Point Detection
Python In Computer Vision: Pygame
Game development frameworkNow has basic Computer Vision supportBeing Python it can be used with other Python tools -integrates well with numpy/scipy
pygame can be downloaded from: http://www.pygame.org
Brian Thorne Computer Vision in Python
Motivation & BackgroundComputer Vision in Python
More InformationSummary
ToolsImage AcquisitionImage FilteringBackground SubtractionFeature Point Detection
Python In Computer Vision: Pygame
Game development frameworkNow has basic Computer Vision supportBeing Python it can be used with other Python tools -integrates well with numpy/scipy
pygame can be downloaded from: http://www.pygame.org
Brian Thorne Computer Vision in Python
Motivation & BackgroundComputer Vision in Python
More InformationSummary
ToolsImage AcquisitionImage FilteringBackground SubtractionFeature Point Detection
Python In Computer Vision: Pygame
Game development frameworkNow has basic Computer Vision supportBeing Python it can be used with other Python tools -integrates well with numpy/scipy
pygame can be downloaded from: http://www.pygame.org
Brian Thorne Computer Vision in Python
Motivation & BackgroundComputer Vision in Python
More InformationSummary
ToolsImage AcquisitionImage FilteringBackground SubtractionFeature Point Detection
Pycam
This is the project with all the examples for this presentation. Hasa bunch of simple examples like filtering and backgroundsubtraction, face detection.
Contains two video player classes that can work with differentbackend setups, and can incorporate optional processfunctions.Examples of intergrating OpenCV with pygame - eg for eyeand face detection.OpenCV camera class that allows an opencv camera to beused with pygame (No longer required in latest pygame)
VideoCapturePlayerFor the rest of this presentation, examples will use the videocapture code (with error checking) from pycam.
http://pycam.googlecode.comBrian Thorne Computer Vision in Python
Motivation & BackgroundComputer Vision in Python
More InformationSummary
ToolsImage AcquisitionImage FilteringBackground SubtractionFeature Point Detection
Pycam
This is the project with all the examples for this presentation. Hasa bunch of simple examples like filtering and backgroundsubtraction, face detection.
Contains two video player classes that can work with differentbackend setups, and can incorporate optional processfunctions.Examples of intergrating OpenCV with pygame - eg for eyeand face detection.OpenCV camera class that allows an opencv camera to beused with pygame (No longer required in latest pygame)
VideoCapturePlayerFor the rest of this presentation, examples will use the videocapture code (with error checking) from pycam.
http://pycam.googlecode.comBrian Thorne Computer Vision in Python
Motivation & BackgroundComputer Vision in Python
More InformationSummary
ToolsImage AcquisitionImage FilteringBackground SubtractionFeature Point Detection
Pycam
This is the project with all the examples for this presentation. Hasa bunch of simple examples like filtering and backgroundsubtraction, face detection.
Contains two video player classes that can work with differentbackend setups, and can incorporate optional processfunctions.Examples of intergrating OpenCV with pygame - eg for eyeand face detection.OpenCV camera class that allows an opencv camera to beused with pygame (No longer required in latest pygame)
VideoCapturePlayerFor the rest of this presentation, examples will use the videocapture code (with error checking) from pycam.
http://pycam.googlecode.comBrian Thorne Computer Vision in Python
Motivation & BackgroundComputer Vision in Python
More InformationSummary
ToolsImage AcquisitionImage FilteringBackground SubtractionFeature Point Detection
Pycam
This is the project with all the examples for this presentation. Hasa bunch of simple examples like filtering and backgroundsubtraction, face detection.
Contains two video player classes that can work with differentbackend setups, and can incorporate optional processfunctions.Examples of intergrating OpenCV with pygame - eg for eyeand face detection.OpenCV camera class that allows an opencv camera to beused with pygame (No longer required in latest pygame)
VideoCapturePlayerFor the rest of this presentation, examples will use the videocapture code (with error checking) from pycam.
http://pycam.googlecode.comBrian Thorne Computer Vision in Python
Motivation & BackgroundComputer Vision in Python
More InformationSummary
ToolsImage AcquisitionImage FilteringBackground SubtractionFeature Point Detection
Outline
1 Motivation & BackgroundWhat is Computer Vision?Uses & Examples
2 Computer Vision in PythonToolsImage AcquisitionImage FilteringBackground SubtractionFeature Point Detection
3 More InformationDifferent platformsAdditional Tools
Brian Thorne Computer Vision in Python
Motivation & BackgroundComputer Vision in Python
More InformationSummary
ToolsImage AcquisitionImage FilteringBackground SubtractionFeature Point Detection
Acquiring & Display Of An Image
Live image acquisition is such a crucialrole in the majority of CV applications.
Example getting and showing a frameas a most basic, but necessary test
Brian Thorne Computer Vision in Python
Motivation & BackgroundComputer Vision in Python
More InformationSummary
ToolsImage AcquisitionImage FilteringBackground SubtractionFeature Point Detection
Acquiring & Display Of An Image
Live image acquisition is such a crucialrole in the majority of CV applications.
Example getting and showing a frameas a most basic, but necessary test
Brian Thorne Computer Vision in Python
Motivation & BackgroundComputer Vision in Python
More InformationSummary
ToolsImage AcquisitionImage FilteringBackground SubtractionFeature Point Detection
Python OpenCV: Image Capture
Examplefrom opencv import highgui as hgcapture = hg.cvCreateCameraCapture(0)hg.cvNamedWindow("Snapshot")frame = hg.cvQueryFrame(capture)hg.cvShowImage("Snapshot", frame)hg.cvWaitKey(10000)
Brian Thorne Computer Vision in Python
Motivation & BackgroundComputer Vision in Python
More InformationSummary
ToolsImage AcquisitionImage FilteringBackground SubtractionFeature Point Detection
Outline
1 Motivation & BackgroundWhat is Computer Vision?Uses & Examples
2 Computer Vision in PythonToolsImage AcquisitionImage FilteringBackground SubtractionFeature Point Detection
3 More InformationDifferent platformsAdditional Tools
Brian Thorne Computer Vision in Python
Motivation & BackgroundComputer Vision in Python
More InformationSummary
ToolsImage AcquisitionImage FilteringBackground SubtractionFeature Point Detection
Filtering - Gaussian Blur
One of the simplest operations in image processing is blurringan image
Reduce noise,Remove artifacts
Scale an image“cleanly”
Create motion blur -if done in onedirection
OpenCV includes a gaussian filter among many others(cvSmooth function)SciPy has a multi-dimensional Gaussian filter that acts on aNumPy arrayOr you could convolve an image with a filter manually
Brian Thorne Computer Vision in Python
Motivation & BackgroundComputer Vision in Python
More InformationSummary
ToolsImage AcquisitionImage FilteringBackground SubtractionFeature Point Detection
Filtering - Gaussian Blur
One of the simplest operations in image processing is blurringan image
Reduce noise,Remove artifacts
Scale an image“cleanly”
Create motion blur -if done in onedirection
OpenCV includes a gaussian filter among many others(cvSmooth function)SciPy has a multi-dimensional Gaussian filter that acts on aNumPy arrayOr you could convolve an image with a filter manually
Brian Thorne Computer Vision in Python
Motivation & BackgroundComputer Vision in Python
More InformationSummary
ToolsImage AcquisitionImage FilteringBackground SubtractionFeature Point Detection
Filtering - Gaussian Blur
One of the simplest operations in image processing is blurringan image
Reduce noise,Remove artifacts
Scale an image“cleanly”
Create motion blur -if done in onedirection
OpenCV includes a gaussian filter among many others(cvSmooth function)SciPy has a multi-dimensional Gaussian filter that acts on aNumPy arrayOr you could convolve an image with a filter manually
Brian Thorne Computer Vision in Python
Motivation & BackgroundComputer Vision in Python
More InformationSummary
ToolsImage AcquisitionImage FilteringBackground SubtractionFeature Point Detection
Filtering - Gaussian Blur
One of the simplest operations in image processing is blurringan image
Reduce noise,Remove artifacts
Scale an image“cleanly”
Create motion blur -if done in onedirection
OpenCV includes a gaussian filter among many others(cvSmooth function)SciPy has a multi-dimensional Gaussian filter that acts on aNumPy arrayOr you could convolve an image with a filter manually
Brian Thorne Computer Vision in Python
Motivation & BackgroundComputer Vision in Python
More InformationSummary
ToolsImage AcquisitionImage FilteringBackground SubtractionFeature Point Detection
Filtering - Gaussian Blur
One of the simplest operations in image processing is blurringan image
Reduce noise,Remove artifacts
Scale an image“cleanly”
Create motion blur -if done in onedirection
OpenCV includes a gaussian filter among many others(cvSmooth function)SciPy has a multi-dimensional Gaussian filter that acts on aNumPy arrayOr you could convolve an image with a filter manually
Brian Thorne Computer Vision in Python
Motivation & BackgroundComputer Vision in Python
More InformationSummary
ToolsImage AcquisitionImage FilteringBackground SubtractionFeature Point Detection
Filtering - Gaussian Blur
One of the simplest operations in image processing is blurringan image
Reduce noise,Remove artifacts
Scale an image“cleanly”
Create motion blur -if done in onedirection
OpenCV includes a gaussian filter among many others(cvSmooth function)SciPy has a multi-dimensional Gaussian filter that acts on aNumPy arrayOr you could convolve an image with a filter manually
Brian Thorne Computer Vision in Python
Motivation & BackgroundComputer Vision in Python
More InformationSummary
ToolsImage AcquisitionImage FilteringBackground SubtractionFeature Point Detection
Applying a Gaussian Blur with OpenCV
OpenCV Gaussian Blurfrom pycam import VideoCapturePlayer as VCPfrom opencv import cv
def gaussianBlur(im, filterSize=43):result = cv.cvCreateMat(im.rows, im.cols,
im.type )cv.cvSmooth(image,result,
cv.CV_GAUSSIAN, filterSize)return result
if __name__ == "__main__":VCP(gaussianBlur, "Guassian Filter").main()
Brian Thorne Computer Vision in Python
Motivation & BackgroundComputer Vision in Python
More InformationSummary
ToolsImage AcquisitionImage FilteringBackground SubtractionFeature Point Detection
Applying a Gaussian Blur with Scipy
SciPy Gaussian Blur
from scipy.ndimage.filters import gaussian_filterfrom pycam import OpencvVideoCapturePlayer as VCPfrom misc import scipyFromOpenCV
@scipyFromOpenCVdef gaussianBlur(np_image):
result = gaussian_filter(np_image,sigma=(4, 4, 0),order=0, mode=’reflect’)
return result
if __name__ == "__main__":VCP(gaussianBlur,"Scipy Guassian Blur").main()
Brian Thorne Computer Vision in Python
Motivation & BackgroundComputer Vision in Python
More InformationSummary
ToolsImage AcquisitionImage FilteringBackground SubtractionFeature Point Detection
Outline
1 Motivation & BackgroundWhat is Computer Vision?Uses & Examples
2 Computer Vision in PythonToolsImage AcquisitionImage FilteringBackground SubtractionFeature Point Detection
3 More InformationDifferent platformsAdditional Tools
Brian Thorne Computer Vision in Python
Motivation & BackgroundComputer Vision in Python
More InformationSummary
ToolsImage AcquisitionImage FilteringBackground SubtractionFeature Point Detection
Background Subtraction
In video security the camera mostly looks at the same boringbackgroundWhat we are usually interested in is when objects (eg people orvehicles) enter or exit a sceneAim is to isolate the interesting, and ignore the boringAt the most simple level background subtraction is simply acomparison between two image framesAt the more complex level many people have gotten phd’s forbetter background learning techniques, and better differencingalgorithms
Brian Thorne Computer Vision in Python
Motivation & BackgroundComputer Vision in Python
More InformationSummary
ToolsImage AcquisitionImage FilteringBackground SubtractionFeature Point Detection
Background Subtraction
In video security the camera mostly looks at the same boringbackgroundWhat we are usually interested in is when objects (eg people orvehicles) enter or exit a sceneAim is to isolate the interesting, and ignore the boringAt the most simple level background subtraction is simply acomparison between two image framesAt the more complex level many people have gotten phd’s forbetter background learning techniques, and better differencingalgorithms
Brian Thorne Computer Vision in Python
Motivation & BackgroundComputer Vision in Python
More InformationSummary
ToolsImage AcquisitionImage FilteringBackground SubtractionFeature Point Detection
Background Subtraction
In video security the camera mostly looks at the same boringbackgroundWhat we are usually interested in is when objects (eg people orvehicles) enter or exit a sceneAim is to isolate the interesting, and ignore the boringAt the most simple level background subtraction is simply acomparison between two image framesAt the more complex level many people have gotten phd’s forbetter background learning techniques, and better differencingalgorithms
Brian Thorne Computer Vision in Python
Motivation & BackgroundComputer Vision in Python
More InformationSummary
ToolsImage AcquisitionImage FilteringBackground SubtractionFeature Point Detection
Background Subtraction
Simple Frame Differencing1 To let the camera adjust to light levels, ignore the first few
frames.2 Store a frame as the base frame.3 For each new frame that comes in:
1 Take the absolute intensity difference in each channel(Red/Green/Blue) with the base frame.
2 Binary threshold to ignore pixels that are only a bit different.3 Convert difference image to a one channel mask4 Clean up small noise areas in the mask (with median filter,
erode, connected components)5 Return the changed pixels from the original image using the
created mask.
Brian Thorne Computer Vision in Python
Motivation & BackgroundComputer Vision in Python
More InformationSummary
ToolsImage AcquisitionImage FilteringBackground SubtractionFeature Point Detection
Background Subtraction
Simple Frame Differencing1 To let the camera adjust to light levels, ignore the first few
frames.2 Store a frame as the base frame.3 For each new frame that comes in:
1 Take the absolute intensity difference in each channel(Red/Green/Blue) with the base frame.
2 Binary threshold to ignore pixels that are only a bit different.3 Convert difference image to a one channel mask4 Clean up small noise areas in the mask (with median filter,
erode, connected components)5 Return the changed pixels from the original image using the
created mask.
Brian Thorne Computer Vision in Python
Motivation & BackgroundComputer Vision in Python
More InformationSummary
ToolsImage AcquisitionImage FilteringBackground SubtractionFeature Point Detection
Background Subtraction
Simple Frame Differencing1 To let the camera adjust to light levels, ignore the first few
frames.2 Store a frame as the base frame.3 For each new frame that comes in:
1 Take the absolute intensity difference in each channel(Red/Green/Blue) with the base frame.
2 Binary threshold to ignore pixels that are only a bit different.3 Convert difference image to a one channel mask4 Clean up small noise areas in the mask (with median filter,
erode, connected components)5 Return the changed pixels from the original image using the
created mask.
Brian Thorne Computer Vision in Python
Motivation & BackgroundComputer Vision in Python
More InformationSummary
ToolsImage AcquisitionImage FilteringBackground SubtractionFeature Point Detection
Background Subtraction
Simple Frame Differencing1 To let the camera adjust to light levels, ignore the first few
frames.2 Store a frame as the base frame.3 For each new frame that comes in:
1 Take the absolute intensity difference in each channel(Red/Green/Blue) with the base frame.
2 Binary threshold to ignore pixels that are only a bit different.3 Convert difference image to a one channel mask4 Clean up small noise areas in the mask (with median filter,
erode, connected components)5 Return the changed pixels from the original image using the
created mask.
Brian Thorne Computer Vision in Python
Motivation & BackgroundComputer Vision in Python
More InformationSummary
ToolsImage AcquisitionImage FilteringBackground SubtractionFeature Point Detection
Background Subtraction
Simple Frame Differencing1 To let the camera adjust to light levels, ignore the first few
frames.2 Store a frame as the base frame.3 For each new frame that comes in:
1 Take the absolute intensity difference in each channel(Red/Green/Blue) with the base frame.
2 Binary threshold to ignore pixels that are only a bit different.3 Convert difference image to a one channel mask4 Clean up small noise areas in the mask (with median filter,
erode, connected components)5 Return the changed pixels from the original image using the
created mask.
Brian Thorne Computer Vision in Python
Motivation & BackgroundComputer Vision in Python
More InformationSummary
ToolsImage AcquisitionImage FilteringBackground SubtractionFeature Point Detection
Background Subtraction
Simple Frame Differencing1 To let the camera adjust to light levels, ignore the first few
frames.2 Store a frame as the base frame.3 For each new frame that comes in:
1 Take the absolute intensity difference in each channel(Red/Green/Blue) with the base frame.
2 Binary threshold to ignore pixels that are only a bit different.3 Convert difference image to a one channel mask4 Clean up small noise areas in the mask (with median filter,
erode, connected components)5 Return the changed pixels from the original image using the
created mask.
Brian Thorne Computer Vision in Python
Motivation & BackgroundComputer Vision in Python
More InformationSummary
ToolsImage AcquisitionImage FilteringBackground SubtractionFeature Point Detection
Background Subtraction
Simple Frame Differencing1 To let the camera adjust to light levels, ignore the first few
frames.2 Store a frame as the base frame.3 For each new frame that comes in:
1 Take the absolute intensity difference in each channel(Red/Green/Blue) with the base frame.
2 Binary threshold to ignore pixels that are only a bit different.3 Convert difference image to a one channel mask4 Clean up small noise areas in the mask (with median filter,
erode, connected components)5 Return the changed pixels from the original image using the
created mask.
Brian Thorne Computer Vision in Python
Motivation & BackgroundComputer Vision in Python
More InformationSummary
ToolsImage AcquisitionImage FilteringBackground SubtractionFeature Point Detection
Background Subtraction
Simple Frame Differencing1 To let the camera adjust to light levels, ignore the first few
frames.2 Store a frame as the base frame.3 For each new frame that comes in:
1 Take the absolute intensity difference in each channel(Red/Green/Blue) with the base frame.
2 Binary threshold to ignore pixels that are only a bit different.3 Convert difference image to a one channel mask4 Clean up small noise areas in the mask (with median filter,
erode, connected components)5 Return the changed pixels from the original image using the
created mask.
Brian Thorne Computer Vision in Python
Motivation & BackgroundComputer Vision in Python
More InformationSummary
ToolsImage AcquisitionImage FilteringBackground SubtractionFeature Point Detection
Background Subtract
Here I have placed a cellphone on mycluttered deskCan’t tell thats there is no green screenQuick demo
Brian Thorne Computer Vision in Python
Motivation & BackgroundComputer Vision in Python
More InformationSummary
ToolsImage AcquisitionImage FilteringBackground SubtractionFeature Point Detection
Outline
1 Motivation & BackgroundWhat is Computer Vision?Uses & Examples
2 Computer Vision in PythonToolsImage AcquisitionImage FilteringBackground SubtractionFeature Point Detection
3 More InformationDifferent platformsAdditional Tools
Brian Thorne Computer Vision in Python
Motivation & BackgroundComputer Vision in Python
More InformationSummary
ToolsImage AcquisitionImage FilteringBackground SubtractionFeature Point Detection
Feature Point Detection
Feature point detection is implementedin OpenCV you can do it in one call:cvCornerHarris or cvGoodFeaturesTo demonstrate the algorithm though -we will go look at it in scipy.Implementation derived from JanSolem
Brian Thorne Computer Vision in Python
Feature Detection
1 First convert to a grey scaleimage
2 Showing the derivative in the xand y directions
3 showing the millions of points ofinterest
4 filtering them
Motivation & BackgroundComputer Vision in Python
More InformationSummary
ToolsImage AcquisitionImage FilteringBackground SubtractionFeature Point Detection
Augmented Reality
Augmented reality is undergoing massivegrowthOpenCV provides the face detectionAn AR game can easily be made in Pygameusing the webcam and face location as theinterface
Brian Thorne Computer Vision in Python
Motivation & BackgroundComputer Vision in Python
More InformationSummary
Different platformsAdditional Tools
Outline
1 Motivation & BackgroundWhat is Computer Vision?Uses & Examples
2 Computer Vision in PythonToolsImage AcquisitionImage FilteringBackground SubtractionFeature Point Detection
3 More InformationDifferent platformsAdditional Tools
Brian Thorne Computer Vision in Python
Motivation & BackgroundComputer Vision in Python
More InformationSummary
Different platformsAdditional Tools
Running on an OLPC
OLPC - provide children in developingnations with access to knowledge, andopportunities to "explore, experimentand express themselves"Includes Python and a webcam - thatsall you need for computer vision!Here I am running OpenCV’sfacedetection on the XO laptopLots of Computer Vision work on theXO has been done using pygame byNirav Patel (http://eclecti.cc/olpc)
Brian Thorne Computer Vision in Python
Motivation & BackgroundComputer Vision in Python
More InformationSummary
Different platformsAdditional Tools
Running on an OLPC
OLPC - provide children in developingnations with access to knowledge, andopportunities to "explore, experimentand express themselves"Includes Python and a webcam - thatsall you need for computer vision!Here I am running OpenCV’sfacedetection on the XO laptopLots of Computer Vision work on theXO has been done using pygame byNirav Patel (http://eclecti.cc/olpc)
Brian Thorne Computer Vision in Python
Motivation & BackgroundComputer Vision in Python
More InformationSummary
Different platformsAdditional Tools
Running on an OLPC
OLPC - provide children in developingnations with access to knowledge, andopportunities to "explore, experimentand express themselves"Includes Python and a webcam - thatsall you need for computer vision!Here I am running OpenCV’sfacedetection on the XO laptopLots of Computer Vision work on theXO has been done using pygame byNirav Patel (http://eclecti.cc/olpc)
Brian Thorne Computer Vision in Python
Motivation & BackgroundComputer Vision in Python
More InformationSummary
Different platformsAdditional Tools
Running on an OLPC
OLPC - provide children in developingnations with access to knowledge, andopportunities to "explore, experimentand express themselves"Includes Python and a webcam - thatsall you need for computer vision!Here I am running OpenCV’sfacedetection on the XO laptopLots of Computer Vision work on theXO has been done using pygame byNirav Patel (http://eclecti.cc/olpc)
Brian Thorne Computer Vision in Python
Motivation & BackgroundComputer Vision in Python
More InformationSummary
Different platformsAdditional Tools
Outline
1 Motivation & BackgroundWhat is Computer Vision?Uses & Examples
2 Computer Vision in PythonToolsImage AcquisitionImage FilteringBackground SubtractionFeature Point Detection
3 More InformationDifferent platformsAdditional Tools
Brian Thorne Computer Vision in Python
IPython & MatPlotLib
Using IPython, an interactive shell can be used from deepinside a nested loop in a running program.
In the code addfrom IPython.Shell import IPShellEmbed...IPShellEmbed()()
Example
In [1]: from opencv import cvIn [2]: cv.cvAnd(diffImage,image, temp)In [3]: timeit cv.cvAnd(diffImage,image, temp)1000 loops, best of 3: 229 µs per loopIn [4]: from pylab import imshow, showIn [5]: imshow(temp)Out[5]: <AxesImage object at 0x42489d0>In [6]: show()
Motivation & BackgroundComputer Vision in Python
More InformationSummary
Different platformsAdditional Tools
Documentation & Support
The documentation in both SciPy and OpenCV was found to bepretty good.... not entirely complete. The OpenCV book is reallygood.
Remember Python is FreeDocumentation is not going to be as extensive as for a professionalpackage like Matlab.... but you can help!
Support for these open source packages is almost entirely reliant onexperienced members of the community responding to requests onmessage boards or mailing lists.
Brian Thorne Computer Vision in Python
Motivation & BackgroundComputer Vision in Python
More InformationSummary
Summary
For the scholar of computer vision research, using Python canhelp in trying out new algorithms very quickly. The breadth ofthe additional libraries available and the ease of integrating,make new and novel solutions quickly realizable.For someone just wanting to play around with some cool stuff,its easy to dive in!Limitations on using Python for CV
A major limitation of using Python would be when theapplication is being developed for special embedded hardwareor when the best possible performance is required (at YOURexpense)
Brian Thorne Computer Vision in Python
References
Thank You!
Thank you toRaphaël Grasset - supervisor at HitLabNZRichard Green - computer vision lecturerJohn Graves & Cristiano Soares for giving me detailed andhelpful feedback
Brian Thorne Computer Vision in Python
References
For Further Reading I
Library URLPygame http://pygame.orgOpenCV http://opencv.willowgarage.com
Numpy/Scipy http://scipy.orgPycam http://pycam.googlecode.org
G. Bradski, A. KaehlerLearning OpenCV.O’Reilly Media, September 2008.
T. OliphantGuide to NumPy.UT, Trelgol Publishing, 2006.
Brian Thorne Computer Vision in Python