Images & Video Capture

8/16/2019 Images & Video Capture

http://slidepdf.com/reader/full/images-video-capture 1/70

CS3249 (SoC, NUS) Images and Video Capture 1CS3249 (SoC, NUS)

Images and Video Capture 1

1CS3249 (SoC, NUS) Images and Video Capture

Images and Video Capture

Leow Wee !engCS3249 User Inter"a#e $e%e&opment$epartment o" Computer S#ien#e, SoC, NUS



CS3249 (SoC, NUS) Images and Video Capture 2

CS3249 (SoC, NUS) 2Images and Video Capture

' pi#ture speas a t!ousand words



CS3249 (SoC, NUS) Images and Video Capture 3

CS3249 (SoC, NUS) 3Images and Video Capture

*t Images

*t pro%ides 4 #&asses "or !and&ing images

QImage

+ptimised "or I+, dire#t pi-e& a##ess and manipu&ation

QPixmap

+ptimised "or disp&a. on s#reen

QBitmap

In!erited "romQPixmap"or /&a#w!ite images

QPicture

0aint de%i#e t!at re#ords and rep&a.sQPainter#ommands


http://slidepdf.com/reader/full/images-video-capture 4/70CS3249 (SoC, NUS) Images and Video Capture 4CS3249 (SoC, NUS) Images and Video Capture 4

CS3249 (SoC, NUS)Images and Video Capture 4Images and Video Capture 4CS3249 (SoC, NUS)

QLabelpro%ides an eas. wa. to disp&a. an image

#include <QApplication>

#include <QPixmap>#include <QLabel>

int main(int argv, char **args){ QApplication app(argv, args);

QPixmap pic("butterfly.jpg"); // read image QLabel label; label.setPixmap(pic); // put image in label label.show(); return app.exec();}


http://slidepdf.com/reader/full/images-video-capture 5/70CS3249 (SoC, NUS) Images and Video Capture 5CS3249 (SoC, NUS) Images and Video Capture 5

CS3249 (SoC, NUS)Images and Video Capture 5Images and Video Capture CS3249 (SoC, NUS)

QLabel wit! an image



CS3249 (SoC, NUS) Images and Video Capture 6CS3249 (SoC, NUS) Images and Video Capture

0ainting *Widget

or "iner #ontro&, dire#t&. paint QWidget

CreateImageWidgetas su/#&ass o"QWidget

'&&ow user to oom image wit! mouse w!ee&

Need to o%erride two e%ent !and&ers5

paintEvent5 paints in t!e widget wheelEvent5 pro#esses mouse w!ee& input

0rogramming ImageWidgetis rat!er tri#.

Need to write spe#i"i#ation to mae re6uirements #&ear



CS3249 (SoC, NUS) Images and Video Capture 7CS3249 (SoC, NUS) 7Images and Video Capture

Spe#i"i#ation o" ImageWidget

ImageWidgetsie

Initia& sie

$e"au&t depends on p&at"orm and s#reen geometr.

Set to (8, ), widt! : 8, !eig!t :

;inimum sie is gi%en /. minimumSize()

$e"au&t is (, ), w!i#! #auses widget to disappear

C!oose (1, 1)

C&ient #&ass #an #!ange it /. #a&&ingsetMinimumSize()

;a-imum sie is gi%en /.maximumSize() $e"au&t is (177721, 177721)

eep t!is de"au&t

C&ient #&ass #an #!ange it /. #a&&ingsetMaximumSize()



CS3249 (SoC, NUS) Images and Video Capture CS3249 (SoC, NUS) Images and Video Capture CS3249 (SoC, NUS)

Images and Video Capture Images and Video Capture 8CS3249 (SoC, NUS)

$isp&a.ed image sie

$isp&a.ed image sie : oom × input image sie, su/<e#t to

minimumSize()= disp&a.ed image sie =maximumSize() ImageWidgetsie a&wa.s e6ua&s disp&a.ed image sie

setImage(fileName)

Loads image wit! "i&e namefileName Initia& oom s!ou&d /e as &arge as possi/&e /ut = 1

su#! t!at disp&a.ed image sie = initia& widget sie

or image = initia& widget sie, initia& oom : 1

or image > initia& widget sie, initia& oom ? 1



CS3249 (SoC, NUS) Images and Video Capture 9CS3249 (SoC, NUS) Images and Video Capture 9CS3249 (SoC, NUS)

Images and Video Capture 9Images and Video Capture 9CS3249 (SoC, NUS)

@ooming

@oom "a#tor >

C&ient #&ass #an #a&&setZoom(newZoom)to oom

User #an oom in or out using mouse w!ee&

Aa#! mouse w!ee& step in#reases or de#reases oom /. 1



CS3249 (SoC, NUS) Images and Video Capture 1!CS3249 (SoC, NUS) Images and Video Capture 1!CS3249 (SoC, NUS)

Images and Video Capture 1!Images and Video Capture 1CS3249 (SoC, NUS)

// ImageWidget.h

#ifndef IMAGEWIDGET_H#define IMAGEWIDGET_H

#include <QWidget>#include <QImage>#include <QPixmap>

class ImageWidget: public QWidget{ Q_OBJECT Q_PROPERTY(float zoom READ getZoom WRITE setZoom)





0ropert.

Lie #&ass %aria/&e /ut !as addition "eatures5

Q_PROPERTY(type name READ getFunction [WRITE setFunction] [RESET resetFunction] [NOTIFY notifySignal]

...)

type#an /e an. t.pe supported /. QVariant

READde#&ares a##essor "un#tion

WRITE,RESETde#&are modi"ier "un#tions (optiona&)

NOTIFYde#&ares a signa& to /e emitted (optiona&)

*tBs propert. s.stem is p&at"ormindependent





public: ImageWidget(QWidget *parent = 0); void setImage(const QString &fileName);

float getZoom() const { return zoom; } // inline void setZoom(float newZoom);

protected: void paintEvent(QPaintEvent *event); void wheelEvent(QWheelEvent *event);

private: QPixmap input; // Input image. QPixmap display; // Displayed image. float zoom;

};

#endif





// ImageWidget.cpp

#include <QtGui>#include "ImageWidget.h"

ImageWidget::ImageWidget(QWidget *parent): QWidget(parent){ // Set defaults zoom = 0.0; setMinimumSize(10, 10); // width, height}





void ImageWidget::setImage(const QString &fileName){ QImage image = QImage(fileName); // Get image input = QPixmap::fromImage(image); // For display

// Determine appropriate initial zoom

float initWidth = 800; float initHeight = 600; float wRatio = (float) initWidth /

(float) input.width(); float hRatio = (float) initHeight / (float) input.height(); float initZoom = qMin(wRatio, hRatio); if (initZoom >= 1.0)

initZoom = 1.0; // Use original size if possible. setZoom(initZoom);

}





void ImageWidget::setZoom(float newZoom){ if (newZoom <= 0.0) return; // no change

QSize newSize = newZoom * input.size();

if (newSize.width() < minimumWidth() || newSize.height() < minimumHeight() || newSize.width() > maximumWidth() || newSize.height() > maximumHeight())

return; // no change zoom = newZoom;

display = input.scaled(zoom * input.size(), Qt::KeepAspectRatio);

qDebug("zoom %f", zoom); // for debugging resize(display.size());}





input.scaled(...)

S#a&esinputtozoom * input.size()

'&wa.s s#a&es "rominputto get optima& image reso&ution

$onBt res#a&e t!e s#a&ed disp&a.ed image

resize()

Desies widget, t!en sends QResizeEventto widget

WidgetBsresizeEvent()is #a&&ed

resizeEvent()

Widget a&read. !as new sie

't t!e end o" t!e "un#tion, sendsQPaintEventto widget

WidgetBspaintEvent()is #a&&ed

Window s.stem ma. send additiona& resie e%ents





// Event handlers

void ImageWidget::paintEvent(QPaintEvent *event){

QRectF rect = QRectF(QPoint(), size()); QPainter painter(this); painter.drawPixmap(rect, display, rect);}

QRectF(top-left, size)de"ines a re#tangu&ar region QPoint()returns(0, 0)

painter(this)#reates aQPainter"orImageWidget

drawPixmap(target, pixmap, source)

drawssourcepart o"pixmapinto target part o" ImageWidget





Caution

Ca&&ingresize()inresizeEvent()ma. #ause in"initere#ursion

Ca&&ingupdate()orrepaint()inpaintEvent() wi&& #ausein"inite re#ursion





void ImageWidget::wheelEvent(QWheelEvent *event){float step = event->delta() / 1200.0;

float newZoom = zoom + step;

char string[100];

sprintf(string, "%.1f", newZoom); sscanf(string, "%f", &newZoom); // roundingsetZoom(newZoom);

}

delta()returns integer %a&ue in units o" 18 degree

;ost mouse w!ee&s rotate in steps o" 1 degree,ie, 1 (18) : 12 units

$i%ide /. 12 to get oom step o" 1









E!is e-amp&e is in#omp&ete5

$oesnBt disp&a. an image sma&&er t!an minimum widget sie





E!is e-amp&e is in#omp&ete5

C!anging widget sie !as no e""e#t on disp&a.ed image




Image is &arger t!an s#reenF

Ewo wa.s to !and&e &arge images

Use s#ro&& area

0ut widget in s#ro&& area

Aas. to imp&ement

Let user pan image ImageWidgetdisp&a.s a part o" image

User uses mouse /utton to pan image

;ore #!a&&enging to imp&ement

See UI Challenge in 'ssignment 2




Using S#ro&& 'rea

QScrollArea

Su/#&ass o"QWidget

0ro%ides s#ro&&ing %iew onto anot!er widget

0utImageWidgetinQScrollArea

No #!ange toImageWidget S#ro&& /ars appear i"ImageWidgetis too &arge





// main.cpp

#include <QApplication>#include <QScrollArea>

#include "ImageWidget.h"

int main(int argc, char *argv[]){QApplication app(argc, argv);

ImageWidget *widget = new ImageWidget; if (argc > 1) widget->setImage(argv[1]); QScrollArea *scroll = new QScrollArea;

scroll->setWidget(widget); scroll->show();

return app.exec();}

Dea&&. eas.GE!ereBs usua&&.

an eas. wa.to do t!ings in *tG





" i i




Inter"a#ing $e%i#es

;odern #omputers support man. no%e& de%i#es

3 / i i " d i





3 /asi# wa.s to inter"a#e de%i#es5

Wait "or "ramewor de%e&opers to de%e&op de%i#e inter"a#e

eg,QCamerais a%ai&a/&e in *t

How &ong #an .ou waitF

Integrate e-isting too&s

eg, +penCV !as #amera inter"a#e

De&ati%e&. eas. to integrate +penCV into *t

$e%e&op de%i#e dri%ers

De6uire detai&ed now&edge

Ver. di""i#u&t

I t " W / it! + CV




Inter"a#e We/#am wit! +penCV

Crossp&at"orm open sour#e &i/rar. "or #omputer %ision

+penCV 1 was written in C

+penCV 2 pro%ides C, C, 0.t!on, Ja%a inter"a#es

Supports Linu-, Windows, ;a# +S, i+S, 'ndriod

Koing to support 0ara&&e& Ja%as#ript

+penCV 2 e-amp&e "or %ideo #apture wit! we/#am





+penCV 2 e-amp&e "or %ideo #apture wit! we/#am

// webcam.c

#include <core_c.h>#include <highgui_c.h>

int main(int argc, char **args){ int dev = 0; // 0: default, > 0: other cameras.

cvNamedWindow("webcam", 1); // Create OpenCV window.

CvCapture* capture = cvCaptureFromCAM(dev); cvSetCaptureProperty(capture,

CV_CAP_PROP_FRAME_WIDTH, 640); cvSetCaptureProperty(capture, CV_CAP_PROP_FRAME_HEIGHT, 480);

while(1)





while(1) { IplImage *frame = cvQueryFrame(capture);

cvShowImage("webcam", frame);

char key = cvWaitKey(10); if (key == 27) // If key is ESC, break; // break out of loop. }

cvReleaseCapture(&capture);

cvDestroyWindow("webcam"); return 0;}



Integrating +penCV in *t




Integrating +penCV in *t

+penCV %ideo #apture program is se6uentia&

*t is e%entdri%en

Eo integrate +penCV in *t5

Use *t timer e%ent to generate regu&ar&. timed e%ents

Ca&& +penCV "un#tion upon re#ei%ing timer e%ent Con%ert +penCV image toQImage"or disp&a.

Integrate +penCV %ideo #apture "un#tions in *t





Integrate +penCV %ideo #apture "un#tions in *t

+penCV%ideo #apture

"un#tions

We/#am(QWidget)

QTimer

+penCVinter"a#e

init

get "rame

timere%ent

"rame

Nati%es#reen

inter"a#e

disp&a.output

//Webcamh





// Webcam.h

#ifndef WEBCAM_H#define WEBCAM_H

#include <QWidget>#include <QTime>#include <core_c.h>#include <highgui_c.h>

class Webcam: public QWidget{ Q_OBJECT public:

Webcam(int dev, int fps); ~Webcam();

protected:





protected: void showEvent(QShowEvent *event); void hideEvent(QHideEvent *event); void timerEvent(QTimerEvent *event);

void paintEvent(QPaintEvent *event); private: CvCapture* capture; int timerId; int frameRate; // input frame rate

QPixmap pixmap; int nframes; // used to calculate actual frame rate QTime time; // used to calculate actual frame rate

// convert image format QImage IplImage2QImage(const IplImage *iplImage);

};

#endif

//Webcam.cpp





// Webcam.cpp

#include <QtGui>#include "Webcam.h"

Webcam::Webcam(int dev, int fps){ timerId = 0; frameRate = fps; capture = cvCaptureFromCAM(dev);

int width = 640;

int height = 480; cvSetCaptureProperty(capture, CV_CAP_PROP_FRAME_WIDTH, width); cvSetCaptureProperty(capture,

CV_CAP_PROP_FRAME_HEIGHT, height); setFixedSize(width, height);}

Webcam::~Webcam()





(){ cvReleaseCapture(&capture);}

void Webcam::showEvent(QShowEvent *event){ nframes = 0; // init timerId = startTimer(1000 / frameRate); // in msec

time.start(); // start time}

void Webcam::hideEvent(QHideEvent *event){

killTimer(timerId);}

void Webcam::timerEvent(QTimerEvent *event)





(Q ){if (event->timerId() == timerId)

{

IplImage *frame = cvQueryFrame(capture); QImage image = IplImage2QImage(frame); // convert pixmap = QPixmap::fromImage(image); // convert repaint(); // immediate repaint

if (++nframes == 50)

{ qDebug("frame rate: %f", // actual frame rate (float) nframes * 1000 / time.elapsed()); nframes = 0; time.restart(); }

} else QWidget::timerEvent(event);}

void Webcam::paintEvent(QPaintEvent *event)





{ QRectF rect = QRectF(QPoint(), size()); QPainter painter(this);

painter.drawPixmap(rect, pixmap, rect);}

// Convert OpenCV's IplImage to QImage.





QImage Webcam::IplImage2QImage(const IplImage *iplImage){

int height = iplImage->height; int width = iplImage->width;

if(iplImage->depth == IPL_DEPTH_8U && iplImage->nChannels == 3) // colour image {

const uchar *qImageBuffer = (const uchar*) iplImage->imageData; QImage img(qImageBuffer, width, height, QImage::Format_RGB888); return img.rgbSwapped(); }

else if(iplImage->depth == IPL_DEPTH_8U &&



CS3249 (SoC, NUS) Images and Video Capture 44CS3249 (SoC, NUS) Images and Video Capture 44CS3249 (SoC, NUS) Images and Video Capture 44Images and Video Capture 44CS3249 (SoC, NUS)

iplImage->nChannels == 1) // gray image { const uchar *qImageBuffer =

(const uchar*) iplImage->imageData; QImage img(qImageBuffer, width, height, QImage::Format_Indexed8);

QVector<QRgb> colorTable; // set up colour table for (int i = 0; i < 256; i++)

colorTable.append(qRgb(i, i, i));

img.setColorTable(colorTable); return img; } else

{ qWarning() << "Image cannot be converted."; return QImage(); }}

// main.cpp




#include <QApplication>#include "Webcam.h"#include "stdio.h"

int main(int argc, char **args){ QApplication app(argc, args);

int dev = 0; // default device int fps = 25; // 25 frames per second if (argc > 1) sscanf(args[1], "%d", &dev); if (argc > 2) sscanf(args[2], "%d", &fps);

Webcam *view = new Webcam(dev, fps);

view->show(); return app.exec();}

# webcam.pro#R k j tt t fil




# Run qmake -project to generate file.# Then, add the include line.# Then, run qmake followed by make.

TEMPLATE = appTARGET =DEPENDPATH += .INCLUDEPATH += .

# include additional compiler info for OpenCVinclude(opencv.pro)

# InputHEADERS += Webcam.hSOURCES += main.cpp Webcam.cpp

# opencv.pro




QMAKE_CXXFLAGS += -Wno-deprecatedINCLUDEPATH += /usr/include/opencv2/core

/usr/include/opencv2/highguiLIBS += -L/usr/lib -lopencv_core -lopencv_imgproc-lopencv_highgui



Dea&Eime Video 0ro#essing




g

+penCV%ideo #apture

"un#tions

We/#am

(QWidget)

QTimer

+penCVinter"a#e

init

get "rame

timere%ent

"rame

Nati%e

s#reeninter"a#e

disp&a.output

Image

0ro#essor

pro#essedimage

// ImageProc.h



CS3249 (SoC, NUS) Images and Video Capture 5!CS3249 (SoC, NUS) Images and Video Capture 5!CS3249 (SoC, NUS) Images and Video Capture 5!Images and Video Capture CS3249 (SoC, NUS)

#ifndef IMAGEPROC_H#define IMAGEPROC_H

#include <core_c.h>#include <highgui_c.h>

class ImageProc

{public: IplImage *process(IplImage *source, int type);};

#endif



switch(type){




{ case 0: // do nothing return source;

case 1: // demo fast algo: fast smoothing cvSmooth(source, target, CV_GAUSSIAN, 5); return target;

case 2: // demo slow algo: filter with large kernel{




{ int size = 100; CvMat *kernel = cvCreateMat(size, size, CV_32FC1); cvSet(kernel, cvScalar(1.0 / size / size)); cvFilter2D(source, target, kernel); cvReleaseMat(&kernel);

// Filter target is too blurred for display.

// Display the following for demo purpose. cvSmooth(source, target, CV_GAUSSIAN, 5); return target; }

default: // do nothing

return source; }}

// Webcam.h




class Webcam: public QWidget{ Q_OBJECT public: Webcam(int dev, int fps, int type); ~Webcam(); ...

protected: ... void timerEvent(QTimerEvent *event);

private:

... ImageProc *imageProc; int demoType;};

// Webcam.cpp



CS3249 (SoC, NUS) Images and Video Capture 55CS3249 (SoC, NUS) Images and Video Capture 55CS3249 (SoC, NUS) Images and Video Capture 55Images and Video Capture CS3249 (SoC, NUS)

Webcam::Webcam(int dev, int fps, int type){ timerId = 0; frameRate = fps; demoType = type; capture = cvCaptureFromCAM(dev);

int width = 640; int height = 480; cvSetCaptureProperty(capture, CV_CAP_PROP_FRAME_WIDTH, width); cvSetCaptureProperty(capture, CV_CAP_PROP_FRAME_HEIGHT, height);

setFixedSize(width, height); imageProc = new ImageProc;

}

// Webcam.cpp




Webcam::~Webcam(){ cvReleaseCapture(&capture); delete imageProc;}

void Webcam::timerEvent(QTimerEvent *event){




{if (event->timerId() == timerId)

{ IplImage *frame = cvQueryFrame(capture); IplImage *target = imageProc->process(frame, demoType); QImage image = IplImage2QImage(target); pixmap = QPixmap::fromImage(image); repaint();

... }}



Desponse time

rame #apture time : " #




rame #apture time : "( #(

rame pro#essing time : d( "(

Desponse time r( : d( #(

Want r( = τ

ast pro#essing

Desponse time r ?



CS3249 (SoC, NUS) Images and Video Capture 6!CS3249 (SoC, NUS) Images and Video Capture 6!CS3249 (SoC, NUS) Images and Video Capture 6!Images and Video Capture CS3249 (SoC, NUS)

Desponse time r ? τ.

'#tua& "rame rate : desired "rame rate

S&ower pro#essing

Desponse time r M 2τ




Desponse time r M 2 τ

'#tua& "rame rate M 12 desired "rame rate

S&ower rate /ut sti&& responsi%e

*uestion5

Eimer e%ents #an "ire at desired "rame rate

W!at !appen to missing timer e%ents (dotted arrows)F



Kenera& So"tware 'r#!ite#ture




etter to sp&it input and output modu&es

+penCV

%ideo #apture"un#tions

We/#am(QWidget)

QTimer

+penCVinter"a#e

init

get "rame

timer

e%ent

"rame

Nati%es#reen

inter"a#e

disp&a.output

input

output

Nati%e disp&a.;ain

pro#essedimage




+penCV


init

get "rame

"rame

Nati%es#reen

inter"a#e

disp&a.output

Image0ro#essor

We/#am

(#amerainter"a#e)

Window

QTimer

generi##amera

"un#tions

init get"rame

"rame

timere%ent

init

disp&a.;ain

pro#essedimage




+penCV


init

get "rame

"rame

disp&a.output

Image0ro#essor

We/#am

(#amerainter"a#e)

Window

QTimer

generi##amera

"un#tions

init get"rame

"rame

timere%ent

init

simp&i"ied

diagram

disp&a.;ain

pro#essedimage




+penCV


init

get "rame

"rame

disp&a.output

Image0ro#essor

We/#am

(#amerainter"a#e)

Window

generi##amera

"un#tions

init get"rame

"rame

W!i#! pattern

is t!isF

Summar.




Woring wit! images

UseQImage"or inputoutput UseQPixmap"or painting onQWidget

Deimp&ement e%ent !and&ers

paintEvent(),wheelEvent(), et#

Write spe#i"i#ation to #&arit. re6uirements

Integrating de%i#es

Use e-isting too&s wit! !ardware inter"a#es

Integrate too&s into *t

Need to #on%ert data to"rom *t "ormat

C!e# response time

urt!er Deading




2$ and 3$ grap!i#s5 O&an28P, C!ap 8

A-er#ise




;odi". t!e samp&e #odes into 3 &oose&. #oup&ed parts5

;ain window5 disp&a.s %ideo "rame and intera#ts wit! user We/#am5 #aptures %ideo "rame

Image0ro#5 pro#esses images

De"eren#es



CS3249 (SoC, NUS) Images and Video Capture 7!CS3249 (SoC, NUS) 7Images and Video Capture

J &an#!ette and ; Summer"ie&d, C++ GUI Programming withQt 4, 2nd ed, 0renti#e Ha&&, 28

OpenCV Reference Manual

Documents

Images & Video Capture