Face Recognition

in

Low Resolution Images

Trey AmadorScott MatsumuraMatt Yiyang Yan

Introduction

❖ Purpose: low resolution facial recognition➢ Extract

image/video from source

➢ Identify the person in real time given a trained-database

taken from https://github.com/alexjc/neural-enhance

Face Recognition

Libraries

histogram of oriented gradients (HOG)

dlibSupport VectorMachines (SVM)

Process

● Neural Enhance library○ increase the resolution

of low pixel density○ Theano (neural network)

→ Lasagne (train) →upsampled image

● dlib○ Histogram of oriented

gradients (HOG)○ SVM○ feature descriptor for

detecting faces

→

Database

● IMDb○ Internet Movie Database is an online database of

information related to films, television programs and videogames

○ low and high resolution versions of the same image○ high-resolution 'base' image to train the Support Vector

Machine (SVM)

Support Vector Machine

for

Face Recognition

Arnold Schwarzenegger

SVM

Identify the Rock

Imagessimilar to Dwayne Johnson

Image ofDwayne Johnson

SVM ++

+

+

+

-

-

-

--

+ The Rock- not The Rock

Imagessimilar to Dwayne Johnson

Image ofDwayne Johnson

SVM ++

+

+

+

-

-

-

--

Separate data

Imagessimilar to Dwayne Johnson

Image ofDwayne Johnson

SVM ++

+

+

+

-

-

-

--

Which line?

Imagessimilar to Dwayne Johnson

Image ofDwayne Johnson

SVM ++

+

+

+

-

-

-

--

Thickest line

Imagessimilar to Dwayne Johnson

Image ofDwayne Johnson

SVM +

-

-

+

+

+ +

+

- -

-

-

-

Separate data?

Imagessimilar to Dwayne Johnson

Image ofDwayne Johnson

SVM +

-

-

+

+

+ +

+

- -

-

-

-

Non-linear separation

Imagessimilar to Dwayne Johnson

Image ofDwayne Johnson

Generative

Adversarial Network

for

Upsampling Images

GAN

Back with The Rock

Imagessimilar to Dwayne Johnson

Image ofDwayne Johnson

Image ofDwayne Johnson

Imagessimilar to Dwayne Johnson

GAN

Generate this image?

+

Generative Network

produce an imageDiscriminative Network

real or fake

vs

How to train your

Generative

Adversarial Network

Discriminative

Network

real

fake

GAN

Train discriminative

network

Discriminative

Network

Generative

Network

random noise

backpropagation

GAN

Train both networks

negative gradient

positive gradient

Fake

Discriminative

Network

Generative

Network

random noise

backpropagation

GAN

Eventually?

Real

Discriminative

Network

Generative

Network

GAN

Upsampled

Real

Code

can be found at:

https://github.com/PresidentDwayneCamacho/super-res-face

super resolution

video samples

face recognition in enhanced-resolution video

super resolution image enhancement

boring Bruce Springsteen

100 x 100

enhanced Bruce Springsteen

200 x 200

actual Bruce Springsteen

high res

super resolution face recognition

unrecognized Bruce Springsteen

100 x 100

that’s Bruce Springsteen!

200 x 200

experimental paradigm

true face false face

high res high res

low res low res

enhanced res enhanced res

future directions

❖ find robust metric with which to filter data

❖ test efficacy of various algorithms

❖ generate larger dataset

References

[1] W. Zhao, et al. “Face Recognition: A Literature Survey.” ACM Computing Surveys, vol. 35, pp. 399-458, Dec. 2003.

[2] S.C. Park, M.K. Park, and M.G. Kang. “Super-Resolution Image Reconstruction: A Technical Overview.” IEEE Signal Processing Magazine. May

2003.

[3] D. Glasner, S. Bagon, and M. Irani. “Super-Resolution from a Single Image,” in IEEE 12th ICCV, 2009, pp 349-356.

[4] W.W. Zou and P.C. Yuen. “Very Low Resolution Face Recognition Problem.” IEEE Transactions on Image Processing, vol. 21, pp. 327-340, July

2012.

[5] A. Geitgey, "Face Recognition," GitHub repository, [Online]. Available: https://github.com/ageitgey/face_recognition. [Accessed 29 10 2017].

[6] N. Dalal and B. Triggs. “Histogram of Oriented Gradients for Human Detection” in CVPR, 2005, pp. 1-8.

[7] P. Felzenszwalb, et al. “Object Detection with Discriminantly Trained Part Based Models.” IEEE Transactions on Pattern Analysis and Machine

Intelligence, vol. 32, pp. 1627-1645, Sept. 2010.

[8] C. Cortes and V. Vladimir, "Support-Vector Networks," Machine Learning, vol. 20, no. 3, pp. 273-297, 1995.

[9] A. J. Champandard, "Neural Enhance," GitHub repository, [Online]. Available: https://github.com/alexjc/neural-enhance. [Accessed 29 10 2017].

[10] D. G. Lowe, "Object Recognition from Local Scale-Invariant Features," Computer Vision, vol. 2, pp. 1150-1157, 1999.

[11] K. Simonyan, M. O. Parkhi, A. Vedaldi and A. Zisserman, "Fisher Vector Faces in the Wild," British Machine Vision Conference, vol. 2, no. 3, p. 4,

Sept. 2013.

[12] P. Fischer, A. Dosovitskiy and T. Brox, "Descriptor Matching with Convolutional Neural Networks: a Comparison to SIFT," arXiv, p. 10, 22 May

2014.

[13] M. O. Parkhi, A. Vedaldi and A. Zisserman, "Deep Face Recognition," British Machine Vision Conference, vol. 1, no. 3, p. 6, 2015.

[14] U. Karn, "An Intuitive Explanation of Convolutional Neural Networks," The Data Science Blog, [Online]. Available:

https://ujjwalkarn.me/2016/08/11/intuitive-explanation-convnets/. [Accessed 29 10 2017].

[15] C. Ledig, et al. "Photo-Realistic Single Image Super-Resolution Using a Generative Adversarial Network," arXiv, p. 19, 25 May 2016.

[16] A.V. Nefian. “Georgia Tech Face Database.” Nov. 15, 1999. [Online]. Available: www.anefian.com/research/face_reco.htm. [Accessed: Nov. 5,

2017].

[17] Y.D. Wong. “ChokePoint Dataset.” [Online]. Available: arma.sourceforge.net/chokepoint/. [Accessed: Nov. 5, 2017].