Upload
rocke-zoolander
View
212
Download
0
Embed Size (px)
Citation preview
7/28/2019 Creating a Face-recognising Security Cam With a Raspberry Pi Part 1
1/5
Through the Interface
September 12, 201 2
Creating a face-recognising security cam with a Raspberry Pi Part 1
This series of posts builds upon the mini-series on building a motion-detecting sec urity cam based around the
Raspberry Pi. Once y ou have y our motion detecting security c am up and running, y ou should be able to mov e
on to the next stage: enabling that system to reco gnize faces that it has been trained against.My specific project (which Im c alling the Facecam, although I havent applied for a trademark;-) pulls data
do wn from Faceboo k and uses that to train the face recognition system, but thats far from being a requirement
its also very possible to train the database in other ways.
Well go into detail on the various parts ov er the coming weeks (probably once a week, as I want to continue
blogging about other to pics, too), but lets start by introducing the projec t and the system architecture.
Heres the elevator p itch: Facec am is a security camera that recognises a residents Faceboo k friends when they
come to their front door and allows for tailored co mmunication both to the resident and the visitor.
Thiswould basically mean that instead of the resident rec eiv ing a not ificatio n email say ing Y ou have a v isitor
(as in the case of the motion-detection sy stem we saw previously), the email would read John Smith visitedyou. A nd Jo hn c ould potentially receiv e a c ustomized wer e out message while standing at the doo r and eve
a post o n his Faceboo k wall thanking him for the v isit.
A lot of this is clearly contingent o n the accuracy of the facial rec ognition sy ste m something well lo ok at muc
more c losely later on. In case the accurac y isnt adequate, it would be nice at least to prov ide some feedback to
the v isitor, say ing Welco me, John! (even if its actually Megan at the do or ;-).
Right, now lets dive into the sy stem architecture . Heres a quick diagram of the Facecam sy stem:
Heres a look at the new components (in addition to the Motion detection box we saw previously):
1. A componen t to download information on the residents Facebook friends (.NET desktop
application)
This component will build a friend database (stored in a file named facedata.xml) to be used at runtime by the
security cam. The database will be trained with the friends photos as downloaded from Facebook. So the
co mponent will need to access for each friend both their names and the photos in which they have been
http://through-the-interface.typepad.com/.a/6a00d83452464869e2017d3bfcb797970c-pihttp://through-the-interface.typepad.com/.a/6a00d83452464869e2017d3bfcb797970c-pihttp://through-the-interface.typepad.com/.a/6a00d83452464869e2017d3bfcb797970c-pihttp://through-the-interface.typepad.com/.a/6a00d83452464869e2017d3bfcb797970c-pihttp://through-the-interface.typepad.com/.a/6a00d83452464869e2017d3bfcb797970c-pihttp://through-the-interface.typepad.com/.a/6a00d83452464869e2017d3bfcb797970c-pihttp://through-the-interface.typepad.com/.a/6a00d83452464869e2017d3bfcb797970c-pihttp://through-the-interface.typepad.com/.a/6a00d83452464869e2017d3bfcb797970c-pihttp://through-the-interface.typepad.com/.a/6a00d83452464869e2017d3bfcb797970c-pihttp://through-the-interface.typepad.com/.a/6a00d83452464869e2017d3bfcb797970c-pihttp://through-the-interface.typepad.com/.a/6a00d83452464869e2017d3bfcb797970c-pihttp://www.facebook.com/http://www.raspberrypi.org/http://through-the-interface.typepad.com/through_the_interface/2012/09/creating-a-motion-detecting-security-cam-with-a-raspberry-pi-part-2.htmlhttp://through-the-interface.typepad.com/through_the_interface/2012/08/creating-a-motion-detecting-security-cam-with-a-raspberry-pi-part-1.htmlhttp://through-the-interface.typepad.com/through_the_interface/7/28/2019 Creating a Face-recognising Security Cam With a Raspberry Pi Part 1
2/5
Creating a face-
recognising
security cam with a
Raspberry ...
Creating a motion-
detecting security
cam with a
Raspberry ...
Creating a motion-
detecting security
cam with a
Raspberry ...
Creating a face-
recognising
security cam with a
Raspberry ...
tagged (as well as the tag information, of course).
Using the tag information, the co mponent will use OpenCV to attempt to ex tract the friends face (by chec king
the list of detec ted faces against the tag lo cation, to get the closest match), and if it finds it it will resize, c rop
and equalize the grey scale intensity o f the image before saving it to disk for later use in the database training
process.
This is ultimately an offline, CPU-intensive o peration we will simply need the database transferred acro ss to
the Raspberry Pi from time to time so the approach well take is to c reate a .NET desktop application that
performs this task on a higher-powered dev ice.
2. A com ponent to analyse frames saved by th e motion-detection sy stem, checkin g for friends
(face recognition)
This is the guts of the sy stem: the runtime compo nent that will again use OpenCV to process images captured
from the webcam, detect faces and then check the results against the facial reco gnition database on the
Raspberry Pi. This needs to be pretty efficient as the feedback should be in clo se to real-time which may be
challenge: given the relatively low power o f the Raspberry Pi CPU, well have to jump through some interesting
hoops to get this working well enough.
3. A com ponent to provide feedback to the v isitor (LED messaging)
If were going to tell the v isitor that they have b een recognised (or in some way warn unwanted visitors that
their photo has been sent to the pro perty s residents), then were going to need so me kind of screen. The
Raspberry Pi has both HDMI and component v ideo outputs, but Iv e dec ided to go the way o f a USB-powered
LED message-board. More o n this in due co urse.
Right, thats it for this introductory post. Well look in more detail at each of these co mponents ov er the c oming
weeks, starting with the .NET desktop applic atio n to download friend information.
You m igh t also like:
LinkWithin
Posted at 07 :45 AM in Raspberry Pi | Permalink
Comments
Jeremy Tammik dijo...
Hi Kean,
Sounds neat and clear and interesting.
http://thebuildingcoder.typepad.com/http://through-the-interface.typepad.com/through_the_interface/2012/09/creating-a-face-recognising-security-cam-with-a-raspberry-pi-part-1.htmlhttp://through-the-interface.typepad.com/through_the_interface/raspberry-pi/http://www.linkwithin.com/learn?ref=widgethttp://www.dreamcheeky.com/led-message-boardhttp://opencv.org/http://through-the-interface.typepad.com/through_the_interface/2012/09/creating-a-face-recognising-security-cam-with-a-raspberry-pi-part-2.htmlhttp://through-the-interface.typepad.com/through_the_interface/2012/08/creating-a-motion-detecting-security-cam-with-a-raspberry-pi-part-1.htmlhttp://through-the-interface.typepad.com/through_the_interface/2012/09/creating-a-motion-detecting-security-cam-with-a-raspberry-pi-part-2.htmlhttp://through-the-interface.typepad.com/through_the_interface/2012/10/creating-a-face-recognising-security-cam-with-a-raspberry-pi-part-4.html7/28/2019 Creating a Face-recognising Security Cam With a Raspberry Pi Part 1
3/5
I really wo nder what kind of data is going to go into the face data XML file. I would not put pixel data there , but
store that in external indiv idual image files, and reference those .
Some of my friends boy co tt internet-based social networking and thus hopefully cannot be identified by
facebook, so I would enable private non-facebo ok photos from the local system to be used as well.
Cheers, Jeremy.
Responder September 12, 201 2 at 09:11 A M
Kean Walmsley dijo en respuesta a Jere my Tammik...
Hi Jeremy ,
We'll of course talk about the XML format (although it's something this projec t has inherited), which uses an
algorithm called Eigenfaces to greatly r educe the amount of data stored.
And y es - I 'm pr obably not ev en going to share the Faceboo k-spec ific c ode (for once), but will certainly ex plain
more abo ut the file/folder structure y ou'd need to build to train the database in some other way .
Cheers,
Kean
Responder September 12, 201 2 at 09:15 A M
AlexFielder dijo...
Hi Kean,
This loo ks really interesting!
An API I 'm not familiar with (but which I b eliev e might prove useful here) is the Goo gle Picasa Web API; As I
say, I o nly know there ex ists an API from Google which (in some way ) ties in with the Picasa Web o ffering fromthe big G.
In case y ou aren't aware, Picasa (the desktop application) offers y ou the ability to tag faces o f the people it finds
within y our picture librar y , and will (with y our permission) sy nc them to your Picasa Web Pho tos albums. I 've
tagged all my family with it, so if it's possible to use this API instead o f Facebook, that might be another avenue
that would likely get aro und the friends-not-tagged-on-Facebo ok issue that Jeremy mentions.
Here is the best link I co uld find that will be of help:
Retrieving face tag data from the Picasa Web API (which in turn links to this page: Google Groups Picasa issues
page
Cheers,
Alex.
Responder September 12, 201 2 at 04:52 PM
Kean Walmsley dijo en respuesta a Alex Fielder...
Hi Alex ,
That's interesting... it should certainly be possible to swap o ut the Facebo ok-oriented piece to pull down data
http://twitter.com/AlexFielderhttp://blogs.autodesk.com/through-the-interfacehttp://through-the-interface.typepad.com/through_the_interface/2012/09/creating-a-face-recognising-security-cam-with-a-raspberry-pi-part-1.html#comment-6a00d83452464869e2017c31d0245a970bhttp://void%200/https://code.google.com/p/gdata-issues/issues/detail?id=751&sort=-stars%20id&colspec=API%20ID%20Type%20Priority%20Stars%20Summaryhttp://www.klick.com/health/news/blog/retrieving-face-tag-data-from-the-picasa-web-api/http://twitter.com/AlexFielderhttp://through-the-interface.typepad.com/through_the_interface/2012/09/creating-a-face-recognising-security-cam-with-a-raspberry-pi-part-1.html#comment-6a00d83452464869e2017c31ce9048970bhttp://void%200/http://thebuildingcoder.typepad.com/http://blogs.autodesk.com/through-the-interfacehttp://through-the-interface.typepad.com/through_the_interface/2012/09/creating-a-face-recognising-security-cam-with-a-raspberry-pi-part-1.html#comment-6a00d83452464869e2017744ac4fb8970dhttp://void%200/7/28/2019 Creating a Face-recognising Security Cam With a Raspberry Pi Part 1
4/5
from Picasa.
I'll be describing the process I've used to isolate and extract faces to train the database next week.
Cheers,
Kean
Responder September 12, 201 2 at 04:56 PM
Philipp Wagner dijo...
This will probably help you: http://docs.opencv.org/trunk/modules/contrib/doc/facerec/index.html.
Responder September 12, 201 2 at 1 0:39 PM
Philipp Wagner dijo en respuesta a Philipp Wagner. ..
Especially the part o n Face Recognition in Videos might be interesting to yo u:
* http://doc s.opencv.org/trunk/modules/contrib/doc/ facerec/ tutorial/facerec_video_recognition.htmlResponder September 12, 20 12 at 10:45 PM
Kean Walmsley dijo en respuesta a Philipp Wagner...
Thanks, Philipp - that's very helpful.
I'm pretty much do ne with that side of things, but I'll certainly rev isit the co de onc e I've loo ked at the tutorial
more closely.
Kean
Responder September 12, 201 2 at 1 0:58 PM
Hlytin dijo...
That was an interesting thing,security camera purpo se is really a lot specially o n giving a security a shop and
home.In Finland country many of people are still having doubt on putting security cameras howev er most of th
already did.Any way this is really ex cellent one.
Responder November 07 , 2012 at 10:27 AM
Pyry l dijo...
Hi Kean, hope y ou are still following this thread. I'm wondering whether to start a face reco gnition pro ject. My
co ncept would be based on linkedin profile pics, thus only 1 re ference pic per face.
Based on your ex perience is reco gnizing a face with only one reference pic available from a photo by any chanc
possible? Did you encounter in your ex periment cases where the user had only one pic in fb?
Responder March 25 , 201 3 at 03 :22 PM
http://through-the-interface.typepad.com/through_the_interface/2012/09/creating-a-face-recognising-security-cam-with-a-raspberry-pi-part-1.html#comment-6a00d83452464869e2017d424741a3970chttp://void%200/http://twitter.com/pyrylhttp://through-the-interface.typepad.com/through_the_interface/2012/09/creating-a-face-recognising-security-cam-with-a-raspberry-pi-part-1.html#comment-6a00d83452464869e2017d3d5f6744970chttp://void%200/http://through-the-interface.typepad.com/through_the_interface/2012/09/creating-a-face-recognising-security-cam-with-a-raspberry-pi-part-1.html#comment-6a00d83452464869e2017c31d19b3f970bhttp://void%200/http://www.bytefish.de/http://blogs.autodesk.com/through-the-interfacehttp://through-the-interface.typepad.com/through_the_interface/2012/09/creating-a-face-recognising-security-cam-with-a-raspberry-pi-part-1.html#comment-6a00d83452464869e2017c31d18f37970bhttp://void%200/http://docs.opencv.org/trunk/modules/contrib/doc/facerec/tutorial/facerec_video_recognition.htmlhttp://www.bytefish.de/http://www.bytefish.de/http://through-the-interface.typepad.com/through_the_interface/2012/09/creating-a-face-recognising-security-cam-with-a-raspberry-pi-part-1.html#comment-6a00d83452464869e2017744af50af970dhttp://void%200/http://docs.opencv.org/trunk/modules/contrib/doc/facerec/index.html.http://www.bytefish.de/http://through-the-interface.typepad.com/through_the_interface/2012/09/creating-a-face-recognising-security-cam-with-a-raspberry-pi-part-1.html#comment-6a00d83452464869e2017744adea69970dhttp://void%200/7/28/2019 Creating a Face-recognising Security Cam With a Raspberry Pi Part 1
5/5
Kean Walmsley dijo en respuesta a Pyry l...
Hi Pyry l,
Based on my experience the problems would be:
* 1 photo is way to o few, at least for the system I tried to use .
* Profile photos aren't representative enough of how people loo k when they co me to y our doo r (or sit at theirPC, if somehow recognising people's faces v ia their webcam).
Cheers,
Kean
Responder March 25 , 201 3 at 04:00 PM
Pyry l dijo...
Thanks Kean! I think yo u just saved me a week of hacking towards a dead end.
Responder April 01, 20 13 at 12:44 PM
Comment below o r sign in with TypePad Faceboo k Twitter and more...
http://www.typepad.com/sitelogin?uri=http%3A%2F%2Fthrough-the-interface.typepad.com%2Fthrough_the_interface%2F2012%2F09%2Fcreating-a-face-recognising-security-cam-with-a-raspberry-pi-part-1.html&fp=9658899d2e71abe8d5c8188ddc835a8c&view_uri=http%3A%2F%2Fprofile.typepad.com%2F&via=blogside&service=openid&post_uri=http://through-the-interface.typepad.com/through_the_interface/2012/09/creating-a-face-recognising-security-cam-with-a-raspberry-pi-part-1.htmlhttp://www.typepad.com/sitelogin?uri=http%3A%2F%2Fthrough-the-interface.typepad.com%2Fthrough_the_interface%2F2012%2F09%2Fcreating-a-face-recognising-security-cam-with-a-raspberry-pi-part-1.html&fp=9658899d2e71abe8d5c8188ddc835a8c&view_uri=http%3A%2F%2Fprofile.typepad.com%2F&via=blogside&service=twitter&post_uri=http://through-the-interface.typepad.com/through_the_interface/2012/09/creating-a-face-recognising-security-cam-with-a-raspberry-pi-part-1.htmlhttp://www.typepad.com/sitelogin?uri=http%3A%2F%2Fthrough-the-interface.typepad.com%2Fthrough_the_interface%2F2012%2F09%2Fcreating-a-face-recognising-security-cam-with-a-raspberry-pi-part-1.html&fp=9658899d2e71abe8d5c8188ddc835a8c&view_uri=http%3A%2F%2Fprofile.typepad.com%2F&via=blogside&service=facebook&post_uri=http://through-the-interface.typepad.com/through_the_interface/2012/09/creating-a-face-recognising-security-cam-with-a-raspberry-pi-part-1.htmlhttp://www.typepad.com/sitelogin?uri=http%3A%2F%2Fthrough-the-interface.typepad.com%2Fthrough_the_interface%2F2012%2F09%2Fcreating-a-face-recognising-security-cam-with-a-raspberry-pi-part-1.html&fp=9658899d2e71abe8d5c8188ddc835a8c&view_uri=http%3A%2F%2Fprofile.typepad.com%2F&via=blogside&post_uri=http://through-the-interface.typepad.com/through_the_interface/2012/09/creating-a-face-recognising-security-cam-with-a-raspberry-pi-part-1.htmlhttp://through-the-interface.typepad.com/through_the_interface/2012/09/creating-a-face-recognising-security-cam-with-a-raspberry-pi-part-1.html#comment-6a00d83452464869e2017d4270a3bd970chttp://void%200/http://twitter.com/pyrylhttp://through-the-interface.typepad.com/through_the_interface/2012/09/creating-a-face-recognising-security-cam-with-a-raspberry-pi-part-1.html#comment-6a00d83452464869e2017c3818455f970bhttp://void%200/http://twitter.com/pyrylhttp://blogs.autodesk.com/through-the-interface