Thesis presentation december

Augmented reality face recogni2on for mobile devices

Gerry Hendrickx

Promoter: Professor Erik Duval Coordinator: Gonzalo Parra

1

Overview

•  Concept •  Scenario •  Related work •  Privacy issues •  Technical informa2on •  Storyboard •  Paper prototyping •  Planning •  Sta2s2cs

2

Concept

3

Concept Augmented reality face recogni2on for mobile devices

4

Concept Augmented reality face recogni2on for mobile devices…

Real world augmented with extra informa2on, by using a camera

5



Extra informa2on will be about people, by recognizing their faces

6




Camera of smartphone will be used to accomplish goals -‐> Extra informa2on on screen

7




Camera of smartphone will be used to accomplish goals -‐> Extra informa2on on screen

… with social network informa2on

Extra informa2on will be taken from social networks

8

Scenarios

9

Scenarios

Problem: Need to point mobile device to face è Where will you use it?

Need to find scenarios for app!

10

Scenarios

1.  Get contact info 2.  Create a face based contact book 3.  Quick access to slides/publica2ons by

recognizing speaker 4.  Quick access to social network informa2on 5.  Solu2on for prosopagnosia Or… 6.  Find out the name of that cute girl in the bar

11

Related work

12

Related work: TAT Augmented ID

13

Related work

•  Viewdle Social Camera: – Detects facebook friends in photos

•  Animetrics: – Face detec2on + percentage of match – Focused on government use

•  Facebook autotag – Automa2cally tag faced in uploaded pics

14

Related work: More!

•  Project by Parra, Klerkx and Duval

•  Science 2.0 •  Mobile social discovery tool •  Based on QR codes è Not successful

15

Related work: Niels Buekers’ Thesis

•  “Social annota>ons in augmented reality” •  Head mounted display è Not op2mal •  Face.com API •  Difficulty to create UI

16

Privacy issues

17

Privacy Issues

If you could be recognized by people through an iPhone app, would you have privacy concerns?

•  Yes: 13 votes •  No: 8 votes

18

Privacy Issues: Solu2on

•  Only people using the app can be recognized •  Allow user to control displayed informa2on

19

Technical informa2on

20

Technical informa2on: iPhone

•  Why iOS?

•  Why iPhone? –  iPad not user friendly for this kind of use…

21

Technical informa2on: Face recogni2on

•  Viewdle iOS api: ??? •  Betaface: – Windows only – Not tested on iOS – Only paid

•  Piipai – Acquired by Google…

22

Technical informa2on: Face.com

•  Free (rate limits) •  Private namespace •  HTTP get and post •  iOs library •  Very well documented API!

23

Storyboard

24

Storyboard

25

Storyboard

26

Storyboard

27

Storyboard

28

Paper prototyping

29

Paper prototyping

•  Started with basic func2onality •  Ini2al vision: Full AR •  Prac2cal considera2ons: 2 new UI’s

30

Paper prototyping: Itera2on 1

31


32


33


34

•  11 test users, from 18 to 23, 4 iPhone users •  Execute tasks in all 3 UI’s •  Results:

– UI1: Follow person was cumbersome – UI2: Small screen, not much use of camera part – UI3: Less AR, but user friendly

UI1 UI2 UI3

Preference 0% 27% 73%


35

•  Elabora2on of UI3 •  New func2onality: History, Seongs, Overview


36

Paper prototyping: Itera2on 2 tasks

37

1 ) Recognize Gerry Hendrickx 2 ) Check Gerry’s last tweet 3 ) Look for informa2on about Ben Schroyens (who is not around, but you have recognized him before *hint*) 4 ) Delete Ben Schroyens from your history 5 ) Link your account with your Twiier and Mendeley account 6 ) Setup the program so that when someone recognizes you, they can access info from Mendeley, but not from Twiier. 7 ) Delete all history 8 ) Scan Serge Hendrickx, but correct the program when the person is not correctly iden2fied. 9 ) Mail Gerry Hendrickx 10 ) Add Gerry Hendrickx to your iPhone contacts.


38

•  Think aloud •  Note remarks and specific clicks •  USE ques2onnaire •  Ques2ons about iPhone/social network use


39

•  10 test users, from 20 to 23 •  iPhone users:

•  Social networks:

Paper prototyping: Itera2on 2 USE results

40

•  Usefulness: Ok, not life changing but useful •  Ease of use: – Room for improvement – Beier with iOS experience

•  Ease of learning: No score below 5! •  Sa2sfac2on: Fun to use, but divided need to have it


•  Posi2ve: – Clean UI –  iOS style – The concept

41

•  Nega2ve: – Not enough home buions

– Unclear contact icon – Not him/her buion!


•  Adapted to nega2ve points •  Tested with prof Duval and 5 assistants •  Should improve: –  Incorrect buion – S2ll too much clicking to get somewhere => Tabbed Bar

– Delete person from history

42


•  Nice to have: – Preview of own overview page – Presets of social network seongs – Contact buion

43

Planning

44

Planning

45

Planning: Todo next weeks

•  Test technology: – Possibili2es of camera – Possibili2es of Face.com – Possibili2es of iOS5 face detec2on – …

•  Start implementa2on of basis

46

Sta2s2cs

47

Sta2s2cs # Blog posts 16

# Comments on other blogs 5

# Tweets concerning thesis 44

Total # of hours worked 187

# of hours on literature study 20

# of hours on related work 10

# of hours on reports/blog posts 20

# of hours dedicated to other theses 15

# of hours on iOS book 24

# of hours on design 35

# of hours on paper prototype tes2ng 28

# of hours on paper prototype crea2ng and evalua2ng

35

48

Thank you for listening!

Ques2ons?

49

Technology

Thesis presentation december