Master of Industrial Sciences 2014- 2015 Faculty of Engineering Technology, Campus Group T Leuven

This paper is written by (a) student(s) in the framework of a Master’s Thesis

AUGMENTED REALITY DOES NOT VIOLATE YOUR PRIVACY

Dries Holemans, Sam Ghysels

Industrial Sciences Electronics-ICT, focus Internet Computing,

Faculty of Engineering Technology, Campus Group T Leuven

Vesaliusstraat 13, 3000 Leuven, Belgium

Supervisor: Jeroen Buijs

Faculty of Engineering Technology, Campus Group T, Leuven

Vesaliusstraat 13, 3000 Leuven, Belgium

jeroen.buijs@kuleuven.be

ABSTRACT

The purpose of this research is to identify the main problems concerning privacy when using augmented reality

devices. People are very aware that many things can violate their privacy, yet they can’t define the exact reasons

for violation. The first phase involves a social blog where posts concerning privacy problems will ask the audience

what they think about the various types of privacy violations. The final phase is to evaluate different types of

solutions for these problems, using the input of the audience. By identifying the problems, we can set goals to

negate them. This will allow for further research on the different solutions to problems concerning privacy with

augmented reality devices.

Keywords Augmented reality, privacy concerns, Google, data collecting, facial recognition, location tracking, social

networking

2

1 INTRODUCTION

The main reason people do not trust the augmented reality

devices is because they can’t recognize the main problems

concerning privacy. While obvious features such as taking

pictures are considered threatening to privacy, there are far

more hazardous features augmented reality devices can

have for our privacy, such as facial recognition.

“If you know the enemy and know yourself, you need not

fear the result of a hundred battles. If you know yourself

but not the enemy, for every victory gained you will also

suffer a defeat. If you know neither the enemy nor

yourself, you will succumb in every battle.” These famous

words from Sun Tzu also apply to fighting the problems

augmented reality has with privacy concerns. We keep this

quote in mind as we are discussing the various problems.

If we do not know what the problem is, we won’t find a

solution. If we know the problem, but not think enough

about the solution, another problem will rise. Only by both

knowing the problem and solution, we can say that we are

victorious in the battle for our privacy.

2 DISCUSSION

Our paper has been divided into two parts with the above

strategy in mind. We will first discuss the problems that

arise with augmented reality concerning privacy. Later on,

we will use the comments we gathered from our followers

to find solutions to these problems and even how

augmented reality could increase your privacy.

2.1 Privacy threats in augmented reality There are numerous privacy risks when using augmented

reality some are obvious, others are not. In this section we

will discuss the possible privacy threats that come with

augmented reality based on various examples that we

found from our blog.

2.1.1 Facebook environment sharing

Social networking is a multi-billion business today and of

course the well-known Facebook is without a doubt the

biggest player right now (Doe, 2012). Ever since Facebook

acquired Oculus, the maker of the popular virtual reality

head-mounted display ‘Oculus Rift’, they are trying to

boost their presence in the virtual reality market. As

discussed in our blog they are trying to do this by making

an application that enables users to share their virtual

environment with their friends on Facebook. Then they

can use the virtual reality app to visit their friends

environments to experience how the place their friends

visited looks like.

Obviously there are numerous privacy concerns about this

new virtual reality application that Facebook is

developing. First of all there’s even more of your identity

being released on the internet. Instead of a single picture

you now share your entire environment and everyone can

clearly see where you were and what you did. Of course

you might say that you choose what you share on

Facebook but it probably wouldn’t be the first time that

you get tagged in your friends pictures. Now with virtual

environments the chance of being tagged by your friends

is greatly increased and the publicly available information

about you keeps growing and growing. Last but not least

Facebook has shown in the past that it doesn’t really care

about privacy and even shares your data with their partners

to increase their own profit (Munson, 2015).

2.1.2 (Automated) Data collection

A lot of devices nowadays gather data in the background.

A good example of this is when a Facebook user posts a

status, it will show the user’s location. This happens

automatically, without the user knowing. Now you could

say that location might not be such a problem, but other

data is also being gathered by various applications around

the world, all day long. More private information such as

the mortgage on your house could be made public. A real

issue for privacy when such private data is being thrown

around on the internet.

Another example of data collection can be found in an

interesting application for Google Glass where subtitles

are shown during a conversation in real time (Schievink,

2014). At first sight you might not recognize the privacy

risk here but all the conversation data is send to Google’s

servers to do the speech-to-text conversion. Another big

problem for the user’s privacy since it is possible for

Google to collect all the data that is coming to their

servers.

2.1.3 Advertising

Advertisements are usually something people do not pay

too much attention to when using electronics. However, it

is quite important that in augmented reality devices the

advertising could become a more serious problem. With

all the data being gathered from the device, marketeers

(and therefore, big companies too) could easily ask for this

private data just for personal advertisements. We strongly

believe that this is an assault on the user’s privacy.

2.1.4 Facial recognition

A study showed that 72% of the people wouldn’t wear an

augmented reality device such as Google Glass due to

privacy concerns and the most common concerns were

“the ease with which others could record their actions

without their knowledge and the potential for private

actions to become public” (Levy, 2015). So it seems like a

camera that is continuously on scares people the most.

However it probably wouldn’t be such a big problem

without facial recognition.

Research has shown that it is already possible to to identify

a person in real time using publicly available information

and facial recognition software (Acquisti, 2011). One out

of three persons could be correctly identified and while

this might not sound like a lot this number will

exponentially increase in the future when more processing

power, better facial recognition algorithms and more

public data is available. There’s a good reason why Google

banned facial recognition applications from the Google

Glass app-market (Charles, 2013). Facial recognition is a

3

serious privacy threat that can’t be ignored while

developing augmented reality devices.

2.2 Solutions and improvements for your

privacy in augmented reality A privacy risk doesn’t automatically mean that your

privacy will be violated. In this section we offer simple

countermeasures for the privacy threats described above

and also other solutions that could even increase your

privacy when using an augmented reality device. Again

this is based upon examples found in our blog.

2.2.1 Hololens

While we’ve discussed other companies such as Facebook

and Google and their privacy issues, a third major

company is also looking forward to the augmented reality

world. In January 2015, Microsoft unveiled their plans for

the Microsoft HoloLens, a device similar to a combination

of the Google Glasses and the Oculus Rift. The current

intention for the HoloLens is to be able to reproduce 3D

images from the virtual world to your room. While some

people raised some questions about projecting the images

and their privacy, Microsoft made sure only the people

who actually wear the HoloLens are able to see the

projections (Statt, 2015).

2.2.2 Militarized security protocols

Another good reason to not distrust the product is the

origin of augmented reality devices. As with a lot of tools

we use today, augmented reality also comes from military

research. Osterhout Design Group was manufacturing

augmented reality glasses for the military and now wants

them to become publicly available (Takahashi, 2015). The

good news is that this device is already up to date with

military protocols, which also include the security of the

data gathered. After all, the military wouldn’t want their

highly confidential data to be leaked all over the place. A

strong trust was detected from the comments we gathered

from our users concerning the privacy this device can

guarantee. Though they also knew that augmented reality

devices might be used for killing purposes at the military,

that is another ethical question not to be further discussed

in the paper.

2.2.3 Augmented Legality

Also to be noted is that there is already a lot of debate on

laws concerning the use of augmented reality devices.

Augmented Legality, as it is called (McNamee, 2014), is a

great step forward towards a future where both the users,

developers and third parties are being protected from harm

by privacy laws. While it is still all under development, it

is reassuring for the people to know that there is indeed a

lot of thought put in for their privacy concerns.

2.2.4 (Automated) Data collection

Future users were also concerned about the data that is

being trafficked all over the internet. People are very

aware that personal data might be gathered by other

companies and generally don’t like that. However, we

noticed from the comments our followers made that a lot

of people did not really do anything against this. A lot of

simple methods could already help to prevent a lot of data

leaking. A simple method for instance is to not put any

confidential data onto the internet! Also users have to be

aware of what their friends are tagging them for. You

might think that a friend putting a photo of your new car

online might be innocent, but this could give local thief

gangs an opportunity to get your address easily and the

knowledge there’s value over there. Therefore, discussing

these things with your friends and family could already

help you a great deal with your privacy.

2.2.5 Augmented reality shopping

Augmented reality could also help preserve your privacy

in ways not possible before. A good example of this would

be a shopping application used in a crowded mall (Oswald,

2015). While shopping for clothes, people have to undress

in small rooms, which might attract some peeping toms.

Now with augmented reality, these clothes can be

projected onto yourself, which undoes the need for

undressing. This in response gives peeping toms no chance

to take a look. While this is only one example, many other

situations can be conceptualized where augmented reality

helps your privacy in a positive way as a side effect.

2.2.6 Facial recognition

As we discussed before a camera that is continuously

recording and facial recognition are one of the biggest

privacy issues for augmented reality. But also for this

problem multiple solutions exist in an augmented reality

world.

The first one is remarkably low tech and protects you with

infrared light. The frame of your augmented reality device

is studded with infrared LEDs, which continuously beam

out IR light. This infrared light will tamper with the lens

of the camera of other augmented reality devices, making

it like there is light all over the wearer’s face. Additionally

the glasses could have a special coating that reflects the

light back in the direction of the camera to make it even

harder to identify the person wearing the glasses (de

Looper, 2015). This solution makes it unable for someone

else to identify you and thus preserving your privacy.

The second one is a bit more ingenious. Instead of making

you unrecognizable, you simply avoid all the privacy

threats altogether. This works by showing you all the

possible privacy threats on your augmented reality device

such as Google Glass. For example for cameras it will

show you a ‘safe zone’ where you can walk without being

recorded (Veenhof, 2014). So it becomes possible to avoid

all the privacy threats and walk around undetected.

So while facial recognition might pose serious privacy

issue when you let it run its course, there are multiple ways

to counter this problem. Thanks to these solutions you will

no longer be able to be identified by an augmented reality

device with a camera and thus the main privacy issue of

augmented reality is eliminated.

4

3 CONCLUSION

In this paper we have discussed both the potential

problems of augmented reality and their possible

solutions. Many people distrust augmented reality devices

due to the obvious privacy threats like a constantly

recording camera, automated data collection and facial

recognition. However we have proven that most of these

problems are easily countered by very simple solutions

like infrared LEDs. In addition there are applications that

could even enhance your privacy like a shopping

application so you no longer having to undress in a

crowded mall or showing cameras and privacy threats on

your augmented reality device so you can avoid them.

The main goal of this paper is to increase the awareness of

privacy threats in augmented reality devices and

applications but also to offer easy countermeasures that

not only negate the threat but even possibly increases your

privacy in the real world. We hope to have increased the

trust that people have in augmented reality and to have

shown that using augmented reality devices doesn’t

necessarily mean that your privacy, or the privacy of the

people around you, will be violated.

4 REFERENCES

Acquisti, Alessandro, Gross, R. & Stutzman, F. (2011).

Privacy in the age of augmented reality. Proc.

National Academy of Sciences.

Charles, A. (2013). Google 'bans' facial recognition on

Google Glass - but developers persist. Retrieved

April 14, 2015 from

http://www.theguardian.com/technology/2013/ju

n/03/google-glass-facial-recognition-ban

de Looper, C. (2015). AVG Privacy Glasses Remove Fear

Of Facial Recognition. Retrieved April 14, 2015

from

http://www.techtimes.com/articles/36743/20150

303/avg-privacy-glasses-remove-fear-of-facial-

recognition.htm

Doe, J. (2012). Facebook Inc. Profile: How a social

network became a multi-billion dollar business.

Retrieved April 13, 2015 from

http://connection.ebscohost.com/c/case-

studies/82029472/facebook-inc-profile-how-

social-network-became-multi-billion-dollar-

business

Levy, P. (2015). Google Glass was a product looking for

a market. Retrieved April 14, 2015 from

http://phys.org/news/2015-01-google-glass-

product.html

McNamee, S. (2014). An introduction to augmented

reality and the law. Retrieved April 14, 2015

from http://theundisciplined.com/2014/07/21/an-

introduction-to-augmented-reality-and-the-law/

Munson, L. (2015). Facebook's got a new privacy policy,

and it plans to share your data with partners.

Retrieved April 13, 2015 from

https://nakedsecurity.sophos.com/2015/02/02/fa

cebooks-got-a-new-privacy-policy-and-it-plans-

to-share-your-data-with-partners/

Oswald, V. (2015). 3 Retail AR Solutions You Didn't Know

Existed. Retrieved April 14, 2015 from

http://blog.metaio.com/2015/02/11/3-columns-

ar-retail-shopping/

Schievink, B. (2014). Google Glass-app zorgt voor

'ondertiteling" bij gesprekken. Retrieved April

14, 2015 from

http://tweakers.net/nieuws/98853/google-glass-

app-zorgt-voor-ondertiteling-bij-

gesprekken.html

Statt, N. (2015). Microsoft's HoloLens explained: How it

works and why it's different. Retrieved April 13,

2015 from http://www.cnet.com/news/microsoft-

hololens-explained-how-it-works-and-why-its-

different/

Takahashi, D. (2015). Osterhout Design Group packs a

ton of cool tech into next-generation augmented-

reality smartglasses. Retrieved April 13, 2015

from

http://venturebeat.com/2015/01/02/osterhout-

design-group-packs-a-ton-of-cool-tech-in-next-

generation-augmented-reality-smartglasses/

Veenhof, S. (2015). Watch Your Privacy. Retrieved April

14, 2015 from

http://sndrv.com/watchyourprivacy/