6
FreeGlass for developers, “haccessibility”, and Digital Eye Glass + Lifeglogging Research in a (Sur/Sous)Veillance Society Steve Mann, Mir Adnan Ali, Raymond Lo, Han Wu University of Toronto, Dept. Electrical and Computer Engineering Abstract—This paper presents “FreeGlass”, a hands-free user- buildable wearable computer system based on Mann’s “Digital Eye Glass” (also known as EyeTap) concept developed 35 years ago. Like Mann’s “Eye Glass”, FreeGlass is based on free and Open Source principles consistent with a free and open i-Society. FreeGlass is suitable for researchers and creators of AR (Augmediated Reality) Glass, LifeGlogging, etc. in the information society. Two example applications of FreeGlass are also presented: Reality User Interface; and “ALIBEye”, a digital alibi through sousveillance (inverse surveillance). We note the potential for FreeGlass to bring about a transition from a surveillance society (cameras affixed to land and buildings), to a sousveillance-society (cameras held, carried, or worn by individuals). Important elements of a balanced sur/sous-veillance society are also presented with a view toward a middle-ground between a surveillance-only society where large entities record interactions but forbid individuals from keeping their own record of their sensory information, and the coming sousveillance society in which individuals are also equipped with veillance capacity, and the capacity to capture AND authenticate their own recordings as digital alibis. Keywords: Wearable Computing, FreeGlass, Sousveillance, Surveillance, Veillance, Alibi Sousveillance, Augmented Re- ality, Augmediated Reality, Mediated Reality, EyeTap, Life- Glogging, Digital Eye Glass, MannGlas I. SURVEILLANCE IN THE INFORMATION SOCIETY Information society is growing at a tremendous pace, through the business economy of cloud computing. The proper working of this i-Society requires intelligent strategies in response to complex attacks [1], as well as an effective e- Government strategy [2]. In addition to the need for security there is also the need for forensic extraction of data to meet the legitimate needs of law enforcement [3]. With mobile, portable, and wearable computing, there are even further security needs [4]. Surveillance, together with necessary oversight mecha- nisms, has a legitimate and important place in a lawful and just i-Society. Surveillance is a well-known and, for the most- part, accepted practice. The word “surveillance” is a French word that means watching (“veillance”) from above (“sur”). See Fig 1. The i-Society has become a “Surveillance Society” so “Sureillance Studies” is an area of increasing importance [5]. Surveillance is defined, formally, and precisely, as: sur-veil-lance [noun] (a) (b) Fig. 1. (a) Surveillance camera overlooking residential area in Toronto, Canada. (b) Kindergarten student (Age 6) fluent in multiple languages draws: Surveillance/ ¨ Uberwachung/Oversight = watching from Over (above); and Sousveillance/Unterwachung/Undersight = watching from Under (below), e.g. by way of a wearable camera. 1) a watch kept over a person, group, etc., especially over a suspect, prisoner, or the like: ‘‘The suspects were under police surveillance.’’ (Random House, Inc. Dictionary, 2013) Merriam Webster provides four words as synonyms for surveillance: supervision; superintendence; oversight; control, and two examples: “government surveillance of suspected terrorists”; “The bank robbery was recorded by surveillance video cameras.”. Society is becoming more like a panoptic [6] prison, with regards to the one-sided gaze of surveillance in which police, governments, and private corporations are installing surveil- lance cameras throughout entire cities, buildings, etc., while at the same time suspecting anyone who takes photographs of them, or their buildings, or their surveillance infrastructure, or the like [7]. This Surveillance Society [5] characterized by: increased surveillance by authority figures; and the authorities’ simultaneous forbidding of non- authorities from photography. See Fig 2. Business establishments often forbid the use of smart- phones, but sell products that 2-D barcodes (“QR codes”) intended to give customers pre-purchase product information that will help the customers decide what to purchase. These QR codes require cameras or smartphones to read them. See Copyright © i-Society 2013 Technical Co-Sponsored by IEEE Toronto Section 51

FreeGlass for developers, “haccessibility”, and Digital ... · FreeGlass for developers, “haccessibility”, and Digital Eye Glass + Lifeglogging Research in a (Sur/Sous)Veillance

  • Upload
    others

  • View
    3

  • Download
    0

Embed Size (px)

Citation preview

Page 1: FreeGlass for developers, “haccessibility”, and Digital ... · FreeGlass for developers, “haccessibility”, and Digital Eye Glass + Lifeglogging Research in a (Sur/Sous)Veillance

FreeGlass for developers, “haccessibility”, andDigital Eye Glass + Lifeglogging Research in a

(Sur/Sous)Veillance SocietySteve Mann, Mir Adnan Ali, Raymond Lo, Han Wu

University of Toronto, Dept. Electrical and Computer Engineering

Abstract—This paper presents “FreeGlass”, a hands-free user-buildable wearable computer system based on Mann’s “DigitalEye Glass” (also known as EyeTap) concept developed 35 yearsago. Like Mann’s “Eye Glass”, FreeGlass is based on freeand Open Source principles consistent with a free and openi-Society. FreeGlass is suitable for researchers and creatorsof AR (Augmediated Reality) Glass, LifeGlogging, etc. in theinformation society. Two example applications of FreeGlass arealso presented: Reality User Interface; and “ALIBEye”, a digitalalibi through sousveillance (inverse surveillance). We note thepotential for FreeGlass to bring about a transition from asurveillance society (cameras affixed to land and buildings),to a sousveillance-society (cameras held, carried, or worn byindividuals).

Important elements of a balanced sur/sous-veillance society arealso presented with a view toward a middle-ground between asurveillance-only society where large entities record interactionsbut forbid individuals from keeping their own record of theirsensory information, and the coming sousveillance society inwhich individuals are also equipped with veillance capacity, andthe capacity to capture AND authenticate their own recordingsas digital alibis.

Keywords: Wearable Computing, FreeGlass, Sousveillance,Surveillance, Veillance, Alibi Sousveillance, Augmented Re-ality, Augmediated Reality, Mediated Reality, EyeTap, Life-Glogging, Digital Eye Glass, MannGlas

I. SURVEILLANCE IN THE INFORMATION SOCIETY

Information society is growing at a tremendous pace,through the business economy of cloud computing. The properworking of this i-Society requires intelligent strategies inresponse to complex attacks [1], as well as an effective e-Government strategy [2]. In addition to the need for securitythere is also the need for forensic extraction of data to meetthe legitimate needs of law enforcement [3]. With mobile,portable, and wearable computing, there are even furthersecurity needs [4].

Surveillance, together with necessary oversight mecha-nisms, has a legitimate and important place in a lawful andjust i-Society. Surveillance is a well-known and, for the most-part, accepted practice. The word “surveillance” is a Frenchword that means watching (“veillance”) from above (“sur”).See Fig 1.

The i-Society has become a “Surveillance Society” so“Sureillance Studies” is an area of increasing importance [5].Surveillance is defined, formally, and precisely, as:sur-veil-lance [noun]

(a) (b)

Fig. 1. (a) Surveillance camera overlooking residential area in Toronto,Canada. (b) Kindergarten student (Age 6) fluent in multiple languages draws:Surveillance/Uberwachung/Oversight = watching from Over (above); andSousveillance/Unterwachung/Undersight = watching from Under (below),e.g. by way of a wearable camera.

1) a watch kept over a person, group,etc., especially over a suspect, prisoner,or the like: ‘‘The suspects were underpolice surveillance.’’ (Random House, Inc.Dictionary, 2013) Merriam Webster provides four words assynonyms for surveillance: supervision; superintendence;oversight; control, and two examples:

• “government surveillance of suspected terrorists”;• “The bank robbery was recorded by surveillance video

cameras.”.Society is becoming more like a panoptic [6] prison, with

regards to the one-sided gaze of surveillance in which police,governments, and private corporations are installing surveil-lance cameras throughout entire cities, buildings, etc., whileat the same time suspecting anyone who takes photographs ofthem, or their buildings, or their surveillance infrastructure, orthe like [7]. This Surveillance Society [5] characterized by:

• increased surveillance by authority figures; and• the authorities’ simultaneous forbidding of non-

authorities from photography. See Fig 2.Business establishments often forbid the use of smart-

phones, but sell products that 2-D barcodes (“QR codes”)intended to give customers pre-purchase product informationthat will help the customers decide what to purchase. TheseQR codes require cameras or smartphones to read them. See

Copyright © i-Society 2013 Technical Co-Sponsored by IEEE Toronto Section 51

Page 2: FreeGlass for developers, “haccessibility”, and Digital ... · FreeGlass for developers, “haccessibility”, and Digital Eye Glass + Lifeglogging Research in a (Sur/Sous)Veillance

(a) (b) (c)

Fig. 2. (a) Signage at the entrance to a typical supermarket informs shoppersthat (1) they are under surveillance (Closed Circuit TeleVision); and (2) thatphotography is prohibited. (b) Closeup of the sign asserting that the premesisis monitored by CCTV. (c) Closeup of the sign asserting “NO PHOTOS”.

(a) (b)

Fig. 3. (a) A massive advertisement with a QR code in a building in abuiding where cameras and smartphones are prohibited. The act of takingthis picture to document this absurdity was itself an act of civil disobedience.(b) Closeup showing the QR code, meant to be read by a camera.

Fig 3. Other businesses like McDonalds, for example, forbidthe use of smartphone-based translators. For example, a travelagent was physically assaulted by staff for photographing themenu at McDonalds (See http://eyetap.blogspot.ca/).

II. MITIGATING ONE-SIDEDNESS OF SURVEILLANCE

There are two strategies people use against one-sidedsurveillance:

1) Anti-surveillance (counter-surveillance, or opposing thesurveillance) efforts range from activism and legalaction to illegal vandalism of surveillance cameras:http://camover.noblogs.org/faq/faq-in-english/ or to sim-ply blocking, covering up, or “hacking” sensors (“hac-tivism”, etc.). In respose to these legal and illegalchallenges to surveillance, many police departments,governments, and private companies are using hid-den surveillance cameras, e.g. hidden inside streetlightpoles throughtout various cities [http://intellistreets.org,www.lsgc.com/pixelview/, etc.].

2) Adding accountability to surveillance without neces-sarily opposing it. This may be done by: (2.1) wear-ing a display device showing current news events tohelp prevent timecode falsification by entities mak-ing surveillance recordings. This approach will be dis-cussed in Section VII; or (2.2) by way of sousveillance(making one’s own recording), also known as “life-glogging” (lifelong cyborglogging), “lifelogging”, or“moblogging” [8]–[11]. See Fig 1(b). Sousveillance isnot counter-surveillance! In fact, methods (2.1) and (2.2)can be combined so that the two veillances can worktogether in a fair and balanced way, as will be describedin Section VII.

LIGHTSPACESYNTHESISGLASS

NUMERICALDESCRIPTION

10011000...

EYE

"DIGITAL EYE GLASS"

NUMERICALDESCRIPTION

10011000...

LIGHTSPACEANALYSISGLASS

INCOMINGREAL LIGHTRAYS

SUBJECTMATTER

OUTGOINGSYNTHETIC(VIRTUAL)LIGHT RAYS

WEARABLECOMPUTER

Fig. 4. A theory of Glass: rays of eyeward bound light are captured byan “analysis glass”, then processed by the wearable computer in the glasssystem, and then re-synthesized by a “synthesis glass”.

III. A THEORY OF GLASS: BACKGROUND INFORMATION

Digital Eye Glass is an Augmediated Reality system com-prised of three parts: image sensor, e.g. a camera system;image processor, e.g. a wearable computer system responsiveto an input from the image sensor; and image display, e.g. awearable display system responsive to an output of the imageprocessor.

The wearable computer system reconfigures the eye(s) forAR (Augmediated Reality), i.e. a visual reality that can beaugmented or delierately diminished to, for example, helppeople see better. The original example we developed in the1970s was to help people see better while welding (e.g. todiminish the intensity of the arc while augmenting the darkareas of the image, as well as overlaying graphical informationonto the scene) [12]. In this sense we originally referred to thedevice as “Glass” (like a welder’s glass) rather than “Glasses”(the more common plural form of the word). The term DigitalEye Glass has also been used [13]. See Fig. 4

Rays of eyeward-bound light strike a “Lightspace AnalysisGlass” (which need not necessarily be flat, and is thereforedepicted as being curved), are converted to numbers whichmay then be processed by the wearable computer. The result-ing numbers are fed to a “Lightspace Synthesis Glass” to beconverted back into light. This allows the wearable computerto become a visual intermediary, to, for example, diminish thebright areas of the Subject Matter, and Augment/Mediate thebright areas, before resynthesizing the rays of light into theEye, as shown in Fig 4.

A. Generation-1 Glass:

S. Mann built a rough approximation to this glass in 1978,using a television camera as the “Lightspace Analysis Glass”a miniature glass cathode-ray tube display as the “LightspaceSynthesis Glass” (over the right eye), and some electric circuitsas the wearable computer.

Because the camera was located beside the eye the long-term effects after many hours of wearing the apparatus con-sisted of an adaptation to this strange way of seeing, and theadaptation persisted after removal of the apparatus.

Copyright © i-Society 2013 Technical Co-Sponsored by IEEE Toronto Section 52

Page 3: FreeGlass for developers, “haccessibility”, and Digital ... · FreeGlass for developers, “haccessibility”, and Digital Eye Glass + Lifeglogging Research in a (Sur/Sous)Veillance

RIGHTMOST

RAY OF LIGHTLEFTMOST

RAY OF LIGHT

CA

ME

RA

EYE

LEFTMOST

RAY OF

VIRTUAL LIGHT

RIGHTMOST

RAY OF

VIRTUAL LIGHT

DIVERTER

AR

EM

AC

d

d

GENERATION−2 GLASS

Fig. 5. Generation-2 Glass: The eye itself is, in effect, both the camera anddisplay. This eliminates the undesirable “flashback” effects that otherwise arisefrom long-term usage. [14].

Others found similar effects, e.g. George Stratton, in hiswork of 1896 with upside-down glass (done optically ratherthan electrically), found that it took him about a week to adaptto seeing with mediated vision, and about one full day to returnto seeing properly (normally) after removing the glass.

S. Mann observed that mappings that deviate moderatelyfrom what the unaided eye saw, were harder to “forget” thanmappings that were either closer to or further from what theunaided eye saw. Thus the camera must, in effect be the eyeitself, in order to avoid the strange “flashback” effects of theGeneration-1 Glass.

B. Generation-2 Glass

In Generation-2 Glass, Rays of eyeward-bound light werediverted into a camera system that fed to the WearableComputer which then drove an Aremac (Lightspace SynthesisGlass). In this way rays of light that would have entered theunaided eye were collinear with rays of light presented by theGen-2 Glass. See Fig 5.

C. Generation-3 Glass

Particularly in monocular systems, when one eye is focusedon a display, and the other is focused on direct “reality”,there is a difference in focus distance between the two eyes.To overcome this problem, we developed Generation-3 Glass,using a special display called an “aremac” which has theability to present visual information at an apparent focaldistance that can be controlled by the wearable computer. Thecomputer senses where objects are in 3-dimensional space,and makes an inference as to what range of focal distancesthe non-glass-eye can focus to. Information is presented tothe glass-eye in this same focal range, which is automaticallyupdated several times per second.

D. Generation-4 Glass

While looking at objects in various focal planes, such aslooking at a distant object through a nearby screen or chainlinkfence, some problems remained with Gen-3 Glass.

Generation-4 Glass. uses a laser light source with a spatiallight modulator, and related apparatus, to attain an infinitedepth-of-focus, wherein displayed information is always in

RETINA

EYE

EYE LENS

VIRTUAL

LIGHT

INCOMING

REAL LIGHT

PS SLM COND

DIVERTER

CAMERA

"PINHOLE"

AREMAC

PS SLM COND

EYE LENS

EYE

RETINA

RETINA’

EYE’

0th ORDER

EYE LENS’

HIGHER

ORDERS"PINHOLE"

AREMAC

GENERATION-4 GLASSFig. 6. Generation-4 Glass uses a laser light source to achieve infinite depth-of-focus to bypass the eye’s lens. Therefore the glass-eye is not presented withinformation at any particular focal distance, leaving it free to focus at whateverdistance the other eye is focused to [14].

Fig. 8. Leftmost: Generation-4 Glass completed in 1999. The eye itself isthe camera exactly! That is why the “Digital Eye Glass” is also known asthe “GlassEye” (The eye looks exactly like a camera lens). This eliminateslong-term dizziness, vertigo, and flashback effects that can otherwise persistlong after the Glass is removed. Rightmost: Google Glass, 2012, where thecamera being to one side of the eye makes it a Generation-1 Glass, subjectto some of the problems discovered earlier on. [15]

focus regardless of where the eye focuses. The laser eyetapbypasses the lens of the glass-eye. In this way the Gen-4 Glassdoes not cause the eye to focus at any particular distance,leaving it free to focus where the non-glass eye is focused.See Fig 6.

Generations 2 to 4 Glass were known as “EyeTap DigitalEye Glass” [13] (the word “Glass” appears in singular form,not plural, i.e. “Eye Glass” not “Eye Glasses”).

The result was a natural experience with zero eyestrainwhich could be worn continuously many hours a day formany years. The Generation-4 Glass was completed in 1999,described in detail in 2001 [12]. See Fig. 8 (leftmost).

IV. GENERATION-5 GLASS

Generation-5 Glass is now proposed. Generation-5 is de-fined as being bi-directional in its light sensing and lightoutput. In this way Gen-5 Glass senses both the scene infront of the wearer as well as the wearer’s own eyes. It alsoilluminates both the wearer’s eye, and the scene in front ofthe wearer. See Fig 9

Gen-5 Glass is an ongoing effort — a work-in-progress.We propose a reseach platform that others can easily buildand duplicate and use in order to research and develop forGen-5 Glass when it becomes available.

The research platform comprises 2 components:• a simple hardware arrangement that anyone can easily put

together using commercial off-the-shelf parts, along withsimple parts that can be printed on a low-cost 3D printer.The proposed hardware arrangement is based on the

Copyright © i-Society 2013 Technical Co-Sponsored by IEEE Toronto Section 53

Page 4: FreeGlass for developers, “haccessibility”, and Digital ... · FreeGlass for developers, “haccessibility”, and Digital Eye Glass + Lifeglogging Research in a (Sur/Sous)Veillance

(a) (b) (c) (d) (e)

(f) (g) (h) (i) (j)Fig. 7. 3D rendering of a simple hardware configuration that anyone can easily put together from commercial off-the-shelf components. Our 3D Solidworksparts and overall system is released Open Source. (a) 3D model of head with simple harware configuration; (b-d) 3D printable part. (e) part printed on alow-cost home-made 3D printer with components arranged as they fit together. The items were placed on a table having a lattice of holes with 1 inch (25.4mm)spacing to give a rough idea of scale. (f-j) Improved design with a Go-ProTM camera mount on top for parallel 3D HDTV documentation, and other fittings.The new mount also connects with a MindMeshTM BCI (Brain-Computer Interface), as shown in (j), a data projector, AremacTM , eye-tracker, and variousother third-party accessories.

NUMERICAL

DESCRIPTION

10011000...

EYE

NUMERICAL

DESCRIPTION

10011000...

SUBJECT

MATTER

WEARABLE

COMPUTER

INCOMING AND

OUTGOING

LIGHT RAYS

INCOMING AND

OUTGOING

LIGHT RAYS

GENERATION−5 GLASS

LIGHTSPACE

GLASS

(BIDIRECTIONAL)

LIGHTSPACE

GLASS

(BIDIRECTIONAL)

Fig. 9. Generation-5 Glass. Conceptually it is characterized by a bidirec-tional sensing and display modality. Rays of light are sensed from BOTHTHE SCENE AND THE EYE. This facilitates eye-tracking as well as sceneanalysis. Computer-generated rays of light are also sent to BOTH THE EYEAND THE SCENE to facilitate active vision as well as being able to displaycontent to other people who are not wearing the glass.

EPSON Moverio true stereo display glass, together with atime-of-flight 3D camera or PrimeSense 3D sensing cam-era, and an easily 3D printed part to join the two together,as shown in Fig 7. Other options such as a miniature laserdata projector, eye-tracker, IMU (Integrated Motion Unit),and brain-computer-interface are also attached;

• a sophisticated software environment that simulates theEyeTap principle. Because the camera is a 3D camera itcaptures spatial-tonal information, allowing its point-of-view to be re-rendered from a left Point-of-Eye for theleft eye, and, separately, a right Point-of-Eye for the righteye of the wearer.

A. Sousveillance versus surveillance with 3D cameras

Surveillance is an application where cameras are affixed tothe environment, so there is no relative movement betweenthe camera and the static objects in the background. But with

sousveillance, the camera is affixed to a person, thus causingrelative movement between the camera and static backgroundobjects. For this reason, computer vision for sousveillance is amuch harder problem than for surveillance, where all we needto do is subtract the background.

However, 3D sensing cameras, originally designed forsurveillance (being fixed to architectural elements like a shelfin a living room for example), can be applied to sousveillance.

In particular, an important observation to make is the 3Dcamera’s ability to ignore subject matter beyond a certaindistance from the camera. In this way the 3D camera candistinguish between gestures and near-field subject matter andbackground clutter. For this reason, 3D cameras are morewell suited to sousveillance than the surveillance they weredesigned for [16].

Wearing the 3D camera gives a setup optimized for near-field time-of-flight sensing of hand gestures at close range,while ignoring the background — a task that has previ-ously been very difficult with 2D wearable camera systems.Background clutter is eliminated from the scene by simplydetermining which areas of the image that are beyond thedepth-range of the sensor.

V. APPLICATIONS

We present two applications of the Gen-5 Glass:• Reality User Interface;• “ALIBEye: Eye Was Here”.

VI. REALITY USER INTERFACE

Unlike the typical GUI (graphical user interface) found indesktop computing, or mobile devices, our system requiresonly simple gestures, such as the “Outstretched-Hands” ges-ture in front of any subject matter, as shown in Fig. 10. Thisenables seamless interaction between the physical world andDigital Eye Glass. For example, reaching out to a tree, the tree

Copyright © i-Society 2013 Technical Co-Sponsored by IEEE Toronto Section 54

Page 5: FreeGlass for developers, “haccessibility”, and Digital ... · FreeGlass for developers, “haccessibility”, and Digital Eye Glass + Lifeglogging Research in a (Sur/Sous)Veillance

(a) (b) (c)

Fig. 10. (a) A practical approximation to Generation-5 Glass: A stereo vision display with a true time-of-flight 3D camera system. (b) Hand gestureautomatically recognized. Automatic detection of subject matter (tree). Hands and subject matter (tree) segmented from scene using depth informationcaptured by the 3D sensor. (c) Information retrieved and overlayed onto stereo display in real-time. Overlays are rendered next to subject matter for ease ofreading/viewing.

“answers back” with information about itself. Although theLandscaping Department has a database of all trees planted,and thus the knowledge could be obtained “top-down” (i.e.from the authorities), our application allows individuals toacquire, disseminate, and share knowledge in a “bottom-up”way, i.e. through shared sousveillance. The 3D camera in theeyeglass collects information about the environment, enablingindividuals to be producers of — and not merely consumers of— information. Thus the proposed Gen-5 Glass answers thecall of “hacccessibility”, i.e. the DIY (Do-It-Yourself) spiritof community-based knowledge acquisition during and abouteveryday life activities, like enjoying nature.

VII. ALIBI SOUSVEILLANCE

Our second application is called “ALIBEye: Eye Was Here”.It provides a lifeglogging experience that can be used as analibi, in a court-of-law, for example, to prove one’s time andplace.

One way of mitigating the one-sided nature of surveillanceis to hold those doing the surveillance accountable. This can bedone by sousveillance (keeping one’s own recording of whathappens). In this case, it is useful to timestamp one’s ownsousveillance recordings so that they can be used as an alibiin any accusation of wrongdoing. Another way of mitigatingthe one-sided nature of surveillance is to timestamp thesurveillance data. This can be done without the consent of —and even against the will of — the authorities conducting thesurveillance. An example of this is the display of content intothe surveillance data. For example, if you simply walk aroundwith a copy of the front page of a newspaper, and display itto every surveillance camera you see, you’ve greatly reducedthe ability of the authorities to backdate the surveillance video(See Fig 12). Wearing a small 2D bar-code that updates witha hash of the day’s news headlines, for example, can serve asimilar effect.

Ideally a combination of these two methods can be used,in which one records one’s own sousveillance video, whilealso displaying to one’s self AND to other cameras (othersousveillers and the surveiller), a hash of today’s latest news,and then timestamping that sousveillance recording.

The use of a visual recording to provide an alibi requires thatthe time of recording is bounded both before AND after therecording takes place. Standard cryptographic timestampingof a hash of a digital recording provides evidence that therecording took place before the timestamp; i.e. the signed hashdepends on the recording. We now describe approaches tobound the time of recording in the other direction, providingevidence that the recording took place after a certain time,preventing replay attacks, and preventing temporal falsificationin BOTH temporal directions.

Alibi veillance can be used for both surveillance and sous-veillance and in various combinations of these veillances.

A. Prior work: Timestamps are all one-sided

A digital timestamp can prove to the world that a certain setof bits existed before a certain time, namely when the times-tamp was created [17]; examples are formalized in IETF’sRFC 3161 [18], RFC 3339 [19], and ANSI’s X9.95 [20].

B. Alibi Veillance: Another side to timestamping

To create an alibi veillance system, three elements arerequired: (A) an a priori unknowable fact that is widelyobserved, ideally in a precise digital form; and (B) a way toembed A into a real-world scene, and record the embedding,in a way that is very difficult to falsify; and (C) a verifiabletimestamp service, to which a hash of the recording is can beimmediately submitted to.

Steps A and B bound the time in one direction, and step Cbounds it in the other temporal direction, thus resulting in abi-directional bounding of the alibi veillance.

C. Alibi displays

A simple embodiment of this idea is to display a hashof current events, as a QR code. This can be done, forexample, on a GNU Linux wristwatch that runs a “cron job”to automatially display the material now and again, so asto be visible to the wearer of a sousveillance eyeglass, orother wearable camera that can view the wristwatch (as wellas other cameras in the environment, including surveillancecameras, being able to view the wristwatch). See Fig 11.

Copyright © i-Society 2013 Technical Co-Sponsored by IEEE Toronto Section 55

Page 6: FreeGlass for developers, “haccessibility”, and Digital ... · FreeGlass for developers, “haccessibility”, and Digital Eye Glass + Lifeglogging Research in a (Sur/Sous)Veillance

(a) (b)Fig. 11. (a) Video frames from a sousveillance video where a GNU Linuxwristwatch is seen in-frame. The barcode display shows the latest BITCOINhash, which — like the day’s news — is difficult to predict, and due to theglare of the watchface, it is very difficult to falsify the video by editing it. (b)A neckworn display used to visibly advertise a temporal code for the benefitof other parties subjecting us to surveillance or sousveillance.

Fig. 12. A demonstration of the ALIBEye app using a “News Flash”with the Gen-5 Glass. A current news headline (Obama’s Victory Speech)and a 2D bar-code are flashed (briefly projected) onto the environment, e.g.here, another person’s shirt, to create a NEWStampTM that can be seen andrecorded by the wearer and by others. The flash duration can be short enoughto be invisible to the naked eye, but only seen by one or more cameras(including the wearer’s camera) wirelessly synchronized to the “News Flash”.

Another possible embodiment is a necklace that displays thecurrent-events QR hashcode. See Fig 11(b).D. Visual Vicarious Soliloquy

Another approach is for the user to occasionally write outor self-gesture the hash code. A recording, from a wearablecamera, of one writing out a temporal code in one’s ownhandwriting is hard to fake.

E. ALIBEye

ALIBEye is an app that uses Gen-5 Glass to project a pattern(visible or infrared) onto the scene and captures recordings ofthat pattern, as shown in Fig 12. The barcode can also includea hash of a recording of the wearer’s own eye, iris pattern,gaze, eye movements, etc., to further biometrically establishtheir presence at the scene. The resulting complex interactionis difficult to falsify. ALIBEye requires little to no attentionfrom the user, and providing continuous automatic updating tothe latest temporal code, and provides the safety and securitybenefit to others, not just the wearer.

VIII. CONCLUSION

We have described a simple easy-to-build “FreeGlass”Generation-5 Digital Eye Glass system along with some

simple applications that provide userful interaction and canalso mitigate the one-sided effects of surveillance. In oneapplication, the Generation-5 Glass works as a truly interactiveself-gesture-sensing apparatus. Another application, “ALIB-eye” (Eyeworn Alibi Veillance) was presented, to make itdifficult for the wearer to falsify the timestamp of a sous-veillance recording. Sousveillance is not anti-surveillance, and,in fact, ALIBEye can even facilitate a cooperation betweensousveillance and surveillance so that neither party can falsifythe veracity or time-stamp of their veillance recordings.

REFERENCES

[1] E. Hooper. Intelligent detection and response strategies for complexattacks. Aerospace and Electronic Systems Magazine, IEEE, 22(11):3–12, 2007.

[2] D. C. Edwards, P. Zavarsky, R. Ruhl, D. Lindskog, and S. Aghili. E-government system security model (egssm): A multidimensional, riskbased approach to e-government. In Privacy, security, risk and trust(passat), 2011 ieee third international conference on and 2011 ieee thirdinternational conference on social computing (socialcom), pages 1273–1277. IEEE, 2011.

[3] F. Olajide and N. Savage. Forensic extraction of user information incontinuous block of evidence. In Information Society (i-Society), 2011International Conference on, pages 476–481. IEEE, 2011.

[4] D. Fischer, B. Markscheffel, S. Frosch, and D. Buettner. A surveyof threats and security measures for data transmission over gsm/umtsnetworks. In Internet Technology And Secured Transactions, 2012International Conferece For, pages 477–482. IEEE, 2012.

[5] D. Lyon. Surveillance society. Open University Press Buckingham,2001.

[6] M. Foucault. Discipline and Punish. Pantheon books, New York, 1977.Translated from “Surveiller et punir”.

[7] O. H. Gandy. The surveillance society: information technology andbureaucratic social control. Journal of Communication, 39(3):61–76,1989.

[8] S. Mann, J. Nolan, and B. Wellman. Sousveillance: Inventing andusing wearable computing devices for data collection in surveillanceenvironments. Surveillance & Society, 1(3):331–355, 2002.

[9] G. Fletcher, M. Griffiths, and M. Kutar. A day in thedigital life: a preliminary sousveillance study. SSRN,http://papers.ssrn.com/sol3/papers.cfm?abstract id=1923629,September 7, 2011.

[10] K. Michael and M. G. Michael. Sousveillance and point of viewtechnologies in law enforcement: An overview. 2012.

[11] J. Bradwell and K. Michael. Security workshop brings’ sous-veillance’under the microscope. 2012.

[12] S. Mann. Intelligent Image Processing. John Wiley and Sons, November2 2001. ISBN: 0-471-40637-6.

[13] P. Nowak. The world’s first ’cyborg,’ Steve Mann, says always beingconnected to others can make people see the world in a different — andbetter — light. CBC News, Monday Dec. 22, 2003.

[14] S. Mann. Through the glass, lightly. IEEE Technology &Society, 31(3):10–14, 2012; see also supplemental material inhttp://wearcam.org/glass.pdf.

[15] S. Mann. Vision 2.0. IEEE Spectrum, 50(3):42–47, 2013.[16] S. Mann, J. Huang, R. Janzen, R. Lo, V. Rampersad, A. Chen, and

T. Doha. Blind navigation with a wearable range camera and vibrotactilehelmet. In Proceedings of the 19th ACM international conference onMultimedia, pages 1325–1328. ACM, 2011.

[17] B. Schneier. Applied Cryptography (2Nd Ed.): Protocols, Algorithms,and Source Code in C. John Wiley & Sons, Inc., New York, NY, USA,1995.

[18] C. Adams, P. Cain, D. Pinkas, and R. Zuccherato. Internet X.509 PublicKey Infrastructure Time-Stamp Protocol (TSP). RFC 3161 (ProposedStandard), August 2001. Updated by RFC 5816.

[19] G. Klyne and C. Newman. Date and Time on the Internet: Timestamps.RFC 3339 (Proposed Standard), July 2002.

[20] ANSI. Trusted Time Stamp Management and Security, July 2012.

Copyright © i-Society 2013 Technical Co-Sponsored by IEEE Toronto Section 56