Mark Frank - RU HMS Symposium 2003 Deception, digitization, & counter-terror. Mark G. Frank School of Communication, Information, & Library Studies UCSF

Mark Frank - RU HMS Symposium 2003

Deception, digitization, & counter-terror.Deception, digitization, & counter-terror.

Mark G. FrankMark G. Frank Associate Professor

School of Communication, Information, & Library StudiesSchool of Communication, Information, & Library Studies

with special thanks to: Paul Ekman, UCSFUCSF;

Dimitris Metaxas, Sinuk Kang, Amy Marie Keller, Anastacia Kurylo,Maggie Herbasz, Belida Uckun, RutgersRutgers;

Jeff Cohn, PittPitt; Takeo Kanade, CMUCMU; Javier Movellan & Marni Bartlett, UCSDUCSD.

Also thanks to: Office of Naval Research and the National Science Foundation



Signals relevant to counter-terror.Signals relevant to counter-terror.

• Changes in gait with loads as small as 1 kgChanges in gait with loads as small as 1 kg

• Anger prior to imminent attackAnger prior to imminent attack

• Fear/distress when lyingFear/distress when lying



How lies are betrayed.How lies are betrayed.

LieLie

Cognitive cluesCognitive clues

- Contradictory statements- Contradictory statements- HesitationsHesitations

-Speech errorsSpeech errors-Reduced illustratorsReduced illustrators

-Contradictory emblemsContradictory emblems-Reduced detailReduced detail

-Etc.Etc.

Emotional cluesEmotional clues

Lying about feelingsLying about feelings Feelings about lyingFeelings about lying

-Look for -Look for reliablereliable

signs of signs of emotionemotion

-Duping delight-Duping delight-GuiltGuilt

-DetectionDetectionapprehensionapprehension


Facial expressions in humans. Facial expressions in humans.


Some reliable facial muscles.Some reliable facial muscles.

• Reliable angerReliable anger- Orbicularis oris

• Reliable disgustReliable disgust- levator

• Reliable fear/distressReliable fear/distress- medial frontalis

• Reliable enjoymentReliable enjoyment- Orbicularis oculi


Anger.Anger.


Anger.Anger.

• Eyebrows pulled down• Upper lids pulled up• Lower lids pulled up• Red margins of lips

rolled in• Sometimes lips tight


Fear.Fear.


Fear.Fear.

• Eyebrows pulled up and together

• Upper eyelids pulled up• Mouth stretched


The Facial Action Coding System The Facial Action Coding System (FACS)(FACS)

Ekman & Friesen, 1978

Action code: 1, 2, 4, 5, 7, 20, 26

1 Inner brow raise2 Outer brow raise4 Brow lower5 Upper lid raise7 Lid tighten20 Lip stretch26 Jaw drop

46 Action Units


Examples (con’t).Examples (con’t).

• Brow Motion – Upward motion

• AU 1+2– Contraction of the frontalis,

pars medialis and lateralis

– Downward motion• AU 4 or 9

– Contraction of the corrugator supercilii, depressor supercilii (AU 4)

– Contraction of the levator labii superioris alaeque nasi (AU 9)


Subtle brow changes important.Subtle brow changes important.


Using FACS to identify emotion.Using FACS to identify emotion.

• Advantages:Advantages:

- reliably scored

- non-obtrusive

- non-inferential, allows for discovery

• Disadvantage:Disadvantage:

- laborious

• SolutionSolution- automatic computer vision techniques


Technical Challenges.Technical Challenges.

• PosePose• Head motionHead motion• Occlusion from glasses, facial hair, rotation, Occlusion from glasses, facial hair, rotation,

handshands• TalkingTalking• Video qualityVideo quality• Frame rate (blinks)Frame rate (blinks)


Conclusions.Conclusions.

• There are reliable signals associated with There are reliable signals associated with deception and hostile intent.deception and hostile intent.

• We can detect these behaviors. We can detect these behaviors.

• We can represent them digitally.We can represent them digitally.

• Can this make us more secure?Can this make us more secure?

Documents

Mark Frank - RU HMS Symposium 2003 Deception, digitization, & counter-terror. Mark G. Frank School of Communication, Information, & Library Studies UCSF