The experience of cinema Tom Troscianko · O'Regan, J.K. (1992). Solving the "real"...

The experience of cinema

Tom Troscianko

Department of Experimental PsychologyUniversity of Bristol

Collaborators

Kay Nelson (1993)Steve Hinde (2011)

Laura Gregory (2006)Annelise Bhatt (2007)

Roland BaddeleyDerek Carr

Ian Christie (Birkbeck)

The film question

What is a film and how do we perceive it?

The visual cognition questions

• How do we perceive a changing world?• Is the world as depicted in film markedly

different from the one outside the cinema?

• How do any such differences affect perception when viewing a film?

The traditional approach to vision

• Attention• Models in the head

Models in the head?

• We make eye movements and traditional accounts of vision said that we construct a detailed model of the world in our head

• It feels to us that this is what is happening

• But then people started doing experiments…

Questioning the “picture in the head” idea

• If we have a detailed representation in our head, then we should have no difficulty in noticing changes in a scene which occur across interruptions of processing

• “Change blindness” experiments in 1990’s show that we are bad at such tasks

Change blindness• Suppose that we present two images

which are separated in time• And during the separation (e.g. flash) we

remove a salient object• If we have a “full and stable”

representation of the world we should always notice the removal of the object

The trans-saccadic method

Blackmore SJ, Brelstaff G, Nelson K, Troscianko T (1995) Is the richness of our visual world an illusion? - Trans-saccadic memory for complex scenes. Perception 24, 1075-1081.

Movie: continuity error

Show video Office1

More modern accounts of vision

• We pick up information about a scene from eye movements and this is processed in a manner which guides action

• Fixations pick up information about:– The “gist” of the scene– Items with which we are about to interact– The gross geometry of the scene

Sensorimotor theory of vision• O'Regan, J.K., & Noe, A. (2001). A sensorimotor

account of vision and visual consciousness. Behavioral and Brain Sciences, 24(5), 939-1031.

“The central idea of our new approach is that vision is a mode of exploration of the world that is mediated by knowledge of what we call sensorimotor contingencies.”

“It could be said that the outside world acts as an external memory that can be probed at will by the sensory apparatus.”

O’Reagan and the Stable World Assumption

“The visual system is particularly sensitive to visual transients (Breitmeyer & Ganz 1976; Stelmach et al. 1984; Tolhurst 1975). When a visual transient occurs, an automatic, “alerting” or “attention-grabbing” mechanism appears to direct processing to the location14 where the transient occurred (Theeuwes 1991; Yantis 1998). This means that should anything happen in the environment, we will generally consciously see it, since processing will be directed to it. This gives us the impression of “having tabs” on everything that might change, and so, of consciously seeing everything. Were there not the attention-grabbing mechanism, our visual impression would be more similar to the impression we have when we stand with our backs to a precipice: we keenly feel it is there, we know that we can turn and see more of the precipice, but the feeling of presence is much less vivid than when we are actually looking into the precipice.”

O'Regan, J.K. (1992). Solving the "real" mysteries of visual perception: The world as an outside memory. Canadian Journal of Psychology, 46(3), 461-488.

“In cinema viewing, even though the camera cuts continually from one viewpoint to another, viewers have no difficulty apprehending the spatial arrangement of the set. It seems that viewers do not attempt to build up a coherent metric replica of the set, but are satisfied with what might be termed a "semantic" representation of it, containing a number of statements such as: X is talking to Y, they are standing on the beach facing the waves, etc.”

Cinematic viewing

• The visual world of the cinema is quasi-stable

• It keeps jumping to a new shot• But “continuity rules” ensure that

relevant perceptual processes can survive

• However, the basic signal is unstable• What do we know about is instability?

Cutting paper

• Cutting JE, DeLong J, Nothelfer CE (2010) Attention and the evolution of Hollywood film. Psychological Science doi:10.1177/0956797610361679

• Studies shot length statistics for 150 films from 1935 to 2005

• Measured autocorrelation function for shot length

“setting the actual narrative aside, perhaps being engrossed in a film is, in part, to allow its 1/f temporal structure to drive the mind exogenously”

Cutting paper

Perception of films

• The autocorrelation argument suggests that the “stable world” assumption can be modified to allow for an unstable world whose temporal statistics conforms to a certain law

• We shall return to test this issue directly• Meanwhile, we need a measure of

“movie perception”

Which film to use?

• Paper by Hasson U et al (2004) Intersubjectsynchronization of cortical activity during natural vision. Science 303, 1634-1640

• Measured cortical responses of people watching film “The Good, The Bad, and The Ugly” (Sergio Leone, 1966)

What can we measure?

• Initially, we need to measure “presnece” at various points during the movie

• Where “presence” is defined as the “illusion of non-mediation” i.e. feeling like you are “actually there”

The problem with measuring presence

• Presence is normally measured by giving subjects a questionnaire after the movie ends

• However, this does not permit comparison across scenes in the movie

• We decided to score subjective ratings of presence using a simple line-bisection task

• Each time a sound beeped or a light flashed, subjects bisected the line on a single piece of paper to indicate the extent to which they “felt present” in the movie

What film to use?• Hasson et al (2004) Science 209, 1631-1641• Used film “The Good, the Bad, and the Ugly” (1967,

United Artists)• Measured correlation in scene-dependent activation

across subjects in fMRI study• Established three main types of scene giving reliably

different activation profiles:– Faces (Fusiform gyrus, FFA)– Landscapes/buildings (Colleteral sulcus, PPA)– Hand-related scenes (Post-central sulcus)

• Therefore, measuring presence at these points in the movie should reflect different patterns of cortical activation in viewers.

FFA scenes

FFA scenes

PPA scenes

PPA scenes

Screen size experiments

Thanks to Lee De-Wit for the picture

Experimental details

• Presence is normally measured by giving ss a questionnaire after the movie ends

• However, this does not permit comparison across scenes in the movie

• We decided to score subjective ratings of presence using a simple line-bisection task

• Each time a light flashed, subjects bisected the line on a single piece of paper to indicate the extent to which they “felt present” in the movie

• 40 subjects in a between-groups design

Presence results

Screen size p<0.001

Scene type p<0.001

NS interaction

Presence and memory

The effect of screen size on presence and recognition memory for scenes

Experimental details

• Similar viewing conditions as in Exp 1• 30 subjects in a between-groups design• A beep sounded at key points in film

(first 45 minutes)• Subjects rated presence with the line

bisection task• Then they took part in recognition

memory expt – 28 clips, 2 sec each, half seen, half unseen

Presence results

Screen size p<0.005

Scene type p<0.05

NS interaction

Memory results

Screen size p<0.001

Scene type p<0.001

Size x type interaction p<0.001

Pressence, RT, arousal

The effect of screen size on presence, reaction time, and arousal

Experimental details

• Similar viewing conditions as in Expts 1 & 2

• 30 subjects in between-groups design• Subjects pressed a button in response

to beep – giving RT measure• Pupil diameter was recorded by a

custom-built pupillometer, giving an indication of cognitive load/arousal

Presence results

42

44

46

48

50

52

54

56

small large

Screen size

Rate

d pr

esen

ce

face

land

Screen size NS

Scene type p<0.05

NS interaction

Reaction time results

1.45

1.5

1.55

1.6

1.65

1.7

1.75

1.8

small large

Screen size

Reac

tion

time

(sec

)

face

land

Screen size NS

Scene type p<0.05

NS interaction

Pupil dilation results

0

0.05

0.1

0.15

0.2

0.25

0.3

0.35

small large

Screen size

Pupi

l dila

tion

(mm

)

face

land

Screen size NS

Scene type p<0.001

NS interaction

Preliminary data: colour vs b&w

Presence and edit statistics• We have a useful online measure of presence• Let us return to the “stable world” assumption• A film is not a very stable world – the perceiver sees

edits every few seconds• Clearly, continuity rules and possible “adaptation” to

1/f statistics mask instability• However, we may still expect a film with short cuts to

appear more unstable, and less “natural”, than one with long cuts

• It is therefore interesting to look for effects of mean cut length, and the standard deviation of mean cut length, on presence

Shoot-out scene experiment

• We used shoot-out scenes from obscure Western movies made in early 1960s

• Created a 26 minute showreel and tested presence on two groups of 21 subjects

• In counterbalanced order• In a group setting in a lecture theatre

Presence and mean shot length

Presence and mean shot length: replication with Good, Bad, Ugly

Presence and mean shot length: replication with Good, Bad, Ugly

Presence and mean shot length

Presence and mean shot length

Film clip 1

Presence and mean shot length

Film clip 3

Conclusions• The local presence measure appears to

reflect arousal or depth of processing• Does not easily predict memory performance• Show clear effect of screen size on presence• There is a relationship between presence and

Mean Shot Length• But the causal nature of this relationship is

unclear: does an editor make short edits in an exciting scene, or do short edits cause a scene to be exciting?

• Need further experiments to tease this apart• Also look at colour/bw and stereo/no-stereo

And why is this important?

And why is this important?Most expensive films

Rank Title Year Film costs in millions (est.)

1 Pirates of the Caribbean: At World's End 2007 $300m[3][6][nb 1]

2 Spider-Man 3 2007 $258m (official)[1]

3 Harry Potter and the Half-Blood Prince 2009 $250m[7][8]

4 Avatar 2009 $237m (official)[9][nb 2]

5

The Chronicles of Narnia: Prince Caspian 2008 $225m (official)[10]

Pirates of the Caribbean: Dead Man's Chest 2006 $225m[3][11][nb 1]

7 X-Men: The Last Stand 2006 $210m[12][13]

8 Superman Returns 2006 $209m (official)[14][nb 3]

9 King Kong 2005 $207m[15][16][17]

10

Transformers: Revenge of the Fallen 2009 $200m (official)[18]

2012 2009 $200m (official)[19]

Titanic 1997 $200m[1][20][21]

Spider-Man 2 2004 $200m[22][23]

Quantum of Solace 2008 $200m[24][25]

Terminator Salvation 2009 $200m[26][27]

Toy Story 3 2010 $200m[28][29]

So…• Measures of perception and cognition during

film viewing reveal effects of viewing conditions, edit regime, and use of colour

• Cognitive scientists can learn much about the brain from using these remarkable stimuli– Broader range of techniques (EEG, fMRI, remote

observation of audience)– Plus, they can begin to fill out the “Impact

Statement” on a grant proposal form…• Film producers and theatres can learn about

their audiences’ response to movies• Thus being able to estimate the benefits from

(say) installation of a 3D system

Speculation• We may have expected presence to reduce

when a film is more “unnatural”• I.e. when its shots are short/variable• The opposite seems to be true• It may be that an instability, or “error” signal

can be perceived as pleasurable if the perceiver appears to be “safe”

• Neuro-aesthetics literature shows activations of different parts of brain by pleasurable and aversive stimuli

• But films (and sunsets) may be examples of stimuli which are both pleasurable andaversive