Embed Size (px)
Citation preview
The experience of cinema
Tom Troscianko
Department of Experimental PsychologyUniversity of Bristol
Collaborators
Kay Nelson (1993)Steve Hinde (2011)
Laura Gregory (2006)Annelise Bhatt (2007)
Roland BaddeleyDerek Carr
Ian Christie (Birkbeck)
The film question
What is a film and how do we perceive it?
The visual cognition questions
• How do we perceive a changing world?• Is the world as depicted in film markedly
different from the one outside the cinema?
• How do any such differences affect perception when viewing a film?
The traditional approach to vision
• Attention• Models in the head
Models in the head?
• We make eye movements and traditional accounts of vision said that we construct a detailed model of the world in our head
• It feels to us that this is what is happening
• But then people started doing experiments…
Questioning the “picture in the head” idea
• If we have a detailed representation in our head, then we should have no difficulty in noticing changes in a scene which occur across interruptions of processing
• “Change blindness” experiments in 1990’s show that we are bad at such tasks
Change blindness• Suppose that we present two images
which are separated in time• And during the separation (e.g. flash) we
remove a salient object• If we have a “full and stable”
representation of the world we should always notice the removal of the object
The trans-saccadic method
Blackmore SJ, Brelstaff G, Nelson K, Troscianko T (1995) Is the richness of our visual world an illusion? - Trans-saccadic memory for complex scenes. Perception 24, 1075-1081.
Movie: continuity error
Show video Office1
More modern accounts of vision
• We pick up information about a scene from eye movements and this is processed in a manner which guides action
• Fixations pick up information about:– The “gist” of the scene– Items with which we are about to interact– The gross geometry of the scene
Sensorimotor theory of vision• O'Regan, J.K., & Noe, A. (2001). A sensorimotor
account of vision and visual consciousness. Behavioral and Brain Sciences, 24(5), 939-1031.
“The central idea of our new approach is that vision is a mode of exploration of the world that is mediated by knowledge of what we call sensorimotor contingencies.”
“It could be said that the outside world acts as an external memory that can be probed at will by the sensory apparatus.”
O’Reagan and the Stable World Assumption
“The visual system is particularly sensitive to visual transients (Breitmeyer & Ganz 1976; Stelmach et al. 1984; Tolhurst 1975). When a visual transient occurs, an automatic, “alerting” or “attention-grabbing” mechanism appears to direct processing to the location14 where the transient occurred (Theeuwes 1991; Yantis 1998). This means that should anything happen in the environment, we will generally consciously see it, since processing will be directed to it. This gives us the impression of “having tabs” on everything that might change, and so, of consciously seeing everything. Were there not the attention-grabbing mechanism, our visual impression would be more similar to the impression we have when we stand with our backs to a precipice: we keenly feel it is there, we know that we can turn and see more of the precipice, but the feeling of presence is much less vivid than when we are actually looking into the precipice.”
O'Regan, J.K. (1992). Solving the "real" mysteries of visual perception: The world as an outside memory. Canadian Journal of Psychology, 46(3), 461-488.
“In cinema viewing, even though the camera cuts continually from one viewpoint to another, viewers have no difficulty apprehending the spatial arrangement of the set. It seems that viewers do not attempt to build up a coherent metric replica of the set, but are satisfied with what might be termed a "semantic" representation of it, containing a number of statements such as: X is talking to Y, they are standing on the beach facing the waves, etc.”
Cinematic viewing
• The visual world of the cinema is quasi-stable
• It keeps jumping to a new shot• But “continuity rules” ensure that
relevant perceptual processes can survive
• However, the basic signal is unstable• What do we know about is instability?
Cutting paper
• Cutting JE, DeLong J, Nothelfer CE (2010) Attention and the evolution of Hollywood film. Psychological Science doi:10.1177/0956797610361679
• Studies shot length statistics for 150 films from 1935 to 2005
• Measured autocorrelation function for shot length
“setting the actual narrative aside, perhaps being engrossed in a film is, in part, to allow its 1/f temporal structure to drive the mind exogenously”
Cutting paper
Perception of films
• The autocorrelation argument suggests that the “stable world” assumption can be modified to allow for an unstable world whose temporal statistics conforms to a certain law
• We shall return to test this issue directly• Meanwhile, we need a measure of
“movie perception”
Which film to use?
• Paper by Hasson U et al (2004) Intersubjectsynchronization of cortical activity during natural vision. Science 303, 1634-1640
• Measured cortical responses of people watching film “The Good, The Bad, and The Ugly” (Sergio Leone, 1966)
What can we measure?
• Initially, we need to measure “presnece” at various points during the movie
• Where “presence” is defined as the “illusion of non-mediation” i.e. feeling like you are “actually there”
The problem with measuring presence
• Presence is normally measured by giving subjects a questionnaire after the movie ends
• However, this does not permit comparison across scenes in the movie
• We decided to score subjective ratings of presence using a simple line-bisection task
• Each time a sound beeped or a light flashed, subjects bisected the line on a single piece of paper to indicate the extent to which they “felt present” in the movie
What film to use?• Hasson et al (2004) Science 209, 1631-1641• Used film “The Good, the Bad, and the Ugly” (1967,
United Artists)• Measured correlation in scene-dependent activation
across subjects in fMRI study• Established three main types of scene giving reliably
different activation profiles:– Faces (Fusiform gyrus, FFA)– Landscapes/buildings (Colleteral sulcus, PPA)– Hand-related scenes (Post-central sulcus)
• Therefore, measuring presence at these points in the movie should reflect different patterns of cortical activation in viewers.
FFA scenes
FFA scenes
PPA scenes
PPA scenes
Screen size experiments
Thanks to Lee De-Wit for the picture
Experimental details
• Presence is normally measured by giving ss a questionnaire after the movie ends
• However, this does not permit comparison across scenes in the movie
• We decided to score subjective ratings of presence using a simple line-bisection task
• Each time a light flashed, subjects bisected the line on a single piece of paper to indicate the extent to which they “felt present” in the movie
• 40 subjects in a between-groups design
Presence results
Screen size p<0.001
Scene type p<0.001
NS interaction
Presence and memory
The effect of screen size on presence and recognition memory for scenes
Experimental details
• Similar viewing conditions as in Exp 1• 30 subjects in a between-groups design• A beep sounded at key points in film
(first 45 minutes)• Subjects rated presence with the line
bisection task• Then they took part in recognition
memory expt – 28 clips, 2 sec each, half seen, half unseen
Presence results
Screen size p<0.005
Scene type p<0.05
NS interaction
Memory results
Screen size p<0.001
Scene type p<0.001
Size x type interaction p<0.001
Pressence, RT, arousal
The effect of screen size on presence, reaction time, and arousal
Experimental details
• Similar viewing conditions as in Expts 1 & 2
• 30 subjects in between-groups design• Subjects pressed a button in response
to beep – giving RT measure• Pupil diameter was recorded by a
custom-built pupillometer, giving an indication of cognitive load/arousal
Presence results
42
44
46
48
50
52
54
56
small large
Screen size
Rate
d pr
esen
ce
face
land
Screen size NS
Scene type p<0.05
NS interaction
Reaction time results
1.45
1.5
1.55
1.6
1.65
1.7
1.75
1.8
small large
Screen size
Reac
tion
time
(sec
)
face
land
Screen size NS
Scene type p<0.05
NS interaction
Pupil dilation results
0
0.05
0.1
0.15
0.2
0.25
0.3
0.35
small large
Screen size
Pupi
l dila
tion
(mm
)
face
land
Screen size NS
Scene type p<0.001
NS interaction
Preliminary data: colour vs b&w
Presence and edit statistics• We have a useful online measure of presence• Let us return to the “stable world” assumption• A film is not a very stable world – the perceiver sees
edits every few seconds• Clearly, continuity rules and possible “adaptation” to
1/f statistics mask instability• However, we may still expect a film with short cuts to
appear more unstable, and less “natural”, than one with long cuts
• It is therefore interesting to look for effects of mean cut length, and the standard deviation of mean cut length, on presence
Shoot-out scene experiment
• We used shoot-out scenes from obscure Western movies made in early 1960s
• Created a 26 minute showreel and tested presence on two groups of 21 subjects
• In counterbalanced order• In a group setting in a lecture theatre
Presence and mean shot length
Presence and mean shot length: replication with Good, Bad, Ugly
Presence and mean shot length: replication with Good, Bad, Ugly
Presence and mean shot length
Presence and mean shot length
Film clip 1
Presence and mean shot length
Film clip 3
Conclusions• The local presence measure appears to
reflect arousal or depth of processing• Does not easily predict memory performance• Show clear effect of screen size on presence• There is a relationship between presence and
Mean Shot Length• But the causal nature of this relationship is
unclear: does an editor make short edits in an exciting scene, or do short edits cause a scene to be exciting?
• Need further experiments to tease this apart• Also look at colour/bw and stereo/no-stereo
And why is this important?
And why is this important?Most expensive films
Rank Title Year Film costs in millions (est.)
1 Pirates of the Caribbean: At World's End 2007 $300m[3][6][nb 1]
2 Spider-Man 3 2007 $258m (official)[1]
3 Harry Potter and the Half-Blood Prince 2009 $250m[7][8]
4 Avatar 2009 $237m (official)[9][nb 2]
5
The Chronicles of Narnia: Prince Caspian 2008 $225m (official)[10]
Pirates of the Caribbean: Dead Man's Chest 2006 $225m[3][11][nb 1]
7 X-Men: The Last Stand 2006 $210m[12][13]
8 Superman Returns 2006 $209m (official)[14][nb 3]
9 King Kong 2005 $207m[15][16][17]
10
Transformers: Revenge of the Fallen 2009 $200m (official)[18]
2012 2009 $200m (official)[19]
Titanic 1997 $200m[1][20][21]
Spider-Man 2 2004 $200m[22][23]
Quantum of Solace 2008 $200m[24][25]
Terminator Salvation 2009 $200m[26][27]
Toy Story 3 2010 $200m[28][29]
So…• Measures of perception and cognition during
film viewing reveal effects of viewing conditions, edit regime, and use of colour
• Cognitive scientists can learn much about the brain from using these remarkable stimuli– Broader range of techniques (EEG, fMRI, remote
observation of audience)– Plus, they can begin to fill out the “Impact
Statement” on a grant proposal form…• Film producers and theatres can learn about
their audiences’ response to movies• Thus being able to estimate the benefits from
(say) installation of a 3D system
Speculation• We may have expected presence to reduce
when a film is more “unnatural”• I.e. when its shots are short/variable• The opposite seems to be true• It may be that an instability, or “error” signal
can be perceived as pleasurable if the perceiver appears to be “safe”
• Neuro-aesthetics literature shows activations of different parts of brain by pleasurable and aversive stimuli
• But films (and sunsets) may be examples of stimuli which are both pleasurable andaversive
