The impact of Cultural Context on the Perception of Sound and Musical Language (and its relevance for the development of Computational Models of Audition)

The impact of Cultural Context on the Perception of Sound and Musical Language

(and its relevance for the development of Computational Models of Audition)

The Lisbon Consortium

Luís Gustavo Martins [email protected]

Lisbon, June 26th 2013

CATÓLICA'PORTO+ARTES+

This work is licensed under a Creative Commons Attribution-ShareAlike 3.0 Unported License.


Outline •  Human listening comprises many layers of information analysis and processing.

•  From the transduction of low-level auditory stimuli in the ears… •  … to their organization in the brain,

•  higher-level factors take an important role in the auditory process: •  Memory •  Attention •  Experience •  Culture •  Context •  …

•  As a result, listening in general is influenced by a large number of variables interacting, cooperating and competing with each other in a rather complex network.

•  Recent research in the field of computational sound and music analysis is now more aware of the impact of the cultural background in the perception of sound and music.


Can Culture and Context affect our Perception? Sensation

•  neurological process by which we become aware of our environment (Jandt, 2012).

•  You are not directly aware of what is in the physical world… •  … but, rather, of your own internal sensations.

•  Is variation in human sensation attributable to culture? Source: h"p://www.sagepub.com/upm1data/45975_Chapter_3.pdf=


Can Culture and Context affect our Perception?


Can Culture and Context affect our Perception? “Kids are encouraged to have fun outdoors, playing in the nature.”


Can Culture and Context affect our Perception? “Kids are encouraged to have fun outdoors, playing in the nature.”


Can Culture and Context affect our Perception? “Unattended kids may become extremely violent when playing outdoors!”


Can Culture and Context affect our Perception? “Unattended kids may become extremely violent when playing outdoors!”



Dogs in the Western Culture

Source: http://www.sagepub.com/upm-data/45975_Chapter_3.pdf

PART 1 CULTURE AS CONTEXT FOR COMMUNICATION66

InterpretationThe third step in the perception process is interpretation. This refers to attaching meaning to sense data and is synonymous with decoding. The same situation can be interpreted quite differently by diverse people. A police officer arriving at a crime scene can be experienced by the victim as calming and relief giving but by the criminal as fearsome and threatening.

Here, too, the effect of culture is great. As you encounter people of your own culture, you constantly make judgments as to age, social status, educational background, and the like. The cues you use to make these decisions are so subtle that it’s often difficult to explain how and why you reach a particular conclusion. Do people in the United States, for example, perceive tall men as more credible? Perhaps.

Applying these same cues to someone from another culture may not work. People in the United States, for example, frequently err in guessing the age of Japanese individuals, such as judging a Japanese college student in her mid-20s to be only 14 or 15.

Dogs as Pets or as Food

The meanings you attach to your perceptions are greatly determined by your cultural back-ground. Think of how speakers of English categorize life. Most probably use the categories of human life and animal life. Now think of how you typically categorize animal life—probably into wild animals and domesticated animals. Now think of how you typically categorize domesticated animal life—probably into animals used for food, animals used for sport and recreation, and pets. Look at the picture of the puppy and capture your feelings.

Most of us see this puppy in the category of pet, for which we have learned to relate warm, loving feelings. Puppies are cute, cuddly, warm, loving creatures. Now look at the next picture

of a man holding up a dog, read the caption, and capture your feelings. Most of us who love dogs find this picture uncomfortable and disgusting. How can people eat dogs? They are pets, not food! It all depends on where you categorize them. Dogs are pets in some cultures and food in others. In the Arab world, dogs are acceptable as watchdogs and as hunting dogs but are not kept in the home as pets because they are seen as unclean and a low form of life. To call someone a dog is an insult among Arabs. People in most cultures have strong ideas about which foods are acceptable for human consumption and which are not. People in some countries think the custom in the United States of eating corn on the cob is disgusting because that food is fit only for pigs. Some Ukrainians like to eat salo, raw pig fat with black bread and vodka, which might cause nausea in some, as would knowing that horse meat from California is served in restaurants in Belgium, France, and Japan.

Many consider dogs as pets. (The author’s first dog, Smokey.)



Dogs in the Eastern Culture

Source: http://www.sagepub.com/upm-data/45975_Chapter_3.pdf

CHAPTER 3 Culture’s Influence on Perception 67

Your reaction of disgust to the picture is a culturally learned interpretation—and that inter-pretation can be quite strong. In 1989, California made it a misdemeanor for any person to sell, buy, or accept any animal traditionally kept as a pet with the intent of killing the animal for food. More recently, animal rights groups have protested the sale of live animals, such as turtles, frogs, lobsters, crabs, fish, and chicken, for food at Asian-American markets. Asian tradition is that fresh meat is tastier and more healthful, that the best meat “enters your house still breathing.” Animal rights activists contend that the animals are treated inhumanely in the shops and are killed in ways that cause them unnecessary pain. Asian-American groups argue that eating dogs and cats is an extreme rarity among Southeast Asian immigrants and call the law and the animal rights activists racist.

In some cultures, parts of some animals are categorized as medicine. In other cultures, certain animals are considered sacred and certainly would not be eaten. The Hindu elephant-headed God Ganesh is accompanied by a rat whenever he travels. Rats, like cows, are deified in India. No Hindu worship is complete without an offering to Ganesh and his companion, the rat. Rats are fed and rarely killed in India.

Weather Vane as Christian Cross

The examples so far have been of practices that could offend some English speakers. Let’s turn that around with an example of what speakers of English do that could be offensive to others. Johnston Pump Company, a U.S. company now based in Brookshire, Texas, has been doing business with Saudi Arabia for more than 70 years. By the 1930s, Johnston Pump was well

Can you explain your feelings about this photograph? As China’s economy boomed and affluence spread, attitudes toward dogs changed. Traditional Chinese may have eaten dog meat because it was thought to improve blood circulation. Urban Chinese today are more likely to have dogs as pampered companions.


Can Culture and Context affect our Perception? What do you (can you!) “see” here?


Can Culture and Context affect our Perception? Humans (independent of their cultural context) develop, from early ages, a “model” for “the human face”... So maybe, you see a face...



Or, maybe you know what a saxophone looks like, and how it is (usually) played...


Can Culture and Context affect our Perception? What if you never left your native tribe, deep in the African savana? Can you still perceive something from the figure?


Can Culture and Context affect our Perception? Or maybe you are a kid with strong “cultural” references from cartoons? (and a large imagination)



•  Nisbett (2003) has demonstrated that:

•  humans sense and perceive the world in ways unique to their environments •  by contrasting Eastern and Western cultures.

•  Perception and thought are not independent of the cultural environment

•  therefore, our brains are both shaped by the external world and shape our perception of the external world.

•  Sensation is the neurological process of becoming aware of our environment and is affected by our cultures (Jandt, 2012).

Source: h"p://www.sagepub.com/upm1data/45975_Chapter_3.pdf=


Sensing and Perceiving the World around us

A World of Sounds


A World of Sounds Sound production and propagation


A World of Sounds Sound propagation – waves of energy


A World of Sounds Sound sensors: the Human Ear THE HUMAN AUDITORY SYSTEM

Figure 1: A sketch of the peripheral region of the human auditory system (adaptedfrom [Beranek, 1990]).

a pre-processing of the acoustical signal, the most notable aspect being frequency analysis.

The structure of the peripheral region is represented in Figure 1. Broadly, the periphery

can be divided into three areas: the outer, the middle and the inner ear. The outer ear

(pinna) collects sound energy from the surrounding environment and transmits it through

the ear canal (meatus) to the eardrum (tympanic membrane), giving rise to an oscillatory

motion. The pinnae play a role in spatial hearing, by imposing spectral characteristics on

sound that depend on its direction of incidence with respect to the head. In the middle

hear, the vibrations in the eardrum are transmitted to the fluid-filled cochlea by means

of three tiny bones (the ossicles: the malleus, the incus and the stapes). They form a

mechanical lever system whose main purpose is to match the impedance of air to that of

the cochlear fluids and to protect the ear against aggressive sound intensities by modifying

the transmission gain. The middle ear is filled with air whose pressure is equalized with

that of the outside environment through the Eustachian tube. The inner ear corresponds

to the cochlea, the organ of hearing. It is a tube shaped like a snail shell, filled with fluid,

that is divided along its length by two membranes: Reissner’s membrane and the basilar

membrane. The basilar membrane varies in mass and sti↵ness along its length so that

di↵erent regions of the membrane vibrate at di↵erent resonant frequencies. As a result,

the basilar membrane exhibits a complex pattern of motion in response to sound-induced

13


A world of Sounds Sound sensors: the Human Ear


A World of Sounds Beyond sensing: higher order Auditory processes and their interactions

MUSIC LISTENING

ABSTRACT

KNOWLEDGE

STRUCTURES

EVENT

STRUCTURE

PROCESSING

EXTRACTION

OF

ATTRIBUTES

AUDITORY

GROUPING

PROCESSES

MENTAL

REPRESENTATION OF

SOUND

ENVIRONMENT

TR

AN

SD

UC

TIO

N

TR

AN

SD

UC

TIO

N

ATTENTIONAL

PROCESSES

Figure 2: The main types of auditory processing and their interactions (adaptedfrom [McAdams and Bigand, 1993]).

possible to extract perceptual attributes which provide a representation of each element in

the auditory system.

These attributes can now be interpreted with respect to evoked abstracted knowledge

structures that allow the recognition and identification of events and sequences of events,

as well as the assignment of meaning and significance. All this is performed at the light

of the local stimulus context and the past experience of the listener (see the discussion of

schema-based grouping in section 2.3.1).

These perceived relations among sound events can influence the perception of sub-

sequent events (and even revise past interpretations – e.g. the McAdams-Reynolds

oboe example [McAdams, 1984] or the “old-plus-new” organization principle described

in [Bregman, 1990, pp.261]). This event structure processing influences the establishment

of larger-scale structural relations.

The resulting elements from the event structure processing give rise to the progressive

creation of a mental representation of the sound environment involving the listener. Given

the transitory nature of the acoustic stimulus, information about acoustic events must

be somehow accumulated through time. This accumulated mental representation takes

a crucial part on the ability to dynamically focus attention to new incoming acoustic

16


A World of Sounds Auditory Scene Analysis (ASA) - The lake analogy…

Source: http://www.cns.nyu.edu/~david/courses/perception/lecturenotes/localization/localization.html


A World of Sounds Auditory Scene Analysis – The Cocktail Party Effect

Source: http://online.wsj.com/article/SB10001424052702303459004577361850069498164.html


A World of Sounds Auditory Scene Analysis - Polyphonic, multitimbral, complex musical compositions

•  Listeners can still identify:

•  Melodic lines •  Instruments (or groups of instruments playing) •  Harmony •  Beat •  Rhythm •  …


Perception Principles

The Gestalt Theory of Perception



The Gestalt Theory of Perception



Auditory Scene Analysis (ASA) - the Perceptual Organization of Sound

http://webpages.mcgill.ca/staff/Group2/abregm1/web/


Perception Principles Segregation of a melody from interfering tones

•  Bottom-up VS Top-Down approaches

Source: http://webpages.mcgill.ca/staff/Group2/abregm1/web/downloadstoc.htm

“At the intermediate separations, familiarity with the melody can sometimes allow listeners to hear it out, but there is a still a tendency to hear runs of tones that group melodic and distractor notes.”


Music Perception

How is Similarity defined, for Music? (And what impact has cultural context and background on its definition?)

Query:

Similar Music?


Music Perception


“- This all sounds like Fado to my hears…” “- Not to me… I’m a Fado singer…I can tell a difference!”

=


Music Perception


Saudades de Coimbra (Fado) - Verdes Anos

Amália Rodrigues - Fado Português

João Braga - Fado de Lisboa

“- This all sounds like Fado to my hears…” “- Not to me… I’m a Fado singer…I can tell a difference!”

=


Music Perception How is Similarity defined, for Music? (And what impact has cultural context and background on its definition?)

“- This all sounds like Techno to my hears…” “ – No to me… I’m a DJ… I can tell a difference!”



“- This all sounds like Techno to my hears…” “ – No to me… I’m a DJ… I can tell a difference!”

The Big Fake Traffic Signs (ELECTRO)

Je T'aime (Digital Dog mix) Armand Van Helden (DEEP HOUSE)

Exorcise (A1) Umek (TRANCE)

Memories David Guetta (COMMERCIAL HOUSE)


Music Perception “Music is different things and does different things in different cultures; the bundles of elements and functions which are music for any given culture may overlap minimally with those of another culture, even for those cultures where "music" constitutes a discrete and identifiable category of human activity in its own right.” (Cross, 2001)

http://techno.org/electronic-music-guide/

-  Expert listeners in a genre, or in a given culture can perceive subtle variances in a music genre, and identify sub-genres, artist styles, etc.

-  They make part of a “culture” around that musical genre / style

-  “Music", like speech, is a product of both our biologies and our social interactions (Cross, 2001)



•  It is an ill–posed question…

•  It is multidimensional, may use different and simultaneous criteria, such as: •  Pitch •  Harmony •  Timbre •  Rhythm •  Genre •  Texture •  …

•  It may depend on the context of the query •  Difficult to infer…

•  It will certainly be culturally biased •  Difficult to match expectations


Effects of Culture on Music Perception •  Perception of Pitch (Wong et al., 2012)

•  Studies on Chinese and Canadian population with Amusia •  Amusia: a neurogenetic disorder affecting the music (pitch and rhythm) processing •  Contonese is a tone language, Canadian French and English are not tone languages

•  Cantonese speakers as a group tend to show enhanced pitch perception ability compared to speakers of Canadian French and English.

•  This enhanced ability occurs in the absence of differences in rhythmic perception and persists even after relevant factors such as musical background and age were controlled.

•  These findings not only provide critical evidence for a double association of music and

speech, but also argue for the reconceptualization of communicative disorders within a cultural framework.

•  Along with recent studies documenting cultural differences in visual perception, our auditory evidence challenges the common assumption of universality of basic mental processes and speaks to the domain generality of culture-to-perception influences.

Source: http://www.plosone.org/article/info:doi/10.1371/journal.pone.0033424


Effects of Culture on Music Perception •  Memory and Musical Expectation for Tones (Curtis and Bharucha, 2009)

•  Explored how musical culture shapes one’s listening experience.

•  Western participants heard a series of tones drawn from either: •  the Western major mode (culturally familiar) •  or the Indian thaat Bhairav (culturally unfamiliar)

•  and then heard a test tone.

•  They made a speeded judgment about whether the test tone was present in the prior series of tones. •  False alarm rates were higher for test tones congruent with the major mode than for

test tones congruent with Bhairav. •  In contrast, false alarm rates were lower for test tones incongruent with the major mode

than for test tones incongruent with Bhairav.

•  These findings suggest that one’s internalized cultural knowledge may drive musical expectancies when listening to music of an unfamiliar modal system.

Source: http://openscholar.purchase.edu/meagan_curtis/files/curtisbharucha2009musicperception.pdf


Effects of Culture on Music Perception •  A cross cultural study on Music Mood Perception (Hu and Lee, 2012)

•  People's cultural background is often assumed to be an important factor in music mood perception.

•  This paper reports on a study comparing mood judgments on a set of 30 songs by American and Chinese people.

•  Results show that mood judgments do indeed differ between American and Chinese respondents.

•  Furthermore, respondents’ mood judgments tended to agree more with other respondents from the same culture than those from the other group.

Source: http://ismir2012.ismir.net/event/papers/535-ismir-2012.pdf


Computational Approaches to Sound and Music Analysis

Typical Pipeline

Hearing Representation

Analysis Understanding

Acting Interaction

Signal Processing Feature Extraction

Machine Learning Analysis and Retrieval

Human Computer Interaction



Different perspectives involved

Source: h"p://mires.eecs.qmul.ac.uk/wiki/index.php/Roadmap=

TECHNOLOGICAL PERSPECTIVE

2.1 Musically

relevant data

2.2 Music

representationsModels

2.3 Data processing methdologies

2.4 Knowledge-driven

methdologies

2.5 Estimation of

elements related to musical concepts

Estimated musical

concepts

Other estimatedconcepts

2.6Evaluation

methodologies

SOCIO-CULTURAL PERSPECTIVE

4.1Social aspects

4.2Multiculturality

USER PERSPECTIVE

3.1User behaviour

3.2User interaction

EXPLOITATION PERSPECTIVE

5.1 Music distribution

applications

5.2Creative tools

5.3Other exploitation areas



Technological Perspective:



Challenge: how to include the Social-Cultural Perspective?


TECHNOLOGICAL PERSPECTIVE

2.1 Musically

relevant data

2.2 Music

representationsModels

2.3 Data processing methdologies

2.4 Knowledge-driven

methdologies

2.5 Estimation of

elements related to musical concepts

Estimated musical

concepts

Other estimatedconcepts

2.6Evaluation

methodologies

SOCIO-CULTURAL PERSPECTIVE

4.1Social aspects

4.2Multiculturality

USER PERSPECTIVE

3.1User behaviour

3.2User interaction

EXPLOITATION PERSPECTIVE

5.1 Music distribution

applications

5.2Creative tools

5.3Other exploitation areas



Challenge: how to include the Social-Cultural Perspective? •  Music Information Research (MIR) involves the understanding and modeling of

music-related data in its full contextual complexity.

•  Music is a communication phenomenon that involves people and communities immersed in specific social and cultural context.

•  MIR aims at processing musical data that captures the social and cultural context and at developing data processing methodologies with which to model the whole musical phenomenon.




Challenge: how to include the Social-Cultural Perspective? •  Music Computing research (IT in general) does not respond to the world's

multi-cultural reality.

•  Data models, cognition models, user models, interaction models, ontologies, ... are culturally biased.

•  Music information is more than CDs and metadata.

h"p://compmusic.upf.edu=;=h"p://mires.eecs.qmul.ac.uk/wiki/index.php/Roadmap= ==



Challenge: how to include the Social-Cultural Perspective? •  Promote a multicultural approach to Music Computing research.

•  Advance in the formalization and description of music, making it more accessible to computational approaches.

•  Reduce the gap between audio signal descriptions and semantically/culturally meaningful music concepts.

•  Develop information modelling techniques applicable to different music repertories.

•  Develop computational models to represent culture specific music contexts.

h"p://compmusic.upf.edu=;=h"p://mires.eecs.qmul.ac.uk/wiki/index.php/Roadmap= ==


compmusic

Music information modeling

Sound and Music Computing

Music Information Processing

Computational Musicology

Ontologies

Cognition models

Interaction models

Data models

User models


Challenge: how to include the Social-Cultural Perspective?

h"p://compmusic.upf.edu==


References •  Richard Nisbett, “The Geography of Thought: How Asians and Westerners Think Differently ... and Why”, New York: Free

Press, 2003, ISBN 978-0-7432-1646-3 •  Fred E. Jandt, “An Introduction to Intercultural Communication”, Chapter 3 - “Culture’s Influence on Perception”, SAGE

Publications, 2012, http://www.sagepub.com/upm-data/45975_Chapter_3.pdf

•  Ian Cross, “Music, cognition, culture and evolution”, Annals of the New York Academy of Sciences Vol 930, 2001, pp 28-42.

•  Wong PCM, Ciocca V, Chan AHD, Ha LYY, Tan L-H, et al. (2012) “Effects of Culture on Musical Pitch Perception.”, PLoS ONE 7(4): e33424. doi:10.1371/journal.pone.0033424

•  Meagan E. Curtis, Jamshed J. Bharucha, “Memory and Musical Expectation for Tones in Cultural Context”, Music Perception VOLUME 26, ISSUE 4, PP. 365–375, ISSN 0730-7829, ELECTRONIC ISSN 1533-8312

•  Xiao Hu, Jin Ha Lee, “A CROSS-CULTURAL STUDY OF MUSIC MOOD PERCEPTION BETWEEN AMERICAN AND CHINESE LISTENERS”, 13th International Conference on Music Information Retrieval (ISMIR), 2012, Porto, Portugal

•  Bregman, A. (1990). Auditory Scene Analysis -- The Perceptual Organization of Sound.

•  Bregman, A., & Ahad, P. (1996). Demonstrations of Auditory Scene Analysis: The Perceptual Organization of Sound.

•  Wang, W. (Ed.). (2010). Machine Audition - Principles, Algorithms and Systems. IGI Global.


References •  Xavier Serra, Michela Magas, Emmanouil Benetos, Magdalena Chudy, Simon Dixon, Arthur Flexer, Emilia Gómez, Fabien

Gouyon, Perfecto Herrera, Sergi Jorda, Oscar Paytuvi, Geoffroy Peeters, Jan Schlüter, Hugues Vinet, Gerhard Widmer, “Roadmap for Music Information ReSearch”, Geoffroy Peeters (editor), 2013, Creative Commons BY-NC-ND 3.0 license, ISBN: 978-2-9540351-1-6 - http://mires.eecs.qmul.ac.uk

•  Wang, D., & Brown, G. (2006). Computational Auditory Scene Analysis: Principles, Algorithms and Applications.


THANK YOU!

Luís Gustavo Martins

[email protected] http://lgustavomartins.wordpress.com/about/

http://artes.ucp.pt

http://artes.ucp.pt/citar http://artes.ucp.pt/ccd

Education

The impact of Cultural Context on the Perception of Sound and Musical Language (and its relevance for the development of Computational Models of Audition)