RENZO ALVES DANTAS TORRECUSO

PROCESSING OF SYNTAX IN MUSIC: AN EEG STUDY

Master thesis presented to the Post Graduation

Program in Neuroscience of the Federal University

of Rio Grande do Norte as a requirement to

obtain the title of Master in Neuroscience.

SUPERVISOR: PROF. DR. DIEGO ANDRÉS LAPLAGNE

CO-SUPERVISOR: PROF. DR. SIDARTA TOLLENDAL GOMES RIBEIRO

NATAL

2

Universidade Federal do Rio Grande do Norte - UFRN Sistema de Bibliotecas - SISBI

Catalogação de Publicação na Fonte. UFRN - Biblioteca Setorial Árvore do Conhecimento - Instituto d Cérebro - ICE

Torrecuso, Renzo Alves Dantas. Processing of syntax in music: an EEG study / Renzo Alves Dantas Torrecuso. - Natal, 2017. 46f.: il. Universidade Federal do Rio Grande do Norte. Instituto do Cérebro. Pós graduação em Neurociências. Orientador: Prof. Dr Diego Andrés Laplagne. Coorientador: Prof. Dr Sidarta Tollendal Ribeiro. 1. Sintaxe. 2. Música. 3. Eletroencefalografia. I. Laplagne, Prof. Dr Diego Andrés. II. Ribeiro, Prof. Dr Sidarta Tollendal. III. Título. RN/UF/BSICe CDU 612.8

3

2017

RENZO ALVES DANTAS TORRECUSO

PROCESSING OF SYNTAX IN MUSIC: AN EEG STUDY

Master thesis presented to the Post Graduation

Program in Neuroscience of the Federal University of Rio Grande do Norte as a requirement to

obtain the title of Master in Neuroscience.

Approval date: ___/ ___/______. EXAMINATION COMMITTEE

__________________________________

Prof. Dr. Diego Andrés Laplagne – President (UFRN)

________________________________________________________

Prof. Dr. Emilie Katarina Svahn Leão – Internal examiner (UFRN)

________________________________________________________

Prof. Dr. Peter Maurice Erna Claessens – External examiner (UFABC)

4

ACKNOWLEDGMENTS

Brain Institute, UFRN.

Prof Dr Diego Laplagne, Prof Dr Sidarta Ribeiro, Prof Dr Kia Leão, Prof Dr Peter Claessens

(UFABC), Prof Dr Kerstin Schmidt , Prof Dr Draulio Araujo, Prof Dr Tarciso Velho, Prof Dr

Marcos Costa, Prof Dr Richardson Leão, MSc Daniel Almeida , MSc Annie Costa, Dr Rodrigo

Pavão, Dr Nivaldo Vasconcelos, MSc Bryan Costa, Dr Vitor Santos, Dr Ernesto Soares, Rafael

Pedrosa, MSc Dardo Ferreiro, Dr Ignacio Sanches Gendriz, Dr Sergio Rolim, Dr Raphael Bender

Dr João Bacelos, João Patriota, Ana Raquel Torres, MSc Thawann Borges, MSc Barbara Boerner

MSc Diego Coelho.

Bergen University

Prof Dr Stefan Koelsch, Dr Sebastian Jentschke.

Freie University

Dr Moritz Lehne, Dr Stavros Skouras, Dr Martin Rohrmeier, Dr Joana Piperi, MSc Kamila

Borowiak.

EMUFRN

Prof MSc Juliano Ferreira, Prof Dr Ticiano Maciel Damore, Prof Dr Marcus André.

EC&T- UFRN

Prof MSc Diego Pena, Prof MSc Leonardo Mafra.

5

“Es Denkt In Mir”, Nietzsche.

6

ABSTRACT

In order to make sense out of a sequence of sounds in music, our brain must meaningfully fit

and recombine acoustic events into a hierarchic online stream. Although these information units are

auditively delivered in sequences with local connections (one after the other), it is assumed that

long-term dependencies are established counting on memory traces to sustain recursiveness in time.

Despite theoretical and empirical consensus, there is yet no clear physiological evidence of the

temporal dimension of syntactic relations in music.

We investigated whether there is quantifiable neural activity suggesting the existence of a

mental representation for fundamental music syntax rules like tonic-dominant-tonic chords. For

such, we compared brain electric activity in 24 subjects (12 musicians, 12 non-musicians) aroused

by original and harmonically modified versions of J.S. Bach chorales, using electroencephalography

(EEG). Chorales were built by two phrases: initiated by a tonic and arriving in a dominant chord

two bars away (first phrase), and concluded in a tonic chord three bars after the dominant (second

phrase). Modified versions were created either by elevating or lowering the first, therefore keeping

the second phrase intact. We compared the brain electric response for the last chord in both

versions.

Our data produced event related potentials (ERP) which suggest that the subjects’ brain

processed modified versions differently than originals. We observed an amplitude difference in the

negativities peaking around 210 ms after the last chord onset. This finding replicates previous

studies on harmonic disruptions processed in the same latency, but differs regarding brain source

contributions: our data revealed a more posterior than anterior effect. Considering that last chords

were the same acoustic event in both versions, we hypothesize that the amplitude differences

plausibly indicate that the syntactic expectancy (long-term dependency between the tonic, dominant

and tonic chord) may relate to brain mechanisms of processing non local connections, establishing a

hierarchical storage of acoustic events in memory.

We interpret that our parietal findings parallels with literature on mathematical sequence

processing which suggest posterior brain regions to be engaged on complex calculation requiring

the storage of numbers magnitude in memory for further computation.

Our study overcame an obstacle not to date surpassed: observe and separate brain electric

activity raised by long-term syntactic disruption in music from the overlap of local mismatch

detection.

7

INDEX

1. Introduction……………………………………………………………………………….....7

1.2. EEG wave Polarity…………………………………………………………….........10

1.3. Auditory Event Related Potential (ERP) Components………………….…….......11

1.4. N100………………………………………………….....…………...…………........12

1.5. MMN..........................................................................................................................12

1.6. LAN and ELAN to ERAN..........................................................................................16

2. Objectives...............................................................................................................................22

3. Methods..................................................................................................................................23

3.1. Stimuli..........................................................................................................................23

3.2. Subjects........................................................................................................................24

3.3. Procedure.....................................................................................................................25

3.4. EEG Recordings..........................................................................................................25

4. Results.....................................................................................................................................26

4.1. ERAN (Early Right Anterior Negativity)....................................................................26

4.2. Statistical analysis.........................................................................................................26

4.2. Musicians and Non Musicians.....................................................................................30

4.3. Brain regions...............................................................................................................31

5. Discussion...............................................................................................................................32

6. Bibliography...........................................................................................................................40

LIST OF ABREVIATIONS

ERAN: Early Right Anterior Negativity

MMN: Mismatch Negativity

LAN: Left Anterior Negativity

ELAN: Early Left Anterior Negativity

ERP: Event Related Potential

SLTDM: Syntactic Long-Term Dependencies in Music

8

1. Introduction.

Together with language and mathematics, music reveals our ability as humans to recognize

discrete elements, meaningfully fitting and recombining them into a hierarchical sequence. The

group of principles that guides this sequence is called syntax (Anderson 1967; Patel et al. 1998;

Winograd 1972) and recursion is its core property (Hauser et al., 2002; Lerdahl and Jackendoff,

1985). In the same way that the phrase The man threw his handkerchief is built by fundamental

hierarchic elements (subject, verb and object) which can be recursively stretched out to longer and

more meaningful structures (e.g. The man [ who was the first suspect of the robbery][for which the

charges have been removed] threw [with a blooded hand] his handkerchief) (figure 1, middle), a

western musical phrase can also expand its fundamental hierarchic elements (1st, 4th,5th and 1st

chords) to a chorale or a whole symphony movement (Jackendoff, 2009; Rohrmeier, 2011)(figure 1,

top). A citation from the XX century composer Arnold Schoenberg, stated in Theory of Harmony

(Schoenberg 1978), illustrates this hierarchy conception: “we can consider the chorale, as well as

every larger composition, a more or less big and elaborate cadence” (pg. 290). Cadence in its

elementary form is a particular chord sequence that represents a phrase end or suspension

(Bharucha and Krumhansl 1983; Tillmann and Bigand 2004). In a similar way, a mathematical

equation is organized in hierarchical operations necessary to find the correct solution:

multiplications and divisions precede sums and subtractions, and common factors can be identified

resuming calculation steps (figure 1, bottom).

Even though theorists have developed in the past fifty years musical grammars sustaining

that harmony is governed by recursiveness or long-term dependencies (Hofstadter 2000; Lerdahl

and Jackendoff 1985; Rohrmeier 2011; Schoenberg 1978; Steedman 1984) there is still no

consensus. Authors from music analysis (Clarke 1987; Konečni 1984; Marvin and Brinkman 1999),

musical psychology (McAdams and Bigand 1993) supported by some sparse empirical evidences in

experimental psychology (Bigand et al. 2003; Tillmann and Bigand 2004), argue that the listener

does not perceive music taking account long-term dependencies, but on the contrary, that the local

models are the basis of musical discourse and perception (Tillmann and Bigand 2004)

As most above mentioned experimental psychology works are based on subjects' ratings on

the perception of music harmonic irregularities, and there are still few studies approaching this issue

based on brain measurements analysis on naturalistic stimuli (real music), we wish to shed light on

the topic by an experiment design which combines electroencephalography (EEG) measurements

with subjects ratings using western music short models, J.S. Bach chorales.

9

Figure 1. Music, language and math as sequences based on non-local dependencies. Top: Music

score depicting, in the first two bars( vertical lines), the long-term dependencies between the G Major chord ( 1st

chord of the tonality´s scale, hence I ) and the D Major chord( 5th chord of the tonality´s scale, hence V) which

establishes an expressive tension. In the last three bars, the long-term dependency between the D Major chord,

5th (dominant) and the G Major chord, 1st (tonic), which resolves the expressive tension. Middle: Phrase

exemplifying long-term dependence in language between subject, verb and object. Note that, as in the music

score, sentences can be recursively inserted between subject and verb, and also between verb and object. Bottom:

in blue “1/2” is separated for later calculation while multiplying 3x5 arrives to a two steps fraction sum.

Intermediary steps are calculated until 1/2 is finally computed as a simple sum producing the final result.

We also intend to investigate if musical expertise may alter brain regions involved in

processing the perception of music long-term dependencies. In order to do so, our 24 subjects were

separated in musicians and non-musicians. We designed an experiment in which 64 EEG electrodes

were positioned in the subjects' scalps while they listened to a pseudo random sequence of J.S. Bach

chorales. The chorales had their harmonic structure modified and subjects were made unaware of

10

the harmonic modifications by engaging in a task relevant paradigm during a presentation of a silent

movie.

As we also intend to compare neural responses to perception ratings of long-term

dependencies, after the scanning session ended subjects were presented again to a pseudo random

sequence of some of the chorales listened previously but this time subjects received sheets and were

asked to rate the stimuli according to its conclusiveness.

This work is based on the past 50 years of research using EEG to identify auditory event

related potentials (ERP) associated to the processing of irregularities in a sound sequence, oddball,

mismatch negativity (MMN), of grammatical irregularities with the so called left negativity (LAN)

and the early left anterior negativity (ELAN), and musical syntactic disruptions with the early right

anterior negativity (ERAN)(Loui et al. 2005; Maess et al. 2001a; Poulin-Charronnat, Bigand, and

Koelsch 2006; Sammler, Koelsch, and Friederici 2011). It has also been observed that subjects'

perception of harmonic integration can be verified by a late negativity around 500 ms (N5)(Stefan

Koelsch 2012; Steinbeis and Koelsch 2008). Previous studies have mainly focused on local

harmonic dependencies, e.g. the succession of one chord to another. Here, we propose to investigate

if non local (long-term) syntactic musical disruptions (e.g. the non fulfillment of the relation

Dominant to last Tonic, separated by several chords) can also be detected by ERP components.

One may ask what is the relevance of proving that our brain processes long-term

dependencies in music without focused attention. First, if it does, this provides evidence that our

brain indeed does perceive music as a code based on recursiveness (an automatic pre attentional

hierarchic organization of acoustic features: in other words, we may contemplate the assumption

that before one consciously focus attention on a piece of music our brain organizes and attributes

different weights to the incoming acoustic features). This also contributes to theoretical and

experimental studies proposing that our brain keeps track of concomitant streams of information

governed by hierarchical rules (Dehaene 2015; Jackendoff 2009; Sussman, Ritter, and Vaughan

1999; Tillmann and Bigand 2004). Second, considering that recursiveness is a core aspect of

language and math, our findings may also contribute to studies which point to a common neural

architecture in processing of sequences of information (Maess et al. 2001b; Poulin-Charronnat,

Bigand, and Koelsch 2006; Sammler, Koelsch, and Friederici 2011; Dehaene 2015). This may also

be taken in consideration by studies in evolutionary theories that consider the close emergence of

music and language in human history (Kivy 1959). Third, as will be discussed in next section, most

event related potentials (ERP) studies using EEG have found evidences about pre-attentive brain

mechanisms. By using this technique to investigate the pre-attentional neural traces aroused by a

complex stimuli as polyphonic music, the present research will provide considerably precise time

information about when does our brain engage in cognitive computations without our awareness.

11

1.2 EEG waves' polarity.

The neuron is a brain cell characterized by a morphological regularity (comprising soma,

axons and dendrites) and by its electrical signaling property. The action potential is a well-

documented voltage variation phenomenon responsible for propagating neuronal information.

Nevertheless, the intricate mesh of brain tissue, with thousands of neuronal projections, and the

depth of a neuron's location prevents the action potential of a neuron to be detected by a scalp

electrode.

Figure 2. Location of afferences influence wave polarity (Kandel 2014). Schematic view of

afferents' input on apical dendrites of pyramidal cells (in layers 2 and 3) producing a current sink of cations

which contributes to the detection of a negative deflection by electrodes in the scalp (right). Conversely, similar

neurons with afferents in deeper layers, near the soma, produce a local current source and therefore may

contribute to a positive deflection measured in scalp electrodes (left). .

Therefore, superficial electrodes from EEG machines can only measure the summation of electrical

activity originated by assemblies with thousands (or hundreds of thousands) of neurons. Most

cortical neurons are pyramidal and are positioned in parallel to each other. This geometric disposal

together with the location of synaptic afferents in the apical dendrite contributes to the polarity that

a scalp electron may register. Kandel (Kandel, n.d.) points that excitatory thalamic afferents if near

the current source, around layer 5, will give rise to a positive deflection, while excitatory potentials

12

originated by axons coming from contralateral cortex and reaching the apical dendrite on layer 2

will give rise to a negative deflection because the action potentials will produce a current sink and

therefore generate a negative electric field (figure 2). Conversely, inhibitory potentials produce

negativities in deeper layers and positive deflection in superficial layers. Hereon, we will refer to

negativities according to this rationale.

1.3 Auditory Event Related Potential (ERP) Components

ERP components (e.g. N1, MMN, LAN, ELAN, ERAN) are time labels to multiple brain

processes that occur at given latencies. It is therefore misleading to attribute brain processes to the

emergence of a particular ERP component: a N1 is not necessarily produced by sensorial stimuli, a

MMN wave is not an undeniable evidence of the brain perception of a sound object, nor its absence

excludes the possible occurrence of the processes attributed to this component (e.g. sound object

recognition, comparison to environment, auditory stream discrimination, among others)(Schröger

2007). We stress that ERPs are time symptoms of given brain mechanisms but ERP\s may not be

alone causally linked to certain brain mechanisms, as diagnosis. Being that so, when one reads, for

instance, “N1 reveals frequency specificity”, it means that, according to the experiment design used,

it is within the 100ms latency after stimulus onset that the brain processes, among other stimuli

properties, frequency specification.

The earliest well known works that used EEG to relate a subjects' specific task to his

electrical brain activity took place at the Psychology and Physiology departments of Carlifornia´s

University in Los Angeles, during the 1960´s. Donchin, Lindsley, Haider and colleagues focused on

to what extent cortical activity could be time-locked to events in a reaction time sequence

and how attention could modulate these potentials (Donchin and Lindsley 1966; Haider, Spong, and

Lindsley 1964; Lindsley 1952). The studies used a similar experiment design: three subjects had

brain activity measured with four silver cup electrodes (Oz, T3, and Cz from the international "10-

20" system) of an electroencephalograph Grass model 6, on a frequency modulated tape recorder.

Scanning was conducted while subjects performed a visual task of identifying deviant luminosity

flashes of light from a sequence of standard flashes presented with regular inter stimulus

interval(ISI). Their preliminary results isolated a negative wave peaking around 160 to 200

milliseconds with an average amplitude of 5 micro volts which was time-locked to the deviant

stimuli, a light flash with diminished or increased luminosity compared to a sequence of standard

flashes. This component was obtained by averaging brain activity throughout a task that would be

repeated for around 100 minutes.

13

The technological improvements of the past five decades, mainly the use of computer,

impedance reduction and association with other techniques (e.g. Magnetoencephalography, PET and

fMRI), enabled researches to identify earlier components and divide them in subcomponents

according to their functionality and neural generators (Garrido et al. 2009; Stefan Koelsch 2012;

Risto Näätänen, Jacobsen, and Winkler 2005).

1.4 N100.

The above mentioned negativity, for instance, is nowadays observed in earlier latencies,

around 100ms, reason why it became named N100 or N1. For the purpose of this work the N1 will

not be discussed in depth because our questions relate to the processing of long-term dependencies

in music, which literature has suggested to be observed in latencies between 180-200ms (Stefan

Koelsch et al. 2000) or even later 125-300ms (Friederici 2004). The N1 has been interpreted as

divided into 6 sub components according to their different neural sources and refraction time. The

first three are considered as the “true” N1 and the other ones as an overlap with the MMN (R.

Näätänen and Picton 1987), discussed below.

Regarding its functionality, the “whole” N1 component can be summarized as the first

prominent and stable brain response sensible to the physical energy variances (intensity and

frequency) of a stimuli, even though after a short time repetition has its amplitude diminished (May

et al. 1999). This means that N1 can be used as a neurophysiological parameter of the brain's

response to different sound frequencies (R. Näätänen et al. 1988). In the auditory modality, the N1

is an ERP component more implicated to the sensorial neural response of a sound than its

perceptual trace (Schröger 2007).

1.5 Mismatch Negativity (MMN).

While in N1's latency (80-120ms) we can observe a sensorial brain response, it is beyond

this latency that EEG experiments have identified correlates of many cognitive processes such as

sound patterns and stream discrimination. Therefore the many different labels (MMN, ERAN,

ELAN, EAN) that are attributed to the negativities peaking from 120 to 400ms are findings

suggesting that during this time window a group of specific cognitive tasks are processed and can

be broadly localized in the brain ( “broadly” is used here due to EEG´s typical unreliability for brain

source location). Moreover, taking advantage on EEG´s good temporal precision, studies have

investigated the time threshold of subjects’ conscious auditory perception by linking

electrophysiological responses to unperceived auditory stimuli (Loui et al. 2005; R. Näätänen and

14

Winkler 1999; Sussman, Ritter, and Vaughan Jr. 1998; Winkler, Karmos, and Näätänen 1996).

Among all ERP components studied so far, the MMN has accumulated the biggest literature

which established a landmark in the research on latencies and neural sources of pre attentional

cognitive processing (Escera et al. 2000; Garrido et al. 2009; Risto Näätänen, Jacobsen, and

Winkler 2005; Schröger 2007). The MMN is defined as a negative wave that peaks between 100 -

200ms generated in the secondary auditory cortex in response to an auditory change compared to a

previous homogeneous sequence (Winkler et al. 1990). Schröger argues that in order to process

sound features of an ongoing sound stream the brain must realize at least 3 basic processes: 1-

separate the incoming sound in different features (pitch, intensity, duration, timbre) enough to

isolate a distinct sound object; 2- create and retain a model built with the very recent past sound

information apprehended from environment; 3- compare the incoming stream to the sound object

just built (Schröger 2007). It is widely accepted that all these processes occur before the subject

consciously focus attention on the deviant sound object (K. Alho et al. 1989; Kimmo Alho et al.

1992; Escera et al. 2000; Jääskeläinen et al. 2004; R. Näätänen and Winkler 1999) which leads to

the assumption that sensory memory is active pre attentively, storing spectral, temporal and spatial

information, during this latency after stimulus onset (Alain and Woods 1997).

Investigating MMN´s independence to attention, Näätänens et al., created an experiment in

which subjects were requested to detect intensity and frequency deviants of a sound sequence

delivered in one ear while they received in the other ear another sequence with same type of

deviants. Rather than observing that stimuli coming from both ears (attending or ignoring) elicited

MMN, their results showed that the MMN's amplitudes were not affected by attention for frequency

deviants, but only for intensity deviants. This finding contributed to later studies of the fading out

effect when one focuses attention in a single sound source in a party, the so-called cocktail party

effect (Conway, Cowan, and Bunting 2001). In order to test the possibility that it was the lack of

attention which reduced the MMN´s amplitude for unattended intensity stimuli, another experiment

was conducted where subjects received the same stimuli but this time completely ignored it while

reading a book. Results showed that MMN amplitudes' for unattended intensity stimuli were

smaller than the amplitudes when subjects were completely ignoring all sound during a reading

section. In other words, when subjects are not paying attention at all to the stimuli, their MMNs

amplitudes are higher than the response to one of the ears receiving stimuli while focusing attention

to the other. Based on these findings Näätänens et al., proposed that is was due to a probable

increase in the general demand of brain resources (resulting from the deliberate focus of attention)

that caused a diminished activation of MMN mechanisms, but that it was not attention per se that

influenced the component (R. Näätänen, Paavilainen, et al. 1993; Sams et al. 1985).

15

Following this research line, Sussman et al., investigated how early sensory memory could

segregate sound streams and, also, to which extent it was influenced by attention (Sussman et al.,

1998, 1999). The first work (published after the second) consisted on a sequence of 50ms

alternating high and low tones connected in groups of 6 tones: H1-L1-H2-L2-H3-L3. The deviant

stimuli, presented on a 16% rate, were either low or high deviant, H1-L3-H2-L2-H3-L1 or H3-L1-

H2-L2-H1-L3, respectively. In order to create a sound stream segregation effect, stimuli were

presented to subjects with short or long ISIs, 100 or 750ms respectively. The short ISI produces the

perception of two separated sound streams using the same frequencies as the long ISI. For the

control condition only the low tones were used separately, L1-L2-L3. Subjects were instructed to

ignore the stimuli while reading a book (Figure 2). Authors observed MMNs for the alternating

tones and for control condition, what is to say, the results suggest that subjects did discriminate

sound streams without attention focus.

Figure 3. ISI influenced MMN emergence. (Sussman et al., 1999) (a) with long ISI subjects

presented no MMN to deviant tones; (b, c) the tones presented with short ISI segregate to low- and high-tone

sequences pre attentively creating the memory trace underlying the MMN system maintaining the two

sequences; (d, e) the deviant sequences presented in the high or low streams was offset by one cycle so that the

low- and high-tone deviants did not occur within the same cycle of six tones. The low-tone deviant always

preceded the high-tone deviant .

The second study of Sussman et al. replicated the same experiment but this time with

subjects focusing attention on stimuli in a detection task. The results were in line with Näätänen´s

1993 study, and authors similarly suggested that it is not attention itself that modulates cognitive

16

processes in this latency, but the drain of neural resources caused by attention for a concurrent

stimuli.

Both studies produced important secondary findings: 1. MMNs were found exclusively for

the low tones either when presented alone or embedded in the alternating sequence; 2. all subjects

that presented MMNs for the control condition did not present MMN during the alternated

condition; 3. The control condition produced MMN only in four of nine subjects. An important

observation is that the authors did not make any distinction of musical expertise among the subjects.

Most likely, musical expertise can have influenced subjects to present MMN exclusively for the low

deviants. Western music is based on harmony which relies on the lowest notes of chords to establish

musical context, and harmony is implicit even in a one voice melody (Rohrmeier 2011). We

therefore speculate that subjects trained in music or even long exposed to it may have listened to

Sussman et al. experiments' stimuli differently, inattentively considering more the lowest note,

which holds fundamental information about the harmonic direction.

Converging the explanation of classical ERP studies to the theoretical grounding of this

research, our hypothesis that long exposure or intense practice of music may alter brain plasticity is

based on another of 1993 Näätänen´s study (R. Näätänen, Schröger, et al. 1993). In that work, they

asked whether MMN´s could produce evidences of the development of a memory trace for complex

sounds. The experiment consisted on the delivery of sequences containing complex spectral sound

patterns, a train of 180 standard blocks 365 ms long composed by the succession of 8 segments

with the frequencies 720, 500, 638, 1040, 1175, 565, 815 and 920 Hz. The 20 deviant blocks stimuli

pseudo randomly distributed within the standards were an alteration of the sixth segment (from 565

to 650Hz). Subjects were reading a book during 3 sessions ( early, middle and late) of unattended

Figure 4. Conscious contact with stimuli leads to

development of sensorial memory trace maintained in

further unattended tasks (Näätänen et al. 1993). Between

'early', 'middle' and 'late' sessions subjects tested if they could detect

deviant stimuli. Note increase of MMN´s amplitude(orange shade)

after more detection sessions. Bottom, in blue, depiction of the 8

frequencies used to build the complex spectral stimulus. The deviant

was produced by changing the sixth frequency (arrow).

17

listening, between these sessions they were required to listen to a short session of the stimuli

previously presented and try to identify the deviants. The results showed that subjects' MMN

presented increased amplitudes in progression with the detecting sessions, even though during the

early, middle and late sessions they were not attending to the stimuli (Figure 4).

The control group did not submit to the detection sessions. The results for this group showed

small amplitude MMN and with no differences between the sessions early, middle and late. This

result was interpreted as a consequence of the lack of sessions in which subjects consciously

attended to the stimuli: control subjects that did not go through detecting sessions were prevented

from forming a memory trace for the acoustic pattern, which from there on, would not be

incorporated to their long-term memory. This experiment showed how MMN amplitude increases

may be linked to the formation of sensorial memory traces for complex spectral stimuli. We suggest

that by Näätänen´s study results (R. Näätänen, Schröger, et al. 1993) it is very likely that musicians

ERPs to syntactic musical perception may differ from non-musicians due to the former´s intense

exposure to music.

1.6 Left Anterior and Early Left Anterior to Early Right Anterior Negativity (LAN and

ELAN to ERAN).

Authors differentiate ERP components by the latencies and their suggested functionality, that

is, which cognitive processes are being computed during that specific time window. While the

properties of sound features extraction and retention in sensory memory and comparison with a

built up model are the core distinction between MMN and N1, for the next latencies it is the

topography and time related aspects of long-term memory that establishes the distinguishing line.

Specifically, the border discussion is around studies that have been relating MMN to abstract, and

therefore not to a sensory stimuli (Paavilainen, Jaramillo, and Näätänen 1998; van Zuijen et al.

2006). By abstract the authors referred to a sequence of auditory stimuli which were frequent

ascending standard tone pairs with randomly deviant descending tones: as all tones were presented

in regular intervals, the descendent deviant tone elicited a MMN not due to the acoustical novelty (

since it was presented in the previous tone) but to a generalized abstract rule. Including the

processing of abstract features to the MMN´s latency (100-200ms), such as implicit rule extraction

from a sound stream, leads to the assumption that this component relates to long-term memory

based processes, since subjects necessarily needed time to implicitly learn the abstract rule

governing the auditory stream and then react to it pre-attentively. We argue that even though some

studies suggest that MMN does emerge for irregularities in musical scales, which can point to some

18

kind of syntax memory, the latencies observed in the mentioned studies (Paavilainen et al., 1998;

van Zuijen et al., 2006) happen to occur later than most MMNs, at 250 and 275ms, respectively. In

other words, it is likely that there is within this “late” MMN an overlap of waves generated by

neuronal populations implicated in long-term memory activity.

We deduce the above by examining the past twenty years literature of ERP studies focused

on latencies beyond 200ms. The first ERP studies to migrate from sensorial stimuli to explore pre

attentional long-term memory based processing came from language research. The early left

anterior negativity (ELAN) and left anterior negativity (LAN) were the first labels to comprise the

post MMN latency (between100-300ms, 300-500ms, respectively) and were attributed to morpho-

syntactic violations such as gender and number agreement, pseudo words and verb-argument

incongruities (Friederici, Pfeifer, and Hahne 1993; Gunter, Friederici, and Schriefers 2000; Hahne

and Friederici 1999; Münte, Heinze, and Mangun 1993; Neville et al. 1991). The widespread

experiment design used in the majority of LAN and ERAN studies can be found in Osterhout,

Bersick, and Mclaughlin 1997. A protocol in which phrases containing syntactic (gender)

irregularities were visually presented to subjects during EEG scalp recording. Here the phrase The

man prepared himself/herself to the operation elicited a LAN peaking around 300ms after the onset

of the gender definition violation. In a similar protocol (Hahne and Friederici, 1999) found an

ELAN peaking at 200ms and 400ms after past participle violations (Figure 5).

LAN ELAN

Figure 5. Language syntactic errors eliciting LAN(Osterhout et al., 1997) and ELAN (Hahne

and Friederici, 1999). Left, LAN observed 300ms after the presentation of nouns of phrases which made

definitional gender match (solid line) or mismatch (doted): in response to a final word in sentence which had a

reflexive pronoun (e.g 'herself', 'himself') that matched or not with the phrases' subject. Right, ELAN observed by syntactically incorrect sentences containing a phrase structure error: a preposition

19

appeared after the auxiliary and was directly followed by a past participle: The goose was in the fed . ELAN

peaking around 200ms for 20% of deviants per block of phrases and around 400ms for blocks with 80% of

phrases containing syntactic violations.

Following this line of research Stefan Koelsch et al.(2000) used a similar experiment design

to address the same language question but here applied to music: the authors proposed to test

whether it would be possible to observe ERP components in response to the violation of an

automatic built context in music, to the violation of the expectation on an overall previously learned

set of rules. Even though the word syntax is not mentioned in the article, a previous long-term

memory acquired by implicit learning was indispensable (as happens in language) given the fact

that the ERP was not elicited by an acoustical novelty effect. Such was their suggestion that there

was among the subjects a brain ability to learn a musical syntax by implicit learning, that the

authors named the article “Brain indices of music processing: 'Non musicians' are musical”.

The experiment consisted in auditively presenting non-musician subjects to 172 sequences

of five chords. Among these chords 10% had timbre deviants inserted to create a sound detecting

task. Subjects were instructed to identify these timbre deviants. As shown in the figure 6, the five

chords sequence were harmonically concatenated according to western music theory rules in order

to produce syntactically stable phrases (a, left); slightly unstable phrase with a Neapolitan chord in

the third position(a, middle); and a unstable phrase with a Neapolitan chord in the fifth position( a,

right). The Neapolitan chord is widely used in western tonal music for modulation and expressive

purposes because it is a consonant minor chord (when played isolated) with notes out of the tonality

but which can be inserted in major harmonic contexts (progressions) and sound like a subdominant

chord (Bharucha and Krumhansl, 1983; Krumhansl and Kessler, 1982). This harmonic resource

enabled the authors to access a negativity lateralized to the right hemisphere and peaking around

200ms (figure 6, c). This occurred with the highest amplitude when the Neapolitan chords were

presented in the fifth position (figure 6 c, left) and with a much smaller amplitude when in the third

position(c, middle). These results confirm the theoretical works (Piston, 1948)(page 265) which

explain the Neapolitan to be acceptable as a subdominant, even though it’s out of tonality notes

make it an exotic subdominant, reason why we hypothesize the even though present significantly

smaller negativity.

20

Koelsch and colleagues justify that although the latency was still within the MMN, the

experiment design and stimuli pointed to different brain processing, much related to an automatic

hierarchy recognition than to acoustical events. Observing a high resemblance with LAN and

ELAN components from language, which were elicited for syntactic disruptions, the authors

decided to name this component early right anterior negativity (ERAN) to brain responses to

musical syntactic violations.

Considering that the ERAN was elicited by a brain mechanism involved in grasping musical

syntax pre attentionally, authors hypothesized that intense practicing and exposure to music could

contribute to improve the neural architecture demanded for musical syntactic computing. Using the

exact same protocol as the previous study, Koelsch, Schmidt, and Kansok (2002) investigated the

effects of musical expertise in the emergence of the ERAN. The results showed that indeed

professional musicians with declared daily practice presented ERANs with higher amplitude when

compared to novices (figure 7, left). Replicating the Koelsch 2000 study but seeking to understand

how adulthood influences ERAN, Koelsch et al., (2003) tested children aged 5 and 9 years (figure

7, right). The results suggest that children do process music based on hierarchical rules and that

older kids produce ERANs with higher amplitudes. The study also revealed that no ERAN was

produced when children listened to the Neapolitan chord in the third position (the position in which

adults showed a very small amplitude ERAN). Authors interpreted that adults amplitudes (even

though smaller) on the third position were due to their overall higher specific representation of

music regularities: since children were less exposed to music their mental representation of

harmonic rules is less specific.

Figure 6. Syntactically incorrect chords produce ERAN (Koelsch et al., 2000). a.(left) syntactically correct chord progression without Neapolitan chord, ( middle) slightly incorrect, “expressive”, chord progression with Neapolitan in the third position, (right) syntactically incorrect chord progression ending with Neapolitan on last chord(fifth); (b) sequences of 'a'(left, middle and right) were connected in order to simulate a musical harmonic progression; (c) ERAN peaking around 180ms triggered by the Neapolitan in the incorrect (fifth) position(left), and on the third position(middle). FT8, location of the electrode with strongest ERAN effect (right).

21

Figure 7. ERAN is influenced by expertise and age (Koelsch et al., 2002a; Koelsch et al., 2003). Left: Subjects with music expertise presented ERAN peaking around 200ms with higher amplitudes compared to

non-musicians (novices) for Neapolitan chords presented at 5th position. ERAN was followed by a N500. Right: 5

and 9 year old subjects' ERAN (solid line) elicited by Neapolitan chord in 5th position around 220ms. Dotted line

for in syntactically correct chord (no accidents) in fifth position.

Adding some debate to the ERANs studies, Brattico et al., (2006) extrapolated the concept

of syntax in chords and proposed to examine which ERPs could be elicited by out of tune and out of

key notes in homophonic melodies(without chords, simultaneous sounds). A negativity peaking in

the common latency of both ERAN and MMN was observed for both out of key and out of tune

notes, with higher amplitude for the latter. Authors argued that the neural response to the deviant

stimuli could not be attributed to acoustical or novelty properties since there were no repeated notes

enough to create a standard pattern. The findings relate to the source of this ambiguous ERP:

ERAN are observed with higher peaks at frontal and fronto temporal scalp regions(Koelsch et al.,

2000, 2002a) and the negativity of this study had higher peaks at superior temporal lobe which are

known MMN peaking regions. These results point to a reconsideration about how early (around

200ms) long-term memory based processes of syntactic integration are computed in the brain.

Independently of the labels, many authors within the period of this discussion seeked

support for the existence of a memory based automatic musical syntax recognition by investigating

the brain regions involved in music and language processing (Besson and Schön 2001; Stefan

Koelsch et al. 2002; Maess et al. 2001a; Patel et al. 1998; Schön, Magne, and Besson 2004;

Tervaniemi et al. 2000). Combining different brain imaging techniques, fMRI, MEG and PET,

Koelsch and colleagues gathered important findings which support the overlap of brain regions

involved in music and language recognition (Besson and Schön, 2001; Ehrlé et al., 2001; Halpern et

22

al., 2004; Koelsch, 2005; Koelsch and Friederici, 2003; Koelsch et al., 2002b, 2005; Maess et al.,

2001). There is a general acceptance among the above-mentioned studies that musical and language

processing share common networks in the brain. These evidences strengthen the hypothesis that the

brain has an automatic mechanism able to process music and language according to their inherent

syntactic rules.

23

2. Objectives

General

i. Identify whether there is a brain processing of SLTDM.

ii. Identify if musicians and non-musicians present different EEG brain

measurements when exposed to disruptions of SLTDM

Specific

i. Determine the latency and predominant topographical distribution of the

eventual ERP raised by SLTDM.

ii. Observe eventual differences of this ERP for SLTDM with previous works on

MMN.

iii. Identify the amplitudes and latencies differences among musicians and non-

musicians.

24

3. Methods

3.1. Stimuli

Willing to investigate the perception of real music stimuli we selected two J.S. Bach

chorales, BWV 302 and 373. These chorales are built by two phrases separated by a half cadence on

the last chord of the first phrase and obey the traditional harmonic sequence of 1st - 5th - 1st chords ,

also called Tonic – Dominant – Tonic chord progression.

We created a sequence containing original, modified and timbre deviant chorales. Modified

chorales had the first phrase transposed a second up for BWV 302 (figure 8) or a fourth down(BWV

373). This manipulation was planned in such way that local harmonic relations were preserved

while in the same choral the long-term hierarchy were disrupted. In order to preserve local

dependencies we based the choral selection and manipulation criteria on harmonic rules (Rohrmeier

2011; Schoenberg 1978) as follows: the following chord from the second phrase should be different

from both the tonic (which would conclude the sequence since the previous was the dominant) and

the seventh( which would reinforce the dominant); and should also not produce longer intervals

then a fifth in the lowest notes, since it would create a discrepancy in the general pitch of the second

phrase, and therefore break the local nest.

Note that our manipulation created identical acoustical stimuli but with a long-term

dependence incongruity. As illustrated in figure 7, while in the original version ( A) the 1st -5th -1st

global dependency is fulfilled by the chords D major(tonic)- A Major(dominant)- D Major(tonic), in

the modified version (B) this hierarchy is disrupted since the expected E Major chord as the

conclusion of the choral, fulfilling the E Major( tonic) – B Major ( dominant)- E Major( tonic), does

not occur. Importantly, the final chord is the same in both versions.

Score manipulation were realized using Sibelius 6.2 software( Avid Tech Inc.). In musical

instrument digital interface(MIDI) the two versions were transposed to all 12 keys. Summing an

amount of 48 stimuli ( 2 chorales x 2 versions x 12 keys). Each stimuli was presented 5 times all

stimuli summed 53 minutes. In order to assure that second phrases were acoustically the same in

both conditions, the latter were copied and pasted using sound edition software Audacity 2.0

(audacity.sourceforge.net). Tempo was 100 beats per minute and the timbre was piano sound

for original and modified versions and for timbre deviant version two seconds of bassoon timbre

were inserted in one of the voices of the chorales, in 15% of trials. This insertion was used as a task

relevant paradigm to keep subjects aware and focusing attention on the stimuli but not specifically

in the harmonic disruptions.

25

Figure 8. Chords long-term dependency manipulation of the chorale Ein feste Burg ist unser

Gott (J. S. Bach, BWV 302). (a) The chorale has 2 phrases: the first ( D to A) begins with a D Major chord

in the 1st position (tonic) of the tonality scale and ends on a A Major chord in the 5th position (dominant)

producing a half cadence, suspension. The second phrase ( A to D) begins with the chord in the 5th position

(dominant) and ends on the final chord in the 1st position (tonic). This progression holds the long-term musical

syntactic dependency known as '1st - 5th - 1st progression' where the 5th chord establishes tension relation with

the 1st chord at the beginning and a relaxation, tension resolution with the 1st chord at the end; (b) The long-term

dependency is altered by shifting the whole first phrase( D to A) one tone up ( where previously there was a tone

D went up to E, C to D, E to F, and so on) making it an E to B phrase. This manipulation kept the syntactic

coherence in the first phrase but altered the long-term dependency for the last chord ( D), therefore breaking the

'1st - 5th - 1st progression'.

3.2. Subjects

We gathered 12 musicians and 12 non-musicians, 6 females in each group, age-range 23–39

years, M = 27:7. Musicians were recruited from Universität der Künste (Berlin) and had at least 10

years of formal instrument study. Non musicians stated that had no instrument classes besides

regular school education. All subjects were right handed and none had hearing disabilities nor

absolute pitch. Study was approved by the ethic committee of Freie Universität Berlin and was

runned according to Declaration of Helsinke.

26

3.3. Procedure

Subjects were comfortably siting while listened to the sequence of stimuli delivered through

headphones and watched “The Penguin’s March” in a monitor. Subjects were required to hit the

space bar whenever identified a deviant timbre among the stimuli.

After scan session, all subjects were presented again to twelve of the chorales used

previously( 6 from each version) and were required to fill a questionnaire with a 9 level rating

about how conclusive was the last chord, its valence, pleasantness, and arousal.

3.4. EEG recording

Recording used a 64 silver electrode distributed according to 10-20 system, referenced to the

left mastoid, four extra electrodes were placed around the left eye to capture electrooculogram

(EOG). Impedance was kept under 5 k ohms.

27

4. RESULTS

This study proposed to investigate if there is a mental representation for the

theoretical assumption that we perceive music based not only on non-local relations (one

chord/note after the other) but establishing long-term connections between sounds. As a

consequence, according to basic harmony rules, in order to unfold in time, music requires

our brain to organize incoming notes of a music into a hierarchy (e.g. first and fifth chords

have a bigger weight) and hold this pitch information in memory until it reappears assuming

different harmonic functions (e.g. a bridge to neighbor tonalities or the music´s conclusion).

To test our hypothesis we used harmonically modified J.S.Bach chorales which kept local

relations intact while long-term tonic to dominant chords dependencies were broken. We

measured and compared brain electric activity ERPs from the last chords (the concluding

tonic chords reappearance) in the original and modified version.

4.1. ERAN

As shown in Figure 9, the final chord evoked a negativity peaking around 210 ms after the

chords’ onset with statistically different amplitudes for modified and original versions. The effect

had anterior and posterior distribution with left and right lateralization respectively, figure 10.

Following this effect another more persistent negativity was evoked around 600 ms (figure 9 up).

4.2. Statistical analysis.

Statistical relevance was calculated in matlab as follows. First step, a 64 x 1000 x 120 data

matrix was created (channels x points (1000 pts = 2 seconds) x trials (original or modified)).

Second step, means were computed on the amplitudes of both versions within 100 and 300ms after

the last chord onset. For each channel these time locked two versions amplitudes’ means were

subtracted and with max function assessed the biggest value from this subtraction together with the

vector’s index for this value. Third step, centered on this index, data from a 40ms window ( figure 9

middle) was extracted to run a Wilcoxon signed-rank paired test comparing conditions (original and

modified) for each one of the 24 subjects ( subjects had his original version data compared to his

modified version data), figure 9 middle. The Wilcoxon signed-rank test was used because when we

submitted our data to a Kolmogorov- Smirnov test it resulted zero (not normally distributed), we

therefore discarded using ANOVA since besides not normally distributed we had a low sample, 12

subjects per condition.

28

Figure 9. Brain electric responses (mean from 24 subjects) evoked by the final chord measured in channel F1 (up) and PO8 (bottom). Yellow areas depict statistical relevant difference between Original (blue) and Modified (red) conditions calculated by finding the highest amplitude from subtraction of the 120 trials means taken from a window comprising 100 to 300 ms after stimulus onset. Dotted gray line for the onset of the final chord in both BWV 302 and BWV373 chorales. Early negativity (first yellow area) peaking 210ms after last chord onset, later negativity peaking 610ms. Middle, depiction of data window used for statistical analysis.

29

For statistical analysis of the ERP's, four multiple electrodes clusters with regions of interest (ROI)

were calculated: left anterior (AF3, F1, F3, FC3,F5, C1,C3, AF7,C5); right anterior (AF4, F2, F4,

FC4,F6, C2, C4, AF8, C6), left posterior (O1, P7, CP1, CP3, CP5, P3, P5, PO3, PO7), and right

posterior (O2, P8, CP2, CP4, CP6, P4, P6, PO4, PO8), figure 12. The electrode exclusion criteria

was a visual inspection for noisy electrodes; 2 out of 11 electrodes, of each cluster were excluded,

figure 11.

Figure 11. Two out of 11 electrodes were excluded from each of 4 clusters, ruled out as noise and indistinguishable ERP .

Figure 10. Electrodes where ERP presented

statistical relevant differences between

Original and Modified versions. Head electrode

map, system 10-20, depicting electrodes where the

difference between brain electric responses for

Original and Modified conditions was statistically

relevant.

30

31

Figure 12. Cluster of electrodes for ROI's comparing Regions, Musicians and Non-Musicians. Left, mean of 9 electrodes from regions Left and Right Anterior, Left and Right Posterior calculated on a 200ms window (refer to section 4.2. Statistical analysis); Central, depiction comparing amplitude electrical response raised by Original and Modified last chords considering all 24 subjects; Right, Same as “Central” but here separating Musicians from Non-Musicians brain electrical responses. The former contributes with more homogeneous amplitude differences when compared to the latter.

4.3. Musicians and Non Musicians

As shown in figure 12, when means from electrodes clusters of brain regions (Left and Right

anterior, Left and Right Posterior) were calculated, divided in musicians and non-musicians, the

former presented higher amplitude ranges in the ERAN latency at the Right Posterior and Left

Anterior regions, and lower amplitudes ranges at the Right Anterior. Left Posterior ranges were not

statistically differentiable. Amplitude range was assessed by the distance from the peak and valley

in the window from 100 and 300 after stimuli onset on the original condition. A non-paired

nonparametric Wilcoxon sum rank test was run and concluded that inter group comparisons

amplitude ranges for musicians and non-musicians were different, figure 12.

Figure 12. Musicians and Non Musicians region electrodes clusters differ. Each colored bar represents mean of 9 electrodes from the depicted regions (Left- Right anterior and posterior) of 12 musicians and 12 non-musicians. Note that parietal accounts for the widest amplitude range difference.

32

4.4. Brain Regions.

Despite the lack of topographic precision of EEG measurements we attempted to analyze the recruitment of brain areas in the processing of the last chord. For such, we plotted the amplitudes originated by the last chord unfolding in brain areas on a sequence of time windows, figure 13.

Figure 13. Progression of last chord's processing depicted in brain topographic maps. Top,

time progression of original and modified versions. Note similar response for sensorial processing, N100 (80-100ms), with

bilateral temporal-frontal activation of region near auditory cortex. Conversely, in ERAN latency range, a right temporal

activation is observed in 190ms, quickly suppressed and reappears in parietal-occipital region at 210-230, disappearing in

240, while in original version is observed a bilateral (predominant left) temporal activation. Bottom, mean from 64

channels, in 120 trials of 24 subjects for the modified version’s last chord.

33

5. Discussion.

Our results are in line with previous works which observed the ERAN, a fronto lateral

negativity peaking around 190 to 230 ms, related to harmonic disruptions (Brattico et al. 2006;

Fadiga, Craighero, and D’Ausilio 2009; Stefan Koelsch and Friederici 2003; S. Koelsch et al. 2001;

Maess et al. 2001b). However, in our experiment design we identified that our ERP with statistical

relevance had a predominantly parietal distribution therefore at odds with the ERAN's anterior

description: 11 parieto-ocipital electrodes ( Pz, POz, PO3, Oz, P4, PO4, O2, CP6, P6, PO8, P8)

against only 6 fronto-temporal electrodes( F1, F3, FC1, FC5, F4, C5). Also, the statistically relevant

frontal electrodes were predominantly lateralized to the left. Therefore, the ERP identified in this

study to musical syntactic disruptions differed from previous works regarding predominant effect

source. Although we did find a negativity around 200ms in one statistically relevant right anterior

(frontal) electrode, F4, hence an Early Right Anterior Negativity, we found the effect on 4 left

anterior and 7 right posterior electrodes, what could not sum up to a label since the effect was both

present in anterior and posterior regions although with a predominantly posterior electrode

distribution ( 6 anterior versus 11 posterior, figure 10). We interpret that this discrepancy is due

to our innovative paradigm. Here we used harmony deviant stimuli which were present in the same

chord as the standard, what is to say that the exact same acoustic stimuli relates to harmonically

different sequences stored in memory and sustained until the presentation of the last chord, which is

the same for deviant and standard chorale versions (figure 8). By doing so, we assured that the

ERAN observed in the last chord could not relate to a local disruption perception, like a MMN. This

means that the last chord was presented to the listener in a syntactically acceptable context ( no

local disruption) but the brain computed a long-term comparison to an event, the first chord,

presented around 9 seconds earlier( 9.062ms for the BWV 302 and 9.597ms for the BWV 373).

Locally accepted but globally not.

This long distance harmonic controlled paradigm differs from previous studies (Stefan

Koelsch 2005; Stefan Koelsch et al. 2000, 2002; Loui et al. 2005; Maidhof and Koelsch 2010;

Sammler, Koelsch, and Friederici 2011; Steinbeis and Koelsch 2008) which predominantly used a

similar stimuli design. The first ten years of ERAN studies used chord sequences where the deviant

stimuli, even though inserted in a harmonic progression, cannot be ruled out as a possible local

effect (Figure 6a, figure 11). Our experimental design also differs from a variation of this chord

sequence paradigm (Brattico et al. 2006; Kalda and Minati 2012): instead of chords, subjects were

presented to irregularities in tones of a simple one voice melody (no chords used). The ERAN

observed was attributed to the familiarity with musical scale knowledge, acquired by implicit

34

learning (Brattico et al. 2006; Kalda and Minati 2012; Leino et al. 2007; Rohrmeier and Rebuschat

2012) and therefore a mismatch of a pitch presented out of the subjects musical tonality expectancy.

The result does stand for an ERAN raised by a musical cognitive process (the expectancy to listen

to pitches inside the tonality field) and can be ruled out as an acoustical mismatch. However, since

the deviant event (an out of key or mistuned pitch) was not expected in the event by event

comparison, the data cannot stand for the segregation of the overlapped brain mechanisms recruited

for “something went wrong” from “something went harmonically wrong”, figure 14.

Figure 14. Local mismatch even though it is a strictly harmonic disruption (Koelsch 2005). The deviant last chord belongs to the neighbour tonalities and is presented in an inappropriate position in the harmonic sequence, therefore a purely harmonic violation. However, considering the chord-by-chord processing, the local effect(novelty) is still embedded in the last chord/ target. Observing that our protocol overcame previous experimental design which could not

successfully segregate brain computations for local and global mismatch, we hypothesize that the

more parietal contributions from our data may relate to the theoretical conception of hierarchic

connections between chords. As proposed by (Bharucha and Krumhansl 1983), musical syntax is

based on the expectation for the arrival of a stable chord, which can be understood as the return to

the central tonality of the music, subjectively reported as a relaxing feeling. The first chord of a

music establishes the tonality, the tonal center from which the other chords will come and from

where the neighbour tones relations are created. A basic rule of western tonal music is that the tonal

center of an entire music or phrase is presented in the beginning, hold off by “traveling” through

other neighbour tonalities, and finally resubmitted at the end of the piece/phrase, as a form of

determining its conclusion (Piston 1948).

This syntactical processing requires from the listener a memory process to store the first

chord, make online hierarchical organization with the incoming chords and finally refer to it in the

conclusion. The rationale to hypothesize the parietal contributions to this memory processing comes

from PET and fMRI works investigating arithmetic calculations (Stanislas Dehaene and Cohen

1997; Delazer et al. 2003; Fulbright et al. 2000; Menon et al. 2000) and from studies considering

music and math to hold similarity as sequences based in hierarchic connections (Stanislas Dehaene

35

et al. 2015; Schmithorst and Holland 2004). The establishment of a hierarchy among the incoming

elements of a sequence, e.g. new chords of a chorale or new numbers and operations of an

arithmetic computation, assumes that a constant up date is done in relation to a memory trace

formed by the first event. This memory trace is the very condition for a chord or number to unfold

in time and become a mental representation of a sequence. Not only a number is the temporal

instant unit of a computation but also what is referred to as arithmetic fact, which can be simple

operations repeated enough to become a lexicon, accessed automatically like a word without mental

manipulation(e.g. 2+2=4, 3³=27, cosine(0)=1)(Ashcraft 1992; Stanislas Dehaene 1992)

Neuroanatomical studies on arithmetic processing have come to a concept of a fronto-

parietal network in which arithmetic facts, exact calculations, are linked to the activation of frontal

and prefrontal brain areas while the parietal lobe has been suggested to be engaged in the processing

of complex calculations (e.g. approximation, 2 digit multiplication, 3 operand addition/subtraction

problems)(Chochon et al. 1999; S. Dehaene et al. 1999; Menon et al. 2000; Zago et al. 2001). This

fronto-parietal network was grounded in Dehaene's (Stanislas Dehaene 1992; Stanislas Dehaene and

Cohen 1997) theorization of human number processing architecture model, the so called triple code

model. As later would be fit in different brain areas, this model assumed that number processing

recruits distinct areas for arithmetic facts implicated in the activation of the left perisylvian network

associated with language (Broca and Wernicke), hence verbal code; visual number form associated

to the ventral visual pathway( inferior temporal and fusiform gyrus) activation, visual code; and for

comparison, approximation based on magnitude representation, necessary for time demanding

mental computation, the bilateral inferior parietal areas were recruited, magnitude code (Delazer et

al. 2003; Klein et al. 2016; Pesenti et al. 2000; Schmithorst and Brown 2004), figure 15.

(Dehaene et al., 1995)

36

Figure 15. Fronto-parietal network recruited in arithmetic calculation (Schmithrost, 2004). Top, schematic diagram of the triple code model. Arithmetic facts processed in anterior regions while magnitude

representations and comparisons are processed in posteriorly. Note the articulatory loop connecting anterior and

posterior brain regions. Bottom, function activation in brain regions suggested to be associated to magnitude

code(a), verbal code(b), and visual code (c)(d).

We here propose that the identification of a parietal contribution for complex memory based

arithmetic processing is a plausible parallel to interpret our predominant parietal effect in face of

our stimuli controlled for long-term dependencies in music. Similarly to a number approximation

and a 2 digit multiplication, which requires the sustention of a number or partial sum in memory ,

the syntactic relation of a first chord is kept in memory and sustained until the fulfillment of the

expectancy of its representation, the return to the tonal center. Furthermore, a parallel can also be

drawn between the arithmetic fact and auditory oddballs: the latter, like the former, is an automatic

brain reaction to an already consolidated memory producing a binary response to a right or wrong

stimulus, hence a fact. Considering that oddball paradigms have largely replicated a MMN with

higher amplitudes in fronto temporal regions ( May et al. 1999; Escera et al. 2000; Garrido et al.

37

2009; Risto Näätänen, Jacobsen, and Winkler 2005; Schröger 2007) we are tempted to interpret the

similar topographical brain findings for arithmetic facts ( fronto temporal) as a plausible fit for brain

areas engaged in processing auditory oddballs.

Our assumption that brain pathways engaged in arithmetic processing may be generalized to

the processing of other high level cognitive sequences, such as music syntax, is also based on

fundamental neuroanatomical evidences for a specific brain area involving music perception. First

relevant pinpoint of a musical processing network came from brain lesion cases in which patients

reported lost of music aesthetic perception, difficulty in discriminate sounds, pitch, rhythm and lost

of music´s gestalt. In the words of one of the first patient studied, amusia lesions result on the

difficulty “in grasping music as I used to” (Mazzoni et al. 1993; Mazzucchi et al. 1982). The studied

subjects presented lesions in their right temporal cortex around the middle portion of the sylvian

fissure.

Further studies on subjects reporting difficulty in identifying pitch intervals of semitones (

the smallest pitch unit in Western music) led to the identification of a congenital amusia(CA) (Hyde

and Peretz 2004; I 1985; Peretz and Kolinsky 1993). It is estimated that 4% of world population

suffers from congenital amusia (Kalmus and Fry 1980), and are therefore born with a deficiency in

music perception and production(less oftenly). Studies have been relating one possible cause of the

disease to a reduced white mater in the right inferior frontal gyrus (rIFG) associated to an increase

in gray matter(Hyde et al. 2006, 2007). This tissue abnormality was hypothesized to generate an

imporvished communication between the right auditory cortex and the rIFG(Hyde et al. 2007). This

postulation received contribution from (Loui, Alsop, and Schlaug 2009) findings which identified

less fiber volume of the arcuate fasciculus in subjects with CA. The arcuate fasciculus(AF) is a

bundle of axons with an important role in the posterior – anterior communication of multimodal

neural sites, like Wernicke (BA22) to Brocas(BA 44,45), caudal dorsolateral prefrontal cortex (BA

6, 8) to the inferior parietal lobule (BA 39, 40)(Phillips et al. 2011). Congenital amusia or brain

fronto-temporal damage on either side can provoke musical deficits, but the nature of the deficiency

relates to different brain hemispheres: right relate to melody and left to rhythm impairments (Peretz

1985; Di Pietro et al. 2004; Peretz et al. 2009), figure 16.

38

Figure 16. Arcuate fasciculus connects posterior and anterior multimodal neuronal sites and its reduced volume has been related to amusia (Loui et al.2009). Left, representation of arcuate fasciculus connecting anterior( Broca BA44, 45) to posterior( Wernicke BA21,22) areas; Right, (a) tractography of normal subjects' Afs: left-superior(yellow), left-inferior(pink), right-superior(red), right-inferior(green). Note that in tractography of amusic subjects(b) the right volume is reduced.

Here we propose possible shared mechanisms to process mathematic and music hierarchic

sequences coming from brain lesion cases in the arithmetic fronto-parietal circuit which lead do

acalculia ( difficulty in performing arithmetic calculations)(Cohen et al. 2000; Stanislas Dehaene et

al. 2003; Girelli et al. 1996; van Harskamp and Cipolotti 2001) compared to amusia cases which

also relate to damages in brain structures integrating fronto and parieto areas ( arcuate fasciculus).

In a first glance our assumption may seem pointing to a different direction of the well-

established findings of Tervaniemi et al.(Tervaniemi et al. 2000). Using PET, they observed the

activation of areas BA 42 for deviant chords (when subtracted the activity of standard chords

presented alone to the activity of standard and deviant presented together.) (Figure 17).

39

Figure 17. Deviant chords activate BA 42 of Superior Temporal Gyrus (Tervaniemi et al.

2000). Right, back and top view from PET data showing neuroanatomical correlates for an odd-ball paradigm using A

major( standard) and A minor( deviant) chords. Corroborating with Tervaniemi, Maess et al., 2001 found anterior activations when

functionally deviant chords were presented in different positions of a harmonic progression (the

Neapolitan paradigm designed by Stefan Koelsch et al. (2000). In the former, authors suggest a

ERAN source in the pars opercularis of the inferior frontal gyrus(IFG), figure 18.

Figure 18. Left and right sagittal and axial (parallel to anterior commissure-posterior

commissure line) view (Maess 2001). Grand average of dipole solutions for both magnetic ERAN(blue) and magnetic P200(yellow) refer to two-dipole configurations(one dipole in each hemisphere) for deviant chords.

Although these findings suggest an anterior location of brain sources for musical syntax

40

processing we argue that they reinforce our parietal hypothesis sustained in the rationale “fronto

parietal network for arithmetic – arcuate fasciculus involved in amusia – frontal neuroanatomical

sources for mismatch detection in chord sequences” developed in this section. Note that we refer in

the latter to a mismatch. This is because we understand that due to the fact that previous

experimental designs have not been able to isolate the auditory MMN effect from the musical

syntactic MMN (which has been called ERAN) previous ERAN findings necessarily contain an

overlap of the general detection of an auditory mismatch. We consider that in the case of a local

disruption, there is a general mismatch detecting process of “something went wrong” irrespective

of the nature of this mismatch ( semantic, syntactic, sensorial), which literature has identified as a

fronto-temporal effect often stronger in the right hemisphere for tone paradigms and on the left for

language paradigms (Garrido et al. 2009). Once we removed the local component of mismatch by

using a presumed long-term memory based mismatch, we interpret that our data reflect the

attenuation of the general classical MMN and an increase in posterior regions due to the memory

based musical syntactic relations (figure 13). In figure 13 we can also infer a major difference

between the brain areas recruited in the processing of original version (a normal sequence of music)

and the modified version (a musical sequence which activated a comparison mechanism due to a

global dependence disruption). While the former activates temporal slightly left lateralized areas (

from 190 to 220ms), the latter engages first an activation of a right temporal area at 190ms (

identified in previous works as the ERAN latency) which disappears after 10ms and at 220ms

initiates what we here named a parietal ERAN peaking at 230ms.

Our results are also in line with previous studies which observed an overall wider EEG

amplitudes in musicians when compared to non-musician, not only for ERAN (Brattico et al. 2013;

Stefan Koelsch and Jentschke 2008; Stefan Koelsch, Schmidt, and Kansok 2002) but for ERP's in

other latencies (Besson and Faita 1995; Patel et al. 1998; Rüsseler et al. 2001) and also fMRI

studies (Stefan Koelsch et al. 2005; Minati et al. 2008; Schulze et al. 2011). The mentioned studies

meet a consensus that the intense exposure and practice of music promotes plastic modifications in

the musicians' brains which are presumably related to higher electric amplitudes and voxel intensity

when exposed to musical stimuli. Our parietal findings with enhanced amplitudes for musicians

when compared to non-musicians (figure 12) corroborate with this assumption since the stimuli

used in the present study isolated the activation of brain areas involved in the maintenance of the

first chord in memory not contaminated by a local novelty effect.

We assume the speculative nature of the comparison between parietal findings from

arithmetic studies and our parietal results with musical long distance dependencies stimuli and that

further studies are necessary to better sustain this assumption. We also must state that even though

our findings point to some areas implicated in processing of music syntax, these areas may very

41

likely not be the source of this processing but one of the projections or sum of neural activities

recruited in this cognitive mechanism. EEG's low spatial resolution is a limitation which inspires us

to consider future studies using similar paradigm with better spatial brain imaging techniques like

PET and fMRI.

BIBLIOGRAPHY.

Alain, Claude, and David L. Woods. 1997. “Attention Modulates Auditory Pattern Memory as Indexed by Event-Related Brain Potentials.” Psychophysiology 34 (5): 534–46. doi:10.1111/j.1469-8986.1997.tb01740.x.

Alho, K., M. Sams, P. Paavilainen, K. Reinikainen, and R. Näätänen. 1989. “Event-Related Brain Potentials Reflecting Processing of Relevant and Irrelevant Stimuli During Selective Listening.” Psychophysiology 26 (5): 514–28. doi:10.1111/j.1469-8986.1989.tb00704.x.

Alho, Kimmo, David L. Woods, Alain Algazi, and Risto Näätänen. 1992. “Intermodal Selective Attention. II. Effects of Attentional Load on Processing of Auditory and Visual Stimuli in Central Space.” Electroencephalography and Clinical Neurophysiology 82 (5): 356–68. doi:10.1016/0013-4694(92)90005-3.

Anderson, Robert H. 1967. “Syntax-Directed Recognition of Hand-Printed Two-Dimensional Mathematics.” In Symposium on Interactive Systems for Experimental Applied Mathematics: Proceedings of the Association for Computing Machinery Inc. Symposium, 436–459. New York, NY, USA: ACM. doi:10.1145/2402536.2402585.

Ashcraft, Mark H. 1992. “Cognitive Arithmetic: A Review of Data and Theory.” Cognition, Numerical Cognition, 44 (1): 75–106. doi:10.1016/0010-0277(92)90051-I.

Besson, Mireille, and Frederique Faita. 1995. “An event-related potential (ERP) study of musical expectancy: comparison of musicians with nonmusicians.” Journal of Experimental Psychology. Human Perception and Performance 21: 1278–96.

Besson, Mireille, and Daniele Schön. 2001. “Comparison between Language and Music.” Annals of the New York Academy of Sciences 930 (1): 232–58. doi:10.1111/j.1749-6632.2001.tb05736.x.

Bharucha, Jamshed, and Carol L. Krumhansl. 1983. “The Representation of Harmonic Structure in Music: Hierarchies of Stability as a Function of Context.” Cognition 13 (1): 63–102. doi:10.1016/0010-0277(83)90003-3.

Bigand, Emmanuel, Bénédicte Poulin, Barbara Tillmann, François Madurell, and Daniel A. D’Adamo. 2003. “Sensory versus Cognitive Components in Harmonic Priming.” Journal of Experimental Psychology: Human Perception and Performance 29 (1): 159–71. doi:10.1037/0096-1523.29.1.159.

Brattico, Elvira, Mari Tervaniemi, Risto Näätänen, and Isabelle Peretz. 2006. “Musical Scale Properties Are Automatically Processed in the Human Auditory Cortex.” Brain Research 1117 (1): 162–74. doi:10.1016/j.brainres.2006.08.023.

Brattico, Elvira, Tiina Tupala, Enrico Glerean, and Mari Tervaniemi. 2013. “Modulated Neural Processing of Western Harmony in Folk Musicians.” Psychophysiology 50 (7): 653–63. doi:10.1111/psyp.12049.

Chochon, F., L. Cohen, P. F. van de Moortele, and S. Dehaene. 1999. “Differential Contributions of the Left and Right Inferior Parietal Lobules to Number Processing.” Journal of Cognitive

42

Neuroscience 11 (6): 617–30. doi:10.1162/089892999563689. Clarke, Eric F. 1987. “Levels of Structure in the Organization of Musical Time.” Contemporary

Music Review 2 (1): 211–38. doi:10.1080/07494468708567059. Cohen, L., S. Dehaene, F. Chochon, S. Lehéricy, and L. Naccache. 2000. “Language and

Calculation within the Parietal Lobe: A Combined Cognitive, Anatomical and fMRI Study.” Neuropsychologia 38 (10): 1426–40. doi:10.1016/S0028-3932(00)00038-5.

Conway, Andrew R. A., Nelson Cowan, and Michael F. Bunting. 2001. “The Cocktail Party Phenomenon Revisited: The Importance of Working Memory Capacity.” Psychonomic Bulletin & Review 8 (2): 331–35. doi:10.3758/BF03196169.

Dehaene, S., E. Spelke, P. Pinel, R. Stanescu, and S. Tsivkin. 1999. “Sources of Mathematical Thinking: Behavioral and Brain-Imaging Evidence.” Science 284 (5416): 970–74. doi:10.1126/science.284.5416.970.

Dehaene, Stanislas. 1992. “Varieties of Numerical Abilities.” Cognition, Numerical Cognition, 44 (1): 1–42. doi:10.1016/0010-0277(92)90049-N.

Dehaene, Stanislas, and Laurent Cohen. 1997. “Cerebral Pathways for Calculation: Double Dissociation between Rote Verbal and Quantitative Knowledge of Arithmetic.” Cortex 33 (2): 219–50. doi:10.1016/S0010-9452(08)70002-9.

Dehaene, Stanislas, Florent Meyniel, Catherine Wacongne, Liping Wang, and Christophe Pallier. 2015. “The Neural Representation of Sequences: From Transition Probabilities to Algebraic Patterns and Linguistic Trees.” Neuron 88 (1): 2–19. doi:10.1016/j.neuron.2015.09.019.

Dehaene, Stanislas, Manuela Piazza, Philippe Pinel, and Laurent Cohen. 2003. “Three Parietal Circuits for Number Processing.” Cognitive Neuropsychology 20 (3–6): 487–506. doi:10.1080/02643290244000239.

Delazer, M, F Domahs, L Bartha, C Brenneis, A Lochy, T Trieb, and T Benke. 2003. “Learning Complex Arithmetic—an fMRI Study.” Cognitive Brain Research 18 (1): 76–88. doi:10.1016/j.cogbrainres.2003.09.005.

Di Pietro, Marie, Marina Laganaro, Béatrice Leemann, and Armin Schnider. 2004. “Receptive Amusia: Temporal Auditory Processing Deficit in a Professional Musician Following a Left Temporo-Parietal Lesion.” Neuropsychologia 42 (7): 868–77. doi:10.1016/j.neuropsychologia.2003.12.004.

Donchin, E., and D. B. Lindsley. 1966. “Average Evoked Potentials and Reaction Times to Visual Stimuli.” Electroencephalography and Clinical Neurophysiology 20 (3): 217–23. doi:10.1016/0013-4694(66)90086-1.

Escera, C., K. Alho, E. Schröger, and I. Winkler. 2000. “Involuntary Attention and Distractibility as Evaluated with Event-Related Brain Potentials.” Audiology & Neuro-Otology 5 (3–4): 151–66. doi:13877.

Fadiga, Luciano, Laila Craighero, and Alessandro D’Ausilio. 2009. “Broca’s Area in Language, Action, and Music.” Annals of the New York Academy of Sciences 1169 (1): 448–58. doi:10.1111/j.1749-6632.2009.04582.x.

Friederici, Angela D. 2004. “Event-Related Brain Potential Studies in Language.” Current Neurology and Neuroscience Reports 4 (6): 466–70. doi:10.1007/s11910-004-0070-0.

Friederici, Angela D., Erdmut Pfeifer, and Anja Hahne. 1993. “Event-Related Brain Potentials during Natural Speech Processing: Effects of Semantic, Morphological and Syntactic Violations.” Cognitive Brain Research 1 (3): 183–92. doi:10.1016/0926-6410(93)90026-2.

Fulbright, Robert K., David L. Molfese, Alexander A. Stevens, Pawel Skudlarski, Cheryl M. Lacadie, and John C. Gore. 2000. “Cerebral Activation during Multiplication: A Functional MR Imaging Study of Number Processing.” American Journal of Neuroradiology 21 (6): 1048–54.

Garrido, Marta I., James M. Kilner, Klaas E. Stephan, and Karl J. Friston. 2009. “The Mismatch Negativity: A Review of Underlying Mechanisms.” Clinical Neurophysiology 120 (3): 453–63. doi:10.1016/j.clinph.2008.11.029.

Girelli, L., M. Delazer, C. Semenza, and G. Denes. 1996. “The Representation Of Arithmetical

43

Facts: Evidence from Two Rehabilitation Studies.” Cortex 32 (1): 49–66. doi:10.1016/S0010-9452(96)80016-5.

Gunter, Thomas C., Angela D. Friederici, and Herbert Schriefers. 2000. “Syntactic Gender and Semantic Expectancy: ERPs Reveal Early Autonomy and Late Interaction.” Journal of Cognitive Neuroscience 12 (4): 556–68. doi:10.1162/089892900562336.

Hahne, Anja, and Angela D. Friederici. 1999. “Electrophysiological Evidence for Two Steps in Syntactic Analysis: Early Automatic and Late Controlled Processes.” Journal of Cognitive Neuroscience 11 (2): 194–205. doi:10.1162/089892999563328.

Haider, M., P. Spong, and D. B. Lindsley. 1964. “ATTENTION, VIGILANCE, AND CORTICAL EVOKED-POTENTIALS IN HUMANS.” Science (New York, N.Y.) 145 (3628): 180–82.

Harskamp, N. J. van, and L. Cipolotti. 2001. “Selective Impairments for Addition, Subtraction and Multiplication. Implications for the Organisation of Arithmetical Facts.” Cortex; a Journal Devoted to the Study of the Nervous System and Behavior 37 (3): 363–88.

Hofstadter, Douglas. 2000. Gödel, Escher, Bach: An Eternal Golden Braid (20th Anniversary Edition with a New Preface by the Author). Penguin Books Ltd. http://www.amazon.ca/exec/obidos/redirect?tag=citeulike09-20&path=ASIN/0140289208.

Hyde, Krista L., Jason P. Lerch, Robert J. Zatorre, Timothy D. Griffiths, Alan C. Evans, and Isabelle Peretz. 2007. “Cortical Thickness in Congenital Amusia: When Less Is Better Than More.” Journal of Neuroscience 27 (47): 13028–32. doi:10.1523/JNEUROSCI.3039-07.2007.

Hyde, Krista L., and Isabelle Peretz. 2004. “Brains That Are out of Tune but in Time.” Psychological Science 15 (5): 356–60. doi:10.1111/j.0956-7976.2004.00683.x.

Hyde, Krista L., Robert J. Zatorre, Timothy D. Griffiths, Jason P. Lerch, and Isabelle Peretz. 2006. “Morphometry of the Amusic Brain: A Two-Site Study.” Brain 129 (10): 2562–70. doi:10.1093/brain/awl204.

I, Peretz. 1985. “[Hemispheric asymmetry in amusia].” Revue neurologique 141 (3): 169–83. Jääskeläinen, Iiro P., Jyrki Ahveninen, Giorgio Bonmassar, Anders M. Dale, Risto J. Ilmoniemi,

Sari Levänen, Fa-Hsuan Lin, et al. 2004. “Human Posterior Auditory Cortex Gates Novel Sounds to Consciousness.” Proceedings of the National Academy of Sciences of the United States of America 101 (17): 6809–14. doi:10.1073/pnas.0303760101.

Jackendoff, Ray. 2009. “Parallels and Nonparallels between Language and Music.” Music Perception: An Interdisciplinary Journal 26 (3): 195–204. doi:10.1525/mp.2009.26.3.195.

Kalda, Tiina, and Ludovico Minati. 2012. “Detecting Scale Violations in Absence of Mismatch Requires Music-Syntactic Analysis: A Further Look at the Early Right Anterior Negativity (ERAN).” Brain Topography 25 (3): 285–92. doi:10.1007/s10548-011-0208-8.

Kalmus, H., and D. B. Fry. 1980. “On Tune Deafness (Dysmelodia): Frequency, Development, Genetics and Musical Background.” Annals of Human Genetics 43 (4): 369–82. doi:10.1111/j.1469-1809.1980.tb01571.x.

Kandel, Eric R. n.d. Principles of Neuroscience. Kivy, Peter. 1959. “Charles Darwin on Music.” Journal of the American Musicological Society 12

(1): 42–48. doi:10.2307/829516. Klein, Elise, Julia Suchan, Korbinian Moeller, Hans-Otto Karnath, André Knops, Guilherme Wood,

Hans-Christoph Nuerk, and Klaus Willmes. 2016. “Considering Structural Connectivity in the Triple Code Model of Numerical Cognition: Differential Connectivity for Magnitude Processing and Arithmetic Facts.” Brain Structure and Function 221 (2): 979–95. doi:10.1007/s00429-014-0951-1.

Koelsch, S., T. C. Gunter, E. Schröger, M. Tervaniemi, D. Sammler, and A. D. Friederici. 2001. “Differentiating ERAN and MMN: An ERP Study.” Neuroreport 12 (7): 1385–89.

Koelsch, Stefan. 2005. “Neural Substrates of Processing Syntax and Semantics in Music.” Current Opinion in Neurobiology, Cognitive neuroscience, 15 (2): 207–12. doi:10.1016/j.conb.2005.03.005.

———. 2012. Brain and Music. John Wiley & Sons. Koelsch, Stefan, and Angela D. Friederici. 2003. “Toward the Neural Basis of Processing Structure

44

in Music.” Annals of the New York Academy of Sciences 999 (1): 15–28. doi:10.1196/annals.1284.002.

Koelsch, Stefan, Thomas Fritz, Katrin Schulze, David Alsop, and Gottfried Schlaug. 2005. “Adults and Children Processing Music: An fMRI Study.” NeuroImage 25 (4): 1068–76. doi:10.1016/j.neuroimage.2004.12.050.

Koelsch, Stefan, Thomas C. Gunter, D. Yves v. Cramon, Stefan Zysset, Gabriele Lohmann, and Angela D. Friederici. 2002. “Bach Speaks: A Cortical ‘Language-Network’ Serves the Processing of Music.” NeuroImage 17 (2): 956–66. doi:10.1006/nimg.2002.1154.

Koelsch, Stefan, Tomas Gunter, Angela D. Friederici, and Erich Schröger. 2000. “Brain Indices of Music Processing: ‘Nonmusicians’ Are Musical.” Journal of Cognitive Neuroscience 12 (3): 520–41. doi:10.1162/089892900562183.

Koelsch, Stefan, and Sebastian Jentschke. 2008. “Short-Term Effects of Processing Musical Syntax: An ERP Study.” Brain Research 1212 (May): 55–62. doi:10.1016/j.brainres.2007.10.078.

Koelsch, Stefan, Björn-helmer Schmidt, and Julia Kansok. 2002. “Effects of Musical Expertise on the Early Right Anterior Negativity: An Event-Related Brain Potential Study.” Psychophysiology 39 (5): 657–63. doi:10.1111/1469-8986.3950657.

Konečni, Vladimir J. 1984. “Elusive Effects of Artists’ ‘Messages.’” In Advances in Psychology, edited by W. Ray Crozier and Antlony J. Chapman, 19:71–93. Cognitive Processes in the Perception of Art. North-Holland. //www.sciencedirect.com/science/article/pii/S0166411508623468.

Lamarche, Fabrice, and Stéphane Donikian. 2004. “Crowd of Virtual Humans: A New Approach for Real Time Navigation in Complex and Structured Environments.” Computer Graphics Forum 23 (3): 509–18. doi:10.1111/j.1467-8659.2004.00782.x.

Leino, Sakari, Elvira Brattico, Mari Tervaniemi, and Peter Vuust. 2007. “Representation of Harmony Rules in the Human Brain: Further Evidence from Event-Related Potentials.” Brain Research 1142 (April): 169–77. doi:10.1016/j.brainres.2007.01.049.

Lerdahl, Fred, and Ray Jackendoff. 1985. A Generative Theory of Tonal Music. MIT Press. Lindsley, D. B. 1952. “Psychological Phenomena and the Electroencephalogram.”

Electroencephalography and Clinical Neurophysiology 4 (4): 443–56. Loui, Psyche, David Alsop, and Gottfried Schlaug. 2009. “Tone Deafness: A New Disconnection

Syndrome?” Journal of Neuroscience 29 (33): 10215–20. doi:10.1523/JNEUROSCI.1701-09.2009.

Loui, Psyche, Tineke Grent-’t-Jong, Dana Torpey, and Marty Woldorff. 2005. “Effects of Attention on the Neural Processing of Harmonic Syntax in Western Music.” Cognitive Brain Research 25 (3): 678–87. doi:10.1016/j.cogbrainres.2005.08.019.

Maess, Burkhard, Stefan Koelsch, Thomas C. Gunter, and Angela D. Friederici. 2001a. “Musical Syntax Is Processed in Broca’s Area: An MEG Study.” Nature Neuroscience 4 (5): 540–45. doi:10.1038/87502.

———. 2001b. “Musical Syntax Is Processed in Broca’s Area: An MEG Study.” Nature Neuroscience 4 (5): 540–45. doi:10.1038/87502.

Maidhof, Clemens, and Stefan Koelsch. 2010. “Effects of Selective Attention on Syntax Processing in Music and Language.” Journal of Cognitive Neuroscience 23 (9): 2252–67. doi:10.1162/jocn.2010.21542.

Marvin, Elizabeth West, and Alexander Brinkman. 1999. “The Effect of Modulation and Formal Manipulation on Perception of Tonic Closure by Expert Listeners.” Music Perception: An Interdisciplinary Journal 16 (4): 389–407. doi:10.2307/40285801.

May, Patrick, Hannu Tiitinen, Risto J. Ilmoniemi, Göte Nyman, John G. Taylor, and Risto Näätänen. 1999. “Frequency Change Detection in Human Auditory Cortex.” Journal of Computational Neuroscience 6 (2): 99–120. doi:10.1023/A:1008896417606.

Mazzoni, M, P Moretti, L Pardossi, M Vista, A Muratorio, and M Puglioli. 1993. “A Case of Music Imperception.” Journal of Neurology, Neurosurgery, and Psychiatry 56 (3): 322.

Mazzucchi, A., C. Marchini, R. Budai, and M. Parma. 1982. “A Case of Receptive Amusia with

45

Prominent Timbre Perception Defect.” Journal of Neurology, Neurosurgery, and Psychiatry 45 (7): 644–47.

McAdams, Stephen, and Emmanuel Bigand. 1993. Thinking in SoundThe Cognitive Psychology of Human Audition. Oxford University Press. http://www.oxfordscholarship.com/view/10.1093/acprof:oso/9780198522577.001.0001/acprof-9780198522577.

Menon, V., S. M. Rivera, C. D. White, G. H. Glover, and A. L. Reiss. 2000. “Dissociating Prefrontal and Parietal Cortex Activation during Arithmetic Processing.” NeuroImage 12 (4): 357–65. doi:10.1006/nimg.2000.0613.

Minati, Ludovico, Cristina Rosazza, Ludovico D’Incerti, Emanuela Pietrocini, Laura Valentini, Vidmer Scaioli, Catherine Loveday, and Maria Grazia Bruzzone. 2008. “FMRI/ERP of Musical Syntax: Comparison of Melodies and Unstructured Note Sequences.” Neuroreport 19 (14): 1381–85. doi:10.1097/WNR.0b013e32830c694b.

Münte, Thomas F., Hans-Jochen Heinze, and George R Mangun. 1993. “Dissociation of Brain Activity Related to Syntactic and Semantic Aspects of Language.” Journal of Cognitive Neuroscience 5 (3): 335–44. doi:10.1162/jocn.1993.5.3.335.

Näätänen, R., P. Paavilainen, H. Titinen, D. Jiang, and K. Alho. 1993. “Attention and Mismatch Negativity.” Psychophysiology 30 (5): 436–50. doi:10.1111/j.1469-8986.1993.tb02067.x.

Näätänen, R., and T. Picton. 1987. “The N1 Wave of the Human Electric and Magnetic Response to Sound: A Review and an Analysis of the Component Structure.” Psychophysiology 24 (4): 375–425.

Näätänen, R., M. Sams, K. Alho, P. Paavilainen, K. Reinikainen, and E. N. Sokolov. 1988. “Frequency and Location Specificity of the Human Vertex N1 Wave.” Electroencephalography and Clinical Neurophysiology 69 (6): 523–31.

Näätänen, R., E. Schröger, S. Karakas, M. Tervaniemi, and P. Paavilainen. 1993. “Development of a Memory Trace for a Complex Sound in the Human Brain.” Neuroreport 4 (5): 503–6.

Näätänen, R., and I. Winkler. 1999. “The Concept of Auditory Stimulus Representation in Cognitive Neuroscience.” Psychological Bulletin 125 (6): 826–59.

Näätänen, Risto, Thomas Jacobsen, and István Winkler. 2005. “Memory-Based or Afferent Processes in Mismatch Negativity (MMN): A Review of the Evidence.” Psychophysiology 42 (1): 25–32. doi:10.1111/j.1469-8986.2005.00256.x.

Neville, Helen, Janet L. Nicol, Andrew Barss, Kenneth I. Forster, and Merrill F. Garrett. 1991. “Syntactically Based Sentence Processing Classes: Evidence from Event-Related Brain Potentials.” Journal of Cognitive Neuroscience 3 (2): 151–65. doi:10.1162/jocn.1991.3.2.151.

Osterhout, Lee, Michael Bersick, and Judith Mclaughlin. 1997. “Brain Potentials Reflect Violations of Gender Stereotypes.” Memory & Cognition 25 (3): 273–85. doi:10.3758/BF03211283.

Paavilainen, Petri, Maria Jaramillo, and Risto Näätänen. 1998. “Binaural Information Can Converge in Abstract Memory Traces.” Psychophysiology 35 (5): 483–87. doi:10.1017/S0048577298970895.

Patel, A. D., E. Gibson, J. Ratner, M. Besson, and P. J. Holcomb. 1998. “Processing Syntactic Relations in Language and Music: An Event-Related Potential Study.” Journal of Cognitive Neuroscience 10 (6): 717–33.

Peretz, Isabelle, Elvira Brattico, Miika Järvenpää, and Mari Tervaniemi. 2009. “The Amusic Brain: In Tune, out of Key, and Unaware.” Brain: A Journal of Neurology 132 (Pt 5): 1277–86. doi:10.1093/brain/awp055.

Peretz, Isabelle, and Régine Kolinsky. 1993. “Boundaries of Separability between Melody and Rhythm in Music Discrimination: A Neuropsychological Perspective.” The Quarterly Journal of Experimental Psychology Section A 46 (2): 301–25. doi:10.1080/14640749308401048.

Pesenti, Mauro, Marc Thioux, Xavier Seron, and Anne De Volder. 2000. “Neuroanatomical Substrates of Arabic Number Processing, Numerical Comparison, and Simple Addition: A

46

PET Study.” Journal of Cognitive Neuroscience 12 (3): 461–79. doi:10.1162/089892900562273.

Phillips, Owen R., Kristi A. Clark, Roger P. Woods, Kenneth L. Subotnik, Robert F. Asarnow, Keith H. Nuechterlein, Arthur W. Toga, and Katherine L. Narr. 2011. “Topographical Relationships between Arcuate Fasciculus Connectivity and Cortical Thickness.” Human Brain Mapping 32 (11): 1788–1801. doi:10.1002/hbm.21147.

Piston, Walter. 1948. Harmony. New York: W. W. Norton & Company. Poulin-Charronnat, Bénédicte, Emmanuel Bigand, and Stefan Koelsch. 2006. “Processing of

Musical Syntax Tonic versus Subdominant: An Event-Related Potential Study.” Journal of Cognitive Neuroscience 18 (9): 1545–54. doi:10.1162/jocn.2006.18.9.1545.

Rohrmeier, Martin. 2011. “Towards a Generative Syntax of Tonal Harmony.” Journal of Mathematics and Music 5 (1): 35–53. doi:10.1080/17459737.2011.573676.

Rohrmeier, Martin, and Patrick Rebuschat. 2012. “Implicit Learning and Acquisition of Music.” Topics in Cognitive Science 4 (4): 525–53. doi:10.1111/j.1756-8765.2012.01223.x.

Rüsseler, Jascha, Eckart Altenmüller, Wido Nager, Christine Kohlmetz, and Thomas F. Münte. 2001. “Event-Related Brain Potentials to Sound Omissions Differ in Musicians and Non-Musicians.” Neuroscience Letters 308 (1): 33–36. doi:10.1016/S0304-3940(01)01977-2.

Sammler, Daniela, Stefan Koelsch, and Angela D. Friederici. 2011. “Are Left Fronto-Temporal Brain Areas a Prerequisite for Normal Music-Syntactic Processing?” Cortex 47 (6): 659–73. doi:10.1016/j.cortex.2010.04.007.

Sams, M, P Paavilainen, K Alho, and R Näätänen. 1985. “Auditory Frequency Discrimination and Event-Related Potentials.” Electroencephalography and Clinical Neurophysiology/Evoked Potentials Section 62 (6): 437–48. doi:10.1016/0168-5597(85)90054-1.

Schmithorst, Vincent J, and Rhonda Douglas Brown. 2004. “Empirical Validation of the Triple-Code Model of Numerical Processing for Complex Math Operations Using Functional MRI and Group Independent Component Analysis of the Mental Addition and Subtraction of Fractions.” NeuroImage 22 (3): 1414–20. doi:10.1016/j.neuroimage.2004.03.021.

Schmithorst, Vincent J., and Scott K. Holland. 2004. “The Effect of Musical Training on the Neural Correlates of Math Processing: A Functional Magnetic Resonance Imaging Study in Humans.” Neuroscience Letters 354 (3): 193–96. doi:10.1016/j.neulet.2003.10.037.

Schoenberg, Arnold. 1978. Theory of Harmony. University of California Press. Schön, Daniele, Cyrille Magne, and Mireille Besson. 2004. “The Music of Speech: Music Training

Facilitates Pitch Processing in Both Music and Language.” Psychophysiology 41 (3): 341–49. doi:10.1111/1469-8986.00172.x.

Schröger, Erich. 2007. “Mismatch Negativity.” Journal of Psychophysiology 21 (3–4): 138–46. doi:10.1027/0269-8803.21.34.138.

Schulze, Katrin, Stefan Zysset, Karsten Mueller, Angela D. Friederici, and Stefan Koelsch. 2011. “Neuroarchitecture of Verbal and Tonal Working Memory in Nonmusicians and Musicians.” Human Brain Mapping 32 (5): 771–83. doi:10.1002/hbm.21060.

Steedman, Mark J. 1984. “A Generative Grammar for Jazz Chord Sequences.” Music Perception: An Interdisciplinary Journal 2 (1): 52–77. doi:10.2307/40285282.

Steinbeis, Nikolaus, and Stefan Koelsch. 2008. “Shared Neural Resources between Music and Language Indicate Semantic Processing of Musical Tension-Resolution Patterns.” Cerebral Cortex 18 (5): 1169–78. doi:10.1093/cercor/bhm149.

Sussman, Elyse, Walter Ritter, and Herbert G. Vaughan. 1999. “An Investigation of the Auditory Streaming Effect Using Event-Related Brain Potentials.” Psychophysiology 36 (1): 22–34. doi:10.1017/S0048577299971056.

Sussman, Elyse, Walter Ritter, and Herbert G Vaughan Jr. 1998. “Attention Affects the Organization of Auditory Input Associated with the Mismatch Negativity System.” Brain Research 789 (1): 130–38. doi:10.1016/S0006-8993(97)01443-1.

Tang, Antony, and Alice Yip. 2010. “Collision Avoidance Timing Analysis of DSRC-Based Vehicles.” Accident Analysis & Prevention 42 (1): 182–95. doi:10.1016/j.aap.2009.07.019.

47

Teo, G., T. Schmidt, J. Szalma, G. Hancock, and P. Hancock. 2014. “The Effects of Individual Differences on Vigilance Training and Performance in a Dynamic Vigilance Task.” Proceedings of the Human Factors and Ergonomics Society Annual Meeting 58 (1): 964–68. doi:10.1177/1541931214581202.

Tervaniemi, M., S. V. Medvedev, K. Alho, S. V. Pakhomov, M. S. Roudas, T. L. Van Zuijen, and R. Näätänen. 2000. “Lateralized Automatic Auditory Processing of Phonetic versus Musical Information: A PET Study.” Human Brain Mapping 10 (2): 74–79.

Tillmann, Barbara, and Emmanuel Bigand. 2004. “The Relative Importance of Local and Global Structures in Music Perception.” The Journal of Aesthetics and Art Criticism 62 (2): 211–22. doi:10.1111/j.1540-594X.2004.00153.x.

Winkler, I., G. Karmos, and R. Näätänen. 1996. “Adaptive Modeling of the Unattended Acoustic Environment Reflected in the Mismatch Negativity Event-Related Potential.” Brain Research 742 (1–2): 239–52.

Winkler, I., P. Paavilainen, K. Alho, K. Reinikainen, M. Sams, and R. Näätänen. 1990. “The Effect of Small Variation of the Frequent Auditory Stimulus on the Event-Related Brain Potential to the Infrequent Stimulus.” Psychophysiology 27 (2): 228–35.

Winograd, Terry. 1972. “Understanding Natural Language.” Cognitive Psychology 3 (1): 1–191. doi:10.1016/0010-0285(72)90002-3.

Zago, Laure, Mauro Pesenti, Emmanuel Mellet, Fabrice Crivello, Bernard Mazoyer, and Nathalie Tzourio-Mazoyer. 2001. “Neural Correlates of Simple and Complex Mental Calculation.” NeuroImage 13 (2): 314–27. doi:10.1006/nimg.2000.0697.

Zuijen, Titia L. van, Veerle L. Simoens, Petri Paavilainen, Risto Näätänen, and Mari Tervaniemi. 2006. “Implicit, Intuitive, and Explicit Knowledge of Abstract Regularities in a Sound Sequence: An Event-Related Brain Potential Study.” Journal of Cognitive Neuroscience 18 (8): 1292–1303. doi:10.1162/jocn.2006.18.8.1292.